Train/validation Discipline

Train/validation Discipline stages the evaluation protocol on the chart. Recent history splits chronologically into training, validation and holdout segments separated by label-horizon purge gaps, and a small demonstration classifier is fitted and judged by the rules of train/validation discipline. Shading shows which bars may teach, which judge and which stay locked; the dashboard renders the verdict.

How to Trade the Train/validation Discipline?

• Leakage Inflation: the validation bars re-scored under a deliberately corrupted twin — full-sample scaler, no purge gap, straddling labels; the difference is measured leakage inflation.
• Generalization Gap: in-sample minus disciplined validation accuracy; a wide gap means the model memorized its training data.
• Selection noise: every variant scored on validation spends a trial; the dashboard estimates how much best-of-K luck explains the score.

Train/validation Discipline Settings

• Label Horizon (default 10) and Extra Embargo (Bars) (default 0): the label lookahead, plus extra buffer on the automatic purge.
• Analysis Window (default 3000): recent bars in the audit.
• Training % (default 60) and Validation % (default 25): segment shares; what's left after both gaps is the holdout.
• Reveal Final Holdout (default off): scores the holdout — every look spends it.
• Feature Length (default 14): base lookback of the features — RSI, z-scored SMA distance, rate of change.
• Tune On Validation (Spend Trials) (default on): fits four feature-length variants and keeps the best, counting the trials.
• Score Leaky Twin (default on): runs the corrupted twin for comparison.
• Leakage Inflation Alert (pp) (default 5.0) and Generalization Gap Alert (pp) (default 15.0): audit thresholds.
• Show Dashboard (on, Top Right, Small); Shade Segments, Boundary Lines, Segment Labels (all on).

Frequently Asked Questions

How does this relate to Walk-Forward Analysis?

This build audits a single chronological split — one fit, one validation score, one locked holdout. Walk-Forward Analysis rolls the exercise forward window by window; each re-fit must obey this discipline.

Why does the holdout label say LOCKED?

Every evaluation converts it into validation data, so the segment is not scored until Reveal Final Holdout is switched on — reveal after tuning ends, treat the number as final.

Are the accuracy figures a trading signal?

No. They score a deliberately modest model against the 50% chance level so leakage, overfitting and selection noise become visible. The point is the difference between protocols, not the level of any score.