Backtesting & Validation

Out-of-sample testing and parameter stability: the checks that separate an edge from a curve fit.

Validation is where a trading idea meets evidence. This family covers the methodology that keeps a backtest honest: testing on data the strategy has never seen, and checking that results survive small changes in settings.

The recurring theme is that an impressive backtest is easy to produce and easy to misread, because any result can be inflated by fitting parameters to history. The entries here explain how to structure tests so a good result is more likely to reflect a real effect than a curve fit.

Start here

In-sample / Out-of-sample Split

The basic defense against overfitting: tune a strategy on one stretch of history and judge it on another.

1 indicators

Parameter Stability

Checks whether performance holds across nearby parameter values, so a result rests on a broad plateau rather than a single lucky setting.

1 indicators

Validation methodology

Procedures that help separate genuine edges from curve fits, including holding out unseen data and checking stability across nearby parameter values.

In-sample / Out-of-sample Split1 Parameter Stability1