Concept
Bias Taxonomy
Bias Taxonomy, also known as data snooping, survivorship, look-ahead, repaint, is a Performance, Backtesting & Validation concept. First implementations are in the build queue: the write-up leads, the indicators follow.
What is a bias taxonomy?
A bias taxonomy is a structured checklist of the systematic errors that inflate backtested performance: the recurring ways a historical test can flatter a strategy that has no real edge. The best-known entries are look-ahead bias, where the simulation uses information that was not available at decision time; survivorship bias, where the tested universe silently excludes delisted or failed instruments; data snooping, where an edge is manufactured by testing many variations against one history and keeping the winner; and repainting, where an indicator revises its past values so historical signals look cleaner than anything a live trader ever saw.
The taxonomy exists because backtests rarely fail from one dramatic mistake. They fail from an accumulation of small, structural flaws, each of which biases results in the same direction: upward. Unlike random noise, these errors do not cancel out, so a backtest containing several of them can show a smooth equity curve for a strategy that loses money live. Naming the failure modes turns a vague worry about curve-fitting into an auditable checklist.
Traders care because each bias has a distinct fingerprint and a distinct cure. Look-ahead bias is fixed by strict point-in-time data and careful signal timing; survivorship bias by using delisting-inclusive datasets; repainting by repaint-safe engineering that only reads confirmed bars; and data snooping by a multiple-testing correction or an honest in-sample / out-of-sample split. Diagnosing which bias is present matters more than knowing that some bias exists.
How traders use it
- As a pre-mortem checklist before trusting any backtest: walk through look-ahead, survivorship, snooping, repainting, and cost omissions one by one, and document how each was ruled out or bounded.
- As a debugging guide when live results undershoot the backtest: the gap's character often points to the culprit, for example an immediate collapse suggests repainting or look-ahead, while slow erosion suggests snooping or optimistic costs.
- As a review standard when evaluating someone else's published results, since third-party backtests rarely disclose universe construction, signal timing, or how many variants were tried before the reported one.
- As a design constraint from the start: choosing point-in-time data, confirmed-bar signals, and a limited pre-registered set of parameter trials prevents most biases more cheaply than detecting them afterward.
- Alongside quantitative audits such as the probability of backtest overfitting, which estimate how much damage the snooping entry of the taxonomy has already done.
Bias taxonomy vs related concepts
Repaint-safe Engineering: The taxonomy names the disease; repaint-safe engineering is the cure for one specific entry. It covers the coding practices, such as evaluating on confirmed bars and avoiding future-referencing functions, that prevent repainting and intra-bar look-ahead in indicator logic.
Multiple-testing Correction: A statistical remedy for the data-snooping entry specifically. It adjusts significance thresholds for the number of variants tried, but does nothing about survivorship, look-ahead, or repainting, which are data and code problems rather than inference problems.
Cost-model Realism: Omitted or understated trading costs act like a bias, inflating every backtest they touch, but they are usually treated as a separate audit because the fix is economic modeling rather than data hygiene or statistics.
Related concepts · Validation methodology
Concept family
Performance, Backtesting & Validation
30 concepts mapped · 30 in the Library
Bias Taxonomy FAQ
Which backtest bias is the most damaging?
It depends on the strategy. High-turnover systems are most often sunk by look-ahead and cost errors, long-only equity screens by survivorship bias, and heavily optimized systems by data snooping. Repainting tends to produce the most dramatic gap between backtest and live results.
How do I check for look-ahead bias?
Audit the timestamp of every input: a signal computed on bar t must only use data finalized by bar t, and fills should occur at prices available after the signal. Shifting all signals one bar later and re-running is a crude but effective smoke test; if performance collapses, timing was leaking information.
Does survivorship bias matter outside stocks?
Yes, though less severely. Crypto has a large graveyard of delisted tokens, and futures histories can embed selection effects through contract choices and roll assumptions. Any universe defined by today's membership imports some survivorship into the past.
Can a backtest ever be fully bias-free?
No. The goal is bounded, documented bias rather than none. A careful workflow rules out the mechanical biases and quantifies the statistical ones, then treats the remaining backtest as evidence, not proof, pending forward confirmation.
Build Bias Taxonomy your way.
Quant writes, tests, and refines it with you — then it runs on LuxAlgo charting or ports to TradingView.