Backtesting Traps: Common Errors to Avoid
A backtest is a simulation whose value depends on its data, rules, and execution assumptions. A high return can reflect a useful idea, but it can also reflect future information, repeated tuning, omitted costs, or fills that would not have been available. Audit those assumptions before interpreting the performance.
The main traps are overfitting, incomplete trading costs, look-ahead bias, and treating one market environment as representative of every other. LuxAlgo’s native charts and Quant, our coding agent, help you express and inspect technical strategies. They do not remove the need to review how a result was produced.
1. Mistaking a strong historical fit for a durable strategy
Overfitting happens when research choices adapt too closely to the particular sample, including its noise. It can arise from complicated code or from trying many apparently simple strategies and reporting only the winner.
| Observation | What to investigate | What it does not prove |
|---|---|---|
| Excellent results at one parameter setting | Whether nearby settings produce a very different outcome | That the selected setting will remain optimal |
| Many filters and exceptions | Which rules were added after reviewing losing trades | That complexity has a sound economic explanation |
| Profit concentrated in a few trades | Whether those trades are valid and consistent with the intended payoff pattern | That every strategy with large outliers is invalid |
| A large gap between development and later results | Selection effects, implementation differences, costs, and market conditions | That one explanation accounts for the whole gap |
Some trend strategies intentionally rely on occasional large winners. Inspect those trades and their data rather than deleting them automatically. Conversely, a smooth curve with modest returns can still contain serious bias.
No metric range certifies a backtest
Profit factor compares gross profits with gross losses. Sharpe measures excess return relative to return variability; Sortino uses a downside-risk measure relative to a specified target. Each needs a clear calculation method, return frequency, sample, and treatment of costs.
A profit factor between 1.5 and 2.0 is not a sustainability certificate. A Sharpe or Sortino ratio above 3 is not, by itself, proof of overfitting. There is also no annual-return or Sharpe threshold below which look-ahead bias can be ignored. Audit the process regardless of the headline number.
Reduce selection effects without promising to eliminate them
Use a chronological development period and reserve later observations for evaluation. An 80/20 split is one possible allocation, not a universal standard. The useful split depends on trade frequency, market coverage, and how much data the model requires.
Track rejected versions and parameter trials. If you repeatedly inspect the reserved period and adjust the strategy, it is no longer independent evidence. Research on backtest overfitting explains why a holdout alone does not account for the full strategy-selection process.
Test neighboring parameters and relevant markets as diagnostics. A strategy designed for a particular market need not profit on every unrelated asset, but its scope must be declared before seeing the results. Do not keep expanding the search until you find a favorable exception.
2. Underestimating costs and confusing execution benchmarks
Costs depend on the instrument, broker plan, order size, routing, trading frequency, and market conditions. Use the schedule and assumptions appropriate to the period being simulated.
| Cost or constraint | What to include | Common mistake |
|---|---|---|
| Commissions and exchange charges | Per-share or per-contract rates, minimums, and applicable pass-through charges | Multiplying by a headline rate while ignoring the order minimum |
| Regulatory charges | The relevant transaction side, date, market, and broker treatment | Using an old combined rate for every buy and sell |
| Spread and slippage | A defined reference price and assumptions for both entry and exit | Adding the same spread effect twice |
| Market impact and capacity | How order size and participation may affect execution | Assuming unlimited size fills at the displayed price |
| Financing and short availability | Interest, borrow fees, funding, and whether the position could be opened | Treating unavailable or expensive shorts as frictionless |
Interactive Brokers’ stock commission schedule, for example, distinguishes tiered, fixed, and eligible commission-free arrangements. A quoted per-share rate is not a complete cost model. Broker minimums and additional charges can matter even for small orders.
Regulatory rates also change. The SEC’s fiscal 2026 advisory set the Section 31 rate for covered sales at $20.60 per million from April 4, 2026, following a period at zero. That is not a combined SEC-and-FINRA rate for all transactions. For historical tests, apply the relevant dated rules and the broker’s actual charging method.
A worked slippage example
Consider a hypothetical stock quoted at a $183.50 bid and $183.53 ask. A 100-share market buy fills at $183.57. The quoted spread is $0.03, but the execution shortfall depends on the benchmark:
- Against the displayed ask: ($183.57 − $183.53) × 100 = $4.00.
- Against the $183.515 midpoint: ($183.57 − $183.515) × 100 = $5.50.
- Against the displayed bid: ($183.57 − $183.50) × 100 = $7.00.
These are alternative measurements of the same purchase, not three costs to add together. They exclude commissions and the eventual exit. Record the benchmark consistently when comparing a backtest with subsequent execution.
There is no universal “under 2%” slippage assumption for liquid markets. Calibrate in price units, ticks, or basis points appropriate to the strategy, and test adverse scenarios. A limit order restricts price but can remain unfilled; TWAP and VWAP execution schedules do not guarantee a better outcome.
Trading a higher timeframe or widening a stop changes the strategy and its cash exposure. It is not a general remedy for costs. Likewise, Monte Carlo simulations explore the assumptions built into their scenarios; they do not repair future-data leakage or prove that a fill model is realistic.
3. Using information before it was available
Look-ahead bias occurs when a simulated decision depends on later information. It can enter through data preparation, indicator calculations, portfolio selection, or order timing—not just an obvious reference to tomorrow’s price.
| Trap | Example | Audit question |
|---|---|---|
| Release-date mismatch | Trading on quarterly earnings at quarter-end before publication | When could the strategy first have known the figure? |
| Retrospective indicator placement | Entering at a pivot that requires later bars to confirm | Is the trade placed at confirmation or backdated to the pivot? |
| Full-sample calculations | Normalizing early values with the mean of the entire dataset | Were transformations fitted only on information available then? |
| Surviving-universe selection | Applying today’s index constituents throughout history | Were membership and delistings represented as they occurred? |
| Order-timing mismatch | Using the final closing price to generate a signal and assuming an earlier fill | Could the order have been submitted at the claimed time? |
Point-in-time research practices help separate an observation’s economic date from its availability date. For revised datasets, retain what was known at each historical timestamp rather than silently substituting the latest version.
An event-driven framework can help enforce time ordering, but precomputed features can still contain future information. Inspect the full pipeline. For a sample of trades, reconstruct the inputs and decision using only the data available before the order.
Use automated checks with their limits in mind
Freqtrade’s lookahead analysis compares baseline and altered backtest runs to detect changes in indicators and signals. Its documentation explicitly limits the conclusion to the signals exercised by the test: untriggered conditions can escape verification. Certain limit-order and custom-price behavior can also produce false positives.
Such checks are useful evidence, not a certificate that every strategy path is bias-free. Choose periods that exercise the relevant signals, inspect reported differences, and retain manual timing checks.
4. Treating one market regime as universal
A strategy may depend on persistent trends, frequent reversals, available liquidity, or a particular financing environment. Assess the conditions it actually needs, then compare behavior across relevant historical periods. Calendar labels alone do not describe all those differences.
Research by Paul Geertsema and Helen Lu illustrates the danger of broad rules about factor performance. Their study reports stronger historical bear-market profitability for several long-short factors, largely from their short sides, while highlighting the atypical losses of most examined factors during the COVID-19 episode. That is not a rule that buying value or momentum stocks must work in every bear market.
Separate the strategy’s long and short contributions, volatility exposure, turnover, and concentration. If a regime filter is used to trade, define it from information available at the decision time. A retrospective label such as “the market bottom” cannot serve as a live entry signal.
Stress periods such as the 2008 crisis and the 2020 disruption can be useful when suitable data and instruments exist. Also examine ordinary periods, recoveries, and changing transaction costs. Good results across several past episodes cannot ensure resilience to a different future event.
Audit the strategy on LuxAlgo’s native charts
Ask Quant to implement explicit entry, exit, and sizing rules, then review Code before Run. Check the script’s signal timing, use of historical values, and handling of overlapping trades. A script that compiles can still implement a biased or unintended rule.
In the backtest viewer, review Performance, Trades Analysis, and Trades Log. Inspect suspicious entries, large winners, and the long/short split. Use standard price candles: averaged Heikin Ashi prices can create simulated fills that were not tradable.
Set simulation assumptions in Properties and strategy parameters in Inputs. Check the market-data source, session, and history coverage before comparing runs. A volume rule built on one venue’s activity may behave differently on another feed.
LuxAlgo’s native order-flow tools summarize supported executed-volume data; they are not a complete historical order book or proof of queue priority. Keep those limits separate from candle-derived market-structure indicators and TradingView toolkits. Quant-generated code also does not automatically provide independent walk-forward validation or a live execution model.
Video: how parameter optimization selects historical winners
This official ProRealTime demonstration shows how parameter combinations are compared within a selected historical period. Use it to understand what an optimization result represents: a ranking on that sample, not independent evidence of future performance. The interface shown is from an earlier version.
Keep a reproducible review record
Save the code version, inputs, instrument, timeframe, session, history period, costs, and reason for each change. Report the tests that failed as well as those that passed. ProRealTime’s ProBacktest and PaperTrading workflow is another example of separating historical simulation from prospective observation; the distinction matters whichever software you use.
Before moving beyond development, freeze a version and evaluate it on reserved data. Then record every eligible signal during live paper observation, including missed or skipped trades. Our guide to backtesting versus forward testing explains how to keep those stages distinct.
A useful backtest does not need to look flawless. It needs traceable inputs, defensible assumptions, and enough evidence to explain both its strengths and its limitations.
Read next