Resampling Tests

What are Resampling Tests?

Resampling tests estimate uncertainty by redrawing from the data you already have instead of assuming a theoretical distribution. The two workhorses are the bootstrap (Efron, 1979), which samples observations with replacement to approximate the sampling distribution of a statistic, and the permutation test, which reshuffles the order or labels of the data to build a null distribution in which any structure carried by ordering or labels has been destroyed. If the statistic computed on the real, ordered data sits far in the tail of the shuffled distribution, the structure is unlikely to be luck alone.

On charts the idea usually appears as shuffled-return projections: draw returns from a lookback window, chain them into many forward paths, and plot percentile bands of the outcomes, the same machinery behind Monte Carlo price paths and one common way of building probability cones. One caveat is structural: independent redraws erase autocorrelation and volatility clustering, so plain shuffles understate how losses and turbulence bunch together. Block bootstrapping (resampling contiguous chunks) preserves some of that memory, and no resampling scheme can represent regimes absent from the sampled window.

How traders use it

• Projection envelopes: resample a window of past returns and compound them forward to draw percentile cones around price, read as scenario ranges consistent with recent behavior rather than forecasts.
• Backtest significance: permute entry signals (or shuffle returns) and re-run the strategy many times; an edge that random orderings reproduce easily is statistically indistinguishable from luck, a check that pairs naturally with an in-sample / out-of-sample split.
• Confidence intervals: bootstrap trade-level results to put empirical error bars on win rate, expectancy, or Sharpe ratio instead of quoting a single point estimate.