Forward Testing & Tracking

advanced7 min read

Comparing live results to the backtest expectation — your early-warning system for decay.

Forward testing is the ongoing discipline of *comparing your live (or paper) results against what the backtest led you to expect*. It turns your backtest from a one-time exam into a living benchmark you continuously measure reality against.

The key idea: *your backtest gave you an expectation — a distribution of likely returns, win rates and drawdowns — and forward testing checks whether live reality is tracking that expectation or diverging from it.* This is your early-warning system. Some divergence is normal (variance — Module 1’s law of large numbers; a few bad trades mean nothing). But a persistent, significant gap between live and expected performance is a red flag that something is wrong: the edge may be decaying (crowding/regime change), or your backtest may have been subtly overfit/leaky (live is exposing what historical optimism hid). Forward testing lets you *distinguish normal bad luck from genuine breakdown* — and Monte Carlo (Module 8) is what calibrates this, by telling you the range of outcomes to expect, so you know whether a 20% drawdown is “within bounds” or “alarm.” Without forward tracking you’re flying blind, unable to tell a temporary slump from a dead strategy. With it, you have an objective tripwire for when to investigate or retire (next lesson).

What it is — continuously compare live/paper results to the backtest’s expected returns, win rate and drawdown.
Normal vs alarming — short-term variance is expected; a persistent, significant divergence signals trouble.
What divergence means — decaying edge (crowding/regime change) or an overfit/leaky backtest now exposed.
Calibrate with Monte Carlo — the expected range of outcomes tells you if a drawdown is within bounds or a red flag.

ExampleYour backtest (and Monte Carlo) said “expect ~15% annual return, worst-case drawdown ~25%, win rate ~55%.” Six months live you’re at a 40% drawdown and a 42% win rate — well outside the expected range. That persistent divergence is an objective signal to investigate: the edge may be decaying or the backtest was optimistic. A 12% return with a 22% drawdown, by contrast, is just normal tracking.

Key takeawayForward testing continuously compares live results to the backtest’s expected range (calibrated by Monte Carlo). It’s your early-warning system: short-term variance is normal, but persistent, significant divergence signals a decaying edge or an overfit backtest. It lets you tell bad luck from genuine breakdown — don’t trade without it.

FAQs

How do I know if underperformance is bad luck or a broken strategy?

Use the *expected range* from your backtest and Monte Carlo simulation. If live results are within the plausible band of outcomes (even a normal drawdown), it’s likely variance — stay the course. If they fall *persistently and significantly outside* that band, it’s a credible signal of decay or backtest flaws warranting investigation. The discipline is judging against a pre-defined expectation, not emotion.