Flexible models memorise noise even faster than humans. Regularisation and humility.
Overfitting — fitting noise instead of signal (Module 3) — is worse in ML than in hand-built strategies, because flexible models can memorise noise far faster and more thoroughly than a human ever could. In markets’ low-signal environment, this is ML’s defining danger.
The core tension:
the very flexibility that makes ML powerful is what makes it overfit so dangerously in markets. A complex model has enormous capacity to fit
any pattern in the training data — including the vast noise — so it can achieve near-perfect
training performance that means
nothing about the
futureA binding agreement to buy or sell at a set price on a future date.. The defences are the same in spirit as Module 3 but sharper:
(1) Regularisation — deliberately penalising complexity to force the model toward simpler, more general patterns.
(2) Simpler models — in noisy markets, a simple model (even linear) often
beats a deep one, because it has less room to memorise noise (Occam’s razor again).
(3) Rigorous out-of-sample / time-series validation — the only honest test (and watch for the leakage that fakes it).
(4) Humility — assume your impressive result is overfit until repeatedly proven otherwise. The paradox mirrors the whole quant track:
more power and complexity usually means more overfitting and worse live results in this domain. The disciplined ML practitioner
fights their own model’s flexibility at every step.
- Why worse in ML — flexible models memorise noise faster/more completely than humans, and markets are mostly noise.
- Regularisation — penalise complexity to push the model toward simpler, more generalisable patterns.
- Prefer simpler models — in low-signal markets, simple (even linear) models often beat deep ones.
- Validate + stay humble — honest time-series OOSTesting a strategy on data it was never built on. testing (leak-free), and assume overfit until proven otherwise.
ExampleA deep neural net hits 99% accuracy on training data — then performs no better than a coin flip live. It memorised the training noise perfectly. A heavily-regularised simple model scoring a modest 55% in training but holding ~54% out-of-sample is the better model — its lower, stable score reflects real (if small) signal rather than memorised noise.
Key takeawayML overfits worse than hand-built strategies because flexible models memorise noise fast — deadly in noisy markets. Defend with regularisation (penalise complexity), simpler models (often beat deep ones here), rigorous leak-free time-series validation, and humility. More complexity usually means more overfitting and worse live results.