Overfitting in ML

advanced7 min read

Flexible models memorise noise even faster than humans. Regularisation and humility.

Overfitting — fitting noise instead of signal (Module 3) — is worse in ML than in hand-built strategies, because flexible models can memorise noise far faster and more thoroughly than a human ever could. In markets’ low-signal environment, this is ML’s defining danger.

The core tension: the very flexibility that makes ML powerful is what makes it overfit so dangerously in markets. A complex model has enormous capacity to fit any pattern in the training data — including the vast noise — so it can achieve near-perfect training performance that means nothing about the future. The defences are the same in spirit as Module 3 but sharper: (1) Regularisation — deliberately penalising complexity to force the model toward simpler, more general patterns. (2) Simpler models — in noisy markets, a simple model (even linear) often beats a deep one, because it has less room to memorise noise (Occam’s razor again). (3) Rigorous out-of-sample / time-series validation — the only honest test (and watch for the leakage that fakes it). (4) Humility — assume your impressive result is overfit until repeatedly proven otherwise. The paradox mirrors the whole quant track: more power and complexity usually means more overfitting and worse live results in this domain. The disciplined ML practitioner fights their own model’s flexibility at every step.

Why worse in ML — flexible models memorise noise faster/more completely than humans, and markets are mostly noise.
Regularisation — penalise complexity to push the model toward simpler, more generalisable patterns.
Prefer simpler models — in low-signal markets, simple (even linear) models often beat deep ones.
Validate + stay humble — honest time-series OOS testing (leak-free), and assume overfit until proven otherwise.

ExampleA deep neural net hits 99% accuracy on training data — then performs no better than a coin flip live. It memorised the training noise perfectly. A heavily-regularised simple model scoring a modest 55% in training but holding ~54% out-of-sample is the better model — its lower, stable score reflects real (if small) signal rather than memorised noise.

✓ You learnedML overfits worse than hand-built strategies because flexible models memorise noise fast — deadly in noisy markets. Defend with regularisation (penalise complexity), simpler models (often beat deep ones here), rigorous leak-free time-series validation, and humility. More complexity usually means more overfitting and worse live results.

FAQs

Are deep neural networks better for trading than simple models?

Usually not, in most retail/quant contexts. Markets’ low signal-to-noise means deep models’ extra flexibility mostly fits noise, while simpler, regularised models generalise better. Deep learning can help with genuinely complex, data-rich problems (e.g. alternative data, NLP), but for typical price-based prediction, simpler models with disciplined validation often win — and are easier to trust.