🤖 ML Models
ML - Bias-Variance Tradeoff
The fundamental tension in ML: simple models underfit (high bias), complex models overfit (high variance). Every model sits somewhere on this spectrum. Understanding where your model falls is the first step to improving it.
2
Minutes
6
Concepts
+45
XP
1
The Decomposition
Total prediction error breaks into three components:
Total Error = Bias² + Variance + Irreducible Noise
- Bias²: Error from wrong assumptions in the model (too simple to capture the pattern)
- Variance: Error from sensitivity to fluctuations in training data (memorizes noise)
- Irreducible Noise: Random error inherent in the data — you can't fix this no matter what
You can only control the first two. Reducing one typically increases the other.