Bias-Variance Tradeoff
Also known as · Bias-Variance Trade-off
Every model sits between two failure modes: high bias (too simple, underfits, high training and test error) and high variance (memorises noise, overfits, low training error but high test error). The aim is the balance with the lowest test error.
When to use
Use it to diagnose a model from its training and test errors: fix high bias with a more complex model or more features, and high variance by regularising (Ridge Regression, Lasso Regression), getting more data or simplifying. The lecture's examples are a linear model on curved data versus a degree-10 polynomial on 30 points.
See DS Lec 4.