Ridge Regression

L2-regularised linear regression: it adds the sum of squared coefficients, times a strength λ\lambda, to the loss, so all coefficients shrink towards zero but none reach exactly zero.

When to use

Use it to cut Overfitting when all features are relevant; Lasso Regression is the choice when many are irrelevant. Scale the features first, and pick the strength (alpha) by Cross-Validation: too small barely regularises, too large pushes every coefficient to zero and underfits. Also in PS 02 and PS 04.

See DS Lec 4.

Appears in