L1 and L2 Penalty

Also known as · L1 Penalty · L2 Penalty · regularization

Regularisation terms added to the loss: L1 adds λ∑j∣βj∣\lambda \sum_j |\beta_j| (sum of absolute coefficients), L2 adds λ∑jβj2\lambda \sum_j \beta_j^2 (sum of squared coefficients).

When to use

L2 (Ridge Regression) shrinks all coefficients without zeroing any; L1 (Lasso Regression) can zero some exactly. In PS 02 raising alpha from 0.01 to 1.0 pushes many more Lasso coefficients to zero. Choose the penalty strength with Cross-Validation.

See Lec 4.

Appears in

No references yet.