OLS

Ordinary least squares: the estimator that picks the coefficients minimising the sum of squared residuals, RSS=∑i(yi−y^i)2\text{RSS} = \sum_i (y_i - \hat{y}_i)^2.

When to use

The closed form is β^=(X⊤X)−1X⊤y\hat{\boldsymbol{\beta}} = (X^\top X)^{-1} X^\top \mathbf{y}, though scikit-learn uses numerical optimisation instead of inverting the matrix. OLS predictions do not change with feature scaling.

See DS Lec 4.

Appears in