PS4

  1. 1

    A seller has a 7-year-old car. Using the simple model's equation (slope and intercept from above):

    1. Calculate the predicted price by hand.
    2. Verify with simple_model.predict([[7]]).
    3. The actual listing price is ₪82K. What is the residual? Did the model over- or underestimate?
    4. Is R² good enough for Dana's pricing engine? Why or why not?
  2. 2
    • Compare the car_age_years and km_driven_thousands coefficients for Ridge and Lasso (both trained on standardised features, so they are directly comparable).
    • Which model shrinks coefficients more aggressively?
    • Why does regularisation help with multicollinearity?

Recitation walkthrough

Setup, EDA and split

python
import pandas as pd, numpy as np
from sklearn.model_selection import train_test_split, cross_val_score
from sklearn.linear_model import LinearRegression, Ridge, Lasso
from sklearn.preprocessing import PolynomialFeatures, StandardScaler
from sklearn.metrics import mean_absolute_error, mean_squared_error, r2_score

df = pd.read_csv("autobid_used_cars.csv")

X = df.drop(columns=["listing_price_thousands", "listing_id"])
y = df["listing_price_thousands"]

# One-hot encode categoricals; drop_first=True makes BMW (brand) and Diesel (fuel) the baselines
X = pd.get_dummies(X, columns=["fuel_type", "car_brand", "city"], drop_first=True)

X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2, random_state=42)

What the EDA showed (the notebook's words): price is right-skewed (mean above median, a tail of luxury and electric cars); age vs price is clearly linear and downward; engine size vs price is curved (steep then flat); and car_age_years and km_driven_thousands are strongly correlated (r about 0.75), which is Multicollinearity.

Simple regression: one feature

python
simple_model = LinearRegression()
simple_model.fit(X_train[["car_age_years"]], y_train)
print(simple_model.coef_[0], simple_model.intercept_)   # slope = depreciation per year, intercept = price at age 0

The slope is read as depreciation per year and the intercept as the predicted price of a brand-new car. Because km_driven is missing, the age slope soaks up the km effect too.

Multiple regression: every feature gets a "price tag"

python
full_model = LinearRegression().fit(X_train, y_train)
r2_full = r2_score(y_test, full_model.predict(X_test))

Read each coefficient as "all else equal, this feature changes the predicted price by this much". Examples quoted in the notebook: car_brand_Hyundai is about −₪44K against the BMW baseline, and fuel_type_Electric is about +₪35K against Diesel.

Why the age coefficient shrinks

When km_driven_thousands joins the model, car_age_years shrinks, because the two share information and the shared effect is now split between them. Remove km and age grows back toward its simple-regression value.

Polynomial term on one feature

python
poly = PolynomialFeatures(degree=2, include_bias=False)
engine_train = poly.fit_transform(X_train[["engine_size_cc"]])   # columns: engine_cc, engine_cc^2

The notebook adds engine_size_cc² only to the one feature that visibly curves. It then shows the over-the-top version: polynomial features on all numeric columns at degree 1, 2 and 3. Training R² keeps climbing but test R² drops at degree 3: classic overfitting (see Bias-Variance Tradeoff).

Regularisation: Ridge and Lasso

python
scaler = StandardScaler()
X_train_sc = pd.DataFrame(scaler.fit_transform(X_train), columns=X_train.columns, index=X_train.index)
X_test_sc  = pd.DataFrame(scaler.transform(X_test),      columns=X_test.columns,  index=X_test.index)

ridge_model = Ridge(alpha=1.0).fit(X_train_sc, y_train)
lasso_model = Lasso(alpha=0.5).fit(X_train_sc, y_train)
  • Ridge (L2) shrinks all coefficients towards zero; none vanish.
  • Lasso (L1) can shrink some coefficients exactly to zero, which is automatic feature selection. In the notebook Lasso dropped has_sunroof among others.
  • You must standardise first: the penalty is on coefficient size, so a feature measured in thousands of cc (1000 to 3500) would be penalised unfairly next to a 0/1 feature.

Cross-validation and the grand comparison

python
cv_scores = cross_val_score(LinearRegression(), X, y, cv=5, scoring="r2")

from sklearn.pipeline import make_pipeline
models = {
    "Linear Regression": LinearRegression(),
    "Ridge (α=1.0)": make_pipeline(StandardScaler(), Ridge(alpha=1.0)),
    "Lasso (α=0.5)": make_pipeline(StandardScaler(), Lasso(alpha=0.5)),
}

Putting the scaler inside a make_pipeline means each CV fold fits the scaler on its own training part only, which prevents leakage (see Scikit-Learn Pipeline). The notebook's recommendation for Dana: Ridge for production (stable, accurate, interpretable, copes with correlated features), Lasso as a side analysis of which features matter. Its quoted figure is a CV R² of about 0.76 with a std of about 0.03.

Related: Lecture 4 - Regression, Lecture 3 - Classification, KNN, Random Tree

Source

Official course notebook recitation4.ipynb (Recitation 4, Spring 2026). The two exercises and their solutions come straight from the notebook. The notebook had no saved outputs and its CSV is not available, so numbers below are only those the notebook's own text states.

What this problem set is about

Recitation 3 predicted labels (churn). Here you predict a number: the listing price of a used car for the marketplace AutoBid. The pattern is the same, create → .fit() → .predict(), but accuracy is replaced by MAE, RMSE and R-Squared.

The AutoBid case

Dana (AutoBid's product lead) wants a price prediction engine: sellers who overprice get no offers, sellers who underprice lose money. The data has 350 listings with car_age_years, km_driven_thousands, engine_size_cc, num_previous_owners, has_sunroof, fuel_type, car_brand, city and the target listing_price_thousands (in ₪K).

Recap

Item Tool Takeaway
Q1 LinearRegression, predict, residual Prediction = intercept + slope × 7; residual = actual − predicted; R² about 0.55 is too low for a pricing tool
Q2 Ridge, Lasso on standardised data Regularisation stabilises correlated features; Lasso can also drop them