PS4
- #data-science
- #intro-data-science
- #recitation
- #regression
- #linear-regression
- #ridge
- #lasso
- #cross-validation
Toolkit
- 1
A seller has a 7-year-old car. Using the simple model's equation (slope and intercept from above):
- Calculate the predicted price by hand.
- Verify with
simple_model.predict([[7]]). - The actual listing price is ₪82K. What is the residual? Did the model over- or underestimate?
- Is R² good enough for Dana's pricing engine? Why or why not?
- 2
- Compare the
car_age_yearsandkm_driven_thousandscoefficients for Ridge and Lasso (both trained on standardised features, so they are directly comparable). - Which model shrinks coefficients more aggressively?
- Why does regularisation help with multicollinearity?
- Compare the
Recitation walkthrough
Setup, EDA and split
import pandas as pd, numpy as np
from sklearn.model_selection import train_test_split, cross_val_score
from sklearn.linear_model import LinearRegression, Ridge, Lasso
from sklearn.preprocessing import PolynomialFeatures, StandardScaler
from sklearn.metrics import mean_absolute_error, mean_squared_error, r2_score
df = pd.read_csv("autobid_used_cars.csv")
X = df.drop(columns=["listing_price_thousands", "listing_id"])
y = df["listing_price_thousands"]
# One-hot encode categoricals; drop_first=True makes BMW (brand) and Diesel (fuel) the baselines
X = pd.get_dummies(X, columns=["fuel_type", "car_brand", "city"], drop_first=True)
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2, random_state=42)
What the EDA showed (the notebook's words): price is right-skewed (mean above median, a tail of luxury and electric cars); age vs price is clearly linear and downward; engine size vs price is curved (steep then flat); and car_age_years and km_driven_thousands are strongly correlated (r about 0.75), which is Multicollinearity.
Simple regression: one feature
simple_model = LinearRegression()
simple_model.fit(X_train[["car_age_years"]], y_train)
print(simple_model.coef_[0], simple_model.intercept_) # slope = depreciation per year, intercept = price at age 0
The slope is read as depreciation per year and the intercept as the predicted price of a brand-new car. Because km_driven is missing, the age slope soaks up the km effect too.
Multiple regression: every feature gets a "price tag"
full_model = LinearRegression().fit(X_train, y_train)
r2_full = r2_score(y_test, full_model.predict(X_test))
Read each coefficient as "all else equal, this feature changes the predicted price by this much". Examples quoted in the notebook: car_brand_Hyundai is about −₪44K against the BMW baseline, and fuel_type_Electric is about +₪35K against Diesel.
Why the age coefficient shrinksWhen
km_driven_thousandsjoins the model,car_age_yearsshrinks, because the two share information and the shared effect is now split between them. Remove km and age grows back toward its simple-regression value.
Polynomial term on one feature
poly = PolynomialFeatures(degree=2, include_bias=False)
engine_train = poly.fit_transform(X_train[["engine_size_cc"]]) # columns: engine_cc, engine_cc^2
The notebook adds engine_size_cc² only to the one feature that visibly curves. It then shows the over-the-top version: polynomial features on all numeric columns at degree 1, 2 and 3. Training R² keeps climbing but test R² drops at degree 3: classic overfitting (see Bias-Variance Tradeoff).
Regularisation: Ridge and Lasso
scaler = StandardScaler()
X_train_sc = pd.DataFrame(scaler.fit_transform(X_train), columns=X_train.columns, index=X_train.index)
X_test_sc = pd.DataFrame(scaler.transform(X_test), columns=X_test.columns, index=X_test.index)
ridge_model = Ridge(alpha=1.0).fit(X_train_sc, y_train)
lasso_model = Lasso(alpha=0.5).fit(X_train_sc, y_train)
- Ridge (L2) shrinks all coefficients towards zero; none vanish.
- Lasso (L1) can shrink some coefficients exactly to zero, which is automatic feature selection. In the notebook Lasso dropped
has_sunroofamong others. - You must standardise first: the penalty is on coefficient size, so a feature measured in thousands of cc (1000 to 3500) would be penalised unfairly next to a 0/1 feature.
Cross-validation and the grand comparison
cv_scores = cross_val_score(LinearRegression(), X, y, cv=5, scoring="r2")
from sklearn.pipeline import make_pipeline
models = {
"Linear Regression": LinearRegression(),
"Ridge (α=1.0)": make_pipeline(StandardScaler(), Ridge(alpha=1.0)),
"Lasso (α=0.5)": make_pipeline(StandardScaler(), Lasso(alpha=0.5)),
}
Putting the scaler inside a make_pipeline means each CV fold fits the scaler on its own training part only, which prevents leakage (see Scikit-Learn Pipeline). The notebook's recommendation for Dana: Ridge for production (stable, accurate, interpretable, copes with correlated features), Lasso as a side analysis of which features matter. Its quoted figure is a CV R² of about 0.76 with a std of about 0.03.
Related: Lecture 4 - Regression, Lecture 3 - Classification, KNN, Random Tree
SourceOfficial course notebook
recitation4.ipynb(Recitation 4, Spring 2026). The two exercises and their solutions come straight from the notebook. The notebook had no saved outputs and its CSV is not available, so numbers below are only those the notebook's own text states.
What this problem set is about
Recitation 3 predicted labels (churn). Here you predict a number: the listing price of a used car for the marketplace AutoBid. The pattern is the same, create → .fit() → .predict(), but accuracy is replaced by MAE, RMSE and R-Squared.
The AutoBid caseDana (AutoBid's product lead) wants a price prediction engine: sellers who overprice get no offers, sellers who underprice lose money. The data has 350 listings with
car_age_years,km_driven_thousands,engine_size_cc,num_previous_owners,has_sunroof,fuel_type,car_brand,cityand the targetlisting_price_thousands(in ₪K).
Recap
| Item | Tool | Takeaway |
|---|---|---|
| Q1 | LinearRegression, predict, residual |
Prediction = intercept + slope × 7; residual = actual − predicted; R² about 0.55 is too low for a pricing tool |
| Q2 | Ridge, Lasso on standardised data |
Regularisation stabilises correlated features; Lasso can also drop them |