Week 8

Goodness of Fit & Multicollinearity

Goodness of Fit & Multicollinearity

Part of: Intro Econometrics | Key concepts: R-squared, Adjusted R-squared, Total Sum of Squares, Multicollinearity, Variance of the OLS Estimator, F-test


Where this lecture sits

The thread from last week

Last week was about joint hypothesis testing with the F-test: build a restricted and an unrestricted model, compare their fit, and ask whether the extra variables in the unrestricted model add anything. This week pushes on the same idea — how well does a model fit? — but turns it into a single summary number, R-squared, and then studies a problem that quietly inflates the variance of our estimates: multicollinearity.

The lecture has two halves:

  1. Goodness-of-fit — defining R2R^2, deriving it from the way OLS splits the variance of yy, its limitations, and the fix (adjusted R2R^2). Plus a neat re-write of the F-statistic purely in terms of R2R^2.
  2. Multicollinearity — what happens when the explanatory variables are linearly related to each other. Perfect collinearity makes estimation impossible; partial collinearity makes it imprecise.

Part 1 — Goodness-of-Fit: the R2R^2 statistic

The basic idea

Every fitted OLS model splits each observed outcome into two pieces:

yi=y^i⏟explained by the model+u^i⏟residual (unexplained)y_i = \underbrace{\hat{y}_i}_{\text{explained by the model}} + \underbrace{\hat{u}_i}_{\text{residual (unexplained)}}

where y^i=β^0+β^1x1i+⋯+β^KxKi\hat{y}_i = \hat\beta_0 + \hat\beta_1 x_{1i} + \dots + \hat\beta_K x_{Ki} is the predicted value. A good model makes the explained part big and the residual part small. R2R^2 turns "big" and "small" into a percentage.

The intuition in one line

R2R^2 = the fraction of the variance in yy that the model accounts for. The closer to 1, the more of the ups-and-downs in yy your xx's capture.

Two algebraic properties of OLS we need

To define R2R^2 cleanly we first need two facts that fall straight out of the OLS first-order conditions (FOCs). Recall that minimising ∑u^i2\sum \hat u_i^2 gives K+1K+1 FOCs:

∑i=1nu^i=0,∑i=1nu^ix1i=0,…,∑i=1nu^ixKi=0\sum_{i=1}^{n}\hat u_i = 0, \qquad \sum_{i=1}^{n}\hat u_i x_{1i} = 0, \quad \dots, \quad \sum_{i=1}^{n}\hat u_i x_{Ki} = 0

The very first one — ∑u^i=0\sum \hat u_i = 0 — comes from the intercept β^0\hat\beta_0, and it is the workhorse of everything below.

Property (1): the predicted values have the same mean as yy

1N∑iyi=1N∑iy^i\frac{1}{N}\sum_{i} y_i = \frac{1}{N}\sum_i \hat y_i Derivation: sum yi=y^i+u^iy_i = \hat y_i + \hat u_i over all ii and divide by NN: 1N∑yi=1N∑y^i+1N∑u^i.\frac1N\sum y_i = \frac1N\sum \hat y_i + \frac1N\sum \hat u_i. The last term is zero by the intercept FOC (∑u^i=0\sum \hat u_i = 0), so yˉ=y^ˉ\bar y = \bar{\hat y}. The regression line passes through the point of sample means.

Property (2): predicted values and residuals are uncorrelated

cov⁡(y^i,u^i)=0\operatorname{cov}(\hat y_i, \hat u_i) = 0 Derivation sketch: write cov⁡(y^i,u^i)=1N−1∑(y^i−y^ˉ)u^i\operatorname{cov}(\hat y_i,\hat u_i) = \frac{1}{N-1}\sum(\hat y_i - \bar{\hat y})\hat u_i. Substituting y^i=β^0+β^1x1i+…\hat y_i = \hat\beta_0 + \hat\beta_1 x_{1i}+\dots and pulling the constants out of the sum, every term becomes one of the FOC sums (∑u^i=0\sum \hat u_i = 0, ∑xjiu^i=0\sum x_{ji}\hat u_i = 0), all of which equal zero. Hence the covariance is exactly zero — by construction, not by luck.

Partitioning the variance

Because the predicted values and residuals have zero covariance, the variance of yy splits cleanly:

var⁡(yi)=var⁡(y^i)+var⁡(u^i)+2cov⁡(y^i,u^i)⏟= 0\operatorname{var}(y_i) = \operatorname{var}(\hat y_i) + \operatorname{var}(\hat u_i) + 2\underbrace{\operatorname{cov}(\hat y_i, \hat u_i)}_{=\,0}

Multiplying the sample-variance version through by N−1N-1 gives the sum-of-squares decomposition:

∑i(yi−yˉ)2⏟TSS=∑i(y^i−yˉ)2⏟MSS+∑iu^i2⏟RSS\boxed{\underbrace{\sum_{i}(y_i - \bar y)^2}_{\text{TSS}} = \underbrace{\sum_i(\hat y_i - \bar y)^2}_{\text{MSS}} + \underbrace{\sum_i \hat u_i^2}_{\text{RSS}}}
Term Name Meaning
TSS — Total Sum of Squares ∑(yi−yˉ)2\sum(y_i-\bar y)^2 Total variation in the outcome
MSS — Model Sum of Squares ∑(y^i−yˉ)2\sum(\hat y_i-\bar y)^2 Variation the model explains
RSS — Residual Sum of Squares ∑u^i2\sum \hat u_i^2 Variation the model leaves unexplained
Where Property (1) sneaks in

The MSS term is centred on yˉ\bar y, not y^ˉ\bar{\hat y}. We are only allowed to do that because Property (1) told us y^ˉ=yˉ\bar{\hat y} = \bar y. Without it, the cross-term wouldn't vanish and the tidy decomposition would fall apart.

Defining R2R^2

Divide the decomposition through by var⁡(yi)\operatorname{var}(y_i) (equivalently by TSS):

1=var⁡(y^i)var⁡(yi)+var⁡(u^i)var⁡(yi)1 = \frac{\operatorname{var}(\hat y_i)}{\operatorname{var}(y_i)} + \frac{\operatorname{var}(\hat u_i)}{\operatorname{var}(y_i)}

The first fraction is the explained share — that's R2R^2:

R2=var⁡(y^i)var⁡(yi)=MSSTSS=1−RSSTSS\boxed{R^2 = \frac{\operatorname{var}(\hat y_i)}{\operatorname{var}(y_i)} = \frac{\text{MSS}}{\text{TSS}} = 1 - \frac{\text{RSS}}{\text{TSS}}}

Because the two shares add to 1 and both are non-negative, 0≤R2≤10 \le R^2 \le 1 and we can read it as a percentage: how much of the variance in the dependent variable the model explains.

Reading R2R^2 in practice

Income–consumption model: R2=0.983R^2 = 0.983 → income explains 98.3% of the variance in consumption; only 1.7% is unexplained. Housing prices (price ~ bdrms + sqrft): R2=0.632R^2 = 0.632 → area and bedrooms explain 63.2% of the variance in apartment prices; 36.8% is left to everything else.

R output
Call: lm(formula = price ~ bdrms + sqrft, data = mydata)
Coefficients:
              Estimate Std. Error t value Pr(>|t|)
(Intercept) -19315.00   31046.62  -0.622    0.536
bdrms        15198.19    9483.52   1.603    0.113
sqrft          128.44      13.82   9.291  1.4e-14 ***
Multiple R-squared:  0.6319,  Adjusted R-squared:  0.6233
F-statistic: 72.96 on 2 and 85 DF,  p-value: < 2.2e-16

The boundary cases

When is R2=0R^2 = 0 or R2=1R^2 = 1?
  • R2=1R^2 = 1 ⟺ all residuals are zero, i.e. every point lies exactly on the regression surface. Random error always exists, so this never truly happens — not even if you observed the entire population.
  • R2=0R^2 = 0 ⟺ the model has no explanatory power at all: the xx's tell you nothing about yy. A model without meaning.

Why R2R^2 can mislead

R2R^2 depends on the variance of the outcome

R2R^2 is sensitive to how spread-out yy is. If var⁡(y)\operatorname{var}(y) is large, R2R^2 can look small even when the model fits well; if yy is a smooth fiscal aggregate it can look huge.

  • Wage models rarely break ~50% — individual wages are noisy.
  • Fiscal series (GDP, consumption, investment, demand) routinely exceed 90% — they move slowly and predictably.

⟹ You may only compare R2R^2 across models if the outcome variable is identical. A model of yy and a model of log⁡y\log y are not comparable on R2R^2.


Adjusted R2R^2 — comparing models of different size

The mechanical flaw in R2R^2

Adding any regressor — even pure noise — weakly increases R2R^2. Why? The new variable can only shrink the RSS (OLS will set its coefficient to 0 in the worst case, never making the fit worse). So R2R^2 rewards throwing variables at the model, which is not what we want.

The fix penalises the model for each extra parameter, using degrees of freedom:

Rˉ2=1−RSS/(N−K−1)TSS/(N−1)=1−(1−R2) N−1N−K−1\boxed{\bar{R}^2 = 1 - \frac{\text{RSS}/(N-K-1)}{\text{TSS}/(N-1)} = 1 - (1-R^2)\,\frac{N-1}{N-K-1}}
How to read it

Adjusted R2R^2 rises only if a new variable improves fit by more than would be expected from a random regressor. If you add junk, Rˉ2\bar R^2 falls even though R2R^2 rises. It is the right tool for choosing between models with different numbers of explanatory variables.

Simulation: adding a noise variable

True model y=β0+β1x1+β2x2+uy = \beta_0 + \beta_1 x_1 + \beta_2 x_2 + u. Now add an unrelated noise variable Z∼N(0,5)Z \sim N(0,5):

  • R2R^2 increases (mechanically, as always)
  • Adjusted R2R^2 decreases — correctly flagging that ZZ added nothing.
Limits that apply to both measures

Neither R2R^2 nor Rˉ2\bar R^2 can compare models with different outcome variables or models with no intercept. Adjusted R2R^2 buys you only one extra thing: comparison across different numbers of regressors.

R2R^2 Adjusted R2R^2
Range [0,1][0,1] Can be negative
Adding a regressor Never decreases Decreases if the variable is useless
Compare different # of regressors? ❌ No ✅ Yes
Compare different outcome variables? ❌ No ❌ No
Compare models without intercept? ❌ No ❌ No
Best use Single-model fit summary Model selection

The F-statistic, rewritten in terms of R2R^2

The F-test for qq restrictions started life as a ratio of residual sums of squares:

F=(RSSR−RSSUR)/qRSSUR/(N−M)F = \frac{(\text{RSS}_R - \text{RSS}_{UR})/q}{\text{RSS}_{UR}/(N-M)}

Divide numerator and denominator by (N−1)var⁡(Y)(N-1)\operatorname{var}(Y) and use RSSN−1=var⁡(u^i)\dfrac{\text{RSS}}{N-1} = \operatorname{var}(\hat u_i) together with var⁡(u^i)var⁡(Y)=1−R2\dfrac{\operatorname{var}(\hat u_i)}{\operatorname{var}(Y)} = 1 - R^2:

F=(RUR2−RR2)/q(1−RUR2)/(N−M)\boxed{F = \frac{(R^2_{UR} - R^2_R)/q}{(1 - R^2_{UR})/(N-M)}}
The hidden assumption

This clean form assumes var⁡(Y)\operatorname{var}(Y) is identical in the restricted and unrestricted models. That holds when the restriction is βj=0\beta_j = 0 (same dependent variable), but fails for restrictions of the form H0:βj=bH_0: \beta_j = b with b≠0b \ne 0 — there you must go back to the RSS version.

Testing whether the whole model is significant

A special case: does the model as a whole explain anything? Restrict every slope to zero.

  • Unrestricted: yi=β0+β1x1+⋯+βkxk+uiy_i = \beta_0 + \beta_1 x_1 + \dots + \beta_k x_k + u_i
  • Restricted: yi=β0+uiy_i = \beta_0 + u_i (so RR2=0R^2_R = 0)
  • Null: H0:β1=β2=⋯=βk=0H_0: \beta_1 = \beta_2 = \dots = \beta_k = 0

Plugging RR2=0R^2_R = 0 into the formula:

F=RUR2/(M−1)(1−RUR2)/(N−M)F = \frac{R^2_{UR}/(M-1)}{(1-R^2_{UR})/(N-M)}

This is exactly the F-statistic printed at the bottom of every lm summary. A small p-value means at least one regressor matters.

Wage model — overall significance

ln_wage ~ s + exp + IQ_test + age + hours: R2=0.301R^2 = 0.301, F = 46.02 on 5 and 534 df, p < 2.2e-16. The model is jointly highly significant — reject H0H_0 that none of the variables explain wages.


Part 2 — Multicollinearity

Multicollinearity is a linear relationship among the explanatory variables themselves. It comes in two flavours, with very different consequences.

Perfect multicollinearity — estimation is impossible

Definition

One regressor is an exact linear function of one or more others, e.g. x1=5x2x_1 = 5x_2, or x1=−x2x_1 = -x_2, or x1=x2+x3+x4x_1 = x_2 + x_3 + x_4. Then OLS cannot be estimated.

Crucially, this is about linear dependence only. Variables can be strongly — even perfectly — correlated in a non-linear way and still be fine:

  • cor⁡(x1,x2)=0.57\operatorname{cor}(x_1, x_2) = 0.57 → fine
  • x2=x12x_2 = x_1^2 → fine (a quadratic is not a linear transformation)
Where it really comes from: misspecification

Perfect collinearity is almost always a modelling mistake, such as:

  • the same variable entered twice in different units (age in months and age in years),
  • two variables linked by an exact identity (age in years and year of birth).

R's lm silently drops one of the offending variables and reports NA:

R output
Coefficients: (1 not defined because of singularities)
            Estimate Std. Error t value  Pr(>|t|)
educ        0.058048   0.006694   8.672   < 2e-16 ***
age         0.024560   0.004878   5.035   6.2e-07 ***
y_birth           NA         NA      NA        NA
Why it breaks — the FOC argument

Suppose x1=−x2x_1 = -x_2. The OLS FOCs include ∑u^ix1i=0and∑u^ix2i=0.\sum \hat u_i x_{1i} = 0 \quad\text{and}\quad \sum \hat u_i x_{2i} = 0. Substituting x1i=−x2ix_{1i} = -x_{2i} into the first gives −∑u^ix2i=0-\sum \hat u_i x_{2i} = 0 — which is just the negative of the second FOC, carrying no new information. We are left with KK independent equations for K+1K+1 unknowns: an underdetermined system with infinitely many solutions. No unique β^\hat\beta exists.

Partial multicollinearity — estimation is possible but imprecise

When the xx's are correlated but not perfectly, none of the Gauss–Markov assumptions break. OLS stays BLUE (unbiased, minimum variance among linear unbiased estimators). The catch:

The cost of partial collinearity

The estimator's variance — though still minimal in the BLUE sense — is larger the more the regressors are correlated. Bigger variance ⟹ bigger standard errors ⟹ smaller t-stats ⟹ harder to find anything "significant", even when the true effect is real.

The variance-of-the-estimator formula

In a single-variable regression:

var⁡(β^1)=σu2∑i(x1i−xˉ1)2\operatorname{var}(\hat\beta_1) = \frac{\sigma_u^2}{\sum_i (x_{1i} - \bar x_1)^2}

In a multivariate regression the formula gains one extra factor:

var⁡(β^j)=σu2(1−Rj2)∑i(xji−xˉj)2\boxed{\operatorname{var}(\hat\beta_j) = \frac{\sigma_u^2}{(1 - R_j^2)\sum_i (x_{ji} - \bar x_j)^2}}

where Rj2R_j^2 is the R2R^2 from regressing xjx_j on all the other explanatory variables:

xji=β~0+β~1x1i+⋯+β~k−1xki+u~i(omitting xj)x_{ji} = \tilde\beta_0 + \tilde\beta_1 x_{1i} + \dots + \tilde\beta_{k-1} x_{ki} + \tilde u_i \quad(\text{omitting } x_j)
How to read each piece of the formula
  • σu2\sigma_u^2 ↑ → more noise in yy → larger variance.
  • ∑(xji−xˉj)2\sum(x_{ji}-\bar x_j)^2 ↑ → more spread in the regressor → smaller variance (more leverage to identify the slope).
  • Rj2R_j^2 ↑ → xjx_j is well explained by the other regressors (high collinearity) → (1−Rj2)(1-R_j^2) shrinks → variance explodes. This is the multicollinearity penalty. 1/(1−Rj2)1/(1-R_j^2) is the Variance Inflation Factor.
Activity 8 — the two extremes of Rj2R_j^2

Model yi=β0+β1x1i+β2x2i+uiy_i = \beta_0 + \beta_1 x_{1i} + \beta_2 x_{2i} + u_i.

  • R12=1R_1^2 = 1: x1x_1 is a perfect linear function of x2x_2 — perfect multicollinearity. (1−R12)=0(1-R_1^2)=0, so var⁡(β^1)\operatorname{var}(\hat\beta_1) is undefined (division by zero). This matches the "impossible to estimate" result above.
  • R12=0R_1^2 = 0: zero collinearity between x1x_1 and x2x_2. The variance of β^1\hat\beta_1 is the same as in the simple regression yi=β0+β1x1i+uiy_i = \beta_0 + \beta_1 x_{1i} + u_i (only the error's standard deviation may differ).
The classic bias–variance / fit–precision trade-off

Adding regressors tends to raise Rj2R_j^2 for the variables already in the model (they get more correlated company), so each estimate's variance climbs and its significance falls — even as the overall model fit (R2R^2) improves. More variables = better fit but noisier individual coefficients.

Rule of thumb and diagnosis

Spotting partial multicollinearity

Rj2R_j^2 lives in [0,1][0,1]. Exactly 0 or exactly 1 are both rare. The practical danger zone is Rj2R_j^2 close to 1 — rule of thumb > 0.75–0.8. Diagnosis is cheap: before running the regression, inspect the correlation matrix of the explanatory variables.


Worked example: parental education in a wage regression

The model

ln⁡(wagei)=β0+βE educi+βA agei+βX experi+βQ IQi+βM meduci+βF feduci+ui\ln(\text{wage}_i) = \beta_0 + \beta_E\,\text{educ}_i + \beta_A\,\text{age}_i + \beta_X\,\text{exper}_i + \beta_Q\,\text{IQ}_i + \beta_M\,\text{meduc}_i + \beta_F\,\text{feduc}_i + u_i where meduc/feduc are mother's/father's years of schooling. Question: does family background affect wages?

Step 1 — the full regression. Both parental-education coefficients look insignificant:

R output
             Estimate Std. Error t value Pr(>|t|)
educ         0.042323   0.009237   4.582  5.5e-06 ***
age          0.016799   0.005709   2.943  0.00337 **
exper        0.015232   0.004584   3.323  0.00094 ***
IQ           0.005082   0.001188   4.279  2.2e-05 ***
meduc        0.017660   0.009823   1.798  0.07267 .
feduc        0.001900   0.009325   0.204  0.83859
Multiple R-squared:  0.1965,  Adjusted R-squared:  0.1891

It is tempting to drop both and conclude "family background doesn't matter." Don't — test them jointly first.

Step 2 — the joint F-test. H0:βM=βF=0H_0:\beta_M = \beta_F = 0, via anova(full, restricted):

R output
Model 1: lwage ~ educ + age + exper + IQ + meduc + feduc
Model 2: lwage ~ educ + age + exper + IQ
  Res.Df    RSS Df Sum of Sq      F  Pr(>F)
1    655 90.381
2    657 92.283 -2   -1.9017 6.8908 0.00109 **

The joint test rejects at 1%. So we cannot remove both — together they matter, even though neither is individually significant.

The t-test vs F-test contradiction — and its cause

Each individual t-test says "insignificant"; the joint F-test says "significant." This is the textbook fingerprint of multicollinearity. The correlation matrix reveals why:

R output
       feduc meduc
feduc   1.00  0.88
meduc   0.88  1.00

Mother's and father's education are correlated at 0.88. Each inflates the other's standard error (via the 1/(1−Rj2)1/(1-R_j^2) factor), drowning their individual t-stats — yet jointly they clearly explain wages.

Step 3 — confirm by dropping one. Remove feduc and meduc jumps to significant; remove meduc and feduc jumps to significant. Each only looked weak because of the other's presence.

The practical conclusion

Don't include two near-identical regressors. Use only meduc, or only feduc, or their average as a single "parental education" measure. The clean result: each year of a parent's education raises the child's wage by ~2%.


Practice exam: the income–consumption model

Setup

CONSi=β0+β1 INCi+ui\text{CONS}_i = \beta_0 + \beta_1\,\text{INC}_i + u_i 34 yearly observations. INC‾=70,941.9\overline{\text{INC}} = 70{,}941.9, CONS‾=35,956.1\overline{\text{CONS}} = 35{,}956.1. Estimates: β^0=−2189 (se 1377)\hat\beta_0 = -2189\ (se\ 1377), β^1=0.8155 (se 0.01914)\hat\beta_1 = 0.8155\ (se\ 0.01914), R2=0.983R^2 = 0.983, df=32df = 32.

Q1 — Predicted consumption at average income? By Property (1), the OLS line passes through the sample means. So at INC=INC‾\text{INC} = \overline{\text{INC}} the prediction is exactly CONS‾=35,956.09\overline{\text{CONS}} = \mathbf{35{,}956.09} — no arithmetic needed.

Q2 — Does the line pass through the origin? H0:β0=0H_0:\beta_0 = 0 vs H1:β0≠0H_1:\beta_0\ne 0.

t^=−21891377=−15.9,tcrit(df=30, 5%,two-sided)=2.042.\hat t = \frac{-2189}{1377} = -15.9, \qquad t_{crit}(df{=}30,\,5\%,\text{two-sided}) = 2.042.

∣−15.9∣>2.042|-15.9| > 2.042 ⟹ reject H0H_0: the intercept is significantly non-zero, so the line does not pass through the origin.

Q3 — Is the marginal propensity to consume below 90%? One-sided: H0:β1≥0.9H_0:\beta_1 \ge 0.9 vs H1:β1<0.9H_1:\beta_1 < 0.9.

t^=0.8155−0.90.01914=−4.415,tcrit(one-sided, 5%)=1.697.\hat t = \frac{0.8155 - 0.9}{0.01914} = -4.415, \qquad t_{crit}(\text{one-sided},\,5\%) = 1.697.

−4.415<−1.697-4.415 < -1.697 ⟹ reject H0H_0: the MPC is significantly less than 0.9.

Q4 — Re-estimate in NIS instead of USD (exchange rate 3.5 NIS/USD), so both variables are scaled by 3.5: CONS_1=α0+α1 INC_1+u\text{CONS}\_1 = \alpha_0 + \alpha_1\,\text{INC}\_1 + u.

Scaling both sides leaves the slope (and its se) unchanged

Multiplying xx and yy by the same constant cc cancels in the slope: α^1=cov⁡(cy,cx)var⁡(cx)=c2cov⁡(y,x)c2var⁡(x)=β^1\hat\alpha_1 = \dfrac{\operatorname{cov}(cy, cx)}{\operatorname{var}(cx)} = \dfrac{c^2\operatorname{cov}(y,x)}{c^2\operatorname{var}(x)} = \hat\beta_1. So α^1=β^1=0.8155\hat\alpha_1 = \hat\beta_1 = \mathbf{0.8155} and se(α^1)=se(β^1)=0.01914se(\hat\alpha_1) = se(\hat\beta_1) = \mathbf{0.01914} — both identical. (Only the intercept rescales: α^0=3.5 β^0\hat\alpha_0 = 3.5\,\hat\beta_0.)


Summary

  1. OLS splits every yiy_i into a predicted part y^i\hat y_i and a residual u^i\hat u_i. Two FOC-driven properties — y^ˉ=yˉ\bar{\hat y} = \bar y and cov⁡(y^,u^)=0\operatorname{cov}(\hat y,\hat u)=0 — let us split the variance: TSS = MSS + RSS.
  2. R2=MSS/TSS=1−RSS/TSSR^2 = \text{MSS}/\text{TSS} = 1 - \text{RSS}/\text{TSS} is the share of variance explained, ∈[0,1]\in[0,1]. It depends on var⁡(y)\operatorname{var}(y), so only compare it across models with the same outcome.
  3. R2R^2 always rises when you add a variable, even noise. Adjusted R2R^2 penalises extra parameters and is the right tool for comparing models of different size (but still not different outcomes / no-intercept models).
  4. The F-statistic can be written purely in R2R^2: F=(RUR2−RR2)/q(1−RUR2)/(N−M)F = \frac{(R^2_{UR}-R^2_R)/q}{(1-R^2_{UR})/(N-M)}, valid when var⁡(Y)\operatorname{var}(Y) is unchanged. Setting RR2=0R^2_R=0 gives the overall model significance test reported by lm.
  5. Perfect multicollinearity (exact linear dependence) makes the FOCs redundant → no unique solution → estimation impossible. It's almost always a misspecification.
  6. Partial multicollinearity keeps OLS unbiased but inflates variance via var⁡(β^j)=σu2(1−Rj2)∑(xji−xˉj)2\operatorname{var}(\hat\beta_j) = \frac{\sigma_u^2}{(1-R_j^2)\sum(x_{ji}-\bar x_j)^2}. High Rj2R_j^2 (rule of thumb > 0.75–0.8) → tiny t-stats. The tell-tale sign is insignificant t-tests but a significant joint F-test (the parental-education example). Diagnose with the correlation matrix; fix by combining or dropping redundant regressors.