Heteroskedasticity
- #intro-econometrics
- #heteroskedasticity
- #weighted-least-squares
- #white-correction
- #robust-standard-errors
- #GLS
- #OLS
Heteroskedasticity
Part of: Intro Econometrics | Key concepts: Heteroskedasticity, Weighted Least Squares, White Correction, Robust Standard Errors, Gauss-Markov Theorem, Classical Assumptions, Generalized Least Squares
Where this lecture sits
The thread from the classical modelThe OLS inference machinery — the t-statistics and F-statistics we have leaned on since Lec_08-Goodness of Fit & Multicollinearity — rests on a quiet but crucial assumption: that every observation's error has the same variance. The last lecture (Lec_11-Interaction Terms) already hinted at the cure when it noted that running separate regressions per group lets each group's error variance differ. This lecture confronts that assumption head-on. When heteroskedasticity is present, OLS coefficients are still fine, but the standard errors are wrong, and every hypothesis test built on them is invalid. We need either to (a) transform the model so the errors become well-behaved (WLS), or (b) keep OLS but compute a corrected standard error that is robust to the problem (White correction).
The lecture has three strands:
- The diagnosis: what heteroskedasticity is, where it comes from, exactly which results break (and which survive), and an informal "eyeball" test.
- Weighted Least Squares (WLS): when we know the functional form of the variance, transform the model to restore the classical assumptions and recover efficiency.
- White robust standard errors: when we don't know the form, keep the OLS estimates but fix the standard-error formula.
Part 0 — Recall: the assumption at stake (A4)
The classical assumption of homoskedasticityIn the linear model with , assumption A4 (homoskedasticity) states the conditional variance of the error is constant across all observations:
Heteroskedasticity is precisely the failure of A4: the error variance depends on — on the level of an explanatory variable, on group membership, or on time:
The classic intuition — returns to educationWhen estimating the returns to education, ability is unobserved and sits inside the error . At low education levels there is little spread in ability; among the highly educated, ability varies enormously (some have a PhD by talent, some by persistence). So the spread of — its variance — grows with the education level. The variance of the thing we cannot see is itself a function of the thing we can.
Part 1 — Where heteroskedasticity comes from
The error variance can be non-constant for several structurally different reasons. Recognising which one you face dictates the fix.
| Pattern | Mechanism | Canonical example |
|---|---|---|
| Varies with a regressor | rises/falls with an | Income–consumption: spending is more variable at high income |
| Differs across groups | Each subpopulation has its own variance | Wage model: differs for men vs women |
| Changes over time | Stable vs turbulent periods | Macro models: calm decades vs crisis periods |
Heteroskedasticity is not "in the model" — it is in the errorsThe variance can depend on a variable exogenously, even one not included as a regressor. The point is about the spread of the unobserved , not about which controls you put in the equation. Adding the variable as a control does not automatically cure it.
Part 2 — The problem: what breaks and what survives
This is the single most important slide of the lecture, and the most common exam trap. Be precise about which results die and which live.
Consequences of heteroskedasticityWhat SURVIVES (the good news):
- OLS estimators are still unbiased and consistent. Assumptions A1 (linearity) and A2 (mean independence ) are untouched — and unbiasedness rests only on those. The coefficient point estimates are fine.
What BREAKS (the bad news):
- The usual formula for assumes constant variance (A4). Under heteroskedasticity it is biased and inconsistent.
- Therefore the standard errors are wrong, so t-statistics and F-statistics are invalid — you cannot trust any hypothesis test or confidence interval.
- Gauss–Markov no longer holds, so OLS is no longer efficient (not BLUE) — some other linear unbiased estimator has a smaller variance.
The practical reading of R outputUnder heteroskedasticity, in the
summary(ols)table only the first column (the coefficient Estimates) is reliable. The Std. Error, t value, and Pr(>|t|) columns are all built on the broken variance formula and must not be trusted until corrected.
Why unbiasedness survives but efficiency dies — the intuitionUnbiasedness is a statement about the average of over repeated samples; it only needs the error to be mean-zero and independent of (A1–A2). The error variance never enters that argument. Efficiency, by contrast, is about how tightly clusters — and that depends entirely on the variance structure. OLS weights every observation equally; when some observations are far noisier than others, equal weighting is wasteful, so a better (smaller-variance) estimator exists.
Part 3 — Detecting it: the "eyeball" test
Since heteroskedasticity lives in the variance of the unobservable , we proxy with the residual and look at how its squared size moves with a suspect regressor.
The eyeball (graphical) testFor :
- Estimate the model by OLS to get .
- Compute the squared residuals . (Squaring because variance is about magnitude, not sign — is the natural sample analogue of when the mean is zero.)
- Plot (vertical) against the suspicious exogenous variable .
ols <- lm(lwage ~ schooling + age + age_sq, mydata) plot(ols$model[,'schooling'], (ols$residuals)^2)
What the cloud of points tells you:
| Shape of vs | Diagnosis | Implied form |
|---|---|---|
| Flat band, no trend | Homoskedastic | |
| Fans out as rises | Increasing variance | |
| Fans in as rises | Decreasing variance |
Why "eyeball"This is an informal visual diagnostic, not a formal hypothesis test (formal tests like Breusch–Pagan or White's test exist but are not the focus here). It tells you both whether there is a problem and, crucially, what functional form the variance might take — which is exactly the input WLS needs.
Part 4 — Weighted Least Squares (known variance form)
The strategyIf we know the functional form of the error variance — that it is times a known function of the regressors — we can transform the model so the transformed errors are homoskedastic, then run OLS on the transformed model. This is Weighted Least Squares (WLS).
The transformation
Suppose
Divide every term of the original equation by :
Rewrite as a new starred model with transformed variables , , and :
The coefficients are the same 'sThe transformation does not change what we are estimating — the in model are the same structural parameters as in the original. We have only re-weighted the observations. (Note the constant becomes a genuine regressor — the transformed model often has no ordinary intercept.)
Why the transformed error is homoskedastic — the derivation
This is the heart of the lecture; the exam can ask you to reproduce it.
Mean zero is preserved. Since is a function of alone (treated as fixed given ):
The variance becomes constant. Because the conditional mean is zero, , and pulls out of the expectation as a constant:
The payoffIn model the errors satisfy all the classical assumptions, including A4 (, constant). Gauss–Markov therefore applies to : the OLS estimates of are efficient (BLUE) and their standard errors are unbiased. The WLS estimates differ numerically from the original OLS estimates — but both are unbiased; WLS is simply the more efficient of the two.
Worked WLS examples — picking the right divisor
The whole trick is matching the divisor to the assumed variance form so that the leftover variance collapses to a constant .
Example 1 —The variance is proportional to , so and we divide by . Checking the transformed variance: (The original slide writes the proportional-to- shorthand ; the operative case here divides by the square root of the variance factor.)
Example 2 —The variance is proportional to , so divide by (i.e. ). The corrected model is Notice now appears as the intercept of the new model, and is the single slope variable. Transformed variance:
Example 3 —Here variance falls with , so we multiply by (equivalently ). The corrected model: Transformed variance:
The unifying ruleYou are scaling each observation by the inverse of its error's standard deviation, so noisy and quiet observations end up equally reliable. After the division, always verify the leftover variance reduces to a constant.
WLS as re-weighting — the deeper view
WLS minimises a weighted sum of squared residualsEquivalently, WLS solves Each observation is weighted by the inverse of its error variance. OLS weights every observation equally; WLS down-weights observations with high .
Why inverse-variance weighting is the right thingAn observation with a large error variance is less reliable — its carries more noise about the true line. Trusting it as much as a precise observation throws away information. Inverse-variance weighting gives each observation exactly the say it deserves. OLS is the special case of WLS where all variances are equal; and WLS is itself a special case of Generalized Least Squares (GLS) (not covered in this course).
Part 5 — White robust standard errors (unknown variance form)
When you don't know the formWLS needs you to know — and to have for all . Often you suspect heteroskedasticity but cannot specify its shape (or the implied weights come out negative). The White correction keeps the ordinary OLS estimates and only replaces the standard-error formula with one that remains valid under arbitrary heteroskedasticity. These are also called robust or heteroskedasticity-consistent standard errors.
Where the usual formula fails — the Lecture 4 derivation revisited
Recall for the single-regressor model the slope error decomposes as
Taking the variance (the constant drops; the denominator is a constant so it squares and pulls out; A2 lets the 's be treated as fixed):
This is exactly as far as you can go without A4Getting from the sum's variance to the sum of the variances uses A3 (errors uncorrelated across observations). The step that we cannot take without A4 is replacing every with a single common . Under homoskedasticity that substitution collapses the numerator to and yields the familiar . Under heteroskedasticity each is different, so the clean formula is simply wrong.
White's fix — plug in the squared residuals
White's insight: we cannot know each , but we can estimate it observation-by-observation with the squared residual . For coefficient in the multiple-regression model:
where are the residuals from regressing on all the other regressors (the part of orthogonal to the rest), and are the residuals from the main regression of on the 's.
What White's correction does and does not do
- It does not change the coefficient estimates — the model still has heteroskedasticity, so OLS is still not efficient.
- It does produce a standard error that accounts for the heteroskedasticity, hence is unbiased/consistent — so the t- and F-tests become valid again.
- Trade-off versus WLS: White is robust and assumption-light but leaves you with inefficient estimates; WLS restores efficiency but demands you know (and correctly specify) the variance form.
Doing it in R
Thecoeftestworkflowols <- lm(lwage ~ schooling + age + age_sq, mydata) coeftest(ols, vcov = hccm, type = "hc0") # libraries: car, zoo, lmtestThe output looks like
summary(ols), but:
- The Estimate column is identical to plain OLS (coefficients unchanged).
- The Std. Error column is recomputed robustly, so the t-statistic and p-value columns differ.
type = "hc0"selects one of several flavours of the correction (hc0, hc1, hc3, …) that differ in small-sample degrees-of-freedom adjustments.
Part 6 — Heteroskedasticity across groups
When the groups themselves have different variancesIf the heteroskedasticity is between groups (men vs women) or time periods (calm vs crisis), the cleanest fix is to estimate a separate regression for each group. This directly allows each group its own error variance.
This ties back to the two versions of the Chow test:
| Pooled with dummies/interactions | Separate regressions per group | |
|---|---|---|
| Error variance assumption | Common across groups | Group-specific (each estimated separately) |
| Under homoskedasticity | Identical results | Identical results |
| Under group heteroskedasticity | Inefficient | Preferred — yields efficient coefficient estimates |
The Chow connection, sharpenedUnder homoskedasticity the dummy-version and separate-regression versions of the Chow test are identical. The moment the groups differ in error variance, the separate-regression version is preferred because each regression uses its own variance and so produces efficient estimates — the pooled-dummy version is stuck with one common variance.
Part 7 — WLS vs White: the decision
WLS summary
- Requires knowing the functional form of , with positive weights .
- The coefficient estimates change (different from OLS), and so does .
- The standard errors and hypothesis tests change.
- The estimates are efficient (BLUE).
White correction summary
- No need to know the functional form of .
- The coefficient estimates do not change, so neither does .
- The standard errors and hypothesis tests change.
- The estimates are not efficient.
Which to use — the practical rule
Use WLS when… Use White when… You "know" the variance is proportional to a known individual-level variable You don't know the functional form of the variance for all Weights would be negative / form unknown You want efficient (BLUE) estimates You don't mind inefficient estimates, just want valid tests In modern applied practice White/robust standard errors are the default workhorse precisely because they ask so little: you rarely know the true variance form, and getting valid inference usually matters more than squeezing out the last drop of efficiency.
Practice exam — model choice, OVB and assumptions
This question appeared on the slides as a worked exam item (medium/high difficulty). It revisits A2/A4 and omitted-variable reasoning — good integrative revision.
SetupResearchers believed and alone explain (all of strictly positive): A researcher suspects also matters and estimates by OLS:
1(a) — On which number did the researcher conclude affects the outcome? The p-value of , here . It is far below 5%, so the coefficient is significant and the variable is relevant.
1(b) — On which numbers did the researcher conclude model (3) beats model (2)? The values: (the RSSs are equally acceptable). Model (3) explains more variation, so the specification fits better.
2 — If and are correlated, which assumption does model (1) violate? (assume model 3 is the true DGF)If model (3) is true, model (1) omits the relevant regressor , which is absorbed into the error . Since and are correlated, the error in model (1) is now correlated with the included regressor : . This violates A2 — answer (ii), . Note it is A2 that fails, not A4 (this is an omitted-variable/endogeneity problem, not a variance problem) — a deliberate trap in a heteroskedasticity lecture.
3 — If and are uncorrelated, how do and compare?Omitting normally biases the other coefficients (OVB), so we'd expect . But OVB only transmits through correlation between the omitted and included regressors. If , dropping does not bias the coefficient on : — answer (iii).
Summary
- Heteroskedasticity is the failure of assumption A4: the conditional error variance varies across observations instead of being a constant . It arises when the variance moves with a regressor, differs across groups, or changes over time.
- Consequences: OLS coefficients stay unbiased and consistent (A1, A2 untouched), but the usual variance estimate is biased, so t- and F-tests are invalid and OLS is no longer efficient (Gauss–Markov fails). In R output, only the Estimate column is trustworthy.
- Detection: the informal eyeball test — plot squared residuals against a suspect regressor. A fanning pattern signals heteroskedasticity and even suggests the functional form.
- Weighted Least Squares (WLS) — when the variance form is known: divide every term by . The transformed errors satisfy A4 (), so the estimates become efficient (BLUE) with valid standard errors. The boxed rule: ; always verify the leftover variance collapses to a constant.
- WLS is inverse-variance weighting: . Noisy (high-variance) observations get less weight because they are less reliable. OLS is the equal-weight special case; WLS is a special case of GLS.
- White / robust standard errors — when the form is unknown (or weights would be negative): keep OLS estimates, replace the standard error with , plugging in for each unknown . Coefficients and unchanged (still inefficient), but tests become valid. In R:
coeftest(ols, vcov = hccm, type = "hc0"). - Across groups/time, estimate separate regressions per group — this lets each group keep its own variance and is the preferred Chow-test version under heteroskedasticity.
- Choosing: WLS for efficiency when you trust the variance form; White when you don't — robust SEs are the modern default because valid inference usually beats squeezing out efficiency.
Related Notes
- Previous: Lec_11-Interaction Terms — separate-group regressions and the Chow test, whose efficiency argument this lecture completes
- Foundational: Lec_08-Goodness of Fit & Multicollinearity — the t- and F-tests that heteroskedasticity invalidates, and the auxiliary-regression residuals reused in White's formula
- Earlier: Lecture 4 — the slope-variance derivation that White's correction picks up where homoskedasticity is dropped
- Concepts: Heteroskedasticity, Weighted Least Squares, White Correction, Robust Standard Errors, Generalized Least Squares, Gauss-Markov Theorem, Classical Assumptions, Chow Test, Omitted Variable Bias
- Hub: Intro Econometrics