Exam Review
- #intro-econometrics
- #exam-review
- #OLS
- #hypothesis-testing
- #F-test
- #formula-sheet
- #revision
Exam Review
Part of: Intro Econometrics | Key concepts: OLS Estimation, Gauss-Markov Theorem, F-test, t-test, Confidence Interval, Multicollinearity, Omitted Variable Bias, Heteroskedasticity, Dummy Variable, Interaction Term
What this lecture is
A whole-course map, not new materialLecture 13 is the final-exam review. It does two things: (1) explains how to use the formula sheet and the statistical tables you are given in the exam, and (2) walks the entire course from first principles — deriving OLS, listing the classical assumptions and what they buy you, the inference machinery (t, F, confidence intervals, ), the three things that "go wrong" (multicollinearity, irrelevant/omitted variables, heteroskedasticity), and the "fancier" interpretation models (unit changes, regression through the origin, logs, quadratics, dummies, interactions, Chow tests). This note is therefore a single-sitting revision sheet for the whole module, with every result reasoned through rather than just listed.
The course splits into three blocks, mirrored in the sections below:
- Basic building blocks — deriving OLS, the sampling distribution of the estimators, t-tests, p-values, confidence intervals, F-tests, goodness-of-fit.
- Things that can go wrong — multicollinearity, including irrelevant variables, omitting relevant variables, heteroskedasticity.
- Fancier models — changing units, regression through the origin, log models, quadratics, categorical dummies, interaction terms, and Chow tests.
Part 0 — Exam mechanics: the formula sheet & tables
Read the table instructions carefully — they cost easy marksThe exam gives you a formula sheet (on Moodle, in the Practice Exam section: OLS formulas, omitted-variable bias, regression through the origin, inference, etc.) and statistical tables. You must use the printed tables, not R's
qt()/qf()quantile functions — the values can differ slightly and the exam expects the table value.
Degrees of freedom — the number people get wrongEverywhere you read a critical value, you need the right degrees of freedom.
- t-distribution: , where = number of observations and = number of estimated parameters (first-order conditions), including the intercept if one is used.
- One- vs two-tailed: use the 1-tailed column for one-sided tests ( or ) and the 2-tailed column for two-sided tests (). Equivalently, a two-sided test at uses the one-sided critical value at .
- F-distribution: numerator df (number of restrictions); denominator df . There is a separate table for each significance level () — pick the right one.
Activity 13 (part 1) — reading the F-tableModel on observations: where are dummies for the categorical variable (so is the base category). Test whether is jointly significant at : .
- Restrictions (two dummies set to zero).
- Parameters ( through ), so denominator .
- Critical value: read the 5% F-table at numerator , denominator , giving .
- Decision rule: reject (POSITION is significant) if the computed .
1. Basic Building Blocks
1.1 The population model and the sample
The data-generating process (DGP)We assume a population model with outcome , explanatory variable , random error (everything else that affects ), intercept , and slope (the effect of on ). For a random sample of size , observation obeys .
We observe and but never the or the true 's, so cannot be recovered exactly — it must be estimated.
1.2 Deriving the OLS estimators
OLS minimises the sum of squared residualsDefine the residual — the gap between the observed and the fitted value. (Distinguish it from the unobservable error , which is the gap from the true line.) OLS chooses to solve
The two first-order conditions (FOCs) are:
Two equations, two unknowns. Solving FOC (1) gives two facts that recur throughout the course:
Two results that fall straight out of FOC (1)
- The fitted line passes through the sample means: .
- The intercept is pinned down by the slope: .
Substituting back and using FOC (2) gives the slope:
Reading the slope formulais the sample covariance of and divided by the sample variance of . Positive covariance ⇒ positive slope. This is the single most important formula in the course: nearly every later result (OVB, regression through the origin, the WLS transformations) is a manipulation of it.
Multivariate case. For , OLS gives FOCs of the same shape: and for each regressor . The residuals are mechanically orthogonal to every regressor.
1.3 Sampling distribution & the variance of the estimators
Variance of the OLS estimatorsSingle-variable model: Multivariate model (coefficient ): where is the from regressing on all the other regressors. The unknown error variance is estimated from the residuals:
What drives precision — read the variance formulais larger (less precise estimate) when: the error variance is large; there is little spread in (small ); or is highly explained by the other regressors (, i.e. multicollinearity). This one expression already foreshadows Part 2.
1.4 Properties of estimators & the classical assumptions
Three properties
- Sampling distribution: how variation across samples translates into variation in .
- Unbiased: — on average, dead on.
- Efficient: has the smallest variance among all unbiased estimators.
The five classical assumptions
- A1 — mean-zero errors: .
- A2 — exogeneity: for all regressors .
- A3 — no autocorrelation: for .
- A4 — homoskedasticity: (constant).
- A5 — normality: .
What each bundle of assumptions buys you
Result Needs Statement Unbiasedness A1 + A2 Gauss–Markov (BLUE) A1–A4 OLS has the minimum variance of all linear unbiased estimators Normality of A1–A5 , enabling exact t/F inference
Why this table is the spine of the courseEach "problem" in Part 2 is best understood as which assumption fails and therefore which property is lost: omitted variables kill A2 (so biased); heteroskedasticity kills A4 (still unbiased, but no longer efficient and SEs wrong). Memorise which assumption → which casualty.
1.5 Simple hypothesis testing — the t-test
Standardising the estimator into a t-statisticBecause is normal, subtracting its mean and dividing by its standard deviation gives a standard normal; replacing the unknown by turns it into a t-distribution with df:
The four steps of a t-test
- State and . A claim about one parameter; two-sided ( vs ) or one-sided ( vs ). always contains the equality.
- Compute the test statistic under : , plugging in the value that asserts.
- Find the critical value from the t-table at significance and (use for two-sided).
- Decide: reject if the statistic is more extreme than the critical value.
| Two-sided test | One-sided test (right) | |
|---|---|---|
| Hypotheses | vs | vs |
| Critical value | ||
| Reject if | $ | t(\hat\beta_1) |
| Rejection region | both tails ( each) | one tail () |
Significance testing & p-valuesSignificance testing is the special two-sided case vs . The statistic is what R prints. The p-value answers: "if were true, what is the probability of seeing a t-statistic at least as extreme as the one we got?" — formally . It is the lowest significance level at which you would still reject. Small p ⇒ reject.
1.6 Confidence intervals
The CI of an OLS estimatorA confidence interval satisfies . Inverting the t-statistic gives with, for the single-variable model, .
The CI ⇔ two-sided test dualityA two-sided test of at level and the CI are the same test: So you can do a two-sided test just by checking whether the hypothesised value sits in the interval. (A common exam shortcut.)
1.7 Multiple hypothesis testing — the F-test
The five steps of an F-test
- State , — a claim about any number of parameters and how they relate; contains the equalities.
- Write the unrestricted model (the full model).
- Write the restricted model — the model assuming is true. This may create a new dependent variable (if ) or a new explanatory variable (if ).
- Compute the F-statistic from the two residual sums of squares.
- Decide against the F-table critical value.
The F-statistic
- , — residual sums of squares of the restricted and unrestricted models.
- — number of restrictions in (numerator df).
- — df of the unrestricted model ( = parameters/FOCs there). Imposing a restriction can only raise RSS, so and . A large means the restriction hurt the fit a lot ⇒ reject.
How F and t relateWhen (a single restriction) the two tests are equivalent, via For a joint test (e.g. ): if every coefficient is separately significant on its t-test, the joint F-test will be significant too — but the converse can fail (joint significance with individually-insignificant coefficients is the classic multicollinearity signature).
1.8 Goodness-of-fit: and the model-significance F-test
— the fraction of the variance in the model explains. Closer to 1 is "better fit." To compare models with different numbers of regressors, use adjusted , which penalises extra variables.
The model-significance (overall-F) testTests whether the regressors jointly explain anything: . The restricted model is just (so ), and the F-statistic can be written purely in terms: This is the "F-statistic" line at the bottom of R's
summary().
2. Things That Can Go Wrong
The master summary
Problem Assumption hit Bias? Efficiency / SEs Fix Multicollinearity none (data issue) Unbiased High variance, big SEs More data / drop a collinear var / accept it Irrelevant variable none Unbiased Higher variance only Drop it (after checking it's not collinearity) Omitted relevant variable A2 Biased SEs also unreliable Add the variable (need its data) Heteroskedasticity A4 Unbiased Not efficient; SEs wrong WLS (known form) or White (unknown)
2.1 Multicollinearity
DefinitionA linear relationship among the explanatory variables.
- Perfect: an exact linear transformation exists, e.g. for all — OLS cannot be computed (the model is not identified).
- Partial: a strong but not exact linear relation, e.g. .
The effect — inflated varianceRecall . High collinearity ⇒ high ⇒ small ⇒ variance blows up. You can't disentangle the separate effects of the collinear variables. Estimates stay unbiased, just imprecise.
Detecting partial multicollinearityTell-tale signs when and are partially collinear:
- With both in the model: neither is significant.
- With only : it is significant, and adjusted barely changes versus the both-in model.
- With only : same story.
- Rule of thumb: is high (above ~0.7). This doesn't always occur under partial collinearity, but when it does it is a strong indicator.
2.2 Including an irrelevant variable
Consequence — noise, not biasAdding a variable that doesn't belong leaves OLS unbiased (A1–A2 intact). The only cost is higher variance of the estimates. It's easy to spot (the coefficient is insignificant) and easy to fix (drop it and re-estimate).
Don't confuse "irrelevant" with "collinear"A seemingly irrelevant variable (small t-stat) might actually be collinear with another regressor — multicollinearity also produces small t-stats. Before dropping a variable, check the correlation matrix of the regressors and run an F-test for joint significance. Drop only if it's genuinely irrelevant.
2.3 Omitting a relevant variable (OVB)
How the bias arisesSuppose the true DGP (satisfying A1–A4) is , but you estimate . The dropped folds into the error. If and are correlated, the error is now correlated with — this violates A2, and becomes biased.
The bias formula and its signwhere is the true effect of the omitted variable and is the sample covariance between and ( the variance of , always ). The sign of the bias:
biased up biased down biased down biased up Special cases: if () ⇒ no bias; if ⇒ the omitted variable was irrelevant anyway. With several omitted variables the direction is generally indeterminate.
Bias vs variance — the modelling ruleOmitting a relevant variable biases the estimates (a serious, hard-to-detect, hard-to-fix problem); including an irrelevant one only raises variance (mild). Since we always prefer unbiasedness over minimal variance, the rule of thumb is: add any plausibly relevant regressor. A small variance cost is worth avoiding bias.
2.4 Heteroskedasticity
Definition and consequencesA4 fails: is not constant. Then OLS stays unbiased (A1, A2 hold) but the usual SE formula (which assumes constant variance) is biased, so t- and F-tests are invalid, and Gauss–Markov fails so OLS is not efficient. In R output, only the coefficient Estimate column is trustworthy.
The two fixes
Weighted Least Squares (WLS) White correction Needs the variance form? Yes — know with No Coefficients change? Yes (and changes) No ( unchanged) SEs / tests change? Yes Yes Efficient (BLUE)? Yes No WLS: divide every term by so the transformed errors satisfy A4 (verify the leftover variance collapses to a constant). White: keep OLS, plug each unknown with the squared residual to get a robust SE. Across groups/time, run separate regressions so each keeps its own variance.
See Lec_12-Heteroskedasticity for the full derivations.
3. Fancier Models (Interpretation)
3.1 Changing units of measurement
What rescaling does to the estimates
Transformation Intercept Slope Add constant to unchanged Multiply by Add constant to unchanged Multiply by unchanged Standard errors scale the same way as their coefficients, so t-statistics and significance are unaffected by a change of units.
Method for "multiply by " derivationsTo find how new coefficients relate to the originals: (1) rearrange the rescaled equation into form; (2) match ; (3) match . E.g. rearranges to , so and .
3.2 Regression through the origin
No-intercept slopeForcing gives (versus the usual ).
- The two coincide only if .
- Regression through the origin is unbiased only if the true intercept is genuinely zero (). Forcing it otherwise reintroduces bias.
3.3 Logarithms — marginal effect, rate of change, elasticity
The three economic moments
- Marginal effect: (units of per unit of ).
- Rate of change: (% change in per unit of ).
- Elasticity: (% change in per % change in ).
Slope interpretation by model
Model Equation Interpretation of Level–level unit ⇒ units Log–level unit ⇒ percent Log–log percent ⇒ percent (elasticity; no ) Level–log percent ⇒ units
Full moment table — every model has all three moments
Model Marginal effect Rate of change Elasticity Level–level Log–level Log–log
3.4 Quadratic terms
Capturing a turning pointAdding makes the relation parabolic: the effect of on rises then falls (if the coefficient on is negative) or falls then rises (if positive). Use it when you expect the relationship to change direction. Test relevance by the significance of the squared term's coefficient. Rule: as a general rule, don't drop the un-squared (linear) term even if it's insignificant — keep the hierarchy.
3.5 Categorical variables (dummies)
A single dummy shifts the interceptFor where :
- Not in A: .
- In A: . Same slope, intercept shifted by . So is the A-vs-not-A difference, holding fixed.
Representing a -category variableFor a variable with categories, create dummies; the left-out category is the base (control), where all dummies . Each dummy's coefficient is the effect relative to the base. E.g. for with the base: — each group gets its own intercept.
Test categorical significance jointly with an F-testTo ask "does the whole categorical variable matter?" test all its dummies together: via an F-test (not separate t-tests). A single binary category can be tested with a t-test on its one dummy. Including all dummies and an intercept is the dummy-variable trap (perfect multicollinearity) — always omit one.
Ordinal categories ⇒ stepwise interceptsFor an ordered category (Left / Center / Right with Right as base), gives each level its own intercept, producing a stepwise pattern across the ordered groups.
3.6 Interaction terms & Chow tests
Interaction = different slopes per group
- Women (): intercept , slope .
- Men (): intercept , slope . The dummy shifts the intercept; the interaction changes the slope. So interactions let the marginal effect of differ by group.
Chow test — do two groups share one regression?Version 1 (dummies/interactions). Unrestricted: . Restricted: . Standard F-test with Reject if the intercept differs (), the slope differs (), or both.
Version 2 (separate regressions). Fit the model separately for each group (, ). The unrestricted RSS is their sum: testing . Under homoskedasticity the two versions agree; under group heteroskedasticity the separate-regression version is preferred (each group keeps its own variance).
Summary — the one-page recap
- OLS minimises , giving and ; the fitted line passes through and residuals are orthogonal to the regressors.
- The classical assumptions map cleanly to properties: A1+A2 ⇒ unbiased; A1–A4 ⇒ Gauss–Markov (BLUE); A1–A5 ⇒ normal , which powers exact t/F inference.
- t-test (, ): one statistic per restriction; two-sided uses . A CI is the dual of the two-sided test — reject iff lies outside it. The p-value is the smallest that rejects.
- F-test (): for joint restrictions; restricted model embeds (new or new ). . The overall-F (: all slopes zero) tests model significance and can be written in .
- Things that go wrong: multicollinearity (unbiased, inflated variance, joint-sig-but-individually-insig), irrelevant variable (unbiased, more variance — check it's not collinearity before dropping), omitted variable (biased via A2, — prefer unbiasedness, so include relevant vars), and heteroskedasticity (A4 fails → unbiased but inefficient with wrong SEs → WLS if form known, White if not).
- Fancier models: unit changes scale coefficients but not t-stats; regression through the origin is unbiased only if ; logs give level/percent/elasticity readings; quadratics capture turning points (keep the linear term); dummies ( of them, one base category) shift intercepts and are tested jointly by F; interactions change slopes; Chow tests check whether two groups share one regression.
- Exam mechanics: use the printed tables, get df right (: ; : over ), pick the table matching , and watch the one- vs two-tailed distinction.
Related Notes
- Covers/integrates: Lec_08-Goodness of Fit & Multicollinearity, Lec_10-Dummy Variables, Lec_11-Interaction Terms, Lec_12-Heteroskedasticity
- Concepts: OLS Estimation, Classical Assumptions, Gauss-Markov Theorem, t-test, F-test, Confidence Interval, p-value, R-squared, Adjusted R-squared, Multicollinearity, Omitted Variable Bias, Heteroskedasticity, Weighted Least Squares, White Correction, Dummy Variable, Base Category, Dummy Variable Trap, Interaction Term, Chow Test, Elasticity, Regression Through the Origin
- Hub: Intro Econometrics