Week 13

Exam Review

Exam Review

Part of: Intro Econometrics | Key concepts: OLS Estimation, Gauss-Markov Theorem, F-test, t-test, Confidence Interval, Multicollinearity, Omitted Variable Bias, Heteroskedasticity, Dummy Variable, Interaction Term


What this lecture is

A whole-course map, not new material

Lecture 13 is the final-exam review. It does two things: (1) explains how to use the formula sheet and the statistical tables you are given in the exam, and (2) walks the entire course from first principles — deriving OLS, listing the classical assumptions and what they buy you, the inference machinery (t, F, confidence intervals, R2R^2), the three things that "go wrong" (multicollinearity, irrelevant/omitted variables, heteroskedasticity), and the "fancier" interpretation models (unit changes, regression through the origin, logs, quadratics, dummies, interactions, Chow tests). This note is therefore a single-sitting revision sheet for the whole module, with every result reasoned through rather than just listed.

The course splits into three blocks, mirrored in the sections below:

  1. Basic building blocks — deriving OLS, the sampling distribution of the estimators, t-tests, p-values, confidence intervals, F-tests, goodness-of-fit.
  2. Things that can go wrong — multicollinearity, including irrelevant variables, omitting relevant variables, heteroskedasticity.
  3. Fancier models — changing units, regression through the origin, log models, quadratics, categorical dummies, interaction terms, and Chow tests.

Part 0 — Exam mechanics: the formula sheet & tables

Read the table instructions carefully — they cost easy marks

The exam gives you a formula sheet (on Moodle, in the Practice Exam section: OLS formulas, omitted-variable bias, regression through the origin, inference, etc.) and statistical tables. You must use the printed tables, not R's qt() / qf() quantile functions — the values can differ slightly and the exam expects the table value.

Degrees of freedom — the number people get wrong

Everywhere you read a critical value, you need the right degrees of freedom.

  • t-distribution: df=N−Mdf = N - M, where NN = number of observations and MM = number of estimated parameters (first-order conditions), including the intercept if one is used.
  • One- vs two-tailed: use the 1-tailed column for one-sided tests (H1: >H_1:\ > or <<) and the 2-tailed column for two-sided tests (H1: ≠H_1:\ \neq). Equivalently, a two-sided test at α\alpha uses the one-sided critical value at α/2\alpha/2.
  • F-distribution: numerator df =q= q (number of restrictions); denominator df =N−M= N - M. There is a separate table for each significance level (α=10%,5%,1%\alpha = 10\%, 5\%, 1\%) — pick the right one.
Activity 13 (part 1) — reading the F-table

Model on N=30N = 30 observations: y=β0+β1X1+β2X2+β3 LEFT+β4 RIGHT+u,y = \beta_0 + \beta_1 X_1 + \beta_2 X_2 + \beta_3\,LEFT + \beta_4\,RIGHT + u, where LEFT,RIGHTLEFT, RIGHT are dummies for the categorical variable POSITION∈{LEFT,MIDDLE,RIGHT}POSITION \in \{LEFT, MIDDLE, RIGHT\} (so MIDDLEMIDDLE is the base category). Test whether POSITIONPOSITION is jointly significant at α=5%\alpha = 5\%: H0: β3=β4=0H_0:\ \beta_3 = \beta_4 = 0.

  • Restrictions q=2q = 2 (two dummies set to zero).
  • Parameters M=5M = 5 (β0\beta_0 through β4\beta_4), so denominator df=N−M=30−5=25df = N - M = 30 - 5 = 25.
  • Critical value: read the 5% F-table at numerator df=2df = 2, denominator df=25df = 25, giving F0.05(2, 25)≈3.39F_{0.05}(2,\,25) \approx 3.39.
  • Decision rule: reject H0H_0 (POSITION is significant) if the computed F>3.39F > 3.39.

1. Basic Building Blocks

1.1 The population model and the sample

The data-generating process (DGP)

We assume a population model y=β0+β1x+u,y = \beta_0 + \beta_1 x + u, with outcome yy, explanatory variable xx, random error uu (everything else that affects yy), intercept β0\beta_0, and slope β1\beta_1 (the effect of xx on yy). For a random sample of size nn, observation ii obeys yi=β0+β1xi+uiy_i = \beta_0 + \beta_1 x_i + u_i.

We observe {yi}\{y_i\} and {xi}\{x_i\} but never the {ui}\{u_i\} or the true β\beta's, so β1\beta_1 cannot be recovered exactly — it must be estimated.

1.2 Deriving the OLS estimators

OLS minimises the sum of squared residuals

Define the residual u^i=yi−(β^0+β^1xi)\hat u_i = y_i - (\hat\beta_0 + \hat\beta_1 x_i) — the gap between the observed yiy_i and the fitted value. (Distinguish it from the unobservable error uiu_i, which is the gap from the true line.) OLS chooses β^0,β^1\hat\beta_0, \hat\beta_1 to solve min⁡β^0,β^1 ∑i=1nu^i2=min⁡β^0,β^1 ∑i=1n(yi−β^0−β^1xi)2.\min_{\hat\beta_0,\hat\beta_1}\ \sum_{i=1}^{n}\hat u_i^2 = \min_{\hat\beta_0,\hat\beta_1}\ \sum_{i=1}^{n}\big(y_i - \hat\beta_0 - \hat\beta_1 x_i\big)^2.

The two first-order conditions (FOCs) are:

∂∂β^0: −2∑iu^i=0  ⇒  ∑iu^i=0,∂∂β^1: −2∑ixiu^i=0  ⇒  ∑ixiu^i=0.\frac{\partial}{\partial\hat\beta_0}:\ -2\sum_i \hat u_i = 0 \;\Rightarrow\; \sum_i \hat u_i = 0,\qquad \frac{\partial}{\partial\hat\beta_1}:\ -2\sum_i x_i\hat u_i = 0 \;\Rightarrow\; \sum_i x_i\hat u_i = 0.

Two equations, two unknowns. Solving FOC (1) gives two facts that recur throughout the course:

Two results that fall straight out of FOC (1)
  • The fitted line passes through the sample means: yˉ=β^0+β^1xˉ\bar y = \hat\beta_0 + \hat\beta_1\bar x.
  • The intercept is pinned down by the slope: β^0=yˉ−β^1xˉ\boxed{\hat\beta_0 = \bar y - \hat\beta_1\bar x}.

Substituting back and using FOC (2) gives the slope:

β^1=∑i(xi−xˉ)(yi−yˉ)∑i(xi−xˉ)2=sxysxx\boxed{\hat\beta_1 = \frac{\sum_i (x_i-\bar x)(y_i-\bar y)}{\sum_i (x_i-\bar x)^2} = \frac{s_{xy}}{s_{xx}}}
Reading the slope formula

β^1\hat\beta_1 is the sample covariance of xx and yy divided by the sample variance of xx. Positive covariance ⇒ positive slope. This is the single most important formula in the course: nearly every later result (OVB, regression through the origin, the WLS transformations) is a manipulation of it.

Multivariate case. For yi=β^0+β^1x1i+⋯+β^KxKi+u^iy_i = \hat\beta_0 + \hat\beta_1 x_{1i} + \dots + \hat\beta_K x_{Ki} + \hat u_i, OLS gives K+1K+1 FOCs of the same shape: ∑iu^i=0\sum_i \hat u_i = 0 and ∑iu^ixji=0\sum_i \hat u_i x_{ji} = 0 for each regressor jj. The residuals are mechanically orthogonal to every regressor.

1.3 Sampling distribution & the variance of the estimators

Variance of the OLS estimators

Single-variable model: Var⁡(β^1)=σu2∑i(xi−xˉ)2\operatorname{Var}(\hat\beta_1) = \frac{\sigma_u^2}{\sum_i (x_i-\bar x)^2} Var⁡(β^0)=σu2[1N+xˉ2∑i(xi−xˉ)2]\operatorname{Var}(\hat\beta_0) = \sigma_u^2\left[\frac{1}{N} + \frac{\bar x^2}{\sum_i (x_i-\bar x)^2}\right] Multivariate model (coefficient jj): Var⁡(β^j)=σu2(1−Rj2)∑i(xji−xˉj)2\boxed{\operatorname{Var}(\hat\beta_j) = \frac{\sigma_u^2}{(1-R_j^2)\sum_i (x_{ji}-\bar x_j)^2}} where Rj2R_j^2 is the R2R^2 from regressing xjx_j on all the other regressors. The unknown error variance is estimated from the residuals: σ^u2=1N−M∑i=1Nu^i2(M=number of parameters).\hat\sigma_u^2 = \frac{1}{N-M}\sum_{i=1}^{N}\hat u_i^2 \qquad (M = \text{number of parameters}).

What drives precision — read the variance formula

Var⁡(β^j)\operatorname{Var}(\hat\beta_j) is larger (less precise estimate) when: the error variance σu2\sigma_u^2 is large; there is little spread in xjx_j (small ∑(xji−xˉj)2\sum (x_{ji}-\bar x_j)^2); or xjx_j is highly explained by the other regressors (Rj2→1R_j^2 \to 1, i.e. multicollinearity). This one expression already foreshadows Part 2.

1.4 Properties of estimators & the classical assumptions

Three properties
  • Sampling distribution: how variation across samples translates into variation in θ^\hat\theta.
  • Unbiased: E[θ^]=θ\mathbb{E}[\hat\theta] = \theta — on average, dead on.
  • Efficient: has the smallest variance among all unbiased estimators.
The five classical assumptions
  • A1 — mean-zero errors: E[u]=0\mathbb{E}[u] = 0.
  • A2 — exogeneity: E[u∣xk]=0\mathbb{E}[u \mid x_k] = 0 for all regressors kk.
  • A3 — no autocorrelation: Cov⁡(ui,uj)=0\operatorname{Cov}(u_i, u_j) = 0 for i≠ji \neq j.
  • A4 — homoskedasticity: Var⁡(u∣xk)=σ2\operatorname{Var}(u \mid x_k) = \sigma^2 (constant).
  • A5 — normality: ui∼N(0,σu2)u_i \sim N(0, \sigma_u^2).
What each bundle of assumptions buys you
Result Needs Statement
Unbiasedness A1 + A2 E[β^j]=βj\mathbb{E}[\hat\beta_j] = \beta_j
Gauss–Markov (BLUE) A1–A4 OLS has the minimum variance of all linear unbiased estimators
Normality of β^\hat\beta A1–A5 β^1∼N(β1,Var⁡(β^1))\hat\beta_1 \sim N\big(\beta_1, \operatorname{Var}(\hat\beta_1)\big), enabling exact t/F inference
Why this table is the spine of the course

Each "problem" in Part 2 is best understood as which assumption fails and therefore which property is lost: omitted variables kill A2 (so biased); heteroskedasticity kills A4 (still unbiased, but no longer efficient and SEs wrong). Memorise which assumption → which casualty.


1.5 Simple hypothesis testing — the t-test

Standardising the estimator into a t-statistic

Because β^1\hat\beta_1 is normal, subtracting its mean and dividing by its standard deviation gives a standard normal; replacing the unknown σu2\sigma_u^2 by σ^u2\hat\sigma_u^2 turns it into a t-distribution with N−MN-M df: t(β^1)=β^1−β1s.e.⁡(β^1)∼tN−M\boxed{t(\hat\beta_1) = \frac{\hat\beta_1 - \beta_1}{\operatorname{s.e.}(\hat\beta_1)} \sim t_{N-M}}

The four steps of a t-test
  1. State H0H_0 and H1H_1. A claim about one parameter; two-sided (== vs ≠\neq) or one-sided (≤\leq vs >>). H0H_0 always contains the equality.
  2. Compute the test statistic under H0H_0: t(β^1)=β^1−bs.e.⁡(β^1)t(\hat\beta_1) = \dfrac{\hat\beta_1 - b}{\operatorname{s.e.}(\hat\beta_1)}, plugging in the value bb that H0H_0 asserts.
  3. Find the critical value from the t-table at significance α\alpha and df=N−Mdf = N-M (use α/2\alpha/2 for two-sided).
  4. Decide: reject H0H_0 if the statistic is more extreme than the critical value.
Two-sided test One-sided test (right)
Hypotheses H0:β1=bH_0:\beta_1 = b vs H1:β1≠bH_1:\beta_1 \neq b H0:β1≤bH_0:\beta_1 \leq b vs H1:β1>bH_1:\beta_1 > b
Critical value tα/2(df)t_{\alpha/2}(df) tα(df)t_{\alpha}(df)
Reject if $ t(\hat\beta_1)
Rejection region both tails (α/2\alpha/2 each) one tail (α\alpha)
Significance testing & p-values

Significance testing is the special two-sided case H0:βk=0H_0:\beta_k = 0 vs H1:βk≠0H_1:\beta_k \neq 0. The statistic t(β^k)=β^k/s.e.⁡(β^k)t(\hat\beta_k) = \hat\beta_k/\operatorname{s.e.}(\hat\beta_k) is what R prints. The p-value answers: "if H0H_0 were true, what is the probability of seeing a t-statistic at least as extreme as the one we got?" — formally p=P(∣Tdf∣≥∣t(β^k)∣)p = \mathbb{P}(|T_{df}| \geq |t(\hat\beta_k)|). It is the lowest significance level at which you would still reject. Small p ⇒ reject.


1.6 Confidence intervals

The CI of an OLS estimator

A 1−α1-\alpha confidence interval satisfies P(L<θ<U)=1−α\mathbb{P}(L < \theta < U) = 1-\alpha. Inverting the t-statistic gives β^1±tα/2(df)⋅s.e.⁡(β^1)\boxed{\hat\beta_1 \pm t_{\alpha/2}(df)\cdot \operatorname{s.e.}(\hat\beta_1)} with, for the single-variable model, s.e.⁡(β^1)=σ^u2/∑i(xi−xˉ)2\operatorname{s.e.}(\hat\beta_1) = \sqrt{\hat\sigma_u^2/\sum_i(x_i-\bar x)^2}.

The CI ⇔ two-sided test duality

A two-sided test of H0:β1=bH_0:\beta_1 = b at level α\alpha and the 1−α1-\alpha CI are the same test: Reject H0  ⟺  b lies OUTSIDE the 1−α confidence interval.\text{Reject } H_0 \iff b \text{ lies OUTSIDE the } 1-\alpha \text{ confidence interval.} So you can do a two-sided test just by checking whether the hypothesised value sits in the interval. (A common exam shortcut.)


1.7 Multiple hypothesis testing — the F-test

The five steps of an F-test
  1. State H0H_0, H1H_1 — a claim about any number of parameters and how they relate; H0H_0 contains the equalities.
  2. Write the unrestricted model (the full model).
  3. Write the restricted model — the model assuming H0H_0 is true. This may create a new dependent variable (if H0:βj=b, b≠0H_0:\beta_j = b,\ b\neq 0) or a new explanatory variable (if H0:βj=kβiH_0:\beta_j = k\beta_i).
  4. Compute the F-statistic from the two residual sums of squares.
  5. Decide against the F-table critical value.
The F-statistic

F=(RSSR−RSSUR)/qRSSUR/(N−M)∼Fq, N−M\boxed{F = \frac{(RSS_R - RSS_{UR})/q}{RSS_{UR}/(N-M)} \sim F_{q,\,N-M}}

  • RSSRRSS_R, RSSURRSS_{UR} — residual sums of squares of the restricted and unrestricted models.
  • qq — number of restrictions in H0H_0 (numerator df).
  • N−MN - M — df of the unrestricted model (MM = parameters/FOCs there). Imposing a restriction can only raise RSS, so RSSR≥RSSURRSS_R \geq RSS_{UR} and F≥0F \geq 0. A large FF means the restriction hurt the fit a lot ⇒ reject.
How F and t relate

When q=1q = 1 (a single restriction) the two tests are equivalent, via F1, N−M=(tN−M)2.F_{1,\,N-M} = \big(t_{N-M}\big)^2. For a joint test (e.g. H0:β1=β2=β3=0H_0:\beta_1=\beta_2=\beta_3=0): if every coefficient is separately significant on its t-test, the joint F-test will be significant too — but the converse can fail (joint significance with individually-insignificant coefficients is the classic multicollinearity signature).


1.8 Goodness-of-fit: R2R^2 and the model-significance F-test

R2R^2

R2=Var⁡(Y^i)Var⁡(Yi)=1−Var⁡(u^i)Var⁡(Yi)R^2 = \frac{\operatorname{Var}(\hat Y_i)}{\operatorname{Var}(Y_i)} = 1 - \frac{\operatorname{Var}(\hat u_i)}{\operatorname{Var}(Y_i)} — the fraction of the variance in yy the model explains. Closer to 1 is "better fit." To compare models with different numbers of regressors, use adjusted R2R^2, which penalises extra variables.

The model-significance (overall-F) test

Tests whether the regressors jointly explain anything: H0:β1=β2=⋯=βk=0H_0:\beta_1=\beta_2=\dots=\beta_k = 0. The restricted model is just yi=β0+uiy_i = \beta_0 + u_i (so RR2=0R_R^2 = 0), and the F-statistic can be written purely in R2R^2 terms: F=RUR2/(M−1)(1−RUR2)/(N−M)∼FM−1, N−M.F = \frac{R_{UR}^2/(M-1)}{(1-R_{UR}^2)/(N-M)} \sim F_{M-1,\,N-M}. This is the "F-statistic" line at the bottom of R's summary().


2. Things That Can Go Wrong

The master summary
Problem Assumption hit Bias? Efficiency / SEs Fix
Multicollinearity none (data issue) Unbiased High variance, big SEs More data / drop a collinear var / accept it
Irrelevant variable none Unbiased Higher variance only Drop it (after checking it's not collinearity)
Omitted relevant variable A2 Biased SEs also unreliable Add the variable (need its data)
Heteroskedasticity A4 Unbiased Not efficient; SEs wrong WLS (known form) or White (unknown)

2.1 Multicollinearity

Definition

A linear relationship among the explanatory variables.

  • Perfect: an exact linear transformation exists, e.g. x1i=5x2ix_{1i} = 5x_{2i} for all ii — OLS cannot be computed (the model is not identified).
  • Partial: a strong but not exact linear relation, e.g. x1i∼N(x2i,1)x_{1i}\sim N(x_{2i},1).
The effect — inflated variance

Recall Var⁡(β^j)=σu2(1−Rj2)∑i(xji−xˉj)2\operatorname{Var}(\hat\beta_j) = \dfrac{\sigma_u^2}{(1-R_j^2)\sum_i(x_{ji}-\bar x_j)^2}. High collinearity ⇒ Rj2R_j^2 high ⇒ (1−Rj2)(1-R_j^2) small ⇒ variance blows up. You can't disentangle the separate effects of the collinear variables. Estimates stay unbiased, just imprecise.

Detecting partial multicollinearity

Tell-tale signs when XX and ZZ are partially collinear:

  • With both in the model: neither is significant.
  • With only XX: it is significant, and adjusted R2R^2 barely changes versus the both-in model.
  • With only ZZ: same story.
  • Rule of thumb: cor⁡(X,Z)\operatorname{cor}(X,Z) is high (above ~0.7). This doesn't always occur under partial collinearity, but when it does it is a strong indicator.

2.2 Including an irrelevant variable

Consequence — noise, not bias

Adding a variable that doesn't belong leaves OLS unbiased (A1–A2 intact). The only cost is higher variance of the estimates. It's easy to spot (the coefficient is insignificant) and easy to fix (drop it and re-estimate).

Don't confuse "irrelevant" with "collinear"

A seemingly irrelevant variable (small t-stat) might actually be collinear with another regressor — multicollinearity also produces small t-stats. Before dropping a variable, check the correlation matrix of the regressors and run an F-test for joint significance. Drop only if it's genuinely irrelevant.

2.3 Omitting a relevant variable (OVB)

How the bias arises

Suppose the true DGP (satisfying A1–A4) is yi=β~0+β~1x1i+β~2x2i+u~iy_i = \tilde\beta_0 + \tilde\beta_1 x_{1i} + \tilde\beta_2 x_{2i} + \tilde u_i, but you estimate yi=β0+β1x1i+uiy_i = \beta_0 + \beta_1 x_{1i} + u_i. The dropped x2x_2 folds into the error. If x1x_1 and x2x_2 are correlated, the error is now correlated with x1x_1 — this violates A2, and β^1\hat\beta_1 becomes biased.

The bias formula and its sign

bias=β~2⋅S12S11\boxed{\text{bias} = \tilde\beta_2 \cdot \frac{S_{12}}{S_{11}}} where β~2\tilde\beta_2 is the true effect of the omitted variable and S12S_{12} is the sample covariance between x1x_1 and x2x_2 (S11S_{11} the variance of x1x_1, always >0> 0). The sign of the bias:

S12>0S_{12} > 0 S12<0S_{12} < 0
β~2>0\tilde\beta_2 > 0 biased up biased down
β~2<0\tilde\beta_2 < 0 biased down biased up

Special cases: if S12=0S_{12}=0 (x1⊥x2x_1\perp x_2) ⇒ no bias; if β~2=0\tilde\beta_2 = 0 ⇒ the omitted variable was irrelevant anyway. With several omitted variables the direction is generally indeterminate.

Bias vs variance — the modelling rule

Omitting a relevant variable biases the estimates (a serious, hard-to-detect, hard-to-fix problem); including an irrelevant one only raises variance (mild). Since we always prefer unbiasedness over minimal variance, the rule of thumb is: add any plausibly relevant regressor. A small variance cost is worth avoiding bias.

2.4 Heteroskedasticity

Definition and consequences

A4 fails: Var⁡(ui)\operatorname{Var}(u_i) is not constant. Then OLS stays unbiased (A1, A2 hold) but the usual SE formula (which assumes constant variance) is biased, so t- and F-tests are invalid, and Gauss–Markov fails so OLS is not efficient. In R output, only the coefficient Estimate column is trustworthy.

The two fixes
Weighted Least Squares (WLS) White correction
Needs the variance form? Yes — know hi>0h_i>0 with Var⁡(ui)=σ2hi\operatorname{Var}(u_i)=\sigma^2 h_i No
Coefficients change? Yes (and R2R^2 changes) No (R2R^2 unchanged)
SEs / tests change? Yes Yes
Efficient (BLUE)? Yes No

WLS: divide every term by hi\sqrt{h_i} so the transformed errors satisfy A4 (verify the leftover variance collapses to a constant). White: keep OLS, plug each unknown Var⁡(ui)\operatorname{Var}(u_i) with the squared residual u^i2\hat u_i^2 to get a robust SE. Across groups/time, run separate regressions so each keeps its own variance.

See Lec_12-Heteroskedasticity for the full derivations.


3. Fancier Models (Interpretation)

3.1 Changing units of measurement

What rescaling does to the estimates
Transformation Intercept β^0\hat\beta_0 Slope β^1\hat\beta_1
Add constant cc to yy +c+c unchanged
Multiply yy by cc ×c\times c ×c\times c
Add constant cc to xx −cβ^1-c\hat\beta_1 unchanged
Multiply xx by cc unchanged ÷c\div c

Standard errors scale the same way as their coefficients, so t-statistics and significance are unaffected by a change of units.

Method for "multiply yy by cc" derivations

To find how new coefficients (β~0,β~1)(\tilde\beta_0,\tilde\beta_1) relate to the originals: (1) rearrange the rescaled equation into y=a+bxy = a + bx form; (2) match a=β0a = \beta_0; (3) match b=β1b = \beta_1. E.g. y⋅c=β~0+β~1xy\cdot c = \tilde\beta_0 + \tilde\beta_1 x rearranges to y=β~0/c+(β~1/c)xy = \tilde\beta_0/c + (\tilde\beta_1/c)x, so β~0=β0c\tilde\beta_0 = \beta_0 c and β~1=β1c\tilde\beta_1 = \beta_1 c.

3.2 Regression through the origin

No-intercept slope

Forcing β0=0\beta_0 = 0 gives β~1=∑ixiyi∑ixi2\tilde\beta_1 = \dfrac{\sum_i x_i y_i}{\sum_i x_i^2} (versus the usual β^1=∑i(xi−xˉ)yi∑i(xi−xˉ)2\hat\beta_1 = \dfrac{\sum_i(x_i-\bar x)y_i}{\sum_i(x_i-\bar x)^2}).

  • The two coincide only if xˉ=0\bar x = 0.
  • Regression through the origin is unbiased only if the true intercept is genuinely zero (β0=0\beta_0 = 0). Forcing it otherwise reintroduces bias.

3.3 Logarithms — marginal effect, rate of change, elasticity

The three economic moments
  • Marginal effect: dydx=ΔyΔx\dfrac{dy}{dx} = \dfrac{\Delta y}{\Delta x} (units of yy per unit of xx).
  • Rate of change: Δy/Δxy=ΔyΔx⋅1y\dfrac{\Delta y/\Delta x}{y} = \dfrac{\Delta y}{\Delta x}\cdot\dfrac{1}{y} (% change in yy per unit of xx).
  • Elasticity: ΔyΔx⋅xy\dfrac{\Delta y}{\Delta x}\cdot\dfrac{x}{y} (% change in yy per % change in xx).
Slope interpretation by model
Model Equation Interpretation of β^1\hat\beta_1
Level–level y=β^0+β^1xy = \hat\beta_0 + \hat\beta_1 x x↑1x\uparrow 1 unit ⇒ y↑β^1y\uparrow \hat\beta_1 units
Log–level ln⁡y=β^0+β^1x\ln y = \hat\beta_0 + \hat\beta_1 x x↑1x\uparrow 1 unit ⇒ y↑100β^1y\uparrow 100\hat\beta_1 percent
Log–log ln⁡y=β^0+β^1ln⁡x\ln y = \hat\beta_0 + \hat\beta_1\ln x x↑1x\uparrow 1 percent ⇒ y↑β^1y\uparrow \hat\beta_1 percent (elasticity; no ×100\times100)
Level–log y=β^0+β^1ln⁡xy = \hat\beta_0 + \hat\beta_1\ln x x↑1x\uparrow 1 percent ⇒ y↑β^1/100y\uparrow \hat\beta_1/100 units
Full moment table — every model has all three moments
Model Marginal effect Rate of change Elasticity
Level–level β^1\hat\beta_1 β^11y\hat\beta_1\dfrac{1}{y} β^1xy\hat\beta_1\dfrac{x}{y}
Log–level β^1y\hat\beta_1 y β^1\hat\beta_1 β^1x\hat\beta_1 x
Log–log β^1yx\hat\beta_1\dfrac{y}{x} β^11x\hat\beta_1\dfrac{1}{x} β^1\hat\beta_1

3.4 Quadratic terms

Capturing a turning point

Adding x2x^2 makes the relation parabolic: the effect of xx on yy rises then falls (if the coefficient on x2x^2 is negative) or falls then rises (if positive). Use it when you expect the relationship to change direction. Test relevance by the significance of the squared term's coefficient. Rule: as a general rule, don't drop the un-squared (linear) term even if it's insignificant — keep the hierarchy.

3.5 Categorical variables (dummies)

A single dummy shifts the intercept

For yi=β0+β1xi+β2 is.Ai+uiy_i = \beta_0 + \beta_1 x_i + \beta_2\,is.A_i + u_i where is.Ai∈{0,1}is.A_i\in\{0,1\}:

  • Not in A: yi=β0+β1xi+uiy_i = \beta_0 + \beta_1 x_i + u_i.
  • In A: yi=(β0+β2)+β1xi+uiy_i = (\beta_0 + \beta_2) + \beta_1 x_i + u_i. Same slope, intercept shifted by β2\beta_2. So β2\beta_2 is the A-vs-not-A difference, holding xx fixed.
Representing a KK-category variable

For a variable with KK categories, create K−1K-1 dummies; the left-out category is the base (control), where all dummies =0=0. Each dummy's coefficient is the effect relative to the base. E.g. for {B,C,D}\{B, C, D\} with DD the base: yi=β0+β1xi+β2 is.Bi+β3 is.Ci+uiy_i = \beta_0 + \beta_1 x_i + \beta_2\,is.B_i + \beta_3\,is.C_i + u_i — each group gets its own intercept.

Test categorical significance jointly with an F-test

To ask "does the whole categorical variable matter?" test all its dummies together: H0:β2=β3=0H_0:\beta_2 = \beta_3 = 0 via an F-test (not separate t-tests). A single binary category can be tested with a t-test on its one dummy. Including all KK dummies and an intercept is the dummy-variable trap (perfect multicollinearity) — always omit one.

Ordinal categories ⇒ stepwise intercepts

For an ordered category (Left / Center / Right with Right as base), yi=β0+β1 is.Lefti+β2 is.Centeri+uiy_i = \beta_0 + \beta_1\,is.Left_i + \beta_2\,is.Center_i + u_i gives each level its own intercept, producing a stepwise pattern across the ordered groups.

3.6 Interaction terms & Chow tests

Interaction = different slopes per group

yi=β0+β1xi+β2 malei+β3(xi⋅malei)+ui.y_i = \beta_0 + \beta_1 x_i + \beta_2\,male_i + \beta_3 (x_i\cdot male_i) + u_i.

  • Women (male=0male=0): intercept β0\beta_0, slope β1\beta_1.
  • Men (male=1male=1): intercept β0+β2\beta_0+\beta_2, slope β1+β3\beta_1+\beta_3. The dummy β2\beta_2 shifts the intercept; the interaction β3\beta_3 changes the slope. So interactions let the marginal effect of xx differ by group.
Chow test — do two groups share one regression?

Version 1 (dummies/interactions). Unrestricted: yi=β0+β1xi+β2 mani+β3(xi mani)+uiy_i = \beta_0 + \beta_1 x_i + \beta_2\,man_i + \beta_3 (x_i\,man_i) + u_i. Restricted: yi=β0+β1xi+uiy_i = \beta_0 + \beta_1 x_i + u_i. Standard F-test with F=(RSSR−RSSUR)/qRSSUR/(N−M)∼Fq, N−M.F = \frac{(RSS_R - RSS_{UR})/q}{RSS_{UR}/(N-M)} \sim F_{q,\,N-M}. Reject if the intercept differs (β2≠0\beta_2\neq0), the slope differs (β3≠0\beta_3\neq0), or both.

Version 2 (separate regressions). Fit the model separately for each group (RSSmRSS_m, RSSfRSS_f). The unrestricted RSS is their sum: F=(RSSR−(RSSm+RSSf))/q(RSSm+RSSf)/(N−M)∼Fq, N−M,F = \frac{\big(RSS_R - (RSS_m + RSS_f)\big)/q}{(RSS_m + RSS_f)/(N-M)} \sim F_{q,\,N-M}, testing H0: β0m=β0f, β1m=β1fH_0:\ \beta_0^m=\beta_0^f,\ \beta_1^m=\beta_1^f. Under homoskedasticity the two versions agree; under group heteroskedasticity the separate-regression version is preferred (each group keeps its own variance).


Summary — the one-page recap

  1. OLS minimises ∑u^i2\sum\hat u_i^2, giving β^1=sxy/sxx\hat\beta_1 = s_{xy}/s_{xx} and β^0=yˉ−β^1xˉ\hat\beta_0 = \bar y - \hat\beta_1\bar x; the fitted line passes through (xˉ,yˉ)(\bar x,\bar y) and residuals are orthogonal to the regressors.
  2. The classical assumptions map cleanly to properties: A1+A2 ⇒ unbiased; A1–A4 ⇒ Gauss–Markov (BLUE); A1–A5 ⇒ normal β^\hat\beta, which powers exact t/F inference.
  3. t-test (t=(β^−b)/s.e.⁡t = (\hat\beta-b)/\operatorname{s.e.}, df=N−Mdf=N-M): one statistic per restriction; two-sided uses α/2\alpha/2. A CI β^±tα/2s.e.⁡\hat\beta \pm t_{\alpha/2}\operatorname{s.e.} is the dual of the two-sided test — reject iff bb lies outside it. The p-value is the smallest α\alpha that rejects.
  4. F-test (F=(RSSR−RSSUR)/qRSSUR/(N−M)F = \tfrac{(RSS_R-RSS_{UR})/q}{RSS_{UR}/(N-M)}): for joint restrictions; restricted model embeds H0H_0 (new yy or new xx). F1,df=tdf2F_{1,df}=t_{df}^2. The overall-F (H0H_0: all slopes zero) tests model significance and can be written in R2R^2.
  5. Things that go wrong: multicollinearity (unbiased, inflated variance, joint-sig-but-individually-insig), irrelevant variable (unbiased, more variance — check it's not collinearity before dropping), omitted variable (biased via A2, bias=β~2S12/S11\text{bias}=\tilde\beta_2 S_{12}/S_{11} — prefer unbiasedness, so include relevant vars), and heteroskedasticity (A4 fails → unbiased but inefficient with wrong SEs → WLS if form known, White if not).
  6. Fancier models: unit changes scale coefficients but not t-stats; regression through the origin is unbiased only if β0=0\beta_0=0; logs give level/percent/elasticity readings; quadratics capture turning points (keep the linear term); dummies (K−1K-1 of them, one base category) shift intercepts and are tested jointly by F; interactions change slopes; Chow tests check whether two groups share one regression.
  7. Exam mechanics: use the printed tables, get df right (tt: N−MN-M; FF: qq over N−MN-M), pick the table matching α\alpha, and watch the one- vs two-tailed distinction.