Causality & the regression model 7

  • Causal Effect:

    The change in yy from a one-unit change in xx with everything else held fixed; a counterfactual quantity that is never directly observed. Lec 1

  • Causality vs Correlation:

    A regression slope shows association; a causal reading needs the other factors held fixed, since Cor(x,y)\mathrm{Cor}(x,y) can be zero, strong or even opposite in sign to the causal effect. Lec 1

  • Confounding Factor:

    A third variable linked to both xx and yy that creates, hides or reverses an apparent relationship between them. Lec 1

  • Randomised Experiment:

    Random assignment of xx, which makes xx independent of the error uu so that differences in outcomes are causal. Lec 1

  • Population Regression Function:

    The true (unobserved) model y=β0+β1x+uy=\beta_0+\beta_1 x+u whose parameters regression tries to estimate. Lec 1

  • Error Term:

    uu: all the factors other than xx that affect yy; unobserved and assumed uncorrelated with xx for OLS to work. Lec 1

  • Data Generating Process:

    The assumed true mechanism yi=β0+β1xi+uiy_i=\beta_0+\beta_1 x_i+u_i that produces the data, used in simulations and proofs. Lec 1

Statistics review 8

  • Expected Value:

    The probability-weighted mean of a random variable, E[X]E[X]; it is linear, so E[aX+b]=aE[X]+bE[aX+b]=aE[X]+b. Lec 2

  • Variance:

    The expected squared distance from the mean, Var(X)=E[(X−E[X])2]\mathrm{Var}(X)=E[(X-E[X])^2]; the variance of a sum needs covariance terms. Lec 2

  • Covariance:

    Joint variation of two variables, Cov(X,Y)=E[XY]−E[X]E[Y]\mathrm{Cov}(X,Y)=E[XY]-E[X]E[Y]; its sign shows whether they move together. Lec 2

  • Statistical Independence:

    Joint density equals the product of the marginals; implies zero covariance, but zero covariance does not imply independence. Lec 2

  • IID Sample:

    Independent draws from one common distribution, the sampling assumption behind the standard OLS results. Lec 2

  • Estimator:

    A function of the sample used to estimate a population parameter; it is itself a random variable. Lec 2

  • Correlation:

    Strength and sign of the linear association between two variables, between −1-1 and 11. Lec 2

  • Conditional Mean:

    E(y∣x)E(y\mid x), the average of yy given xx, which regression estimates. Lec 2

OLS estimation 5

  • Simple Linear Regression:

    One regressor: y=β0+β1x+uy=\beta_0+\beta_1 x+u, fitted by OLS. Lec 2

  • OLS Estimation:

    Ordinary least squares: picks the coefficients that minimise the sum of squared residuals, from the first-order conditions. Lec 2

  • Sample Regression Function:

    The estimated line y^=β^0+β^1x\hat y=\hat\beta_0+\hat\beta_1 x, the sample counterpart of the unobserved population regression function. Lec 2

  • Residuals:

    u^i=yi−y^i\hat u_i=y_i-\hat y_i, the gap between observed and fitted values; an estimate of the error uiu_i, and sums to zero when there is an intercept. Lec 2

  • Multiple Regression:

    A regression with KK regressors, y=β0+β1x1+⋯+βKxK+uy=\beta_0+\beta_1x_1+\dots+\beta_Kx_K+u; each βk\beta_k is the effect of xkx_k holding the others fixed, and OLS has K+1K+1 first-order conditions. Lec 3

Functional form & logs 9

  • Functional Form:

    The shape chosen for the relationship (level, log, quadratic), which sets how coefficients are interpreted. Lec 3

  • Linear in Parameters:

    A model linear in the β\betas after transforming the variables (logs, squares), so OLS still applies. Lec 3

  • Marginal Effect:

    Change in yy from a one-unit change in xx; constant in levels, varies with xx in quadratic or interaction models. Lec 3

  • Rate of Change:

    (dy/dx)/y(dy/dx)/y: the proportional change in yy per unit of xx, which is the slope of a log-level model. Lec 3

  • Log-Level Model:

    ln⁡y\ln y on xx: a one-unit rise in xx changes yy by about 100β100\beta percent. Lec 3

  • Log-Log Model:

    ln⁡y\ln y on ln⁡x\ln x: the slope is an elasticity. Lec 3

  • Level-Log Model:

    yy on ln⁡x\ln x: the slope divided by 100 is the change in yy (in its own units) for a 1% rise in xx. Lec 3

  • Elasticity:

    Percentage change in yy for a 1% change in xx; the slope of a log-log model. Lec 3

  • Prediction:

    Plugging values of the regressors into the fitted equation to get y^\hat y. Lec 3

Properties of OLS & the classical assumptions 9

  • Classical Assumptions:

    A0 to A4: linearity, random sampling, no perfect collinearity, zero conditional mean and homoskedasticity (plus normality, A5, for exact tests); they underpin the OLS properties. Lec 4

  • Zero Conditional Mean:

    Assumption A2, E[u∣x]=0E[u\mid x]=0: the error is unrelated to the regressor, which implies Cov(x,u)=0\mathrm{Cov}(x,u)=0 and drives unbiasedness. Lec 4

  • Homoskedasticity:

    Assumption A4, Var(u∣x)=σ2\mathrm{Var}(u\mid x)=\sigma^2 for every xx: constant error variance, needed for the usual variance formula. Lec 4

  • Serial Correlation:

    Correlation between the errors of different observations; assumption A3 rules it out. Lec 4

  • Sampling Distribution:

    The distribution of an estimator's values across repeated samples of size NN. Lec 4

  • Unbiasedness:

    The mean of the estimator's sampling distribution equals the true parameter, E[θ^]=θE[\hat\theta]=\theta. Lec 4

  • Efficiency:

    Minimum variance among all unbiased estimators of the parameter. Lec 4

  • Gauss-Markov Theorem:

    Under assumptions A1 to A4 the OLS estimator is the best linear unbiased estimator (BLUE). Lec 4

  • Variance of the OLS Estimator:

    The sampling variance of β^\hat\beta, σ2/∑(xi−xˉ)2\sigma^2/\sum(x_i-\bar x)^2 in the simple model: smaller with more data and more spread in xx, larger when regressors are collinear. Lec 4

Confidence intervals & units 7

  • Standard Error:

    The square root of the estimated variance of an estimator, se(β^)se(\hat\beta); the yardstick for tests and intervals. Lec 5

  • Degrees of Freedom:

    Observations left free to vary once the parameters are estimated; N−K−1N-K-1 in OLS with KK regressors. Lec 5

  • Normality of Errors (A5):

    Assumption ui∼N(0,σ2)u_i\sim N(0,\sigma^2), which makes the OLS estimators normal and gives exact t and F tests. Lec 5

  • t-Distribution:

    The distribution of an estimator standardised with the estimated variance; fatter tails than the normal, converging to it as NN grows. Lec 5

  • Critical Value:

    The cutoff tα/2[N−K−1]t_{\alpha/2}[N-K-1] leaving α/2\alpha/2 in each tail; used to reject a null and to build confidence intervals. Lec 5

  • Confidence Interval:

    Range β^±tcrit se(β^)\hat\beta \pm t_{crit}\,se(\hat\beta) that covers the true coefficient with stated probability. Lec 5

  • Change of Variables:

    How coefficients and intercepts transform when a variable is rescaled or its units or currency change (y+cy+c shifts the intercept; cxcx divides the slope by cc). Lec 5

t-tests & p-values 10

  • Hypothesis Testing:

    Framework of a null and an alternative hypothesis, a test statistic and a significance level (5%). Lec 6

  • Null Hypothesis:

    The claim β=b\beta=b that a test tries to reject, against an alternative H1H_1. Lec 6

  • t-statistic:

    (β^−b)/se(β^)(\hat\beta-b)/se(\hat\beta): how many standard errors the estimate is from the null value. Lec 6

  • t-test:

    Tests one coefficient, t=β^/se(β^)t=\hat\beta/se(\hat\beta) against a critical value or p-value. Lec 6

  • Significance Level:

    α\alpha, the probability of rejecting a true null (5% by default). Lec 6

  • Statistical Significance:

    H0:βk=0H_0:\beta_k=0 is rejected at the chosen level, read from the t value or Pr(>|t|) column of the output. Lec 6

  • p-value:

    Probability of a result at least as extreme as observed if the null were true; reject when below the significance level. Lec 6

  • One-Sided Test:

    A test that puts all of α\alpha in one tail, used when the alternative has a direction (β>b\beta>b or β<b\beta<b). Lec 6

  • P-hacking:

    Trying specifications until some p<0.05p<0.05 appears, which makes the reported significance misleading. Lec 6

  • Regression Through the Origin:

    A model with no intercept, forcing the line through (0,0)(0,0); R2R^2 is then not comparable. Lec 6

Joint tests & the F-test 7

  • F-test:

    Joint test that several coefficients are zero (or that a restricted model fits as well), built from the change in R2R^2 or SSR. Lec 7

  • Joint Hypothesis Test:

    A test that several coefficient restrictions hold simultaneously; H1H_1 is true if any one of them fails. Lec 7

  • Restricted Model:

    The original regression with the null hypothesis imposed, so it has fewer free coefficients. Lec 7

  • Unrestricted Model:

    The original regression with all coefficients free; it always fits at least as well as the restricted model. Lec 7

  • Residual Sum of Squares:

    RSS=∑u^i2RSS=\sum\hat u_i^2, the sum of squared OLS residuals; RSSUR≤RSSRRSS_{UR}\le RSS_R, and the F-test compares the two. Lec 7

  • Returns to Scale:

    How output or cost scales with inputs; in a log-log Cobb-Douglas model an output elasticity below 1 means increasing returns (a natural monopoly). Lec 7

  • Cobb-Douglas Function:

    A multiplicative cost or production function made linear in parameters by taking logs, so the slopes are elasticities. Lec 7

Goodness of fit & multicollinearity 5

  • R-squared:

    Share of the variation in yy explained by the regression, R2=1−SSR/TSSR^2 = 1 - SSR/TSS; never falls when a regressor is added. Lec 8

  • Adjusted R-squared:

    R2R^2 penalised for the number of regressors, so it can fall when a useless variable is added. Lec 8

  • Total Sum of Squares:

    TSS=∑(yi−yˉ)2TSS=\sum (y_i-\bar y)^2, the total variation in yy that R2R^2 is measured against. Lec 8

  • Multicollinearity:

    Regressors that are highly (but not perfectly) correlated, inflating standard errors and shrinking t-stats while OLS stays unbiased. Lec 8

  • Perfect Multicollinearity:

    An exact linear relation among regressors, so OLS cannot be computed. Lec 8

Misspecification & quadratics 6

  • Model Misspecification:

    Wrong functional form, omitted relevant variables or included irrelevant ones, biasing or weakening estimates. Lec 9

  • Omitted Variable Bias:

    Bias in β^\hat\beta when a relevant variable correlated with the included regressor is left out. Lec 9

  • Irrelevant Variable:

    A regressor whose true coefficient is zero; including it keeps OLS unbiased but raises the variance. Lec 9

  • Quadratic Model:

    Regression with xx and x2x^2, giving a marginal effect that changes with xx and a turning point at −β1/(2β2)-\beta_1/(2\beta_2). Lec 9

  • Turning Point:

    The xx where a quadratic's slope is zero, −β1/(2β2)-\beta_1/(2\beta_2): a peak if β2<0\beta_2<0, a trough if β2>0\beta_2>0. Lec 9

  • Bad Control:

    A control that is itself affected by the variable of interest (a mediator), so including it distorts the effect you want. Lec 9

Dummy variables & interactions 8

  • Dummy Variable:

    A 0/1 regressor marking membership of a category, shifting the intercept for that group. (Alias: Dummy Variables.) Lec 10

  • Base Category:

    The omitted group whose mean is the intercept; every dummy coefficient is measured relative to it. Lec 10

  • Categorical Variables:

    A variable whose values are categories (e.g. family status); KK categories enter as K−1K-1 dummies relative to a base category, tested jointly with an F-test. Lec 10

  • Dummy Variable Trap:

    Including a dummy for every category plus an intercept, which makes the regressors perfectly collinear. Lec 10

  • Interaction Term:

    Product of two regressors letting the effect of one depend on the level of the other (different slopes per group). Lec 11

  • Chow Test:

    F-test of whether one regression pools two groups or the coefficients differ across groups. Lec 11

  • Chow-Fisher Test:

    The Chow test written as a joint F-test on a group dummy and all its interactions. Lec 11

  • Hedonic Pricing:

    Regressing a product's price on its characteristics to value each attribute (e.g. computer prices). Lec 11

Heteroskedasticity 5