Econometrics uses data and statistical models to estimate economic relationships and test theories. This cheat sheet covers ordinary least squares, common inference problems, and the identification strategies used to make causal claims. Students need these tools to interpret regression tables, diagnose weak research designs, and understand when estimates can be trusted.
It is especially useful for applied microeconomics, labor, development, finance, and policy evaluation courses.
The core idea of OLS is to choose coefficients that minimize the sum of squared residuals, giving beta_hat = (X'X)^(-1)X'y when the required matrix inverse exists. Under exogeneity, the OLS estimator is unbiased or consistent for the population relationship of interest. Inference depends on correct treatment of heteroskedasticity, serial correlation, clustering, omitted variables, and functional form.
Identification strategies such as randomized experiments, instrumental variables, fixed effects, regression discontinuity, and difference-in-differences connect regression estimates to causal effects.
Key Facts
- The OLS estimator minimizes SSR = sum_i (y_i - x_i'beta)^2 and equals beta_hat = (X'X)^(-1)X'y when X'X is invertible.
- The residual for observation i is e_i = y_i - y_hat_i, where y_hat_i = x_i'beta_hat.
- The key exogeneity condition for causal interpretation is E[u_i | X_i] = 0, meaning the error term is unrelated to the regressors.
- Omitted variable bias in a simple regression has sign determined by Bias(beta_hat_1) = beta_2 * Cov(x_1, x_2) / Var(x_1).
- Heteroskedasticity means Var(u_i | X_i) is not constant, so robust standard errors are usually needed for valid inference.
- A 95 percent confidence interval is beta_hat_j plus or minus 1.96 * SE(beta_hat_j) in large samples under standard regularity conditions.
- Instrumental variables require relevance, Cov(Z, X) != 0, and exogeneity, Cov(Z, u) = 0, to identify a causal effect.
- Difference-in-differences identifies treatment effects using the parallel trends assumption: without treatment, treated and control groups would have changed similarly over time.
Vocabulary
- Ordinary Least Squares
- Ordinary least squares is a regression method that estimates coefficients by minimizing the sum of squared prediction errors.
- Exogeneity
- Exogeneity means the explanatory variables are not systematically related to the unobserved error term.
- Omitted Variable Bias
- Omitted variable bias occurs when a missing factor affects the outcome and is correlated with an included regressor.
- Heteroskedasticity
- Heteroskedasticity occurs when the variance of the regression error changes with the values of the regressors.
- Instrumental Variable
- An instrumental variable affects the endogenous regressor but affects the outcome only through that regressor.
- Identification
- Identification is the set of assumptions and variation that allow a parameter to be interpreted as a causal effect.
Common Mistakes to Avoid
- Calling any regression coefficient causal is wrong because OLS is causal only when the identification assumptions justify E[u | X] = 0 or a valid alternative strategy.
- Ignoring omitted variables is wrong because a missing confounder correlated with both the regressor and outcome can bias the coefficient upward or downward.
- Using usual standard errors under heteroskedasticity or clustering is wrong because the coefficient may be consistent while the reported t-statistics and p-values are misleading.
- Controlling for post-treatment variables is wrong because variables affected by the treatment can block part of the causal effect or create selection bias.
- Treating a statistically insignificant result as proof of no effect is wrong because low power, large standard errors, or weak design can hide meaningful effects.
Practice Questions
- 1 Suppose beta_hat = 2.40 and SE(beta_hat) = 0.50. Compute the t-statistic for testing H0: beta = 0 and state whether it exceeds 1.96 in absolute value.
- 2 In a simple regression, the true model is y = beta_0 + beta_1 x_1 + beta_2 x_2 + u, but x_2 is omitted. If beta_2 > 0 and Cov(x_1, x_2) < 0, what is the direction of omitted variable bias in beta_hat_1?
- 3 A researcher estimates wage_i = beta_0 + beta_1 education_i + u_i and worries that ability is omitted. Explain why ability could bias the estimated return to education.
- 4 Why does a valid instrument need both relevance and exogeneity, and what fails if only one of these conditions holds?
Understanding Econometrics OLS and Identification
A regression coefficient is easiest to read as a comparison made after holding the included control variables fixed. If a wage regression includes education, experience, location, and age, the education coefficient compares people who are similar on those measured factors. It does not automatically compare people with the same ability, family background, motivation, or job connections.
Those unmeasured factors can sit in the error term. If they affect wages and are related to education, the estimated education effect can be misleading.
This is why adding controls can change a coefficient sharply. The change is evidence that the original comparison mixed together different kinds of people.
Identification is about finding variation in a possible cause that is not driven by the usual confounding factors. Random assignment is the cleanest case. A scholarship lottery, for example, can create two groups that differ in scholarship access for reasons unrelated to student choices.
Researchers can then compare later outcomes such as enrollment or earnings. In nonexperimental settings, the hard work is explaining why the comparison group is credible. A good design makes treated and untreated units similar in the relevant way.
A regression table alone cannot prove this. The institution, policy rule, timing, and data construction matter as much as the reported coefficient.
Standard errors describe uncertainty from using a sample rather than the whole population. They become larger when observations are noisy, when a key variable has little variation, or when the sample is small. They can be badly understated when observations share shocks.
Students in the same school, workers in the same firm, or counties hit by the same recession are not fully independent observations. In such cases, clustering standard errors by school, firm, or county is often needed.
Robust standard errors handle changing error spread across observations, but they do not repair omitted variables, reverse causation, poor measurement, or an invalid research design. Statistical significance is therefore not the same as practical importance or causal credibility.
Each major identification method has a specific weak point to inspect. An instrument must affect the treatment strongly enough to be useful, while having no separate route to the outcome. This second claim is often the difficult one.
Fixed effects remove stable differences within a person, firm, school, or place, but they cannot remove factors that change over time. Difference in differences relies on the idea that groups would have followed similar trends without the policy. Graphing outcomes before treatment helps assess that claim.
Researchers should check whether effects appear before the event, whether groups change composition, and whether another policy starts at the same time. When reading a study, pay attention to the unit of observation, the source of variation, the comparison group, the timing, and the assumptions needed for the conclusion.