Linear regression is a tool for modeling how a response variable changes with one or more predictor variables. It is widely used because the fitted equation is easy to interpret and can make predictions from data. The results are most trustworthy when several assumptions about the data and errors are reasonably satisfied.
Checking these assumptions helps you avoid misleading slopes, confidence intervals, and predictions.
The core assumptions are linearity, independence, constant variance, and approximately normal residuals. Residual plots, fitted value plots, histograms, and Q-Q plots help reveal when these assumptions are weak or violated. When assumptions fail, a model may still produce a line, but its p-values, standard errors, and forecasts may be unreliable.
Common fixes include transforming variables, adding missing predictors, using weighted regression, or choosing a different model.
Key Facts
- Simple linear regression model: y = β0 + β1x + ε
- Residual for point i: ei = yi - ŷi
- Linearity means the average value of y changes approximately as a straight-line function of x.
- Independence means residuals are not systematically related to one another, especially over time or space.
- Homoscedasticity means residuals have roughly constant spread for all fitted values.
- Normal residuals are mainly needed for reliable t tests, F tests, and confidence intervals in small samples.
Vocabulary
- Residual
- A residual is the difference between an observed response value and the value predicted by the regression model.
- Linearity
- Linearity is the assumption that the relationship between predictors and the mean response can be represented by a linear equation.
- Independence
- Independence is the assumption that one observation's error does not influence or predict another observation's error.
- Homoscedasticity
- Homoscedasticity is the assumption that residuals have about the same variance across the range of fitted values.
- Q-Q Plot
- A Q-Q plot compares residual quantiles to theoretical normal quantiles to check whether residuals are approximately normal.
Common Mistakes to Avoid
- Ignoring curved patterns in a residual plot is wrong because a clear curve suggests the linear model is missing important nonlinear structure.
- Using regression on time-ordered data without checking independence is wrong because autocorrelated residuals can make standard errors too small and p-values too optimistic.
- Assuming equal variance when the residual plot fans outward is wrong because nonconstant spread can make confidence intervals and prediction intervals inaccurate.
- Treating a high R² as proof that all assumptions are satisfied is wrong because R² measures fit strength, not residual behavior or model validity.
Practice Questions
- 1 A regression model predicts ŷ = 12 + 3x. For x = 5, the observed value is y = 31. Find the predicted value and the residual.
- 2 A fitted model has residuals with standard deviations about 2 for small fitted values and about 9 for large fitted values. Which regression assumption is likely violated, and why?
- 3 A residual plot shows points randomly scattered around zero with no curve, but a plot of residuals versus time shows long runs of positive residuals followed by long runs of negative residuals. Explain which assumption is most concerning and how it could affect inference.