Linear regression is a tool for modeling how a response variable changes with one or more predictor variables. It is widely used because the fitted equation is easy to interpret and can make predictions from data. The results are most trustworthy when several assumptions about the data and errors are reasonably satisfied.
Checking these assumptions helps you avoid misleading slopes, confidence intervals, and predictions.
The core assumptions are linearity, independence, constant variance, and approximately normal residuals. Residual plots, fitted value plots, histograms, and Q-Q plots help reveal when these assumptions are weak or violated. When assumptions fail, a model may still produce a line, but its p-values, standard errors, and forecasts may be unreliable.
Common fixes include transforming variables, adding missing predictors, using weighted regression, or choosing a different model.
Understanding Statistics: Regression Assumptions
The assumptions concern the residuals, which are the differences between observed values and the values predicted by the fitted line. A good model leaves behind differences that look like ordinary random scatter. Plot residuals against fitted values and against each predictor.
The points should form a loose horizontal band centered near zero. A U shape or an S shape suggests that the relationship bends. In that case, a straight line gives a simplified average that misses a systematic feature.
Adding a squared predictor, using a transformed scale, or fitting a model designed for curves may describe the pattern better. A few unusual points deserve attention too. One point with an extreme predictor value can pull the slope strongly, even if its residual is small.
Independence is about whether one observation carries information about another observation's error. It matters greatly in time series. Daily temperatures, stock prices, and measurements from the same patient on consecutive days often resemble nearby observations.
If residuals occur in long runs above the line, followed by long runs below it, they are related over time. Data from nearby places can have the same issue. Test scores from pupils in one classroom may be more alike than scores from pupils chosen from many unrelated schools.
Treating related observations as independent makes the sample seem larger than it really is. This can make an effect appear more certain than the evidence supports. Time plots, maps, and knowledge of how the data were collected are often more useful than a single statistical test.
Equal variance means that the typical prediction error stays at a similar size across the range of the data. A residual plot that spreads out like a fan shows unequal variance. For example, the cost of a house may vary by a few thousand pounds for low priced homes but by far more for expensive homes.
The line may still show the average trend reasonably well, yet its standard errors can be wrong. A logarithmic transformation is sometimes useful when values are positive and spread grows with their size. Weighted regression gives more weight to observations measured with greater precision.
Normality has a different role. The response values themselves do not need to form a bell shape. It is the residual distribution, after accounting for predictors, that matters for small sample inference.
A histogram and a Q-Q plot can reveal strong skewness or heavy tails. Large samples often reduce the practical effect of mild nonnormality.
Assumption checks are not a pass or fail ritual. Real data rarely produce perfectly random plots. Focus on patterns that are clear enough to change a conclusion or prediction.
Start by checking units, recording errors, duplicate rows, and whether important groups were left out. A curved residual pattern may mean a missing physical variable rather than a need for a more complicated equation. For instance, fuel use depends on speed, vehicle mass, road slope, and driving conditions.
A model using only speed can leave structured errors because the omitted factors matter. Finally, do not confuse good assumptions with safe prediction everywhere. Predictions far beyond the observed predictor range are extrapolations, and they can fail even when every residual plot looks acceptable.
Key Facts
- Simple linear regression model: y = β0 + β1x + ε
- Residual for point i: ei = yi - ŷi
- Linearity means the average value of y changes approximately as a straight-line function of x.
- Independence means residuals are not systematically related to one another, especially over time or space.
- Homoscedasticity means residuals have roughly constant spread for all fitted values.
- Normal residuals are mainly needed for reliable t tests, F tests, and confidence intervals in small samples.
Vocabulary
- Residual
- A residual is the difference between an observed response value and the value predicted by the regression model.
- Linearity
- Linearity is the assumption that the relationship between predictors and the mean response can be represented by a linear equation.
- Independence
- Independence is the assumption that one observation's error does not influence or predict another observation's error.
- Homoscedasticity
- Homoscedasticity is the assumption that residuals have about the same variance across the range of fitted values.
- Q-Q Plot
- A Q-Q plot compares residual quantiles to theoretical normal quantiles to check whether residuals are approximately normal.
Common Mistakes to Avoid
- Ignoring curved patterns in a residual plot is wrong because a clear curve suggests the linear model is missing important nonlinear structure.
- Using regression on time-ordered data without checking independence is wrong because autocorrelated residuals can make standard errors too small and p-values too optimistic.
- Assuming equal variance when the residual plot fans outward is wrong because nonconstant spread can make confidence intervals and prediction intervals inaccurate.
- Treating a high R² as proof that all assumptions are satisfied is wrong because R² measures fit strength, not residual behavior or model validity.
Practice Questions
- 1 A regression model predicts ŷ = 12 + 3x. For x = 5, the observed value is y = 31. Find the predicted value and the residual.
- 2 A fitted model has residuals with standard deviations about 2 for small fitted values and about 9 for large fitted values. Which regression assumption is likely violated, and why?
- 3 A residual plot shows points randomly scattered around zero with no curve, but a plot of residuals versus time shows long runs of positive residuals followed by long runs of negative residuals. Explain which assumption is most concerning and how it could affect inference.