Checking linear regression assumptions helps you decide whether a fitted line gives reliable estimates, tests, and predictions. A regression model can have a strong-looking equation but still fail if the residuals show patterns or extreme points dominate the fit. This cheat sheet focuses on the diagnostic checks students need after fitting a simple or multiple linear regression model.
It is designed as a quick reference for interpreting plots, statistics, and warning signs.
Key Facts
- The linear regression model is , where the errors should have mean .
- A residual is , and residuals should be randomly scattered around when plotted against fitted values.
- Linearity is supported when the residual plot has no curve, meaning is well approximated by a straight-line relationship in the predictors.
- Homoscedasticity means the error variance is constant, written as for all observations.
- Independence means errors are not correlated, commonly written as for .
- Normality for inference means the errors are approximately , which is mainly checked with a Q-Q plot of residuals.
- Leverage for observation is , and a common warning cutoff is .
- Cook's distance measures influence, and values such as often deserve closer investigation.
Vocabulary
- Residual
- A residual is the observed response minus the predicted response, written as .
- Linearity
- Linearity means the mean response changes according to a linear function of the predictors, so the model form is appropriate.
- Homoscedasticity
- Homoscedasticity means the residuals have roughly constant spread across fitted values or predictor values.
- Q-Q Plot
- A Q-Q plot compares residual quantiles to normal quantiles to check whether the residual distribution is approximately normal.
- Leverage
- Leverage measures how far an observation's predictor values are from the center of the predictor space.
- Influence
- Influence describes how much a fitted regression model would change if a particular observation were removed.
Common Mistakes to Avoid
- Trusting alone, because a high does not prove that residuals are linear, independent, homoscedastic, or normally distributed.
- Ignoring a funnel shape in the residual plot, because increasing or decreasing residual spread suggests nonconstant variance and can make standard errors unreliable.
- Treating every outlier as an error, because an unusual response may be valid data and should be investigated before being removed.
- Using a normality check on the raw values instead of the residuals, because regression inference assumes normal errors, not necessarily a normal response variable.
- Forgetting time order or grouping, because residuals from time series, repeated measures, or clustered data may violate the independence assumption.
Practice Questions
- 1 A regression has observations and predictors. Using the cutoff , what leverage value should be flagged?
- 2 For one observation, and . Compute the residual and interpret its sign.
- 3 A model has observations. Using the rule , what Cook's distance value should trigger closer investigation?
- 4 A residual plot shows a clear curved pattern but nearly constant vertical spread. Which regression assumption is most directly violated, and what model change might help?
Understanding Checking Linear Regression Assumptions
Start with residuals because they reveal what the equation leaves unexplained. Make a residual versus fitted value plot after fitting the model. A roughly even cloud suggests that a straight line may be reasonable.
A U shape or an S shape suggests missing curvature. In practice, this can mean that growth slows at high values, a treatment has a threshold effect, or two predictors interact.
Try a transformed predictor, a squared term, or an interaction only when there is a sensible subject reason. Adding terms just to remove a pattern can make a model hard to interpret and less useful on new data.
The spread of residuals matters as much as their average position. If the cloud opens into a funnel, predictions are more variable for some cases than for others. House prices often show this pattern because expensive homes differ more widely in price.
Test scores can show the reverse pattern when results bunch near the top or bottom of the scale. Unequal spread does not necessarily make the estimated trend useless, but it can make standard errors, confidence intervals, and significance tests unreliable. A response transformation can sometimes help.
Another option is a method that allows unequal variability. Students should describe the shape they see rather than simply writing that an assumption passed or failed.
Independence depends mainly on how the data were collected. Measurements from different randomly chosen people may be close to independent. Repeated measurements from the same person are not independent because one result tends to resemble that person's earlier results.
Data recorded over time can have similar dependence. Daily temperatures, sales figures, and sensor readings often follow nearby observations. Plot residuals in collection order when time is involved.
Long runs above zero followed by long runs below zero suggest dependence. Ignoring this issue usually makes the model seem more certain than it really is. A time series or multilevel model may be needed when observations belong to groups or sequences.
A normal probability plot checks whether residuals have the shape needed for many small sample inference procedures. Points near a straight reference pattern are reassuring. Strong bends or isolated tail points suggest skewness, heavy tails, or outliers.
Normality is less important for estimating the fitted line than for exact tests and intervals, especially in large samples. Outliers, leverage, and influence are different ideas. An outlier has an unusual response compared with the model.
A high leverage case has unusual predictor values. An influential case changes the fitted results noticeably when removed. Inspect the original record before deleting anything.
It may be a typing error, a measurement problem, or a real rare case that the model should explain. Compare the fitted coefficients with and without influential cases, then report any important change honestly.