Residuals measure how far each observed data value is from the value predicted by a regression model. This cheat sheet helps students calculate residuals, build residual plots, and decide whether a linear model is reasonable. Residual plots are important because they reveal patterns that a scatterplot or correlation value may hide.
Students use them to check model fit, spot outliers, and compare predictions to real data.
The main formula is , where is the residual, is the observed value, and is the predicted value. A residual plot places the explanatory variable on the horizontal axis and the residuals on the vertical axis. A good linear model usually has residuals randomly scattered around .
Curved patterns, changing spread, or extreme points suggest the model may not be appropriate.
Key Facts
- A residual is calculated using , where is the observed value and is the predicted value.
- A positive residual means the observed value is above the regression line because .
- A negative residual means the observed value is below the regression line because .
- For a least-squares regression line, the residuals always have a sum of , up to rounding error.
- A residual plot graphs each point as , using the original explanatory variable and the residual .
- A residual plot with random scatter around supports using a linear model.
- A curved pattern in a residual plot suggests that a nonlinear model may fit the data better than a line.
- A fan-shaped residual plot shows nonconstant spread, meaning prediction errors change size as changes.
Vocabulary
- Residual
- A residual is the difference between an observed response value and the value predicted by a model, calculated as .
- Predicted Value
- A predicted value, written , is the response value estimated by a regression equation for a given .
- Residual Plot
- A residual plot is a graph of residuals against the explanatory variable, usually shown as points .
- Least-Squares Regression Line
- A least-squares regression line is the line that minimizes the sum of squared residuals, .
- Outlier
- An outlier is a data point with an unusually large residual or an unusual position compared with the rest of the data.
- Nonlinear Pattern
- A nonlinear pattern occurs when residuals show a curve or systematic shape instead of random scatter around .
Common Mistakes to Avoid
- Reversing the residual formula is wrong because gives the opposite sign; use .
- Thinking a high correlation always means a good linear model is wrong because a residual plot can reveal curvature or changing spread.
- Ignoring the sign of a residual is wrong because positive residuals mean the point is above the line and negative residuals mean it is below the line.
- Using instead of on the vertical axis of a residual plot is wrong because the plot must show prediction errors, not original response values.
- Calling any large residual an error in the data is wrong because an outlier may be real and should be investigated before being removed.
Practice Questions
- 1 A regression model predicts for a data point with observed value . Find the residual .
- 2 For the regression equation , find the residual when and the observed value is .
- 3 A point has residual and predicted value . Find the observed value .
- 4 A residual plot shows a clear U-shaped pattern around . Explain what this suggests about using a linear model for the data.
Understanding Residuals & Residual Plots
Least-squares regression chooses a line by making the total squared error as small as possible. Squaring matters because errors above and below the line would otherwise cancel. It also gives extra weight to large misses.
This is useful when a model must make predictions, but it means one unusual observation can affect the line strongly. A point far from the center of the explanatory values has high leverage. If that point pulls the line noticeably when removed, it is influential.
Students should examine such points rather than automatically deleting them. An unusual value may be a recording mistake, or it may reveal an important real situation.
The shape of the leftover errors gives clues about what the line is missing. A U-shaped pattern often means the relationship bends. For example, the distance a ball travels may increase with launch angle up to a point, then decrease.
A straight line cannot represent that rise and fall well. A pattern that alternates above and below zero can show cycles, such as seasonal sales or daily temperatures. Clusters can suggest that different groups are mixed together.
Heights and weights for children of several age groups may form groups with different behavior. In these cases, adding a relevant variable or separating the groups can improve the analysis.
The vertical spread matters as much as the center of the pattern. If errors become wider as the explanatory value increases, predictions are less reliable in that region. House price models often show this effect.
A prediction for a low-cost home may miss by a few thousand dollars, while a prediction for an expensive home may miss by much more. The line can still describe the average trend, but it does not have equal accuracy everywhere.
Students should avoid saying that a model is good simply because the points center near zero. They should check whether the amount of scatter stays fairly similar across the graph.
Residuals connect statistics to decisions made with imperfect information. A school might use study time to predict a test score. A positive leftover error could reflect effective study methods, prior knowledge, sleep, tutoring, or an easier test version.
The error is not meaningless noise. It represents influences the model did not include. This helps explain why a regression prediction is an estimate, not a guarantee.
When reading a plot, first identify any curve, widening spread, clusters, or isolated points. Then state what that feature means for the model.
Finally, remember that a good pattern check does not prove that one variable causes another. A line can summarize an association without explaining its cause.