A residual measures how far a data point is from the value predicted by a regression model. It is the vertical difference between the observed y-value and the predicted y-value on a fitted line or curve. Residuals matter because they show the errors that remain after a model has been used.
Looking at residuals helps you judge whether a model is reliable or whether it is missing an important pattern.
A residual plot places the explanatory variable or predicted values on the horizontal axis and the residuals on the vertical axis. A good linear model usually produces residuals that are randomly scattered around the horizontal line at residual = 0. Curved patterns, funnels, clusters, or long runs above and below zero suggest that the model may be inappropriate.
The standard error of the estimate summarizes the typical size of the residuals, giving a numerical measure of how far predictions usually miss.
Understanding Statistics: Residuals and Residual Plots
A fitted line gives every x-value a predicted response. The gap for each student, house, test score, or other observation tells a small story about that case. Suppose a line predicts that a plant will be 18 centimetres tall after six weeks, but the plant is actually 21 centimetres tall.
Its residual is plus 3 centimetres. The line underestimated that plant.
A residual of minus 3 centimetres means the line overestimated by 3 centimetres. Residuals keep the original units of the response variable, so their size has a practical meaning.
Least-squares regression chooses a line by making the squared residuals as small as possible in total. Squaring does two useful things. It prevents positive and negative errors from cancelling out.
It gives much more weight to a large miss than to a small miss. A residual of 6 contributes four times as much as a residual of 3 after squaring. This is why one unusual point can pull a regression line toward itself.
The standard error of the estimate is based on these squared gaps. It estimates the typical prediction error, though a few extreme points can make it larger.
The shape in a residual plot matters more than whether every point is close to zero. A smooth U-shaped pattern suggests that the relationship curves, so a straight line consistently misses in opposite directions across the data range. A pattern that spreads wider from left to right shows that prediction errors become less consistent for larger explanatory values.
This is called nonconstant spread. For example, a model for house price may have small errors for cheaper homes but large errors for expensive homes. Groups or separate bands can signal that another variable matters, such as location, age group, or type of product.
Students should inspect individual points before trusting a summary number. A large residual may come from a recording mistake, an unusual but real case, or a factor not included in the model. A point far from the others horizontally can have high leverage.
It may strongly affect the slope even if its residual is not large. Removing a point simply because it is inconvenient is poor practice. First check the source and explain any decision clearly.
Residual analysis does not prove that one variable causes another. It only checks whether the chosen model describes the observed pattern reasonably well within the data range.
Key Facts
- Residual = observed y - predicted y
- e = y - yhat
- A positive residual means the observed value is above the model prediction.
- A negative residual means the observed value is below the model prediction.
- For a least-squares regression line, the residuals sum to 0: Σe = 0
- Standard error of the estimate: s = sqrt(Σe^2 / (n - 2)) for simple linear regression
Vocabulary
- Residual
- A residual is the difference between an observed response value and the value predicted by a model.
- Predicted value
- A predicted value is the response value a model estimates for a given input.
- Residual plot
- A residual plot is a graph that shows residuals against x-values or predicted values to check model fit.
- Least-squares regression
- Least-squares regression is a method that chooses a model by minimizing the sum of the squared residuals.
- Standard error of the estimate
- The standard error of the estimate is the typical size of the prediction errors measured by the spread of the residuals.
Common Mistakes to Avoid
- Using predicted minus observed instead of observed minus predicted is wrong when the class or software defines residual as e = y - yhat. It reverses the signs and changes the interpretation of points above and below the model.
- Judging a model only from the scatterplot is incomplete because the residual plot can reveal hidden curvature or unequal spread. Always check the residual plot after fitting a model.
- Ignoring a curved pattern in the residual plot is wrong because random scatter is expected for a good linear model. A clear curve suggests that a nonlinear model may fit better.
- Treating a large residual as proof of a data error is not justified. A large residual may be an outlier, an unusual observation, or evidence that the model is missing an important variable.
Practice Questions
- 1 A regression model predicts yhat = 18 for a data point with observed value y = 23. Find the residual and state whether the point is above or below the model prediction.
- 2 For four data points, the residuals are -2, 1, 3, and -4. Find the sum of squared residuals. If n = 4 for a simple linear regression, compute the standard error of the estimate using s = sqrt(Σe^2 / (n - 2)).
- 3 A residual plot shows residuals that are mostly negative for small x-values, positive for middle x-values, and negative again for large x-values. Explain what this pattern suggests about using a linear model.