A residual measures how far a data point is from the value predicted by a regression model. It is the vertical difference between the observed y-value and the predicted y-value on a fitted line or curve. Residuals matter because they show the errors that remain after a model has been used.
Looking at residuals helps you judge whether a model is reliable or whether it is missing an important pattern.
A residual plot places the explanatory variable or predicted values on the horizontal axis and the residuals on the vertical axis. A good linear model usually produces residuals that are randomly scattered around the horizontal line at residual = 0. Curved patterns, funnels, clusters, or long runs above and below zero suggest that the model may be inappropriate.
The standard error of the estimate summarizes the typical size of the residuals, giving a numerical measure of how far predictions usually miss.
Key Facts
- Residual = observed y - predicted y
- e = y - yhat
- A positive residual means the observed value is above the model prediction.
- A negative residual means the observed value is below the model prediction.
- For a least-squares regression line, the residuals sum to 0: Σe = 0
- Standard error of the estimate: s = sqrt(Σe^2 / (n - 2)) for simple linear regression
Vocabulary
- Residual
- A residual is the difference between an observed response value and the value predicted by a model.
- Predicted value
- A predicted value is the response value a model estimates for a given input.
- Residual plot
- A residual plot is a graph that shows residuals against x-values or predicted values to check model fit.
- Least-squares regression
- Least-squares regression is a method that chooses a model by minimizing the sum of the squared residuals.
- Standard error of the estimate
- The standard error of the estimate is the typical size of the prediction errors measured by the spread of the residuals.
Common Mistakes to Avoid
- Using predicted minus observed instead of observed minus predicted is wrong when the class or software defines residual as e = y - yhat. It reverses the signs and changes the interpretation of points above and below the model.
- Judging a model only from the scatterplot is incomplete because the residual plot can reveal hidden curvature or unequal spread. Always check the residual plot after fitting a model.
- Ignoring a curved pattern in the residual plot is wrong because random scatter is expected for a good linear model. A clear curve suggests that a nonlinear model may fit better.
- Treating a large residual as proof of a data error is not justified. A large residual may be an outlier, an unusual observation, or evidence that the model is missing an important variable.
Practice Questions
- 1 A regression model predicts yhat = 18 for a data point with observed value y = 23. Find the residual and state whether the point is above or below the model prediction.
- 2 For four data points, the residuals are -2, 1, 3, and -4. Find the sum of squared residuals. If n = 4 for a simple linear regression, compute the standard error of the estimate using s = sqrt(Σe^2 / (n - 2)).
- 3 A residual plot shows residuals that are mostly negative for small x-values, positive for middle x-values, and negative again for large x-values. Explain what this pattern suggests about using a linear model.