Cook's distance measures how much a regression model changes when one observation is removed. This cheat sheet helps students identify influential points that can strongly affect fitted coefficients, predictions, and conclusions. It is especially useful in college statistics when checking linear regression assumptions and diagnosing unusual data points.
The core idea is to combine residual size with leverage, because a point is influential when it is both unusual in outcome and positioned far from the center of the predictor values. Important tools include the hat matrix, leverage values, studentized residuals, and Cook's distance. Rules of thumb such as or help flag observations for further investigation, not automatic deletion.
Key Facts
- In linear regression, Cook's distance for observation can be written as .
- The residual for observation is , which measures vertical distance from the fitted regression line or plane.
- The leverage value is , the th diagonal entry of the hat matrix .
- The average leverage in a regression model is , where is the number of estimated parameters including the intercept.
- A common leverage flag is or, more conservatively, .
- A common Cook's distance screening rule is , while is often considered a stronger warning sign.
- An observation with a large residual but low leverage may be an outlier without being highly influential.
- An observation with high leverage and a small residual may have little influence on the fitted model, because it already follows the fitted pattern.
Vocabulary
- Cook's distance
- A diagnostic measure that estimates how much the fitted regression model changes when one observation is removed.
- Influential point
- An observation that substantially changes regression coefficients, fitted values, or statistical conclusions when included or excluded.
- Leverage
- A measure of how far an observation's predictor values are from the center of the predictor space.
- Residual
- The difference between an observed response and its predicted value, written as .
- Hat matrix
- The matrix that maps observed responses to fitted values in linear regression.
- Studentized residual
- A residual scaled by an estimate of its standard deviation so residuals from different observations are easier to compare.
Common Mistakes to Avoid
- Deleting every point with large Cook's distance, which is wrong because a flagged value requires investigation of data quality, context, and model sensitivity before removal.
- Treating as an absolute rule, which is wrong because it is only a screening guideline and not a formal hypothesis test.
- Confusing outliers with influential points, which is wrong because an outlier has an unusual response value while influence depends on both residual size and leverage.
- Ignoring high-leverage observations with small residuals, which is wrong because they may still control the fitted line even if they do not appear unusual on a residual plot.
- Forgetting that includes the intercept, which is wrong because leverage averages and Cook's distance formulas use the total number of estimated regression parameters.
Practice Questions
- 1 A regression has observations and estimated parameters. Compute the average leverage and the common high-leverage cutoff .
- 2 For one observation, , , , and . Compute .
- 3 A data set has observations. Use the rule to find the Cook's distance screening cutoff.
- 4 Explain why a point with high leverage but a very small residual might not have a large Cook's distance, yet still deserves attention in regression diagnostics.