The coefficient of determination, written R², tells how well a statistical model explains the variation in a response variable. In simple linear regression, it measures how closely the data points fit around the regression line. A larger R² means the model accounts for more of the observed spread in the data.
This makes R² useful for judging whether a linear model gives a strong or weak summary of a relationship.
R² is based on comparing the total variation in the data to the variation left over after fitting the model. If the predictions are close to the actual values, the unexplained variation is small and R² is high. If the predictions are not much better than using the mean, R² is low.
For simple linear regression with one explanatory variable, R² is equal to r², where r is the correlation coefficient.
Understanding Statistics: The Coefficient of Determination (R-squared)
A regression model starts with a baseline prediction. The baseline is the mean of all response values. For example, imagine recording test scores for a class.
If no other information is available, the mean score is the same prediction for every student. A model that uses hours of study tries to improve on that basic guess.
Each actual score differs from its predicted score by a residual. Residuals show the part of the outcome that the model missed.
To calculate the result, statisticians square the residuals before adding them. Squaring matters because positive and negative errors would otherwise cancel out. A prediction that is ten points too high should count as an error just like a prediction that is ten points too low.
Squaring gives larger mistakes extra weight. The model's leftover squared error is compared with the squared error from the mean-only baseline.
The fraction of error removed by the model becomes the coefficient of determination. This comparison explains why the statistic depends on the data being studied, not just on the slope of a line.
A value of 0.64 can be read as sixty-four percent of the variation in the response being captured by the fitted model. It does not mean that sixty-four percent of one student's score was caused by the explanatory variable. It does not prove that studying causes higher scores.
Sleep, prior knowledge, test anxiety, teaching quality, and random chance can affect scores too. A strong fit can occur when two variables move together for another reason. Students should keep prediction separate from cause and effect.
Real data often produce a respectable R-squared value while still containing important problems. A scatter plot may show a curved pattern even when a straight-line model reports a fairly high value. Residual plots help reveal this issue.
Residuals should look like an unstructured cloud around zero. A fan shape suggests that prediction errors grow or shrink across the data range.
One unusual point can change a regression line sharply, especially in a small sample. Always inspect the graph, the residuals, and the context before trusting one summary number.
R-squared usually rises when more predictors are added, even if a new predictor adds little useful information. This makes comparisons between models tricky. Adjusted R-squared applies a penalty for adding predictors that do not improve prediction enough.
Testing a model on new data is even more convincing because it checks whether the pattern holds beyond the original sample. A high result on training data can come from overfitting, where the model learns accidental details instead of a reliable relationship. The best model balances fit, simplicity, sensible assumptions, and usefulness for the situation.
Key Facts
- R² = explained variation / total variation
- R² = 1 - SSE / SST, where SSE is the sum of squared residuals and SST is the total sum of squares
- 0 ≤ R² ≤ 1 for standard linear regression with an intercept
- R² = 0 means the model explains none of the variation in the response beyond the mean
- R² = 1 means the model explains all the variation and every data point lies on the regression line
- For simple linear regression, R² = r², so r = 0.80 gives R² = 0.64
Vocabulary
- Coefficient of determination
- The statistic R² that measures the proportion of variation in the response variable explained by a model.
- Regression line
- The best fit line used to predict the response variable from the explanatory variable in a linear model.
- Residual
- The difference between an observed value and the value predicted by the model.
- Total variation
- The overall spread of the response values around their mean before using a model.
- Correlation coefficient
- The statistic r that measures the direction and strength of a linear relationship between two quantitative variables.
Common Mistakes to Avoid
- Saying R² is the percent of data points on the regression line. R² describes variation explained, not the fraction of points that lie exactly on the line.
- Using R² to decide whether the slope is positive or negative. R² is never negative and does not show direction, so you need the slope or r to know the direction.
- Assuming a high R² proves causation. A strong fit can occur because of lurking variables, coincidence, or common trends, so the study design matters.
- Comparing R² values without checking the model type and data context. A higher R² is not automatically better if the model is inappropriate, overfit, or based on a different response variable.
Practice Questions
- 1 A regression model has SST = 250 and SSE = 40. Calculate R² and interpret the result as a percentage of variation explained.
- 2 In a simple linear regression, the correlation coefficient is r = -0.70. Find R² and explain what the negative sign of r does and does not tell you.
- 3 Two scatterplots both have R² = 0.81. One shows a clear curved pattern around the regression line, while the other shows random scatter around the line. Which model is more appropriate for linear regression, and why?