Correlation and Regression

Understanding Correlation and Regression

The least-squares line is chosen by comparing its predictions with the actual data values. For each point, the residual is the actual y value minus the predicted y value from the line. Points above the line have positive residuals, while points below it have negative residuals.

If ordinary residuals were added, positive and negative values could cancel. Least squares avoids this by squaring every residual, then choosing the line with the smallest total of these squared values. Large errors count much more heavily after squaring, so one distant point can pull the line noticeably toward itself.

The slope has units, which makes it useful in context. If x is hours studied and y is test score points, a slope of three means the model predicts about three more score points for each additional study hour. This is an average pattern in the observed data, not a guarantee for every student.

The intercept can be meaningful only when zero is a realistic x value. A model relating adult height to body mass may produce an intercept at zero height, but that situation has no practical meaning. Students should always state what a slope means using the original units.

The value called r squared comes from comparing the spread of the actual y values with the remaining spread around the regression line. If r squared is close to one, the line accounts for much of the observed variation in y. If it is close to zero, knowing x does little to improve prediction beyond using the mean y value.

A high r squared does not mean every prediction is accurate. The residual plot gives extra evidence. A useful linear model tends to leave residuals scattered randomly around zero, with a fairly similar vertical spread across x values.

A curved pattern suggests that a straight line misses an important feature. A widening fan shape suggests that prediction errors become less consistent.

Regression appears whenever people use one measurement to estimate another. Health researchers may relate dosage to a response. A school may examine attendance and course marks.

A shop may relate advertising spending to sales. In each case, the data must represent the group being studied. A relationship found among one class, season, or location may not hold elsewhere.

Predictions are most reliable within the x values already observed. Extending a line far beyond the data is extrapolation, and it can fail badly when real conditions change. Finally, a strong pattern can arise because of a third factor.

Ice cream sales and sunburn cases may rise together because hot weather affects both. A regression line describes the data pattern. It does not by itself establish a cause.