The least squares regression line is a straight line used to model the relationship between two quantitative variables. It is called a best-fit line because it balances the data points so that the overall prediction error is as small as possible. This matters because regression lets us describe trends, make predictions, and compare how strongly variables move together.
In a scatterplot, the regression line gives a clear visual summary of a data cloud.
Key Facts
- The least squares regression line has the form y hat = a + bx, where b is the slope and a is the y-intercept.
- A residual is residual = observed y - predicted y = y - y hat.
- Least squares means the line minimizes the sum of squared residuals: SSE = sum(y - y hat)^2.
- The slope can be found with b = r(s_y / s_x), where r is correlation, s_y is the standard deviation of y, and s_x is the standard deviation of x.
- The intercept is a = y bar - b x bar, so the regression line always passes through the point (x bar, y bar).
- Example: If b = 2.5, a = 4, and x = 6, then y hat = 4 + 2.5(6) = 19.
Vocabulary
- Least squares regression line
- The line that minimizes the sum of the squared vertical distances between the observed data points and the predicted values.
- Residual
- The vertical difference between an observed y-value and the y-value predicted by the regression line.
- Slope
- The amount the predicted y-value changes for each 1-unit increase in x.
- Intercept
- The predicted y-value when x equals 0, if that value of x is meaningful in context.
- Correlation
- A number from -1 to 1 that describes the direction and strength of a linear relationship between two quantitative variables.
Common Mistakes to Avoid
- Using horizontal distances as residuals is wrong because residuals in ordinary least squares are vertical distances from each data point to the line.
- Forgetting to square residuals is wrong because positive and negative errors would cancel, hiding the total size of prediction errors.
- Interpreting the intercept outside the data context is wrong because x = 0 may not be realistic or included in the observed range.
- Using the regression line to predict far beyond the data is wrong because extrapolation assumes the same linear pattern continues where no data were observed.
Practice Questions
- 1 A regression line is y hat = 12 + 3x. Predict y when x = 7, and find the residual if the observed y-value is 35.
- 2 A data set has r = 0.80, s_x = 5, s_y = 10, x bar = 20, and y bar = 50. Find the slope b, the intercept a, and the regression equation.
- 3 A scatterplot shows a curved pattern, but a least squares regression line is drawn through it. Explain why the line may give misleading predictions even if it passes through the middle of the data.