Linear Regression & Correlation Cheat Sheet

Key Facts

• The least-squares regression line has the form $\hat{y}=a+bx$, where $\hat{y}$ is the predicted response, $a$ is the intercept, and $b$ is the slope.
• The slope of the regression line is $b=r\frac{s_y}{s_x}$, where $r$ is correlation and $s_x$ and $s_y$ are the sample standard deviations.
• The intercept is $a=\bar{y}-b\bar{x}$, so the regression line always passes through the point $(\bar{x},\bar{y})$.
• The correlation coefficient is $r=\frac{1}{n-1}\sum \left(\frac{x_i-\bar{x}}{s_x}\right)\left(\frac{y_i-\bar{y}}{s_y}\right)$ and satisfies $-1\le r\le 1$.
• A residual is $e_i=y_i-\hat{y}_i$, and positive residuals mean the actual value is above the predicted value.
• The least-squares line minimizes the sum of squared residuals, written $\sum (y_i-\hat{y}_i)^2$.
• The coefficient of determination is $r^2$, which gives the proportion of variation in $y$ explained by the linear relationship with $x$.
• Use a regression line for interpolation within the data range, but avoid extrapolation far outside the observed $x$ values.

Vocabulary

Scatterplot

A graph of paired quantitative data values $(x,y)$ used to show the form, direction, strength, and outliers in a relationship.

Correlation coefficient

The number $r$ that measures the direction and strength of a linear association between two quantitative variables.

Least-squares regression line

The line $\hat{y}=a+bx$ that minimizes the sum of squared residuals for a set of data.

Residual

A residual is the prediction error $e=y-\hat{y}$ for one data point.

Coefficient of determination

The value $r^2$ is the proportion of variation in the response variable explained by the regression model.

Extrapolation

Extrapolation is using a regression model to predict values outside the range of the original data.