Linear Regression Inference Cheat Sheet

Key Facts

• The population regression model is $\mu_y = \beta_0 + \beta_1x$, where $\beta_0$ is the intercept and $\beta_1$ is the slope.
• The least-squares regression line is $\hat{y} = b_0 + b_1x$, where $b_0$ and $b_1$ estimate $\beta_0$ and $\beta_1$.
• The residual standard error is $s = \sqrt{\frac{\sum (y_i - \hat{y}_i)^2}{n - 2}}$, which estimates the typical vertical prediction error.
• The standard error of the slope is $SE_{b_1} = \frac{s}{\sqrt{\sum (x_i - \bar{x})^2}}$.
• A confidence interval for the population slope is $b_1 \pm t^*SE_{b_1}$ with $df = n - 2$.
• The test statistic for testing $H_0: \beta_1 = 0$ is $t = \frac{b_1 - 0}{SE_{b_1}}$ with $df = n - 2$.
• The four main conditions are linear pattern, independent observations, approximately normal residuals, and roughly constant residual spread.
• A small $P$-value gives evidence against $H_0: \beta_1 = 0$, meaning the data suggest a linear relationship in the population.

Vocabulary

Population slope

The population slope $\beta_1$ is the true average change in the mean response for each $1$-unit increase in the explanatory variable.

Sample slope

The sample slope $b_1$ is the slope of the least-squares regression line computed from sample data.

Residual

A residual is the difference between an observed value and its predicted value, written as $e_i = y_i - \hat{y}_i$.

Standard error of the slope

The standard error $SE_{b_1}$ measures the typical sampling variability of the sample slope $b_1$.

Degrees of freedom

For linear regression inference, the degrees of freedom are $df = n - 2$ because two parameters, $b_0$ and $b_1$, are estimated.

Prediction interval

A prediction interval estimates a likely range for an individual future response value at a given explanatory value.