Linear Regression Inference Cheat Sheet

Linear regression inference helps students decide whether a linear relationship in sample data gives evidence of a real relationship in the population. This cheat sheet covers inference for the slope, including confidence intervals, hypothesis tests, conditions, and interpretation. It is useful when students already know how to fit a least-squares regression line and now need to make conclusions beyond the sample data.

The main parameter is the population slope $\beta_1$ , which describes the average change in the response variable for each $1$ -unit increase in the explanatory variable. Inference uses the sample slope $b_1$ , its standard error $SE_{b_1}$ , and a $t$ distribution with $df = n - 2$ . Students must check linearity, independence, normality of residuals, and equal variability before trusting a confidence interval or significance test.

Key Facts

The population regression model is $\mu_y = \beta_0 + \beta_1x$ , where $\beta_0$ is the intercept and $\beta_1$ is the slope.
The least-squares regression line is $\hat{y} = b_0 + b_1x$ , where $b_0$ and $b_1$ estimate $\beta_0$ and $\beta_1$ .
The residual standard error is $s = \sqrt{\frac{\sum (y_i - \hat{y}_i)^2}{n - 2}}$ , which estimates the typical vertical prediction error.
The standard error of the slope is $SE_{b_1} = \frac{s}{\sqrt{\sum (x_i - \bar{x})^2}}$ .
A confidence interval for the population slope is $b_1 \pm t^*SE_{b_1}$ with $df = n - 2$ .
The test statistic for testing $H_0: \beta_1 = 0$ is $t = \frac{b_1 - 0}{SE_{b_1}}$ with $df = n - 2$ .
The four main conditions are linear pattern, independent observations, approximately normal residuals, and roughly constant residual spread.
A small $P$ -value gives evidence against $H_0: \beta_1 = 0$ , meaning the data suggest a linear relationship in the population.

Vocabulary

Population slope: The population slope $\beta_1$ is the true average change in the mean response for each $1$ -unit increase in the explanatory variable.
Sample slope: The sample slope $b_1$ is the slope of the least-squares regression line computed from sample data.
Residual: A residual is the difference between an observed value and its predicted value, written as $e_i = y_i - \hat{y}_i$ .
Standard error of the slope: The standard error $SE_{b_1}$ measures the typical sampling variability of the sample slope $b_1$ .
Degrees of freedom: For linear regression inference, the degrees of freedom are $df = n - 2$ because two parameters, $b_0$ and $b_1$ , are estimated.
Prediction interval: A prediction interval estimates a likely range for an individual future response value at a given explanatory value.

Common Mistakes to Avoid

Interpreting $b_1$ as proof of causation is wrong because regression inference can show association, but causation requires a well-designed experiment or strong causal evidence.
Using regression inference without checking residual plots is wrong because curved patterns, outliers, or changing spread can make the $t$ procedures unreliable.
Forgetting that $df = n - 2$ is wrong because linear regression estimates both an intercept and a slope before calculating residual variation.
Interpreting a confidence interval for $\beta_1$ as a range for $y$ is wrong because it estimates the population slope, not individual response values.
Extrapolating far beyond the observed $x$ -values is wrong because the linear pattern may not continue outside the range of the data.

Practice Questions

1 A regression analysis gives $b_1 = 2.4$ , $SE_{b_1} = 0.6$ , and $n = 18$ . Compute the test statistic for $H_0: \beta_1 = 0$ and state the degrees of freedom.
2 A sample gives $b_1 = -1.8$ , $SE_{b_1} = 0.5$ , and $df = 12$ . If $t^* = 2.179$ , find a $95\%$ confidence interval for $\beta_1$ .
3 For a regression with $n = 25$ , the residual sum of squares is $184$ . Compute the residual standard error $s = \sqrt{\frac{\sum (y_i - \hat{y}_i)^2}{n - 2}}$ .
4 A residual plot shows a clear curved pattern. Explain why a linear regression inference procedure for $\beta_1$ may not be appropriate.

Understanding Linear Regression Inference

A regression slope is a sample estimate, so it changes from sample to sample. Inference measures how much that estimate would usually move if the study were repeated with similar data. The residual standard error is central because it describes the size of the vertical misses around the fitted line.

Large residuals mean that points sit far from the line, making the slope less precise. Precision improves when the data have less scatter and when the explanatory variable values are spread across a wider range. A study of study hours from zero to ten hours can estimate a trend more reliably than one where nearly every student studies between four and five hours.

The t statistic compares the observed slope with the slope claimed by a null model. For a test of no linear relationship, the null model says that the population slope is zero. The statistic tells how many estimated slope standard errors separate the sample result from zero.

A large positive or negative value is harder to explain as ordinary sampling variation. The P value is calculated from this value using a t distribution.

It is not the probability that the null model is true. It is the probability of getting a result at least this far from zero if repeated samples came from a population with no linear trend.

A confidence interval gives a range of plausible values for the population slope. Its meaning is about the long run method. If researchers made many intervals from many random samples in the same way, about the stated percent would contain the true slope.

It does not mean there is a stated percent chance that one fixed population slope lies in one particular interval. Students should read interval endpoints in the original units.

For example, an interval from two to five means each one unit increase in the explanatory variable is associated with an average response increase somewhere between two and five units. If zero is outside a matching confidence interval, a two sided test at the matching significance level rejects a zero slope.

Graphs are not optional checks. A residual plot can reveal a curve even when the overall scatterplot looks roughly straight. In that case, a linear slope summarizes the data poorly.

A fan shape in residuals shows changing spread, which can make the usual standard error unreliable. One unusual point can pull the line strongly, especially if its explanatory value is far from all the others. Such a high leverage point deserves investigation, not automatic deletion.

Regression can describe an association, but it cannot by itself prove that one variable causes the other. Confounding variables, biased sampling, and reverse direction can produce convincing slopes without a causal effect. Predictions need extra care too.

A mean response interval estimates the average response at a chosen input. A prediction interval is wider because one individual result has its own random variation. Predictions outside the observed input range are extrapolations, and the line may no longer reflect the real pattern there.

Sign in to save

Sign in to save

Linear Regression Inference Cheat Sheet

Related Tools

Related Labs

Related Worksheets

Related Infographics

Study as Flashcards

Key Facts

Vocabulary

Common Mistakes to Avoid

Practice Questions

Understanding Linear Regression Inference