Sign in to save

Bookmark this page so you can find it later.

Sign in to save

Bookmark this page so you can find it later.

Logistic Regression Reference cheat sheet - grade 11-12

Click image to open full size

Statistics Grade 11-12

Logistic Regression Reference Cheat Sheet

A printable reference covering logit models, logistic probability, odds ratios, maximum likelihood, classification thresholds, and model evaluation for grades 11-12.

TopicsData AnalysisProbabilityFunctionsGraphsStatisticsStudy Guide
Download PNG

Related Tools

Related Labs

Related Worksheets

Related Infographics

Study as Flashcards

Logistic regression is used when the response variable has two outcomes, such as yes or no, pass or fail, or disease or no disease. This cheat sheet helps students connect a linear predictor to a probability using the logit link. It is useful because ordinary linear regression can give impossible probabilities below 00 or above 11.

Logistic regression keeps predictions between 00 and 11 and supports classification decisions.

Key Facts

  • For binary logistic regression, the response is often coded as Y=1Y = 1 for success and Y=0Y = 0 for failure.
  • The logistic model is p=P(Y=1∣x)=11+e−(β0+β1x)p = P(Y = 1 \mid x) = \frac{1}{1 + e^{-(\beta_0 + \beta_1x)}} for one predictor.
  • The logit form is log⁡(p1−p)=β0+β1x\log\left(\frac{p}{1-p}\right) = \beta_0 + \beta_1x, where p1−p\frac{p}{1-p} is the odds of success.
  • The odds can be found from the logit by p1−p=eβ0+β1x\frac{p}{1-p} = e^{\beta_0 + \beta_1x}.
  • A one-unit increase in xx multiplies the odds by eβ1e^{\beta_1} when all other predictors are held constant.
  • In multiple logistic regression, log⁡(p1−p)=β0+β1x1+β2x2+⋯+βkxk\log\left(\frac{p}{1-p}\right) = \beta_0 + \beta_1x_1 + \beta_2x_2 + \cdots + \beta_kx_k.
  • A common classification rule predicts Y^=1\hat{Y} = 1 if p^≥0.5\hat{p} \ge 0.5 and predicts Y^=0\hat{Y} = 0 if p^<0.5\hat{p} < 0.5.
  • Maximum likelihood chooses the coefficients that make the observed outcomes most likely under the model.

Vocabulary

Binary response
A variable with two possible outcomes, usually coded as 00 and 11.
Probability
The long-run chance that an event occurs, written as pp with 0≤p≤10 \le p \le 1.
Odds
The ratio of the probability of success to the probability of failure, written as p1−p\frac{p}{1-p}.
Logit
The natural logarithm of the odds, written as log⁡(p1−p)\log\left(\frac{p}{1-p}\right).
Odds ratio
The factor by which the odds change for a one-unit increase in a predictor, often written as eβ1e^{\beta_1}.
Classification threshold
A cutoff value such as 0.50.5 used to turn a predicted probability p^\hat{p} into a predicted class.

Common Mistakes to Avoid

  • Treating β1\beta_1 as the change in probability is wrong because β1\beta_1 changes the log-odds, not pp directly.
  • Forgetting to convert from logit to probability is wrong because β0+β1x\beta_0 + \beta_1x can be any real number, while a probability must be between 00 and 11.
  • Interpreting eβ1e^{\beta_1} as an added amount is wrong because an odds ratio multiplies the odds rather than adding to them.
  • Using accuracy alone to judge the model can be misleading because a model may predict the majority class well while missing many important minority cases.
  • Assuming a threshold of 0.50.5 is always best is wrong because the best cutoff depends on the costs of false positives and false negatives.

Practice Questions

  1. 1 For the model log⁡(p1−p)=−2+0.8x\log\left(\frac{p}{1-p}\right) = -2 + 0.8x, find the predicted probability when x=3x = 3.
  2. 2 If a logistic regression coefficient is β1=0.4\beta_1 = 0.4, calculate the odds ratio eβ1e^{\beta_1} and interpret it for a one-unit increase in xx.
  3. 3 A model gives p^=0.72\hat{p} = 0.72 for one student and p^=0.41\hat{p} = 0.41 for another. Using the threshold 0.50.5, classify each student as Y^=1\hat{Y} = 1 or Y^=0\hat{Y} = 0.
  4. 4 Explain why logistic regression is more appropriate than ordinary linear regression when the response variable is binary.

Understanding Logistic Regression Reference

The model works by first calculating a score from the predictor values. That score can be any real number, so it is not yet a probability. A curved conversion then maps the score onto a value from zero to one.

The curve is flat near zero and near one, then steepest around the middle. This shape matches many real situations. A factor may have little visible effect when an event is already very unlikely.

Near the middle, the same change can shift the predicted chance much more. As the event becomes nearly certain, there is less room for the probability to rise.

Coefficients need careful interpretation. A positive coefficient means higher values of that predictor are linked with higher odds of the outcome, after accounting for the other predictors in the model. A negative coefficient means lower odds.

The coefficient itself is not usually a change in probability. Its exponential gives an odds ratio. For example, an odds ratio of two means the odds are multiplied by two for each one unit increase in the predictor.

This does not mean the probability doubles. The difference matters most when the starting probability is high or low.

Students should state the unit clearly. One extra hour of study, one year of age, or one point on a test scale can produce very different interpretations.

Maximum likelihood fits the model by comparing predicted probabilities with what actually happened. A good fit gives high predicted probabilities to cases where the outcome occurred and low predicted probabilities to cases where it did not occur. One incorrect prediction does not automatically make a model poor.

The method considers every case together. Predictions that are confidently wrong are punished more heavily than uncertain predictions.

This is why logistic regression is not fitted by simply drawing the closest line through zero and one values. Computer software usually does the calculations, but students should inspect the data first for missing values, unusual observations, and predictors that are nearly duplicates of each other.

Turning probabilities into labels requires a threshold, but zero point five is only a convention. In medical screening, missing a true illness can be more serious than sending a healthy person for another test. A lower threshold may catch more true cases, though it creates more false alarms.

In spam filtering, a higher threshold may prevent important messages from being wrongly blocked. A confusion matrix counts true positives, false positives, true negatives, and false negatives. Accuracy alone can mislead when one outcome is rare.

Sensitivity describes how well the model finds actual positive cases. Specificity describes how well it rejects actual negative cases.

Finally, association does not prove cause. A coefficient can reflect hidden differences in the groups being compared, so conclusions need context and careful study design.