Logistic Regression Reference Cheat Sheet

Key Facts

• For binary logistic regression, the response is often coded as $Y = 1$ for success and $Y = 0$ for failure.
• The logistic model is $p = P(Y = 1 \mid x) = \frac{1}{1 + e^{-(\beta_0 + \beta_1x)}}$ for one predictor.
• The logit form is $\log\left(\frac{p}{1-p}\right) = \beta_0 + \beta_1x$, where $\frac{p}{1-p}$ is the odds of success.
• The odds can be found from the logit by $\frac{p}{1-p} = e^{\beta_0 + \beta_1x}$.
• A one-unit increase in $x$ multiplies the odds by $e^{\beta_1}$ when all other predictors are held constant.
• In multiple logistic regression, $\log\left(\frac{p}{1-p}\right) = \beta_0 + \beta_1x_1 + \beta_2x_2 + \cdots + \beta_kx_k$.
• A common classification rule predicts $\hat{Y} = 1$ if $\hat{p} \ge 0.5$ and predicts $\hat{Y} = 0$ if $\hat{p} < 0.5$.
• Maximum likelihood chooses the coefficients that make the observed outcomes most likely under the model.

Vocabulary

Binary response

A variable with two possible outcomes, usually coded as $0$ and $1$.

Probability

The long-run chance that an event occurs, written as $p$ with $0 \le p \le 1$.

Odds

The ratio of the probability of success to the probability of failure, written as $\frac{p}{1-p}$.

Logit

The natural logarithm of the odds, written as $\log\left(\frac{p}{1-p}\right)$.

Odds ratio

The factor by which the odds change for a one-unit increase in a predictor, often written as $e^{\beta_1}$.

Classification threshold

A cutoff value such as $0.5$ used to turn a predicted probability $\hat{p}$ into a predicted class.