Regression Multiple, Logistic, GLM Overview Cheat Sheet

Key Facts

• Multiple linear regression uses the model $Y = \beta_0 + \beta_1X_1 + \cdots + \beta_pX_p + \varepsilon$, where $\varepsilon$ is the random error term.
• The coefficient $\beta_j$ in multiple regression estimates the expected change in $Y$ for a one-unit increase in $X_j$, holding the other predictors constant.
• The fitted value in linear regression is $\hat{Y} = b_0 + b_1X_1 + \cdots + b_pX_p$, and the residual is $e_i = y_i - \hat{y}_i$.
• Logistic regression for a binary outcome uses $\log\left(\frac{p}{1-p}\right) = \beta_0 + \beta_1X_1 + \cdots + \beta_pX_p$.
• In logistic regression, the odds ratio for a one-unit increase in $X_j$ is $e^{\beta_j}$ when all other predictors are held constant.
• A generalized linear model has three parts: a response distribution, a linear predictor $\eta = X\beta$, and a link function $g(\mu) = \eta$.
• Common GLM choices include normal with identity link for continuous data, binomial with logit link for binary data, and Poisson with log link for counts.
• Model fit can be assessed with tools such as residual plots, $R^2$, adjusted $R^2$, deviance, AIC, likelihood ratio tests, and cross-validation.

Vocabulary

Multiple Regression

A regression model that predicts one response variable using two or more explanatory variables.

Logistic Regression

A regression model used for binary outcomes that models the log odds of success as a linear function of predictors.

Generalized Linear Model

A framework that models a response variable using a probability distribution, a linear predictor, and a link function.

Link Function

A function $g(\mu)$ that connects the mean response $\mu$ to the linear predictor $\eta = X\beta$.

Odds Ratio

A multiplicative change in the odds of success, usually calculated as $e^{\beta_j}$ for predictor $X_j$ in logistic regression.

Deviance

A GLM fit measure based on likelihood that compares a fitted model with a saturated model.