Time Series ARIMA Reference Cheat Sheet

Key Facts

• An $ARIMA(p,d,q)$ model uses autoregressive order $p$, differencing order $d$, and moving average order $q$.
• The first difference is $\nabla y_t = y_t - y_{t-1}$, and the second difference is $\nabla^2 y_t = y_t - 2y_{t-1} + y_{t-2}$.
• An $AR(p)$ model has the form $y_t = c + \phi_1 y_{t-1} + \phi_2 y_{t-2} + \cdots + \phi_p y_{t-p} + \varepsilon_t$.
• An $MA(q)$ model has the form $y_t = \mu + \varepsilon_t + \theta_1\varepsilon_{t-1} + \theta_2\varepsilon_{t-2} + \cdots + \theta_q\varepsilon_{t-q}$.
• A stationary series has a constant mean, constant variance, and autocovariance that depends only on lag $h$, not on time $t$.
• The autocorrelation at lag $h$ is $\rho_h = \frac{\gamma_h}{\gamma_0}$, where $\gamma_h$ is the autocovariance at lag $h$.
• Model comparison often uses $AIC = 2k - 2\ln(\hat{L})$, where $k$ is the number of estimated parameters and $\hat{L}$ is the maximized likelihood.
• A forecast interval is wider than a point forecast because it accounts for future error uncertainty, often summarized by $\hat{y}_{t+h} \pm z_{\alpha/2}\,SE(\hat{y}_{t+h})$.

Vocabulary

ARIMA

An ARIMA model is a time series model that combines autoregression, differencing, and moving average components to model autocorrelated data.

Stationarity

Stationarity means the statistical behavior of a series stays stable over time, especially its mean, variance, and autocorrelation structure.

Differencing

Differencing transforms a series by subtracting earlier values, such as $\nabla y_t = y_t - y_{t-1}$, to reduce trend and improve stationarity.

Autocorrelation Function

The autocorrelation function, or ACF, measures the correlation between $y_t$ and $y_{t-h}$ for different lags $h$.

Partial Autocorrelation Function

The partial autocorrelation function, or PACF, measures the direct correlation between $y_t$ and $y_{t-h}$ after accounting for shorter lags.

White Noise

White noise is a random error sequence with mean $0$, constant variance $\sigma^2$, and no meaningful autocorrelation.