ARIMA models are used to describe and forecast time series data with trends, autocorrelation, and random shocks. This cheat sheet summarizes the notation, assumptions, identification tools, and workflow students need when fitting ARIMA models. It is designed as a quick reference for choosing model orders, checking stationarity, and interpreting forecasts.
Students use ARIMA to connect statistical theory with practical forecasting tasks in economics, science, engineering, and business.
An model combines autoregressive terms, differencing, and moving average error terms. The main ideas are stationarity, autocorrelation, partial autocorrelation, residual diagnostics, and forecast uncertainty. Differencing removes many types of trend by applying one or more times.
A good ARIMA model has residuals that behave like white noise, meaning they have mean , constant variance, and little remaining autocorrelation.
Key Facts
- An model uses autoregressive order , differencing order , and moving average order .
- The first difference is , and the second difference is .
- An model has the form .
- An model has the form .
- A stationary series has a constant mean, constant variance, and autocovariance that depends only on lag , not on time .
- The autocorrelation at lag is , where is the autocovariance at lag .
- Model comparison often uses , where is the number of estimated parameters and is the maximized likelihood.
- A forecast interval is wider than a point forecast because it accounts for future error uncertainty, often summarized by .
Vocabulary
- ARIMA
- An ARIMA model is a time series model that combines autoregression, differencing, and moving average components to model autocorrelated data.
- Stationarity
- Stationarity means the statistical behavior of a series stays stable over time, especially its mean, variance, and autocorrelation structure.
- Differencing
- Differencing transforms a series by subtracting earlier values, such as , to reduce trend and improve stationarity.
- Autocorrelation Function
- The autocorrelation function, or ACF, measures the correlation between and for different lags .
- Partial Autocorrelation Function
- The partial autocorrelation function, or PACF, measures the direct correlation between and after accounting for shorter lags.
- White Noise
- White noise is a random error sequence with mean , constant variance , and no meaningful autocorrelation.
Common Mistakes to Avoid
- Using ARIMA before checking stationarity is wrong because nonstationary data can produce misleading coefficients and overly confident forecasts.
- Choosing and only by visual guesswork is unreliable because ACF and PACF patterns can be ambiguous, so students should also compare diagnostics and information criteria such as .
- Over-differencing the series is a mistake because too large a value of can add unnecessary noise and create artificial negative autocorrelation.
- Trusting a model with autocorrelated residuals is wrong because remaining residual structure means the ARIMA model has not captured all predictable time dependence.
- Interpreting forecast intervals as fixed guarantees is incorrect because a forecast interval describes long-run coverage under the model, not certainty for one future value.
Practice Questions
- 1 For the series values , , , and , compute the first differences , , and .
- 2 An analyst fits . Identify the values of , , and , and state what each part represents.
- 3 Two fitted models have and . Which model is preferred by AIC, and why?
- 4 A residual ACF plot shows several significant spikes after fitting an ARIMA model. Explain what this suggests about the model and what the analyst should check next.
Understanding Time Series ARIMA Reference
A useful analysis starts before any model is fitted. Plot the observations in time order and mark events that may explain sudden changes. A sales series may jump after a price change.
Daily electricity use may change during a heat wave. A long upward path can come from genuine growth, but it can also come from a one time change in how data were recorded. Differencing can help with a changing level, yet it removes information and shortens the usable series.
Too much differencing often creates a jagged pattern with strong negative dependence from one observation to the next. Use the smallest amount that makes the behavior stable enough to model. A log transformation is often useful when larger values have much larger swings than smaller values.
The autocorrelation plot and partial autocorrelation plot provide clues, not automatic answers. An autoregressive pattern often shows a partial autocorrelation that becomes small after a few lags, while a moving average pattern often shows this behavior in the autocorrelation plot. Real data rarely follow these clean textbook shapes.
Sampling noise can make a small spike look important, especially when the series is short. Look for a consistent pattern across several lags and compare a few reasonable candidate models. Seasonal repetition needs special attention.
Monthly data may have a yearly cycle, while quarterly data may repeat every four observations. Ordinary differencing cannot always remove this pattern. A seasonal version of the model may be needed when values at the same point in each cycle remain closely related.
After estimation, residual checks show whether the model has left useful structure unexplained. Plot residuals over time, inspect their autocorrelations, and look at their distribution. Long runs of positive or negative residuals suggest that forecasts are missing a pattern.
A Ljung Box test can check several residual lags together, but its result should not replace plots and judgment. Very large residuals may be outliers, data entry mistakes, or real unusual events. They should be investigated rather than silently deleted.
A model can fit historical data well while failing after a policy change, a new competitor, or a sensor replacement. These are structural breaks, and no choice of orders can fully repair a model trained on an outdated regime.
Forecasting should be tested in the same direction that it will be used. Fit the model using an earlier block of time, then forecast a later block that was hidden during fitting. Repeating this process at several cutoff dates gives a more honest view of performance than testing on the training data.
Compare forecasts with simple baselines such as the latest observed value or the value from the same season last year. Measures such as mean absolute error make the size of typical mistakes easier to interpret. For forecasts many steps ahead, uncertainty grows because each future value depends partly on earlier forecasts.
Prediction intervals communicate this growing uncertainty. In real settings, a forecast is most useful when its likely error is considered alongside the decision it supports, such as ordering stock, scheduling staff, or planning energy supply.