Lecture 15: Introduction to Box-Jenkins Methodology of

Forecasting

1
 Box-Jenkins Methodology
The Box-Jenkins methodology of forecasting is
different from most methods because it does not
assume any particular pattern in the historical data
of the series to be forecast.

• It uses an iterative approach of identifying a

possible model from a general class of models.
• The chosen model is then checked against the
historical data to see whether it accurately
describes the series.
• The model fits well if the residuals are generally
small and randomly distributed and contain no
useful information.
2
• If the specified model is not satisfactory, the
Flow Chart

3
Definition

4
Example

5
 Implementing the model Building Strategy
• Step 1: Model Identification
The first step in model identification is to determine whether
the series is stationary— that is, whether the time series
appears to vary about a fixed level. It is useful to look at a plot
of the series along with the sample autocorrelation function.

6
 Differencing Required
• If the series is not stationary, it can often be
converted to a stationary series by differencing.
That is, the original series is replaced by a series
of differences. An ARMA model is then specified
for the differenced series. In effect, the analyst is
modeling changes rather than levels.

Once a stationary
series has been
obtained, the analyst
must identify the
form of the model to
be used by
comparing the
autocorrelations and
partial 7
 Partial Auto Correlations

• Partial autocorrelations are

used to measure the degree
of association between Yt
and Yt-k, when the effects of
other time lags (1, 2, 3, …,
k – 1) are removed.
• The partial autocorrelations
at lags 1, 2, 3, …, make up
the partial autocorrelation
function or PACF.

8
 Model Identification
• Identifying the model form is
accomplished by comparing the
autocorrelations and partial
autocorrelations computed from the data
to the theoretical autocorrelations and
partial autocorrelations for the various
ARIMA models.
Theoretical Behavior for AR(1) Model

10
 Theoretical Behavior for AR(2) Model

If the sample autocorrelations die out exponentially to

zero and the sample partial autocorrelations cut off, the
model will require autoregressive terms. 11
Theoretical Behavior for MA(1) Model

12
 Theoretical Behavior for MA(2) Model

If the sample autocorrelations cut off and the sample

partial autocorrelations die out, the model will require
moving average terms. 13
 Theoretical Behavior for ARMA(1,1) Model

If both the sample

autocorrelations and the
sample partial
autocorrelations die out,
both the autoregressive
and the moving average
terms are indicated.
14
Theoretical Behavior for ARMA(1,1) Model

By counting the number of significant sample

autocorrelations and partial autocorrelations, the
orders of the MA and AR parts can be determined.

However, there may be some ambiguity in

determining an appropriate ARIMA model from the
patterns of the sample autocorrelations and partial
autocorrelations. 15
 Model Identification
• To judge their significance, both the sample
autocorrelations and the sample partial
autocorrelations are usually compared
with ,where n is the number of observations
in the time series.

• These limits work well when n is large.

16
 Principal of Parsimony
• All things being equal, simple models are
preferred to complex models. This is known as
the principle of parsimony.
• With a limited amount of data, it is relatively easy to
find a model with a large number of parameters that
fits the data well.
• However, forecasts from such a model are likely to be poor
because much of the variation in the data is due to the
random error that is being modeled.
• The goal is to develop the simplest model that provides an
adequate description of the major features of the data.

17
Example : Model Identification
Neither pattern appears
to die out in a declining
manner at low lags.
Lynn decided to fit both
ARIMA(1, 1, 0) and
ARIMA(0, 1, 1) models to
the Transportation
Index.

18
 Step 2: Model Estimation
Once a tentative model has been selected, the
parameters for that model must be estimated.
• The parameters in ARIMA models are estimated by
minimizing the sum of squares of the fitting errors.
These least squares estimates must, in general, be
obtained using a nonlinear least squares procedure.
• A nonlinear least squares procedure is simply an
algorithm that finds the minimum of the sum of
squared errors function.
• Once the least squares estimates and their standard
errors are determined, t values can be constructed
and interpreted in the usual way.
• Parameters that are judged significantly different
from zero are retained in the fitted model; 19
 Residual Mean Square Error

The residual mean square error is useful for

assessing fit and comparing different models.
It is also used to calculate forecast error limits.
20
 Example: Model Estimation
• If denotes the Transportation Index, then the differenced
series is - , and Lynn’s models are

The Minitab outputs for Lynn’s models are shown

on next slide.

21
22
 Example: Estimation Results

Lynn also noticed that the constant in the ARIMA(0,

1, 1) model is estimated to be , (essentially) the
sample mean of the differences.

23
 Step 3: Model Checking
• Before using the model for forecasting, it must be checked
for adequacy. Basically, a model is adequate if the
residuals cannot be used to improve the forecasts. That is,
the residuals should be random.

1. Many of the same residual plots that are useful in

regression analysis can be developed for the residuals from an
ARIMA model. A histogram and normal probability plot (to
check for normality) and a time sequence plot (to check for
outliers) are particularly helpful.

2. The individual residual autocorrelations should be small and

generally within of zero. Significant residual autocorrelations
at low lags or seasonal lags suggest that the model is
inadequate and that a new or modified model should be
selected.
24
Ljung-Box Q statistic

25
 Example: Model Diagnostics
ARIMA(1, 1, 0): Model for Transportation Index

ARIMA(0, 1, 1): Model for Transportation Index

The Ljung-Box Q statistics computed for groups of lags

m=12, 24, 36, and 48 are not significant, as indicated by the
large p-values for each model. 26
Lynn decided that either model is adequate.
 Residual Autocorrelation Plot

Figure 9 shows there is no significant residual autocorrelation for the

ARIMA(1, 1, 0) model. Although the residual autocorrelation function
27
for the ARIMA(0, 1, 1) model is not shown, the results are similar.
 Step 4: Forecasting with the Model
1. Once an adequate model has been found, forecasts for one period or several
periods into the future can be made. Prediction intervals based on the forecasts
can also be constructed.

In general, for a given confidence level, the longer the forecast lead time, the
larger the prediction interval. This is sensible, since the uncertainty is expected
to be greater for a forecast of a distant value than it is for a forecast of, say, the
next observation. Calculating forecasts and prediction intervals is tedious and
best left to the computer. Computer programs that fit ARIMA models generate
forecasts and prediction intervals at the analyst’s request.

2. As more data become available, the same ARIMA model can be used to
generate
revised forecasts from another time origin.

3. If the pattern of the series appears to be changing over time, the new data
may be
used to re-estimate the model parameters or, if necessary, to develop an entirely
new model.
28
 Monitor the Forecast Errors
• It is a good idea to monitor the forecast errors. If
the most recent errors tend to be consistently
larger than the previous errors, it may be time to
reevaluate the model. At this point, another
iteration of the model-building strategy may be
required. The same holds true if the recent
forecast errors tend to be consistently positive
(underpredicting) or negative (overpredicting).

29
 Example: Forecasting with the Model
• Lynn opted for the ARIMA(1, 1, 0) model on the
basis of its slightly better fit. She checks the
forecast for period 66 for this model as follows:

30