UNIT 14

TIME SERIES MODELLING

TECHNIQUES
Structure
14.1 Introduction qth-order Moving Average Models

Expected Learning Outcomes 14.5 Autoregressive Moving

Average Models
14.2 Time Series Models
Various Forms of ARMA Models
14.3 Autoregressive Models
14.6 Autoregressive Integrated
First-order Autoregressive Models
Moving Averages Models
Second-order Autoregressive
Various Forms of ARIMA Models
Models
14.7 Time Series Model
pth-order Autoregressive Models
Selection
14.4 Moving Average Models
14.8 Summary
First-order Moving Average Models
14.9 Terminal Questions
Second-order Moving Average
14.10 Solutions /Answers
Models

14.1 INTRODUCTION
In the previous units (Units 12 and 13), you have learnt stationary and
nonstationary time series, the concept of autocorrelation and partial
autocorrelation in time series. When the data is autocorrelated, then most of
the standard modelling methods may become misleading or sometimes even
useless because they are based on the assumption of independent
observations. Therefore, we need to consider alternative methods that take
into account the autocorrelation in the data. Such types of models are known
as time series models. In this unit, you will study time series models, such as
autoregressive (AR), moving average (MA) autoregressive moving average
(ARMA), autoregressive integrated moving average (ARIMA), etc.
In this unit, you will learn some time series models. We begin with a simple
introduction of the necessity of time series models instead of ordinary
regression models in Sec. 14.2. In Secs. 14.3 and 14.4, we discuss the
autoregressive and moving average models with their types and properties,
respectively. In Sec. 14.5, the autoregressive moving average models are
117
* Dr. Prabhat Kumar Sangal, School of Sciences, IGNOU, New Delhi
Block 3 Time Series Analysis
explained. The AR, MA and ARMA models are used for stationary time series.
If a time series is nonstationary then we use autoregressive integrated moving
average (ARIMA), therefore, you will learn it in Sec. 14.6. When you deal with
real-time series data, then the first question may arise in your mind how you
know which time series model is most suitable for a particular time series data.
For that, we discuss time series model selection in Sec. 14.7.

Expected Learning Outcomes

After studying this unit, you would be able to:
 explain the necessity of time series models;
 describe and use autoregressive models;
 explain and use moving average models;
 explore autoregressive moving average model;
 describe and use autoregressive integrated moving average models;
and
 selection of a particular time series model for real-life time series data.

14.2 TIME SERIES MODELS

In the previous classes, you have learnt about linear regression, and it is one of
the most common methods for identifying and quantifying the relationship
between a dependent variable and a single (simple linear regression) or
multiple (multivariate linear regression) independent variables. The dependent
variable (Y ) is also called the regress, explained or forecast variable
whereas the independent variable(X) is also called the predictor, regressor
or explanatory variable.
In the simplest case, the regression model allows for a linear relationship
between the forecast variable Y and a single predictor variable X. In the
case of a single independent variable, the regression equation is as follows:
Y =β0 + β1X + ε

The coefficients β0 and β1 denote the intercept and the slope of the
regression line, respectively. The intercept β0 represents the predicted value
of Y when X = 0 and the slope β1 represents the average predicted change
in Y resulting from a one-unit change in X. Also each observation Y is
consisting of the systematic or explained part of the model, β0 + β1X and
random error ε . The term "error" in this context refers to a departure from the
underlying straight line model rather than a mistake and it includes everything
affecting Y other than predictor variable X. The error term has the following
assumptions:
• The mean of the error term should be zero, i.e., E [ ε ] =0 .

• The error term should have constant variance, i.e., Var [ ε ] =σ2 =constant

• It should not correlate with the predictor variable, i.e.,

118 Cor [ ε, X] =Cov [ ε, X] =E [ ε, X] =0

Unit 14 Time Series Modelling Techniques
• The error term should not correlate with its previous values, i.e.,
Cor [ εi , εi−1 ]= Cov [ εi , εi−1 ]= E [ εi , εi−1 ]= 0

• It should be normally distributed with mean 0 and constant variance.

We may use the regression model in time series, but it has some issues which
are as follows:
• The primary issue is that linear regression models assume that the data
points of a variable are independent of each other, while time series
models work with data points that have a temporal relationship with one
another (autocorrelated). For example, there exists a strong relationship
between today’s temperature and the next day’s temperature.
• A second issue is the error distribution. In the regression model, we
assume that the errors are independent and identically distributed (“IID”)
and the error in the model at one time is uncorrelated with the error at
other times, which is usually not fulfilled for time series data, which are
generally autocorrelated.
However, time series data have their historical features of the trend seasonal,
cyclical and irregular components, therefore, they might not fit well with
traditional regression models. In such cases, we use Autoregressive (AR)
models, Moving Average (MA) models, Autoregressive Moving Average
(ARMA) models or Autoregressive Integrated Moving Average (ARIMA)
models, etc. Out of these, AR, MA and ARMA models are used when the time
series is stationary whereas the ARIMA model is used when the time series is
nonstationary. We will discuss one at a time.

14.3 AUTOREGRESSIVE MODELS

In time series data, it is generally observed that the value of a variable
observed in the current time period will be similar to its value in the previous
period, or even the period before that, and so on. For example, if the
temperature is quite high today, it is reasonable to assume that the
temperature will also be high tomorrow. Therefore, the past temperature has
an impact more on the future temperature. In other words, we can say that
there exists a strong relationship between today’s temperature and the next
day’s temperature. Therefore, to forecast the temperature, we can take
previous days’ temperatures as a predictor when fitting a regression model for
such time series data. Consider one more example, suppose we want to
predict today's behaviour of a child. Generally, we see the behaviour of a child
matches with his/her father/mother or grandfather/grandmother. So if today’s
behaviour of a child depends on the behaviour of his/her father/mother then
we can take the father/mother as a predictor/lag because the father/mother is
the previous generation or grandfather/grandmother as lag two generations as
previous to the previous generation. So we can say that there exists
autocorrelation and we can predict today's behaviour of a child using the
behaviour of his/her parents or grandparents. The regression models in which
the predictors are the past values of the series instead of independent
variables are called autoregressive (AR) models. Because such a regression
model uses data from the same input variable at previous time steps,
therefore, it is referred to as an autoregression (regression of self). We can 119
 Block 3 Time Series Analysis
define autoregressive models as
Autoregressive (AR) models are the models in which the value of a
variable in the current period is regressed against its own lagged values,
that is, the value in the previous period.
The AR models are very useful in situations where the next forecasted value is
a function of the previous time period, such as, if it's rainy today, the data
suggests that it's more likely to rain tomorrow than if it's clear today. The
autoregressive models are used for stationary time series. On the basis of the
correlation with the previous values, the autocorrelation model has different
types which are discussed in the next sub-sections.
14.3.1 First-order Autoregressive Models
The autoregressive model is a model in which the value of a variable in the
current period is regressed against its previous value then it is called first-
order autoregression. The number of lags used as regressors is called the
order of autoregression. So, the preceding model is a first-order
autoregression and it is written as AR(1) where 1 represents the order.
Since in time series, we have measured values of a variable over time,
therefore, we use “t” as a subscript in the variable of time series models. If y t
and y t −1 are the values of a variable at time t and t − 1 , respectively then we
can express the first-order autoregressive model as follows:
y t = δ + φ1y t −1 + ε t

The model expresses the present value as a linear combination of the mean of
the series δ (read as delta), the previous value of the variable y t −1 and the
error term ɛt (read as epsilon). The magnitude of the impact of the previous
value on the present value is quantified using a coefficient denoted with φ1
(read as phai). The "error term " is called white noise and it is normally
distributed with mean zero and constant variance (σ2 ).

We now study the properties of the model.

Mean and Variance
If X and Y are We can find mean of an AR(1) model as follows:
independent random
variables and a, b & c E [ y t ]= μ= E [δ] + φ1E [ y t −1 ] + E [ ε t ]
are constant then
Since time series is stationary, therefore,
 
Mean= δ + φ1μ + 0  E [ y t ]= E [ y t −1 ]= μ and 
 
 ε t  N 0,σ  , therefore, E [ ε t ] = 0
2


If X and Y are Thus,

independent random δ
variables and a, b & c Mean =
1 − φ1
are constant then
Similarly,
We can find the variance of an AR(1) model as follows:

120
Var
= [ y t ] Var [δ] + φ12 Var [ y t −1 ] + Var [ε t ]
 Unit 14 Time Series Modelling Techniques
Since time series is stationary, therefore, Var [ y t ] = Var [ y t −1 ]

Also, δ is a constant, therefore, Var [δ] = 0 and ε t  N 0,σ 2  , therefore,

Var [ ε t ] = σ 2 .

Therefore,
Var [ y t ] =
φ12 Var [ y t ] + σ 2

σ2
Var [=
yt ] 2
≥ 0 when φ12 < 1
1 − φ1

Autocovariance and Autocorrelation Functions

We now find the autocovariance of a stationary AR(1) model as follows:
=γk Cov [ y t ,=
y t +k ] Cov [δ + φ1y t −1 + ε t , y t +k ]

= Cov [δ, y t +k ] + +φ1Cov [ y t −1, y t +k ] + Cov [ ε t , y t +k ]

Since the covariance between a constant and a variable is zero, therefore,

Cov [δ, y t +k ] = 0,

Also, Cov [ y t −1, y t +k ] = γk −1 and by the property of ε t

Cov [ ε t , y t +k ] = 0

Therefore,
γk = φ1γk −1
Hence,
σ2  σ2 
γ1 = φ1Var ( y t ) =
φ1γ 0 = φ1
1 − φ12
 Since Var [ y t ] = 
1 − φ12 


σ2  σ2 
γ2 =φ1γ1 =φ12  Since γ 1 = φ1 
1 − φ12  1 − φ12 
σ2
γk = φ1k
φ1γk −1 =
1 − φ12
The autocorrelation function(ACF) for an AR(1) model is as follows:
γk
ρk = φ1k for k =
= 0,1,2,...
γ0

Since φ1 lies between ̶ 1 to 1, therefore, for a positive value of φ1 , the ACF

( φ ) exponentially decreases to 0 as the lag k increases. For the
k
1

negative value of φ1 , the ACF also exponentially decays to 0 as the lag

increases, but the algebraic signs for the autocorrelations alternate between
positive and negative.
Conditions for Stationarity
An autoregressive model can be used on time series data if and only if the
time series is stationary. Therefore, some constraints on the values of the
parameters are required for stationarity which are as follows:

φ1 < 1 ⇒ −1 < φ1 < 1

121
Block 3 Time Series Analysis
Since stationarity is necessary for applying autoregressive models, therefore,
before applying it to time series data, you will have to check whether the time
series is stationary or not. If it is nonstationary, then you will have to apply
some transformation methods (such as differencing, log transformation, etc. as
discussed in Unit 13) to transform the series into a stationary and we use the
ARIMA model. We will discuss it later in this unit.
14.3.2 Second-order Autoregressive Models
The autoregressive model in which the value of a variable in the current period
is regressed against its two previous values then the autoregressive model is
called the second-order autoregression model.
The second-order autoregression model is written as AR(2) where 2
represents the order.
If y t , y t −1 and y t − 2 are the values of a variable at time t, t − 1 and t − 2 ,
respectively then the second-order autoregressive model is expressed as
follows:
y t = δ + φ1y t −1 + φ2 y t − 2 + ε t

We now study the properties of the AR(2) model as we have studied for AR(1).
Mean and Variance
We can find the mean and variance of an AR(2) model as we have obtained
for AR(1). Here, we just write these as follows:
δ
Mean =
1− φ1 − φ2
Similarly,
σ2
Var [ y t ]
= ≥ 0 when φ12 + φ22 < 1
1 − φ12 − φ22
Autocovariance and Autocorrelation Functions
We can find the autocovariance of an AR(2) model as we have obtained for
AR(1) model. Here, we just write these as follows:
σ 2 if k = 0
γk = Cov [ y t , y t +k ] = φ1γk −1 + φ2 γk − 2 + 
0 if k > 0
Therefore,
γ 0 = φ1γ1 + φ2 γ 2
and
γk = φ1γk −1 + φ2 γk − 2 + σ 2 ; k = 1,2,...

The autocorrelation function for an AR(2) model can be obtained by dividing

γk by γ 0 as follows:
γk
ρk = = φ1ρk −1 + φ2ρk − 2 for k = 1,2,...
γ0
Therefore,
ρ1 =φ1ρ0 + φ2ρ−1

122 ρ1 − φ2ρ1 =φ1 (Since ρ0 = 1,ρ−1 = ρ1 )

 Unit 14 Time Series Modelling Techniques
φ1
ρ1 =
1 − φ2
ρ2 =φ1ρ1 + φ2ρ0 =φ1ρ1 + φ2

ρ3 =φ1ρ2 + φ2ρ1

Conditions for Stationarity

Since an autoregressive model can be used on time series data if and only if
the time series is stationary. Therefore, some constraints on the values of the
parameters are required for stationarity which are as follows:
• φ2 < 1 ⇒ −1 < φ2 < 1

• φ1 + φ2 < 1
• φ2 − φ1 < 1

After understanding the first and second-order autoregressive models, you

are interested to know the general form of the autoregressive model. Let’s
discuss the general form of the autoregressive model.
14.3.3 pth-order Autoregressive Models
More generally, a pth-order autoregression, written as AR(p), is a multiple
linear regression in which the value of the series at any time t is a linear
function of the values at times t−1,t−2,…,t−p.
If y t , y t −1,..., y t −p are the values of a variable at time t, t−1,t−2,…,t−p,
respectively, then the pth-order autoregressive model is expressed as follows:
y t = δ + φ1y t −1 + φ2 y t − 2 + ... + φp y t −p + ε t

We now study the properties of the AR(p) model as we have studied for AR(1)
and AR(2) models.
Mean and Variance
We can find the mean and variance of an AR(p) model as we have obtained
for AR(1). Here, we just write these as follows:
δ
Mean =
1 − φ1 − φ2 − ... − φp
and
σ2
=Var [ y t ] ≥ 0 when φ12 + φ22 + ... + φp2 < 1
1 − φ12 − φ22 − ... − φp2
Autocovariance and Autocorrelation Functions
We can find the autocovariance of an AR(2) model as we have obtained for
AR(1). Here, we just write these as follows:
σ 2 if k = 0
γk = Cov [ y t , y t +k ] = φ1γk −1 + φ2 γk − 2 + ... + φp γk −p + 
0 if k > 0
Therefore,
γ 0 = φ1γ1 + φ2 γ 2 + ... + φp γp
and
γk = φ1γk −1 + φ2 γk − 2 + ... + φp γk −p + σ 2 ; k = 1,2,...
123
Block 3 Time Series Analysis
The autocorrelation function for an AR(2) model can be obtained by dividing γk
by γ 0 as follows:

ρk =φ1ρk −1 + φ2ρk − 2 + ... + φpρk −p for k = 1,2,...

Conditions for Stationarity

When p ≥ 3, the restrictions for stationarity are much more complicated,
therefore, we have not discussed them.
After understanding the autoregressive models, let's look at an example which
helps you to understand how to check stationarity and calculate mean,
variance, autocovariance and autocorrelation functions of an autoregressive
model.
Example 1: Consider a time series model
yt =
10 + 0.2y t −1 + ε t

where ε t  N [0,1]

(i) Is this a stationary time series?

(ii) What are the mean and variance of the time series?
(iii) Calculate the autocorrelation function.
(iv) If the current observation is y100 = 7.5, would you expect the next
observation to be above or below the mean?
Solution:
For checking the stationarity of the time series, first of all, we find the
parameters of the time series model. Since the variable in the current period is
regressed against its previous value, therefore, it is the first-order
autoregressive model. We now compare it with its standard form, that is,
y t = δ + φ1y t −1 + ε t and we obtain

δ = 10 and φ1 =0.2

The condition for stationarity of the first-order autoregressive model is

−1 < φ1 < 1 . Since −1 < =
φ1 0.2 < 1, therefore, the series is stationary.

We now calculate the mean and variance of the series as

δ 10
Mean
= = = 12.5
1 − φ1 1 − 0.2
σ2 1 1
[yt ]
Var= = 2
= = 1.04
1 − φ1 1 − 0.04 0.96

We now calculate the autocorrelation function as

σ2
γ1 =
φ1γ 0 =
φ1 0.2 × 1.04 =
= 0.21
1 − φ12

γ2 =φ1γ1 =
0.2 × 0.21 =
0.04

γ3 =
φ1γ 2 =
0.2 × 0.04 =
0.008

124 We can forecast, the next value of y100 using the prediction model as
Unit 14 Time Series Modelling Techniques
ŷ=
t 10 + 0.2y t −1

ŷ101 =10 + 0.2y100 =10 + 0.2 × 7.5 =10 + 0.15 =11.15

Therefore, if the current observation is y100 = 7.5, then the next observation
y101 = 11.15 will be above the mean (12.5).
You may like to try the following Self Assessment Question before studying
further.

SAQ 1
Consider the time series model
yt =
5 + 0.8y t −1 − 0.5y t − 2 + ε t

where ε t  N [0,2]

(i) Is this a stationary time series?

(ii) What are the mean and variance of the time series?
(iii) Calculate the autocorrelation function.
(iv) Plot the correlogram.
(v) Suppose the observations for time periods 50 and 51 are 38 and 40 then
forecast the next observation.

14.4 MOVING AVERAGE MODELS

In the previous section, you learned the autoregression models in which the
value of a variable in the current period is regressed against its own lagged
values, that is, the value in the previous period. In some cases, the forecasting
model is unable to capture all series patterns, and therefore some information
is left over in model residuals which are called forecasting errors. For example,
in our example of today’s behaviour of a child, it may be possible that the
behaviour of a child does not depend on the behaviour of his/her family, and it
may be possible that it depends on the behaviour of his/her friends, therefore,
whatever his/her friend perform in the past the child performs today. It means
that the behaviour may also depend on other unknown factors. In other words,
we can say that we are not aware of that factors and they are not considered
in the model. As you know, the factors which are not considered in the model
come under the residual. It means that the current value of a variable may also
depend on the past residuals. The goal of the moving average models is to
capture patterns in the residuals, if they exist, by modelling the relationship
between the current value of a variable, the error term(∈t), and the past error
(residuals) terms of the models. We can define moving average models as

Moving average (MA) models are the models in which the value of a
variable in the current period is regressed against the residuals in the
previous period.

A moving average model, states that the current value is linearly dependent on
the past error terms. 125
 Block 3 Time Series Analysis
A Moving Average model is similar to an autoregressive model, except that
instead of being a linear combination of past time series values, it is a linear
combination of the past error/residual/white noise terms.
Concept of Invertibility
An autoregressive model can be used on time series data if and only if the
time series is stationary. The moving average models are always stationary.
But some restrictions also are imposed on the parameters of the moving
average model as in the case of the autoregressive model.
As by the definition of the moving average model, the value of a variable in the
current period is regressed against the residuals in the previous period and if
The MA model is defined
in many textbooks and
y t is the value of a variable at time t and ε t −1 is the residuals at time t –1 then
computer software with we can express moving average model of first-order as
minus sign before the
terms. Although this y t =μ + ε t + θ1ε t −1
switches the algebraic
signs of estimated We can write the above expression as
coefficient values and
(unsquared) theta terms ε t = y t − μ − θ1ε t −1
in formulas for ACFs and
variances. It has no ε t = y t − μ − θ1 ( y t −1 − μ − θ1ε t − 2 )
effect on the model’s
overall theoretical = μ ( θ1 − 1) + y t + θ1y t −1 + θ12 ε t − 2
features. In order to
accurately construct the
estimated model, you 
must examine your ∞
software to make sure ε t= c + ∑ θ1i yit
that either to make sure i=0
that either negative or
positive signs are used.
It means that we can convert/invert the past residuals/errors//noises into past
R software uses positive
signs in its underlying observations. In other words, we can convert/invert a moving average model
model, as we take here. into the autoregressive model. This property is called invertibility. This notion
is very important if one wants to forecast the future values of the dependent
variable, otherwise, the forecasting task will be impossible (i.e., the residuals
in the past cannot be estimated, as it cannot be observed). Actually, when the
model is not invertible, the innovations can still be represented by
observations of the future, this is not helpful at all for forecasting purposes.
Any autoregressive process is necessarily invertible but a stationarity condition
must be imposed to ensure the uniqueness of the model for a particular
autocorrelation structure. A moving average process on the other hand is
stationary but in order for there to be a unique model for a particular
autocorrelation structure an invertbility condition must be imposed.
We now discuss different types of moving average models on the basis of the
correlation with the residuals of the previous periods in the following sub-
sections.

14.4.1 First-order Moving Average Models

The moving average model in which the value of a variable in the current
period is regressed against its previous residual is called the first-order moving
average. For example, if today's price of a share depends on whatever has
happened in the other factor on the previous day except for the price of the
126 share on the previous day then we use the first-order moving average.
 Unit 14 Time Series Modelling Techniques
If y t is the value of a variable at time t and ε t −1 is the residuals at time t –1
then the moving average model of first-order is given as follows:

y t =μ + ε t + θ1ε t −1

A first-order moving average model expresses the present value of a variable

as a linear combination of the mean of the series µ, the present error term ε t
and the past error term ε t −1 . Of course, we do not observe the values of ε t .
The magnitude of the impact of past residual on the present value is quantified
using a coefficient denoted with θ1. We use θ1 for that instead of ϕ1 like in the
autoregressive model, to avoid confusion.
The number of residuals used as regressors is called the order of the moving
average model. So, the first-order moving average is written as MA(1) where 1
represents the order.
After understanding the expression of MA(1) model, we now study its
properties.
Mean and Variance
We can find the mean of a MA(1) model as follows:

E [ y t ] =E [μ] + E [ ε t ] + θ1E [ ε t −1 ]

=E [ ε t ] E=
[ε t −1 ] 0 because ε t  N 0,σ 2  
=μ + 0 + θ1 × 0  
and μ is a constant so E (μ) =μ 

Therefore,
Mean of MA(1) = μ

Similarly,

We can find the variance of a MA(1) model as follows:

Var [ y t ] =
Var [μ] + θ12 Var [ ε t −1 ] + Var [ ε t ]

Since

Var [μ] = 0 because μ is a constant

Var [ ε t ] Var
= = [ε t −1 ] σ 2 because ε t  N 0,σ 2 
Therefore,
[ y t ] θ12σ 2 + σ 2
Var =

Var [ y t ] = ( )
1 + θ12 σ 2 ≥ 0

Autocovariance and Autocorrelation Functions

The autocovariance function of an MA(1) model is as follows:
σ2
=γ 0 Var
= ( yt )
1 + θ12

γ1 = θ1σ 2
127
Block 3 Time Series Analysis
γk 0; k > 1
=

Similarly, we have the autocorrelation function of MA(1) model as

γ1 θ1
ρ
=1 =
γ 0 1 + θ12

ρk 0; k > 1
=

It indicates that the autocorrelation function for MA(1) model becomes zero
after lag 1.
Conditions for Invertibility
The moving average models are always stationary. However, some
restrictions are also imposed on the parameters of the moving average models
otherwise the model can not converge. Therefore, some constraints on the
values of the parameters are required for the invertibility of the MA (1)
model which is as follows:
θ1 < 1 ⇒ −1 < θ1 < 1

14.4.2 Second-order Moving Average Models

The moving average model in which the value of a variable in the current
period is regressed against its two previous residuals then it is called the
second-order moving average. It is represented by MA(2). For example, if
today's price of a share depends on whatever has happened in the other
factor in the previous day and the day of the previous day then we use a
second-order moving average.
If y t is the value of a variable at time t and ε t , ε t −1 and ε t − 2 are the residuals at
time t, t – 1 and t – 2, respectively, then the moving average model of second
order is expressed as follows:
y t =μ + ε t + θ1ε t −1 + θ2 ε t − 2

As for MA (1) model, the coefficients θ1 & θ2 represent the magnitude of the
impact of past residuals on the present value.
Let us study the properties of the MA(2) model.

Mean and Variance

The mean and variance of MA(2) model are given as follows:

=E [ ε t ] E=
[ε t −1 ] E=
[ε t −2 ] 0 
Mean = μ + 0 + θ1 × 0 + θ2 × 0  
 because ε t  N 0,σ 2  

Mean of MA(2) = μ

Similarly, We can find the variance of a MA(2) model as follows:

Var [ y t ] =
Var [μ] + θ12 Var [ ε t −1 ] + θ22 Var [ ε t − 2 ] + Var [ ε t ]

Since

128
Var [μ] = 0 because μ is constant
Unit 14 Time Series Modelling Techniques
Var [ ε t ] Var
= = [ε t −1 ] Var
= [ε t −2 ] σ because ε t  N 0,σ 
2 2

Therefore,
Var [ y t ] = θ12σ 2 + θ22σ 2 + σ 2

( )
Var [ y t ] = 1 + θ12 + θ22 σ 2 ≥ 0

Autocovariance and Autocorrelation Functions

The autocovariance function of an MA(2) model is given as follows:

( )
γ 0 =Var ( y t ) = 1 + θ12 + θ22 σ 2

γ1
= ( θ1 + θ1θ2 ) σ 2
γ 2 = θ2σ 2

γk 0; k > 2
=

Similarly, we have the autocorrelation function of MA(2) model as

γ1 θ1 + θ1θ2
ρ
=1 =
γ 0 1 + θ12 + θ22

γ2 θ2
ρ=
2 =
γ 0 1 + θ12 + θ22

ρk 0; k > 2
=

It indicates that the autocorrelation function of MA( 2) model becomes zero

after lag 2.
Conditions for Invertibility
The constraints on the values of the parameters for the invertibility of the
MA (2) model are as follows:
• θ2 < 1 ⇒ −1 < θ2 < 1

• θ1 + θ2 < 1
• θ2 − θ1 < 1

After understanding the first and second-order moving average models, you
are interested to know the general form of the moving average model. Let’s
discuss the general form of the moving average model.

14.4.3 qth-order Moving Average Models

A moving average model states that the current value is linearly dependent on
the current and past error terms.
If y t is the value of a variable at time t and ε t , ε t −1 , ε t − 2 ,...,ε t − q are the residuals
at time t,t − 1,t − 2,...,t − q , respectively, then the moving average model of the
qth order is expressed as the present value ( y t ) as a linear combination of the
mean of the series (μ) , the present error term ( ε t ) , and past error terms ( ε t −1 ,
ε t − 2 ,...,ε t − q ). Mathematically, we express a general moving average model as
follows: 129
Block 3 Time Series Analysis
y t =μ + ε t + θ1ε t −1 + θ2 ε t − 2 + ... + θqε t − q

where θ1,θ2 ,...,θq represent the magnitude of the impact of past errors on the
present value.
After understanding the form of the general moving average model, we now
study the properties of the model.
Mean and Variance
The mean and variance of MA(q) model are given as follows:
Mean = μ

( )
Var [ y t ] = 1 + θ12 + θ22 + ... + θ2q σ 2 ≥ 0

Autocovariance and Autocorrelation Functions

The autocovariance function of a MA(q) model is given below:

(
γ 0 = 1 + θ12 + θ22 + ... + θ2q σ 2)
( θ + θ1θk +1 + ... + θq−k θq ) σ 2 ;k =
1,2,...,q
γk =  k
0;k > q

Similarly, we have the autocorrelation function of MA(q) model as

 θk + θ1θk +1 + ... + θq−k θq
γk  ;k = 1,2,...,q
ρ=
k =  1 + θ12 + θ22 + ... + θ2q
γ0 
0;k > q

It indicates that the autocorrelation function of MA (q) model becomes zero

after lag q.
Conditions for Invertibility
The constraints on the values of the parameters for invertibility of the MA (q)
when q ≥ 3, are much more complicated, therefore, we are not discussing
them.
After understanding the moving average models, let's look at an example
which helps you to understand how to check invertibility and calculate mean,
variance, autocovariance and autocorrelation functions for the moving average
models.

Example 2: Consider the time series model

y t =2 + ε t + 0.8ε t −1

where ε t  N [0,1]

(i) Identify the model.

(ii) Is this a moving average model? If yes check whether it is invertible.

(iii) What are the mean and variance of the time series?

(iv) Calculate autocovariance and autocorrelation functions.

130 (v) Plot the correlogram.

 Unit 14 Time Series Modelling Techniques
(vi) If the residual at t = 100 is 2.3, then would you expect the next
observation to be above or below the mean?

Solution: Since the variable in the current period is regressed against its
previous residual, therefore, it is the first-order moving average model. To
check whether it is invertible, first of all, we find the parameters of the time
series model. We now compare it with its standard form, that is,
y t =μ + ε t + θ1ε t −1

We obtain
μ = 10 , θ1 = 0.8

The invertibility constraints for MA(1) is −1 < θ1 < 1. Since θ1 lies between –1
and 1, therefore, the time series model MA(1) is invertible.

We now calculate the mean and variance of the series as

Mean = 2

( )
Var [ y t ] = 1 + θ12 σ 2 = (1 + 0.64 ) × 1 = 1.64

We now calculate the autocorrelation function as

=γ 0 Var
= ( y t ) 1.64
γ1= θ1σ 2= 0.8 × 1= 0.8

γk 0; k > 1
=

Similarly, we calculate the autocovariance function of MA(1) as

γ1 0.8
ρ
=1 = = 0.49
γ 0 1.64

ρk 0; k > 1
=

We can forecast, the next value of y100 using the prediction model as
ŷ t= 2 + 0.8ε t −1

ŷ101 =2 + 0.8ε100 =2 + 0.8 × 2.3 =2 + 0.18 =2.18

Therefore, if the residual at t = 100 is 2.3, then the next observation y101 = 2.18
will be above the mean (2).

You may try the following Self Assessment Question before studying further.

SAQ 2
Consider the time series model

y t = 42 + ε t + 0.7ε t −1 − 0.2ε t − 2

where ε t  N [0,2]

(i) Identify the model.

131
Block 3 Time Series Analysis
(ii) Is this a moving average model? If yes check whether it is invertible.
(iii) What are the mean and variance of the time series?

(iv) Calculate the autocorrelation function.

(v) Suppose the residual errors for time periods 20 and 21 are 0.23 and 0.54
then forecast the next observation.

14.5 AUTOREGRESSIVE MOVING AVERAGE

MODELS
In the previous sections, you have learnt autoregressive and moving average
models which are used to model time series data. The autoregressive(AR)
models are used when the current value of the time series variable depends
on the past values of the series whereas the moving average models are used
when the current value of the time series variable depends on the
unpredictable shocks (residuals) in the previous periods. But in real-life data,
we also observe that the current value of the time series variable depends not
only on its past values but also on the residuals in the previous periods. For
example, the sale of a product of a company at a current time depends on the
prior sales happening in the past time which plays a role of the AR component
and it also depends on the time-based campaigns launched by the company,
such as distribution of coupons, by one get one free, etc. will increase sales
temporarily and such change in sales is captured by the moving average
component. Therefore, we need models that simultaneously use past data as
a foundation for estimates, and can also quickly adjust to unpredictable shocks
(residuals). In this section, we are going to talk about one such model, called
autoregressive moving average (ARMA), which takes into account past values
as well as past errors when constructing future estimates.
Autoregressive moving average (ARMA) models play a key role in the
modelling of time series. An ARMA process consists of two models: an
autoregressive (AR) model and a moving average (MA) model. In analysis, we
tend to put the residuals at the end of the model equation, so that’s why the
“MA” part comes second. As compared with the pure AR and MA models,
ARMA models provide the most effective linear model of stationary time series
since they are capable of modelling the unknown process with the minimum
number of parameters. In this model, the impact of previous lags along with the
residuals is considered for forecasting the future values of the time series. We
can define ARMA model as follows:
Autoregressive moving average models are simply a combination of an AR
model and an MA model.
Autoregressive Moving Average (ARMA) models are models in which
the value of a variable in the current period is related to its own values
in the previous period as well as values of the residual in the previous
period.

Since ARMA is a combination of both autoregressive terms(p) and moving

average(q) terms, therefore, we represent it as ARMA (p,q). It is also used
132 for stationary time series.
 Unit 14 Time Series Modelling Techniques
On the basis of different values of p and q, the ARMA model has different
types which are discussed as follows:

14.5.1 Various Forms of ARMA Models

The ARMA model has various forms for different values of the parameters p
and q of the model. We discuss some standard forms as follows:

ARMA(1,1) Models
ARMA(1,1) models are the models in which the value of a variable in the
current period is related to its own value in the previous period as well as
values of the residual in the previous period. It is a mixture of AR(1) and
MA(1).
If y t and y t −1 are the values of a variable at time t and t –1, respectively and if
ε t and ε t −1 are the residuals at time t and t –1, respectively then the
ARMA (1,1) model is expressed as follows:
y t = δ + φ1y t −1 + θ1ε t −1 + ε t

As usual, the coefficients δ and ε t denote the intercept/constant factor and

error term at time t, respectively whereas the coefficients φ1 and θ1 represent
AR and MA coefficients and represent the magnitude of the impact of past
values and past error on the present value, respectively.
After understanding the form of the ARMA(1,1) model, we now study the
properties of the model.
Mean and Variance
We can find the mean and variance of an ARMA(1, 1) model as we have
found in AR and MR models which are given as follows:
δ
Mean =
1 − φ1

Var [ y t ]
=
(1 + 2φ θ1 1 )
+ θ12 σ 2
≥ 0 when φ12 < 1
2
1− φ 1

Autocovariance and Autocorrelation Functions

The autocovariance function of an ARMA(1, 1) model is given as follows:

(1 + 2φ θ )
+ θ12 σ 2
=γ 0 Var
= [yt ] 1 1

1− φ 2
1

γ1 =
( φ1 + θ1 )(1 + φ1θ1 ) σ 2
1 − φ12

γk =
φ1γk −1 for k =
2,3,...

Similarly, the autocorrelation function of an ARMA(1,1) model is given as

follows:
 ( φ1 + θ1 )(1 + φ1θ1 )
γk  for k = 1
ρ=
k =  1 + 2φ1θ1 + θ12
γ0  k
φ1 ρk −1 for k = 2,3,...
133
Block 3 Time Series Analysis
The autocorrelation function of an ARMA(1, 1) model exhibits exponential
decay and/or sinusoid pattern towards zero. It does not cut off but gradually
decreases as lag k increases. Also the autocorrelation function of an ARMA(1,
1) model displays the shape of an AR(1) process. The partial autocorrelation
function of an ARMA(1, 1) model also gradually dies out (the same property as
a moving average model) as k increases. It is relatively difficult to the
identification of the order of the ARMA model.
Conditions for Stationarity and Invertibility
The stationarity of the ARMA (1, 1) is related to the AR component in the
ARMA (1, 1) model. Therefore, stationarity conditions which are discussed for
AR(1) are also for ARMA (1,1) model which is as follows:
φ1 < 1 ⇒ −1 < φ1 < 1

Similarly to the stationarity conditions, the invertibility of an ARMA(1,1) model

is related to the MA(1) component. Therefore, the invertibility conditions which
are discussed for MA(1) are also for ARMA (1,1) model which is as follows:
θ1 < 1 ⇒ −1 < θ1 < 1

ARMA (1, 2) Models

ARMA(1,2) models are the models in which the value of a variable in the
current period is related to its own value in the previous period as well as
the residuals of two previous periods. It is a mixture of AR(1) and MA(2)
models.
If y t and y t −1 are the values of a variable at time t and t –1, respectively, and if
ε t ,ε t −1 and ε t − 2 are the residuals at time t, t –1 and t – 2, respectively, then the
ARMA (1,2) model is expressed as follows:
y t = δ + φ1y t −1 + θ1ε t −1 + θ2 ε t − 2 + ε t

The expression for mean, variance, autocovariance and autocorrelation

functions of ARMA(1, 2) model are more complicated, therefore, we are not
giving the expression of these.
Conditions for Stationarity and Invertibility
The stationarity of the ARMA (1, 2) model is related to the AR component in
the ARMA (1, 1) model. Therefore, stationarity conditions which are discussed
for AR(1) are also for ARMA (1,2) model which is as follows:
φ1 < 1 ⇒ −1 < φ1 < 1

Similarly, the invertibility conditions, of an ARMA(1,1) model is related to the

MA(2) component which are as follows:
• θ2 < 1 ⇒ −1 < θ2 < 1

• θ1 + θ2 < 1
• θ2 − θ1 < 1

ARMA (p, q) Models

More generally, ARMA(p,q) models are the models in which the value of a
134 variable in the current period is related to its own p values in the previous
Unit 14 Time Series Modelling Techniques
periods as well as q values of the residual in the previous periods. It is a
mixture of AR(p) and MA(q) models.
If y t , y t −1,..., y t −p are the value of a variable at time t,t − 1,...,t − p , respectively
and if ε t ,ε t −1,...,ε t − q are the residuals at time t,t − 1,...,t − q , respectively, then
the ARMA (p, q) model is expressed as follows:
y t = δ + φ1y t −1 + φ2 y t − 2 + ... + φp y t −p + θ1ε1 + θ2 ε t − 2 + ... + θqε t − q + ε t

The expression for mean, variance, autocovariance and autocorrelation

functions of ARMA (p, q) models are more complicated, therefore, we are not
giving the expression of these.
Autocorrelation and Partial Autocorrelation Functions
The autocorrelation function of an ARMA(p, q) model exhibits exponential
decay and/or sinusoid pattern towards zero. It does not cut off but gradually
decreases as lag k increases. Also the autocorrelation function of an
ARMA(p, q) model displays the shape of an AR(p) process. The partial
autocorrelation function of an ARMA(p, q) model also gradually dies out (the
same property as a moving average model) as k increases. It is relatively
difficult to the identification of the order of the ARMA model.
Conditions for Stationarity and Invertibility of ARMA (p, q) model
When p ≥ 3, the restrictions for stationarity are much more complicated.
Similarly, when q ≥ 3, the restrictions for invertibility become more
complicated, therefore, we are not discussing them here.
After understanding the ARMA models, let's look at an example which helps
you to understand how to identify the order, check stationary and invertible
and calculate mean, variance, autocovariance and autocorrelation functions
for the ARMA models.
Example 3: Consider the time series model
y t = 20 + ε t − 0.5y t −1 + 0.7ε t −1

Assuming that the variance of the white noise is 2.

(i) Identify the model.

(ii) Check whether the model is stationary and invertible.

(iii) Calculate the autocorrelation function ρ1,ρ2 and ρ3 .

Solution:
Since the variable in the current period is regressed against its previous value
as well as previous residual, therefore, it is the ARMA model of order (1, 1). To
check whether it is stationary and invertible, first of all, we find the parameters
of the time series model. We now compare it with its standard form, that is,
y t = δ + φ1y t −1 + θ1ε t −1 + ε t

We obtain
δ = 20 , φ1 =−0.5 , θ1 = 0.7
The stationary constraints for ARMA(1, 1) is −1 < φ1 < 1 . Since φ1 lies
between –1 and 1, therefore, the time series model ARMA(1, 1) is stationary. 135
Block 3 Time Series Analysis
Similarly, the invertibility constraints for ARMA(1,1) is −1 < θ1 < 1. Since θ1
lies between –1 and 1, therefore, the time series model ARMA(1,1) is
invertible.
We now calculate the autocovariance function of ARMA(1, 1) as

 ( φ1 + θ1 )(1 + φ1θ1 )
γk  for k = 1
ρ=
k =  1 + 2φ1θ1 + θ12
γ0  k
φ1 ρk −1 for k = 2,3,...

ρ1 =
( φ1 + θ1 )(1 + φ1=
θ1 ) ( −0.5 + 0.7 )(1 − 0.5 × 0.7
=
) 0.13
= 0.165
1 + 2 × −0.5 × 0.7 + ( 0.7 )
2 2
1 + 2φ1θ1 + θ1 0.79

φ12ρ1 =
ρ2 = −0.5 × −0.5 × 0.165 =
0.041

( −0.5 ) × 0.041 =
3
φ13ρ2 =
ρ3 = −0.005

Now, Let us try some more Self Assessment Questions.

SAQ 3
Consider the ARMA time series model
y t =+
27 0.8y t −1 + 0.3ε t −1 + ε t

Assuming that the variance of the white noise is 1.5.

(i) Is the process stationary and invertible?
(ii) Find ρ1,ρ2 and ρ3 for the process.

14.6 AUTOREGRESSIVE INTEGRATED MOVING

AVERAGE MODELS
In the previous sections of this unit, you have learnt different time series
modes such as autoregressive (AR), moving average (MA) and autoregressive
moving average (ARMA). These models are based on the assumption that the
time series is stationary. But in the real world, most of the time series variables
are nonstationary. In general, trends, and periodicity exist in many time series
data. Hence, the AR, MA, and ARMA models do not apply to nonstationary
time series so there is a need to remove these effects before applying such
models. Therefore, if the input time series is nonstationary, then first we have
to transform the series from a nonstationary into a stationary and after that, we
shall apply models such as the AR, MA, and ARMA. For transforming a
nonstationary time series to stationary, we may use differencing, as discussed
in Unit 13, once, twice or three times, and so on until the series is at least
approximately stationary. As AR and MA processes are described by the
order, in a similar way, the differencing process is also described by the order
of differencing, as 1, 2, 3.... Therefore, to describe a model for nonstationary
time series, the elements make up a triple (p,d,q) instead of two (p, q) that
defines the type of model applied where the degree of the differencing is
represented by the d parameter. Combining the differencing of a nonstationary
time series with the ARMA model provides a powerful family of models that
can be applied in a wide range of situations. The model is described as an
136 autoregressive moving average (ARMA) model. In this form, the letter “I” in
Unit 14 Time Series Modelling Techniques
ARIMA refers to the fact that the time series data has been initially differenced
and when the modelling is completed the results then have to be summed
or integrated to produce the final estimations and forecasts. Box and Jenkins
played a significant role in the development of this extended variant of the
model, therefore, ARIMA models are also referred to as Box-Jenkins models.
The ARIMA model is discussed below:

Autoregressive Integrated Moving Average (ARIMA) model is a

combination of differencing with autoregressive and moving average
models.
We can express the ARIMA model as follows:
y′t = δ + φ1y′t −1 + φ2 y′t − 2 + ... + φp y′t − q − θ1ε1 − θ2 ε t − 2 − ... − θqε t − q + ε t

where y′t is the differenced series which may have been differenced more
than once and p and q are the orders of autoregressive and moving average
parts.
For the first difference, we can write the ARIMA model as
y t − y t −1 = δ + φ1 ( y t −1 − y t − 2 ) + φ2 ( y t − 2 − y t −3 ) + ... + φp ( y t − q − y t − q−1 )
− θ1ε1 − θ2 ε t − 2 − ... − θqε t − q + ε t

On the basis of different values of p, q and d, the ARIMA model has different
types which are discussed as follows:
14.6.1 Various Forms of ARIMA Models

The ARIMA model has various forms for different values of the parameters
p, d and q of the model. We discuss some standard forms as follows:

Model Name Form Use

The errors are

ARIMA(0,0,0) White noise y t = εt
uncorrelated across time.
first-order The series is stationary
ARIMA(1,0,0) autoregressive y t = µ + φ1y t −1 + ε t and autocorrelated with
model its previous value.
y t − y t −1 = µ + ε t The series is not
ARIMA(0,1,0) Random walk
y t = µ + y t −1 + ε t stationary.

Differenced The time series is not

y t − y t −1 = µ + φ1 ( y t − − y t − 2 ) + ε t
first-order stationary and the
ARIMA(1,1,0)
autoregressive y t = µ + y t −1 + φ1 ( y t − − y t − 2 ) + ε t autocorrelated with its
model previous values.
Simple The time series is not
ARIMA(0,1,1) exponential y t − y t −1 = θ1ε t −1 + ε t stationary and the errors
with constant smoothing y t y t −1 + θ1ε t −1 + ε t
= are correlated across
model time.
Simple The time series is not
exponential y t − y t −1 = µ + θ1ε t −1 + ε t stationary and the errors
ARIMA(0,1,1)
smoothing with y t = µ + y t −1 + θ1ε t −1 + ε t are correlated across
growth time.
Damped-trend
linear y t − y t −1 = µ + φ1y t −1 + θ1ε t −1 + ε t The series is not
ARIMA(1,1,1) exponential stationary and the series
smoothing y t = µ + (1 + φ1 ) y t −1 + θ1ε t −1 + ε t has an upward trend
model
137
Block 3 Time Series Analysis
After understanding various time series models, we now discuss an important
topic how to select a suitable time series model for real-life time series data in
the next section.

14.7 TIME SERIES MODEL SELECTION

I hope you understand the various time series models. Broadly, you can divide
all time series models into two categories: the models which are used for
stationary time series such as AR, MA, ARMA and the models which are used
for nonstationary time series such as ARIMA. When you deal with real-time
series data, the first question that may arise in your mind is how you know
which time series model is most suitable for a particular time series data. Don’t
worry about that here we describe the methodology for the same in steps so
that you can easily identify/select the model and its order for the given time
series data. It has the following steps:

It is important that the Step 1: Since there are two types of models which are used for stationary and
choice of order makes nonstationary time series, therefore, first of all, we plot the time series
sense. For example,
suppose you have had data and check whether the time series data is stationary or
blood pressure readings nonstationary as you have learned in Unit 13.
for every day over the
past two years. You Step 2: If the time series is stationary, we have to decide which model out
may find that an AR(1)
or AR(2) model is of AR, MA and ARMA is suitable for our time series data. To
appropriate for distinguish among them, we calculate the autocorrelation function
modeling blood
pressure. However, the (ACF) and the partial autocorrelation function (PACF) as discussed in
PACF may indicate a Unit 13. After that, we plot the ACF and PACF versus the lag, that is,
large partial
autocorrelation value at
correlogram as discussed in Unit 13 and try to identify the pattern of
a lag of 17, but such a both. The ACF plot is most useful for identifying the AR model and
large order for an PACF plot for the order of the AR model whereas the PACF plot is
autoregressive model
likely does not make most useful for identifying the MA model and ACF plot for the order of
much sense. the MA model. We now try to understand how to distinguish between
AR, MA and ARMA models as follows:
Case I (AR model): In the plot of ACF versus the lag (correlogram), if you
see a gradual diminish in amount or exponential decay
then this indicates that the values of the time series are
serially correlated, and the series can be modelled
through an AR mode. For determining the order of an
AR model, we use a plot of PACF versus the lag. If the
PACF output cuts off, which means the PACF is almost
zero at lag p+1, then it indicates that the AR model of
order p. We can also calculate PACF by increasing the
order one by one and as soon as this lies within the
range of ± 2/√n (where n is the size of the time series)
we should stop and take the order as the last significant
PACF as the order of the AR model (see SAQ 4).
Case II (MA model): In the plot of PACF versus the lag, if you see a gradual
diminish in amount or exponential decay then this
indicates that the series can be modelled through an MA
model and if the ACF output cuts off, means the ACF
almost zero, at lag q+1, then it indicates that the MA
138 model of order q.
Unit 14 Time Series Modelling Techniques
Case III (ARMA model): If the autocorrelation function (ACF) as well as the
partial autocorrelation function (PACF) plots show a
gradual diminish in amount (exponential decay) or
damped sinusoid pattern then this indicates that the
series can be modelled through an ARMA model but
it makes the identification of the order of the ARMA
(p, q) model relatively more difficult. For that
extended ACF, generalised sample PACS, etc. are
used which are beyond the scope of this course. For
more detail, you can consult Time Series Analysis
Forecasting and Control, 4th Edition, written by Box,
Jenkins and Reinsel.
Step 3: If the time series is nonstationary, we obtain the first, second, etc.
differences of the time series as discussed in Unit 13 until it becomes
stationary and ensure that trend and seasonal components are
removed and find d. Suppose after the second difference the series
becomes stationary then d is 2. Generally, one or two-stage
differencing is sufficient. The differenced series will be shorter (as you
have observed in Unit 13) than the source series. An ARMA model is
then fitted to the resulting time series. Since ARIMA models have
three parameters, therefore, there are many variations to the possible
models that could be fitted. We should choose the ARIMA models as
simple as possible, i.e. contain as few terms as possible (small values
of p and q). For more detail, you can consult Time Series Analysis
Forecasting and Control, 4th Edition, written by Box, Jenkins and
Reinsel.

Step 4: After identifying the model, we estimate the parameters of the model
using the method of moments, maximum likelihood estimation, least
squares methods, etc. The method of moments is the simplest of
these. In this method, we equate the sample autocorrelation functions
to the corresponding population autocorrelation functions which are
the function of the parameters of the model and solve these
equations for the parameters of the model. However, this method is
not a very efficient method of estimation of parameters. For moving
average processes usually the maximum likelihood method is used
which gives more efficient estimates when n is large. We shall not
discuss this anymore here and if someone is interested in this, may
refer to Time Series Analysis Forecasting and Control, 4th Edition,
written by Box, Jenkins and Reinsel.

Step 5: After fitting the best model, we give a diagnostic check to the
residuals to examine whether the fitted model is adequate or not. It
helps us to ensure no more information is left for extraction and check
the goodness of fit. For the residual analysis, we plot the ACF and
PACF of the residual and check whether there is a pattern or not. For
the adequate model there should be no structure in ACF and PACF of
the residual and should not differ significantly from zero for all lags
greater than one. For the goodness of fit, we use Akaike’s information
criterion (AIC) and Bayesian information criterion (BIC). We have not 139
Block 3 Time Series Analysis
discussed all the above aspects in detail here but interested one
should consult Time Series Analysis Forecasting and Control, 4th
Edition, written by Box, Jenkins and Reinsel.
After understanding the procedure of selection of a time series model, let us
take an example.
Example 4: The temperature (in oC) in a particular area on different days
collected by the meteorological department is as given below:
Day Temperature Day Temperature
1 27 9 28
2 29 10 30
3 31 11 30
4 27 12 26
5 28 13 30
6 30 14 31
7 32 15 27
8 29

(i) Examine which model (AR or MA) is suitable of this data

(ii) Find the order and estimate the parameter of the selected model.
(iii) Write the model.
Solution: First of all, we check whether the given time series is stationary or
nonstationary. For that, we plot the time series data by taking days on the
X-axis and temperature on the Y-axis. We get the time series plot as shown in
Fig. 14.1.

Y
40

38
Temperature

36

34

32

30 X
1 2 3 4 5 6 7 8 9 10 11 12 13
Day

Fig. 14.1: Time series plots of the temperature data.

Fig. 14.1 shows that there is no consistent trend (upward or downward) over
the entire period. The series appears to slowly wander up and down. Also, the
variance is constant. Almost by definition, there is no seasonality or trend. So
we can say that this time series is stationary.
To examine the model and its order, we have to compute the sample
autocorrelation (ACF) and partial autocorrelation (PACF) as we have
discussed in Unit 13. Therefore, for the sack of time, we just write them here

140 r1 = 0.835, r2 = 0.676, r3 = 0.469, r4 = 0.280

Unit 14 Time Series Modelling Techniques
φˆ11 = r1 = 0.835 , φˆ22 =0.088

Since the autocorrelation function (ACF) gradually diminishes (decreases) in

amount, it indicates that the series can be modelled through an AR model and
the PACF output is almost zero, at lag 2, it indicates that the AR model is of
order 1. Hence, we may conclude that the AR (1) model is suitable of this
data. Therefore, the model is
y t = δ + φ1y t −1 + ε t

We now estimate the parameters ( δ and φ1 ) of the model using the method of
moments. In this method, we equate the sample autocorrelation functions to
the population autocorrelation functions which are the function of the
parameters of the model and solve these as below:
r1 = ρ1

0.835 = φ11 ⇒ φ1 = 0.835

For estimating the parameter δ , first, we find the mean of the given data and
δ
then we use the relationship Mean =
1 − φ1
1 15 435
Mean
= ∑=
15 i=1
yi = 29
15

Therefore,
δ
Mean = 29 = ⇒ δ = 29 × 0.165 = 4.785
1 − 0.835
Therefore the suitable model for the temperature data is
yt =
4.785 + 0.835y t −1 + ε t

Before going to the next session, you may like to do some exercise yourself.
Let us try Self Assessment Question.

SAQ 4
A researcher wants to develop an autoregressive model for the data on
COVID-19 patients in a particular city. For that, he collected the data 100 days
and calculated the autocorrelation function which are given as follow:
r1 = 0.73, r2 = 0.39, r3 = 0.07

By calculating sample PACF, estimate the order of the autoregressive model

to be fitted.
We end this unit by giving a summary of what we have covered in it.

14.8 SUMMARY
In this unit, we have discussed

• The necessity of time series models in comparison of regression models.

• Describe various time series modes such as AR, MA, ARMA, ARIMA.
• Explain the various properties of the models.
141
Block 3 Time Series Analysis
• Selection of a particular time series model for real-life time series data.
• The stationarity and invertibility conditions of the model also considered
the role of autocorrelations and partial autocorrelations in the
identification of the models.

14.9 TERMINAL QUESTIONS

1. Define various components of the ARIMA model.
2. Define the parameters of the ARIMA model.
3. For time series data, a researcher obtained the following information:
n = 100, Mean = 26, Error variance = 2.2
r1 = 0.69, r2 = 0.54, r3 =0.38, r4 = 0.38, r5 = 0.29, r6 = 0.0.5
(i) Plot the correlogram
(ii) Which one of the AR and MR models will be more suitable?
(iii) Fit the suitable model.

14.10 SOLUTIONS/ANSWERS
Self Assessment Questions (SAQs)
1. For checking the stationarity of the time series, first of all, we find the
parameters of the time series model. Since the variable in the current
period is regressed against its previous and previous to previous values,
therefore, it is the second-order autoregressive model. We now compare
it with its standard form, that is, y t = δ + φ1y t −1 + φ2 y t − 2 + ε t . We obtain

δ = 5 , φ1 =0.8 , φ2 =−0.5

We now check the stationarity conditions for the second-order

autoregressive model as
−1 < φ2 < 1 ⇒ 0.5 < 1

φ1 + φ2 < 1 ⇒ 0.8 − 0.5 = 0.3 < 1

φ2 − φ1 < 1 ⇒ −0.5 − 0.8 = −1.3 < 1

Since all three conditions for stationarity are satisfied, hence, the time
series is stationary.

We now calculate the mean and variance of the series as

δ 5 5
Mean = = = = 7.14
1 − φ1 − φ2 1 − 0.8 + 0.5 0.7

σ2 2 2
Var [ y t ] = 2
= 2
= = 18.18
1 − φ1 − φ2 1 − 0.64 − 0.25 0.11

We now calculate the autocorrelation function as

φ1 0.8
=ρ1 = = 0.53
142 1 − φ2 1 + 0.5
Unit 14 Time Series Modelling Techniques
ρ2 =φ1ρ1 + φ2 =0.8 × 0.53 − 0.5 =−0.076

−0.061 − 0.265 =
= −0.326
We can forecast, the next value of y52 using the prediction model as
ŷ t =
5 + 0.8y t −1 − 0.5y t − 2

ŷ 52 =
5 + 0.8y 51 − 0.5y 50

ŷ 52 =5 + 0.8 × 0.40 − 0.5 × 0.38 =5.13

2. Since the variable in the current period is regressed against its previous
and previous to previous residuals, therefore, it is a second-order
moving average model. For checking the invertibility of the MA(2) model,
first of all, we find the parameters of the time series model. We now
compare it with its standard form, that is, y t =μ + ε t + θ1ε t −1 + θ2 ε t − 2 . We
have
μ = 42
θ1 = 0.7
θ2 = −0.2

The conditions for the invertibility of the MA(2) model are

• θ2 < 1 ⇒ −1 < θ2 < 1

• θ1 + θ2 < 1
• θ2 − θ1 < 1
Since θ2 = −0.2 , therefore, it lies between –1 and 1

θ1 + θ2 = 0.7 − 0.2 = 0.5 < 1

θ2 − θ1 =
−0.2 − 0.7 =
−0.9 < 1

Since all three conditions for the invertibility of MA(2) model are satisfied,
hence the time series is invertible.
We now calculate the mean and variance of the series as
Mean = 42
σ2 2 2
Var [ y t ] = 2
= 2
= = 1.307
1 + θ1 + θ2 1 + 0.49 + 0.04 1.53

We now calculate the autocorrelation function as

θ1 + θ1θ2 0.7 + 0.7 ( −0.2 ) 0.56
ρ1 = = = = 0.366
2 2
1 + θ1 + θ2 1.53 1.53
θ2 −0.2
ρ2 = 2 2
= = −0.131
1 + θ1 + θ2 1.53

ρk 0; k > 2
=
We can forecast, the next value of y52 using the prediction model as
ŷ t =+
42 0.7y t −1 − 0.2y t − 2

ŷ 22 =
42 + 0.7ε 21 − 0.2ε 20
143
Block 3 Time Series Analysis
ŷ 22 = 42 + 0.7 × 0.54 − 0.2 × 0.23 = 42.332

3. For checking whether it is stationary or invertible, first of all, we find the

parameters of the time series model. We now compare it with its
standard form, that is y t = δ + φ1y t −1 + θ1ε t −1 + ε t

We obtain
δ = 27 , φ1 =0.8 , θ1 = 0.3

The stationary constraints for ARMA(1, 1) is −1 < φ1 < 1 . Since φ1 =0.8

lies between –1 and 1, therefore, the time series model ARMA(1, 1) is
stationary.
Similarly, the invertibility constraints for ARMA(1) is −1 < θ1 < 1. Since
θ1 = 0.3 lies between –1 and 1, therefore, the time series model
ARMA(1) is invertible.
We now calculate the autocovariance function of ARMA(1, 1) as

ρ1 =
( φ1 + θ1 )(1 +=
φ1θ1 ) ( 0.8 + 0.3 )(1 − 0.8 × 0.3 )
=
0.836
= 0.532
1 + 2φ1θ1 + θ12 1 + 2 × 0.8 × 0.3 + ( 0.3 )
2
1.57

( 0.8 ) × 0.532 =
2
φ12ρ1 =
ρ2 = 0.340

( 0.8 ) × 0.340 =
3
φ13ρ2 =
ρ3 = 0.174

4. As we know from Unit 13, the 1st-order partial autocorrelation equals the
1st-order autocorrelation, that is,

φ11 = r1 = 0.73

The range of the PACF function is ±2/√n = ±2/10 = ±0.2

Since φ11 =0.73 lies outside of the range of PACF so we have to

calculate the next order PACF.
The 2nd order (lag) sample partial autocorrelation is

φˆ22 =
(r − r ) =0.39 − ( 0.73 )
2 1
2 2

=−0.31
(1 − r ) 1 − ( 0.73 )
1
2 2

Since φˆ22 =−0.31lies outside the range ±0.2, therefore, the

autoregressive model AR(1) is not possible and we have to calculate
next PACF.
Similarly, We now compute the 3rd order sample partial autocorrelation
function as
1 r1 r1
r1 1 r2
r r r
φ33 =2 1 3
1 r1 r2
r1 1 r3
r2 r1 1
144
Unit 14 Time Series Modelling Techniques
1 r1 r1 1 0.73 0.73
r1 1 r2 = 0.73 1 0.39
r2 r1 r3 0.39 0.73 0.07

1× ( 0.07 − 0.73 × 0.39 ) − 0.73 × ( 0.73 × 0.07 − 0.39 × 0.39 )

=
+ 0.73 × ( 0.73 × 0.73 − 0.39 × 1)

−0.214 + 0.073 + 0.104 =

= −0.037
Similarly,
1 r1 r2 1 0.73 0.39
r1 1 r3 = 0.73 1 0.07
r2 r1 1 0.39 0.73 1

1× (1 − 0.73 × 0.07 ) − 0.73 × ( 0.73 × 1 − 0.39 × 0.07 )

=

+ 0.39 × ( 0.73 × 0.73 − 0.39 × 1)

= 0.949 − 0.513 + 0.056 = 0.492

Therefore,
−0.037
φˆ33 = =−0.075
0.492
Since φˆ22 =−0.075 lies inside the range ±0.2, herefore, AR model of
order 2 will be suitable for this time series. .

Terminal Questions (TQs)

1. The components of the ARIMA model are as follows:

• Autoregression (AR): refers to a model that shows a changing

variable that regresses on its own lagged, or prior, values.

• Integrated (I): represents the differencing of raw observations to

allow the time series to become stationary (i.e., data values are
replaced by the difference between the data values and the
previous values).

• Moving Average (MA): incorporates the dependency between an

observation and a residual error.

2. The ARIMA(p, d, q) model has the parameters p, d, q which are

integers and different values of these indicate the type of ARIMA model
used. The parameters can be defined as:

• p: the number of lag observations in the model, also known as the

lag order.
• d: the number of times the raw observations are differenced; also
known as the degree of differencing.
• q: the size of the moving average window, also known as the order
of the moving average.
3. For the correlogram, we take lags on the X-axis and sample
autocorrelation coefficients on the Y-axis. At each lag, we draw a line, 145
Block 3 Time Series Analysis
which represents the level of correlation between the series and its lags
as shown in the following Fig. 14.2.

In the plot of ACF versus the lag (correlogram), we see a gradual

diminish in amount or exponential decay which indicates that the values
of the time series are serially correlated, and the series can be modelled
through an AR mode. For determining the order of an AR model, we use
PACF. We calculate PACF by increasing the order one by one and as
soon as this lies within the range of ± 2/√n (where n is the size of the
time series) we should stop and take the order as the last significant
PACF as the order of the AR model.

Fig. 14.2: The correlogram of the time series.

The 1st-order partial autocorrelation function equals the 1st-order

autocorrelation function, that is,
φ11 = r1 = 0.691

Since PACF of first-order lies outside the range ± 2/√n = ± 2/√100 = 0.2,
therefore, we calculate second-order PACF as

(
φˆ 22 =
)
r2 − r12
=
0.54 − ( 0.69 )
=
2
0.064
= 0.049
(
1 − r1)
2
1 − ( 0.69 )
2
0.524

Since PACF (2) lies within the range of ± 2/√n = 0.22, therefore, AR(1)
will be suitable for this time series.

146