Chapter 6

Univariate time series modeling and

forecasting

1
„Introductory Econometrics for Finance‟ © Chris Brooks 2008
„Introductory Econometrics for Finance‟ © Chris Brooks 2008
 I. Time Series: Introduction

Time Series: collection of observations made sequentially through time

Objectives of Time Series Analysis:
-Description: using summary statistics and/or graphical methods (Time Plot)
-Modeling: to find suitable statistical model to describe the Data-Generating-
Process.
-Univariate Model of a variable is based only on past values of that
variable
-Structural Model: based on present and past values of other variables
-Forecasting (Predicting): to estimate future values of the series.
-Control: good forecasts enable the analyst to take action so as to control a
given process (economic or industrial)
 Time Series and Forecasting

“Don’t never prophesy: if you prophesy right, nobody is going to remember and if
you prophesy wrong, nobody is going to let you forget”

Mark Twain

Applications of Time Series forecasting include:

- Economic Planning
- Sales forecasting
- Inventory control
- Production and Capacity Planning
- Budgeting
- Financial Risk Management
 II. Univariate Time Series Models: Notations and Concepts

• Where we attempt to predict returns using only information contained in their

past values.

• A Strictly Stationary Process

A strictly stationary process is one where
P{yt1  b1,..., ytn  bn}  P{yt1  m  b1,..., ytn  m  bn}
i.e. the probability measure for the sequence {yt} is the same as that for {yt+m}  m.

• A Weakly Stationary Process

If a series satisfies the next three equations, it is said to be weakly or covariance
stationary
1. E(yt) =  , t = 1,2,...,
2. E ( yt   )( yt   )   2  
3. E ( yt   )( yt   )   t  t  t1 , t2
1 2 2 1
 Univariate Time Series Models (cont’d)

• So if the process is covariance stationary, all the variances are the same and all
the covariances depend on the difference between t1 and t2. The moments
E ( yt  E ( yt ))( yt  s  E ( yt  s ))   s , s = 0,1,2, ...
are known as autocovariances.

• However, the value of the autocovariances depend on the units of measurement

of yt.
• It is thus more convenient to use the autocorrelations which are the
autocovariances normalised by dividing by the variance:
 s , s = 0,1,2, ...
 s
0
• If we plot s against s=0,1,2,... then we obtain the autocorrelation function
(ACF, acf) or correlogram.
 A White Noise Process (purely randomly process)

• A white noise process is one with (virtually) no discernible structure. A

definition of a white noise process is
E (ut )  
Var (ut )   2
 2 if s  0
s  
0 otherwise

Autocorrelation function will be zero apart from a single peak of 1 at s = 0.

• If ut is further normally distributed then s  approximately N(0,1/T)

where T = sample size and s denotes the autocorrelation at lag s

• We can use this to do significance tests for the autocorrelation coefficients

by constructing a confidence interval.
White Noise Process: Example
 Single Tests: s =0

• s  approximately N(0,1/T) where T = sample size and s denotes the
autocorrelation at lag s
• We can use this to do significance tests for the autocorrelation coefficients
by constructing a confidence interval.

• H0: true value of the autocorrelation coefficient at lag s is zero: s =0

1
• For example, a 95% confidence interval would be given by  .196 
T.

• If the sample autocorrelation coefficient, s , falls outside this region for
any value of s, then we reject the null hypothesis that the true value of the
autocorrelation coefficient at lag s is zero.
 Joint Hypothesis Tests: 1 =0, 2=0,…, m =0
(Data are independently distributed)

• We can also test the joint hypothesis that all m of the k autocorrelation
coefficients are simultaneously equal to zero using the Q-statistic developed
by Box and Pierce: m
Q  T   k2
k 1
where T = sample size, m = maximum lag length
• The Q-statistic is asymptotically distributed as a  m.
2

• However, the Box Pierce test has poor small sample properties, so a variant
has been developed, called the Ljung-Box statistic:
m
 k2
Q  T T  2

~  m2
k 1 T k
• This statistic is very useful as a portmanteau (general) test of linear
dependence in time series.
 An ACF Example

• Question:
Suppose that a researcher had estimated the first 5 autocorrelation coefficients
using a series of length 100 observations, and found them to be (from 1 to 5):
0.207, -0.013, 0.086, 0.005, -0.022.
a) Test each of the individual coefficient for significance
b) Use both the Box-Pierce and Ljung-Box tests to establish whether
they are jointly significant.

• Solution:
a) A coefficient would be significant if it lies outside (-0.196,+0.196) at the
5% level, so only the first autocorrelation coefficient is significant.
b) Q=5.09 and Q*=5.26
Compared with a tabulated 2(5)=11.07 at the 5% level, so the 5 coefficients
are jointly insignificant (low power of the joint test when 1 single
coefficient is significant).
 III. Moving Average Processes

• Let ut (t=1,2,3,...) be a white noise process with:

E(ut)=0 and Var (ut )   , then
2

yt =  + ut + 1ut-1 + 2ut-2 + ... + qut-q

is a qth order moving average model MA(q).

• Its properties are:

E(yt)=;
Var(yt) = 0 = (1+ 12   22 ... q2 )2
Covariances

( s   s 1 1   s  2 2  ...   q q  s ) 2 for s  1,2,..., q

s 
0 for s  q
MA(q) is stationary
 Example of an MA Problem

1. Consider the following MA(2) process:

where t is a zero mean white noise process with variance .

(i) Calculate the mean and variance of Xt
(ii) Derive the autocorrelation function for this process (i.e. express
the autocorrelations, 1, 2, ... as functions of the parameters 1 and
2).
(iii) If 1 = -0.5 and 2 = 0.25, sketch the ACF of Xt.
 Solution

(i) If E(ut)=0, then E(ut-i)=0  i.

So

E(Xt) = E(ut + 1ut-1+ 2ut-2)= E(ut)+ 1E(ut-1)+ 2E(ut-2)=0

Var(Xt) = E[Xt-E(Xt)][Xt-E(Xt)]
but E(Xt) = 0, so
Var(Xt) = E[(Xt)(Xt)]
= E[(ut + 1ut-1+ 2ut-2)(ut + 1ut-1+ 2ut-2)]
= E[ u t2   12 u t21   22 u t2 2 +cross-products]

But E[cross-products]=0 since Cov(ut,ut-s)=0 for s0.

 Solution (cont’d)

So Var(Xt) = 0= E [ ]
=
= (1   1   2 )
2 2 2

(ii) The ACF of Xt.

1 = E[Xt-E(Xt)][Xt-1-E(Xt-1)]
= E[Xt][Xt-1]
= E[(ut +1ut-1+ 2ut-2)(ut-1 + 1ut-2+ 2ut-3)]
= E[(  1u t 1   1 2 u t  2 )]
2 2

=  1  2   1 2  2
= ( 1   1 2 ) 2
 Solution (cont’d)

2 = E[Xt-E(Xt)][Xt-2-E(Xt-2)]
= E[Xt][Xt-2]
= E[(ut +1ut-1+2ut-2)(ut-2 +1ut-3+2ut-4)]
= E[(  2 u t2 2 )]
=  2
2

3 = E[Xt-E(Xt)][Xt-3-E(Xt-3)]
= E[Xt][Xt-3]
= E[(ut +1ut-1+2ut-2)(ut-3 +1ut-4+2ut-5)]
=0

So s = 0 for s > 2.
 Solution (cont’d)

We have the autocovariances, now calculate the autocorrelations:


0   0  1
0
1 ( 1   1 2 ) 2 ( 1   1 2 )
1   
 0 (1   12   22 ) 2 (1   12   22 )
2 ( 2 ) 2 2
2   
 0 (1   12   22 ) 2 (1   12   22 )

3   3  0
0

s   s  0s  2
0

(iii) For 1 = -0.5 and 2 = 0.25, substituting these into the formulae above
gives 1 = -0.476, 2 = 0.190.
 ACF Plot

Thus the ACF (autocorrelation) plot will appear as follows:

1.2

0.8

0.6

0.4
acf

0.2

0
0 1 2 3 4 5 6
-0.2

-0.4

-0.6

s
 IV. Autoregressive Processes

• An autoregressive model of order p, an AR(p) can be expressed as

y t     1 y t 1   2 y t  2  ...   p y t  p  u t
where ut is a white noise process with zero mean.
• Or using the lag operator notation:
Lyt = yt-1 Liyt = yt-i
p
y t      i y t i  u t
i 1
p
• or y t      i Li y t  u t
i 1

or  ( L) y t    u t where  ( L)  1  (1 L  2 L2 ... p Lp ) .

 The Stationary Condition for an AR Model

• The condition for stationarity of a general AR(p) model is that the

roots of 1  1z   2 z ... p z  0 all lie outside the unit circle i.e. |z|>1
2 p

• A stationary AR(p) model is required for it to have an MA()

representation.

• Example 1 (Random walk): Is yt = yt-1 + ut stationary?

The characteristic root is 1, so it is a unit root process (so non-stationary)

• Example 2: Is yt = 2yt-1 - 2.75yt-2 + 0.75yt-3 +ut stationary?

 Wold’s Decomposition Theorem

• An AR(p) stationary series with no constant and no other terms can be

expressed as MA(∞)

• For the AR(p) model  ( L) y t  u t , ignoring the intercept, the Wold

decomposition is

y t   ( L)u t
where,

 ( L)  (1  1 L  2 L2 ... p Lp ) 1
 Characteristics of an Autoregressive Process

• The moments of an autoregressive process are as follows. The mean is given

by
E ( yt )  
1 1  2  ...   p

• The autocovariances and autocorrelation functions can be obtained by

solving what are known as the Yule-Walker equations:
1  1  12  ...   p 1 p
 2  11  2  ...   p  2 p
  
 p   p 11   p  22  ...   p
• If the AR model is stationary, the ACF will decay geometrically to zero.
 Example 5.4

Consider the following simple AR(1) model

yt    1 yt 1  ut
(i) Given that the series is stationary, calculate the (unconditional)
mean of yt

For the remainder of the question, set =0 for simplicity.

(ii) Calculate the (unconditional) variance of yt.

(iii) Derive the autocorrelation function for yt.

 V. The Partial Autocorrelation Function (denoted kk )

• Measures the correlation between an observation k periods ago and the

current observation, after controlling for observations at intermediate lags
(i.e. all lags < k).

• So kk measures the correlation between yt and yt-k after removing the effects
of yt-k+1 , yt-k+2 , …, yt-1 .

• At lag 1, the ACF = PACF: 11 = 1

• At lag 2, 22 = (2-12) / (1-12)

• For lags 3+, the formulae are more complex.

 The Partial Autocorrelation Function (denoted kk)
(cont’d)

• The PACF is useful for telling the difference between an AR process and
an ARMA process.

• In the case of an AR(p), there are direct connections between yt and yt-s only
for s p.

• So for an AR(p), the theoretical PACF will be zero after lag p.

• In the case of an MA(q), if it is invertible (roots of characteristic equations

(z)=0 lie outside the unit circle), it can be written as an AR(), so there are
direct connections between yt and all its previous values.

• For an MA(q), the theoretical PACF will be geometrically declining.

 VI. ARMA Processes

• By combining the AR(p) and MA(q) models, we can obtain an

ARMA(p,q) model:  ( L) y     ( L)u
t t

where  ( L)  1  1 L  2 L2 ... p Lp

and  ( L)  1  1L   2 L  ...   q L

2 q

or y t    1 y t 1   2 y t  2  ...   p y t  p   1u t 1   2 u t  2  ...   q u t  q  u t

with E (u t )  0; E (ut )   ; E (ut u s )  0, t  s

2 2
 The Invertibility Condition

• Similar to the stationarity condition, we typically require the MA(q) part of

the model to have roots of (z)=0 greater than one in absolute value.

• The mean of an ARMA series is given by


E ( yt ) 
1  1  2 ...p

• Remark: The ACF for an ARMA process will display combinations of

behaviour derived from the AR and MA parts
 Summary of the Behaviour of the ACF for
AR and MA Processes

An autoregressive process AR(p) has

• a geometrically decaying ACF
• number of non-zero points of PACF = AR order =p

A moving average process MA(q) has

• Number of non-zero points of ACF = MA order =q
• a geometrically decaying PACF

ARMA models have both ACF and PACF geometrically decaying to 0.

 Some sample acf and pacf plots
for standard processes
The ACF and PACF are not produced analytically from the relevant formulae for a model of
that type, but rather are estimated using 100,000 simulated observations with disturbances
drawn from a normal distribution.
ACF and PACF for an MA(1) Model: yt = – 0.5ut-1 + ut
0.05

0
1 2 3 4 5 6 7 8 9 10
-0.05

-0.1

-0.15
acf and pacf

-0.2

-0.25

-0.3
acf
-0.35
pacf

-0.4

-0.45
Lag
 ACF and PACF for an MA(2) Model:
yt = 0.5ut-1 - 0.25ut-2 + ut

0.4

0.3 acf
pacf

0.2

0.1
acf and pacf

0
1 2 3 4 5 6 7 8 9 10

-0.1

-0.2

-0.3

-0.4
Lags
 ACF and PACF for a slowly decaying AR(1) Model:
yt = 0.9yt-1 + ut

0.9
acf

0.8 pacf

0.7

0.6
acf and pacf

0.5

0.4

0.3

0.2

0.1

0
1 2 3 4 5 6 7 8 9 10
-0.1
Lags
 ACF and PACF for a more rapidly decaying AR(1)
Model: yt = 0.5yt-1 + ut

0.6

0.5
acf
pacf
0.4
acf and pacf

0.3

0.2

0.1

0
1 2 3 4 5 6 7 8 9 10

-0.1
Lags
 ACF and PACF for a more rapidly decaying AR(1)
Model with Negative Coefficient: yt = -0.5yt-1 + ut

0.3

0.2

0.1

0
1 2 3 4 5 6 7 8 9 10
acf and pacf

-0.1

-0.2

-0.3

-0.4
acf
-0.5 pacf

-0.6
Lags
 ACF and PACF for a Non-stationary Model
(i.e. a unit coefficient): yt = yt-1 + ut

0.9
acf
pacf
0.8

0.7
acf and pacf

0.6

0.5

0.4

0.3

0.2

0.1

0
1 2 3 4 5 6 7 8 9 10
Lags
 ACF and PACF for an ARMA(1,1):
yt = 0.5yt-1 + 0.5ut-1 + ut

0.8

0.6
acf
pacf
0.4
acf and pacf

0.2

0
1 2 3 4 5 6 7 8 9 10

-0.2

-0.4
Lags
 VII. Building ARMA Models
- The Box Jenkins Approach

• Box and Jenkins (1970) were the first to approach the task of estimating
an ARMA model in a systematic manner. There are 3 steps to their
approach:

1. Identification
2. Estimation
3. Model diagnostic checking

Step 1: Identification

- Involves determining the order of the model.

- Use of graphical procedures (Time plot, ACF, PACF)
 Building ARMA Models
- The Box Jenkins Approach (cont’d)

Step 2: Estimation
- Estimation of the parameters
- Can be done using least squares or maximum likelihood depending
on the model.

Step 3: Model diagnostic checking

Box and Jenkins suggest 2 methods:

- overfitting (fitting a larger model than required to capture the
dynamics of the data, extra terms would be insignificant)
- residual diagnostics (checking the residuals for evidence of linear
dependence: ACF, PACF, Ljung-Box Test. If linear dependence is
present, the model is not appropriate). More common.
 Some More Recent Developments in
ARMA Modelling
• We want to form a parsimonious model: one with all features of data using
as few parameters as possible

• This gives motivation for using information criteria, which embody 2

factors:
- a term which is a function of the RSS (fitness of the model)
- some penalty for adding extra parameters (parsimonious model)

• The objective is to choose the parameters which minimise the

information criterion.
 Information Criteria for Model Selection

• The information criteria vary according to how stiff the penalty term is.
• The three most popular criteria are Akaike’s (1974) information criterion (AIC),
Schwarz’s (1978) Bayesian information criterion (SBIC), and the Hannan-
Quinn criterion (HQIC).
AIC  ln( 2 )  2 k / T
k
SBIC  ln(ˆ 2 )  ln T
T
2k
HQIC  ln(ˆ 2 )  ln(ln(T ))
T
where k = p + q + 1, T = sample size. So we min. IC s.t. p  p, q  q
SBIC embodies a stiffer penalty term than AIC.
• Which IC should be preferred if they suggest different model orders?
– SBIC is strongly consistent but inefficient.
– AIC is not consistent, but more efficient, and will typically pick “bigger”
models.
 ARIMA Models

• As distinct from ARMA models, the I stands for integrated.

• An integrated autoregressive process is one with a characteristic root

on the unit circle.

• Typically researchers difference the variable as necessary and then

build an ARMA model on those differenced variables.

• An ARMA(p,q) model in the variable differenced d times is

equivalent to an ARIMA(p,d,q) model on the original data.
 XI. Forecasting in Econometrics

• Forecasting = prediction: determine the values that a series is likely to

take

• Examples:
- Forecasting tomorrow‟s return on a particular share
- Forecasting the price of a house given its characteristics
- Forecasting the riskiness of a portfolio over the next year
- Forecasting the volatility of bond returns

• We can distinguish two approaches:

- Econometric (structural) forecasting: relates a dependent variable to one
or more independent variables
- Time series forecasting: involves trying to forecast future values given
previous values and/or previous values of errors

• The distinction between the two types is somewhat blurred (e.g, VARs).
 In-Sample Versus Out-of-Sample

• Expect the “forecast” of the model to be good in-sample.

• Say we have some data - e.g. monthly FTSE returns for 120 months:
1990M1 – 1999M12. We could use all of it to build the model, or keep
some observations back:

• A good test of the model since we have not used the information from
1999M1 onwards when we estimated the model parameters.
 How to produce forecasts

• One-step-ahead forecast: forecast generated for the next observation only

• Multi-step ahead: forecasts generated for 1, 2, 3… s steps ahead
• Recursive forecasting for s steps ahead: initial date is fixed, add one
observation at a time to the estimation period
• Rolling windows for s steps ahead: length of in-sample period is fixed, start
and end dates increase successively by 1

• To understand how to construct forecasts, we need the idea of conditional

expectations: E(yt+1  t )
• Optimal forecast for a white noise process: E(ut+s  t ) = 0  s > 0.

• The two simplest forecasting “methods”

1. Assume no change : E(yt+s  t )= yt
2. Forecasts are the long term average f(yt+s) = y
 Models for Forecasting

• Structural models
e.g. y = X + u
yt  1   2 x2t     k xkt  ut
To forecast y, we require the conditional expectation of its future
E  yt t 1   E 1   2 x2t     k xkt  ut 
value:

= 1   2 E  x2t      k E  xkt 
But what are ( x 2t ) etc.? We could use x 2 , so
E  yt    1   2 x 2     k x k
= y !!!
 problem!!!
 Models for Forecasting (cont’d)

• Time Series Models

The current value of a series, yt, is modelled as a function only of its previous
values and the current value of an error term (and possibly previous values of
the error term).

• Models include:
• simple unweighted averages
• exponentially weighted averages
• ARIMA models
• Non-linear models – e.g. threshold models, GARCH, bilinear models, etc.
 Forecasting with ARMA Models

The forecasting model typically used is of the form:

p q
f t , s      i f t , s  i    j ut  s  j
i 1 j 1

where ft,s = yt+s , s 0; ut+s = 0, s > 0

= ut+s , s  0
 Forecasting with MA Models

• An MA(q) only has memory of q.

e.g. say we have estimated an MA(3) model:

yt =  + 1ut-1 +  2ut-2 +  3ut-3 + ut

yt+1 =  +  1ut +  2ut-1 +  3ut-2 + ut+1
yt+2 =  +  1ut+1 +  2ut +  3ut-1 + ut+2
yt+3 =  +  1ut+2 +  2ut+1 +  3ut + ut+3

• We are at time t and we want to forecast 1,2,..., s steps ahead.

• We know yt , yt-1, ..., and ut , ut-1

 Forecasting with MA Models (cont’d)

ft, 1 = E(yt+1  t ) = E( +  1ut +  2ut-1 +  3ut-2 + ut+1)

=  +  1ut +  2ut-1 +  3ut-2

ft, 2 = E(yt+2  t ) = E( +  1ut+1 +  2ut +  3ut-1 + ut+2)

=  +  2ut +  3ut-1

ft, 3 = E(yt+3  t ) = E( +  1ut+2 +  2ut+1 +  3ut + ut+3)

=  +  3ut

ft, 4 = E(yt+4  t ) = 

ft, s = E(yt+s  t ) =  s4

 Forecasting with AR Models

• Say we have estimated an AR(2)

yt =  + 1yt-1 +  2yt-2 + ut
yt+1 =  +  1yt +  2yt-1 + ut+1
yt+2 =  +  1yt+1 +  2yt + ut+2
yt+3 =  +  1yt+2 +  2yt+1 + ut+3

ft, 1 = E(yt+1  t ) = E( +  1yt +  2yt-1 + ut+1)

=  +  1E(yt) +  2E(yt-1)
=  +  1yt +  2yt-1

ft, 2 = E(yt+2  t ) = E( +  1yt+1 +  2yt + ut+2)

=  +  1E(yt+1) +  2E(yt)
=  +  1 ft, 1 +  2yt
 Forecasting with AR Models (cont’d)

ft, 3 = E(yt+3  t ) = E( +  1yt+2 +  2yt+1 + ut+3)

=  +  1E(yt+2) +  2E(yt+1)
=  +  1 ft, 2 +  2 ft, 1

• We can see immediately that

ft, 4 =  +  1 ft, 3 +  2 ft, 2 etc., so

ft, s =  +  1 ft, s-1 +  2 ft, s-2

• Can easily generate ARMA(p,q) forecasts in the same way.

 How can we test whether a forecast is accurate or not?

• For example, say we predict that tomorrow‟s return on the FTSE will be 0.2,
but the outcome is actually -0.4. Is this accurate? Define ft,s as the forecast made
at time t for s steps ahead (i.e. the forecast made for time t+s), and yt+s as the
realised value of y at time t+s.

• Some of the most popular criteria for assessing the accuracy of time series
forecasting techniques are:
N
1
Mean Squared Error:
MSE 
N

t 1
( yt  s  f t , s ) 2

N
Mean Absolute Error: MAE  1  yt  s  f t , s
N t 1

1 N yt s  ft ,s
Mean Absolute Percentage Error: MAPE  100 
N t 1 yt s
Example
 How can we test whether a forecast is accurate or not?
(cont’d)

• It has, however, also recently been shown (Gerlow et al., 1993) that the
accuracy of forecasts according to traditional statistical criteria are not
related to trading profitability.

• A measure more closely correlated with profitability:

1 N
% correct sign predictions =  zt  s
N t 1

where zt+s = 1 if (yt+s . ft,s ) > 0

zt+s = 0 otherwise
 Forecast Evaluation Example

• Given the following forecast and actual values, calculate the MSE, MAE and
percentage of correct sign predictions:

Steps Ahead Forecast Actual

1 0.20 -0.40
2 0.15 0.20
3 0.10 0.10
4 0.06 -0.10
5 0.04 -0.05

• MSE = 0.079, MAE = 0.180, % of correct sign predictions = 40

 What factors are likely to lead to a
good forecasting model?

• “signal” versus “noise”

• simple versus complex models

• financial or economic theory

 Statistical Versus Economic or
Financial loss functions

Limits of forecasting: What can and cannot be forecast?

• Forecasting models are prone to break down around turning points

• Series subject to structural changes or regime shifts cannot be forecast

• Predictive accuracy usually declines with forecasting horizon

• Forecasting is not a substitute for judgement

 Back to the original question: why forecast?

• Why not use “experts” to make judgemental forecasts?

• Judgemental forecasts bring a different set of problems:
e.g., psychologists have found that expert judgements are prone to the
following biases:
– over-confidence
– inconsistency
– recency
– illusory patterns
– “group-think”…

• Optimal Approach
To use a statistical forecasting model built on solid theoretical
foundations supplemented by expert judgements and interpretation.