Univariate Time Series Models

Gianluca Cubadda
Universita di Roma "Tor Vergata"
 Program

Stationary time series: Basic concepts. Stationarity, Total and partial

autocorrelation, Ergodicity, Linear stationary processes, ARMA mod-
els, Outliers, Forecasting.

Nonstationary time series: ARIMA models, The Beveridge-Nelson

Trend-Cycle decomposition, Seasonality,

Statistical inference: Estimation, Identi cation, Diagnostic checking.

Unit roots in economic and nancial time series: Deterministic trends

vs. random walks, Unit-roots tests, Impulse response function and
measures of persistence.
 List of references

Brockwell and Davis (2002) Introduction to Time Series and Forecast-

ing, second edition, Springer-Verlag, New York.

Hamilton (1994), Time Series Analysis, Princeton University Press.

Wei (2006) Time Series Analysis: Univariate and Multivariate Meth-

ods, second edition, Addison-Wesley.
1 Univariate time series analysis: Basic concepts

We consider a univariate time series, yt; t = 1; : : : ; T .

The information set is the series itself and its position in time.

We now review some basic concepts in time series analysis, along with
simple and essential tools for descriptive analysis.

The main descriptive tool is the plot of the series, by which we represent
the pair of values (t; yt) on a Cartesian plane.

The graph can immediately reveal the presence of important features, such
as trend and seasonality, structural breaks and outliers, and so forth.

The series may be a transformation of the original measurements: loga-

rithms; changes, log-di erences, etc.
 German Industrial Production Index (1995=1) US Gross Domestic Product
IPI IPI_SA
9.5 ln(GDP)
1.2 GDP
∆ 4 ln GDP
9.0

1.1
8.5

1.0
8.0

0.9 7.5

1995 2000 1950 1960 1970 1980 1990 2000

US CPI Standard & Poors 500 log returns
CPI ∆ 1 2 ln CPI DLs&p
15
0.1

10 0.0

-0.1
5

-0.2
0
1950 1960 1970 1980 1990 2000 1950 1960 1970 1980 1990 2000

Figure 1: Plots of various time series

2 Stationary stochastic processes

Stochastic process: a collection of random variables fyt(! ); ! 2 ; t 2 Zg

de ned on a probability space ( ; F; P ), where the integer number t is a
time-index, is the sample space, F is a sigma algebra de ned on and
P is a probability measure on . A time series is a realization of the
stochastic process for a given ! 2 and t = 0; 1; 2; :::; T .

Stationarity: yt is weakly stationary if 8t; k 2 Z :

E(yt) = ; j j<1
E(yt )2 = (0) < 1
E(yt )(yt k ) = (k)
yt is strictly stationary if 8t; k; h 2 Z :
d
(yt; yt+1; :::yt+h) = (yt+k ; yt+1+k ; :::yt+h+k )
Strict stationarity implies weak stationarity whereas the viceversa is in gen-
eral not true. The exception are Gaussian processes, i.e., if the distribution
of (yt; yt+1; :::yt+h) is a multivariate Gaussian for 8t; h 2 Z .

Autocovariance function, (k), is symmetric: (k) = ( k).

The partial autocovariance function at lag k is the covariance between yt
and yt k having removed the e ects of wt = (yt 1; ; yt k+1), i.e.
g (k) = E [yt E(ytjwt)][yt k E(yt k jwt)]

Autocorrelation function (ACF):

(k) = (k)= (0)
i) (0) = 1; ii) j (k)j < 1; iii) (k) = ( k).
The partial autocorrelation function (PACF):
n o1=2
r(k) = g (k)= E[yt E(ytjwt)]2E[yt k E(yt k jwt)] 2
White noise (WN): "t WN( 2);
E("t) = 0; 8t;
E("2t ) = 2 < 1 8t;
E("t"t k ) = 0; 8t; 8k 6= 0:

Lag operator: Lk yt = yt k ; L is an algebraic operator.

Wold theorem: (almost) any weakly stationary stochastic process can be

represented as a linear process, i.e.
yt = + " t + 1 "t 1 + 2 "t 2 + = + (L)"t;
P P1 2 < 1 (square sum-
where (L) = 1 j=0 j Lj , with
0 = 1 and j=0 j
mability).
For a stationary linear process we have:
E(yt) = ; (0) = 2 P1 2 (k) = 2 P1
j=0 j < 1; j=0 j j+k :
2.1 Estimation
T
sample mean ^ = y = T 1 P y
t
t=1

P
sample variance: ^ (0) = T 1 T
t=1 (yt y )2

P
sample autocovariance: ^ (k) = T 1 T
t=k+1 (yt y )(yt k y)

The ACF is estimated by ^(k) = ^ (k)= ^ (0); the barplot (k; ^(k)) is
d
the correlogram. If yt WN( 2), then T 1=2 ^(k) ! N(0; 1).
2.2 Ergodicity

Ergodicity: A stationary stochastic process is ergodic for the rst and

second moments when the sample mean and autocovariance function are
mean-square consistent. Notice that stationarity itself does not imply er-
godicity.
A su cient condition for ergodicity for the rst two moments of a linear
process is that
P1
j=0 j j j < 1
(absolute summability). This implies square summability.
The above condition is equivalent to require that the autocovariance func-
tion is absolutely summable:
P1
k=0 j (k )j < 1
3 Genesis and Properties of Autoregressive -
Moving Average (ARMA) processes

A problem narises witho linear stationary process: an in nite number of

coe cients j ; j > 0 need to be estimated.

P1
Since j=0 j j j < 1 implies that j!1
lim j = 0, we could approximate
(L) by its "truncated" version e(L) such that
(
e = j; j m
j 0; j > m
where m ! 1 and m=T ! 0 as T ! 1.
However, the "best" approximation of a 1 order polynomial is obtained
by a rational polynomial, i.e.
(L)
(L) ' ;
(L)
where
Pp j; p < 1
(L) = 1 j=1 j L
Pq
(L) = 1 + j=1 j Lj ; q < 1

Autoregressive-Moving average (ARMA) processes: A linear stationary

process such that (L) = (L)= (L), which can be rewritten as
(L)yt = (L)"t;
Pp Pq
yt = j=1 j yt j + "t + j=1 j "t j
It is denoted as yt ARMA(p; q ), where p is the AR order and q is the MA
order.
Inversion of a rst-order polynomial: Consider (L) = (1 L) such that
j j < 1. From the relation

1 ( L)n+1 = (1 L)[1 + L + ( L)2 + + ( L)n];

we obtain that
1
1 [1 + L + ( L)2 + + ( L)n] X
j
= lim = ( L)
1 L n!1 1 ( L)n+1 j=0
Assume now that (L) = 1 and (L) = (1 L) with j j < 1. Hence,
we have
2" "t
yt = "t + "t 1 + t 2 + = = "t + yt 1:
1 L
3.1 Autoregressive (AR) processes

AR(1) process: The autoregressive process of order 1, AR(1), is generated

by the equation
yt = m + yt 1 + "t
The process is stationary if j j < 1. Indeed, by recursive substitution we
obtain the Wold representation:

yt = m=(1 ) + " t + "t 1 + + n "t n +

P1 P1
Hence, the condition j=0j j j = j=0j j j < 1 is satis ed i j j < 1 or,
equivalently, i the root of 1 L, is greater then 1 in modulus.
The expected value can be easily computed from either the AR or the Wold
representation:
E(yt) = = m=(1 )

The demeaned AR(1) process: In view of the above equation, we see that
any stationary AR(1) process can be rewritten as a zero-mean AR(1)
process:
(yt ) = (yt 1 ) + "t
From the demeaned AR(1) process we easily get
Var(yt) = (0) = E[(yt )yt] = E[ (yt 1 )yt + "tyt]
= (1) + 2
since E[(yt )"t] = E[("t + "t 1 + 2"t 2 + )"t] = 2.
(1) = E[(yt )yt 1] = E[ (yt 1 )yt 1 + "tyt 1]
= (0)
since E[(yt 1 )"t] = E[("t 1 + "t 2 + 2"t 3 + )"t] = 0.

Replacing (1) in the expression for (0), we obtain:

2
(0) = 2
1

Moreover, (k) = (k 1) for k 1, so that (k) = k (0).

The autocorrelation function (ACF) is thus

(k) = k

The partial autocorrelation function (PACF) is easily obtained as

(
(k); k 1
r(k) =
0; k > 1
since yt E[ytj(yt 1; ; yt k+1)] = "t for k > 1.
Figure 2: ACF and PACF of an AR(1) process
Random Walk (RW): the RW is generated by the equation

yt = m + yt 1 + "t
It is a non-stationary AR(1) process since = 1. By recursive substitutions
we get
Pt 1
yt = y0 + mt + j=0"t j
Treating the initial condition y0 as xed, it's easy to compute the moments:
E(yt) = y0 + mt
Var(yt) = t 2
Cov(yt; yt k ) = (t k) 2 0 k<t
which vary over time.
The term m is called the drift of the RW.
AR(2) processes: The AR(2) process is generated by the equation

yt = m + 1yt 1 + 2yt 2 + "t

It can be shown that yt is stationary if the roots of 1 2 are

1L 2L
greater than 1 in modulus (lie outside the unit circle).

The above condition is equivalent to: i) j 2j < 1 and ii) j 1j < 1 2.

The stationarity region of the AR(2) parameters lies inside the triangle
with vertices (-2,-1),(2,-1),(0,1). A pair of complex conjugate roots arises
for 21 + 4 2 < 0.
Figure 3: The complex unit circle
Figure 4: Stationarity region of an AR(2) process
Under stationarity, we can easily compute the rst two moments of the
AR(2) process:

Expected value: E(yt) = = m=(1 1 2 ).

The demeaned AR(2) process:

(yt ) = 1(yt 1 ) + 2(yt 2 ) + "t

Autocovariance function: it is given recursively by

(k) = 1 (k 1) + 2 (k 2); k = 2; 3; : : :
with starting values
(1 2
2)
(0) = 2 2]
; (1) = 1 (0)=(1 2 ):
(1 + 2)[(1 2) 1
The expression for (k) can be derived from the demeaned AR(2)
equation as follows:
(0) = E[ 1(yt 1 )yt + 2(yt 2 )yt + "tyt]
= 1 (1) + 2 (2) + 2
(1) = E[ 1(yt 1 )yt 1 + 2(yt 2 )yt 1 + "tyt 1]
= 1 (0) + 2 (1)
(k) = E[ 1(yt 1 )yt k + 2(yt 2 )yt k + "tyt k ]
= 1 (k 1) + 2 (k 2); k = 2; 3; : : :
Compute (1) from the second equation, and substitute in the equa-
tion for (2), then replace for (1) and (2) in the rst expression to
get (0).
ACF:
(k) = 1 (k 1) + 2 (k 2); k = 2; 3; : : :
with starting values

(0) = 1; (1) = 1=(1 2)

It is such that (k) ! 0 as k ! 1. If the roots of the AR polynomial

are complex the ACF describes a damped cosine wave.

PACF: It has a cut-o (i.e. it's equal to zero) after k = 2 since

yt E[ytj(yt 1; ; yt k+1)] = "t; k > 2:

Figure 5: ACF and PACF of an AR(2) process
AR(p) processes: The AR(p) process is generated by the equation

yt = m + 1yt 1 + 2yt 2 + + pyt p + "t; "t WN( 2)

(L)yt = m + "t; (L) = 1 p

1L pL :

yt is stationary if the p roots of (L) are outside the unit circle.

E(yt) = = m= (1), where (1) = 1 1 p.

The demeaned process is (L)(yt ) = "t

The Autocovariance Function is
(k) = 1 (k 1) + + p (k p); for k > 0
(k) = 1 (k 1) + + p (k p) + 2; for k = 0

ACF is given by the Yule-Walker system of equations:

(k) = 1 (k 1) + 2 (k 2) + + p (k p); k = 1; 2; : : : ; p

PACF: It has a cut-o after k = p since

yt E[ytj(yt 1; ; yt k+1)] = "t; k > p:

3.2 Moving Average (MA) processes

In the Wold representation set j = j; j q and j = 0; j > q . This

gives the MA(q ) process

yt = + "t + 1 "t 1 + 2 " t 2 + + q "t q

where "t WN( 2).

P
Stationarity: Since the condition j j j j < 1 holds, the MA(q ) process
is always stationary and ergodic.
MA(1) processes: The MA(1) process is generated by the equation

yt = + "t + " t 1 = + (1 + L)"t

The moments are obtained as follows
E(yt) = + E("t) + E("t 1) =
(0) = E(yt )2 = E("t + "t 1)2
= E("2t ) + 2 E("t"t 1) + 2E("2t 1) = 2(1 + 2)
(1) = E[(yt )(yt 1 )]
= E[("t + "t 1)("t 1 + "t 2)] = 2
(k) = E[(yt )(yt k )]
= E[("t + "t 1)("t k + "t k 1)] = 0; k > 1
 ACF has a cuto at k = 1:
(0) = 1
(1) = 1+ 2
(k ) = 0 ; k>1

Invertibility: yt MA(1) is invertible if j j < 1. Consider the process

y~t = + "t + ~"t 1

with ~ = 1= and "t WN(~ 2).
The process y~t has the same moments , (0) and (1), as

yt = + "t + " t 1
with 2 = ~2 ~ 2. Hence, (1) = 1 =(1 + 2) = =(1 + 2) in both cases.
The two processes have identical properties and cannot be discriminated
from the rst two moments. This problem is known as non-identi ability
and is remedied upon by constraining in the interval (-1,+1).

The term invertibility stems from the possibility of rewriting the process as
an in nite autoregression, AR(1), with coe cients j that are convergent:
1
X
yt + 1yt 1 + 2yt 2 + + k yt k + = m + "t ; j jj < 1
j=1

The sequence of weights j = ( )j converges if and only if j j < 1.

PACF: Since an invertible MA(1) process can be rewritten as AR(1),

its PACF has no cuto but it decays exponentially.
Figure 6: ACF and PACF of an MA(1) process
MA(q ) processes: The MA(q ) process is generated by the equation
yt = + "t + 1 "t 1 + + q "t q
is invertible if the roots of (L) = 0 are outside the unit circle.
The moments are obtained as follows
E(yt) =
(0) = E(yt )2 = E("t + 1"t 1 + + q "t q )2
= 2(1 + 12 + + q2)
(k) = E[("t + 1"t 1 + + q "t q )("t k + 1"t k 1 + + q "t k q )]
= 2( k + k+1 1 + + q k q)
(k) = 0; k > q

ACF has a cuto at k = q .

PACF has no cuto , it is similar as the ACF of an AR(q ) process.

3.3 ARMA processes

ARMA(p; q ) processes: The ARMA(p; q ) process is generated by the equa-

tion
(L)yt = m + (L)"t
where

(L) = 1 p
1L pL ;
(L) = 1 + 1L + + q
qL

Remark: the polynomials (L) and (L) should have no common roots,
otherwise a reduction of the orders p; q would occur after cancelling the
common roots.
Stationarity: yt is stationary if the roots of the AR polynomial (L) lie
outside the unit circle.
Stationarity implies that yt can be written as an MA(1) process with
declining coe cients:

yt = (1) 1m + (L) 1 (L)"t = + (L)"t

Invertibility: yt is invertible if the roots of the MA polynomial (L) lie

outside the unit circle.
Invertibility implies that yt can be written as an AR(1) process with declin-
ing coe cients:

(L) 1 (L)(yt ) = (L)(yt ) = "t

ARMA(1; 1) processes: The ARMA(1; 1) process is generated by the equa-
tion
yt = m + yt 1 + "t + "t 1

The expected value is:

E(yt) = m + E(yt 1) = m + = m=(1 )

The demeaned process is:

(yt ) = (yt 1 ) + "t + " t 1

The autocovariance function is obtained as follows
(0) = E[(yt )yt] = E[ (yt 1 )yt + "tyt + "t 1yt]
= (1) + 2 + E f "t 1[ (yt 1 ) + "t + "t 1]g
= (1) + 2(1 + + 2)
(1) = E[(yt )yt 1] = E[ (yt 1 )yt 1 + "tyt 1 + "t 1yt 1]
= (0) + 2
(k) = E[yt(yt k )] = (k 1); k > 1

Both ACF and PACF have no cuto !

Figure 7: ACF and PACF of an ARMA(1,1) process.
3.4 Outliers in ARMA models
Many economic and nancial time series display outlying observations,
which are due to speci c events (errors in the data, strikes, changes
in regulations, natural disasters, etc.).

The presence of these outliers may induce misspeci cation of the

ARMA model as well as biases in parameter estimation.

Outliers can be modelled as additional deterministic components to

the basic ARMA set-up. Their treatment leads to ARMA models with
exogenous variables (ARMAX) that are impulse dummies:
(
1 if =t
It ( ) =
0 if 6= t
where is the time in which the event generating the outlier occurs.
Additive Outlier (AO):
(L)
yt = + "t + !It( ); ! 2 R
(L)
An AO shows up at time only with magnitude ! , subsequent obser-
vations are una ected.

Innovational Outlier (IO):

(L)
yt = + [!It( ) + "t];
(L)
An IO is characterized by having an impact of magnitude h! at time
+ h for h = 0; 1; 2; :::
Transient Change (TC):
(L) !
yt = + "t + It( ); j j < 1
(L) 1 L
A TC is characterized by having an impact of magnitude h! at time
+ h. Asymptotically, E(y +h) returns to .

Level Shift (LS):

(L) !
yt = + "t + It( );
(L) 1 L
A LS implies that E(y +h) move permanently to ( + ! ). In contrast
to a TC, a LS a ects all the subsequent observations forever.
3.5 Forecasting from ARMA Models

Let yt ARMA(p; q ) and It = fyt; yt 1; g. The best linear unbiased

predictor of yt+h is given by:
yt(h) = E(yt+hjIt); h = 1; 2; :::;
where E(yt+hjIt) is the expected value of yt+h conditional to It, which is
the called the natural ltration of the process yt.

From the expression

yt+h = m + 1yt+h 1 + + pyt+h p + "t+h + 1"t+h 1 + + q "t+h q
we get
yt(h) = m + 1E(yt+h 1jIt) + + pE(yt+h pjIt)
+ E("t+hjIt) + 1E("t+h 1jIt) + + q E("t+h q jIt)
It is then possible to recursively compute the optimal h step ahead pre-
dictor yt(h) given that
(
yt+h i; i h
E(yt+h ijIt) =
yt(h i); i < h
(
"t+h i; i h
E("t+h ijIt) =
0; i < h

Example: Let assume that yt ARMA(1; 1), we get

yt(h) = m + E(yt+h 1jIt) + E("t+hjIt) + E("t+h 1jIt);

which implies
yt(1) = m + yt + "t
yt(h) = m + yt(h 1) = m(1 + + + h 2) + h 1yt(1)
= m(1 + + + h 1) + hyt + h 1 "t; h > 1
Since any ARMA(p; q ) admits the Wold representation yt = + (L)"t,
where (L) = (L)= (L), we can rewrite h step ahead predictor as
P 1 P1 P1
yt(h) = E( h "
j=0 j t+h j + " jI
j=h j t+h j t ) = j=h j "t+h j
Hence, the h step ahead prediction error is
Ph 1
"t(h) = yt+h yt(h) = j=0 j "t+h j
Since "t(h) MA(h 1), we have that
E["t(h)] = 0;
2 (h) Var["t(h)] = 2 Ph 1 2 ;
j=0 j
We note that 2(h) is a non-decreasing function of h such that

lim 2 (h) = (0)

h!1
When "t is a Gaussian white-noise, it follows that

"t(h)= (h) N (0; 1)

since "t(h) is a linear combination of i.i.d. N (0; 2) random variables.
Hence, the 100(1 )% con dence interval for yt+h is

yt(h) z =2 (h) < yt+h < yt(h) + z =2 (h)

Remark: When the model parameters are estimated, the above formula
underestimates the true sample variability.
3.6 Forecasting the random walk

In order to compute the h step ahead prediction of the RW (with drift),

it is convenient to rely on the expression:

yt+h = mh + yt + "t+h + "t+h 1 + + "t+1;

which is easily obtained by recursive substitutions.
From the above equation we get:

yt(h) = mh + yt
"t(h) = "t+h + "t+h 1 + + "t+1
2 (h) = h 2

Remark: lim 2 (h) = 1 ) it is less and less likely to nd yt+h close to

h!1
yt(h) as the forecasting horizion h increases.
4 Nonstationary processes

Integrated processes: An ARMA process is integrated of order d, denoted

as yt I (d) or yt ARIMA(p; d; q ), if
(L) dyt = m + (L)"t;
where = 1 L; and the roots of both (L) and (L) lie outside the
unit circle.
The simplest I (1) process is the RW, i.e the ARIMA(0; 1; 0):
yt = m + "t
The simplest I (2) process is the Integrated Random Walk, i.e the ARIMA(0; 2; 0):
2y = (1 2L + L2)yt = "t;
t
which can be rewritten by recursive substitutions as:
Pt 1Pt 1
yt = y0 + y0t + i=0 j=i "t j
4.1 The Beveridge-Nelson decomposition

Beveridge and Nelson (1981) proved that any ARIMA(p; 1; q ) process can
be decomposed as the sum of a RW (with drift) and an I(0) process.
Indeed, by expanding the polynomial (L) on 1 we get
(L) = (1) + (1 L) (L)
1
P 1
P
where (L) = j LJ is such that j = i and lim j j j = 0.
j=0 i>j j!1
Hence, we can rewrite the Wold representation as
yt =
+ (1)"t + (1 L) (L)"t;
By cumulating both sides of the above equation we nally have
tX1
yt = y0 + t + (1)"t j + (L)" ;
| {z }t
j=0 I(0) = "cycle"
| {z }
RW with drift = "stochastic trend"
4.2 Seasonality

Seasonal Di erences:

s =1 Ls = (1 L)(1 + L + + Ls 1)
where s denotes the number of seasons in an year. Usually, s = 4 or 12.

Seasonal Integration (s = 4):

4 = (1 L)(1 + L + L2 + L3) = (1 L)(1 + L)(1 + L2);

(1 + L2) = (1 iL)(1 + iL);
where i = ( 1)1=2. Hence, 4 has 4 roots on the unit circle. A similar
result applies to the monthly case (s = 12) as well.
Let us write f! (t) = a cos(t! ) + b sin(t! ), a function with period P =
2 =! , where ! 2 [0; ]. Then we have
(1 L)f0(t) = (1 L)[a cos(t0)] =0
(1 + L)f (t) = (1 + L)[a cos(t )] =0
(1 + L2)f =2(t) = (1 + L2)[a cos(t =2) + b sin(t =2)] = 0
Hence, we de ne

yt I (1) i (1 + L)yt I (0)

yt I =2(1) i (1 + L2)yt I (0)
yt I0(1) i (1 L)yt I (0)
A process yt such that
(1 L4)yt I (0);
is then I (1) at both 0 and seasonal frequencies ; =2. It is denoted
yt SI (1). A similar de nition applies to the monthly case as well.
Seasonal Processes: The most popular seasonal generalization of ARIMA
models leads to the following ARIMA(p; d; q ) (P; D; Q)s model:

(L) (Ls) d D
s yt = + (L) (Ls)"t;
where (Ls) = 1 s 2s P s is the seasonal AR
1L 2L PL
polynomial in Ls with order P , and (Ls) = 1 + 1Ls + 2L2s + +
Qs is the seasonal MA polynomial with order Q.
QL

An important special case is the Airline model ARIMA(0; 1; 1) (0; 1; 1)s:

(1 L)(1 Ls)yt = (1 + L)(1 + Ls)"t;

with j j < 1; j j < 1.
The autocovariance function of the Airline model is
(0) = (1 + 2)(1 + 2) 2
(1) = (1 + 2) 2
(k) = 0 for k = 2; : : : ; s 2
(s 1) = 2
(s) = (1 + 2) 2
(s + 1) = 2
(k) = 0 for k > s + 1
4.3 The Box-Jenkins Approach

1. Identi cation of the orders p; d; q (and P; D; Q for seasonal models).

The integration order d (and D for seasonal models) is determined

rst. In the past, this was basically done by graphical methods.
Nowadays, we rely on unit roots tests, which we will see later.

In the past, The MA and AR orders p; q (and P; Q for seasonal

models) were determined from the analysis of the correlogram.
Nowadays, we rely on information criteria, which we will see soon.
2. Estimation of the parameters

Conditional maximum likelihood (CML). Given a sample of T ob-

servations, the initial values (y1; :::yp) and ("1; :::"q ) are treated as
xed ) they do not enter in the likelihood function.

Unconditional ML (UML). The initial values are treated as real-

izations of the stochastic process ) they enter in the likelihood
function.

In large samples, CML and UML are equivalent.

For pure AR processes, CML is the same as OLS.

3. Diagnostic checking

Signi cance tests for the parameters. t tests, F tests.

b(L)
Normality tests on residuals et = b yt. Jarque{Bera test; under
(L)
H0 : " t i:i:d:N (0; 2), the test statistic (based on the sample
3rd and 4th moments of the residuals) converges to a 2(2).

Autocorrelation tests on residuals. Ljung-Box test; under H0 : "t

W N , the test statistic
m
X
Q(m) = T (T + 2) (T k) 1 ^2e (k)
k=1
converges to a 2 (m p q ).

Goodness of t. Coe cient of determination.

Information criteria (IC): composed by (i) an inverse measure of t; (ii) an
increasing function of the number of parameters. Given a set of candidate
models, the preferred model is the one with the minimum IC ) best
compromise between t and parsimony.

In practice, estimate all the ARMA models whose orders are at maximum
pmax,qmax, then choose the ARMA(p ; q ) such that one of the following
is satis ed:
2 p+q
(p ; q ) = arg min AIC (p; q ) = ln ^ + 2 ;
T
p+q
(p ; q ) = arg min HQIC (p; q ) = ln ^ 2 + 2 ln ln T ;
T
p+q
(p ; q ) = arg min BIC (p; q ) = ln ^ 2 + ln T :
T
where AIC (HIC) [BIC] stands for Akaike (Hannan-Quinn) [Bayes] IC.
Remarks:

For T > 15, the penalty term of AIC (BIC) is the smallest (largest)
one ) for the sample sizes typically used in economics and nance,
AIC tends to choose less parsimonious models than BIC, HQIC is a
middle way.

BIC and HQIC asymptotically choose the true orders with probability
one, whereas AIC has a non-null probability of choosing an overpara-
meterized model even in the limit.

Notwithstanding the above, AIC is often preferred in empirical appli-

cations because the cost of underparameterization, i.e. lack of consis-
tency, is statistically larger than the one of overparameterization i.e.
lack of e ciency.
5 Unit-root tests

A stylized fact: Many (possibly log-transformed) economic and nancial

time series display a tendency to grow approximately linearly over time.

Nelson and Plosser (1982) contrast 2 candidate data generating processes:

Di erence-stationary (DS) processes: yt I(1) + drift.
Trend-stationary (TS) processes: yt I(0) + linear deterministic trend
Both are nested into the unobserved components (UC) process:

yt = + t + ut;
(L)ut = "t:
If (L) = (1 L) (L) and all the (p 1) roots of (L) lie outside the
unit circle ) yt is DS. Notice that in this case we have (1) = 0:
If all the roots of (L) lie outside the unit circle ) yt is TS.
Dickey-Fuller (DF) test: assume that ut is an AR(1) process. By premul-
tiplying both sides of the rst UC equation with (L) we get:

yt = [ + (1) ] + (1) t + yt 1 + "t; (1) = 1 ;

which we may reparametrize as

yt = ( ) +( ) t + yt 1 + " t ; =( 1);
| {z } | {z }

and perform a t test for H0 : = 0, i.e. = 1, vs. H1 : 2 ( 2; 0), i.e.

j j < 1.

Remark 1: Under H0, we have yt = + "t ) the slope annihilates.

Remark 2: The limit distribution of the t test statistic under H0 is no

longer a N (0; 1) since the CLT does not apply to I(1) processes.
DF test with no trend: assume that the data do not display a trending
behavior. Hence, = 0 in the UC process. The DF regression becomes

yt = + yt 1 + " t ; =( 1);
and we still perform a t test for H0 : = 0 vs. H1 : 2 ( 2; 0). Notice
that under H0 we have: yt = "t ) the drift is null.

DF test with no constant: assume that any deterministic term is present in

the data. With = = 0 in the UC process, the DF regression becomes

yt = yt 1 + "t; =( 1);
and we again perform a t test for H0 : = 0 vs. H1 : 2 ( 2; 0).

Remark: The distribution of the test statistic is not invariant to the deter-
ministic kernel. Each of the three tests has its own critical values. They
were tabulated by Fuller (1977).
Augmented Dickey-Fuller (ADF) test: assume that ut is an AR(p) process.
Rewriting the AR polynomial as

(L) = (1)L + y (L);

with
p
X
y (L) y y p 1; y
=1 1L ::: p 1 L j = i
i=j+1
and premultiplying both sides of the rst UC equation with (L) we get:
pX1
y
yt = ( + y(1) )+ ( )t+ yt 1 + j yt j +"t; = (1):
| {z } | {z }
j=1

Finally, we perform a t test for H0 : = 0 vs. H1 : < 0. Notice that

under H0 we have: y(L) yt = y(1) + "t ) the slope annihilates.
The limit distribution of the test statistic is the same as the DF one.
6 Impulse response function and persistence mea-
sures

The impulse response function (IRF) is a standard tool in illustrating the

dynamic behavior of a time series model, e.g. yt ARIMA(p; 1; q ).

The IRF measures the e ect of an innovation occurring at time t, yt

E[ytjIt 1], on yt+h; h = 1; : : : ; 1.
@yt+h
IRF(h) = ; h = 0; 1; : : :
@"t

In a linear time series model the IRF is time invariant and a function of h
alone.
If yt I (1),
yt = + (L)"t;
the IRF is given by the deterministic rst order di erence equation:

IRF(h) = IRF(h 1) + h
with starting value IRF(0) = 1; thus,

IRF(h) = 1 + 1 + + h
(cumulation of psi-weights)
Measures of persistence: Let yt I (1), yt = + (L)"t.

Campbell and Mankiw (1987) measure

lim IRF(h) = (1) = (1)= (1)
h!1
Equivalently, we can interpret (1) as the revision in the long run
prediction of yt due to the occurrence of a unit shock at t ("t = 1):
lim [E(yt+hjIt) E(yt+hjIt 1)] = (1)"t;
h!1

Cochrane (1988) normalized variance ratio

V = [ (1) ]2= 0
It normalizes the variance of the innovations of the stochastic trend
with the unconditional variance of yt.