Notation and some Linear Algebra 1

14.384 Time Series Analysis, Fall 2007

Professor Anna Mikusheva
Paul Schrimpf, scribe
September 27, 2007
revised October 5, 2009

Lecture 11

VARs

Notation and some Linear Algebra

Let
p

yt = aj yt−j + et (1)
j=1

where yt and et are k × 1, and aj is k × k. et is white noise with Eet et = Ω and Eet es = 0
 p   p 
Lemma 1. yt is stationary if det Ik − j =1 aj z j = 0 for all |z | ≤ 1, i.e. all roots of det Ik − j=1 aj z j
are outside the unit circle.
Definition 2. Companion form of (1) is:
Yt = AYt−1 + Et ,
where
⎡ ⎤
⎡ ⎤ a1 a2 ... ap ⎡ ⎤
yt ⎢ ⎥ et
⎢ ⎥ ⎢ I 0 ... 0 ⎥ ⎢ 0 ⎥
⎢ yt−1 ⎥ ⎢ ⎥ ⎢ ⎥
Yt = ⎢ ⎥ ,A=⎢ 0 I 0 ... ⎥ , Et = ⎢ .. ⎥
⎣ ... ⎦ ⎢ .. .. .. ⎥ ⎣ . ⎦
⎣ . . . ⎦
yt−p+1 0
I 0
so Yt and Et are kp × 1 and A is kp × kp.
So, any VAR(p) can be wrote as a multi-dimensional VAR(1). From a companion form one can note that
ΣY = AΣY A + ΣE
This may help to calculate variance-covariance structure of VAR. In particular, we may use the following
formula from linear algebra:
vec(ABC) = (C  ⊗ A)vec(B),
here ⊗ stays for tensor ⎤ vec(A) transforms a matrix to a vector according to the following
⎡ product and
a11 a12

semantic rule: if A = ⎣ a21 a22 ⎦, then vec(A) = [a11 , a12 , a21 , a22 , a31 , a32 ]
a31 a32

vec(ΣY ) =vec (AΣY A + ΣE )

=(A ⊗ A)vec(ΣY ) + vec(ΣE )
⇒
−1
vec(ΣY ) = (I − (A ⊗ A)) vec(ΣE )

Cite as: Anna Mikusheva, course materials for 14.384 Time Series Analysis, Fall 2007. MIT OpenCourseWare (http://ocw.mit.edu),
Massachusetts Institute of Technology. Downloaded on [DD Month YYYY].
 Estimation 2

Estimation

Lemma 3. MLE (with normal error assumption)= OLS equation-by-equation with Ω̂ = 1
T êt eˆt
Intuition: all variables are included in all equations, so there is nothing gained by doing SUR. This also
implies that OLS equation by equation is asymptotically efficient. The usual statements of consistency and
asymptotic normality hold, as well as OLS formulas for standard errors.

Granger Causality
Granger Causality is a misleading name. It would be better called Granger predictability.

Definition 4. y fails to Granger cause x if it’s not helpful in linear predicting x (in MSE sense). More
formally,
 
MSE Ê(xt+s |xt , xt−1 , ...) = MSE Ê(xt+s |xt , ..., yt , yt−1 , ...) , ∀s > 0

where Ê(xt |z) denotes the best linear prediction of xt given z

A test of Granger causality is to run OLS:

xt = α1 xt−1 + ... + αp xt−p + β1 yt−1 + ... + βp yt−p + et

and test H0 : β1 = β2 = ... = 0.

Note that:
• Granger causality is not related to economic causality, it’s more about predictability.

• There could be simultaneous casualty or omitted variable problems. For example, there may be a
variable z that causes both y and x but with the different lag(sluggish response). If one does not
include z (omitted variable), it may look like x causes y.
• Forward looking (rational) expectations may even reverse the causality. For example, suppose analysts
rationally predict that a stock is going to pay high dividends tomorrow. That will provoke people
to buy the stock today, and the price will rise. In the data you would observe that the price rise is
followed by high dividends. So, we would find that prices Granger cause dividends, even though it was
really that anticipated high dividends caused high prices. Or increase in orange juice price Granger
causes bad weather in Florida.

How to do Granger causality in multivariate case?

Assume y1t is k1 × 1 vector and y2t is k2 × 1. Assume that we have VAR system
     
y1t A1 (L)y1t−1 + A2 y2t−1 e1t
= +
y2t B1 (L)y1t−1 + B2 y2t−1 e2t

Group of variables y2 fails to Granger cause y1 if A2 = 0. To perform this test we have to run unrestricted
regression y1t = A1 (L)y1t−1 +A2 y2t−1 +eut and restricted regression y1t = A1 (L)y1t−1 +ert . Then we estimate
T  T 
the corresponding variance-covariance matrix Ωu = T1 t=1 etu etu and Ωr = T1 t=1 ert ert . The test statistic
compares these matrices:
LR = T (log |Ωr | − log |Ωr |)
Under the null (absence of Granger Causality) LR statistic is asymptotically χ2 with the degrees of freedom
equal to the number of restrictions imposed.

Cite as: Anna Mikusheva, course materials for 14.384 Time Series Analysis, Fall 2007. MIT OpenCourseWare (http://ocw.mit.edu),
Massachusetts Institute of Technology. Downloaded on [DD Month YYYY].
 Reporting Results 3

Figure 1: Impulse-response function

Impulse Response

2.5

Rep
2 orting Results

Reporting the matrix of coefficients is not very informative. There are too many of them, and the coeffi-
Amplitude

cients 1.5
are difficult to interprete anyway. Instead, people present impulse-response functions and variance
decompositions.

Impulse-resp
1 onse
Suppose
0.5 yt = a(L)yt−1 + et

with MA representation
0
yt = c(L)et
0 5 10 15 20 25
and Dut = et such that ut are orthonormal, i.e. Eu u = I.
Samples
t t Let c̃(L) = c(L)D, so

yt = c̃(L)ut

∂yti
Definition 5. The impulse-response function is k
∂ut−j
. It is the change in yti in response to a unit change
in ukt−j holding all other shocks constant. We can plot the impulse-response function as in figure 1.
To estimate an impulse-response, we would

1. Estimate VAR by OLS – â

2. Invert to MA
3. Find and apply rotation D to get orthonormal shocks – the impulse response is given by ĉ˜

Cite as: Anna Mikusheva, course materials for 14.384 Time Series Analysis, Fall 2007. MIT OpenCourseWare (http://ocw.mit.edu),
Massachusetts Institute of Technology. Downloaded on [DD Month YYYY].
 Impulse-response 4

Standard Errors
Delta-method To calculate standard errors, we can apply the delta-method to â – the ĉ˜ are just some
complicated function of â. In practice, we can do this recursively:
yt =a1 yt−1 + ... + ap yt−p + et
=a1 (a1 yt−2 + ... + ap yt−p−1 + et−1 ) + a2 yt−2 + ... + yt−p + et
..
.
so, c1 = a1 , c2 = a2 + a21 , etc. We can apply the delta-method to each of these coefficients. We’d also need to
apply the delta-method to our estimate of D. Sometimes, this is done in practice. However, it is not really
the best way, for two reasons:
• We estimate many aj from not all that big of a sample, so our asymptotics may not be very good.
• This is made even worse by the fact that ck are highly non-linear transformations of aj
Instead of the delta-method, we can use the bootstrap.

Bootstrap The typical construction of bootstrap confidence sets would be the following:
ˆ
1. run regression yt = c + a1 yt−1 + ... + ap yt−p + et to get c, a1 , ..., 
ap and residuals et
2. Invert the estimated AR process to get the estimates of impulse response ĉj from 
a1 , ..., 
ap
3. For b = 1..B
∗
(a) Form yt,b =
c+ ap yt∗−p,b + e∗t,b , where e∗t,b is sampled randomly with replacement
a1 yt∗−1,b + ... + 
from {êt }
∗
(b) Run regression yt,b = c + a1 yt∗−1,b + ... + ap yt∗−p,b + et to get  a∗p,b
a∗1,b , ..., 
(c) Invert the estimated AR process to get the estimates of impulse response ĉ∗j,b from  a∗1,b , ..., 
a∗p,b
4. sort ĉ∗j,b in ascending order ĉ∗j,(1) ≤ ... ≤ ĉ∗j,(B)
5. Form α confidence interval.
There are at least three way’s of forming a confidence set:
cj as a test statistics. The interval is [ĉ∗j,([Bα/2]) , ĉ∗j,([B(1−α/2)]) ]
1. Efron’s interval (percentile bootstrap): uses 

2. Hall’s interval (“turned around” bootstrap) : uses  cj − cj as a test statistics. It employs the idea of
bias correction. The interval is a solution to inequalities
ĉ∗j,([Bα/2]) −  cj − cj ≤ ĉ∗j,([B(1−α/2)]) − 
cj ≤  cj
cj − ĉ∗j,([B(1−α/2)]) , 2
or [2 cj − cˆ∗j,([Bα/2]) ]

cj −cj
3. studentized bootstrap : uses t-statistics statistictj = s.e.(
cj ) . The interval is a solution to inequalities

cj − c j

t∗j,([Bα/2]) ≤ ≤ t∗j,([B(1−α/2)])
s.e.(cj )
cj − t∗j,([B(1−α/2)]) s.e.(
or [ cj − t∗j,([Bα/2]) s.e.(
cj ),  cj )]
Remark 6. The bootstrap is still an asymptotic procedure. One advantage of the bootstrap is its simplicity.
There is no need to apply the delta-method.
Remark 7. There are variations of the bootstrap that also work. For example, you could sample the errors
ˆ This would be called a parametric bootstrap because we’d be
from a normal distribution with variance Ω.
relying on a parametric assumption to create our simulated samples.

Cite as: Anna Mikusheva, course materials for 14.384 Time Series Analysis, Fall 2007. MIT OpenCourseWare (http://ocw.mit.edu),
Massachusetts Institute of Technology. Downloaded on [DD Month YYYY].
 Impulse-response 5

Bootstrap-after-bootstrap Simulations show that bootstrap works for impulse responses somewhat bet-
ter than asymptotic (delta-method). This is due to final sample correction - remember that the dependence
between {aj } and {cj } is non-linear. However, the coverage of these intervals is still very far from ideal,
especially for very persistent processes. The main reason for that is aj are very biased estimates of aj . To
correct this a bootstrap-after-bootstrap was suggested.

1. run regression yt = c + a1 yt−1 + ... + ap yt−p + et to get c,

ˆ ap and residuals et
a1 , ..., 
2. Invert the estimated AR process to get the estimates of impulse response ĉj from 
a1 , ..., 
ap

3. For b = 1..B
∗
(a) Form yt,b =
c+ ap yt∗−p,b + e∗t,b , where e∗t,b is sampled randomly with replacement
a1 yt∗−1,b + ... + 
from {êt }
∗
(b) Run regression yt,b = c + a1 yt∗−1,b + ... + ap yt∗−p,b + et to get 
a∗1,b , ..., 
a∗p,b
B
4. calculate bias corrected estimates of aj : 
aj = 2
aj − 1
B a∗j,b
b=1 

5. For b = 1..B
∗
(a) Form yt,b a1 yt∗−1,b + ... + 
= ap yt∗−p,b + e∗t,b , where e∗t,b is sampled randomly with replacement from
{êt }
∗
(b) Run regression yt,b = c + a1 yt∗−1,b + ... + ap yt∗−p,b + et to get 
a∗1,b , ..., 
a∗p,b
(c) Invert the estimated AR process to get the estimates of impulse response ĉ∗j,b from 
a∗1,b , ..., 
a∗p,b
6. Form α confidence interval.

Cite as: Anna Mikusheva, course materials for 14.384 Time Series Analysis, Fall 2007. MIT OpenCourseWare (http://ocw.mit.edu),
Massachusetts Institute of Technology. Downloaded on [DD Month YYYY].
MIT OpenCourseWare
http://ocw.mit.edu

14.384 Time Series Analysis

Fall 2013

For information about citing these materials or our Terms of Use, visit: http://ocw.mit.edu/terms.