Spectral Analysis

Jesus Fernandez-Villaverde
University of Pennsylvania

1
 Why Spectral Analysis?

We want to develop a theory to obtain the business cycle properties

of the data. Burns and Mitchell (1946).

General problem of signal extraction.

We will use some basic results in spectral (or harmonic) analysis.

Then, we will develop the idea of a lter.

2
 Riesz-Fisher Theorem

First we will show that there is an intimate link between L2 [ ; ]

and l2 ( 1; 1).

Why? Because we want to represent a time series in two di erent

ways.

Riesz-Fischer Theorem: Let fcng1 n= 1 2 l2 ( 1; 1). Then, there

exist a function f (! ) 2 L2 [ ; ] such that:
1
X
f (! ) = cj e i!j
j= 1

This function is called the Fourier Transform of the series.

3
 Properties

Its nite approximations converge to the in nite series in the mean

square norm:
Z n 2
X
lim cj e i!j f (! ) d! = 0
n!1
j= n

It satis es the Parseval's Relation:

Z 1
X
jf (! )j2 d! = jcnj2
n= 1

Inversion formula:
Z
1
ck = f (! ) ei!k d!
2

4
 Harmonics

n oj=1
The functions e i!j are called harmonics and constitute an
j= 1
orthonormal base in L2 [ ; ].

The orthonormality follows directly from:

Z
e i!j ei!k d! = 0

if j 6= k and
Z
e i!j ei!j d! = 1

The fact that they constitute a base is given by the second theorem
that goes in the opposite direction than Riesz-Fischer: given any func-
tion in L2 [ ; ] we can nd an associated sequence in l2 ( 1; 1).
5
 Converse Riesz-Fischer Theorem

Let f (! ) 2 L2 [ ; ]. Then there exist a sequence fcng1

n= 1 2 l2 ( 1; 1)
such that:
1
X
f (! ) = cj e i!j
j= 1
where:
Z
1
ck = f (! ) ei!k d!
2
and nite approximations converge to the in nite series in the mean square
norm:
Z n 2
X
lim cj e i!j f (! ) d! = 0
n!1
j= n

6
 Remarks I

.
The Riesz-Fisher theorem and its converse assure then that the Fourier
transform is an bijective from l2 ( 1; 1) into L2 [ ; ].

The mapping is isometric isomorphism since it preserves linearity and

distance: for any two series fxng1 n= 1 , fyn g1
n= 1 2 l2 ( 1; 1)
with Fourier transforms x (! ) and y (! ) we have:
1
X
x (! ) + y (! ) = (xn + yn) ei!j
j= 1
X1
x (! ) = xnei!j
j= 1
0 11
Z 1 1
X 2
1 2
jx (! ) y (! )j2 d! =@ jxk yk j2A
2 j= 1

7
 Remarks II

.
There is a limit in the amount of information in a series.

We either have concentration in the original series or concentration in

the Fourier transform.

This limit is given by:

0 1
1
X Z 1
2 1 4
@ j 2 cj A ! 2 jf (! )j2 d! cj
j= 1 1 4

This inequality is called the Robertson-Schr•

odinger relation.

8
 Stochastic Process

X fXt : ! Rm, m 2 N, t = 1; 2; :::g is a stochastic process de-

ned on a complete probability space ( ; =; P ) where:

1. = Rm 1 limT !1 T Rm
t=0

2. = limT !1 =T limT !1 T t=0 B (R m) B R m 1 is just

the Borel -algebra generated by the measurable nite-dimensional
product cylinders.

3. P T (B ) P (B ) j=T P Y T 2 B , 8B 2 =T .

0
De ne a T segment as XT X1; :::; XT with X 0
0 0 = f;g and a
0
realization of that segment as xT x01; :::; x0T .
9
 Moments

Moments, if they exist, can be de ned in the usual way:

Z
t = Ext = xtdP T
Rm
Z
tj = Ext xt j = (xt ) xt j dP T
Rm (j+1)

If both t and tj are independent of t for all j , X is a covariance-

stationary or weakly stationary process.

Now, let us deal with X : covariance-stationary process with zero mean

(the process can be always renormalized to satisfy this requirement).
10
 z Transform
P1
If j= 1 j < 1, de ne the operator gx : C ! C :
1
X
gx (z ) = j
jz
j= 1
where C is the complex set.

This mapping is known as the autocovariance generating function.

Dividing this function by 2 and evaluating it at e i! (where ! is a

real scalar), we get the spectrum (or power spectrum) of the process
xt :
1
1 X i!j
sx (! ) = je
2 j= 1

11
 Spectrum

The spectrum is the Fourier Transform of the covariances of the

process, divided by 2 to normalizes the integral of sx ( ) to 1.

The spectrum and the autocovariances are equivalent: there is no

information in one that is not presented in other.

That does not mean having two representations of the same informa-
tion is useless: some characteristics of the series, as its serial correla-
tion are easier to grasp with the autocovariances while others as its
unobserved components (as the di erent uctuations that compose
the series) are much easier to understand in the spectrum.

12
 Example

General ARMA model: (L) xt = (L) "t.

P . (1) representation of xt in the lag
Since xt = 1
j=0 j " t j ; an M A
operator (L) and variance 2 satis es:

gx (L) = j (L)j2 = (L) L 1 2

Wold's theorem assures us that we can write any stationary ARMA as

an M A (1) and then since we can always make:

(L) = (L) (L)

we will have
(L) L 1
gx (L) = (L) L 1 2 = 2
(L) L 1
and then the spectrum is given by:
2 e i!j ei!j
sx (! ) = e i!j
2 e i!j ei!j

13
 Working on the Spectrum

Since j = j: .
0 1
1
X 1
X h i
1 i!j 1 @ i!j i!j A
sx (! ) = je = 0+ j e +e
2 j= 1 2 j=1
0 1
1
X
1 @ A
= 0+2 j cos (!j )
2 j=1
iz iz
by Euler's Formula ( e +e2 = cos z )

Hence:
1. The spectrum is always real-valued.
2. It is the sum of an in nite number of cosines.
3. Since cos (! ) = cos ( ! ) = cos (! + 2 k), k = 0; 1; 2; :::, the
spectrum is symmetric around 0 and all the relevant information is
concentrated in the interval [0; ].
14
 Spectral Density Function

.
Sometimes the autocovariance generating function is replaced by the
autocorrelation generating function (where every term is divide by 0).

Then, we get the spectral density function:

0 1
1
X
1 @
s0x (! ) = 1+2 j cos (!j )
A
2 j=1
where j = j = 0.

15
 Properties of Periodic Functions

.
Take the modi ed cosine function:

yj = A cos (!j )

! (measured in radians): angular frequency (or harmonic or simply

the frequency ).

2 =! : period or whole cycle of the function.

A: amplitude or range of the function.

: phase or how much the function is translated in the horizontal axis.

16
 Di erent Periods

.
! = 0, the period of uctuation is in nite, i.e. the frequency associ-
ated with a trend (stochastic or deterministic).

! = (the Nyquist frequency), the period is 2 units of time, the

minimal possible observation of a uctuation.

Business cycle uctuations: usually de ned as uctuations between 6

and 32 quarters. Hence, the frequencies of interest are those comprised
between =3 and =16.

We can have cycles with higher frequency than . When sampling in

not continuous these higher frequencies will be imputed to the fre-
quencies between 0 and . This phenomenon is known as aliasing.
17
 Inversion
Using the inversion formula, we can nd all the covariances from the
spectrum: .
Z Z
sx (! ) ei!j d! = sx (! ) cos (!j ) d! = j

When j = 0, the variance of the series is:

Z
sx (! ) d! = 0

Alternative interpretation of the spectrum: integral between [ ; ] of

it is the variance of the series.

In general, for some ! 1 and and ! 2 :

Z !
2
sx (! ) d!
!1
is the variance associated with frequencies in the [! 1; ! 2]

Intuitive interpretation: decomposition of the variances of the process.

18
 Spectral Representation Theorem

Any stationary process can be written in its Cramer's Representation:

Z . Z
xt = u (! ) cos !td! + v (! ) sin !td!
0 0

A more general representation is given by:

Z
xt = (! ) d!

where ( ) is any function that satis es an orthogonality condition:

Z
(! 1) (! 2)0 d! 1d! 2 = 0 for ! 1 6= ! 2

19
 Relating Two Time Series

Take a zero mean, covariance stationary random process Y with real-

. process X , with realization xt:
ization yt and project it against the

yt = B (L) xt + "t
where
1
X
B (L) = bj Lj
j= 1
and Ext"t = 0 for all j .

Adapting our previous notation of covariances to distinguish between

di erent stationary processes we de ne:
l = Eltlt j
j

20
 Thus:
0 10 1
1
X 1
X
ytyt j = @ bs x t s A @ b r xt j r A
s= 1 r= 1
0 1 0 1
1
X 1
X
+@ b s xt s A "t j + @ br xt j r A "t + "t"t j
s= 1 r= 1

Taking expected values of both sides, the orthogonality principle im-

plies that:
1
X 1
X
y
j = bsbr xj+r s + "j
s= 1 r= 1

With these covariances, the computation of the spectrum of y is direct:

0 1
1 1 1
1 X y i!j 1 @ X x i!j
X
" e i!j A
sy (! ) = e = b r b s j+r s e + j
2 j= 1 j 2 j;s;r= 1 j= 1
21
 If we de ne h = j + r s:

e i!j = e i!(h r+s) = e i!he i!sei!r

we get:
1
X 1
X X1 X1
1 1
sy (! ) = br ei!r bse i!s x e i!h +
h
" e i!j
j
r= 1 s= 1 2 r= 1 2 j= 1
= B ei!r B e i!s sx (! ) + s" (! )

Using the symmetry of the spectrum:

2
sy (! ) = B e i! sx (! ) + s" (! )

22
 LTI Filters

Given a process X , a linear time-invariant lter (LTI- lter) is a op-

erator from the space of sequences into itself that generates a new
process Y of the form: .

yt = B (L) xt

Since the transformation is deterministic, s" (! ) = 0 and we get:

2
sy (! ) = B e i! sx (! )

B e i! : frequency transform or the frequency response function is

the Fourier transform of the coe cients of the lag operator.

G (! ) = B e i! is the gain (its modulus).

2 2
B e i! is the power transfer function (since B e i! is a
quadratic form, it is a real function). It indicates how much the spec-
trum of the series is changed at each particular frequency.
23
 Gain and Phase

The de nition of gain implies that the ltered series has zero variance
P i0 = P1
at ! = 0 if and only if B e i0 .= 1 j= 1 jb e j= 1 bj = 0.

Since the gain is a complex mapping, we can write:

B e i! = B a (! ) + iB b (! )

where B a (! ) and B b are both real functions.

Then we de ne the phase of the lter

!
B b (! )
(! ) = tan 1
B a (! )
The phase measures how much the series changes its position with
respect to time when the lter is applied. For a given (! ), the lter
shifts the series by (! ) =! time units.

24
 Symmetric Filters I

A lter is symmetric if bj = b j :
.
1
X
B (L) = b0 + bj Lj + L j
j=1

Symmetric lters are important for two reasons:

1. They do not induce a phase shift since the Fourier transform of

B (L) will always be a real function.

2. Corners of the a T segment of the series are di cult to deal

with since the lag operator can only be applied to one side.

25
 Symmetric Filters II
P1
If j= 1 bj = 0, symmetric lters can remove trend frequency compo-
nents, either deterministic or stochastic up to second order (a quadratic
deterministic trend or a double unit root):
.
1
X 1
X 1
X
bj Lj = bj Lj +L j 2 bj Lj
j= 1 j=1 j= 1
X1 1
X
= bj Lj + L j 2 = bj 1 Lj 1 L j
j=1 j=1
1
X jX1
= (1 L) 1 L 1 bj (k jhj) Lh
j=1 h= j+1
= (1 L) 1 L 1 B 0 (L)

where we used 1 Lj = (1 L) 1 + L + ::: + Lj 1 and

1
X jX1
(1 + L + :::) 1 + L 1 + ::: = bj (k jhj) Lh
j=1 h= j+1
if the sum is well de ned.
26
 Ideal Filters

An ideal lter is an operator B (L) such that the new process Y only
.
has positive spectrum in some speci ed part of the domain.

Example: a band-pass lter for the interval f(a; b) [ ( b; a)g 2

( ; ) and 0 < a b , we need to choose B e i! such
that:
(
= 1; for ! 2 (a; b) [ ( b; a) ;
B e i! =
= 0 otherwise
Since a > 0, this de nition implies that B (0) = 0. Thus, a band-pass
lter shuts o all the frequencies outside the region (a; b) or ( b; a)
and leaves a new process that only uctuates in the area of interest.

27
 How Do We Build an Ideal Filter?

P1 2
Since X is a zero mean, covariance stationary process, n= 1 jxn j <
.
1, the Riesz-Fischer Theorem holds.

If we make f (! ) = B e i! , we can use the inversion formula to set

the Bj :
Z
1
bj = B e i! ei!j d!
2

Substituting B e i! by its value:

Z Z Z b !
1 1 a
bj = B e i! ei!j d! = ei!j d! + ei!j d!
2 2 b a
Z
1 b i!j
= e + e i!j d!
2 a
R a i!j Rb i!j d! .
where the second step just follows from b e d! = ae

28
 Building an Ideal Filter

Now, using again Euler's Formula,

Z b
1 1 b sin jb sin ja
bj = 2 cos !jd! = .
sin !jja = 8 j 2 N n f0g
2 a j j

Since sin z = sin ( z ),

sin ( jb) sin ( ja) sin jb + sin ja sin jb sin ja
b j= = = = bj
j j j

Also:
Z
1 b b a
b0 = 2 cos ! 0d! =
2 a
and we have all the coe cients to write:
0 1
1
X
b a sin jb sin ja
yt = @ + Lj + L j A xt
j=1 j

The coe cients bj converge to zero at a su cient rate to make the

sum well de ned in the reals.
29
 Building an Low-Pass Filter

Analogously, a low-pass lter allows only the low frequencies (between

.
( b; b)), implying a choice of B e i! such that:
(
= 1; for ! 2 ( b; b) ;
B e i! =
= 0 otherwise

Just set a = 0.
sin jb
bj = b j = 8 j 2 N n f0g
j
b
b0 =
0 1
1
X
b sin jb j
yt = @ + L + L j A xt
j=1 j

30
 Building an High-Pass Filter

Finally a high-pass allows only the high frequencies:

( .
i! = 1; for ! 2 (b; ) [ ( ; b) ;
B e =
= 0 otherwise

This is just the complement of a low-pass ( b; b):

0 1
1
X
b sin jb j
yt = @1 L + L j A xt
j=1 j

31
 Finite Sample Approximations

With real, nite, data, it is not possible to apply any of the previous
formulae since they require an in. nite amount of observations.

Finite sample approximations are this required.

We will study two approximations:

1. the Hodrick-Prescott lter
2. the Baxter-King lters.

We will be concern with the minimization of several problems:

1. Leakage: the lter passes frequencies that it was designed to elim-
inate.
2. Compression: the lter is less than one at the desired frequencies.
3. Exacerbation: the lter is more than one at the desired frequencies.
32
 HP Filter

Hodrick-Prescott (HP) lter:

1. It is a coarse and relatively uncontroversial procedure to represent
the behavior of macroeconomic variables and their comovements.
2. It provides a benchmark of regularities to evaluate the comparative
performance of di erent models.

Suppose that we have T observations of the stochastic process X ,

fxtgt=1
t=1 .

HP decomposes the observations into the sum of a trend component,

xtt and a cyclical component xct:
xt = xtt + xct

33
 Minimization Problem

How? Solve:
T
X TX1 h i2
t 2
min xt xt + xtt+1 xtt xtt xtt 1
xtt t=1 t=2

Intuition.

Meaning of :

1. = 0 )trivial solution (xtt = xt).

2. = 1 )linear trend.
34
 Matrix Notation

To compute the HP lter is easier to use matrix notation, and rewrite

minimization problem as:
t 0 0
min x x x xt + Ax t Axt
xt
0 0
where x = (x1; :::; xT ) , xt = t t
x1; :::; xT and:
0 1
1 2 1 0 0
B C
B 0 1 2 1 0 C
B ... ... ... ... . . . ... ... C
A=B
B
C
C
B ... ... ... 1 2 1 0 C
@ A
0 1 2 1
(T 2) T

35
 Solution

First order condition:

xt x + A0Axt = 0
or
1
xt = I + A 0 A x

I + A0A 1 is a sparse matrix (with density factor (5T 6) =T 2).

We can exploit this property.

36
 Properties I

To study the properties of the HP lter is however more convenient

.
to stick with the original notation.

We write the rst-order condition of the minimization problem as:

h i
0 = 2 xt xtt +2 xtt t
xt 1 t
xt 1 xt 2t
h i h i
4 xtt+1 xtt xtt t
xt 1 + 2 xtt+2 xtt+1 xtt+1 xtt

Now, grouping coe cients and using the lag operator L:

h i h i h i
xt = xtt + 1 2L + L2 2 L 1 2+L + L 2 2L 1 +1 xtt
h i
= L 2 4 L 1 + (6 + 1) 4 L+ L2
2 1 2
= 1+ (1 L) 1 L = F (L) xtt

37
 Properties II

De ne the operator C (L) as:

2
(1 L)2 1 L 1
C (L) = (F (L) 1) F (L) . 1 = 2
1+ (1 L)2 1 L 1

Now, if we let, for convenience, T = 1, and since

xtt = B (L) xt
xct = (1 B (L)) xt
we can see that
B (L) = F (L) 1
1 B (L) = 1 F (L) 1 = (F (L) 1) F (L) 1 = C (L)
i.e., the cyclical component xct is equal to:
2 1 2
(1 L) 1 L
xct = 2
xt
1+ (1 L)2 1 L 1
a stationary process if xt is integrated up to fourth order.
38
 Properties III

Remember that:
2 2
sxc (! ) = 1 B e i! sx (! ) = C e i! sx (! )
.

We can evaluate at e i! and taking the module, the gain of the cyclical
component is:
2 2
1 e i! 1 ei!
Gc (! ) = 2 2
1+ 1 e i! 1 ei!
4 (1 cos (! ))2
=
1 + 4 (1 cos (! ))2
where we use the identity 1 e i! 1 ei! = 2 (1 cos (! )).

This function gives zero weight to zero frequencies and close to unity
on high frequency, with increases in moving the curve to the left.

Finally note that since the gain is real, (! ) = 0 and the series is not
translated in time.
39
 Butterworth Filters

Filters with gain:

.
2 0 12d3 1
sin !
6 2 A 7
G (! ) = 41 + @ !0 5
sin 2
that depends on two parameters, d, a positive integer, and ! 0, that
sets the frequency for which the gain is 0:5.

Higher values of d make the slope of the band more vertical while
smaller values of ! 0 narrow the width of the lter band.

40
 Butterworth Filters and the HP Filter

Using the fact that 1 cos (! ) = 2 sin2 ! ; the HP gain is:

2
.
1
Gc (! ) = 1
1 + 16 sin4 !2

Since B (L) = 1 C (L), the gain of the trend component is just:

t 1
Gxt (! ) = 1 Gc ( ! ) =
1 + 16 sin4 !2
a particular case of a Butterworth lter of the sine version with d = 2
and ! 0 = 2 arcsin 1
20:25 .

41
 Ideal Filter Revisited

P1 jx
Recall that in our discussion about the ideal lter yt = j= 1 b j L t
we derived the formulae:
sin jb sin ja
bj = b j = 8j2N
j
b
b0 =

for the frequencies a = 2pu and b = 2p where 2 pl < pu < 1 are

l
the lengths of the uctuations of interest.

Although this expressions cannot we evaluate for all integers, they

suggest the feasibility of a more direct approach to band-pass ltering
using some form of these coe cients.
42
 Baxter and King (1999)

Take the rst k coe cients of the above expression.

.
Then, since we want a gain that implies zero variance at ! = 0, we
normalize each coe cient as:
Pk
b0 + 2 j=1 bj
bfj = bj
2k + 1
Pk f = 0 as originally proposed by Craddock (1957).
to get that b
j= k j

This strategy is motivated by the remarkable result that the solution

to the problem of minimizing
Z
1 i! 2
Q= B e Bk e i! d!
2
where B ( ) is the ideal band-pass lter and Bk ( ) is its k-approximation,
is simply to take the rst k coe cients bj from the inversion formula
and to make all the higher order coe cients zero.
43
 Pk f
To see that, write the minimization problem s.t. j= k bj = 0 as:
Z k
X
1 0
L= !; eb e
!; b d! + bfj
2 j= k
where
2 3
1
X
!; eb = 4B e i! bj e i!j 5
e
j= 1

First order conditions:

Z
1 0 0 i!j
e i!j e
!; b + e
!; b e d! =
2
k
X
0= bfj
j= k
for j = 1; :::; k.
44
Since the harmonics e i!j are orthonormal

0 = 2 bj bfj +
Pk
j= k bj
= 2
2k + 1
for j = 1; :::; k, that provides the desired result.

Notice that, since the ideal lter is a step function, this approach
su ers from the same Gibbs phenomenon that arises in Fourier analysis
(Koopmans, 1974).

45