Eigenvectors and Diagonalization

EE263 Autumn 201213
Stephen Boyd
Lecture 11
Eigenvectors and diagonalization
eigenvectors
dynamic interpretation: invariant sets
complex eigenvectors & invariant planes
left eigenvectors
diagonalization
modal form
discrete-time stability
111
Eigenvectors and eigenvalues

C is an eigenvalue of A Cnn if
X () = det(I A) = 0
equivalent to:
there exists nonzero v Cn s.t. (I A)v = 0, i.e.,
Av = v
any such v is called an eigenvector of A (associated with eigenvalue )
there exists nonzero w Cn s.t. wT (I A) = 0, i.e.,
wT A = wT
any such w is called a left eigenvector of A
112
if v is an eigenvector of A with eigenvalue , then so is v, for any

C, 6= 0
even when A is real, eigenvalue and eigenvector v can be complex
when A and are real, we can always find a real eigenvector v
associated with : if Av = v, with A Rnn, R, and v Cn,
then
Av = v,
Av = v
so v and v are real eigenvectors, if they are nonzero
(and at least one is)
conjugate symmetry : if A is real and v Cn is an eigenvector
associated with C, then v is an eigenvector associated with :
taking conjugate of Av = v we get Av = v, so
Av = v
well assume A is real from now on . . .
113
Scaling interpretation
(assume R for now; well consider C later)
if v is an eigenvector, effect of A on v is very simple: scaling by
Ax
v
x
Av
(what is here?)
114
R, > 0: v and Av point in same direction

R, < 0: v and Av point in opposite directions
R, || < 1: Av smaller than v
R, || > 1: Av larger than v
(well see later how this relates to stability of continuous- and discrete-time
systems. . . )
115
Dynamic interpretation
suppose Av = v, v 6= 0
if x = Ax and x(0) = v, then x(t) = etv
several ways to see this, e.g.,
x(t) = etAv
I + tA +
(tA)
+ v
2!
(t)2
v +
= v + tv +
2!
= etv
(since (tA)k v = (t)k v)
116
for C, solution is complex (well interpret later); for now, assume

R
if initial state is an eigenvector v, resulting motion is very simple
always on the line spanned by v
solution x(t) = etv is called mode of system x = Ax (associated with
eigenvalue )
for R, < 0, mode contracts or shrinks as t

for R, > 0, mode expands or grows as t
117
Invariant sets
a set S Rn is invariant under x = Ax if whenever x(t) S, then

x( ) S for all t
i.e.: once trajectory enters S, it stays in S
trajectory
vector field interpretation: trajectories only cut into S, never out

118
suppose Av = v, v 6= 0, R
line { tv | t R } is invariant
(in fact, ray { tv | t > 0 } is invariant)
if < 0, line segment { tv | 0 t a } is invariant
119
Complex eigenvectors
suppose Av = v, v 6= 0, is complex
for a C, (complex) trajectory aetv satisfies x = Ax
hence so does (real) trajectory
t
x(t) = ae v
= e
vre vim
cos t sin t
sin t cos t

where
v = vre + ivim,
= + i,
a = + i
trajectory stays in invariant plane span{vre, vim}

gives logarithmic growth/decay factor
gives angular velocity of rotation in plane

1110
Dynamic interpretation: left eigenvectors

suppose wT A = wT , w 6= 0
then
d T
(w x) = wT x = wT Ax = (wT x)
dt
i.e., wT x satisfies the DE d(wT x)/dt = (wT x)
hence wT x(t) = etwT x(0)
even if trajectory x is complicated, wT x is simple
if, e.g., R, < 0, halfspace { z | wT z a } is invariant (for a 0)

for = + i C, (w)T x and (w)T x both have form
et ( cos(t) + sin(t))
1111
Summary
right eigenvectors are initial conditions from which resulting motion is
simple (i.e., remains on line or in plane)
left eigenvectors give linear functions of state that are simple, for any
initial condition
1112
1 10 10
0
0 x
example 1: x = 1
0
1
0
block diagram:
1/s
x1
1
1/s
x2
10
1/s
x3
10
X (s) = s3 + s2 + 10s + 10 = (s + 1)(s2 + 10)
eigenvalues are 1, i 10
1113
trajectory with x(0) = (0, 1, 1):
x1
2
1
0
1
0
0.5
1.5
2.5
3.5
4.5
0.5
1.5
2.5
3.5
4.5
0.5
1.5
2.5
3.5
4.5
x2
0.5
0
0.5
1
0
x3
1
0.5
0
0.5
0
1114
left eigenvector associated with eigenvalue 1 is
0.1
g= 0
1
lets check g T x(t) when x(0) = (0, 1, 1) (as above):
1
0.9
0.8
0.7
gT x
0.6
0.5
0.4
0.3
0.2
0.1
0
0
0.5
1.5
2.5
3.5
4.5
t
1115
eigenvector associated with eigenvalue i 10 is
0.554 + i0.771
v = 0.244 + i0.175
0.055 i0.077
so an invariant plane is spanned by
0.554
vre = 0.244 ,
0.055
vim
0.771
= 0.175
0.077
1116
for example, with x(0) = vre we have
x1
1
0
0.5
1.5
2.5
3.5
4.5
0.5
1.5
2.5
3.5
4.5
0.5
1.5
2.5
3.5
4.5
x2
0.5
0.5
0
x3
0.1
0.1
0
1117
Example 2: Markov chain

probability distribution satisfies p(t + 1) = P p(t)
Pn
pi(t) = Prob( z(t) = i ) so i=1 pi(t) = 1
Pn
Pij = Prob( z(t + 1) = i | z(t) = j ), so i=1 Pij = 1
(such matrices are called stochastic)
rewrite as:
[1 1 1]P = [1 1 1]
i.e., [1 1 1] is a left eigenvector of P with e.v. 1
hence det(I P ) = 0, so there is a right eigenvector v 6= 0 with P v = v
it can be shown thatPv can be chosen so that vi 0, hence we can
n
normalize v so that i=1 vi = 1
interpretation: v is an equilibrium distribution; i.e., if p(0) = v then

p(t) = v for all t 0
(if v is unique it is called the steady-state distribution of the Markov chain)
1118
Diagonalization
suppose v1, . . . , vn is a linearly independent set of eigenvectors of
A Rnn:
Avi = ivi, i = 1, . . . , n
express as
define T =

...
v1 vn
v1 vn
and = diag(1, . . . , n), so
v1 vn
AT = T
and finally
T 1AT =
1119
T invertible since v1, . . . , vn linearly independent
similarity transformation by T diagonalizes A

conversely if there is a T = [v1 vn] s.t.
T 1AT = = diag(1, . . . , n)
then AT = T , i.e.,
Avi = ivi,
i = 1, . . . , n
so v1, . . . , vn is a linearly independent set of n eigenvectors of A

we say A is diagonalizable if
there exists T s.t. T 1AT = is diagonal
A has a set of n linearly independent eigenvectors
(if A is not diagonalizable, it is sometimes called defective)

1120
Not all matrices are diagonalizable

example: A =
0 1
0 0
characteristic polynomial is X (s) = s2, so = 0 is only eigenvalue

eigenvectors satisfy Av = 0v = 0, i.e.

0 1
0 0
so all eigenvectors have form v =

v1
0
v1
v2
where v1 6= 0
=0
thus, A cannot have two independent eigenvectors

1121
Distinct eigenvalues
fact: if A has distinct eigenvalues, i.e., i 6= j for i 6= j, then A is
diagonalizable
(the converse is false A can have repeated eigenvalues but still be

diagonalizable)
1122
Diagonalization and left eigenvectors

rewrite T 1AT = as T 1A = T 1, or
w1T
w1T
.. A = ..
wnT
wnT
where w1T , . . . , wnT are the rows of T 1

thus
wiT A = iwiT
i.e., the rows of T 1 are (lin. indep.) left eigenvectors, normalized so that
wiT vj = ij
(i.e., left & right eigenvectors chosen this way are dual bases)
1123
Modal form
suppose A is diagonalizable by T
define new coordinates by x = T x

, so
Tx
= AT x
x
= T 1AT x
x
=
x
1124
in new coordinate system, system is diagonal (decoupled):

1/s
x
1
1/s
x
n
n
trajectories consist of n independent modes, i.e.,
x
i(t) = eitx
i(0)
hence the name modal form
1125
Real modal form
when eigenvalues (hence T ) are complex, system can be put in real modal
form:
S 1AS = diag (r , Mr+1, Mr+3, . . . , Mn1)

where r = diag(1, . . . , r ) are the real eigenvalues, and
Mj =
j
j
j
j
j = j + ij ,
j = r + 1, r + 3, . . . , n
where j are the complex eigenvalues (one from each conjugate pair)
1126
block diagram of complex mode:
1/s
1/s
1127
diagonalization simplifies many matrix expressions

e.g., resolvent:
(sI A)1 =
=
sT T

1 1
T T

1 1
T (sI )T
= T (sI )1T 1

1
1
= T diag
,...,
T 1
s 1
s n
powers (i.e., discrete-time solution):
A
=
=
T T
T T

1 k
1
= T k T 1
T T
= T diag(k1 , . . . , kn)T 1
(for k < 0 only if A invertible, i.e., all i 6= 0)
1128
exponential (i.e., continuous-time solution):

eA = I + A + A2/2! +
= I + T T
+ T T

1 2
/2! +
= T (I + + 2/2! + )T 1
= T eT 1
= T diag(e1 , . . . , en )T 1
1129
Analytic function of a matrix

for any analytic function f : R R, i.e., given by power series
f (a) = 0 + 1a + 2a2 + 3a3 +
we can define f (A) for A Rnn (i.e., overload f ) as
f (A) = 0I + 1A + 2A2 + 3A3 +
substituting A = T T 1, we have
f (A) = 0I + 1A + 2A2 + 3A3 +
= 0T T 1 + 1T T 1 + 2(T T 1)2 +
1
2
= T 0 I + 1 + 2 + T
= T diag(f (1), . . . , f (n))T 1
1130
Solution via diagonalization

assume A is diagonalizable
consider LDS x = Ax, with T 1AT =
then
x(t) = etAx(0)
= T etT 1x(0)
n
X
=
eit(wiT x(0))vi
i=1
thus: any trajectory can be expressed as linear combination of modes

1131
interpretation:
(left eigenvectors) decompose initial state x(0) into modal components
wiT x(0)
eit term propagates ith mode forward t seconds
reconstruct state as linear combination of (right) eigenvectors
1132
application: for what x(0) do we have x(t) 0 as t ?

divide eigenvalues into those with negative real parts
1 < 0, . . . , s < 0,
and the others,
s+1 0, . . . , n 0
from
x(t) =
n
X
eit(wiT x(0))vi
i=1
condition for x(t) 0 is:

x(0) span{v1, . . . , vs},
or equivalently,
wiT x(0) = 0,
i = s + 1, . . . , n
(can you prove this?)

1133
Stability of discrete-time systems

suppose A diagonalizable
consider discrete-time LDS x(t + 1) = Ax(t)
if A = T T 1, then Ak = T k T 1
then
x(t) = Atx(0) =
n
X
i=1
ti(wiT x(0))vi 0
as t
for all x(0) if and only if

|i| < 1,
i = 1, . . . , n.
we will see later that this is true even when A is not diagonalizable, so we
have
fact: x(t + 1) = Ax(t) is stable if and only if all eigenvalues of A have
magnitude less than one
1134

Eigenvectors and Diagonalization

Uploaded by

Copyright:

Available Formats

Eigenvectors and Diagonalization

Uploaded by

Document Information

Original Description:

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Eigenvectors and Diagonalization

Uploaded by

Copyright:

Available Formats

EE263 Autumn 201213

Eigenvectors and eigenvalues

if v is an eigenvector of A with eigenvalue , then so is v, for any

R, > 0: v and Av point in same direction

Eigenvectors and diagonalization

for C, solution is complex (well interpret later); for now, assume

for R, < 0, mode contracts or shrinks as t

Eigenvectors and diagonalization

a set S Rn is invariant under x = Ax if whenever x(t) S, then

vector field interpretation: trajectories only cut into S, never out

(in fact, ray { tv | t > 0 } is invariant)

if < 0, line segment { tv | 0 t a } is invariant

Eigenvectors and diagonalization

trajectory stays in invariant plane span{vre, vim}

gives angular velocity of rotation in plane

Dynamic interpretation: left eigenvectors

if, e.g., R, < 0, halfspace { z | wT z a } is invariant (for a 0)

Eigenvectors and diagonalization

Eigenvectors and diagonalization

X (s) = s3 + s2 + 10s + 10 = (s + 1)(s2 + 10)

trajectory with x(0) = (0, 1, 1):

Eigenvectors and diagonalization

left eigenvector associated with eigenvalue 1 is

eigenvector associated with eigenvalue i 10 is

Eigenvectors and diagonalization

for example, with x(0) = vre we have

Eigenvectors and diagonalization

Example 2: Markov chain

interpretation: v is an equilibrium distribution; i.e., if p(0) = v then

and = diag(1, . . . , n), so

T invertible since v1, . . . , vn linearly independent

similarity transformation by T diagonalizes A

so v1, . . . , vn is a linearly independent set of n eigenvectors of A

A has a set of n linearly independent eigenvectors

(if A is not diagonalizable, it is sometimes called defective)

Not all matrices are diagonalizable

characteristic polynomial is X (s) = s2, so = 0 is only eigenvalue

so all eigenvectors have form v =

thus, A cannot have two independent eigenvectors

(the converse is false A can have repeated eigenvalues but still be

Eigenvectors and diagonalization

Diagonalization and left eigenvectors

where w1T , . . . , wnT are the rows of T 1

define new coordinates by x = T x

Eigenvectors and diagonalization

in new coordinate system, system is diagonal (decoupled):

Real modal form

S 1AS = diag (r , Mr+1, Mr+3, . . . , Mn1)

Eigenvectors and diagonalization

block diagram of complex mode:

Eigenvectors and diagonalization

diagonalization simplifies many matrix expressions

exponential (i.e., continuous-time solution):

Eigenvectors and diagonalization

Analytic function of a matrix

= T diag(f (1), . . . , f (n))T 1

Eigenvectors and diagonalization

Solution via diagonalization