MATH 609-600

Sample Exam #1
Numerical Linear Algebra

The exam will be based on the material covered in the area of numerical linear algebra. You
should be able to prove any major result that we had in class.
Also there will be a substantial body of problems that you need to compute certain quantities
in order to get an answer to the question. Remember the Taussky’s maxim and use it when you
can: if a conjecture about a matrices is false, it can usually be disproved with a 2 × 2 exapmle.
Here is a set of practice problems you can work on in order to prepare yourself for the exam. Do
not expect that problems on the exam will be part of these problems, though there might be some
similar problems.

Below we shall use the following matrices:

     
3 2 0 1 3 2 0 0 2 1 0 0
 2 5 1 1   2 4 0 0   1 2 1 0 
W =  0 2
 V =  Z= 
3 0   0 0 1 1   0 1 2 1 
1 1 0 3 0 0 1 1 0 0 1 2

1. Some simple and not so simple problems

(1) Prove that the inverse of a nonsingular block upper triangular matrix is a upper block-
triangular matrix.
(2) Here A ∈ Rn×n is nonsingular and u, v ∈ Rn .

0 uT
 
S=
v A
Under what condition S is invertible, and give a formula for the inverese when it exists.
(3) Let D be a matrix in partitioned form:
 
A B
D=
C I
Prove that if A − BC is nonsingular, then so is D.
(4) Is matrix Z positive definite ? Prove or disprove.
(5) Assume A and B are symmetric matrices in Rn×n . Prove that AB has real eigenvalues.
(6) Find the spectral radius of the matrix V .
(7) For what values of a is the matrix A is not positive definite ?
 
1 a
A=
a 1

(8) Is the matrix W invertible ?

(9) Let k · k be a subordinate matrix norm, and let S be a nonsingular matrix. Define kAk∗ =
kSAS −1 k. Show that k · k∗ is a subordinate norm.
(10) Assume that A and B are such that for some subordinate matrix norm k · k the following
holds true: kI − BAk < 1. Show that both A and B are nonsingular.
(11) Assume that A and B are such that for some subordinate matrix norm k · k the following
is true: kI − BAk < 1. Prove or disprove that kI − ABk < 1.
1
  
A1 0
(12) Consider the matrix A ∈ R2n×2n of the block form A = , where C is an arbi-
C A2
trary matrix in Rn×n and A1 , A2 ∈ Rn×n are given by
   
2 1 0 ... 0 0 2 -1 0 ... 0 0

 1 2 1 ... 0 0 


 -1 2 -1 ... 0 0 

 0 1 2 ... 0 0   0 -1 2 ... 0 0 
A1 =   and A2 =  

 ... ... ... ... ... ... 


 ... ... ... ... ... ... 

 0 0 0 ... 2 1   0 0 0 ... 2 -1 
0 0 0 ... 1 2 0 0 0 ... -1 2

(1) Show that A is a regular matrix (that means, nonsingular).

(2) Find a formula for its inverse in terms of A−1 −1
1 , A2 and C. Hint: Realise that the
inverse of A is an upper block triangular matrix, so the formula should provide expressions
for the corresponding blocks.
(3) If C = 0 then A is symmenrtic. Is this matrix positive definite ? (4) Find its spectrum
for n = 2.
(13) Consider the Jacobi iteration for the system Zx = b. Does it converege ? Explain !
(14) Write down the Richardson method for the system Zx = b. What is the optimal parameter
τ for the Richardson iteration.
(15) Can you show convergence of an iterative method of your choice for the system W x = b .
(16) Show that Jacobi method for the system A1 x = b (A1 is the matrix defined in Problem 12)
is convergent.
(17) Show that Jacobi method for the system A2 x = b (A2 is the matrix defined in Problem 12)
is convergent.
(18) Let U be a real upper tiangular n × n matrix, i.e. uij = 0 for i > j. Answer the following
questions:
(a) When this matrix is nonsingular;
(b) find its spectral radius;
(c) if U is nonsingular show that its inverse is also upper triangular.
(19) Let A and B be matrices in Rn×n . Assume that A is nonsingular and A and B satisfy the
inequality kA−1 k kBk < 1 for some subordinate matrix norm. Show that the matrix A + B
is nonsingular.
(20) Given the matrices
     
0 1 0 1 -2 1
A= , B= , C= .
-2 0 -1/2 0 0 2

(a) Compute kAk2 and kCk2 .

(b) Does the Jacobi iterative method for Cx = b converge ?
(c) Does the iteration x(k+1) = B x(k) + c converge for any intial guess x(0) ?
(21) Consider the linear system Ax = b, x ∈ Rn with A an SPD matrix. Let λn and λ1 be the
largest and the smallest eigenvalues of A. Recall, that Richardson method
2
x(k+1) = (I − τ A)x(k) + τ b, with optimal parameter τ =
λn + λ1

satisfies the following error transfer estimate (here e(k) = x(k) − x):

λn − λ1 (k)
ke(k+1) k2 ≤ ke k2 , where k · k2 is the Euclidean norm.
λn + λ1
 Using the eigenvectors (and eigenvalues) of A (this is called Fourier mode analysis) prove
that the following estimate is also valid
λn − λ1 (k) 1
ke(k+1) kA ≤ ke kA , where kxkA = (Ax, x) 2 .
λn + λ1
(22) Consider the linear system Ax = b, x ∈ Rn with A nonsingular, but NOT positive (neither
negative) definite matrix. This means that A has positive and negative eigenvalues. Show
that there is NO parameter τ so that the Richardson iteration
x(k+1) = (I − τ A)x(k) + τ b
converges.
(23) Consider the linear system Ax = b, x ∈ Rn with A an SPD matrix. Assume that the error
e(m) = x(m) − x of an iterative method satisfies the estimate
ke(m) kA ≤ kQm (A)kA ke(0) kA ,
where Qm (A) is a polynomial of degree which satrisfies Qm (0) = I. Show that
kQm (A)kA = max |Qm (λi )|.
i=1,...,n

where λi are the eigenvalues of A.

Remark. This problem is a generalization of Problem 21, but the proof is essentially the
same and relies on the representation of any vector in Rn by the eigenvectors of A.
(24) Show that the norm of the plynomial Qm from Problem 23 satisfies the following estimate
 1 − √ ξ m λ1
max |Qm (λi )| ≤ 2 √ , where ξ = .
i=1,...,n 1+ ξ λ n

2. Solutions
Problem 1
Prove that the inverse of a nonsingular block upper triangular matrix is a upper block-triangular
matrix.

Solution: The general form of block upper triangular matrix is

 
A O
U=
C B
where A and B are nonsingular square matrices. The inverse of U in general will have the following
block form  
X Y
U −1 =
W Z
where the the unknown blocks shoud be chosen in such way that U U −1 = I. Performing the block
multiplications we get
AX = I, AY = 0, AW = C, CX + BW = 0, CY + BZ = I.
Since A is nonsingular, from AY = 0 we get Y = 0 as a matrix and, therefore, U −1 is an upper block
triangular matrix.
Note that from these equations we can determine the other unknown blocks:
X = A−1 , W = −B −1 CA−1 , Z = B −1
so that we get the following formula for U −1 :
A−1
 
0
U −1 = .
−B −1 CA−1 B −1
 Problem 2
Here A ∈ Rn×n is nonsingular and u, v ∈ Rn .

0 uT
 
S=
v A

Under what condition S is invertible, and give a formula for the inverese when it exists.

This problem is quite similar to one of the probelms from your HW#1:

Problem 3
Let D be a matrix in partitioned form:
 
A B
D=
C I

Prove that if A − BC is nonsingular, then so is D.

This problem is quite similar to one of the probelms from your HW#1:

Problem 4
Let k · k be a subordinate matrix norm, and let S be a nonsingular matrix. Define kAk∗ =
kSAS −1 k. Show that k · k∗ is a subordinate norm.

One possible solution: First, using the fact that k · k is a subordinate matrix norm, we show that
k · k∗ is a norm:
(1) kAk∗ = kSAS −1 k ≥ 0. S is nonsingular, kAk∗ = kSAS −1 k = 0 if and only if A is a zero
matrix.
(2) kαAk∗ = kαSAS −1 k = αkSAS −1 k = αkAk∗ , for any scalar α ∈ C.
(3) kA + Bk∗ = kS(A + B)S −1 k = kSAS −1 + SBS −1 k ≤ kSAS −1 k + kSBS −1 k = kAk∗ + kBk∗ .
Now prove that ·k∗ is a subordinate matrix norm.

kAk∗ = kSAS −1 k
kSAS −1 xk
= max
n
x∈C , x6=0 kxk
kSAyk
= max (y = S −1 x)
y∈C n , y6=0 kSyk

Define a vector norm kxk∗ = kSxk, then

kAyk∗
kAk∗ = max
y∈C n, y6=0 kyk∗

so kAk∗ is the matrix norm induced by the vector norm kxk∗ .

Problem 5
 This is similar to one of your HW problems:

Problem 6

This is similar to one of your HW problems:

Problem 7

This is similar to one of your HW problems:

Problem 8

Solution:
There are many ways to address this question. (1) Reduce the matrix to lower triangula matris via
elementary transformations; (2) compute the determinats; (3) solve the homogeneous sytem W x = 0.
The easiest of these is to compute the determinant
     
5 1 1 2 1 1 2 5 1
det(W ) = W = 3 det  2 3 0  − 2 det  0 3 0  − det  0 2 3  = 71
1 0 3 1 0 3 1 1 0
Thus, the matrix is nonsingular and therefore invertible.

Problem 9

This is one of your HW problems:

Problem 10

Solution:
Assume that A is singular. Then there is a nonzero vector ψ such that Aψ = 0. then
k(I − BA)xk k(I − BA)ψk kψk
kI − BAk = sup ≥ = = 1.
x∈Rn kxk kψk kψk
However, this contradict the assumption that kI − BAk < 1. Therefoore, A cannot be singular. The
same way we show that B cannot be singular.

Problem 11

Solution:
Use the Taussky’s maxim to disprove the statement with a construction of 2 × 2 exapmle.

Problem 12
 Solution:

(1) A is a block-triangular matrix, so to show that it is nonsingular it is sufficient to show that the
diagonal blocks are nonsingular matrices. Thus we need to check that A1 and A2 are nonsingular. This
is one of the HW problems, so you repeat the arguments from there.
(2) The solution has been provided in Problem # 1 of this set of problems.
(3) Since the diagonal blocks are symmetric, then the matrix (with C = 0) is symmetric. The positive
definiteness forllows for the fact that both diagonal blocks are positive definite, but you HAVE to show
this (this is one of the HW problems).