Lectures in Functional Analysis

Roman Vershynin
Department of Mathematics, University of Michigan, 530 Church
St., Ann Arbor, MI 48109, U.S.A.
E-mail address: romanv@umich.edu
 Preface
These notes are for a one-semester graduate course in Functional Analysis,
which is based on measure theory. The notes correspond to the course Real Analysis
II, which the author taught at University of Michigan in the Fall 2010. The course
consists of about 40 lectures 50 minutes each.
The student is assumed to be familiar with measure theory (both Lebesgue and
abstract), have a good command of basic real analysis (epsilon-delta) and abstract
linear algebra (linear spaces and transformations).
The course develops the theory of Banach and Hilbert spaces and bounded
linear operators. Main principles of are covered in depth, which include Hahn-
Banach theorem, open mapping theorem, closed graph theorem, principle of uni-
form boundedness, and Banach-Alaoglu theorem. Fourier series are developed for
general orthogonal systems in Hilbert spaces. Compact operators and basics of
Fredholm theory are covered.
Spectral theory for bounded operators is studied in the second half of the course.
This includes the spectral theory for compact self-adjoint operators, functional
calculus and basic spectral theory of general (non-compact) operators, although
the latter needs to be expanded a bit.
Topics not covered include: Krein-Milman theorem (although this can be done
with one extra lecture), unbounded linear operators, and Fourier transform. Most
applications to ODE and PDE are not covered, however the integral operators serve
as a main example of operators in this course.
The material has been compiled from several textbooks, including Eidelman,
Milman and Tsolomitis “Functional Analysis”, Kirillov and Gvishiani “Theorems
and problemsin functional analysis”, Reed and Simon “Methods of modern mathe-
matical physics. I. Functional analysis”, V. Kadets “A course in functional analy-
sis” (Russian), and P. Knyazev, “Functional analysis”. Minor borrowings are made
from Yoshida “Functional analysis”, Rudin “Functional analysis”, and Conway “A
course in functional analysis”. For some topics not covered, one may try R. Zimmer
“Essential results of functional analysis”.
Acknowledgement. The author is grateful to his students in the Math 602
course Real Analysis II, Winter 2010, who suggested numerous corrections for these
notes. Special thanks are to Matthew Masarik for his numerous thoughtful remarks,
corrections, and suggestions, which improved the presentation of this material.
 Contents

Preface iv
Chapter 1. Banach and Hilbert spaces 1
1.1. Linear spaces and linear operators 1
1.2. Normed spaces 7
1.3. Banach spaces 15
1.4. Inner product spaces 19
1.5. Hilbert spaces 25
1.6. Fourier series 28
Chapter 2. Bounded linear operators 39
2.1. Bounded linear functionals 39
2.2. Representation theorems for linear functionals 42
2.3. Hahn-Banach theorem 47
2.4. Bounded linear operators 56
Chapter 3. Main principles of functional analysis 69
3.1. Open mapping theorem 69
3.2. Closed graph theorem 74
3.3. Principle of uniform boundedness 76
3.4. Compact sets in Banach spaces 83
3.5. Weak topology 87
3.6. Weak topology. Banach-Alauglu’s theorem 91
Chapter 4. Compact operators. Elements of spectral theory 94
4.1. Compact operators 94
4.2. Fredholm theory 98
4.3. Spectrum of a bounded linear operator 101
4.4. Properties of spectrum. Spectrum of compact operators. 103
Chapter 5. Self-adjoint operators on Hilbert space 110
5.1. Spectrum of self-adjoint operators 110
5.2. Spectral theorem for compact self-adjoint operators 113
5.3. Positive operators. Continuous functional calculus 116
5.4. Borel functional calculus. Spectral theorem for self-adjoint operators 122
Bibliography 126

Index 127

v
 CHAPTER 1

Banach and Hilbert spaces

1.1. Linear spaces and linear operators

Lec.1: 09/08/10
1.1.1. From individual functions to function spaces. In this section, our
discussion is general and not very precise. For now, our goal is to see a big picture.
? In antiquity, properties of individual numbers were of great importance. Whether
2 is rational, for example, was a source of a great discussion. Later, with the
development of calculus in the XVII century, the focus shifted from numbers to
functions. A function wraps up individual numbers into some assignment rule,
and it is the assignment that now matters rather than individual numbers. Later,
in the second half of XIX century, the interest of mathematicians shifted further
from studying individual functions (their differentiability, integrability) to function
spaces. One may say that a function space wraps up functions of interest into one
geometric object. The geometry of the function space reflects important properties
of functions. This lead to the development of the field of functional analysis.
In functional analysis, we view functions as points or vectors in a function
space. Since we can add functions on a common domain (say ra, bs) by defining
pf gqpxq : f pxq gpxq and multiply them by defining paf qpxq  af pxq, we see
that a function space is a linear vector space.
Furthermore, we can envision a kind of distance on a function space, which
would quantify similarity (or dissimilarity) of functions. The choice of a distance
depends on application. One choice of distance between f and g is the “sup-
distance”
}f
g}8 : sup |f pxq
gpxq|.
Pr s
x a,b

This is clearly a metric, so a function space becomes not only a linear vector space
but also a metric space. Such spaces will be called normed spaces later. Another
natural choice of a distance would be the
»b
}f
g}1 : |f pxq
gpxq| dx
a

Heuristically, a control of }f
g }8 forces the values of f and g stay close everywhere

on ra, bs, while a control of }f
g }1 forces the values of f and g stay close “on
average”.

1.1.2. Examples of linear vector spaces. Classical example of linear vector

spaces given in linear algebra courses are Rn and Cn . Here are some examples of
function spaces that are linear vector spaces (check!):
1. F  tall functions R Ñ Ru. This space is too large, and is never studied.
2. {all solutions of a linear homogeneous PDE}
3. L1 ra, bs  tall Lebesgue integrable functions on ra, bsu
1
 1.1. LINEAR SPACES AND LINEAR OPERATORS 2

4. L8 ra, bs  tall bounded almost everywhere functions on ra, bsu

5. C ra, bs  tall continuous functions on ra, bsu
6. C 1 ra, bs  tall continuously differentiable functions on ra, bsu
7. C 8 ra, bs  tall infinitely differentiable functions on ra, bsu
8. P pxq  tall univariate polynomialsu
9. Pn pxq  tall univariate polynomials of degree at most nu
Also there are many natural examples of sequence spaces that are linear vector
spaces (check!):
1. s  tall sequences of real numbers pan q8
n1 u. This space is too large, and is
never studied. °
2. `1  tall absolutely summable sequences, i.e. satisfying n |an |   8u
3. `8  tall bounded sequences of real numbersu
4. c  tall convergent sequences of real numbersu
5. c0  tall sequences of real numbers converging to zerou
6. c00  tall sequences of real numbers with finite supportu

1.1.3. Subspaces. A (linear) subspace of a linear vector space is a subset that

is closed under the operations of addition of vectors and multiplication by scalars:
Definition 1.1.1. A subset A1 of a linear vector space E is called a (linear)
subspace if x, y P E1 , a, b P R implies ax by P E1 .
Example 1.1.2. One can check that the following set-theoretic inclusions hold:1
Pn pxq € P pxq € C 8 ra, bs € C 1 ra, bs € C ra, bs € L8 ra, bs € L1 ra, bs € F,
c00 € `1 € c0 € c € `8 € s.
Some of these inclusions encode non-trivial theorems of undergraduate analysis.
All these inclusions are subspace inclusions, i.e. Pn pxq is a subspace of P pxq,
etc. (check this!)
Exercise 1.1.3. Let E be a linear vector space. Show that t0u and E
are subspaces of E. Show that the intersection of an arbitrary collection
of subspaces of E is again a subspace of E.

Lec. 2: 09/10/10
1.1.4. Hamel basis. As we know, every finite dimensional linear vector space
E has a basis tx1 , . . . , xn u. A basis is a maximal linearly independent subset of
vectors in E. The number n of basis elements is called the dimension of E; this
number is independent of the choice of the basis. Every vector x P E can be
uniquely expressed as a linear combination of the basis elements:
¸
n
(1.1) x ak xi , for some ak P R.
k 1
The notion of basis can be generalized to arbitrary, possibly infinite dimensional
linear vector spaces E.

pq pq
1For consistency of these inclusions, we restrict the functions in P x , P x and F onto
n
ra, bs.
 1.1. LINEAR SPACES AND LINEAR OPERATORS 3

Definition 1.1.4 (Hamel basis). A subset X of a linear vector space E is called

a Hamel basis of E if every vector x P E can be uniquely expressed as a finite linear
combination of some elements of X :
¸
n
(1.2) x ak xk

k 1
for some nonzero scalars ak and vectors xk P X.
Exercise 1.1.5. Show that each of the following two statements gives
an equivalent definition of Hamel basis:
(1) A Hamel basis is a maximal linearly independent2 subset X € E.
(2) A Hamel basis is a linearly independent subset of E which spans
E. The latter means that the linear span of X , defined as
! ¸
n )
SpanpX q : x  ak xk : ak P R, xk P X , n P N ,

k 1
coincides with E.
Since we have no topology on E, we have to consider finite sums in (1.2). This
requirement is too strong to be put in practice, which makes Hamel bases essentially
impractical (except in theory). We will come across the more practical notion of
Schauder basis later.
Proposition 1.1.6. Every linear vector space E has a Hamel basis.
For finite dimensional spaces E, this result is usually proved in undergraduate
linear algebra using induction. One keeps adding linearly independent elements
to a set until it spans the whole E. This argument can be pushed into infinite
dimensions as well, where the usual induction is replaced by transfinite induction.
The transfinite induction is best done with Zorn’s lemma (review a Wikipedia article
on Zorn’s lemma if you are uncomfortable with all notions it uses):
Lemma 1.1.7 (Zorn’s lemma). A partially ordered set in which every chain has
an upper bound contains a maximal element.
Proof of Proposition 1.1.6. Consider a family F of all linearly indepen-
dent subsets of E, which is partially ordered by inclusion. We claim that F has a
maximal element; this would obviously complete the proof by Exercise 1.1.5. We
will get a maximal element from Zorn’s lemma. Let us check its assumption. Con-
sider a chain pXα q of elements in F. The elements Xα are linearly independent
subsets of E totally ordered by inclusion. Therefore, their union Yα Xα is again a
linearly independent subset of E (check!) Hence this union is an element of F, and
it is clearly an upper bound for the chain pXα q. The assumption of Zorn’s lemma
is therefore satisfied, and the proof is complete 
As in the finite dimensional case, the cardinality of Hamel basis of E is called
the dimension of E; one can show that the dimension is independent of the choice
of a Hamel basis.
Example 1.1.8. Here we consider some of the examples of linear vector spaces
given in Section 1.1.2.
2Linear independence means that every finite subset of X is linearly independent in the
°
ordinary sense. In other words, if n 
k1 ak xk P P P 
0 for some ai R, xk X , n N, then all ak 0.
 1.1. LINEAR SPACES AND LINEAR OPERATORS 4

1. dimpRn q  n, dimpCn q  n.
2. dimpPn pxqq  n 1, the monomials t1, x, x2 , . . . , xn u form a basis.
3. dimpP pxqq  8, the monomials t1, x, x2 , . . .u form a Hamel basis.
4. dimpc00 q  8, the coordinate vectors p0, . . . , 0, 1, 0, . . .q form a Hamel basis.
Remark 1.1.9. Unfortunately, the notion of Hamel basis is too strong. Except
in spaces P pxq and c00 (which are isomorphic - why?) no explicit constructions are
known in any other infinite dimensional vector space. It would be great to have
a construction of a Hamel basis in C r0, 1s, for example. However, Hamel bases
usually have to be uncountable; see a later exercise.
1.1.5. Quotient spaces. The notion of quotient space allows one easily to
collapse some directions in linear vector spaces. One reason for doing this is when
one has unimportant directions and would likes to neglect them; see the construction
of L1 below.
Definition 1.1.10 (Quotient space). Let E1 be a subspace of a linear vector
space E. Consider an equivalence relation on E defined as
xy if x
y P E1 .
The quotient space E {E1 is then defined as the set of equivalence classes (cosets)
rxs for all x P E.
The quotient space is a linear space, with operations defined as
rxs rys : rx y s, arxs : raxs for x, y P E, a P R.
The dimension of the quotient space is called the codimension of E1 , thus
codimpE1 q : dimpE {E1 q.
Exercise 1.1.11. Prove that the operations above are well defined,
and that quotient space is indeed a linear space.

Remark 1.1.12. 1. Observe that rxs is an affine subspace:

rxs  x E1 : tx h : h P E1 u.
2. The definition of the equivalence relation x  y is meant to ignore the
directions in E1 , and thus to identify points x, y if they only differ by a vector from
E1 .
3. From undergraduate linear algebra we know that if E is finite dimensional
then all of its subspaces E1 satisfy
dimpE1 q codimpE1 q  dimpE q.
Example 1.1.13 (Space L1 ). The notion of quotient space comes handy when
we define the space of integrable functions L1  L1 pΩ, Σ, µq where pΩ, Σ, µq is an
arbitrary measure space. We first consider
E : tall integrable functions f on pΩ, Σ, µqu.
To identify functions that are equal µ-almost everywhere, we consider the subspace
we would like to neglect:
E1 : tall functions f  0 µ-almost everywhereu
Then we define
L1  L1 pΩ, Σ, µq : E {E1 .
 1.1. LINEAR SPACES AND LINEAR OPERATORS 5

This way, the elements of L1 are, strictly speaking, not functions but classes of
equivalences.3 But in practice, one thinks of an f P L1 as a function, keeping in
mind that functions that coincide µ-almost everywhere are “the same”.
Example 1.1.14 (Space L8 ). A similar procedure is used to define the space
of essentially bounded functions L8  L8 pΩ, Σ, µq. A real valued function f on Ω
is called essentially bounded if there exists a bounded function g on Ω such that
f  g µ-almost everywhere. Similar to the previous example, we consider the linear
vector space
E : tall essentially bounded functions f on pΩ, Σ, µqu
and the subspace we would like to neglect:
E1 : tall functions f  0 µ-almost everywhereu
Then we define
L8  L8 pΩ, Σ, µq : E {E1 .
Example 1.1.15. As we know, the space c0 of sequences converging to zero is
a subspace of the space c of all convergent sequences. Let us observe that
codimpc0 q  1.
Indeed, every sequence x P c can be uniquely represented as
x  a1 z for some a P R, z P c0
where 1  p1, 1, . . .q. (How do we choose the value of a?). Hence
rxs  ar1s rzs  ar1s.
It follows that every element rxs P c{c0 is a constant multiple of the element r1s.
Therefore, dimpc{c0 q  1 as claimed.
This example shows that the space c0 makes up almost the whole space c,
except for one dimension given by the constant sequences. This explains why the
space c is rarely used in practice; one prefers to work with c0 which is almost the
same as c but has the advantage that we know the limits of all sequences there
(zero).
1.1.6. Linear operators. This is a quick review of the classical linear algebra
concept.
Definition 1.1.16 (Linear operator). A map T : E Ñ F between two linear
vector spaces E and F is called a linear operator if it preserves the operations of
addition of vectors and multiplication by scalars, i.e.
T pax by q  aT pxq bT py q for all x, y P E, a, b P R.
4
The kernel and image of T is defined respectively as
kerpT q  tx P E : T x  0u; ImpT q  tT x : x P E u.

3Even more strictly speaking, the representative functions f in L may take infinite values,
1
too. However, every integrable function is finite a.e. So every such function is equivalent to a
function that is finite everywhere.
pq
4One usually writes T x instead of T x
 1.1. LINEAR SPACES AND LINEAR OPERATORS 6

Example 1.1.17 (Differential operator). The simplest example of a differential

operator is given by taking the derivative of a function:
T pf q  f 1 .
Such operator is well be defined e.g. on the space of polynomials T : P pxq Ñ P pxq.
But usually one prefers to have a differential operator on a larger space; for example
T : C 1 r0, 1s Ñ C r0, 1s is also well defined.
Example 1.1.18 (Embedding and quotient map). Given a subspace E1 of a
linear vector space E, there are two canonical linear operators associated with it:
1. Embedding ι : E1 Ñ E, which acts as an identity ιpxq  x;
2. Quotient map q : E Ñ E {E1 , which acts as q pxq  rxs.
Example 1.1.19 (Shifts on sequence spaces). On any sequence space such as
c00 , c0 , c, `8 , `1 , one can define the right and left shift operators respectively as
Rpxq  p0, x1 , x2 , . . .q; Lpxq  px2 , x3 , . . .q for x  px1 , x2 , . . .q.
Exercise 1.1.20. Compute the images and kernels of the embedding,
quotient map, and the shift operators in the examples above.

1.1.7. Additional Exercises.

Exercise 1.1.21. Show that the intersection of an arbitrary collection
of subspaces of a linear vector space E is again a subspace of E.

Exercise 1.1.22. Show that every linearly independent subset of a

linear vector space E can be extended to a Hamel basis of E.

Exercise 1.1.23. [Complementary subspaces] Let E1 be a subspace of

a linear vector space E. Prove that there exists a subspace E2 of E such
that
E1 X E2  t0u, SpanpE1 Y E2 q  E.
(Hint: extend a Hamel basis from E1 onto E; use the extension to con-
struct E2 ). Such subspaces E1 , E2 are called complementary to each
other. Show that E1 , E2 are complentary if and only if every vector
x P E can be uniquely represented as the sum
x  x1 x2 for some x1 P E1 , x2 P E2 .
Exercise 1.1.24. [Injectivization] This is a linear version of the fun-
damental theorem on homomorphisms for groups. Consider a linear
operator T : E Ñ F acting between linear spaces E and F . The operator
T may not be injective; we would like to make it into an injective op-
erator. To this end, we consider the map T̃ : X { ker T Ñ Y which sends
every coset rxs into a vector T x, i.e. T̃ rxs  T x.
(i) Prove that T̃ is well defined, i.e. rx1 s  rx2 s implies T x1  T x2 .
(ii) Check that T̃ is a linear and injective operator.
(iii) Check that T is surjective then T̃ is also surjective, and thus T̃ is a
linear isomorphism between X { ker T and Y .
 1.2. NORMED SPACES 7

(iv) Show that T  T̃  q, where q : X Ñ X { ker T is the quotient map. In

other words, every linear operator can be represented as a compo-
sition of a surjective and injective operator.

1.2. Normed spaces

Lec. 3: 09/13/10
1.2.1. Definition and examples. A norm is a general concept of length of
vectors. Once we have a norm we can geometrize analysis in some sense, because
we would have a metric on our linear vector spaces. For example, this would allow
us to study functions through geometry of function spaces.
A norm is an assignment of a non-negative number }x} to every vector x in a
linear vector space E. In order to have a meaning of length, this assignment must
satisfy some natural axioms:
Definition 1.2.1 (Normed space). Let E be a linear vector space. A norm on
E is a function }  } : E Ñ R which satisfies the following axioms:
(i) }x} ¥ 0 for all x P E; }x}  0 if and only if x  0;
(ii) }ax}  |a|}x} for all x P E, a P R (or C);
(iii) }x y } ¤ }x} }y } for all x, y P E.
The linear vector space E equipped with the norm }  } is called a normed space,
and denoted X  pE, }  }q.
Axiom (iii) is called triangle inequality for the following reason. Given an
arbitraty triangle in E with vertices x, y, z P E, its lengths satisfy the inequality
(1.3) }x
z} ¤ }x
y} }y
z},
which follows from norm axiom (iii). For the usual Euclidean length on the plane,
this is the ordinary triangle inequality.
The normed space is naturally a metric space, with the metric defined by
dpx, y q : }x
y }.
The norm axioms, and in particular triangle inequality (1.3), show that this is
indeed a metric (check!)
Exercise 1.2.2. [Normed spaces `8 , c, c0 , `1 , C pK q, L1 , L8 ] Many of
linear vector spaces introduced in Section 1.1.2 and Example 1.1.12 are
in fact normed spaces. Check the norm axioms for them:
1. The space of bounded sequences `8 is a normed space, with the norm
defined as
(1.4) }x}8 : sup |xi |.
i

2. The spaces c and c0 are normed spaces, with the same sup-norm as in
(1.4).
3. The space of summable sequences `1 is a normed space, with the norm
defined as
8̧
}x}1 : |xi |.

i 1
 1.2. NORMED SPACES 8

4. The space C pK q of continuous functions on a compact topological

space K is a normed spaces with the norm5
}f }8 : max
K
|f pxq|.
5. The space L1  L1 pΩ, Σ, µq is a normed space, with the norm defined
as6 »
}f }1 : |f pxq| dµ.
Ω
Note that `1 is a partial case of the space L1 pΩ, Σ, µq where Ω  N and
µ is the counting measure on N.
6. The space L8  L8 pΩ, Σ, µq is a normed space, with the norm defined
as the essential supremum:
}f }8 : esssup |f ptq| : ginf sup |g ptq|.
tPΩ f a.e. tPΩ

Here the infimum is taken over all g P L8 that are equal to f µ-almost
everywhere. Note that `8 is a partial case of the space L8 pΩ, Σ, µq
where Ω  N and µ is the counting measure on N.

Exercise 1.2.3. [Essential supremum] Show that the norm in L8 pΩ, Σ, µq

can be equivalently computed as
}f }8  µpinf
Aq0
sup |f ptq|
P z
t Ω A

where the infimum is over all subsets A € Ω of measure zero.

Exercise 1.2.4. [Continuity of norms] Prove that the norm assignment

x ÞÑ }x} is a continuous function on the normed space. Specifically, show
that if }xn
x} Ñ 0 then }xn } Ñ }x}.

1.2.2. Convexity of norms and balls. The geometry of a normed space

can be very different from that of the usual Euclidean geometry. The balls do not
need to be round anymore. For example, the ball of `8 looks like a cube (why?)
Nevertheless, one important property still holds: the balls are always convex sets,
and the norm is a convex function. The convexity considerations are very helpful
when one works in normed spaces.
Let us first recall some notions coming from geometry of metric spaces.
Definition 1.2.5 (Balls, spheres of normed spaces). Let X be a normed space.
A (closed) ball centered at a point x0 P X and with radius r ¡ 0 is defined as
BX px0 , rq : tx P X : }x
x0 } ¤ ru.
The (closed) unit ball of X is defined as
BX : BX p0, 1q  tx P X : }x} ¤ 1u.
The unit sphere of X is the boundary of the unit ball, that is
SX : tx P X : }x}  1u.
5The maximum is attained because K is compact.
6Formally, as we know the elements of L are cosets f
³ 1 rs
rather than functions. We should
}r s} 
define the norm f 1 : | p q| rs
Ω f x dµ where f is an arbitrary element in the coset f . Check
that this way, the norm is well defined. The same concerns the definition of L8 below.
 1.2. NORMED SPACES 9

Definition 1.2.6 (Convex functions and sets). Let E be a linear vector space.
A function f : E Ñ R is convex if for all x, y P E, λ P r0, 1s one has
f pλx p1
λqyq ¤ λf pxq p1
λqf pyq.
A set K „ E is convex if for all x, y P K, λ P r0, 1s one has
λx p1
λqy P K.
A geometric meaning of convexity is the following. A function f is convex on
E if its graph restricted to any interval rx, y s € E lies below the interval joining
the points px, f pxqq and py, f py qq; see the picture. A set K is convex if, together
with any two points x, y P K, it contains the interval rx, y s.

Figure 1.1. Convex function f on a linear vector space E

Proposition 1.2.7 (Norm axioms imply convexity). Let X be a normed space.

Then:
1. The function x Ñ }x} is convex on X.
2. The unit ball BX is a closed, origin-symmetric7, and convex set in X.
Proof. 1. Convexity of the norm follows from the norm axioms. Indeed, for
every x, y P E, λ P r0, 1s we have
}λx p1
λqyq} ¤ }λx} }p1
λqyq}  λ}x} p1
λq}y}.
2. Closedness of BX follows from continuity of the norm (see Exercise 1.2.4).
Origin-symmetry follows from norm axiom (ii) with λ 
1. Finally, to prove
convexity of BX we choose arbitrary x, y P BX , λ P r0, 1s and use inequality above
to obtain
}λx p1
λqy} ¤ λ}x} p1
λq}y} ¤ λ p1
λq  1.
It follows that λx p1
λqy P BX as required. 
The converse to Proposition 1.2.7 also holds:
Proposition 1.2.8 (Convexity implies triangle inequality). Let x ÞÑ }x} be a
real-valued function defined on a linear vector space E. Assume that this function
satisfies norm axioms (i) and (ii). Then:
7Origin-symmetric means that x P BX implies
x P BX
 1.2. NORMED SPACES 10

1. If the function x Ñ }x} is convex, then the triangle inequality is satisfied, and
}  } is a norm on E.
2. If the sublevel set tx P X : }x} ¤ 1u is convex, then }  } is a norm on E.
Proof. 1. Convexity ensures that for every x, yP E, λ P r0, 1s we have
}λx p1
λqyq} ¤ λ}x} p1
λq}y}.
Triangle inequality follows from this for λ  1{2.
2. This statement is less trivial, and can not be obtained from the first one
directly. Indeed, while it is true that the sublevel sets of a convex functions are
convex sets, the converse statement may fail (construct an example!)
The assumption states that, for u, v P E we have:
(1.5) if }u} ¤ 1, }v } ¤ 1, λ P r0, 1s, }λu p1
λqv} ¤ 1.
then
Let x, y P E; we want to show that }x y } ¤ }x} }y }. This is equivalent to
 

}x} }y} }x} }y}  ¤ 1.

 x y 


We obtain this inequality from (1.5) with

u
x
}x} , v  }yy} , λ  }x}}x}}y} .
This completes the proof. 
Lec. 4: 09/15/2010
1.2.3. Spaces Lp . Minkowski inequality. We have already come across
the spaces L1 and `1 . They are partial cases of a big family of spaces Lp and `p
which we will study now.
Consider a measure space pΩ, Σ, µq and an exponent p P r1, 8q. We define
the space of p-integrable functions Lp  Lp pΩ, Σ, µq as the set of all measurable
functions f : Ω Ñ R such that
»
|f pxq|p dµ   8.
Ω

Proposition 1.2.9. Lp pΩ, Σ, µq is a linear vector space for p P r1, 8q.

Proof. The only non-trivial point is that Lp is closed under addition, that is
f, g P Lp implies f g P Lp . We will obtain this by the convexity of the function
z ÞÑ |z |p on R for p ¥ 1. The convexity implies the pointwise inequality

| pq
 f t

g ptq| p
 ¤ |f ptq|
p
|gptq|p , t P Ω.
2 2
Integrating both sides of this inequality, we obtain the required claim. 

We turn Lp into a normed space by defining

» 1{p
}f }p : |f pxq|p dµ for f P Lp pΩ, Σ, µq.
Ω

Proposition 1.2.10. Lp pΩ, Σ, µq is a normed spaces with the norm }f }p for

p P r1, 8q.
 1.2. NORMED SPACES 11

Proof. Norm axioms (i) and (ii) are straightforward. Axiom (iii), triangle
inequality, will follow from Proposition 1.2.8. To this end, it suffices to check that
the sublevel set
Bp : tf P Lp : }f }p ¤ 1u
is a convex set. To prove this, let us fix f, g P Bp and λ P r0, 1s. Since the function
z ÞÑ |z |p is convex on R for p ¥ 1, we have a pointwise inequality
 
λf t p q p1
λqgptqp ¤ λ|f ptq|p p1
λq|gptq|p .
Integrating both sides of this inequality implies
» » »
 
p q p1
λqgptqp dµ ¤ λ |f ptq|p dµ p1
λq |gptq|p dµ ¤ λ p1
λq  1.
λf t
Ω Ω Ω
We have showed that }λf p1
λqg }p ¤ 1, hence λf p1
λqg P Bp . Therefore,
the sublevel set Bp is convex. The proof is finished by Proposition 1.2.8. 
Writing out the triangle inequality }f g }p ¤ }f }p }g}p in analytic form, we
obtain the classical Minkowski inequality:
Theorem 1.2.11 (Minkowski inequality in Lp ). Let p P r1, 8q. Then, for every
two functions f, g P Lp pΩ, Σ, µq one has

» 1{p
» 1{p
» 1{p
|f ptq g ptq|p dµ ¤ |f ptq|p dµ |gptq|p dµ .
Ω Ω Ω

1.2.4. Spaces `p and `np . An important partial case of the space Lp pΩ, Σ, µq
is the space `p obtained by choosing Ω  N and µ to be the counting measure on
N. Equivalently, for p P r1, 8q, the space of p-summable sequences `p is defined to
consist of sequences x  pxi q8
i1 for which
8̧
|xi |p   8.

i 1
We turn `p into a normed space with the norm

8̧ 1{p
}x}p : |xi |p .

i 1
Writing down Minkowski inequality for this specific measure space, we obtain:
Theorem 1.2.12 (Minkowski inequality in `p ). Let p P r1, 8q. Then, for every
two sequences x, y P `p one has

8̧ 1{p
8̧ 1{p
8̧ 1{p
|xi yi |p ¤ |xi |p |yi |p .

i 1 
i 1 
i 1

A remarkable family of finite-dimensional spaces Lp pΩ, Σ, µq is formed by con-
sidering Ω to be a finite set, say Ω  t1, . . . , nu and µ to be the counting measure on
Ω. The resulting space is called `np . The functions in `np can be obviously identified
with vectors in Rn . Thus `np  pRn , }  }p q with the norm

¸
n 1{p
}x}p : |xi |p .

i 1
 1.2. NORMED SPACES 12

When p  2, this space is the usual Euclidean space Rn . However, for p  2,

the geometry of `np is quite different from Euclidean. Indeed, in two-dimensional
spaces, the unit ball of `21 is a diamond with vertices p1, 0q, p0, 1q, p
1, 0q, p0,
1q.
The unit ball of `28 is the square with vertices p1, 1q, p1,
1q, p
1, 1q, p
1,
1q.
Exercise 1.2.13. [`8 as the limit of `p ] This exercise explains the index
8 in the name of the spaces `8 , L8 .
1. Show that if x P `p for some p0 ¥ 1 then
0

}x}p Ñ }x}8 as p0 ¤ p Ñ 8.
2. Consider the space L8  L8 pΩ, Σ, µq with finite total measure µpΩq.
Show that if f P L8 then
}f }p Ñ }f }8 as p Ñ 8.
1.2.5. Subspaces of normed spaces.
Definition 1.2.14 (Subspace). Let X be a normed space. A subspace Y of X
is a linear subspace equipped with the norm induced from that of X.
This concept should be familiar from topology, where a subspace is a subset of
a topological space with the induced topology.
Example 1.2.15. 1. The space of polynomials P pxq is a dense subspace of
C r0, 1s. This is the statement of Weierstrass approximation theorem.
2. The set of all continuous functions C r0, 1s forms a dense subspace of L1 r0, 1s. (of
course, both spaces are considered in the L1 norm!) This follows from a theorem
in measure theory that states that an integrable function can be approximated
by a continuous function (why?)
Exercise 1.2.16. 1. Show that the set of convergent sequences c and
the set of sequences converging to zero c0 are closed subspaces of `8 .
2. For all p P r1, 8q, show that the set of p-summable sequences `p is a
closed subspace of `8 but is a dense subspace of c0 .

1.2.6. Quotient spaces of normed spaces. In Section 1.1.5, we defined

quotient spaces of linear vector spaces. If the ambient space is a normed space,
then we can also induce the norm onto the quotient space as follows.
Definition 1.2.17 (Quotient space of a normed space). Let X be a normed
space and Y be a closed subspace of X. We define a norm on X {Y as follows. For
every coset rxs  x Y , we put
}rxs} : yinf
PY
}x y }.

It is easy to understand the norm in the quotient space geometrically as the

distance from the origin to the coset rxs. Indeed, let distpb, Aq denote the distance
in X from a point b to a set A:
distpb, Aq  inf }b
a}.
P
a A
Then clearly
(1.6) }rxs}  distp0, rxsq.
 1.2. NORMED SPACES 13

Proposition 1.2.18. The definition of }rxs} above indeed produces a norm on

X {Y .
Proof. First we observe that the number }rxs} is well defined, i.e. it does not
depend on a choice of a representative x in the coset rxs. This clearly follows from
the geometric definition (1.6).
Next, we have to check the three norm axioms.
(i) Assume that }rxs}  0. Then, from the geometric definition (1.6) we see
that 0 is a limit point of rxs. Since Y is closed, so is rxs  x Y . Therefore 0 P rxs.
Hence rxs  r0s, which verifies norm axiom (i).
(ii) Let x P X and λ P R. Then
}rλxs}  yinf
PY
}λx y }  inf }λx
P
y Y
λy }  λ  inf }x
P
y Y
y }  λ}rxs}.

This verifies norm axiom (ii).

(iii) Let us fix x1 , x2 P X; we want to show that }rx1 x2 s} ¤ }rx1 s} }rx2 s}.
To this end, fix an arbitrary ε ¡ 0. By the definition of the quotient norm there
exist y1 , y2 P Y so that
}x1 y1 }
ε ¤ }rx1 s} ¤ }x1 y1 }, }x2 y2 }
ε ¤ }rx2 s} ¤ }x2 y2 }.
Using triangle inequality for the norm in X, we obtain
}x1 x2 y1 y2 } ¤ }x1 y1 } } x2 y2 } ¤ }rx1 s} }rx2 s} 2ε.
We conclude that
}rx1 x2 s}  inf }x1
P
y Y
x2 y } ¤ }x1 x2 y1 y2 } ¤ }rx1 s} }rx2 s} 2ε.

Since ε ¡ 0 is arbitrary, this completes the proof of norm axiom (iii). 

Exercise 1.2.19. Consider the subspace Y of C pK q that consists of

constant functions. Derive the following formula for the norm in the
quotient space C pK q{Y :


}rf s}  21 max f ptq
min f ptq
P
t K P
t K
for f P C pK q.
Exercise 1.2.20. Derive the following formula for the norm in the
quotient space `8 {c0 :
}ras}  lim sup |ai | for a  pai q8
i1 P `8 .

1.2.7. Additional Exercises.

Exercise 1.2.21. [Direct sum of two normed spaces] Let X and Y be
two normed spaces. Consider their direct (Cartesian) product
X `1 Y  tpx, yq : x P X, y P Y u.
Show that X `1 Y is a normed space, with the norm defined as
}px, yq} : }x} }y}.
 1.3. BANACH SPACES 14

Exercise 1.2.22. [Minkowski functional] Consider a closed, convex,

origin-symmetric set K in Rn with nonempty interior. Minkowski func-
tional of K is the function defined on Rn by
(
}x}K : inf t ¡ 0 : x{t P K .

Show that }  }K is a norm on Rn , and the unit ball of this normed space
is K.

Exercise 1.2.23. [Seminorms] A seminorm on a linear vector space

E is a function }  } : E Ñ R which satisfies all norm axioms except the
second part of axiom (i). That is, there may exist nonzero vectors x for
which }x}  0.
Show that one can convert a seminorm into a norm by factoring out
the zero directions. Mathematically, show that kerppq : tx P E : }x}  0u
is a linear subspace of E. Show that the quotient space E { kerppq is a
normed space, with the norm defined as

}rxs} : }x} for x P E.

Illustrate this procedure by constructing the normed space L8 from

the semi-normed space of all essentially bounded functions, with the
essential sup-norm.

Exercise 1.2.24. [Convex hull] The convex hull of a subset A of a linear

vector space X is defined to be the smallest convex set that contains A.
The convex hull of A is denoted convpAq. (See the picture for an example).

Figure 1.2. The shaded region is the convex hull convpx1 , . . . , x5 q

A convex combination of vectors x1 , . . . , xn in a linear vector space X

is any vector of the form
¸
n
x λ k xk

k 1
°n
where λk ¥ 0 are some numbers such that k1 λk  1. Prove that convpAq
coincides with the set of all convex combinations of a finite number of
vectors from A.
 1.3. BANACH SPACES 15

1.3. Banach spaces

1.3.1. Definition. Completeness of C pK q. It turns out that the concept of
Lec.5: 09/17/10

normed space is deficient; many results in analysis can not be obtained just based
on the norm axioms. An additional axiom is needed, which is completeness.
Recall that a metric space X is called complete if every Cauchy sequence in X
converges to a point in X. For example, R is a complete metric space while Q is
not.
Specializing to normed spaces X, recall that a sequence pxi q8
i1 in X is Cauchy
if
}xn
xm } Ñ 0 as n, m Ñ 8,
i.e. for every ε ¡ 0 there exists N  N pεq such that
}xn
xm }   ε for all n, m ¡ N.
Definition 1.3.1 (Banach space). A complete normed space X is called a
Banach space.
Exercise 1.3.2. [Subspaces, quotients of Banach spaces] Let X be a
Banach space and Y be a (linear) subspace of X. Show that:
1. Y is a Banach space if and only if Y is closed.
2. If Y is closed8, then X {Y is a Banach space.
Many classical spaces are Banach spaces.
Theorem 1.3.3. For a compact topological space K, C pK q is a Banach space.
Proof. Most proofs of completeness work out by reducing the problem to the
completeness of R. To this end, consider a Cauchy sequence pfn q in C pK q, that is
(1.7) }fn
fm }8 Ñ 0, n, m Ñ 8.
Therefore, for every t P K, we have |fn ptq
fm ptq| Ñ 0. In other words, pfn ptqq
is a Cauchy sequence in R for every t. By completeness of R, this sequence has a
limit which we call f ptq. We have constructed a function f ptq such that fn Ñ f
pointwise.
We now claim that fn Ñ f uniformly, i.e. }fn
f }8 Ñ 0. This would
complete the proof, since the limit of a uniformly convergent sequence of continuous
functions on the compact space K is a continuous function (as we know from the
undergraduate analysis). Let us prove our claim then. By (1.7), for every ε there
exists N  N pεq such that
|fn ptq
fm ptq|   ε for all n, m ¡ N, t P K.
Letting m Ñ 8 in this inequality (while keeping everything else fixed), we conclude
that
|fn ptq
f ptq|   ε for all n, m ¡ N, t P K.
This means that }fn
f }8 Ñ 8, which is what we wanted. 
Exercise 1.3.4. [Banach spaces `8 , c0 , L8 ]
1. Show that `8 and L8 are Banach spaces. (Hint: modify the proof of
Theorem 1.3.3.)
2. Show that c0 is a Banach space (Hint: use Exercise 1.2.16.)

{
8Recall that Y has to be closed in order for X Y to be well defined.
 1.3. BANACH SPACES 16

3. Show that the set of functions

tf P C r0, 1s : f p0q  f p1q  0u
is a Banach space with respect to the sup-norm. (Hint: check that
this is a closed subspace of C r0, 1s.)
4. Show that the set of polynomials P pxq restricted to r0, 1s is not a
Banach space with respect to the sup-norm. (Hint: recall that P pxq
is dense in C r0, 1s by Weierstrass approximation theorem.)
5. Show that c00 is not a Banach space with respect to any }  }p norm,
1 ¤ p ¤ 8. (Hint: show that c00 is dense in `p for p P r1, 8q, and is not
closed in `8 .)
Lec.6: 09/20/10
1.3.2. Series in Banach spaces. Completeness of Lp . We are going to
give a useful criterion of completeness of normed spaces in terms of convergence of
series rather than sequences. We shall use this criterion to prove the completeness
of the Lp spaces.
Definition 1.3.5. Series Let pxk q be a sequence of vectors in a normed space
X. If the partial sums
¸
n
sn : xk
k 1  °
converge to some vector x P X as n Ñ 8, then we say that the series k xk
converges in X, and we write
8̧
xk  x.
°
k 1
A series k xk is called absolutely convergent if
8̧
}xk }   8.

k 1

Recall that in the scalar case, where X  R or C, absolute convergence of

series implies convergence (but not vice versa). As the following theorem shows,
this happens precisely because of the completeness of R and C.
Theorem 1.3.6 (Completeness criterion). A normed space X is a Banach space
if and only if every absolutely convergent series in X converges in X.
Proof. 1. Necessity. Let X be a Banach space, and consider an absolutely
convergent series, i.e.
8̧
(1.8) }xk }   8.

k 1
°
We want to prove that the series k xk converges. By completeness of X, it suffices
to show that the partial sums of this series are Cauchy, i.e. that }sn
sm } Ñ 0 as
n ¡ m Ñ 8. To this end, we use triangle inequality and our assumption (1.8) to
obtain
 ¸
m  ¸
m
}sn
sm }   xk  ¤

}xk } Ñ 0.

k n 1 
k n 1
This completes the proof of necessity.
 1.3. BANACH SPACES 17

2. Sufficiency. Assume that X in incomplete; we want to construct a divergent

series which is absolutely convergent. By incompleteness, there exists a Cauchy
sequence pvn q in X which diverges.
Every subsequence of pvn q diverges (check!). Therefore, there exists a subse-
quence pwn q of pvn q which diverges but which is “rapidly Cauchy”, i.e.

(1.9) }w2
w1 } ¤ 21 , }w3
w3 } ¤ 212 , }w4
w4 } ¤ 213 , . . .

(Construct it!) It follows that the sequence pxk q defined as
x1 : w2
w1 , x2 : w3
w2 , x3 : w4
w3 , . . .
forms the terms of an absolutely convergent series:
8̧
}xk } ¤ 12 1
22
1
23
    1.

k 1

Nevertheless, the partial sums

¸
n
xk  wn 1
w1
k 1 
diverge. So we have constructed an absolutely convergent series in X which di-
verges. This completes the proof. 
Exercise 1.3.7. Validate the two missing steps in the proof of Theo-
rem 1.3.6. Let Xn be a normed space.
1. Let pvn q be a Cauchy sequence X which diverges. Prove that every
subsequence of pvn q diverges.
2. Let pvn q be a Cauchy sequence in X. Construct a “rapidly Cauchy”
subsequence pwn q of pvn q, i.e. one that satisfies (1.9).

Theorem 1.3.8. For every p P r1, 8q, the space Lp  Lp pΩ, Σ, µq is a Banach
space.
Proof. Let functions pfn q in Lp form the terms of an absolutely convergence
series, i.e.
8̧
}fk }p : M   8.
k 1
By
° the completeness criterion, Theorem 1.3.6, it suffices to show that the series
k fk converges in Lp . °n
Case 1: all fk ¥ 0 pointwise. The partial sums k1 fk form a pointwise non-
°8
decreasing sequence of functions. Denote the pointwise limit by k1 fk ; it may
be infinite at some points.
The triangle inequality (which is Minkowski’s inequality) implies that the par-
tial sums are bounded:
¸n  ¸
n



fk  ¤ }fk }p ¤ M,

k 1
p
k 1 
which in other words is »
¸
n p
fk dµ ¤ M p .
Ω 
k 1
 1.3. BANACH SPACES 18

We
°napplyLebesgue monotone convergence theorem for the sequence of functions
p
k 1 f k and get
»
¸
n p »
8̧ p
fk dµ Ñ fk dµ.
Ω 
k 1 Ω k 1 
The right hand ° side exists and is finite since the left hand side is bounded by M p
8
for all n. Thus k1 fk P Lp .
°8
It remains to check that the series k1 fk P Lp converges not only pointwise
but also in Lp . By the pointwise convergence and positivity, the tails of this series
8̧
rn : fk

k n 1

form a pointwise non-increasing sequence of positive functions in Lp that pointwise

converge to 0. Applying Lebesgue monotone convergence theorem °8 for the sequence
rnp , we conclude that }rn }p Ñ 0. We conclude that the series k1 fk converges in
Lp as required.
Case 1: arbitraty fk . As is usual in the theory of Lebesgue integration, the claim
for arbitrary (not necessarily non-negative) functions fk will follow by decomposing
them into positive and negative parts:
fk : maxpfk , 0q, fk
: maxp
fk , 0q pointwise.
Then fk  fk
fk
. Since 0 ¤ fk ¤ |fk |, we have
8̧ 8̧
}fk }p ¤ }fk }p   8.

k 1 
k 1
°8
So, by the first part of the argument, the series k1 fk converges in Lp . Similarly
°8 °8 °8 °8
we show that k1 fk
converges in Lp . Therefore, k1 fk  k1 fk
k1 fk

converges in Lp . This completes the proof. 
Lec.7: 09/22/10
1.3.3. Completion. From a course in point-set topology we know that every
metric space can be turned into a complete metric space. This procedure is called
completion. We will illustrate it in the context of normed spaces.
Theorem 1.3.9 (Completion). Let X be a normed space. There exists a Ba-
nach space X,p called the completion of X, with the following properties. One can
find a linear map ι : X Ñ Xp such that:
(i) }ιx}  }x} for all x P X (i.e. ι is an isometric embedding of X into X;
p
(ii) Impιq is dense in X.
p
The completion of X is unique up to an isometry.9
Exercise 1.3.10. Prove Theorem 1.3.9. Mimic the construction of R
from Q, or the more general construction of completion of metric spaces.
Namely, consider the space Y of all Cauchy sequences pxk q in X, equipped
with the semi-norm
}pxk q} : lim }xk }. k

p 1 , there exists an injective and bijective

9Precisely, this means that for any other completion X
linear map T : X p Xp 1 such that T x
Ñ } }} } x for all x X. P
 1.4. INNER PRODUCT SPACES 19

Then turn this space into a normed space Xp by taking quotient over the
kernel of the semi-norm (see Exercise 1.2.23).
The concept of completion suggests an alternative construction of the space
Lp ra, bs, p P r1, 8q. From measure theory we know that the set of continuous
functions is a dense subset of Lp ra, bs (why?). Furthermore, Lp ra, bs is a com-
plete normed space. Therefore (by the uniqueness of completion), Lp ra, bs is the
completion of the space C ra, bs in the }  }p norm.
This is an alternative definition of Lp ra, bs. This gives also an alternative con-
struction of Lebesgue integral. Indeed, on for continuous functions, Riemann and
Lebesgue integrals coincide. Therefore, the space C ra, bs with }  }1 can be con-
structed using only Riemann integral. But its completion gives rise to Lebesgue
integral.

1.3.4. Additional Exercises.

Exercise 1.3.11. [Space of continuous periodic functions] Show that
the set of functions
tf P C r0, 1s : f p0q  f p1qu
is a Banach space with respect to the sup-norm. (Hint: identify this
space with C pTq where T is a one-dimensional torus.)

Exercise 1.3.12. [Space of continuously differentiable functions] Show

that the space C k r0, 1s of k-times differentiable functions is not a Banach
space with respect to the sup-norm.
Show that C k r0, 1s is a Banach space with respect to the norm
}f }C  }f }8 }f 1 }8    }f pkq }8 .
k

Exercise 1.3.13. [Completeness of a direct sum] Let X and Y be Ba-

nach spaces. Show that the direct sum X `1 Y defined in Exercise 1.2.21
is a Banach space.

1.4. Inner product spaces

1.4.1. Definition. Cauchy-Schwarz inequality. Hilbert spaces form an
important and simplest class of Banach spaces. Speaking imprecisely, Hilbert spaces
are those Banach spaces where the concept of orthogonality of vectors is defined.
Hilbert spaces will arise as complete inner product spaces.
Definition 1.4.1 (Inner product space). Let E be a linear space over C. An
inner product on E is a function x, y : E  E Ñ C which satisfies the following
three axioms:
(i) xx, xy ¥ 0 for all x P E; xx, xy  0 if and only if x  0;
(ii) xax by, z y  axx, z y bxy, z y for all x, y, z P E and a, b P C;
(iii) xx, y y  xy, xy for all x, y P E.
The space E with an inner product is called an inner product space.
Inner products over R are defined similarly, except there is no conjugation in
axiom (iii).
 1.4. INNER PRODUCT SPACES 20

Remark 1.4.2. The inner product is (congjugate) linear in the second argu-
ment:
xx, ay bzy  axx, yy bxx, zy.
This follows from axioms (ii) and (iii) of the inner product.
Definition 1.4.3 (Orthogonality). If xx, y y  0 we say that vectors x and y
are orthogonal and write x K y.
Example 1.4.4. The canonical example of a finite-dimensional Hilbert space
is the Euclidean space Cn equipped with the inner product
¸
n
xx, yy  xk yk .

k 1

Theorem 1.4.5 (Cauchy-Schwarz inequality). Let X be an inner product space.

Then every two vectors x, y P X satisfy
|xx, yy| ¤ xx, xy1{2 xy, yy1{2 .
Proof. Case 1: xx, yy P R. For all t P R, by the axioms of the inner product
we have:
0 ¤ xx ty, x ty y  t2 xy, y y 2txx, y y xx, xy.
A quadratic polynomial that is everywhere non-negatve must have a non-positive
discriminant, i.e.
xx, yy2
xx, xyxy, yy ¤ 0.
This is precisely Cauchy-Schwarz inequality.
Case 2: xx, y y P C arbitrary. We will multiply y by a unit scalar so that xx, y y
becomes a real number, and use the first part. Indeed, polar decomposition
xx, yy  |xx, yy|ei Argxx,yy
implies that
|xx, yy|  xx, y1 y
where y 1  ei Argxx,yy y.
Now using the first part of the proof we conclude that
|xx, yy|  xx, y1 y ¤ xx, xyxy1 , y1 y ¤ xx, xyxy, yy
as required. 
Corollary 1.4.6 (Norm in an inner product space). Let X be an inner product
space. Then X is a normed space, with the norm defined as
}x} : xx, xy1{2 .
Proof. Of the three norm axioms, only the triangle inequality is non-trivial.
Let us check it. For x, y P X we have by Cauchy-Schwarz inequality that
}x y }2  xx y, x yy  xx, xy xx, yy xy, xy xy, yy
 }x}2 2 Rexx, yy }y}2 ¤ }x}2 2|xx, yy| }y}2
 p}x} }y}q2 .
This completes the proof. 
Remark 1.4.7 (Pythagorean theorem). The calculation above clearly implies
Pythagorean theorem: if x K y then
}x y }2  }x}2 }y}2 .
 1.4. INNER PRODUCT SPACES 21

Remark 1.4.8 (Angle between vectors). The concept of inner product makes
it possible to define the angle between two vectors x, y in an inner product space
X. Recall that in Euclidean space Rn , the inner product can be computed by the
formula
xx, yy  }x}}y} cos θpx, yq
where θpx, y q is the angle between x and y. Therefore, in a general inner product
space X, it makes sense to define the angle between x, y by

cos θpx, y q 
xx, yy .
}x}}y}
Cauchy-Schwarz inequality guarantees that the right hand side lies in r
1, 1s, so
the angle exists. Nevertheless, the concept of angle is rarely used; one prefers to
work with inner product directly.
Lec.8: 09/24/10

1.4.2. The space L2 . Hölder’s inequality. The basic example of an inner

product space is L2  L2 pΩ, Σ, µq:
Proposition 1.4.9 (Canonical inner product on L2 ). For f, g P L2 , the quan-
tity »
xf, gy : f ḡ dµ
is finite, and it defines an inner product on L2 . This inner product obviously agrees
with the L2 norm, i.e. }f }2  xf, f y1{2 .
Proof. The only non-trivial fact to prove is that xf, g y is finite, i.e. that
f ḡ is integrable. Since f , ḡ and f ḡ belong to L2 , we have that f 2 , ḡ 2 and
pf ḡq  f f ḡ ḡ are integrable. Hence f ḡ is integrable, as required. 
2 2 2

We can recast Cauchy-Schwarz inequality in this specific space L2 as follows.

Corollary 1.4.10 (Cauchy-Schwarz inequality in L2 ). For every f, g P L2 one
has » 
» 1{2
» 1{2
 
 f ḡ dµ ¤ |f |2 dµ |g|2 dµ .

The left
³ hand side of Cauchy-Schwarz inequality can be replaced by the larger
quantity |f g | dµ. (This can be seen by applying Cauchy-Schwarz inequality for
|f |, |g|.) Thus Cauchy-Schwarz inequality can be written as
}f g}1 ¤ }f }2 }g}2 .
Cauchy-Schwarz inequality is a partial case of the more general Hölder’s in-
equality:
Corollary 1.4.11 (Hölder’s inequality for functions). Let p, q P p1, 8q be
adjoint, i.e. p1 1q  1. Then for every f P Lp and g P Lq one has
» 
» 1{p
» 1{q
 
 f ḡ dµ ¤ |f |
p
dµ |g|q dµ .

Before proving this result, note that similarly to Cauchy-Schwarz inequality,

the
³ left hand side in Hölder’s inequality can be replaced by the larger quantity
|f g| dµ. Thus Hölder’s inequality can be written as
(1.10) }f g}1 ¤ }f }p }g}q .
 1.4. INNER PRODUCT SPACES 22

Proof of Corollary 1.4.11. We are going to prove inequality (1.10). By

rescaling, we can assume that }f }p  }g }q  1 (In order to see this, divide both
sides by }f }p and }g }q ). We shall use Young’s inequality
ap bq
ab ¤for a, b ¥ 0,
p q
which follows by taking logarithm of both sides and using the concavity of the
function logpxq. Therefore, we have the pointwise inequality

|f ptqgptq| ¤ |f pptq| |gptq|q

p
for all t P Ω.
q
Integrating yields »
|f g| ¤ p1 1
q
1
as required. 
Using Hölder’s inequality we can clarify the scale of spaces Lp  Lp pΩ, Σ, µq
for various p. Assume that µ is a finite measure (this is important!) Then L8 is
the smallest space, L1 is the largest, and all other Lp , p P r1, 8q lie in between:
Corollary 1.4.12 (Scale of Lp spaces). Let pΩ, Σ, µq is a probability space,
and 1 ¤ r ¤ s ¤ 8. Then
}f }r ¤ }f }s for all f P Ls  Ls pΩ, Σ, µq.
In particular, we have the inclusion
Ls „ Lr .
Proof. For s  8 this inequality is clearly true, so let s   8. We apply
Hölder’s inequality for |f |r and the constant function 1, and with p  s{r:
» »
» r{s
» 1{q
}f }rr  |f |r dµ  |f |r  1 dµ ¤ |f |rs{r 1q dµ

» r{s
 |f |s µpΩq  }f }rs .

This completes the proof. 

1.4.3. The space `2 . Recall that the space of square-summable sequences `2
is a particular case of L2  L2 pΩ, Σ, µq for the counting measure µ on Ω  N.
Therefore, `2 is also an inner product space. The inner product formula reads as
8̧
xx, yy  xi yi for x  pxi q, y  pyi q P `2 .

i 1
Let us write out Cauchy-Schwarz inequality in this case:
Corollary 1.4.13 (Cauchy-Schwarz inequality in `2 ). For every two sequences
x  pxi q and y  pyi q in `2 one has
 8̧ 
8̧ 1{2
8̧ 1{2
xi yi  ¤ |xi |2 |yi |2
 
 .

i 1 
i 1 
i 1
°8
As before, the left hand side can be replaced by the larger quantity  |xi yi |.
i 1
Similarly, Hölder’s inequality in this case takes the following form:
 1.4. INNER PRODUCT SPACES 23

Corollary 1.4.14 (Hölder’s inequality for sequences). Let p, q P p1, 8q be

adjoint, i.e. p1 1q  1. Then for every x  pxi q P `p and y  pyi q P `q one has
8̧
8̧ 1{p
8̧ 1{q
|xi yi | ¤ |xpi |yiq .

i 1 
i 1 
i 1

However, Corollary 1.4.12 on the scale of Lp (on finite measure spaces) does
not hold for `p , because the counting measure on N is not finite. In fact, the scale
is completely reversed in this case: `1 is the smallest space, `8 is the largest, and
the other `p , p P r1, 8q lie in between:
Corollary 1.4.15 (Scale of `p spaces). Let 1 ¤ r ¤ s ¤ 8. Then
}x}s ¤ }x}r for all x P `r .
In particular, we have the inclusion
`r „ `s .
Exercise 1.4.16. Prove Corollary 1.4.15. You don’t have to use Hölder’s
inequality.

1.4.4. The space of matrices. For arbitrary fixed dimensions m, n P N, we

consider the space of matrices
Mm,n : tall m  n matrices with complex entriesu.
One can turn Mm,n into an inner product space by defining the trace inner product
as
¸
m ¸
n
(1.11) xA, B y : trpAB  q  aij bij .
 
i 1j 1

This is clearly an inner product. One way to see this is to identify Mm,n with Cmn
by concatenating the rows of a matrix A P Mm,n into a long vector in Cmn . Then
the canonical inner product in Cmn is the same as the right hand side of (1.11).
The norm defined by the inner product on Mm,n is called Hilbert-Schmidt or
Frobenius norm of matrices:

¸
m ¸
n 1{2
(1.12) }A}HS  xA, Ay1{2  |aij |2
 
i 1j 1

Note³ some similarity between the forms of the inner product in L2 , which is
xf, gy  f ḡ dµ and in Mm,n , which is xA, B y : trpAB  q – the integral is replaced
by the trace, functions by matrices, complex conjugation by transposition, and
product of functions by product of matrices.

1.4.5. A space of random variables. The space L2 arises in probability

theory in a natural way as a space of random variables with finite variance. Indeed,
consider a probability space pΩ, Σ, Pq. Recall that a random variable X is a mea-
surable real-valued function defined on Ω. The expectation of X is, by definition,
the integral of X: »
EX  X pω q dPpω q.
 1.4. INNER PRODUCT SPACES 24

Therefore, the space L2 pΩ, Σ, Pq consists of all random variables X with finite
second moment:
}X }2  pEX 2 q1{2   8.
Exercise 1.4.17. Show that X P L2 pΩ, Σ, Pq if and only if X has finite
variance:
VarpX q  EpX
EX q2  EX 2
pEX q2   8.
The concepts of covariance and correlation coefficient have some geometric
meaning, too. Consider two random variables X and Y , and for simplicity assume
that they have mean zero, i.e. EX  EY  0. Then the covariance of X and Y is
nothing else than the inner product in L2 :
covpX, Y q : EpX
EX qpY
EY q  EXY  xX, Y y.
Similarly, the correlation coefficient between X and Y is
covpX, Y q
corrpX, Y q :
VarpX q1{2 VarpY q1{2
 pEX 2 qEXY
1{2 pEY 2 q1{2
 }XxX, Yy
} }Y } .
2 2

Hence the correlation coefficient is nothing else as the cosine of the angle between
random variables X and Y considered as vectors in L2 (see Remark 1.4.8). This
demonstrates the geometric meaning of correlation – the more random variables X
and Y are correlated, the less the angle between them, and vice versa. Lec.9: 09/27

1.4.6. Parallelogram law. The parallelogram law in planar geometry states

that for every parallelogram, the sums of squares of the diagonals equals the sum of
squares of the sides. This statement remains to be true in all inner product spaces:
Proposition 1.4.18 (Parallelogram law). Let X be an inner product space.
Then for every x, y P X one has
(1.13) }x y }2 }x
y}2  2}x}2 2}y }2 .
Proof. The result follows once we recall that }x y }2  xx y, x yy 
}x} 2 Rexx, yy }y}2 and similarly }x
y}2  }x}2
2 Rexx, yy }y}2 .
2


The parallelogram law characterizes inner product spaces. First recall that in
inner product spaces, the inner product determines the norm (}x}  xx, xy1{2 ).
Vice versa, the inner product is uniquely determined by the norm, and it can be
reconstructed through the polarization identity:
Proposition 1.4.19 (Polarization identity). Let X be an inner product space.
Then for every x, y P X one has


(1.14) xx, yy  14 }x y }2
}x
y }2 i}x iy }2 i}x
y }2 .

Exercise 1.4.20. Prove the polarization identity.

The parallelogram law characterizes inner product spaces:
Theorem 1.4.21 (Characterization of inner product spaces). Let X be a normed
space over C such that every two vectors x, y P X satisfy the parallelogram law
(1.13). Then X is an inner product space. Precisely, polarization formula (1.14)
defines the inner product on X that agrees with the norm, i.e. }x}  xx, xy1{2 .
 1.5. HILBERT SPACES 25

The proof of this result is deferred to the exercises for this section.
It follows from Theorem 1.4.21 that being an inner product space is a “local”
property, since checking the parallelogram law involves just two (arbitrary) vectors.
In particular, if all two-dimensional linear subspaces of a normed space X are inner-
product spaces (with respect to some inner product, possibly different for each
subspace), then X is an inner product space (and there the inner product on all
subspaces is actually the same, induced from X!)

1.4.7. Additional Exercises.

Exercise 1.4.22. [Direct sum of inner product spaces] Let X, Y be
inner product spaces. Show that their direct sum
X `2 Y : tpx, yq : x P X, y P Y u
is also an inner product space, with the inner product defined as
xpx1 , y1 q, px2 , y2 qy  xx1 , x2 y xy1 , y2 y.
Derive a formula for the norm in X `2 Y . Show that if X, Y are Hilbert
spaces then so is X `2 Y .

Exercise 1.4.23. Show that the inner product is a continuous function

on the direct sum X `2 X of a Hilbert space X with itself.

Exercise 1.4.24. [Characterization of inner product spaces] Prove The-

orem 1.4.21. For simplicity, do this for spaces over R. (Formulate the
polarization identity in this case). Axioms (i) and (iii) of inner product
are simple to check. To prove axiom (iii) (linearity), you may follow this
sequence of steps:
(i) Show that xx1 x2 , y y xx1
x2 , y y  2xx1 , y y for all x1 , x2 , y P X.
(ii) Deduce the additivity property: xu, y y xv, y y  xu v, y y for all
u, v, y P X.
(iii) From additivity property, deduce the multiplicativity property xtx, y y 
txx, y y for all x, y P X, t P R. Indeed, for t P N this follows by induc-
tion; deduce the same for all t P Q and extend to all t P R by
continuity.

Exercise 1.4.25. Show that C pK q, c0 , Lp r0, 1s, `p for p P r1, 8s, p  2, are
not inner product spaces. (More accurately, it is not possible to define
an inner product on those spaces which would agree with their norms).
Use Theorem 1.4.21.

1.5. Hilbert spaces

1.5.1. Definition. Orthogonal complements. As in the case of normed
spaces, we will really need an extra axiom – completeness – in order to study
geometry of inner product spaces.
Definition 1.5.1. A complete inner product space is called a Hilbert space.
 1.5. HILBERT SPACES 26

Example 1.5.2. As we know, L2 pΩ, Σ, µq is a Hilbert space. In particular,

`2 is a Hilbert space. Also, Cn and Rn are Hilbert spaces; therefore the space of
matrices Mm,n is also a Hilbert space.
The geometry of Hilbert spaces is dominated by the concept of orthogonality.
We will exploit it in detail now.
Definition 1.5.3 (Orthogonal complement). Let A be a subset of an inner
product space X. The orthogonal complement of A is defines as
AK : tx P X : xx, ay  0 for all a P Au.
Proposition 1.5.4. Let A be a subset of an inner product space X. Then AK
is a closed linear subspace of X. Moreover,
AK X A „ t0u.
Proof. It is easy to check that AK is a linear subspace of X. To show that
AK is a closed set, express it as
£
AK  tauK .
aPA
K
Therefore it suffices to check that tau is a closed set for every a P A. So we fix
10

a P A and consider a sequence xn P tauK such that xn Ñ x for some x P X. We

would like to show that x P tauK . To this end, notice that continuity of the inner
product (Exercise 1.4.23) implies that
0  xxn , ay Ñ xx, ay.
Hence xx, ay  0 as required.
Finally, to show that AK X A „ t0u, consider x P AK X A; it follows that
xx, xy  0 which implies x  0. The proof is complete. 
1.5.2. Orthogonality principle. Orthogonal decompositions.
Theorem 1.5.5 (Orthogonality principle). Let Y be a closed linear subspace of
a Hilbert space X, and let x P X. Then the following holds.
(i) There exists a unique closest point y P Y to x, i.e. such that
}x
y}  ymin 1
1 PY }x
y }.
(ii) The point y is the unique vector in Y such that
x
y P Y K.
The point y is called the orthogonal projection of x onto the subspace Y .
Proof. (i). Existence. Denote the distance by
d : inf
y 1 PY
}x
y1 }.
Let us choose a sequence pyn q in Y which satisfies
}x
yn } Ñ d.
Since Y is closed, it is a Banach space. Therefore it suffices to show that pyn q is a
Cauchy sequence. (By continuity of the norm it would follow for y  limn yn that
}x
y}  d.)
10Recall that the intersection of closed sets is closed.
 1.5. HILBERT SPACES 27

To bound }yn
ym }, we use parallelogram law. We apply it for the parallelogram
with vertices x, yn , ym (and whose fourth vertex is determined by these three, see
the picture.)

Figure 1.3. Proof of Theorem 1.5.5

Parallelogram law then yields

}yn
ym }2 4}x

1
2
pyn ym q}2  2}x
yn }2 2}x
ym }2 .

By definition of d, we have }x
12 pyn ym q} ¥ d, and by construction we have

}x
yn } Ñ d, }x
ym } Ñ d. With this, we conclude that
0 ¤ lim inf }yn
ym }2 ¤ 2d2 2d2
4d2  0.
Therefore pyn q is a Cauchy sequence as required.
Uniqueness. Suppose there existed two different closest points y1 , y21 for x.
Then the alternating sequence py1 , y2 , y1 , y2 , . . .q would not be Cauchy, contradict-
ing the argument above. Part (i) of Theorem is proved. Lec.10: 09/29

(ii). Orthogonality. Assume that x
y R Y K , so

xx
y, y1 y  0 for some y 1 P Y.
By multiplying y 1 by an appropriate complex scalar, we can assume that xx
y, y 1 y
is a real number. We will show that by moving y in the direction of y 1 , one can
improve the distance from x, which will contradict the definition of y. Namely, for
every t P R the definition of y implies that
}x
y}2 ¤ }x
y ty 1 }2  }x
y}2 2txx
y, y 1 y t2 }y 1 }2 .
This implies that the quadratic polynomial in t satisfies
}y1 }2 t2 2xx
y, y1 yt ¥ 0 for all t P R.
This can only happens if xx
y, y 1 y  0, which contradicts our assumption.
Uniqueness. Suppose there are two vectors y 1 , y 2 P Y which satisfy x
y 1 P Y K ,
x
y 2 P Y K . Since Y K is a linear subspace, subtracting yields that y 1
y 2 P Y K .
But Y is also a linear subspace, so y
y 1 P Y . Since Y X Y K  t0u, it follows that
y
y 1  0, hence y 1  y 2 . Theorem is proved. 
 1.6. FOURIER SERIES 28

In the proof of part (i) of Theorem 1.5.5, we used convexity rather than linearity
of Y . (Indeed, we needed that together with two points yn , ym P Y their midpoint
2 pyn ym q is contained in Y ). Therefore, our argument implies the following more
1

general result:
Theorem 1.5.6 (Hilbert’s projection theorem). Given a closed convex set Y
in a Hilbert space X and a point x P X, there exists a unique closest point y P Y .
The map that takes x into the closest point y is called a projection onto convex
set Y and is abbreviated POCS. This map appears in several applied fields.
The orthogonality principle immediately implies that a Hilbert space X can be
decomposed into the orthogonal sum of a subspace Y and its complement Y K :
Corollary 1.5.7 (Orthogonal decomposition). Let X be a Hilbert space and
Y be a closed subspace. Then every vector x P X can be uniquely represented as
xy z, y P Y, z P Y K.
This orthogonal decomposition is usually abbreviated as
X  Y ` Y K.
Definition 1.5.8 (Orthogonal projection). In the setting of Corollary 1.5.7,
the map
PY : X Ñ X, PY x  y
is called the orthogonal projection in X onto Y .
Exercise 1.5.9. Show that the orthogonal projection PY is a linear
map. Check that ImpPY q  Y and kerpPY q  Y K . Also check that the
identity map IX on X can be decomposed as
IX  PY PY K .

1.5.3. Additional Exercises.

Exercise 1.5.10. Let A be a subset of a Hilbert space. Show that
AK  ĀK
where Ā denotes the closure of A.

Exercise 1.5.11. [Projection onto constants] Let Y denote the sub-

space of constant functions in L2  L2 pΩ, Σ, µq. Compute PY f for an
arbitrary function f P L2 .

1.6. Fourier series

Hilbert space gives a geometric framework for studying Fourier series. The
classical Fourier analysis studies situations where a function f : r
π, π s Ñ C can
be expanded as Fourier series
8̧
f ptq  fppk qeikt
k 
8
 1.6. FOURIER SERIES 29

with Fourier coefficients

»π
fppk q  f ptqe
ikt dt
1
(1.15)
2π
π
are called the Fourier coefficients of f . In order to make Fourier analysis rigorous,
one has to understand what functions f can be written as Fourier series, and in
what sense the Fourier series converges.
In order to do so, it is of great advantage to depart from this specific situation
and carry out Fourier analysis in an abstract Hilbert space. We will regard the
function f ptq as a vector in the function space L2 r
π, π s. The exponential functions
e
ikt will form a set of orthogonal vectors in this space. Fourier series will then
become an orthogonal decomposition of a vector f with respect to an orthogonal
system of coordinates.
1.6.1. Orthogonal systems.
Definition 1.6.1 (Orthogonal system). A sequence pxk q in a Hilbert space X
is called an orthogonal system if
xxk , xl y  0 for all k  l.
If additionally }xk }  1 for all k, the sequence pxk q is called an orthonormal system.
Equivalently, pxk q is an orthonormal system if
xxk , xl y  δkl
where δkl equals 1 if k  l and 0 otherwise (it is called Kronecker’s delta function).
Example 1.6.2 (Canonical basis of `2 ). In the space `2 , consider the vectors
xk  p0, . . . , 0, 1, 0, . . .q
whose all coordinates are zero except the k-th equals 1. The sequence pxk q8
k1 is
clearly an orthonormal system in `2 .
Example 1.6.3 (Fourier basis in L2 ). In the space11 L2 r
π, π s, consider the
exponentials

ek ptq  ? eikt , t P r
π, π s.

1
(1.16)
2π
8
Then pek qk
8 is an orthonormal system in L2 r
π, π s (check!).
Example 1.6.4 (Trigonometric system in L2 ). Closely related to the Fourier
basis is the trigonometric system. Note that we can write the exponentials from
the previous example as

fk ptq  ? cospktq i sinpktq .
1
2π
Considering the real and imaginary parts separately, we see that the system
! 1 )
? , ?1π cosptq, ?1π sinptq, ?1π cosp2tq, ?1π sinp2tq, . . .
2π
is an orthonormal system in L2 r
π, π s (check!)
11The space L
2r
s p q
π, π can be identified with L2 T where T is the unit torus in C. We can
think of elements of this space as 2π-periodic functions.
 1.6. FOURIER SERIES 30

Exercise 1.6.5. Prove that an orthogonal system is a linearly inde-

pendent set.

1.6.2. Orthogonal series. The main interest in orthogonal systems is that

they allow us to form orthogonal expansions of every vector x P X. Such expansions
are infinite series. So our first task will be to clarify when orthogonal series converge.
Theorem 1.6.6 (Convergence of orthogonal series). Let pxk q be an orthogonal
system in a Hilbert space X. Then the following are equivalent:
°
(i) °k xk converges in X;
(ii) °k }xk }2   8;
(iii) k xk converges unconditionally in X, i.e. for every reordering of terms.
In case of convergence, we have
 ¸ 2 ¸
(1.17)



xk   }xk }2 .
k k

The proof of this result is based on its finite version, which may be called the
Pythagorean theorem in higher dimensions:
Lemma 1.6.7 (Pythagorean theorem). Let pxk q be an orthogonal system in a
Hilbert space X. Then for every n P N one has
¸n 2 ¸
n
(1.18)



xk   }xk }2 .

k 1 
k 1

Proof. Using orthogonality, we see that the left hand side of (1.18)
A¸
n ¸
n E ¸
n ¸
n
xk , xk  xxk , xl y  xxk , xk y

k 1 
k 1 
k,l 1 
k 1

which equals the right hand side of (1.18). 

° Proof of Theorem 1.6.6. (i) ô (ii). By the Cauchy criterion, the series
x
k k converges if and only if its partial sums form a Cauchy sequence in X, i.e.
 ¸
m 2
(1.19)



xk  Ñ0 as n, m Ñ 8.

k n

Note
°m that 2by Pythagorean theorem (Lemma 1.6.7), the quantity in (1.19) equals
kn }xk } . So using Cauchy°
criterion again we see that (1.19) is equivalent to the
convergence of the series k }xk }2 , as°
required.
(ii) ñ (iii). The scalar series k }xk }2 converges absolutely, therefore also
unconditionally (as we know from an analysis
° course). Hence, by the equivalence
of (i) and (ii) proved above, the series k xk converges unconditionally.
(iii) ñ (i) is trivial.
The last part of the theorem, identity (1.17), follows by taking limit in Pythagorean
identity (1.18). The theorem is proved. 
°8

8 ak e
ikt
Exercise 1.6.8. For what coefficients ak does the series
converge in L2 r
π, π s?
k
 1.6. FOURIER SERIES 31

Lec.11: 10/01
1.6.3. Fourier series.
Definition 1.6.9 (Fourier series). Consider an orthonormal system pxk q8
k1 in
a Hilbert space X and a vector x P X. The Fourier series of x with respect to pxk q
is the formal series ¸
xx, xk yxk .
k
The coefficients xx, xk y are called the Fourier coefficients of x.
In order to understand the convergence of Fourier series, we will first focus on
the finite case, and study the partial sums of Fourier series
¸
n
Sn pxq : xx, xk yxk .

k 1

Lemma 1.6.10. Sn pxq is the orthogonal projection of x onto12 Spanpx1 , . . . , xn q.

Proof. By the definition of the orthogonal projection (see the orthogonality
principle, Theorem 1.5.5), the lemma states that
x
Sn pxq P pSpanpx1 , . . . , xn qqK .
It suffices to check that x
Sn pxq K xk for all k  1, . . . , n. We have
xx
Sn pxq, xk y  xx, xk y
xSn pxq, xk y.
By definition of Sn pxq and orthonormality of pxk q we see that xSn pxq, xk y  xx, xk y.
Therefore we conclude that xx
Sn pxq, xk y  0 as required. 

Let us estimate the size of Sn pxq. Since x
Sn pxq K Sn , by Pythagorean

theorem we have }Sn pxq}2 }x
Sn pxq}2  }x}2 . Hence
}Sn pxq}2 ¤ }x}2 .
On the other hand, by Pythagorean theorem and orthonormality,
¸
n ¸
n
}Sn pxq}2  }xx, xk yxk }2  |xx, xk y|2 .

k 1 
k 1

Combining these two inequalities and letting n Ñ 8, we arrive at the following

result.
Theorem 1.6.11 (Bessel’s inequality). Let pxk q be an orthonormal system in
a Hilbert space X. Then for every x P X one has
¸
|xx, xk y|2 ¤ }x}2 .
k

This result along with the convergence criterion for orthogonal series, Theo-
rem 1.6.6, shows that Fourier series always converge.
Corollary 1.6.12. ° Let pxk q be an orthonormal system in a Hilbert space X.
Then the Fourier series k xx, xk yxk of every vector x P X converges in X.

12
°n Recall that the ( p
linear span of vectors x1 , . . . , xn is defined as Span x1 , . . . , xn q x 
k1 ak xk : ak P C .
 1.6. FOURIER SERIES 32

In general, Fourier series of x needs not converge to x (why?) Still we can

compute the point where it converges. Indeed, letting n Ñ 8 in Lemma 1.6.10
and interpreting the conclusion via the orthogonality principle (Theorem 1.5.5), we
conclude:
Theorem 1.6.13 (Optimality of Fourier series). Consider an orthonormal sys-
tem pxk q8
k1 in a Hilbert space X and a vector x P X.
°
(i) The Fourier series k xx, xk yxk is the orthogonal projection of x onto Spanpxk q
(the closure of the linear span). °
(ii) Among all convergent series of the form S  k ak xk , the approximation
error }x
S } is minimized by the Fourier series of x.
Exercise 1.6.14. Prove Theorem 1.6.13.

1.6.4. Orthonormal bases. Fourier expansions. It is now easy to identify

an extra condition so that the Fourier series of every vector x converges to x.
Definition 1.6.15 (Complete systems). A system of vectors pxk q in a Banach
space X is called complete if
Spanpxk q  X.
A complete orthonormal system in a Hilbert space X is called an orthonormal basis
of X.
Theorem 1.6.16 (Fourier expansions). Let pxk q be an orthonormal basis of a
Hilbert space X. Then every vector x P X can be expanded in its Fourier series:
¸
(1.20) x xx, xk yxk .
k
Consequently, Parseval’s identity holds:
¸
}x}2  |xx, xk y|2 .
k

Proof. The first part follows from the Optimality Theorem 1.6.13, since by
completeness the orthogonal projection onto Spanpxk q  X is the identity map in
X. Parseval’s identity follows from Fourier expansion (1.20), Pythagorean identity
(1.17) for orthogonal series, and the normalization condition }xk }  1. 
Exercise 1.6.17. Prove that Parseval’s identity holds for an orthonor-
mal system pxk q if and only if pxk q is complete. Therefore the equality
cases of Bessel’s inequality hold exactly when the system is complete.
Now we describe some classical examples of complete sets and orthonormal
bases.
Example 1.6.18 (Monomials). Weierstrass approximation theorem states that
the system of monomials ptk q8 k0 is a complete system in C r0, 1s. We claim that
this is also a complete system in L2 r0, 1s.
Indeed, C r0, 1s is dense in L2 r0, 1s. This means that for every f P L2 r0, 1s
and ε ¡ 0, there exists g P C r0, 1s such that }f
g }2 ¤ ε{2. By Weierstrass
approximation theorem, there exists h P Spanptk q8 k0 such that }g
f }8 ¤ ε{2.
Hence }g
f }2 ¤ }g
f }8 ¤ ε{2, so by triangle inequality we conclude that
}f
h}2 ¤ ε{2 ε{2  ε. We have proved that Spanptk q8k0 is dense in L2 r0, 1s as
required.
 1.6. FOURIER SERIES 33

Example 1.6.19 (Exponentials). By a general version of Weierstrass approx-

imation theorem (called Stone-Weierstrass theorem), the exponential monomials
peitk qkPZ is a complete system in C r
π, πs. Repeating the argument in Exam-
ple 1.6.18, we can check that this is also a complete system in L2 r
π, π s.
Therefore, the system of exponentials

xk ptq  ? eikt , k P Z,
1
(1.21)
2π
forms an orthonormal basis of L2 r
π, π s. Reformulating Theorem 1.6.16 in this
case, we obtain a basic result in classical Fourier analysis:
Theorem 1.6.20 (Classical Fourier series). Every function f P L2 r
π, πs can
be represented by its Fourier series
8̧ »π
f ptq  fppk qeikt , fppk q  f ptqe
ikt dt.
1
where
k 
8 2π
π
The coefficients fppk q are all finite; the Fourier series converges in L2 r
π, π s.
Example 1.6.21 (Trigonometric system). In a similar way we can show that
the trigonometric system considered in Example 1.6.4 is an orthonormal basis in
L2 r
π, π s (do this!) Therefore a version of Theorem 1.6.20 holds for the trigono-
metric system, and it reads as follows:
8̧  
f ptq  ak cospktq bk sinpktq
a0
2 
k 1

where » »
1 π 1 π
ak  f ptq cospktq dt, bk  f ptq sinpktq dt.
π
π π
π
This again holds for every function f P L2 r
π, π s; the coefficients ak , bk are all
finite, and the Fourier series converges in L2 r
π, π s.
Lec.12: 10/04

1.6.5. Gram-Schmidt orthogonalization. There is a general way of cre-

ating an orthonormal basis phk q in a Hilbert space X out of some other, possibly
non-orthogonal system pxk q. One orthogonalize the system pxk q one element at a
time. This procedure is called Gram-Schmidt orthogonalization.
So let us consider a linearly independent system of vectors pxk q8
k1 in X. We
define the system phk q8
k 1 inductively as follows:

h1  }xx1 } , hn 1  }PPn xxn 1 } , n  1, 2, . . .

1 n n 1

where Pn denotes the orthogonal projection in X onto Spanph1 , . . . , hn qK . Geomet-

rically, one “rotates” the new vector xn 1 so it becomes orthogonal to the vectors
hk constructed earlier, normalizes it, and calls hn 1 ; see the picture.
One can effectively compute the vectors Pn xn 1 used in this process. Indeed,
by Lemma 1.6.10, the orthogonal projection of a vector x onto Spanph1 , . . . , hn q is
the partial sum of Fourier series:
¸
n
Sn pxq  xx, hk yhk .

k 1
 1.6. FOURIER SERIES 34

Figure 1.4. Gram-Schmidt orthogonalization

Now, Pn pxq is the othogonal projection onto the orthogonal complement, so

¸
n
Pn x  x
Sn pxq  x
xx, hk yhk .

k 1
So
¸
n
Pn x n 1  xn 1
xxn 1 , hk yhk .

k 1

Proposition 1.6.22. Let pxk q be a linearly independent system in a Hilbert

space X. Then the system phk q obtained by Gram-Schmidt orthogonalization of
pxk q is an orthonormal system in X, and
Spanphk qnk1  Spanpxk qnk1 for all n P N.
Proof. The system phk q is orthonormal by construction. Also, we obviously
have the inclusion of the subspaces Spanphk qnk1 „ Spanpxk qnk1 , and the dimen-
sions of these subspaces both equal n by construction. Therefore, these subspaces
are equal. 
Exercise 1.6.23. [Legendre polynomials] Prove that Gram-Schmidt
orthogonalization of the monomials ptk q8
k0 in the space L2 r
1, 1s gives
the system of Legendre orthogonal polynomials
1 dk 2
Pk ptq  pt
1qk ,
2k k! dtk
up to normalization constants. More precisely, Pk ptq form an orthogonal
basis in L2 r
1, 1s and }Pk }22  2{p2k 1q.
In a similar way, by orthogonalization of monomials in different spaces, one
produces other classical systems of orthogonal polynomials, such as Hermite (see
Exercise 1.6.38 and Chebychev polynomials.
1.6.6. Existence of orthogonal bases. As a consequence of Proposition 1.6.22,
we will obtain that every Hilbert space that is not “too large” has an orthonormal
bases. Such are all separable Hilbert spaces.
Recall that a metric space is separable if it contains a countable dense subset.
Exercise 1.6.24. Prove that the following Banach spaces X are sepa-
rable by describing a countable dense subset in X: C r0, 1s, L2 r0, 1s, c0 , `p
for p P r1, 8q.
 1.6. FOURIER SERIES 35

Exercise 1.6.25. Prove that the spaces `8 and L8 are not separable
by constructing an uncountable separated subset A, i.e. such that
inf t}x
y } : x, y P A, x  yu ¡ 0.
For Banach spaces, separability is formally equivalent to a slightly stronger
property:
Lemma 1.6.26 (Separable spaces). A Banach space X is separable if and only
if it contains a system of vectors pxk q8
k1 whose linear span is dense in X, i.e.

Spanpxk q  X.
Proof. Necessity. If X is separable, it contains a system of vectors pzk qnk1
whose linear span is dense in X. We construct pxk q inductively as a subset of pzk q.
Namely, we include z1 , and if zn 1 R pz1 , . . . , zn q we include zn for n  1, 2, . . .. By
construction, pxk q is linearly independent and Spanpx1 , . . . , xn q  Spanpz1 , . . . , zn q
for all n  1, 2, . . . Letting n Ñ 8 we conclude that Spanpx1 , . . . , xn q  Spanpz1 , . . . , zn q 
X as required.
Sufficiency.
°n If Spanpxk q is dense in X, so is the set of all finite linear combina-
tions k1 ak xk with ak P Q, which is a countable set. The lemma is proved. 

Theorem 1.6.27. Every separable Hilbert space has an orthonormal basis.

Proof. Let X be a separable Hilbert space. By Lemma 1.6.26, there is a
system of vectors pxk q in X such that Spanpxk q  X. Applying the Gram-Schmidt
orthogonalization, we obtain an orthonormal system phk q in X. Using Proposi-
tion 1.6.22 and letting n Ñ 8, we conclude that
Spanphk q  Spanpxk q  X.
This proves the completeness of phk q. Hence phk q is an orthonormal basis in X. 

Exercise 1.6.28. Prove the converse for Theorem 1.6.27. Namely, if

a Hilbert space X has an orthonormal basis then X is separable.

Remark 1.6.29. We developed the theory for countable orthogonal systems

and bases. One can generalize it for systems of arbitrary cardinality, but we will
not do this here.
1.6.7. Isometry of all separate Hilbert spaces. We are ready to show
that all Hilbert spaces of the same cardinality have “the same geometry”:
Theorem 1.6.30. All infinite-dimensional separable Hilbert spaces are isomet-
ric to each other. Precisely, for every such spaces X and Y , one can find a linear
bijective map T : X Ñ Y which preserves the inner product, i.e.
(1.22) xT x, T yy  xx, yy for all x, y P X.
Proof. Let pxk q and pyk q be orthonormal bases of spaces X and Y respectively.
Let T be the map that takes xk to yk . More precisely, define T by

¸ ¸
T ak xk  a k yk .
k k
 1.6. FOURIER SERIES 36

°
Note that every x P X has the form x  k ak xk for some (Fourier) coefficients ak ,
so the definition makes sense. Also, by Parseval’s identity,
¸ 2 ¸ ¸ 2
(1.23)



a k xk   |ak |2   
ak yk  .
k k k

Therefore, T is well defined on X, its inverse is also well defined as

¸ ¸
T
1 ak yk  ak xk ,
k k

so T is bijective and clearly linear. Additionally, (1.23) shows that

}T x}  }x} for all x P X.
So T preserves the norm. Since by polarization formula, the inner product is
uniquely determined by the norm, T must also preserve the inner product, i.e.
(1.22) holds. This completes the proof. 

Remark 1.6.31. It follows from Theorem 1.6.30 that }T x}  }x} for all x P X.
Thus by linearity,
}T x
T y}  }x
y} for all x, y P X.
Thus T preserves all pairwise distances; hence the name “isometry”.
Remark 1.6.32. Since `2 and L2 r0, 1s are separable Hilbert spaces, it follows
that every Hilbert space is isometric to `2 and L2 r0, 1s.

1.6.8. Additional Exercises.

Exercise 1.6.33. [Frames] Consider the space L2 ra, bs where
π   a  
b   π. Show that the system of exponentials (1.21) satisfies Parseval’s
identity for every function x P L2 ra, bs, although it is not an orthogo-
nal system. General systems that satisfy Parseval’s identity are called
frames. They are important in signal processing.

Exercise 1.6.34. [Haar system] Consider the function

#
t P r0, 1{2q
hptq 
1,

1, t P r1{2, 1q

and define the functions

hkl ptq  hp2k t
lq, k  0, 1, 2, . . . , l  0, 1, 2, . . . , 2k
1.
Together with the constant function 1, this system of functions is called
the Haar system; the individual functions is called the Haar wavelets
and the function hptq is called the Haar mother wavelet. Show that,, the
Haar system is an orthonormal basis in L2 r0, 1s. (Hint: first show that
Haar system is complete in C r0, 1s.)
 1.6. FOURIER SERIES 37

Exercise 1.6.35. [Rademacher system] Consider the functions

rk ptq  p
1qr2 s, P N Y t0u,
k
x
k
where rs denotes the integer part of a number. This system of functions
is called the Rademacher system. Show that the Rademacher system is
an orthonormal system in L2 r0, 1s, but is not complete (therefore not an
orthonormal basis).

Exercise 1.6.36. [Walsh system] Consider the functions wA ptq, A €

N Y t0u (indexed by subsets A rather than numbers!) defined by
¹
wA ptq  rk ptq
P
k A
where rk ptq are the Rademacher functions. This system of functions is
called the Walsh system. Show that the Walsh system is an orthonormal
basis in L2 r0, 1s.

Exercise 1.6.37. [Characterization of complete systems] Consider a

system of vectors pxk q (not necessarily orthogonal) in a Hilbert space X.
Prove that pxk q is complete if and only if the only vector orthogonal to
all of xk is zero.

Exercise 1.6.38. [Hermite polynomials] Suppose wptq is a continuous

weight function R Ñ R . Consider the Hilbert space L2 pR, wptq dtq, i.e. the
measure on R is given by wptq dt. The Gram-Schmidt orthogonalization of
monomials ptk q8
k0 produces a system orthogonal polynomials Pk ptq with
repsect to the weight wptq, i.e.
»
Pk ptqPl ptqwptq dt  δkl .
R

Now consider the weight wptq  ?12π e
t {2 , i.e. the standard normal
2

density. Prove that the orthogonal polynomials with respect to this

weight is the system of Hermite polynomials
{2 d e
t2 {2 ,
k
Pk ptq  p
1qk et
2

dt k

up to normalization constants. More precisely, Pk ptq form an orthogonal

basis in L2 pR, wptq dtq and }Pk }22  k!.

Exercise 1.6.39. [Space of almost periodic functions] A function f :

R Ñ C is called almost periodic if it is a superposition of a finite number
of frequencies, i.e. f has the form
¸
n
f ptq  ak eiwk t , where ak P C, wk P R, n P N.

k 1
Note that the frequencies wk are allowed to take arbitrary real values.
Denote the space of almost periodic functions by X0 , and equip it with
the inner product

1 »T 1{2
xf, gy  Tlim
Ñ8 2T
|f ptq|2 dt .
T
 1.6. FOURIER SERIES 38

Show that the inner product is well defined on X0 , and

¸
n ¸
n ¸
n
xf, gy  ak bk where f ptq  ak eiwk t , g ptq  bk eiwk t

k 1 
k 1 
k 1
(and where the frequencies wk are all different). The completion X of the
inner product space X0 is called the space of almost periodic functions.
Prove that X is a non-separable Hilbert space by showing that the
system of functions
teitw , w P Ru
is an orthonormal system in X.
 CHAPTER 2

Bounded linear operators

Lec.13: 10/06
In this chapter we study certain transformations of Banach spaces. Because
these spaces are linear, the appropriate transformations to study will be linear
operators. Furthermore, since Banach spaces carry topology, it is most appropriate
to study continuous transformations, i.e. continuous linear operators. They are
also called bounded linear operators for the reasons that will become clear shortly.

2.1. Bounded linear functionals

The most basic but rich class of linear operators are linear functionals, those
that map a space into R or C.
2.1.1. Definition and examples. At this moment, the topology does not
matter, so we define linear functionals on general linear vector spaces.
Definition 2.1.1 (Linear functionals). Let E be a linear space over C (the
real case is similar). A linear functional on E is a linear operator f : E Ñ C.
Equivalently, a function f : E Ñ C is a linear functional if
f pax by q  af pxq bf py q for all x, y P E, a, b P C.
Example 2.1.2 (Integration). The integral of an integrable function is a basic
example of a linear functional. Specifically, the map
»
F pg q  g ptq dµ

is clearly a linear functional on L1 pΩ, Σ, µq.

Similarly, for a fixed weight function w P L1 r0, 1s, the map
»1
F pg q  g ptqwptq dt
0
defines a linear functional on L8 r0, 1s. (Check!)
Example 2.1.3 (Point evaluation functional). For a fixed t0 P r0, 1s, the map
(2.1) F pg q  g pt0 q
is clearly a linear functional on C r0, 1s. It is called the point evaluation functional
at t0 .
Physicists view the point evaluation functional as a special case of integration
with weight:
»1
(2.2) g pt0 q  g ptqδ pt
t0 q dt
0
The weight here is given by Dirac delta function δ ptq, which is zero for all values t
³1
except δ pt0 q  8, and such that 0 δ ptq dt  1. Dirac delta function does not exist
39
 2.1. BOUNDED LINEAR FUNCTIONALS 40

as a function r0, 1s Ñ R, and should be understood as a linear functional. The

“integral” (2.2) of a function against the Dirac delta function should be understood
as the point evaluation functional (2.1).
Exercise 2.1.4. [Functionals on Cn ] Show that every linear functional
f on Cn has the form
¸
n
f p xq  xk yk  xx, yy, x  px1 , . . . , xn q,

k 1

for some y  py1 , . . . , yn q P Cn .

Example 2.1.5. More generally, we will soon show that every liner functional
on a Hilbert space X has the form
f pxq  xx, y y
for some y P X. For now, we note that f defined this way is indeed a linear
functional.
2.1.2. Continuity and boundedness. Dual space.
Definition 2.1.6 (Continuity, boundedness). Let f be a linear functional on
a normed space X.
(i) Recall that f is continuous if
xn Ñ x in X implies f pxn q Ñ f pxq.
(ii) f is called bounded if there exists a number C such that
|f pxq| ¤ C }x} for all x P X.
Proposition 2.1.7. Continuity and boundedness of linear functionals are equiv-
alent.
Proof. Assume that f is bounded, and let xn Ñ x. Then
|f pxn q
f pxq|  |f pxn
xq| ¤ C }xn
x} Ñ 0.
Thus f is continuous.
Vice versa, assume that f is not bounded. Then we can find a sequence pxn q
of nonzero vectors in X such that
|f pxn q| ¥ n}xn }, n  1, 2, . . .
Dividing both sides by n}xn } we obtain


xn 
 ¥ 1, n  1, 2, . . .

f
n}x } n

On the other hand, n}xxnn } Ñ 0 as the norm of these vectors equals 1{n. This implies
that f is not continuous. 

Exercise 2.1.8. Prove that if f is continuous at a single point x0 PX

then f is continuous (everywhere on X).
 2.1. BOUNDED LINEAR FUNCTIONALS 41

Definition/Proposition 2.1.9 (Dual space). Let X be a normed space. The

space of all linear functionals f on X is a linear vector space. It is called the dual
space and is denoted X  . The dual space is a normed space, with the norm defined
as
}f }  sup |f}pxx}q|  sup |f pxq|, f P X  .
x0 }x}1
Exercise 2.1.10. Prove the facts stated in this definition/proposition –
that X  is a normed space, the identity between the two norm definitions,
and that }f } indeed defines the norm on X  .

Remark 2.1.11. The definition implies the following useful inequality:

|f pxq| ¤ }f } }x} for all x P X, f P X .
Also, }f } is the smallest number in this inequality that makes it valid for all x P X.
Exercise 2.1.12. Compute the norms of the integration and the point
evaluation functionals considered in Examples 2.1.2 and 2.1.3.

2.1.3. Hyperplanes as level sets of linear functionals. General functions,

and in particular linear functionals f , on a linear vector space E may be visualized
by describing their level sets
tx P X : f pxq  cu
for various values c P C. The level set corresponding to c  0 is the kernel of f .
It turns out that ker f is a hyperplane, i.e. a subspace of E of codimension
1. All other level sets of f are obviously the translates of ker f , see the picture.
Moreover, there is a canonical correspondence between the linear functionals and
the hyperplanes in X. This is clarified in the following proposition.

Figure 2.1. Level sets of a linear functional f on a linear vector

space E

Proposition 2.1.13 (Linear functionals and hyperplanes). Let E be a linear

vectors space.
(i) For every linear functional f on E, ker f is a hyperplane in E, i.e. codimpker f q 
1.
(ii) If f, g  0 are linear functionals on E such that ker f  ker g, then f  ag
for some scalar a  0.
 2.2. REPRESENTATION THEOREMS FOR LINEAR FUNCTIONALS 42

(iii) For every hyperplane H € E there exists a linear functional f  0 such that
ker f  H.
Proof. (i) Follows from a linear version of the fundamental theorem on ho-
momorphisms, Exercise 1.1.24. Indeed, the injectivization f˜ : E { ker f Ñ C of f
establishes a linear bijection (isomorphism) between E { ker f and the range C of f .
Thus dimpE { ker f q  dimpCq  1, so ker f is a hyperplane in E.
(ii) Since ker f  ker g : H, the injectivizations f˜, g̃ : E {H Ñ C are linear
functionals on the one-dimensional space E {H. A moment’s thought yields that
such linear functionals must be equal up to some constant factor a, i.e. f˜ 
ag̃. Then f˜rxs  ag̃ rxs for all x P E. On the other hand, by construction of
injectivization, f pxq  f˜rxs and g pxq  g̃ rxs. Therefore f pxq  ag pxq as required.
(iii) Since dimpE {H q  1, we have
E {H  tarx0 s : a P Cu
for some x0 P E. Let x P E be arbitrary; then rxs  arx0 s for some a  apxq P C,
which implies x  ax0 h for some h P H. Let us define f on E by f pxq  a. Then
f is a linear functional (check!), and clearly ker f  H. 

Proposition 2.1.14. Let f be a bounded linear functional, i.e. f P X  then

ker f is closed.
Proof. ker f is the pre-image of the closed set t0u under the continuous map
f , so it must be closed. 

Remark 2.1.15. Using injectivization of f , one can show that the converse also
holds. So, a linear functional f is bounded if and only if ker f is closed. It follows
that the kernel of a linear functional is either closed or dense in X. (Why?)
Lec.14: 10/08

2.2. Representation theorems for linear functionals

In concrete Banach spaces, the bounded linear functionals usually have a spe-
cific and useful form. Generally speaking, all linear functionals on function spaces
(such as Lp and C pK q) act by integration of the function (with respect to some
weight or measure). Similarly, all linear functionals on sequence spaces (such as `p
and c0 ) act by summation with weights.

2.2.1. Dual of a Hilbert space: Riesz representation theorem. We

start by characterizing bounded linear functionals on a Hilbert space X. The
following theorem says that every functional f acts as an inner product with some
vector in X.
Theorem 2.2.1 (Riesz representation theorem). Let X be a Hilbert space.
(i) For every y P X, the function
(2.3) f pxq  xx, y y, xPX
is a bounded linear functional on X, and its norm is }f }  }y }.
(ii) Conversely, for every bounded linear functional f P X  there exists a unique
vector y P X such that (2.3) holds. Moreover, }f }  }y }.
 2.2. REPRESENTATION THEOREMS FOR LINEAR FUNCTIONALS 43

Proof. (i) By Cauchy-Schwarz inequality,

|f pxq|  |xx, yy| ¤ }x} }y}, x P X.
Hence f is bounded, and }f } ¤ }y }. Conversely, for x  y we have
f pxq  xx, xy  }x}2 .
Hence }f } ¥ }y }.
(ii) Let f P X  . By Proposition 2.1.13, ker f is a hyperplane in X. Since f is
bounded, ker f is closed (see Proposition 2.1.14). Therefore, X can be represented
by the orthogonal decomposition
X  ker f ` Spanpy0 q for some y0 P pker f qK .
(Use Theorem 1.5.7). Consider the map
g pxq : xx, y0 y, x P X.
We have g P X  by part (i). Moreover,
ker g  ty0 uK  ker f.
Therefore, by Proposition 2.1.13, the functionals f and g are equal up to some
constant factor a, that is f  ag. It follows that
f pxq  axx, y0 y  xx, ay0 y,
and the conclusion follows with y : ay0 .
The uniqueness of y is simple. (Why?) 
Remark 2.2.2. In a concise form, the statement of Riesz representation theo-
rem can be expressed as
X   X.
Although X  and X are formally different spaces, they can be canonically identified
as in Riesz representation theorem.
Let us rewrite Riesz representation theorem for the Hilbert space L2 .
Corollary 2.2.3 (L2  L2 ). Consider the space L2  L2 pΩ, Σ, µq.
(i) For every weight function g P L2 , integration with weight
»
Gpf q : f g dµ, f P L2
³
is a bounded linear functional on L2 , and its norm is }G}  }g }2  p |g |2 dµq1{2 .
(ii) Conversely, every bounded linear functional G P L2 can be represented as
integration with weight for some unique weight function g P L2 . Moreover,
}G}  }g}2 .
2.2.2. Application: proof of Radon-Nikodym theorem. Riesz represen-
tation theorem can be used to give a “soft” proof of Radon-Nikodym theorem in
measure theory. This argument is due to von Neumann (1940).
Consider two measures µ, ν on the same σ-algebra. Recall that ν is called
absolutely continuous with respect to µ, abbreviated ν ! µ, if
µpAq  0 implies ν pAq  0
for measurable sets A.
 2.2. REPRESENTATION THEOREMS FOR LINEAR FUNCTIONALS 44

Theorem 2.2.4 (Radon-Nikodym theorem). Consider two finite measures1 µ, ν

such that ν ! µ. Then ν can be expressed as
»
ν pAq  g dµ for all measurable sets A,
A

where g ¥ 0 is a measurable function. Moreover, g is uniquely determined µ-a.e.;

it is called the Radon-Nikodym derivative and is denoted g : dν {dµ.

Proof. We shall prove the existence part only.

The linear functional
»
F pf q : f dµ

is a bounded linear functional on the space L2 pµq, and therefore also on the space
L2 pµ ν q. By Riesz representation theorem, there exists h P L2 pµ ν q such that
» » » »
(2.4) f dµ  f h d pµ νq  f h dµ f h dν for all f P L2 pµ ν q.

Rearranging the terms, we obtain

» »
(2.5) f h dν  f p1
hq dµ for all f P L2 pµ ν q.

We claim that

(2.6) 0 h¤1 µ-a.e.

Indeed, consider the set A 

³ th ¤ 0u and the indicator function f  1A . In this
case (2.4) becomes µpAq  A h dpµ ν q ¤ 0, hence µpAq  0. Similarly, consider
the set B  th ¡³ 1u and the indicator function f  1B . If µpB q ¡ 0 then (2.4)
becomes µpB q  B h dpµ ν q ¡ pµ ν qpB q ¥ µpB q, a contradiction. This proves
(2.6).
Since ν ! µ, we moreover have

(2.7) 0 h¤1 pµ ν q-a.e.

By Monotone Convergence Theorem, one can show that (2.5) holds for arbitrary
pµ ν q-measurable functions f such that f ¥ 0 pµ ν q-a.e. (Indeed, consider the
truncation fn ptq : minpf ptq, nq and let n Ñ 8.) The convention is that if one side
of (2.5) is infinite then the other is infinite, too.
Now, given a measurable set A, we choose f so that f h  1A . In other words,
we consider
f :
1A
h
and apply the identity in (2.5). We obtain
»
1
h
ν pAq  dµ.
A h

The proof is complete with g : p1
hq{h. 

 2.2. REPRESENTATION THEOREMS FOR LINEAR FUNCTIONALS 45

Lec.15: 10/11
2.2.3. The dual of Lp . A version of the representation theorem for L2 , Corol-
lary 2.2.3, holds in fact for all Lp spaces. In short, it states that Lp  Lp1 where p
and p1 are conjugate exponents as in Hölder’s inequality, i.e.
1
p p1
1
 1, 1   p, p1   8,
and p1  8 if p  1. The rigorous statement is the following:
Theorem 2.2.5 (Lp  Lp1 ). Consider the space Lp  Lp pΩ, Σ, µq with finite
of σ-finite measure µ, and where 1 ¤ p   8. Let p1 be the conjugate exponent of p.
(i) For every weight function g P Lp1 , integration with weight
»
Gpf q : f g dµ, f P Lp
is a bounded linear functional on Lp , and its norm is }G}  }g }p1 .
(ii) Conversely, every bounded linear functional G P Lp can be represented as
integration with weight for some unique weight function g P Lp1 . Moreover,
}G}  }g}p1 .
We shall prove Theorem 2.2.5 for the particular case of spaces `p . The general
case is somewhat similar, and it relies on an application of Radon-Nykodim theorem.
For the spaces `p , Theorem 2.2.5 reads as follows:
Corollary 2.2.6 (`p  `p1 ). Let 1 ¤ p  8 and let p1 be the conjugate
exponent of p.
(i) For every y P `p1 , summation with weight
8̧
Gpxq : xk yk , x P `p
k 1 
is a bounded linear functional on `p , and its norm is }G}  }y }p1 .
(ii) Conversely, every bounded linear functional G P `p can be represented as
summation with weight for some unique weight y P `p1 . Moreover, }G}  }y }p1 .
Proof. We will only prove the case where 1   p, p1   8; the case p  1,
p1  8 is an exercise.
(i) By Hölder’s inequality, we have
¸ 
|Gpxq|   xk yk  ¤ }x}p }y }p1 .

k
It follows that F is a bounded linear functional on `p , and }G} ¤ }y }p1 . To prove
the converse inequality, note that Hölder’s inequality is sharp. Namely, for every
y P `p1 there exists x P `p such that
¸ 
|Gpxq|   xk yk   }x}p }y }p1 .

k
Indeed, one can check that this holds for x  pxk q defined as
1
xk  e
i Argpyk q |yk |p
1 .
For this x, it follows that }G} ¥ }y }p1 , so part (i) of the theorem is proved.
1The result can be extended (by decomposition) to σ-finite measures.
 2.2. REPRESENTATION THEOREMS FOR LINEAR FUNCTIONALS 46

(ii) Consider G P `p . Let ek denote as usual the coordinate vectors in `p ,

that is ek  p0, . . . , 0, 1, 0, . . .q with k-th coordinate equal 1. By the linearity and
continuity of G we have

8̧ 8̧
Gpxq  G xk ek  xk Gpek q, x  pxk q P `p .

k 1 k 1 
We claim that the conclusion of (ii) follows for
y  pyk q : pGpek qq.
°
Obviously, Gpxq  k xk yk as required. To prove that y P `p1 , consider
¸
n ¸
n
1
y pnq : yk ek and xpnq : e
i Argpyk q |yk |p
1 ek , n  1, 2, . . .

k 1 
k 1

Then using the equality case of Hölder’s inequality, we see that

¸
n 
}G}}xpnq }p ¥ |Gpxpnq q|   xk yk   }xpnq }p }y pnq }p1 .

k 1 
Canceling }xpnq }p on both sides, we conclude that

¸ 1{p1
|yk |p1
n
}ypnq }p1  ¤ }G}.
k 1 
Letting n Ñ 8 we conclude that y P `p1 and that }y }p1 ¤ }G}. By part (i), we
actually have }y }p1  }G}. This completes the proof. 
Exercise 2.2.7. [c0  `1 ] Prove that c0  `1 . The meaning of this is
the same as in Corollary 2.2.6, i.e. the functionals on c0 are given by
summation with weight from `1 .

2.2.4. The dual of C pK q. Finally, we state without proof the following char-
acterization of bounded linear functionals on C pK q.
Theorem 2.2.8 (C pK q ). Consider the space C pK q where K is a compact
topological space.
(i) For every Borel regular signed measure2 µ on K, integration
»
Gpf q : f dµ, f P C pK q
is a bounded linear functional on C pK q, and its norm is the total variation
}G}  |µ|pK q.
(ii) Conversely, every bounded linear functional G P C pK q can be represented
as integration with respect to a unique Borel regular signed measure µ on K.
Moreover, }G}  |µ|pK q.

2A signed measure µ is an extension of the notion of measure by allowing it to take on

negative values. By Hahn decomposition theorem, a signed measure µ can be uniquely represented

as µ µ
µ
where µ and µ
are measures. The measure µ µ | |
µ
is called the variation
| |p q
of µ, and the value µ K is called the total variation of µ.
 2.3. HAHN-BANACH THEOREM 47

2.2.5. Additional Exercises.

Exercise 2.2.9. [Span of point evaluation functionals] Compute the
norm of the following linear functional on C r0, 1s:
¸
n
F pf q  ak f ptk q

k 1
where ak are fixed scalars, and t1 , . . . , tn are fixed distinct points in r0, 1s.
Deduce that the linear span of point evaluation functionals δt1 , . . . , δtn
in C r0, 1s is isometric to `n1 .

2.3. Hahn-Banach theorem

Lec.16: 10/13
Hahn-Banach theorem allows one to extend continuous linear functionals f from
a subspace to the whole normed space, while preserving the continuity of f . Hahn-
Banach theorem is a major tool in functional analysis. Together with its variants
and consequences, this result has applications in various areas of mathematics,
computer science, economics and engineering.
Let X be a normed space, and let X0 be a subspace of X. Consider a bounded
linear functional f0 defined on X0 , i.e. f0 P X0 . An extension of f0 to the whole
space X is a bounded linear functional f P X  whose restriction on X0 coincides
with f0 , i.e.
f |X0  f0 |X ,
0 meaning that f0 px0 q  f pxq for all x0 P X0 .
Constructing extensions is a nontrivial problem because of the continuity (=bound-
edness) requirement for f .
Exercise 2.3.1. Show that if one does not require continuity of f , one
can construct f using a Hamel basis.

2.3.1. Extension by continuity. Before we state Hahn-Banach theorem, let

us address the simpler problem of extending a continuous linear functional from a
dense subspace to the whole space.
Proposition 2.3.2 (Extension by continuity). Let X0 be a dense subspace of a
normed space X. Then every functional f0 P X0 admits a unique extension f P X  .
Moreover, }f }  }f0 }.
Proof. Let x P X be arbitrary. By density, we can find a sequence pxn q € X0
such that
xn Ñ x.
Then pf0 pxn qq is a Cauchy sequence, since
|f0 pxn q
f0 pxm q| ¤ }f0 } }xn
xm } Ñ 0 as n, m Ñ 8.
By completeness of R or C, the sequence converges. So we define
f pxq : lim f0 pxn q
n
and claim that f satisfies the conclusion of the proposition.
First of all, f is well defined, i.e. it depends only on x and not on the choice of
an approximating sequence pxn q. Indeed, if x1n Ñ x then
|f0 pxn q
f0 px1n q| ¤ }f0 } }xn
x1n } Ñ 0
 2.3. HAHN-BANACH THEOREM 48

since the limits of xn and x1n coincide.

Next, f is a linear functional on X, since for xn Ñ x and yn Ñ y we have
f pax by q  lim f0 paxn byn q  a lim f0 pxn q b lim f0 pyn q  af pxq bf py q.
n n n
Finally, f is a bounded linear functional. Indeed, for xn Ñ x we have
|f pxq|  lim
n
|f0 pxn q| ¤ }f0 } lim
n
}xn }  }f0 } }x}.
This shows that f P X  and }f } ¤ }f0 }. Note that the reverse inequality }f } ¥ }f0 }
trivially holds (for any extension f ). This completes the proof. 
Example 2.3.3 (Lebesgue integral as an extension of Riemann integral). Let
X0 be the space of continuous functions C r0, 1s equipped with the norm
»1
}f }  |f ptq| dt (Riemann integral).
0
The Riemann integration
»1
F0 pf q  f ptq dt
0
is clearly a bounded linear functional on X0 . Let X denote the completion of
C r0, 1s. One can define Lebesgue integral as the extension F P X  of F0 , and one
can define L1 r0, 1s : X.
Exercise 2.3.4. Check that this indeed gives an equivalent definition
of Lebesgue integral and of the space L1 r0, 1s.

2.3.2. Hahn-Banach theorem. Now we address the more difficult problem

of extending linear funcitonals from arbitrary subspaces. The result is the same as
for dense subspaces, except the extensions need not be unique.
Theorem 2.3.5 (Hahn-Banach theorem). Let X0 be a subspace of a normed
space X. Then every functional f0 P X0 admits an extension f P X  such that
}f }  }f0 }.
Proof. We will prove this result for spaces over R only.
Step 1. Extension by one dimension. Let us first assume that X0 is a hyperplane
in X, i.e. codim X0  1. Fix a vector z P X zX0 . Every vector x P X is then
uniquely represented as
x  az x0 , a P R, x0 P X0 .
Then an extension f has the form
f pxq  af pz q f px0 q  af pz q f0 px0 q.
So f is determined by just one number
f pz q : C.
Without loss of generality we can assume that }f0 }  1 (by rescaling). We are
looking for an extension f such that }f }  1, which means that
|f pxq| ¤ }x}, x P X.
This requirement is equivalent to
f pxq ¤ }x}, x P X.
 2.3. HAHN-BANACH THEOREM 49

(Why?) Expressing x and f as above, we write this equivalently as

(2.8) aC f0 px0 q ¤ }az x0 }, a P R, x0 P X0 .
The desired extension f will be constructed if we are able to show that this inequal-
ity has solutions in C.
For a  0, inequality (2.8) trivially holds by the assumption that }f0 }  1. For
a ¡ 0, inequality (2.8) can be written as

x0 
x
C ¤ z a

f0
a
0
, a P R , x0 P X0 .
For a 
b   0, inequality (2.8) can be written as

 
¥ f0 xb1
 xb1
z, b P R , x1 P X1 .
C

Existence of a number C P R that satisfies both these inequalities is equivalent to

the inequality
 


x
 x0 
x
sup f0
1

 xb1
z ¤ aPR inf
, x PX

z 
f0
0
.
P P
b R , x 1 X1 b 0 0 a a
Collecting the similar terms we see that this in turn is equivalent to the inequality

x x1  x0   
f0
0
a b
¤ z a

 x1

b

z, a, b P R ; x0 , x1 P X0 .
But this inequality is true; it follows from the fact that }f0 }  1, i.e.

x x1  x0 x1 
f0
a
0
b
¤ a b
,

and from the triangle inequality.

Step 2. Transfinite induction. This standard argument relies on Zorn’s Lemma 1.1.7
in a similar way to the proof of Proposition 1.1.6 on the existence of Hamel bases.
We consider the set Γ of all extensions of f0 , precisely
(
Γ pY, gq : „ Y „ X a subspace, g P Y  an extension of f0 , }g}  }f0 } .
X0
The proof will be complete if we show that Γ contains an element pY, g q with Y  X.
We introduce a partial order on Γ by
pY1 , g1 q ¤ pY2 , g2 q if Y1 „ Y2 , g2 is an extension of g1 .

Every chain pYα , gα q α has an upper bound pY, g q in Γ, namely
¤
Y : Yα , g pxq : gα pxq if x P Yα .
α

(Why is g a well defined bounded linear functional on Y ?) Therefore, by Zorn’s

Lemma 1.1.7, there exists a maximal element pY, g q in Γ.
We claim that Y  X. Indeed, otherwise by Step 1 we could extend g to a
subspace larger than Y , which would contradict the maximality of pY, g q in Γ. The
proof is complete. 

Exercise 2.3.6. Deduce Hahn-Banach theorem for spaces X over C.

 2.3. HAHN-BANACH THEOREM 50

Lec.17: 10/15
2.3.3. Supporting functionals. Hahn-Banach theorem has a variety of con-
sequences, both analytic and geometric. One of the basic tools guaranteed by
Hahn-Banach theorem is the existence of a supporting functional f P X  for every
vector x P X.
Proposition 2.3.7 (Supporting functional). Let X be a normed space. For
every x P X there exists f P X  such that
}f }  1, f pxq  }x}.
The functional f is called the supporting functional of x.
Proof. Consider the one-dimensional subspace X0  Spanpxq, and define a
functional f0 P X0 by
f0 ptxq  t}x}, t P R or C.
Then }f0 }  1. An extension f P X  of f0 guaranteed by Hahn-Banach theorem
clearly satisfies the conclusion. 

Exercise 2.3.8. Consider a normed space X  pR2 , }  }q and a unit

vector x0 P X. Let f be a supporting functional of x0 . Interpret geomet-
rically the level set tx : f pxq  1u as a tangent hyperplane for the unit
ball BX at point x0 .
Construct an example of a normed space for which the supporting
functional of x is not unique.
Recall that the norm of a functional f P X  is defined as
}f }  sup |f}pxx}q|
x0

Generally it is not true that every functional f attains its norm on some vector x,
i.e. that the supremum above can be replaced by the maximum.
Exercise 2.3.9. Construct a bounded linear functional on C r0, 1s which
does not attain its norm.
However, every vector x does attain its norm on some functional f P X  ,
namely the supporting functional. This immediately follows from Proposition 2.3.7:
Corollary 2.3.10. For every vector x in a normed space X, one has

}x}  max |f pxq|

f 0 }f }

where the maximum is taken over all non-zero functionals f P X .

Hahn-Banach theorem implies that there are enough bounded linear functionals
f P X  on every space X. One manifestation of this is the following:
Corollary 2.3.11 (X  separates the points of X). For every two vectors x1 
x2 in a normed space X, there exists a functional f P X  such that f px1 q  f px2 q.
Proof. The supporting functional f P X  of the vector x  x1
x2 must
satisfy f px1
x2 q  }x1
x2 }  0, as required. 
 2.3. HAHN-BANACH THEOREM 51

2.3.4. Second dual space. Let X be a normed space as usual. The func-
tionals f  are designed to act on vectors x P X via f : x ÞÑ f pxq. Vice versa, we
can say that vectors x P X act on functionals f P X  via

(2.9) x:f ÞÑ f pxq, f P X .

Thus a vector x P X can itself be considered as a function from X  to R.
This function (2.9) is clearly linear, so we may consider x as a linear functional
on X  . Also, the inequality
|f pxq| ¤ }x} }f }
shows that this functional is bounded, so

x P X 

and the norm of x as a functional is }x}X  ¤ }x}. Considering the supporting

functional f P X  of x we see that actually

}x}X   }x}.
We demonstrated that there exists a canonical embedding of X into into X  .
We summarize this as follows.

Theorem 2.3.12 (Second dual space). Let X be a normed space. Then X can
be considered as a linear subspace of X  . For this, a vector x P X is considered
as a bounded linear functional on X  via the action

x:f ÞÑ f pxq, f P X .
Example 2.3.13. As we know from Section 2.2.3, c0  `1 while `1  `8 , so
c
0  `8 .
The space c0 of sequences converging to zero is indeed canonically embedded into
the larger space `8 of all bounded sequences (and with the same sup-norm).

Definition 2.3.14 (Reflexive spaces). A normed space X is called reflexive if

X   X (under the canonical embedding).

Example 2.3.15. As we know from Section 2.2.3, pLp q  Lp1 where 1 ¤ p   8

and p1 is the conjugate index of p. Therefore, Lp is a reflexive space for 1   p   8.
One can show that L1 and L8 are not reflexive spaces.

Proposition 2.3.16. Let X be a reflexive space. Then every functional f P X

attains its norm on X.

Proof. By reflexivity, the supporting functional of f is a vector x P X   X,

thus }x}  1 and f pxq  }f }, as required. 

The converse of Proposition 2.3.16 is also true. If every functional f P X  on

a Banach space X attains its norm, then X is reflexive. This is James’ theorem
(1971).
 2.3. HAHN-BANACH THEOREM 52

2.3.5. Hahn-Banach theorem for sublinear functions. A quick inspec-

tion of the proof of Hahn-Banach theorem in Section 2.3.2 reveals that we have not
used all the norm axioms there. We used just these two – positive homogeneity
and triangle inequality, precisely
(i) }ax}  a}x} for all x P X, a ¥ 0;
(ii) }x y } ¤ }x} }y } for all x, y P X.
Definition 2.3.17 (Sublinear function). Let X be a linear space. Functions
}  } : X Ñ r0, 8q that satisfy (i) and (ii) above are called sublinear functions.
It is an exercise to check that our proof of Hahn-Banach theorem is valid not
only for norms but for sublinear functions:
Theorem 2.3.18 (Hahn-Banach theorem for sublinear functions). Let X0 be a
subspace of a linear vector space X. Let }} be a sublinear function on X. Consider
a linear functional f on X such that
f0 pxq ¤ }x} for all x P X0 .
Then f0 admits an extension f to the whole space X such that
f pxq ¤ }x} for all x P X.
Exercise 2.3.19. Prove Theorem 2.3.18 by modifying the proof of
Hahn-Banach Theorem.
Lec.18: 10/20

Considering sublinear functionals instead of norms offers us more flexibility

in geometric applications. Sublinear functionals arise as Minkowski functionals of
convex sets.
Definition 2.3.20. A subset K of a linear vector space X is called absorbing
if ¤
X  tK
¥
t 0
where tK  ttx : x P K u.
Exercise 2.3.21. Let K be a subset of a normed space X such that

0 P K. Then K is an absorbing set.

Proposition 2.3.22 (Minkowski functional). Let K be a absorbing convex sub-

set of a linear vector space X such that 0 P K. Then Minkowski functional
(
}x}K  inf t ¡ 0 : x{t P K
is a sublinear functional on X.
Conversely, let }  } be a sublinear functional on a linear vector space X. Then
the sub-level set
K  tx P X : }x} ¤ 1u
is an absorbing convex set, and 0 P K.
Exercise 2.3.23. Prove Proposition 2.3.22.
Proposition 2.3.22 should be compared to a similar result for norms that we
proved earlier in Exercise 1.2.22.
 2.3. HAHN-BANACH THEOREM 53

2.3.6. Separation of convex sets. Hahn-Banach theorem has some remark-

able geometric implications, which are grouped together under the name of separa-
tion theorems. Under some mild topological requirements, these results guarantee
that two convex sets A, B can always be separated by a hyperplane. As we know
from Section 2.1.3, the hyperplanes correspond to the level sets of linear functionals
f . Therefore, we expect that a separation theorem for A, B would give us a linear
functional f and a number C such that
f paq ¤ C ¤ f pbq, a P A, b P B.
In this case, the sets A and B get separated by the hyperplane tx : f pxq  C u.
Let us start from the simpler case when one of the two sets is a point.
Theorem 2.3.24 (Separating a point from a convex set). Let K be an open
convex subset of a normed space X, and consider a point x0 R K. Then there exists
a functional f P X  , f  0, such that
f p xq ¤ f p x0 q , x P K.
Proof. Translating K if necessary, we can assume without loss of generality
that 0 P K. (Why?) By Exercise 2.3.21, K is an absorbing set. Therefore, by
Proposition 2.3.22, Minkowski functional }}K of K is a sublinear functional on X.
Since 0 P K and K is open, it contains a centered ball BX p0, rq for some radius
r ¡ 0 (see the figure). The set inclusion BX p0, rq „ K implies the inequality for
the Minkowski functionals:
1
r
}x} ¥ }x}K , x P K.
(Why?)

Figure 2.2. Separation of a point x0 from the set K by a func-

tional f

Consider the one-dimensional subspace

X0  Spanpx0 q
and define a linear functional f0 on X0 by
f0 ptxq  t}x}, t P R.
 2.3. HAHN-BANACH THEOREM 54

Then f0 is dominated by }  }K on X0 , since for t ¥ 0 we have

f0 ptx0 q  }tx0 }K ;
f0 p
tx0 q 
tf px0 q 
t}x0 }K ¤ 0 ¤ }tx0 }K .
By Hahn-Banach theorem for sublinear functionals (Theorem 2.3.18), f0 admits an
extension f onto the whole space X such that the domination is preserved, i.e.
f pxq ¤ }x}K , x P X.
To finish the proof, we need to check that f is bounded and that it separates
x0 from K as required. The boundedness follows from the inequality

f pxq ¤ }x}K ¤ }x}, x P X,

1
r
so f P X  . To check the separation, consider x P K. Since x0 R K, we have
f pxq ¤ }x}K ¤ 1 ¤ }x0 }  f0 px0 q  f px0 q.
This completes the proof. 
Theorem 2.3.25 (Separation of open convex sets). Let A, B be disjoint open
convex subsets of a normed space X.
(i) Assume that A is open. Then there exists a functional f P X  and a number
C P R such that
f paq ¤ C ¤ f pbq, a P A, b P B.
(ii) If both A and B are open, then the stronger inequality holds:
f paq   C   f pbq, a P A, b P B.
Proof. (i) Consider the Minkowski difference set
K  A
B : ta
b : a P A, b P B u.
The set K is open and convex. (Check!) Since A and B are disjoint, 0 P K.
Using Theorem 2.3.24, we obtain a functional f P X  , f  0 such that
f pa
bq ¤ f p0q  0, a P A, b P B.
Hence f paq ¤ f pbq for all a P A, b P B, so letting C : supaPA f paq we obtain
f paq ¤ C ¤ f pbq, a P A, b P B.
Since A is open, by considering a small neighborhood of a in A (check!) we obtain
the strict inequality
f paq   C ¤ f pbq, a P A, b P B.
(ii) This part is similar, and follows by considering small neighborhoods of a in
A and of b in B. 
Exercise 2.3.26. Fill in the details in the proof of Theorem 2.3.25.

Corollary 2.3.27 (Separation of closed convex sets). Let A, B be disjoint

closed convex subsets of a normed space X. Assume B is compact. Then there
exists a functional f P X  and a number C P R such that
sup f paq   inf f pbq.
P
a A P
b B
 2.3. HAHN-BANACH THEOREM 55

Proof. Let
r  distpA, B q : aPA,infbPB }a
b}.
By the assumptions, r ¡ 0. (Why?) Therefore, the open r{3-neighborhoods Ar{3
of A and Br{3 of B are disjoint, open and convex sets. Applying Theorem 2.3.25,
we obtain a functional f P X  that separates the neighborhoods:
sup f paq ¤ inf f pbq.
P
a A r {3 bPBr{3

From this the conclusion easily follows. (How?) 

Exercise 2.3.28. Fill in the details in the proof of Corollary 2.3.27.

Remark 2.3.29. Suppose that in Theorem 2.3.24, the set K is either open (i.e.
as stated) or closed. Then the strict separation holds:
f pxq   f px0 q, x P K.
Indeed, for open sets this follows from Theorem 2.3.25, while for closed sets this
follows from Corollary 2.3.27.
2.3.7. Convex sets are intersections of half-spaces.
Corollary 2.3.30. Every closed convex subset K of a normed space X is the
intersection of all (closed) half-spaces that contain K.
Recall that the half-space is what lies on one side of a hyperplane; therefore
half-spaces have the form
tx P X : f pxq ¤ au

for some f P X , a P R. See the picture illustrating Corollary 2.3.30.

Figure 2.3. Convex set K is the intersection of half-spaces

Proof. K is trivially contained in the intersection of the half-spaces that con-

tain K. To prove the reverse inclusion, choose a point x0 R K and use Separation
Theorem 2.3.27 for A  K and B  tx0 u. We thus obtain a functional f P X 
such that
a : sup f pxq   f px0 q.
P
x K
It follows that the half-space tx PX : f pxq ¤ au contains K but not x0 . This
completes the proof. 
 2.4. BOUNDED LINEAR OPERATORS 56

2.3.8. Additional exercises.

Exercise 2.3.31. [Closed convex sets that can not be strictly sep-
arated] Show that the compactness assumption in Corollary 2.3.27 is
essential. Construct two closed convex sets on the plane R2 that can not
be strictly separated.

Exercise 2.3.32. [Convex sets that can not be separated] Show that
the openness assumption in Theorem 2.3.25 is essential.
To this end, consider the linear space P of all polynomials in one
variable and with real coefficients. Let the subset A consist of polyno-
mials with negative leading coefficient, and let the subset B consists of
polynomials with all non-negative coefficients. Show that A and B are
disjoint convex subsets of P, and that there does not exist a nonzero
linear functional f on P such that
f paq ¤ f pbq for all a P A, b P B.
(Hint: assume that for some C P R one has f paq ¤ C ¤ f pbq, a P A, b P B;
deduce from 0 P B that C ¤ 0 and by considering monomials that C ¥ 0.

Exercise 2.3.33. [Functionals that annihilate a subspace] Let X0 be a

closed subspace of a normed space X. Prove that there exists a functional
f P X  such that
f pxq  0 for all x P X0 .
You may deduce this from Hahn-Banach theorem directly or from a
separation theorem.

2.4. Bounded linear operators

In this section, we shall study continuous linear operators T : X Ñ Y between
Lec.19: 10/22

normed spaces X, Y . Linear functionals can be seen as a particular case of linear

operators with Y being the scalar field, R or C. So several results for linear operators
will be generalizations of those we have already seen for linear functionals; there will
be important differences though (e.g. a natural extension Hahn-Banach theorem
fails for linear operators).
2.4.1. Operator norm. Continuity and boundedness. The notions of
boundedness, continuity and norm of linear operators are similar to those for linear
functionals given in Section 2.1.2:
Definition 2.4.1 (Operator norm). A linear operator T : X Ñ Y acting
between normed spaces X and Y is called bounded if there exists a number C such
that
}T x} ¤ C }x} for all x P X.
The norm of T is the smallest constant C in this inequality, so it is defined as

}T }  sup }}Txx}}  sup }T x}.

x0 }x}1
Remark 2.4.2. We always have the inequality
}T x} ¤ }T } }x}, x P X.
 2.4. BOUNDED LINEAR OPERATORS 57

Exercise 2.4.3. [Composition of operators] If T : X Ñ Y and S : Y ÑZ

are bounded linear operators between normed spaces, then ST : X ÑZ
is a bounded linear operator, and
}ST } ¤ }T } }S }.
A version of Proposition 2.1.7 for linear functionals holds for linear operators,
and with a similar proof:
Proposition 2.4.4. Continuity and boundedness of linear operators are equiv-
alent.
Exercise 2.4.5. Prove that a linear operator T : X Ñ Y is bounded
if and only if it maps sequences that converge to zero to bounded se-
quences.

2.4.2. Space of operators. Let X and Y be normed spaces. The space of all
bounded linear operators T : X Ñ Y equipped with the operator norm is denoted
LpX, Y q.
As an example, the dual space is a space of operators that map to scalars, i.e.
X  LpX, Rq.
Proposition 2.4.6. LpX, Y q is a normed space. Moreover, if Y is a Banach
space then LpX, Y q is also a Banach space. In particular, the dual space X  is
always a Banach space, even if X is incomplete.
Proof. The norm axioms are straightforward. (Check!) To prove complete-
ness, let Tn P LpX, Y q be a Cauchy sequence, i.e.
}Tn
Tm } Ñ 0, n, m Ñ 8.
Applying this to an arbitrary x P X and noting that
}Tn x
Tn x} ¤ }Tn
Tm } }x} Ñ 0, n, m Ñ 8
we see that pTn xq is a Cauchy sequence in Y . By the completeness of Y it converges.
Define the map T as
T x : lim Tn x.
n
We claim that T is the limit of Tn in LpX, Y q, which would complete the proof.
It is easy to check that T : X Ñ Y is a linear operator. (Check!)
To show that T is bounded, we choose an arbitrary x P X and use continuity
of the norm:
}T x}  lim }Tn x} ¤ sup }Tn }}x}.
n n
Since a Cauchy sequence is always bounded (why?), supn }Tn }   8. It follows that
T is a bounded linear operator, i.e. T P LpX, Y q.
It remains to show that Tn Ñ T in LpX, Y q, i.e. in the operator norm. Since
T is Cauchy, for every ε ¡ 0 there exists a N P N such that
}Tn
Tm }   ε for n, m ¡ N.
Applying this to an arbitrary x P X we obtain
}Tn x
Tm x}   ε}x} for n, m ¡ N.
 2.4. BOUNDED LINEAR OPERATORS 58

Letting m Ñ 8, we get
}Tn x
T x}   ε}x} for n ¡ N.
Since x is arbitrary it follows that
}Tn
T }   ε for n ¡ N.
This means that Tn Ñ T in LpX, Y q as required. 

2.4.3. Operators on the finite-dimensional Euclidean space. Let X 

Y  `n2 ; recall that `n2 is the Euclidean space pRn , }  }2 q. As we know from linear
algebra, T can be identified with its n  n matrix ptij q, where
tij  xT ej , ei y, i, j  1, . . . , n
and where pei q denotes the canonical basis of `n2 . This way, the i-th coordinate of
the vector T x can be computed as
¸
n
(2.10) pT xqi  tij xj .

j 1

Proposition 2.4.7. Every linear operator T : `n2 Ñ `n2 is bounded. Specifically,

}T } ¤ }T }HS
where }T }HS denotes the Hilbert-Schmidt norm of T defined as

¸
n 1{2
}T }HS  |tij |2 .
i,j 1
Recall that we already encountered Hilbert-Schmidt (or Frobenius) norm of
matrices in (1.12).

Proof. Using (2.10) and applying Cauchy-Schwarz inequality, we obtain

¸
n n  ¸
¸ n 2 n
¸
¸ n
¸
n
}T x}22  |pT xqi |2  


tij xj  ¤ |tij |2 |xj |2  }T }2HS }x}22 .

i 1 
i 1 j 1 
i 1 
j 1 
j 1

The claim follows. 

2.4.4. Hilbert-Schmidt integral operators. A similar construction in func-

tion spaces L2 leads to the notion of Hilbert-Schmidt integral operators. To this
end, consider a function k pt, sq P L2 pr0, 1s2 q which we call the kernel. Define a
linear operator T : L2 r0, 1s Ñ L2 r0, 1s as
»1
(2.11) pT f qptq  k pt, sqf psq ds.
0

We can view this definition as a continuous version of (2.10), where kernel k pt, sq
can be considered as a continuous version of matrix. The operator T defined this
way is called Hilbert-Schmidt integral operator with kernel k pt, sq.
Proposition 2.4.8. A Hilbert-Schmidt integral operator T : L2 r0, 1s Ñ L2 r0, 1s
with kernel k pt, sq P L2 pr0, 1s2 q is bounded. Specifically,
}T } ¤ }k}2 .
 2.4. BOUNDED LINEAR OPERATORS 59

Proof. Our argument is a continuous version of the proof of Proposition 2.4.7.

Applying Cauchy-Schwarz and Fubini inequalities, we obtain
»1 » 1 2 »1
» 1
» 1
}T f } 
2
2

dt  p qpq 
k t, s f s ds ¤ dt |kpt, sq|
2
ds |f psq|2 ds
0 0 0 0 0
 }k}22 }f }22 .
The claim follows. 
Exercise 2.4.9. Why is pT f qptq defined for almost all t?
Lec.20: 10/25

Remark 2.4.10 (Fredholm integral equations). Hilbert-Schmidt integral opera-

tors arise from the study of Fredholm integral equations. The so-called homogeneous
Fredholm equation of the first kind is
»1
g ptq  k pt, sqf psq ds.
0
Given a kernel k pt, sq and the left hand side g ptq, the problem is to find the function
f psq.
Fredholm equations can be written as
Tf g
where T is the corresponding Hilbert-Schmidt operator. Therefore, Fredholm equa-
tions are linear equations. They can be thought of as continuous versions of matrix
linear equations Ax  b, where A is an m  n matrix, b P Rm and x P Rn .
Remark 2.4.11. The particular measure space r0, 1s does not play any role in
the discussion above, and can be replaced with an arbitrary measure space.
2.4.5. Volterra operator and differential operators. Volterra operator
T : L2 r0, 1s Ñ L2 r0, 1s is defined as
»t
(2.12) pT f qptq  f psq ds.
0
By Lebesgue differentiation theorem, T f is the antiderivative of f .
Volterra operator is clearly a particular case of Hilbert-Schmidt integral oper-
ator with kernel k pt, sq  1ts¤tu , and is therefore a bounded linear operator.
Exercise 2.4.12. Compute the norm of Volterra operator.
One would naturally think about the inverse of Volterra operator, which should
be the differential operator
Df  f 1 .
Clearly D can only defined on an appropriate dense subspace of L2 , such as C 1 ,
the space of continuously differentiable functions.
However, D is an unbounded linear operator on L2 r
π, π s.3 To see this, con-
sider the Fourier basis pek qkPZ of L2 r
π, π s, which we defined in Example 1.6.3
as
ek ptq  ? eikt .
1
2π
3The interval r
π, πs is chosen only for convenience; a similar result holds for an arbitrary
interval.
 2.4. BOUNDED LINEAR OPERATORS 60

Differentiating yields
Dek  pikqek , k P Z.
Hence
}Dek }2  k}ek }2  k, k P Z.
Letting k Ñ 8 implies that D is an unbounded operator.
This simple example suggests that various differential operators are unbounded
even on good function spaces. They may be studied through their inverses, which
are bounded integral operators.

2.4.6. Orthogonal projections and partial Fourier series. Let X0 be a

closed subspace of a Hilbert space X. Consider the orthogonal projection P in X
onto X0 . As we know from Section 1.5.2, P : X Ñ X is a linear map. Since P x
and x
P x are orthogonal vectors, we have
}x}2  }P x}2 }x
P x}2 , x P X,
so
}P x} ¤ }x}, x P X.
We have shown:
Proposition 2.4.13. The orthogonal projection P in a Hilbert space X onto a
closed subspace X0 is a bounded linear operator.
Exercise 2.4.14. Show that }P }=1.
As an example, consider the n-th partial sum of the Fourier series of a function
f P L2 r
π, πs:
¸
n
Sn f  xf, ek yek
k 
n
where ek denotes the Fourier basis of exponentials (1.16). As we know from
Lemma 1.6.10, Sn is an orthogonal projection onto Spanpek qnk
n . This subspace
is finite-dimensional and thus closed. (Why?) So by Proposition 2.4.13, Sn is a
bounded linear operator in L2 r
π, π s.
A convenient and classical way to represent Sn is via convolution with Dirichlet
kernel. Indeed,
»π
(2.13) pSn f qptq  2π1 Dn pt
sqf psq ds 
1
pDn  f qptq

π 2π
where

(2.14) Dn pθq 
¸
n
eikθ
pn 2 qθ
 sinsin
1

1
k 
n θ 2

is called the Dirichlet kernel.

Exercise 2.4.15. Prove this identity.
We see that Sn acts as a convolution with Dirichlet kernel Dn . In particular,
Sn is a Hilbert-Schmidt operator with Dirichlet kernel k pt, sq  2π
1
Dn pt
sq.
 2.4. BOUNDED LINEAR OPERATORS 61

2.4.7. Isomorphisms, isometries. An important class of bounded linear

operators is formed by isomorphisms. Recall that a transformation T : X Ñ Y
between topological spaces X and Y is called an isomorphism if T is invertible
and both T , T
1 are continuous. Spaces X and Y are called isomorphic if there
exists an isomorphism between them. So a linear map T : X Ñ Y between normed
spaces X and Y is an isomorphism if and only if T is invertible and T P LpX, Y q,
T
1 P LpY, X q.
Exercise 2.4.16. Show that a surjective linear map T : X Ñ Y between
normed spaces is an isomorphism if and only if there exist C, c ¡ 0 such
that
c}x} ¤ }T x} ¤ C }x} for all x P X.
In other words, an isomorphism preserves all distances up to a multiplicative
factor C {c. An isometry is an isomorphism T such that }T }  }T
1 }  1, i.e.
}T x}  }x} for all x P X.
Isometries preserve all distances exactly.
Example 2.4.17. Isometries on a Hilbert space are commonly called unitary
operators. As an example, in Section 1.6.7 we constructed an isometry T between
any pair of infinite-dimensional separable Hilbert spaces X and Y .
Example 2.4.18. The right shift Rpx1 , x2 , . . .q  p0, x1 , x2 , . . .q on any classical
sequence space considered in Example 1.1.19 is not an isometry because R is not
surjective, although }Rx}  }x}. The left shift Lpx1 , x2 , . . .q  px2 , x3 , . . .q is
obviously not invertible but is bounded, and }L}  1. (Why?)
Exercise 2.4.19. [Isomorphisms preserve vital structures in normed
spaces] Show that an isomorphism between normed spaces maps open
sets to open sets, closed sets to closed sets, convegrent sequences to
convergent sequences, complete spaces to complete spaces.
Unfortunately, most infinite-dimensional Banach spaces are not isomorphic to
each other. For example, A. Pelczynski proved that among the spaces c0 , Lp r0, 1s
and `q , 1 ¤ p, q ¤ 8, there are exactly two isomorphic ones, L2 r0, 1s and `2 , see
[8].
However, as we know all separable Hilbert spaces are isomorphic (even iso-
metric) to each other. We will also see in Section 3.1.5 that all finite-dimensional
Banach spaces of same dimension are isomorphic (but not isometric to each other).
The inverse mapping theorem of Section 3.1.2 below is a powerful tool to construct
isomorphisms.

2.4.8. Extensions and projections. We will now address the extension

problems for bounded linear operators. Similar to Section 2.3, we consider a normed
space X and its subspace X0 Consider an operator T0 P LpX0 , Y q where Y is some
normed space. An extension of T0 to the whole space X is an operator T P LpX, Y q
whose restriction on X0 coincides with T0 , i.e.
T |X0  T0 |X .
0

As we know from Section 2.3, every bounded linear functional can be extended
from either dense or closed subspace to the whole space. For dense subspaces, we
 2.4. BOUNDED LINEAR OPERATORS 62

can extend by continuity, while for closed subspaces the extension is guaranteed by
Hahn-Banach theorem.
For general linear operators, extension by continuity holds with the same proof
as in Proposition 2.3.2:
Proposition 2.4.20 (Extension by continuity). Let X0 be a dense subspace of
a normed space X, and Y be a Banach space. Then every operator T0 P LpX0 , Y q
admits a unique extension T P LpX, Y q. Moreover, }T }  }T0 }.
Unfortunately, extension from a closed subspace is not always possible, and
Hahn-Banach theorem does not generalize to bounded linear operators. There is
a simple geometric description of the situations when such extensions are possible.
To state it we need a general notion of projections in normed space (not necessarily
orthogonal).
Definition 2.4.21 (Projection). Let X0 be a closed subspace of a normed
space X. An operator P P LpX, X q is called a projection in X onto X0 if
(i) P pX q „ X0 ;
(ii) P x  x for all x P X0 , i.e. P |X0  IX0 .
Example 2.4.22. Any orthogonal projection in a Hilbert space is clearly a
projection in this sense. However, even in a Hilbert space there is a plenty of
non-orthogonal projections. (Construct one in a two-dimensional space.)
The following observation characterizes the subspaces from which extensions of
linear operators are possible.
Proposition 2.4.23 (Extensions of operators and projections). Let X0 be a
closed subspace of a normed space X. Then the following are equivalent:
(i) There exists a projection in X onto X0 . In this case we say that X0 is a
complemented subspace of X;
(ii) For every normed space Y , every operator T0 P LpX0 , Y q admits an extension
T P LpX, Y q.
Proof. Assume P is a projection in X onto X0 . Then for every operator
T0 P LpX0 , Y q, the operator T : T0 P P LpX0 , Y q is an extension.
Vice versa, consider the identity map I : X0 Ñ X0 . Its extension P : X Ñ X0
is clearly a projection in X onto X0 . 
Since every closed subspace in a Hilbert space is complemented, the extension
problem in Hilbert spaces always has a positive solution.
Unfortunately, in general normed spaces there may be uncomplemented sub-
spaces. Here is a synopsis of some of the known results, without proof:
Theorem 2.4.24 (Complemented subspaces). In what follows, X0 stands for
a closed subspace of a Banach space X.
(i) If either dim X0   8 or codim X   8 then X0 is complemented in X.
(ii) c0 is not complemented in `8 [10].
(iii) Every complemented subspace X0 of X is isomorphic to X if X is one of the
spaces `p , p P r1, 8s or c0 . These results are due to Pelczynski and Linden-
strauss [9, 6].
(iv) Every Banach space that is not isomorphic to a Hilbert space has an uncom-
plemented subspace. This is a theorem of Lindenstrauss and Tzafriri [7].
 2.4. BOUNDED LINEAR OPERATORS 63

(v) There exists a Banach space X without non-trivial complemented subspaces

X0 (i.e. such that dim X0  codim X0  8). This was proved in 1993 by
Gowers and Maurey [5].
(vi) If X0 is isomorphic to `8 then it is complemented in X.
Lec. 21: 10/27

2.4.9. Adjoint operators. The concept of adjoint operator is a generaliza-

tion of matrix transpose in linear algebra. Recall that if A  paij q is an n  n
matrix with complex entries, then the Hermitian transpose of A is the n  n matrix
A  paji q. The transpose thus satisfies the identity
(2.15) xA x, yy  xx, Ayy, x, y P Cn .
Now we would like to extend this to a general definition of the adjoint T  for a
linear operator T : X Ñ Y acting between normed spaces X and Y .
Definition 2.4.25 (Adjoint operator). Let T P LpX, Y q. The adjoint operator
T P LpY  , X  q is defined by
pT  f qpxq : f pT xq, f P Y  , x P X.
In order to see a similarity with (2.15), we adopt the following alternative
notation for the action of functionals, one that resembles the inner product:
(2.16) xf, xy : f pxq, f P X , x P X.
Notice that if X is a Hilbert space, this notation agrees with the inner product by
Riesz representation theorem (up to complex conjugation). In general, xf, xy does
not define an inner product since the arguments are taken from different spaces.
Then the definition of the adjoint reads as
xT  f, xy  xf, T xy, f P Y , x P X
and we see that this is a general form of (2.15).
Remark 2.4.26 (Adjoint operators on Hilbert spaces). For operators T on a
Hilbert space X  Y  H, the definition (2.4.25) of the adjoint operator T  takes
place with x, y denoting the inner product on H. This makes a small difference
– the inner product is conjugate linear in the second argument, while the action
of functionals (2.16) is just linear. So, for operators on a Hilbert space one has
paT q  aT  for a P C, while the general definition of adjoint for Banach spaces
incurs paT q  aT  .
One point has not been justified in Definition 2.4.25, why T  is a bounded
linear operator. We shall prove this now:
Proposition 2.4.27. For every T P LpX, Y q, we have T  P LpY  , X  q, and
}T  }  }T }.
Proof. Denoting as usual SX the unit sphere of X, and using notation (2.16),
we have
}T  }  sup }T  f }X   sup sup |xT  f, xy|  sup sup |xf, T xy|
f PS 
Y f PS  xPS
Y X xPS f PS  X Y

 sup }T x}Y (choosing f as a supporting functional of T x)

xPSX

 }T }
as required. 
 2.4. BOUNDED LINEAR OPERATORS 64

Exercise 2.4.28. Let T be the Hilbert-Schmidt integral operator with

kernel k pt, sq. Show that T  is also the Hilbert-Schmidt integral operator
with kernel k ps, tq.

Exercise 2.4.29. Let R and L denote the right and left shift operators
on `2 . Prove that R  L.

Exercise 2.4.30. (i) Let T P LpX, Y q and S P LpY, Z q. Show that

pST q  T  S  .
(ii) Let S, T P LpX, Y q and a, b P C. Show that paS bT q  āS  b̄T  .
(iii) Let T P LpX, Y q be such that T
1 P LpY, X q. Show that pT
1 q 
pT  q
1 .
The kernel and image of bounded linear operators are in a duality relation.
To state it, we consider a generalization of the notion of orthogonal complement,
which we studied in Section 1.5.1 for Hilbert spaces.

Definition 2.4.31 (Annihilator). An annihilator of a subset A of a normed

space X is the set AK „ X  defined as

AK  tf P X  : xf, xy  0 for all x P Au.

Proposition 2.4.32 (Duality of kernel and image). Let T P LpX, Y q. Then
pIm T qK  ker T  .
Proof. Let f P Y  . Then f P ker T  means that T  f  0, which is equivalent
to xT  f, xy  xf, T xy  0 for all x P X, which means that f P pIm T qK . 

Corollary 2.4.33. Let H be a Hilbert space, and T P LpH, H q. Then the

orthogonal decomposition holds:

 Im T ` ker T  .
H

Proof. By Proposition 2.4.32, we have pIm T qK  pIm T qK  ker T  . (Why

does the first identity hold?) By Proposition 1.5.7, the proof is complete. 

Exercise 2.4.34. [Duality of kernel and image II] For a subset A € X  ,

define the “pre-anihilator” as

AK  tx P X : xf, xy  0 for all f P Au.

Let T P LpX, Y q. Prove that

ker T  pIm T  qK .
Deduce that
pker T qK … Im T  .
Give an example of a linear operator such that pker T qK  Im T  .
 2.4. BOUNDED LINEAR OPERATORS 65

2.4.10. Application: ergodic theory. Ergodic theorems allow one to com-

pute space averages as time averages. Let us first state and prove a preliminary
form of von Neumann’s ergodic theorem; its interpretation will follow.
Theorem 2.4.35 (von Neumann ergodic theorem). Let U be a unitary operator
on a Hilbert space H. Let P denote the orthogonal projection onto the invariant
subspace tx P X : U x  xu. Then, for all x P H, we have
N
1
1 ¸ n
lim U x  P x.
N Ñ8 N
n0

Proof. It suffices to prove the result for x P ker P and for x P Im P , because
then the result will follow for all x P H by the orthogonal decomposition H 
ker P ` Im P . (Check!)
For x P Im P the result is trivial because in this case U n x  U x  x and
P x  x. So let x P ker P . We will first find a convenient representation of ker P .
By definition, Im P  kerpI
U q  kerpI
U  q because for unitary operators,
U x  x if and only if U  x  x. (Check!) Therefore, using the duality between
kernels and images, Corollary 2.4.33, we have
ker P  pkerpI
U  qqK  ImpI
U q.
Therefore, every x P ker P can be approximated arbitrarily well by vectors of the
form pI
U qy. For x  pI
U qy, we arrive at a telescoping sum
N
1
1 ¸ n
U x  p x
U N xq Ñ 0 Ñ 8.
1
as N
N n0 N

This carries over to vectors x P ImpI
U q by a simple approximation argument.

The proof is complete. 

Exercise 2.4.36. Write down the approximation argument in this

proof.
Now we explain the implications of Theorem 2.4.35 for time and space averages.
Consider first the simple example of a discrete dynamical system studied by Weyl.
Let T denote the unit circle. We put a particle on the circle, and consider its
consecutive rotations by some fixed angle θ P p0, 2π q. Equivalently, we consider the
sequence
nθ mod 2π, n  0, 1, 2, . . .
If θ{2π is rational then this sequence has finitely many values. If θ{2π is irrational,
then the number of values is infinite. Moreover, in the latter case simulations
suggest that the values become uniformly distributed for large N on the circle, see
the picture.
This observation was formalized by the Weyl’s ergodic theorem, which can be
stated as follows. For every number θ P p0, 2π q such that θ{2π is irrational, and
every measurable subset A „ r0, 2π s, one has

(2.17)
|tn ¤ N : mod 2π P Au|
nθ pAq
Ñ µ2π as N Ñ 8.
N
where µ denotes the Lebesgue measure.
 2.4. BOUNDED LINEAR OPERATORS 66

Figure 2.4. Rotations by angle θ are equidistributed on the circle

Theorem 2.4.35 implies a very general form of Weyl’s ergodic theorem, for an
arbitrary ergodic measure-preserving transformation of a probability space (instead
of an irrational rotation of the circle).
Definition 2.4.37 (Ergodic transformation). Let pΩ, Σ, µq be a probability
space. A transformation T : Ω Ñ Ω is called measure-preserving if
µpT
1 pAqq  µpAq
for all measurable subsets A „ Ω.4 A one-to-one, measure preserving transforma-
tion T is ergodic if the only functions f P L2 pΩ, Σ, µq which satisfy f pT ω q  f pω q
for almost all ω P Ω are the constant functions.
Exercise 2.4.38. Show that T is ergodic if and only if for all measur-
able subset A „ Ω, T
1 pAq  A implies µpAq  0 or µpAq  1.

Theorem 2.4.39. Consider a measure-preserving, ergodic transformation T on

a probability space pΩ, Σ, µq. For every f P L2 pΩ, Σ, µq one has
N
1 »
1 ¸
(2.18) lim f pT ω q 
n
f dµ
N Ñ8 N
n0 Ω

where the convergence is in the L2 norm.

Proof. To see a connection with ergodic Theorem 2.4.35, define an operator
U : L2 pΩ, Σ, µq Ñ L2 pΩ, Σ, µq by
Uf  f  T, i.e. pU f qpωq  f pT ωq, ω P Ω.
Since T is measure-preserving, U is a unitary operator. (Check!) Since T is ergodic,
the invariant subspace of U is the space of constants. The orthogonal projection
P in L2 pΩ, Σ, µq onto the subspace of constants is the integral in the right side of
(2.18) (see Exercise 1.5.11). The claim then follows from Theorem 2.4.35. 

Weyl’s ergodic theorem (2.17) is a partial case of Theorem 2.4.39 for Ω being
the unit circle, T being the rotation of the circle by an irrational angle θ (why is T
ergodic?) and for f  1A .

p q  tω P Ω :
4Here T 1 A Tω P Au is the preimage of A under T .
 2.4. BOUNDED LINEAR OPERATORS 67

2.4.11. Additional Exercises.

Exercise 2.4.40. [Integral operators in C r0, 1s] Consider the integral
operator T defined by (2.11) with kernel k pt, sq P C pr0, 1s2 q. Then T maps
continuous functions into continuous functions, i.e. T : C r0, 1s Ñ C r0, 1s.
Compute the norm of T .

Exercise 2.4.41. [Schur’s test] Assume that a kernel function k pt, sq

satisfies
»1
sup |kpt, sq| ds : M1   8,
Pr s
t 0,1 0
»1
sup |kpt, sq| dt : M2   8.
Pr s
s 0,1 0

Show that the intergal operator (2.11) T : L2 r0, 1s Ñ L2 r0, 1s with kernel
k pt, sq is bounded, and5 a
}T } ¤ M1 M2 .
Exercise 2.4.42. [Multiplication operator] Consider a multiplier func-
tion k ptq P C r0, 1s, and define a linear operator T : C r0, 1s Ñ C r0, 1s by
pT f qptq  kptq f ptq.
Show that T is a bounded linear operator and compute its norm.

Exercise 2.4.43. [Annihilators] Let A, B be subsets of a normed space

X. Prove the following:
(i) AK is a closed linear subspace of X  .
(ii) If A „ B then AK … B K .
(iii) pA Y B qK  AK X B K . Give an example where pA X B qK  AK Y B K .
(iv) AK  pSpan AqK .
(v) pĀqK  AK . (This property was used in the proof of Corollary 2.4.33.)
(vi) Suppose X0 is a closed linear subspace of X. Then X0K  t0u is
equivalent to X0  X.

Exercise 2.4.44. [Isomorphisms] Let X be a Banach space. Show that

the isomorphisms on X form an open subset of LpX, X q, and that the
inversion map T ÞÑ T
1 is continuous on this subset.

Exercise 2.4.45. [Sesquilinear forms] Let H be a Hilbert space. A

sesquilinear form on H is a function B : H  H Ñ C which is linear in the
first argument and conjugate-linear in the second argument, i.e.
B pa1 x1 a2 x2 , y q  a1 B px1 , y q a2 B px2 , y q,
B px, b1 y1 b2 y2 q  b1 B px, y1 q b2 B px, y2 q.
An example of a sesquilinear form is B px, y q  xT x, y y where T P LpH, H q.
p r sÑ r s
5In fact, under the conditions of Schur’s test, T is bounded as an operator L 0, 1 Lp 0, 1
P r 8s
for all p 1, . Indeed, for p  1 and p 8 this is an exercise. The result for intermediate p
follows at once by Riesz-Thorin theorem (which we do not discuss here).
 2.4. BOUNDED LINEAR OPERATORS 68

Consider a sesquilinear form B px, y q which satisfies

|B px, yq| ¤ M }x}}y}, x, y PH
for some number M . Show that there exists an operator T P LpH, H q
with }T } ¤ M and such that
B px, y q  xT x, y y for all x, y P H.
 CHAPTER 3

Main principles of functional analysis

Lec. 22: 10/29
In this chapter we shall study the three theorems that, together with Hahn-
Banach theorem, form the main principles of functional analysis. Those are the
open mapping theorem, the uniform boundedness principle, and the closed graph
theorem.

3.1. Open mapping theorem

3.1.1. Statement and proof. This result was proved by S. Banach.
Theorem 3.1.1 (Open mapping theorem). Let X, Y be Banach spaces. Then
every surjective linear operator T P LpX, Y q is an open map, i.e. T maps open sets
in X to open sets in Y .
The proof of the open mapping theorem relies on Baire category theorem, which
states that every complete metric space M is a set of second category, i.e. M can
not be represented as a countable union of nowhere dense sets. Recall that a subset
A „ M is nowhere dense set if there is no neighborhood in X on which A is dense.
Equivalently, A is nowhere dense if the interior of the closure of A is empty.
The open mapping theorem states that for every open set U „ X, every y P T U
is an interior point of T U . We claim that it suffices to show this for U being the
unit ball BX and for y  0:
Claim 3.1.2. To prove the open mapping theorem, it suffices to find ε ¡ 0 such
that
(3.1) T BX … εBY .
Proof of Claim. Let U „ X be open and choose y P T U . We find x PU
such that y  T x. Since U is open, there exists δ ¡ 0 such that
U …x δBX .
Applying T to both sides and using (3.1), we conclude that
TU … T px δBX q  y δT BX …y δεBY ,
so y is an interior point of T U . 
We will now prove the Claim. In view of application of Baire category theorem,
we represent ¤
X nBX .
n N P
Therefore ¤
Y  TX  nT BX .
P
n N

69
 3.1. OPEN MAPPING THEOREM 70

By Baire category theorem, there exists n P N such that nT BX is not a nowhere

dense set. Thus T BX is not a nowhere dense set, i.e. its closure has nonempty
interior. So there exist y P Y and ε ¡ 0 such that
T BX …y εBY .
By symmetry, T BX …
y εBY . Hence by convexity (check this!) we have
T BX … εBY .
As we see, we have almost proved the Claim, except for the closure. Unfortu-
nately, in general K … D does not imply K … D even for convex and symmetric
sets K, D in a Banach space. (Give a counterexample!) However, this is true for
perfectly convex sets, defined as follows.
Definition 3.1.3 (Perfectly convex set). A set K in a Banach space Y is called
perfectly
°8
convex if for every
°8
sequence pxk q8
k1 and every numbers λk ¥ 0 such that
k1 λ k  1, one has k 1 λ k x k P K.
Convex sets satisfy this property only for finite sequences pxk q. (Why?) There-
fore, every perfectly convex set is convex, but not vice versa. (Give an example.)
Lemma 3.1.4 (Perfectly convex sets). Let K be a perfectly convex set in a
Banach space Y . If K … εBY for some ε ¡ 0, then K … 2ε BY .

Proof. Assume B : εBY „ K; we would like to show that 1

2B „ K. The
assumption clearly implies that

B „K1
2
B,

for the right side is the ε-neighborhood of K in Y . Iterating this inclusion gives
1
1
B „K 2
K
2
B K
1
2
K
1
4
B


„K 1
2
K
1
4
K
1
2
B K
1
2
K
1
4
K
1
8
B „ 
Therefore1
B „K 1
2
K
1
4
K
1
8
K 
By perfect convexity (check!), we have

B„ K    „ K.
1 1 1 1 1
K K K
2 2 4 8 16
This proves the lemma. 

Exercise 3.1.5. Verify the steps of the above proof where we used
Minkowski sums and series of sets.
Now we are ready to complete the proof of the open mapping theorem. By
Lemma 3.1.4, it suffices to show that K  T BX is perfectly convex. This is easy to

°
° All sums and series of sets are in Minkowski sense. The sum of sets is defined as k Ak 
1
t k ak : ak P Ak u. The same for infinite sums (series), where we insist on the convergence of
°
k ak .
 3.1. OPEN MAPPING THEOREM 71

° consider any sequence pT xk q

check. Indeed, „ T BX with xk P BX , and numbers
λk such that k λk  1. Then
¸
¸
(3.2) λk T xk T λ k xk ,
k k
°
provided that the series k λk xk converges. It indeed converges absolutely:
¸  ¸ ¸



λ k xk  ¤ λk }xk } ¤ λk  1.
k k k
°
By completeness of X, the series k λk xk converges to a vector in BX . It follows
that the right side of (3.2) belongs to T BX  K, as required. This completes the
proof of the open mapping theorem.  Lec. 23: 11/1

3.1.2. Inverse mapping theorem. As an immediate consequence of the

open mapping theorem, we obtain:
Theorem 3.1.6 (Inverse mapping theorem). Let X, Y be Banach spaces. Then
every bijective linear operator T P LpX, Y q is an isomorphism, i.e. T
1 P LpY, X q.
Proof. The open mapping theorem states that the preimages of open sets
under T
1 are open, hence T
1 is a continuous map. 
The inverse mapping theorem is often used to establish stability of solutions of
linear equations. Consider a linear equation in x in Banach space:
(3.3) Tx  b
with T P LpX, Y q and b P Y .
Assume a solution x exists and is unique for every right hand side b. Then,
by inverse mapping theorem, the solution x  xpbq is continuous with respect to b.
In other words, the solution is stable under perturbations of the right hand side of
(3.3).
In case T is not injective (but is surjective) in the inverse mapping theorem,
one can still apply inverse mapping theorem to the injectivization of T :
Corollary 3.1.7 (Surjective operators are essentially quotient maps). Let
X, Y be Banach spaces. Then every surjective linear operator T P LpX, Y q is a
composition of a quotient map and an isomorphism. Specifically,
T  T̃ q,
where q : X Ñ X { ker T is the quotient map, T̃ : X { ker T ÑY is an isomorphism.
Proof. Let T̃ be the injectivization of T constructed in Example 1.1.24. By
construction, T  T̃ q and T̃ is injective. Since T is surjective, T̃ is also surjective.
Therefore, by inverse mapping theorem T̃ is an isomorphism, completing the proof.

3.1.3. Equivalent norms. Sometimes one wants to consider several norms
on the same space, such as }  }8 and }  }1 on C r0, 1s. In this case one naturally
needs to compare them.
Definition 3.1.8 (Equivalent norms). Two norms }  } and ~  ~ on a linear
vector space E are called equivalent if there exist C, c ¡ 0 such that
c}x} ¤ ~x~ ¤ C }x} for all x P E.
 3.1. OPEN MAPPING THEOREM 72

In other words, the norms }  } and ~  ~ are equivalent if the identity operator
(3.4) I : pE, }  }q Ñ pE, ~  ~q
is an isomorphism.
Exercise 3.1.9. Show that two norms on a linear vector space E are
equivalent if and only if they generate the same topology on E.

Proposition 3.1.10 (Domination and equivalence of norms). Consider two

norms }} and ~~ on a linear vector space E such that E is complete with respect
to both norms. Suppose that one norm dominates the other, i.e. one can find C
such that
~x~ ¤ C }x} for all x P E.
Then the two norms are equivalent.
Proof. The claim follows from inverse mapping theorem applied to the iden-
tity map (3.4). 
Proposition 3.1.10 indicates that it is difficult to construct different good norms
on the same space. Either the space will be incomplete or the norms need to be
incomparable. This is a simple way to prove incompleteness of various spaces.
As an example, consider the norms }  }1 and }  }8 on C r0, 1s. On the one
hand, }  }1 ¤ }  }8 . On the other hand, the norms are not equivalent – one can
easily construct functions with }f }1  1 and }f }8 arbitrarily large. (Do this.) By
Proposition 3.1.10, C r0, 1s must be incomplete with respect to one of these norms.
Since it is complete with its natural norm }}8 , it follows that C r0, 1s is incomplete
with respect to }  }1 .
This argument is flexible and applies to a whole range of norms. It implies that
there is essentially only one natural norm on C r0, 1s, namely the sup-norm }  }8 .
Exercise 3.1.11. [Direct sums of normed spaces] Let X and Y be
normed spaces and p P r1, 8s. Define the direct sum of X `p Y as the
Cartesian product X  Y equipped with the norm
}px, yq} : p}x}p }y}p q1{p if p   8, }px, yq} : maxp}x}, }y}q if p  8.
Show that X `p Y is a normed space, and all norms }px, y q}p , p P r1, 8s,
are equivalent to each other.
For this reason, the index p is usually omitted from notation, and
the space X ` Y is called the direct sum of X and Y .

3.1.4. Isomorphic embeddings. As we know, the kernel of every operator

T P LpX, Y q is always a closed subspace. The image of T may or may not be closed.
The following result characterizes operators with closed images.
Proposition 3.1.12 (Isomorphic embeddings). Let T P LpX, Y q be an operator
between Banach spaces X and Y . Then the following are equivalent:
(i) T is an isomorphic embedding, i.e. T acts as an isomorphism between spaces
X and Im T „ Y ;
(ii) T is injective and it has closed image;
(iii) T is bounded below, i.e. one can find c ¡ 0 such that
}T x} ¥ c}x} for all x P X.
 3.1. OPEN MAPPING THEOREM 73

Proof. (i) ñ (ii). Recall that isomorphisms preserve completeness of spaces.

Since X is complete, it follows that the subspace Im T „ Y is complete. By
Exercise 1.3.2, Im T is closed. Injectivity of T is an obvious consequence of the
isomorphic embedding property.
(ii) ñ (iii). Considering T as an operator from X to Im T we see that T is
injective and surjective. By inverse mapping theorem, T is an isomorphism.
(iii) ñ (i). We have
c}x} ¤ }T x} ¤ C }x} for all x P X
where C  }T }. It follows that T is an isomorphic embedding. (How?) 
3.1.5. Finite dimensional normed spaces. Finite dimensional normed spaces
are the simplest examples of Banach spaces. As we will see now, they are all com-
plete, all isomorphic (but not isometric) to each other, all of their subspaces are
closed and all linear operators on them are bounded.
Theorem 3.1.13. Every n-dimensional normed space X is isomorphic to the
Euclidean space `n2 . Consequently, all n-dimensional normed spaces are isomorphic
to each other.
Proof. We construct an isomorphism between X and `n2 which that identifies
a basis of X with the canonical basis of `n2 . To this end, consider a basis u1 , . . . , un
of X, and define the operator T : `n2 Ñ X by
¸
n
Tx  xk uk for x  px1 , . . . , xn q P `n2 .

k 1
Boundedness of T follows from triangle and Cauchy-Schwarz inequalities:
¸
n
¸
n 1{2
¸
n 1{2
}T x} ¤ |xk | }uk } ¤ |xk |2 }uk }2  M }x},

k 1 
k 1 
k 1
°n 1{2
where M  k1 }uk }
2
.
Moreover, T is bijective because puk q is a basis of X. (Why?) By inverse
mapping theorem, T is an isomorphism. 
Exercise 3.1.14. Instead of using the inverse mapping theorem, give
a proof based on the compactness of the unit sphere of `n2 .

Exercise 3.1.15. Any two n-dimensional normed spaces X and Y are

isomorphic but not necessarily isometric. Show that X and Y are iso-
metric if and only if BY  T BX for some invertible linear transformation
T : X Ñ Y . Show that `22 is not isometric to `28 .

Remark 3.1.16 (Banach-Mazur distance). A quantitative notion of isomor-

phism is given by the Banach-Mazur distance dpX, Y q between two isomorphic
normed spaces X and Y . It is defined as
dpX, Y q  inf t}T } }T
1 } : T : X ÑY is an isomorphismu.
A theorem of F. John (see e.g. [1]) is a quantitative form of Theorem 3.1.13. It
states that every n-dimensional normed space X satisfies
dpX, `n2 q ¤
?n.
 3.2. CLOSED GRAPH THEOREM 74

It follows that for every two n-dimensional normed spaces X and Y , one has
dpX, Y q ¤ n.
(Why?) E. Gluskin [4] proved in 1981 that this upper bound is asymptotically
sharp, i.e. there exists an absolute constant c ¡ 0 such that, for every n P N one
can construct n-dimensional normed spaces Xn and Yn with
dpXn , Yn q ¥ cn.
Corollary 3.1.17. (i) Every finite dimensional normed space is a Banach
space.
(ii) Every finite dimensional subspace of a normed space is closed.
(iii) Every linear operator on a finite dimensional normed space is bounded.
(iv) Every two norms on a finite dimensional normed space are equivalent.
Proof. (i) Since an n-dimensional normed space X is isomorphic to the com-
plete space `n2 , X itself is complete.
(ii) Let Y be a finite-dimensional subspace of a normed space X. Then Y
is complete by part (i). Therefore Y is closed. (Indeed, if a sequence pyn q € Y
converges to x P X then pyn q is Cauchy in Y , hence its limit x is in Y .)
(iii) By Proposition 2.4.7, every linear operator on `n2 is bounded, }T } ¤ }T }HS .
Since an n-dimensional normed space X is isomorphic `n2 , the same is true for linear
operators on X.
(iv) Let }  } and ~  ~ be two norms on X. By part (iii), the identity map
I : pX, }  }q Ñ pX, ~  ~q as well as its inverse are bounded. This completes the
proof. 
Exercise 3.1.18. [Operators from finite dimensional spaces] Let X and
Y be normed spaces, and X be finite dimensional. Show that every
linear operator T : X Ñ Y is bounded. (Hint: identify X with `n2 by some
isomorphism, and show boundedness of T similarly to the argument of
Theorem 3.1.13.)
Lec. 24: 11/3

3.2. Closed graph theorem

3.2.1. Statement and proof. Closed graph theorem is an alternative way
to check whether a given linear operator is bounded. This result characterizes
bounded operators in terms of their graphs.
Definition 3.2.1 (Graph of an operator). Let T : X Ñ Y be a linear operator
between normed spaces X and Y . The graph of T is the following subset of the
direct sum2 X `1 Y : (
ΓpT q  px, T xq : x P X .
Clearly, ΓpT q is a linear subspace of the normed space X `1 Y . The main result
of this section is that ΓpT q is closed if and only if T is bounded.
Let us compare these two notions, boundedness (equivalently, continuity) of T
and having closed graph. T is continuous if and only if
(3.5) xn ÑxPX implies T xn Ñ T x.
2The direct sum of normed spaces is studied in Exercise 3.1.11. Instead of X` 1 Y , one can
`
choose to work with X p Y for any p P r 8s
1, . As we know this defines the same topology, and
will not affect the arguments in this section.
 3.2. CLOSED GRAPH THEOREM 75

In contrast, ΓpT q is closed if and only if

(3.6) xn Ñ x P X and T xn Ñ y P Y imply y  T x.
It is clear from these two lines that continuity always implies the closed graph
property:
Proposition 3.2.2. Let T : X Ñ Y be a linear operator between normed spaces
X and Y . If T is bounded then ΓpT q is closed.
The opposite statement is nontrivial and requires completeness of both spaces
X and Y :
Theorem 3.2.3 (Closed graph theorem). Let T : X Ñ Y be a linear operator
between Banach spaces X and Y . If ΓpT q is closed then T is bounded.
Proof. The direct sum X `1 Y is a Banach space (Exercise 1.3.13). The graph
ΓpT q is a closed linear subspace of X `1 Y , hence ΓpT q is a Banach space itself.
Consider the linear operator
u : ΓpT q Ñ X, upx, T xq : x.
Then u is a bounded, surjective and injective linear operator between two Banach
spaces. (Check!) By the open mapping theorem, u
1 is bounded. This means that
there exists a number M such that
}x} }T x}  }px, T xq} ¤ M }x} for all x P X.
The inequality }T x} ¤ M }x} implies that T is bounded. 
3.2.2. Interpretation and an example. Recalling the interpretation of con-
tinuity (3.5) and closed graph property (3.6), we can make advantage of the closed
graph property – the extra assumption that T xn converges in Y . So, to check the
continuity of a linear operator T using the definition (3.5), one can always assume
for free that T xn converges. Checking continuity no longer requires proving that
the limit exists; it reduces to checking the consistency of the limits of xn and T xn .
As an example, consider the simplest differential operator

T  dtd , T : C 1 r0, 1s Ñ C r0, 1s

where C 1 r0, 1s is considered as a subspace of C r0, 1s, i.e. with respect to sup-norm.
Lemma 3.2.4. The differential operator T has closed graph.
Proof. Let fn Ñ f and T fn  fn1 Ñ g in C r0, 1s, i.e. uniformly. Then, by
the theorem on differentiation under the limit,3 plimn fn q1  limn fn1 , i.e. g  T f .
This completes the proof. 
Nevertheless, as we know the differential operator is unbounded. (Why?) This
does not contradict the closed graph theorem, because C 1 r0, 1s is not complete
under the sup-norm. If we consider C 1 r0, 1s under its natural norm }f }8 }f 1 }8
in which it is a Banach space (Exercise 1.3.12), then the differential operator will
obviously be bounded.

p
3The theorem on differentiation under the limit states that lim f q1  limn fn1 provided
that fn1 converges uniformly and fn t0 converges for some point t0 .
n n
p q
 3.3. PRINCIPLE OF UNIFORM BOUNDEDNESS 76

3.2.3. Symmetric operators on Hilbert spaces. A remarkable application

of closed graph theorem is that the symmetry property of an operator always implies
boundedness:
Theorem 3.2.5 (Hellinger-Toeplitz). Let T : H ÑH be a linear operator on
a Hilbert space H. Suppose that
(3.7) xx, T xy  xT x, yy for all x, y P H.
Then T is bounded.
Proof. By the closed graph theorem, it suffices to check that the graph of T
is closed. To this end we choose convergent sequences xn Ñ x, T xn Ñ y in H. We
would like to show that y  T x. It suffices to show that
xz, yy  xz, T xy for all z P H.
(Why?) This follows by using continuity of the inner product and (3.7) twice:
xz, yy  lim
n
xz, T xn y  lim
n
xT z, xn y  xT z, xy  xz, T xy.
The proof is complete. 
Hellinger-Toeplitz theorem identifies the source of considerable difficulties in
mathematical physics. Many natural operators, such as differential, satisfy the
symmetry condition (3.7) but are unbounded. Hellinger-Toeplitz theorem declares
that such operators can not be defined everywhere on the Hilbert space. For ex-
ample, one can never define a useful notion of differentiation that would make all
functions in L2 differentiable.
This explains that working with unbounded operators one has to always keep
track of their domains. For example, a linear operator T on a Hilbert space H is
called symmetric if the domain of T is dense in H, and (3.7) holds. An example of
a symmetric operator is the differential operator on L2 r0, 1s

T  i dtd
with domain
Dom T  tf P L2 r0, 1s : f P C 1 r0, 1s, f p0q  f p1q  1u.
3.3. Principle of uniform boundedness
Lec.25: 11/5
3.3.1. Statement and proof. The principle of uniform boundedness is a
result due to Banach and Steinhaus.
Theorem 3.3.1 (Principle of uniform boundedness). Let X and Y be Banach
spaces. Consider a family of bounded linear operators T „ LpX, Y q. Assume that
T is pointwise bounded, i.e.
(3.8) sup }T x}   8 for every x P X.
P
T T
Then T is uniformly bounded, i.e.
sup }T }   8.
P
T T

Note that the reverse direction is trivially true – uniform boundedness implies
pointwise boundedness. (Why?)
 3.3. PRINCIPLE OF UNIFORM BOUNDEDNESS 77

Proof. For every x P X, define

M pxq  sup }T x}.
P
T T

Pointwise boundedness of T means that

¤
X  Xn , where Xn  tx P X : M pxq ¤ nu.
P
n N

Baire category theorem implies that one of Xn is not a nowhere dense subset of
X. Since Xn is closed (why?), Xn has nonempty interior. Summarizing, we have
shown that there exist n P N, x0 P X and ε ¡ 0 such that
… x0 εBX .
Xn
By symmetry of Xn , we also have Xn …
x0 εBX . Hence by convexity of Xn ,
we have
Xn … εBX .
(Check!) By definition of Xn this means that for x P X,
}x} ¤ ε implies sup }T x} ¤ n.
T T P
It follows that for every x P X,
sup }T x} ¤ }x}.
n
P
T T ε
This implies that
sup }T } ¤  8
n
P
T T ε
as required. 

Exercise 3.3.2. Check that the sub-level sets Xn in the proof above
are closed, convex and symmetric. (All these properties were used in
the argument).

Remark 3.3.3 (Principle of condensation of singularities). Banach and Stein-

haus called their Theorem 3.3.1 the principle of condensation of singularities for the
following reason. Suppose a family T „ LpX, Y q is not uniformly bounded. This
means that the set of vectors tT x : x P BX , T P T u is unbounded. Theorem 3.3.1
states that T is not even pointwise bounded, so there exists one vector x P X with
unbounded trajectory tT x : T P T u. One can say that the unboundedness of the
family T is condensated in a single “singularity” vector x.
Remark 3.3.4 (Completeness). In the proof of Theorem 3.3.1, the complete-
ness of only X was used. So the result still holds if X is a Banach space and Y is
a normed space.
3.3.2. Weak and strong boundedness. Principle of uniform boundedness
can be used to check whether a given set in a Banach space is bounded.
Corollary 3.3.5 (Weak and strong boundedness). Let A be a subset of a
Banach space X. Assume that A is weakly bounded, i.e.
sup |f pxq|   8 for every f P X .
P
x A
 3.3. PRINCIPLE OF UNIFORM BOUNDEDNESS 78

Then A is (strongly) bounded, i.e.

sup }x}   8.
P
x A

Here again the reverse statement is trivially true – (strong) boundedness triv-
ially implies weak boundedness.
Proof. We embed X into X  using the canonical embedding that we studied
in Theorem 2.3.12. So we consider vectors x P A as bounded linear functionals on
X  acting as xpf q : f pxq, f P X  . Rewriting the weak boundedness assumption
as supxPA |xpf q|   8 for f P X  , we may understand this assumption as point-
wise boundedness of the family A „ X   LpX  , Rq. The principle of uniform
boundedness implies that supxPA }x}X   supxPA }x}X   8, as required. 
Remark 3.3.6. Using Corollary 3.3.5, one can weaken the assumption (3.8) in
the principle of uniform boundedness to the following one:
sup |f pT xq|   8 for every x P X, f P Y .
P
T T

(Why?)
3.3.3. Application to convergence of Fourier series. A basic and classi-
cal question in Fourier analysis is – when does Fourier series of a function f on an
interval converge to f ?
Hilbert space technique provides a complete answer to this question in the
space L2 . As we know from Theorem 1.6.20, the Fourier series of every function in
L2 r
π, π s converges to f in the L2 -norm.
In function spaces other than L2 , the answer to this problem is often nontrivial
and even negative. Unfortunately, such is the situation in the space of continuous
functions C r
π, π s. There exist continuous functions f whose Fourier series do
not converge in C r
π, π s (i.e. uniformly). This follows from a somewhat stronger
result, which in turn is a consequence of the principle of uniform boundedness:
Theorem 3.3.7 (Divergent Fourier series). There exists a function f P C r
π, πs
whose partial Fourier sums
¸
n
pSn f qptq  fppk qeikt
k 
n
forms an unbounded sequence of complex numbers at t  0. In particular, the
Fourier series of f is unbounded (hence divergent) at t  0.
Proof. Recall from (2.13) that the partial Fourier sums can be represented
via convolution with Dirichlet kernel Dn :
»π
sinpn 21 qθ
pSn f qptq  2π1 Dn pt
sqf psq ds, where Dn pθq  .

π sin 21 θ
We are interested in the behavior of pSn f qp0q. These are obviously linear functionals
on C r
π, π s, which we denote
»π
φn pf q : pSn f qp0q  Dn psqf psq ds.
1
2π
π
(We used that Dn is an even function).
 3.3. PRINCIPLE OF UNIFORM BOUNDEDNESS 79

Since Dirichlet kernel Dn is continuous, φn are bounded linear functionals on

C r
π, π s, and »π
}φn }  2π1 |Dn psq| ds

π
(Why?)
On the other hand, evaluating these integrals by hand one can see that
(3.9) }φn } Ñ 8 as n Ñ 8.
(see the exercise below). Therefore, pφn qnPZ is not a uniformly bounded family of
linear functionals.
By principle of uniform boundedness, this family is not even pointwise bounded.
This means that there exists a function f P C r
π, π s such that the set of numbers
tφn pf q : n P Nu is unbounded. This is exactly what we wanted to show. 
Exercise 3.3.8. [L1 norm of Dirichlet kernel] Show that
»π
1
|Dn psq| ds ¥ c log n
2π
π
where c ¡ 0 is an absolute constant. This validates step (3.9) in the
argument above. Hint: show that the area under k-th bump of the graph
of the Dirichlet kernel is Á k (see the picture); as there are  n bumps on
r
π, πs this will show that the total area of the bumps is °nk1 1{k Á log n.

Figure 3.1. Dirichlet kernel

Remark 3.3.9 (Convergence of Fourier series). (i) Analyzing the proof of The-
orem 3.3.7 one can show that continuous functions whose Fourier series con-
verge pointwise are rare. Precisely, the set of such functions is of first Baire
category in C r
π, π s (i.e. it is a countable union of nowhere dense sets).
(ii) Nevertheless, for every continuous function f , and even for f P Lp r
π, π s,
p ¡ 1, Fourier series converges to f almost everywhere. So the set of points
of divergence is always small. This is a deep result of L. Carleson [2].
(iii) For functions in L1 , Carleson result is generally false. Kolmogorov (apparently
when he was 19-21!) constructed a function f P L1 r
π, π s whose Fourier series
diverges everywhere.
(iv) If f is differentiable at a point t, then Fourier series of f converges to f at t.
This is called Dirichlet-Dini condition.
 3.3. PRINCIPLE OF UNIFORM BOUNDEDNESS 80

Lec. 26: 11/8

3.3.4. Schauder bases. The notion of Hamel basis, which we studied in Sec-
tion 1.1.4, has a serious drawback. In all infinite-dimensional Banach spaces, Hamel
bases are uncountable, see Exercise 3.3.22. This makes it difficult to use Hamel bases
in practice. There exists an alternative notion of basis, which is more tailored to
the needs of analysis:
Definition 3.3.10 (Schauder basis). A sequence pxk q8k1 in a Banach space X
is called a Schauder basis of X if every vector x P X can be uniquely expressed as
a convergent series
8̧
(3.10) x ak xk

k 1

for some scalars ak .

Exercise 3.3.11. Show that only separable spaces X may have Schauder
bases.
If pxk q is a Schauder basis then the sequence pxk q is clearly linear independent
and complete (recall from Definition 1.6.15 that completeness means that Spanpxk q
is dense in X.) However, the basis property is considerably stronger than linear
independence and completeness. Completeness means that for arbitrary vector
x P X and arbitrary precision ε ¡ 0, one can find a linear combination of basis
elements that approximates x well:
 ¸
n 
(3.11)

x
ak xk  ¤ ε.


k 1

However, the coefficients ak  ak pεq may depend on ε. The limit limεÑ0 ak pεq
generally does not exist, as will be clear from the next exercise. In contrast, the
basis property guarantees that one can achieve approximation (3.11) with the same
coefficients ak independent of ε by only increasing the number of terms n in the
linear combination.
Exercise 3.3.12. [Completeness and basis property] In a Hilbert space
H, find a complete and linearly independent sequence pxk q which is not a
Schauder basis. Hint: construct xk so that it converges to some nonzero
vector in H; show that such sequences are never Schauder bases.

Example 3.3.13 (Bases in sequence spaces). An orthogonal basis of a Hilbert

space is a Schauder basis. (Check the uniqueness of represenation of x!)
In sequence spaces `p , 1 ¤ p   8 and c0 , the canonical (coordinate) system
forms a Schauder basis. (Check!)
In `8 there is no Schauder basis because this space is not separable.
Example 3.3.14 (Bases in Lp ). One can show (albeit somewhat nontrivially)
that Fourier basis (1.16) and the trigonometric system each form Schauder bases
in Lp r
π, π s, 1   p   8.
The Haar system defined in Exercise 1.6.34 also forms a Schauder basis in
Lp r
π, π s, 1 ¤ p   8. (Check!)
In L8 r
π, π s there is no Schauder basis because this space is not separable.
 3.3. PRINCIPLE OF UNIFORM BOUNDEDNESS 81

Example 3.3.15 (Basis of the space of continuous functions). In C r0, 1s, the
natural candidates fail to be Schauder bases. The Fourier basis is not a Schauder
basis – otherwise this would imply that the Fourier series of every continuous func-
tion would converge in C r0, 1s (why?), which would contradict Theorem 3.3.7.
The sequence of monomials 1, t, t2 , . . . is not a Schauder basis of C r0, 1s either.
(Prove this!)
The most known Schauder basis of C r0, 1s is the so-called Schauder system of
wavelets. Its mother wavelet φptq is obtained by integration of Haar mother wavelet,
i.e. #
»t
t P r0, 1{2q
φptq  hpsq ds 
t,
0 1
t, t P r1{2, 1q
Then we consider the translates and dilates of the mother wavelet:
φkl ptq  φp2k t
lq, k  0, 1, 2, . . . , l  0, 1, 2, . . . , 2k
1.
Together with the constant function 1, the system of functions φkl ptq is called the
Schauder system, see the picture. It forms a Schauder basis on C r0, 1s. (Check!)

Figure 3.2. The first few functions of the Schauder system

Exercise 3.3.16. [Verifying Schauder bases] Do all checks in the previ-

ous examples. For C r0, 1s, note that the linear span of Schauder system
consists of piecewise-linear functions on r0, 1s whose nodes are dyadic
points.
A major property of Schauder bases is the uniform bound on the partial sums:
Theorem 3.3.17 (Partial sums of a Schauder basis). Let pxk q be a Schauder
basis of a Banach space X. Then there exists a number M called the basis constant
of pxk q with the following property. The partial sums of the basis expansion (3.10)
of every x P X satisfy
¸n 



ak xk  ¤ M }x}, n  1, 2, . . .

k 1

Proof. Consider the sequence space

! 8̧ )
(3.12) E : a  pak q8
k1 : ak xk converges in X
k 1 
with the norm
¸
n 
(3.13) }a}E : sup  
ak xk .
P
n N 
k 1
 3.3. PRINCIPLE OF UNIFORM BOUNDEDNESS 82

It is an exercise to check that E is a Banach space. We will show that X and E

are isomorphic.4
To this end, we consider the synthesis operator T : E Ñ X defined as
8̧
Ta  ak xk .

k 1
By construction, T is a bounded linear operator:
}T a} ¤ }a}E .
Since pxk q is a Schauder basis, T is surjective and injective. By the inverse mapping
theorem, T is an isomorphism. Therefore one can find a number M such that
}a}E ¤ M }T a}, a P E.
But this means that
¸n 
ak xk  ¤ M }x}, x P X,
 
sup 
P
n N 
k 1
which we wanted to prove. 
Exercise 3.3.18. [Space of coefficients] Let pxk q8
k1 be a sequence of
nonzero vectors in a Banach space X. Define the space of coefficients E
by (3.12) and (3.13). Prove that E is a Banach space. (This property was
used in the proof of Theorem 3.3.17.)
Consider the partial sums of a basis expansion (3.10):
¸
n
Sn pxq : ak xk .
k 1 
By Theorem 3.3.17, we see that Sn is a projection5 in X onto Spanpx1 , . . . , xn q,
and that Sn are uniformly bounded (by the basis constant):
sup }Sn }  M   8.
P
n N
Also, the coefficients ak  ak pxq of the basis expansion (3.10) are obviously
linear functionals on X. They are called biorthogonal functionals of the basis pxk q
and denoted xk , i.e.
xk pxq  ak .
With this notation, the basis expansion of x P X looks as
8̧
x xk pxqxk

k 1
This resembles the Fourier series with respect to orthogonal bases in a Hilbert
space, except now we discuss this in general Banach spaces.
Corollary 3.3.19 (Biorthogonal functionals). The biorthogonal functionals
pxk q of a Schauder basis pxk q are uniformly bounded:
sup }xk } }xk }   8.
P
k N

4For an orthonormal basis x p q

k in a Hilbert space X, this statement means that X is iso-

morphic to E `2 . We have proved this in Section 1.6.7.
5To recall the notion of projection in a Banach space, see Definition 2.4.21.
 3.4. COMPACT SETS IN BANACH SPACES 83

Proof. We estimate the k-th term of basis expansion as

}xk pxqxk }  }Sk pxq
Sk
1 pxq} ¤ }Sk pxq} }Sk
1 pxq} ¤ 2M }x}
where M is the basis constant. On the other hand, }xk pxqxk }  |xk pxq| }xk }. This
clearly completes the proof. 
Example 3.3.20. The biorthogonal functionals for an orthonormal basis in a
Hilbert space coincide with the basis vectors, i.e. xk  xk . The biorthogonal
functionals for the canonical (coordinate) basis of `p , 1 ¤ p   8 are the coordinate
vectors in `q , where p and q are adjoint indices.
Remark 3.3.21 (Basis problem). All classical separable Banach spaces are
known to have Schauder bases. However, there exist separable Banach spaces that
have no Schauder bases. The first constructions of such spaces was proposed by
Enflo [3] as a negative solution to Banach’s basis problem.

3.3.5. Additional Exercises.

Exercise 3.3.22. [Hamel bases are uncountable] Show that a Hamel
basis of an infinite-dimensional Banach space X is always uncountable.
(This is the reason why Hamel bases are rarely used in practice). Hint:
first prove that every finite-dimensional subspace of X is nowhere dense;
then use Baire category theorem.

3.4. Compact sets in Banach spaces

Lec. 27: 11/10
Compactness is a useful substitute of finite dimensionality. We assume that the
reader is familiar with the notion of compactness from a basic course in topology,
but we will give a brief review.
3.4.1. Review of compactness. By definition, a subset A of a topological
space X”is compact if every open cover of A contains a finite sub-cover.
”n Precisely,
if A „ α Uα for some collection of open sets Uα , then A „ k1 Uαk for some
sub-collection. Some basic properties of compact sets are:
(i) compact subsets of a Hausdorff space are closed;
(ii) closed subsets of compact sets are compact;
(iii) images of compact sets under continuous maps are compact;
(iv) continuous functions on compact sets are uniformly continuous and they attain
their maxima and minima.
A set A is called precompact if its closure A is compact.
In metric spaces X, a useful description of compact sets A can be given in
terms of ε-nets. Recall that a subset N is an ε-net of A for some ε ¡ 0 if for every
x P A there exists y P N such that dpx, y q ¤ ε. Equivalently, N is an ε-net of A if
A can be covered by balls of radius ε centered at points in N .
Theorem 3.4.1 (Compactness in metric spaces). For a subset A of a metric
space X, the following are equivalent:
(i) A is precompact;
(ii) Every sequence pxn q in A has a Cauchy subsequence (which furthermore con-
verges in X if X is complete);
 3.4. COMPACT SETS IN BANACH SPACES 84

(iii) For every ε ¡ 0, there exists a finite ε-net of A.

As a consequence, precompact sets in metric spaces are always bounded. The
converse is true in all finite dimensional normed spaces:
Theorem 3.4.2 (Heine-Borel). A subset A of finite dimensional normed space
X is precompact if and only if A is bounded.
Actually, the classical Heine-Borel theorem is the statement for the specific
space X  `n2 . But as we know, all finite dimensional normed spaces X are isomor-
phic to `n2 (Theorem 3.1.13), so the general result is also true.

3.4.2. Compactness in infinite-dimensional normed spaces. In infinite-

dimensional normed spaces, Heine-Borel theorem fails. For example, an orthonor-
mal basis pxk q of `2 is a bounded set but it is not
? precompact, because it does not
have a convergent subsequence (as }xk
kj }  2 for k  j).
Compact sets are almost finite dimensional. This heuristics, which is made
rigorous in the following result, underlies many arguments in analysis:
Lemma 3.4.3 (Approximation by finite dimensional subspaces). A subset A of
a normed space X is precompact if and only if A is bounded and, for every ε ¡ 0,
there exists a finite dimensional subspace Y of X which forms an ε-net of A.
Proof. Necessity. Let A be precompact. Choose a finite ε-net Nε of A; then
the subspace Y : SpanpNε q is finite-dimensional and forms an ε-net of A.
Sufficiency. Since A is bounded, A „ rBX for some finite radius r. Since Y is
an ε-net of A, it follows that pr εqBY is also an ε-net of A. (Check!) Further, since
Y is finite-dimensional, the set pr εqBY is precompact by Heine-Borel theorem.
So we have found a precompact ε-net of A. Therefore A itself is precompact.
(Why?) 

By Heine-Borel theorem, the unit ball BX of a finite-dimensional normed space

X is compact. This never holds in infinite dimensions:
Theorem 3.4.4 (F. Riesz). The unit ball BX of an infinite-dimensional normed
space X is never compact.
Proof. Suppose BX is compact. By approximation Lemma 3.4.3, we can find
a finite dimensional subspace Y of X which forms an 12 -net of BX , i.e.

distpx, Y q ¤ for all x P BX .

1
(3.14)
2
Since X is infinite dimensional and Y is finite dimensional, the quotient space
X {Y is nonzero. So we can find a coset rxs P X {Y with }rxs}  0.9. Since by
definition }rxs}  inf xPrxs }x}, we can further choose a representative x P rxs such
that }x} ¤ 1. Summarizing, distpx, Y q  }rxs}  0.9 and x P BX . This contradicts
(3.14) and completes the proof. 

The next useful result states that pointwise convergence of operators implies
uniform convergence on compacta. We say that a sequence of operators Tn P
LpX, Y q between normed spaces X and Y converges pointwise to some T P LpX, Y q
if
Tn x Ñ T x for all x P X.
 3.4. COMPACT SETS IN BANACH SPACES 85

In contrast, we say that Tn converges uniformly to T on a subset A „ X if

}Tn x
T x} ¤ εn Ñ 0 for all x P A,
where εn ¥ 0 is some sequence of numbers (that does not depend on x).
Theorem 3.4.5 (Convergence on compacta). Let X, Y be Banach spaces, and
Tn , T P LpX, Y q. Assume that Tn Ñ T pointwise. Then Tn Ñ T uniformly on
every precompact subset A € X.
Proof. Since the sequence pTn q is pointwise convergent, it is pointwise bounded.
Therefore, by the principle of uniform boundedness, pTn q is uniformly bounded, i.e.
there exists a number M such that
}Tn } ¤ M for all n.
Let ε ¡ 0 be arbitrary, and choose a finite ε-net Nε of A. Since the set Nε is
finite, Tn Ñ T uniformly on Nε . (Why?) So there exists a number N such that
}Tn y
T y} ¤ ε for all n ¥ N and y P Nε .
For every x P A, we choose y P Nε with }x
y } ¤ ε. Then
}Tn x
T x}  }pTn
T qx} ¤ }pTn
T qy} }pTn
T qpx
yq}
¤ ε p}Tn } }T }q }x
y} ¤ ε 2M ε.
Summarizing, we have proved that for every ε ¡ 0 there exists N such that }Tn x

T x} ¤ p1 2M qε for all n ¥ N and x P A. This means that Tn Ñ T uniformly on
A. 
3.4.3. Compactness criteria in various spaces. There is a useful criterion
of compactness in spaces with Schauder basis (which covers all classical spaces).
Theorem 3.4.6 (Compactness in spaces with basis). Let X be a Banach space
with a Schauder basis pxk q. A subset A „ X is precompact if and only if A is
bounded and the basis expansions of vectors x P A converge uniformly, i.e.
 ¸ 
xk x xk 
pq  }x
Sn x} ¤ εn Ñ 0 for all x P A,
 

¡
k n

where εn ¥ 0 is some sequence of numbers (that does not depend on x).

Proof. Necessity. Since pxk q is a Schauder basis, this means that the projec-
tions Sn Ñ I pointwise (where I is the identity operator in X). By Theorem 3.4.5,
Sn Ñ I uniformly on A, as required.
Sufficiency. Assume that }x
Sn x} ¤ ε for all x P A. Since Sn x P ImpSn q, this
implies that ImpSn q forms an ε-net of A. But ImpSn q has finite dimension n. We
conclude that A is precompact by Lemma 3.4.3. 
Corollary 3.4.7 (Compactness in `p ). A subset A „ `p , p P r1, 8q is precom-
pact if and only if A is bounded and has uniformly decaying tails, i.e.
¸
|ak |p ¤ εn Ñ 0 for all a  pak q P A,
¡
k n

where εn ¥ 0 is some sequence of numbers (that does not depend on a).

Proof. The claim follows immediately by applying Theorem 3.4.6 for the
canonical basis of `p . 
 3.5. WEAK TOPOLOGY 86

Example 3.4.8. The Hilbert cube is the following subset of `2 :

A  tpak q P `2 : |ak | ¤ P Nu.

1
for all k
k
By Corollary 3.4.7, the Hilbert cube is compact.

Exercise 3.4.9. [Compactness in c0 ] Prove that a subset A „ c0 is

precompact if and only if there exists a vector b P c0 that majorizes all
vectors a P A, i.e.
|ak | ¤ bk for all k P N.
(Hint: use Theorem 3.4.6).

Finally, we mention without proof compactness criteria in two other spaces,

C ra, bs and L1 .

Theorem 3.4.10 (Arzela-Ascoli). A subset A „ C ra, bs is precompact if and

only if A is bounded and equicontinuous, i.e. for every ε ¡ 0 there exists δ ¡ 0
such that
|s
t| ¤ δ implies |f psq
f ptq| ¤ ε for all f P A.
Example 3.4.11. Arzela-Ascoli theorem implies that the set of differentiable
functions f with }f 1 }8 ¤ 1 is compact in C r0, 1s.

Theorem 3.4.12 (Compactness in L1 ). A subset A „ L1 r0, 1s is precompact if

and only if A is bounded and uniformly bounded on average, i.e. for every ε ¡ 0
there exists δ ¡ 0 such that
»1
|τ | ¤ δ implies |f pt τ q
f ptq| ¤ ε for all f P A.
0

3.4.4. Additional Exercises.

Exercise 3.4.13. [Stability of compactness] Prove that compactness in

normed spaces is stable under linear operations:
(i) If A, B are precompact sets in a normed space, then Minkowski
sum A B is precompact;
(ii) If A is a precompact subset of X and T P LpX, Y q then T pAq is a
precompact set in Y .

Exercise 3.4.14. [Convex hull of compact sets]

(i) Show that the convex hull of a precompact set in a normed space
is a precompact set.
(ii) Show that the closure of the convex hull of a compact set in a
normed space is compact.
(iii) Construct an example showing that the convex hull of a compact
set in a normed space does not need to be compact.
 3.5. WEAK TOPOLOGY 87

3.5. Weak topology

Every normed space X is a metric space, with the metric given by dpx, y q 
Lec. 28: 11/12

}x
y}, x, y P X. This topology on X is called strong topology . Thus a sequence

xn Ñ x converges (strongly) in X if
(3.15) }xn
x} Ñ 0.
In addition to the strong topology, X carries a different topology called weak
topology. We are going to study the weak topology in this section.

3.5.1. Weak convergence.

Definition 3.5.1 (Weak convergence). A sequence pxk q in a normed space X
converges weakly to a vector x P X if
f pxk q Ñ f pxq for every f P X .
The weak convergence is denoted xk ÝÑ
w
x.
Strong convergence (3.15) clearly implies weak convergence. The converse is
generally not true:
Example 3.5.2. An orthonormal system pxk q in a Hilbert space X converges
weakly to zero, and it clearly does not converge strongly. Indeed, by Riesz repre-
sentation theorem the weak convergence to zero is equivalent to
xxk , xy Ñ 0 for every x P X.
This indeed follows from Bessel’s inequality
8̧
|xxk , xy|2 ¤ }x}2 .

k 1

Even though weak convergence is generally strictly weaker than strong con-
vergence, there are several useful ties between weak and strong properties. Weak
convergence clearly implies weak boundedness, which in turn implies strong bound-
edness by a consequence to the principle of uniform boundedness (Corollary 3.3.5):
Proposition 3.5.3. Weakly convergent sequences in Banach spaces are bounded.

Moreover, we have a good control of the weak limit, given in the next two
results.
Proposition 3.5.4. If xn ÝÑ
w
x in a normed space then }x} ¤ lim inf n }xn }.
Proof. Let f P X  be a supporting functional of x, i.e. }f }  1, f pxq  }x}.
Then f pxn q ¤ }xn } for all n. Taking lim inf of both sides, we conclude that
lim inf }xn } ¥ lim inf f pxn q  f pxq  }x}
n n

as required. 

Lemma 3.5.5 (Mazur). Let xk ÝÑ

w
x in a normed space, then6 x P convpxk q.

p q
6Recall that conv A is the smallest convex set containing A, see Exercise 1.2.24.
 3.5. WEAK TOPOLOGY 88

Proof. Suppose x R K : convpxk q. Using a separation theorem (Corol-

lary refclosed convex separation), we can separate the closed convex set K from the
point txu. Namely, there exists a functional f P X  such that
sup f py q   f pxq.
y K P
Since xk P K, this implies that
sup f pxk q   f pxq,
k
which contradicts weak convergence. 
3.5.2. Criteria of weak convergence. Some known criteria of weak conver-
gence in classical normed spaces rely on the following tool.
Lemma 3.5.6 (Testing weak convergence on a dense set). Let X be a normed
space and A „ X  be a dense set. Then xk ÝÑ x in X if and only if pxk q is bounded
w

and
f pxk q Ñ f pxq for every f P A.
Proof. Nesessity follows by Proposition 3.5.3.
Sufficiency will be proved by a standard approximation argument. Consider
arbitrary g P X  and ε ¡ 0. By density, we can choose f P A such that }g
f } ¤ ε.
Then
lim sup |g pxk
xq| ¤ lim sup |f pxk
xq| lim sup |pg
f qpxk
xq|
k k n
¤ 0 }g
f } lim supp}xk } }x}q ¤ M ε
k
where M  supk }xk } }x}   8 by the boundedness assumption. Since ε ¡ 0
is arbitrary, we conclude that lim supk |g pxk
xq|  0. Hence g pxk q Ñ g pxq as
required. 
Theorem 3.5.7 (Weak convergence in c0 and `p ). Let X  c0 or X  `p ,
p P p1, 8q. Then xk Ý Ñ x in X if and only if the sequence pxk q is bounded and
w

converges to x pointwise, i.e.

xk piq Ñ xpiq for every i P N.
Proof. Necessity. If xn Ý Ñ x then by applying coordinate functionals ei P X 
w

(i.e. those acting as ei pxq  xpiq) we see that xk piq Ñ xpiq as required.
Sufficiency. We are given that pxk q is bounded and that f pxk q Ñ f pxq for
every coordinate functional f  ei . By linearity, we get f pxk q Ñ f pxq for every
f P Spanpei q8
i1 .
On the other hand, the representation theorems (Corollary 2.2.6 and Exer-
cise 2.2.7) state that X   `1 if X  c0 and X   `p1 if X  `p . The functionals
ei P X  get identified with the coordinate vectors p0, . . . , 0, 1, 0, . . .q, which shows
that Spanpei q8 
i1 is dense in X . (Why?)
The proof is finished by applying Lemma 3.5.6 to A  Spanpei q8 i1 . 
Exercise 3.5.8. Consider the sequence xk  p1, . . . , 1, 0, 0, . . .q (with k
ones) in `8 . Use Mazur’s lemma to show that xk does not converge
weakly. Deduce that the criterion of weak convergence in `p , p P p1, 8q,
given in Theorem 3.5.7 fails for `8 . (There is no useful criterion of weak
convergence in `8 .
 3.5. WEAK TOPOLOGY 89

Exercise 3.5.9. State and prove a similar criterion of weak conver-

gence in spaces with Schauder basis.
A similar criterion of weak convergence holds in spaces of continuous functions.
Theorem 3.5.10 (Weak convergence in C pK q). Let K be a compact topological
space. Then xk ÝÑ x in C pK q if and only if the sequence of functions xk ptq is
w

uniformly bounded7 and converges to xptq pointwise, i.e.

xk ptq Ñ xptq for every t P K.
Proof. Necessity. Boundedness of pxk q follows from weak convergence as
before. Pointwise convergence follows by applying point evaluation functionals
δt P C pK q (acting as δt pxq  xptq).
Sufficiency. We need to show that f pxk q Ñ f pxq for all f P C pK q . By
representation Theorem 2.2.8, this is equivalent to claiming that
» »
(3.16) xn dµ Ñ x dµ
K K

for every Borel regular signed measure µ. On the other hand, our assumptions are
that the sequence of functions xn ptq is uniformly bounded and it converges to xptq
pointwise. The Lebesgue dominated convergence theorem implies (3.16). 

A similar criterion of weak convergence holds in Lp spaces. However, it does

not make sense to consider the values of functions x P Lp in individual points.
Instead, we shall consider integrals of xptq over short intervals.

Theorem 3.5.11 (Weak convergence in Lp ). Let p P p1, 8q. A sequence xk ÝÑ

w

x in Lp r0, 1s if and only if the sequence pxk q is bounded in Lp and

»b »b
xk ptq dt Ñ xk ptq dt for every interval ra, bs „ r0, 1s.
a a

Proof. One notices that the set of characteristic functions 1ra,bs ptq for ra, bs „
r0, 1s spans the set of step functions, which is dense in pLp q  Lp1 .(Why?) The
argument is finished similarly to Theorem 3.5.7. 

Remark 3.5.12. The same criterion holds for Lp pRq. (Why?)

Example 3.5.13 (Sliding bumps). A good example of weakly convergent but
strongly divergent sequence of functions is formed by a sliding bump in Lp pRq,
p P p1, 8q. Consider a function x P Lp pRq with compact support. Then the
sequence xk ptq  xpt
k q converges weakly to zero by Theorem 3.5.11.
Remark 3.5.14 (Weak convergence in `1 ). The weak convergence in `1 is un-
usual. (Note that this space is not covered by Theorem 3.5.7.) The weak and strong
convergence in `1 are equivalent. This result is due to I. Schur, and is called Schur
property of `1 .
Exercise 3.5.15. [Weak convergence in finite dimensional spaces] Show
that all finite-dimensional normed spaces X have Schur property, so the
weak and strong convergence in X coincide.
 3.5. WEAK TOPOLOGY 90

Lec. 29: 11/15

3.5.3. Weak topology. Now we broaden the picture and study the weak
topology on X which defines weak convergence. This way, in addition to weak
convergence, we could be able to study other weak properties, such as weak bound-
edness, weak compactness and so on.
Definition 3.5.16 (Weak topology). The weak topology on a normed space X
is defined as the weakest topology in which all maps f P X  (i.e. f : X Ñ R) are
continuous.
Equivalently, the base of the weak topology is given by the cylinders, which are
the sets of the form
tx P X : |fk px
x0 q|   ε, k  1, . . . , N u
where x0 P X, fk P X  , ε ¡ 0, and n P N. So, these cylinders form a local base of
weak topology at point x0 .
To distinguish the two natural topologies on X, the norm topology is sometimes
called strong.
Exercise 3.5.17. Check the equivalence of the two ways to define the
weak topology in the definition above.

Remark 3.5.18. The cylinders are clearly open in the strong topology. Hence
the weak topology is weaker than the strong topology.
Remark 3.5.19. In infinite-dimensional spaces X, the cylinders are rather large
as they contain subspaces of finite codimension tx P X : fk px
x0 q  0, k 
1, . . . , N u. This shows that in infinite dimensions, weakly open sets are unbounded.
Exercise 3.5.20. Prove that in an infinite dimensional normed space
X, weak topology is strictly weaker than the strong topology. Why
does not this contradict Schur property of X  `1 mentioned in Re-
mark 3.5.14?
Nevertheless, some weak and strong properties are equivalent. For example,
weak boundedness and strong boundedness are equivalent. This follows from the
principle of uniform boundedness, see Corollary 3.3.5. (Check that what we called
“weak boundedness” is indeed boundedness in the weak topology).
Also, weak closedness and strong closedness are equivalent for convex sets:
Proposition 3.5.21 (Weak closedness). Let K be a convex set in a normed
space X. Then K is weakly closed if and only if K is (strongly) closed.
Proof. Necessity is trivial.
Sufficiency. Assume K is closed and convex. By Corollary intersection of half-
spaces to Hahn-Banach theorem, K is the intersection of the closed half-spaces that
contain K. Each closed half-space has the form
Af,a  tx P X : f pxq ¤ au
for some f P X  and a P R. Hence Af,a is weakly closed.8 The intersection K of
the closed half-spaces is therefore automatically weakly closed. 
7That is, sup x
k } }8   8
k .
8Indeed, A
f,a is the preimage of the closed subset p
8, as under the function f : X Ñ R,
which is continuous by the definition of weak topology.
 3.6. WEAK TOPOLOGY. BANACH-ALAUGLU’S THEOREM 91

Remark 3.5.22. Convexity assumption is critical in Proposition 3.5.21. Oth-

erwise the result would claim that the weak and strong topologies are equivalent,
which is false.

3.6. Weak topology. Banach-Alauglu’s theorem

On X  , there are two natural weaker topologies. The weak topology that
we already considered makes all functionals in X  continuous functions on X  .
The other topology, called weak topology, is only concerned with continuity of
functionals that come from X € X  .

3.6.1. Weak convergence.

Definition 3.6.1 (Weak convergence). Let X be a normed space. A sequence

of functionals pfk q in X  weak converges to a functional f P X  if

fk pxq Ñ f pxq for every x P X.


The weak convergence is denoted fk ÝwÝÑ f .
In other words, the weak convergence of functionals is pointwise convergence.
While weak convergence of functionals fk P X  is tested on all functionals from
X  , weak convergence of fk is tested on the subset X „ X  . Therefore, weak
convergence implies weak convergence in X  . Of course, for reflexive spaces, weak
and weak convergence coincide.

Example 3.6.2 (Weak convergence of measures). In probability theory, one

says that a sequence of regular Borel measures µn on R converges weakly to a Borel
regular measure µ if
» »
f dµn Ñ f dµ for every f P C pRq.
Assume that the measures µn and µ are compactly supported, say on an interval
ra, bs. By the representation theorem for pC ra, bsq , Theorem 2.2.8, this convergence
is nothing different from
w
µn ÝÝÑ µ in pC ra, bsq .
Summarizing, the weak convergence of measures in probability theory is actually
the weak convergence of measures acting as linear functionals on C ra, bs.

Example 3.6.3 (Dirac delta function). Recall that we understand Dirac delta
function δ ptq as the point evaluation functional at zero, see Example 2.1.3. Equiv-
alently, Dirac delta function may be identified with the probability measure on R
with the only atom at the orgigin. Therefore Dirac delta function is the weak limit
of uniform measures on r
n1 , n1 s as n Ñ 8.
This gives a natural way to approximate Dirac delta function δ ptq (which does
not exists as a function on R) by genuine functions δn ptq, which are the probability
distribution functions of the uniform measures on r
n1 , n1 s, see the picture.
 3.6. WEAK TOPOLOGY. BANACH-ALAUGLU’S THEOREM 92

Figure 3.3. Approximation of Dirac delta function δ ptq by func-

tions δn ptq

3.6.2. Weak topology. Banach-Alaoglu’s theorem. Similarly to the

Definition 3.5.16, we will consider the weak topology on X  .
Definition 3.6.4 (Weak topology). Let X be a normed space. The weak
topology on X  is defined as the weakest topology in which point evaluation maps
f ÞÑ f pxq from X  to R are continuous for all points x P X.
Equivalently, the base of the weak topology is given by the cylinders, which are
the sets of the form
tf P X  : |pf
f0 qpxk q|   ε, k  1, . . . , N u
where f0 P X  , xk P X, ε ¡ 0, and n P N. So, these cylinders form a local base of
weak topology at f0 .
Looking at the first part of this definition and recalling that x P X „ X  ,
we see that weak topology is weaker than the weak topology on X  . However, for
reflexive spaces the two topologies are of course equivalent.
The main result on weak topology is Alaoglu’s theorem. It allows one to bring
back to life compactness arguments in infinite-dimensional normed spaces X, even
though the unit ball of such X is always not compact.
Theorem 3.6.5 (Banach-Alaoglu). For every normed space X, the closed unit
ball BX  is weak compact.
The proof will be based on Tychonoff ’s theorem that states that the product of
any collection of compact topological spaces is compact. Let us briefly recall this
result.
Consider a collection of pXγ qγ PΓ of any number
± (countable or uncountable) of
topological spaces Xγ . The Cartesian product γ PΓ Xγ can be equipped with the
product topology whose base is formed by the sets of the form
" *
¹
Aγ : Aγ is open in Xγ ; all but finitely many of Aγ equal Xγ .
P
γ Γ
±
Tychonoff’s theorem states that if each Xγ is compact then P Xγ is compact
γ Γ
in the product topology.
Proof of Banach-Alaoglu’s theorem. We shall embed BX  into the prod-
uct space of intervals
¹ (
K : r
}x}, }x}s  f :X Ñ R : |f pxq| ¤ }x} for all x P X .
P
x X
 3.6. WEAK TOPOLOGY. BANACH-ALAUGLU’S THEOREM 93

equipped with the product topology.

This is the weakest topology in which the point evaluation maps f ÞÑ f pxq from
K to R are continuous for all x P X. (Check!) We identify a functional f P BX 
with the element of the product space pf pxqqxPX P K. With this identification,
the weak topology on BX  coincides with the product topology on K. (Why?)
Therefore, this identification is a homeomorphic embedding of BX  into K.
It remains to check that BX  is a weak closed subset of K; the proof will
then be finished by Tychonofff’s theorem. This is simple. Indeed, note that BX 
consists of the linear functions in K. So we can represent
£ (
BX   Bx,y,a,b , where Bx,y,a,b  f P K : f pax by q  af pxq bf py q .
P P
x,y X, a,b R

Each set Bx,y,a,b is the preimage of the weak closed set t0u under the map f ÞÑ
f pax by q
af pxq
bf py q which, as we know, is continuous in the product topology.9
Therefore all sets Bx,y,a,b are weak closed, and so is their intersection BX  . This
completes the proof. 
3.6.3. Universality of space C pK q. As an application of Banach-Alaoglu’s
Lec. 30: 11/17

theorem, we will show that the space of continuous functions C pK q is universal in

the sense that it contains every Banach space X as a subspace.10
Theorem 3.6.6 (Universality of C pK q). Every Banach space X can be isomet-
rically embedded into C pK q for some compact topological space K.
Proof. Let K : BX  equipped with weak topology. By Banach-Alaoglu’s
theorem, K is indeed compact. We define the embedding X Ñ C pK q by associating
every x P X the point evaluation function
(3.17) xpf q : f pxq, f P K.
Recall that the point evaluation function is indeed in C pK q by the definition of
weak topology. The map defined by (3.17) is linear by construction. Finally, this
map is an isometric embedding; indeed
}x}C pK q  f PKmax
B 
|f pxq|  }x}X ,
X

where the last inequality uses a consequence of Hahn-Banach theorem, Corol-

lary 2.3.10. 
Exercise 3.6.7. [Universality of `8 ] Show that `8 is a universal space
for all separable Banach spaces. In other words, show that every sepa-
rable Banach space X isometrically embeds into `8 .
Hint: Consider a dense subset pxk q8 k1 of SX , choose supporting func-
tionals fk P SX  of xk , and define the embedding X Ñ `8 by x ÞÑ pfk pxqq8k1 .

9Recall that the point evaluation maps are continuous in the product topology.
10A little disclaimer is that the compact topological space K may depend on X; otherwise
the result is false for spaces X of too large cardinality.
 CHAPTER 4

Compact operators. Elements of spectral theory

4.1. Compact operators

Compact operators form an important class of bounded linear operators. On
the one hand, they are “almost” finite rank operators (in the same way as compact
sets are “almost” finite dimensional). So compact operators do share some proper-
ties of finite rank operators, which facilitates their study. On the other hand, the
class of compact operators is wide enough to include integral and Hilbert-Schmidt
operators, which are important in many applications.
4.1.1. Definition. Integral operators. Throughout this section, X, Y will
denote normed spaces.
Definition 4.1.1 (Compact operators). A linear operator T : X Ñ Y is called
compact if it maps bounded sets in X to precompact sets in Y . The set of compact
operators is denoted K pX, Y q.
Exercise 4.1.2. Show that T is compact if and only if it maps BX to
a precompact set in Y .
Since precompact sets are bounded, compact operators are always bounded,
i.e. K pX, Y q „ LpX, Y q.
Example 4.1.3. Every finite rank operator1 T P LpX, Y q is compact. Indeed,
T pBX q is a bounded subset of a finite dimensional normed space Im T „ Y , so
T pBX q is precompact by Heine-Borel theorem.
The next example is one of the main motivation to study compact operators.
Proposition 4.1.4 (Integral operators are compact). Consider the integral
operator T : C r0, 1s Ñ C r0, 1s defined as
»1
pT f qptq  k pt, sqf psq ds
0

with kernel k pt, sq P C pr0, 1s2 q. Then T is a compact operator.

Proof. We need to show that K : T pBC r0,1s q is a precompact subset of
C r0, 1s, see Exercise 4.1.2. By Arzela-Ascoli Theorem 3.4.10, this would follow
from (uniform) boundedness and equicontinuoity of the set K.
The (uniform) boundedness of K follows from the boundedness of T . (Why?)
To prove equicontinuity, we let ε ¡ 0 and choose δ ¡ 0 such that
|t1
t2 | ¤ δ implies |kpt1 , sq
f pt2 , sq| ¤ ε for all s P r0, 1s.

1T is a finite rank operator if dim Im T   8.

94
 4.1. COMPACT OPERATORS 95

(We can do this by continuity of the kernel k pt, sq.) Now, for every f P BC r0,1s , we
obtain by triangle inequality that
»1
 
p qp q
p qp q ¤
 T f t1 T f t2  |kpt1 , sq
f pt2 , sq| |f psq| ds ¤ ε
0
as |f psq| ¤ 1 for all s. This shows that the set K is equicontinuous, and therefore
precompact. 
Exercise 4.1.5. Show that Volterra operator (2.12) is compact on
C r0, 1s, even though its kernel is discontinuous. See Exercise 4.1.19 for a
more general result.

4.1.2. Basic properties of compact operators.

Proposition 4.1.6 (Properties of K pX, Y q). (i) The set of compact opera-
tors K pX, Y q is a closed linear subspace of LpX, Y q.
(ii) K pX, Y q is an operator ideal. This means that if T P K pX, Y q then the
compositions ST and T S are both compact for every bounded linear operator
S.
Proof. (i) Linearity follows from the observation that the Minkowski sum of
two precompact sets is precompact (see exercise below).
Closedness. Consider a sequence Tn P K pX, Y q such that Tn Ñ T in LpX, Y q;
we want to prove that T P K pX, Y q. Let ε ¡ 0 and choose n P N such that
}Tn
T } ¤ ε. This means that
}Tn x
T x} ¤ ε for every x P BX .
This shows that Tn pBX q is a precompact ε-net of T pBX q. Since ε is arbitrary,
T pBX q is itself precompact. (Why?)
(ii) is straightforward and is left as an exercise. 
Exercise 4.1.7. Prove that Minkowski sum A B of two precompact
subsets A, B of a normed space is a precompact set.

Corollary 4.1.8 (Isomorphisms are not compact). Let X be an infinite di-

mensional normed space. Then the identity operator on X is not compact. More
generally, any isomorphism T : X Ñ Y is not compact.
Proof. For the identity operator on X, the result follows from F. Riesz’s
Theorem 3.4.4 on non-compactness of BX . As for the general statement, if an
isomorphism T : X Ñ Y were compact then the identity operator T
1 T would also
be compact by Proposition 4.1.6, which would be a contradiction. 
As we know, finite rank operators are compact (Example 4.1.3). More generally,
since K pX, Y q is closed, it follows that any operator that can be approximated by
finite rank operators is also compact:
Corollary 4.1.9 (Almost finite rank operators are compact). Suppose a linear
operator T : X Ñ Y can be approximated by finite rank operators Tn P LpX, Y q,
i.e.
}Tn
T } Ñ 0 as n Ñ 8.
Then T is compact. 
 4.1. COMPACT OPERATORS 96

Exercise 4.1.10. Fix a sequence of real numbers pλk q8

k1 , and define
the linear operator T : `2 Ñ `2 by
T x  pλk xk q8
k1 .

For what multiplier sequences pλk q is the operator T (a) well defined?
(b) bounded? (c) compact?

4.1.3. Hilbert-Schmidt operators. This is a most frequently used class of

compact operators in Hilbert spaces. As we will see, it covers the class of Hilbert-
Schmidt integral operators discussed in Section 2.4.4.
Definition 4.1.11 (Hilbert-Schmidt operators). Let H be a separable Hilbert
space, and let pxk q be an orthonormal basis of H. A linear operator T : H Ñ H is
called a Hilbert-Schmidt operator if
8̧
}T xk }2   8.

k 1

The quantity
 8̧
1{2
}T }HS : }T xk } 2


k 1
is called the Hilbert-Schmidt norm of T .
Exercise 4.1.12. For operators in finite dimensional spaces T : Cm Ñ
C , (equivalently, for m  n matrices), we already gave a definition of
n

Hilbert-Schmidt norm in (1.12). Verify that the two definitions do agree.

Proposition 4.1.13. The definition of Hilbert-Schmidt operator and of the

Hilbert-Schmidt norm does not depend on the choice of an orthonormal basis of H.
°
Proof. Assume that k }T xk }2   8 for some orthonormal basis pxk q of H.
Using Parseval’s identity twice, we obtain
¸ ¸ ¸ ¸
(4.1) }T xk }2  |xT xk , xj y|2  |xxk , T  xj y|2  }T  xj }2 .
k k,j k,j k

Let px1k q be another orthonormal basis of H. Then a similar argument gives

¸ ¸ ¸ ¸
}T  xj }2  |xx1k , T  xj y|2  |xT x1k , xj y|2  }T x1k }2 .
j j,k j,k k

This completes the proof. 

Remark 4.1.14. As a byproduct of the proof, we obtained in (4.1) that

}T  }HS  }T }HS .
Lec. 31: 11/19
As we know from Proposition 2.4.7, linear operators on finite-dimensional
spaces satisfy }T } ¤ }T }HS . The same holds in general:
Proposition 4.1.15. Hilbert-Schmidt operators are bounded, and }T } ¤ }T }HS .
 4.1. COMPACT OPERATORS 97

Proof. Let T be a Hilbert-Schmidt operator on a Hilbert space H, and pxk q

be an orthonormal basis of H. It suffices to prove that the restriction of T on the
dense subspace Spanpxk q of H is bounded and has norm at most }T }HS ; the result
would then follow by extension by continuity (Proposition
° 2.4.20).
So let x P Spanpxk q, which means that x  k ak xk for some scalars ak (finite
sum). Then using triangle inequality and Cauchy-Schwarz inequality, we obtain
¸  ¸
¸ 1{2
¸ 1{2
}T x}   ak T xk  ¤

|ak | }T xk } ¤ |ak |2 }T xk }2  }x} }T }HS .
k k k k
This completes the proof. 
An important example of Hilbert-Schmidt operators are integral operators on
L2 , which we introduced in Section 2.4.4.
Proposition 4.1.16 (Hilbert-Schmidt integral operators). Consider the inte-
gral operator T : L2 r0, 1s Ñ L2 r0, 1s defined as
»1
pT f qptq  kpt, sqf psq ds
0
with kernel k pt, sq P L2 pr0, 1s2 q. Then T is a Hilbert-Schmidt operator, and
}T }HS  }k}2 .
Proof. We will view the integral in the definition of T as the inner product
of f with the kernel k. Specifically, consider the function Kt psq  k pt, sq; then
pT f qptq  xKt , f y for every t P r0, 1s.
Let us fix some orthonormal basis pxk q of L2 r0, 1s. Then
¸ ¸» 1 ¸» 1
}T } 
2
HS } T xk 22 }  |pT xk qptq| 2
dt  |xKt , xk y|2 dt
k k 0 k 0
»1 ¸
 |xKt , xk y|2 dt (by monotone convergence theorem)
0 k
»1
 }Kt }22 dt (by Parseval’s identity)
0
 }k}22 (by definition of Kt and Fubini theorem).
This completes the proof. 
4.1.4. Compactness of the adjoint operator. Recall the basic duality
property for bounded linear operators: if T P LpX, Y q then T  P LpY  , X  q and
}T  }  }T }. A similar duality principle holds for compact operators:
Theorem 4.1.17 (Schauder). Let X and Y be Banach spaces. If T P K pX, Y q
then T  P K pY  , X  q.
Proof. Given f P Y  , we are seeking a bound on
(4.2) }T  f }X   sup |pT  f qpxq|  sup |f pT xq|  sup |f pyq|
P
x BX P
x BX P
y K

where K : T pBX q. (Taking the closure here is justified by continuity of f ).

We shall interpret the identity (4.2) in topological terms. Indeed, we know that
K is compact, and we need to prove that G : T  pBY  q is precompact in X  . Let
 4.2. FREDHOLM THEORY 98

us embed the subset G „ X  into C pK q and use Arzela-Ascoli theorem. Namely,

we define the embedding U : G Ñ C pK q by
U pT  f q : f |K
(making some selection of f ; it does not matter in which way). Then identity (4.2)
implies that
}T  f }X   }f |K }C pK q for every f P Y  ,
which shows that U is an isometric (thus homeomorphic) embedding.
Now, U pGq is (uniformly) bounded in C pK q as
}T  f }X  ¤ }T  } }f }Y  ¤ }T } for every f P BY  .
Moreover, U pGq is equicontinuous. Indeed, for every f P BY  and for y1 , y2 P K
we have
 
f |K py1 q
f |K py2 q  |f py1
y2 q| ¤ }f }X  }y1
y2 } ¤ }y1
y2 }.
Arzela-Ascoli theorem completes the proof. 

Remark 4.1.18 (For future). Consider proving the reverse direction in Schauder’s
theorem. Also consider proving that compact operators map weak Cauchy se-
quences to strongly convergent.

4.1.5. Additional Exercises.

Exercise 4.1.19. [Compactness of integral operators] Consider an in-
tergal operator T with kernel k pt, sq : r0, 1s2 Ñ R which satisfies the fol-
lowing:
(i) for each s P r0, 1s, the function ks ptq  k pt, sq is integrable in t;
(ii) the map s ÞÑ ks is a continuous map from r0, 1s to L1 r0, 1s.
Show that the integral operator T is compact in C r0, 1s.

4.2. Fredholm theory

Lec. 32: 11/22
Fredholm theory studies operators of the form “identity plus compact”. They
are conveniently put in the form I
T where I is the identity operator on some
Banach space X and T P K pX, X q.
Fredholm theory is motivated by two applications. One is for solving linear
equations λx
T x  b, and in particular integral equations (T being an integral
operator). Another related application is in spectral theory, where the spectrum of
T consists of numbers λ for which the operator λI
T is invertible. We will discuss
both applications in detail later.

4.2.1. Closed image.

Theorem 4.2.1. Let X be a Banach space and T P K pX, X q. Then operator
I
T has closed image.
 4.2. FREDHOLM THEORY 99

Proof. The argument relies on the characterization of injective operators with

closed image, Proposition 3.1.12. So we consider the operator A  I
T and its
injectivization à : X { ker A Ñ X. Since Im A  Im Ã, it suffices to show that à is
bounded below.
Suppose the contrary, that there exists cosets rxk s with }rxk s}  1 and such
that
Ãrxk s Ñ 0.
We can find representatives xk P rxk s with, say, }xk } ¤ 2 and such that
distpxk , ker Aq  }rxk s}  1 but Axk  Ãrxk s Ñ 0.
So we have Axk  xk
T xk Ñ 0. By compactness of T , we can assume (passing
to a subsequence if necessary) that T xk Ñ z for some z P X. It follows that
xk Ñ z. Therefore Axk Ñ Az; but we already know that Axk Ñ 0, thus Az  0.
So z P ker A. Furthermore, since xk Ñ z it follows that distpxk , ker Aq Ñ 0. This
contradiction completes the proof. 
4.2.2. Fredholm alternative. We now state and prove a partial case of the
so-called Fredholm alternative. The more general theorem is given in Exercise 4.2.4.
Theorem 4.2.2 (Fredholm alternative). Let X be a Banach space and T P
K pX, X q. Then operator A  I
T is injective if and only if A is surjective.
Proof. Necessity. Assume that A is injective but not surjective. Consider the
subspaces of X
Yn : ImpAn q, n  0, 1, . . .
Then
Y0  Y1  Y2    
is a chain of proper inclusions. Indeed, the first inclusion X  ImpAq is proper by
assumption; the claim follows by induction. (Check this!)
Furthermore, Yn are closed subspaces of X. Indeed, by Newton’s binomial
expansion we see that An  pI
T qn has the form A  I
T1 for some compact
operator T1 , so the claim follows from Theorem 4.2.1.
By Hahn-Banach theorem (see Exercise 2.3.33) we can find functionals
fn P Yn such that }fn }  1, fn P YnK 1 .
We can extend fn , again by Hahn-Banach theorem, so that fn P X  . We are
going to show that the sequence pT  fn q has no convergent subsequences. This
will contradict the compactness of T  and, by Schauder’s Theorem 4.1.17, the
compactness of T .
To this end, let us fix n ¡ m and compute the pairwise distances
dn,m : }T  fn
T  fm }  }T  pfn
fm q}  }pI
T q pfn
fm q fm
fn }.
So
dn,m ¥ sup |xT  fn
T  fm , xy|  sup |xfn
fm , pI
T qxy xfm
fn , xy|.
P
x BYn P
x BYn

Now, pI
T qx  Ax P Yn 1 while fn
fm P YnK 1 by construction, so xfn
fm , pI

T qxy  0. Further, x P Yn while fm P YnK , so xfm , xy  0. Therefore
dn,m ¥ sup |xfn , xy|  1
P
x BYn
 4.3. SPECTRUM OF A BOUNDED LINEAR OPERATOR 100

by construction. It follows that the terms of the sequence pT  fn q are pairwise

separated, so there can not be any convergent subsequence. This completes the
proof of the necessity direction.
Sufficiency will follow from a duality argument. We use the relations

(4.3) pIm AqK  ker A , pker AqK … Im A

which we proved in Proposition 2.4.32 and Exercise 2.4.34. So, assume that A 
I
T is surjective. Then A  I
T  is injective by (4.3). Since T  is compact
by Schauder’s theorem, the first part of the proof gives that A is surjective. This
implies that A is injective by (4.3). The proof is complete. 

Remark 4.2.3 (Compactness is essential). Fredholm alternative does not hold

for non-compact operators in general. For example, the right shift operator in `2
is injective but not surjective; the left shift operator in `2 is surjective but not
injective.

The name “Fredholm alternative” is explained by the following application to

solving linear equations of the form

λx
T x  b

where T P K pX, X q, λ P C, b P X. One is interested in existence and uniqueness of

solution. Theorem 4.2.2 states that exactly one the following statements holds for
every λ  0:
either the homogeneous equation λx
T x  0 has a nontrivial
solution,
or the inhomogeneous equation λx
T x  b has a solution
for every b; this solution is automatically unique.
This alternative is particularly useful
³1
for studying integral equations, since
for the integral operator pT f qptq  0 k pt, sqf psq ds, the homogeneous Fredholm
equation is
»1
λf ptq
k pt, sqf psq ds  0,
0

while the inhomogeneous Fredholm equation (“of second kind”) is

»1
λf ptq
k pt, sqf psq ds  bptq.
0

4.2.3. Additional exercises.

Exercise 4.2.4. [General Fredholm alternative] Let X be a Banach

space and T P K pX, X q. Show that operator A  I
T satisfies

dim ker A  dim ker A  codim Im A  codim Im A .

 4.3. SPECTRUM OF A BOUNDED LINEAR OPERATOR 101

4.3. Spectrum of a bounded linear operator

Lec. 33: 11/24
Studying linear operators through their spectral properties is a powerful ap-
proach in analysis and mathematical physics. Recall from linear algebra that the
spectrum of a linear operator T acting on Cn consists of the eigenvalues of T ,
which are the numbers λ P C such that T x  λx for some nonzero vector x P Cn ;
such x are called the eigenvectors of T . Eigenvalues always exist by the funda-
mental theorem of algebra, as they are the roots of the characteristic polynomial
detpT
λI q  0. There are at most n eigenvalues of T , or one can say exactly
n counting multiplicities. Eigenvectors corresponding to different eigenvalues are
linearly independent.2

4.3.1. Examples and definition of spectrum. In infinite-dimensional normed

spaces, the spectrum is a richer concept than in finite-dimensional spaces. Let us
illustrate the difference on two examples.

Example 4.3.1 (Uncoutnable number of eigenvalues). Consider the differential

operator

T 
d
dt
acting, for example, on C 1 pCq. To compute the spectrum of T , we solve the ordinary
differential equation u1  λu. The solution has the form

uptq  Ceλt .

Therefore, every λ P C is an eigenvalue of T .

Example 4.3.2 (No eigenvalues). Consider a multiplication operator on L2 r0, 1s

acting as
pT f qptq  tf ptq.
Suppose λ is an eigenvalue of T with eigenvector f P L2 r0, 1s. This means that the
following identity holds

tf ptq  λf ptq for all t P r0, 1s.

It follows that f  0. Therefore, T has no eigenvalues.

Definition 4.3.3 (Spectrum). Let X be a normed space and T P LpX, X q. A

number λ P C is called a regular point if T
λI is invertible as a bounded linear
operator,3 i.e. pT
λI q
1 P LpX, X q. All other λ are called spectrum points. The
set of all regular points is denoted ρpT q and is called the resolvent set of T . The
set of all spectrum points is denoted σ pT q and is called the spectrum of T .

2Recall however that the eigenvalues do not need to form a basis of Cn . The dimension of

 less
the span of eigenvectors corresponding to a given eigenvalue (the eigenspace) may be strictly


than the multiplicity of that root. This happens, for example, for the Jordan block T
λ 1
0 λ
.

An orthonormal basis of eigenvectors exists if and only if T is normal, i.e. T T 
TT .
3In the future, we will often say “invertible” instead of ‘invertible as a bounded linear
operator”.
 4.3. SPECTRUM OF A BOUNDED LINEAR OPERATOR 102

4.3.2. Classification of spectrum. For operators T acting on a finite di-

mensional space, the spectrum consists of eigenvalues of T . In infinite dimensions,
this is not true, as there are various reasons why T
λI may be non-invertible.
These reasons are listed in the following definition:
Definition 4.3.4 (Classification of spectrum). Let X be a normed space and
T P LpX, X q.
(i) The point spectrum σp pT q is the set of all eigenvalues of T , i.e. the numbers
λ P C satisfying
kerpT
λI q  0.
(ii) The continuous spectrum σc pT q is the set of all λ P C such that
kerpT
λI q  0 and ImpT
λI q is dense in X.
(iii) The residual spectrum σr pT q is the set of all λ P C such that
kerpT
λI q  0 and ImpT
λI q is not dense in X.
So the spectrum of T can be expressed as a disjoint union
σ pT q  σp pT q Y σc pT q Y σr pT q.
Let us now compute and classify the spectrum of come basic linear operators.
Example 4.3.5 (Diagonal operator on `2 ). Let us fix some sequence λk Ñ0
in Czt0u, and consider the operator T on `2 defined as
T ppxk q8 8
k1 q  pλk xk qk1 .
8
As pT
λI qx  ppλk
λqxk q8 k1 , we have pT
λI q

1 y  yk
λk
λ k1 . It follows
that pT
λI q
1
is a bounded operator is and only if λ is not in the closure of
tλk u8k1 , which is tλk u8k1 Y t0u.
All λk are clearly the eigenvalues of T as T ek  λk ek for the canonical basis
pek q of `2 . 0 is not an eigenvalue since T is injective (as all λk  0). So 0 is either
in continuous or residual spectrum. Now, Im T is dense in `2 (why?), so 0 is in the
continuous spectrum. Our conclusion is:
σp pT q  tλk u8
k 1 , σc pT q  t0u, σr pT q  H.
Example 4.3.6 (Multiplication operator on L2 ). Let us consider the same
multiplication operator as in Example 4.3.2, i.e.
pT f qptq  tf ptq.
As pT
λI qf ptq  pt
λqf ptq, we have

(4.4) pT
λI q
1 yptq  t
1 λ yptq.

If λ R r0, 1s then the function t
1λ is bounded, thus pT
λI q
1 is a bounded operator.
Therefore such λ are regular points. Conversely, if λ P r0, 1s then t
1λ R L2 r0, 1s
because of the non-integrable singularity at 0. Hence T
λI is not invertible (at
y ptq  1). Hence all such λ are regular points. Therefore, σ pT q  r0, 1s.
As we noticed in Example 4.3.2, T has no eigenvalues. It follows from (4.4)
that ImpT
λI q is dense in L2 r0, 1s. (Check!) Our conclusion is:
σp pT q  H, σc pT q  r0, 1s, σr pT q  H.
 4.4. PROPERTIES OF SPECTRUM. SPECTRUM OF COMPACT OPERATORS. 103

Remark 4.3.7. If Dirac delta function δ ptq was a genuine function in L2 , then
its translates δλ ptq : δ pt
λq would be the eigenvectors of the multiplication
operator on L2 :
T δλ  λδλ
and δλ would be the eigenfunctions of T . The situation would be similar to the
discrete multiplication operator from Example 4.3.5.
Example 4.3.8 (Shift operator). Consider the right and left shift operators on
`2 , acting on a vector x  px1 , x2 , . . .q as
Rpxq  p0, x1 , x2 , . . .q, Lpxq  px2 , x3 , . . .q.
Since R is clearly injective but Im R is not dense in `2 (why?), 0 is in the residual
spectrum of R. One can show that
σp pRq  H, σc pRq  tλ P C : |λ|  1u, σr pRq  tλ P C : |λ|   1u;
σp pLq  tλ P C : |λ|   1u, σc pRq  tλ P C : |λ|  1u, σr pRq  H.
Exercise 4.3.9. Prove the claims about the spectra of shift operators
made in Example 4.3.8.

4.4. Properties of spectrum. Spectrum of compact operators.

P LpX, X q.
Lec. 34: 11/29
Throughout this section, X denotes a Banach space and T
4.4.1. Resolvent operator. Spectrum is bounded. Studying the spec-
trum of T is convenient via the so-called resolvent operator:
Definition 4.4.1 (Resolvent operator). To each regular point λ P ρpT q we
associate the operator
Rpλq  pT
λI q
1 .
Rpλq is called the resolvent operator of T . So the resolvent is a function R : ρpT q Ñ
LpX, X q.
The resolvent operator can be computed in terms of series expansion involving
T . This technique is based on the following simple lemma:
Lemma 4.4.2 (Von Neumann). Consider an operator S P LpX, X q such that
}S }   1. Then I
S is invertible, and it can be expressed as a convergent series in
LpX, X q:
8̧
pI
S q
1  S k , }pI
S q
1 } ¤ 1
1}S } .

k 0
°8
 S converges absolutely because }S } ¤ }S }k while
k k
Proof. The series
}S }   1. Furthermore,
k 0

8̧ 8̧
pI
S q Sk  S k pI
Sq  I

k 0 
k 0
as telescoping series. (Check!) Finally,
8̧
}pI
S q
1 } ¤ }S }k ¤ 1
1}S } .

k 0
This completes the proof. 
 4.4. PROPERTIES OF SPECTRUM. SPECTRUM OF COMPACT OPERATORS. 104

Proposition 4.4.3. The spectrum σ pT q is a bounded set. Specifically,

σ pT q „ tλ P C : |λ| ¤ }T }u.
Proof. The claim is that |λ| ¡ }T } implies λ P ρpT q, i.e. the operator T
λI
is invertible. We express T
λI  λp λ1 T
I q. Since } λ1 T }   1, von Neumann’s
Lemma 4.4.2 implies that the operator in question is indeed invertible. 
Furthermore, writing the series expansion of the inverse of T
λI 
λpI
λ1 T q
according to von Neumann’s lemma, we immediately obtain:
Proposition 4.4.4 (Bound on the resolvent). If |λ| ¡ }T } then
8̧
Rpλq 
λ
k
1 T k , }Rpλq} ¤ |λ|
1 }T } . 

k 1

4.4.2. Resolvent identity. Spectrum is closed.

Proposition 4.4.5 (Resolvent identity). For all λ, µ P ρpT q we have
Rpλq
Rpµq  pλ
µqRpλqRpµq.
Proof. In the scalar case, the following identity clearly holds:
λ
µ
1
x
λ x
µ

1
 px
λqpx
µq
for all x, y P C, λ, µ  x. It is an exercise to generalize it to the operator-valued
case, which yields the resolvent identity. (Do this!) 
Solving the resolvent identity for Rpµq we obtain for λ, µ P ρpT q that

pµ
λqRpλq
1 Rpλq
 
(4.5) Rpµq  I
Corollary 4.4.6. The regular set ρpT q is an open set. Equivalently, the spec-
trum σ pT q is a closed set. 
Proof. Let λ P ρpT q. Von Neumann’s lemma implies that the operator I

pµ
λqRpλq is invertible for all µ in a suitably small neighborhood of λ. So for such
µ, the right hand side of (4.5) defines a bounded linear opearator. One can check
that in this case identity (4.5) holds (do this!) and therefore µ P ρpT q. 
4.4.3. Resolvent is an analytic function. Spectrum is nonempty. The
proof of Corollary 4.4.6 gives us a bit more information about the resolvent than we
have noticed. Let us go fo back to identity (4.5) and write the series expansion of
the inverse of pλ
µqRpλq
I according to von Neumann’s lemma. We immediately
obtain:
Corollary 4.4.7 (Resolvent expansion). The resolvent Rpλq is an analytic
operator-valued function on its domain ρpT q. Specifically, Rpµq can be expressed as
a convergent power series in a small neighborhood of any point λ P ρpT q:
8̧
(4.6) Rpµq  pµ
λqk
1 Rpλqk . 

k 1

Remark 4.4.8. It follows that for every functional f P LpX, X q , the function
f pRpµqq is a usual (i.e. complex-valued) analytic function on ρpT q.
Theorem 4.4.9. The spectrum σ pT q is a nonempty set.
 4.4. PROPERTIES OF SPECTRUM. SPECTRUM OF COMPACT OPERATORS. 105

Proof. We shall deduce this result from Liouville’s theorem in complex anal-
ysis.4 To this end, assume that σ pT q  H, hence ρpT q  C and the resolvent Rpλq
is an entire function (i.e. analytic on the whole complex plane).
Claim. Rpλq is also bounded function on C with Rpλq Ñ 0 as λ Ñ 8.
Indeed, by Proposition 4.4.4, Rpλq is a bounded in the annulus |λ| ¥ 2}T } and
vanishes at infinity. Since Rpλq is a continuous function by Corollary 4.4.7, Rpλq is
also bounded in the disc |λ| ¤ 2}T }.
Claim. By Liouville’s theorem, Rpλq  0 everywhere.
Indeed, we fix a functional f P LpX, X q and apply the usual Liouville’s theo-
rem for the bounded entire function f pRpλqq. It follows that f pRpλqq is constant,
and since it must vanish at infinity it is zero everywhere. The claim follows.
The last claim contradicts the fact that Rpλq is an invertible operator. 
Summarizing our findings, we can state that the spectrum of every bounded
linear operator is a nonempty compact subset of C.
4.4.4. Spectral radius. The spectrum of any operator T P LpX, X q is a
Lec. 35: 12/1

bounded set by Proposition 4.4.3, and moreover we have a quantitative bound

|λ| ¤ }T } for all λ P σpT q. This bound is not always sharp, and we will try to come
up with a sharp bound.
Definition 4.4.10. The spectral radius of an operator T P LpX, X q is defined
as (
rpT q  max |λ| : λ P σ pT q .
Theorem 4.4.11 (Gelfand’s formula). For every operator T P LpX, X q acting
on a Banach space X, one has
rpT q  lim }T n }1{n
 inf }T n }1{n .
n n

Exercise 4.4.12. Clearly rpT q ¤ }T }n1{n ¤ }T }, so Gelfand’s formula is

an improvement upon Proposition 4.4.3. Give an example where rpT q  
}T }.
Remark 4.4.13. Gelfand’s formula gives an asymptotics for the growth of
operator powers }T n }  rpT qn .
Proof. Upper bound. Let λ P σ pT q. One can easily show that λn P σ pT n q,
see Exercise 4.4.27.5 Therefore, using Proposition 4.4.3, we have |λn | ¤ }T n }, so
|λ| ¤ }T n }1{n . We conclude that
rpT q ¤ inf }T n }1{n .
n
Lower bound will be obtained as an application of the theory of Laurent series
in complex analysis.6 Consider the function f pRpλqq where Rpλq is the resolvent of
T and f P LpX, X q is an arbitrary functional. Recall two facts:
4Liouville’s theorem states that an entire (i.e. analytic on C) and bounded function f : C
C Ñ
is constant everywhere.
5This is a partial case of the spectral mapping theorem we will study later.
6Specifically, we shall use the following theorem of compex analysis. Consider a Laurent
series
8̧
p q
f z p
z0 q k .
ak z
k 
8
 4.4. PROPERTIES OF SPECTRUM. SPECTRUM OF COMPACT OPERATORS. 106

(i) f pRpλqq is an analytic function on the annulus |λ| ¡ rpT q by Corollary 4.4.4;
(ii) f pRpλqq is represented by a convergent Laurent series
8̧
(4.7) f pRpλqq 
λ
k
1 f pT k q

k 1
in the smaller annulus |λ| ¡ }T }.
By the theory of convergence of Laurent series (outlined in the footnote), the
series (4.7) converges in the larger annulus |λ| ¡ rpT q. So the terms of the series
are bounded for such λ:  
sup λ
n
1 f pT n q   8.
n
This means that the sequence pλ
n
1 T n ) is weakly bounded in LpX, X q. By a
consequence of the principle of uniform boundedness (Corollary 3.3.5), this sequence
is (strongly) bounded, i.e.
 
sup λ
n
1 T n  : K   8.
n

Taking n-th root and rearranging the terms, we obtain }T n }1{n ¤ K 1{n λ1 1{n for
all n. It follows that lim supn }T n }1{n ¤ |λ|. Since this happens for all λ such that
|λ| ¡ rpT q, we have proved that
lim sup }T n }1{n ¤ rpT q.
n
So, putting this together with the upper bound, we have proved that
rpT q ¤ inf }T n }1{n ¤ limninf }T n }1{n ¤ lim sup }T n }1{n ¤ rpT q.
n n
This completes the proof. 
4.4.5. Spectrum of compact operators. As compact operators are proxies
of finite rank operators, one is able to fully classify their spectrum. First of all, for
every T P K pX, X q one has
0 P σ pT q
since T is not invertible by Corollary 4.1.8.
Exercise 4.4.14. Construct three examples of compact operators for
which 0 is in the point, continuous, and residual spectrum respectively.

Theorem 4.4.15 (Point spectrum of compact operators). Let T P K pX, X q be

a compact operator on a normed space X. For every ε ¡ 0 there exists a finite
number of linearly independent eigennvectors corresponding to eigenvalues λk with
|λk | ¡ ε.
Consequently, the point spectrum σp pT q is at most countable, and it lies in a
sequence that converges to zero.
It also follows that each eigenvalue λk of T has finite multiplicity, i.e. dim kerpT

λk I q   8.

There exist unique radii r, R P R X t8u such that the Laurent series converges in the annulus
A  t z P C : r   | z
z0 |   R u
and diverges outside the closure of A. Moreover, there exists at least one point on the inner
t P
boundary z C : z z0 |
| u r of A and at least one point on the outer boundary z C : t P
|
| u
z z0 pq
R of A such that f z can not be analytically continues to those points.
 4.4. PROPERTIES OF SPECTRUM. SPECTRUM OF COMPACT OPERATORS. 107

Proof. Clearly, the second and third claims of the theorem follow from the
first one (why?). So, assume the contrary, that there exist ε ¡ 0 and an infinite
sequence of linearly independent vectors pxk q8
k1 such that

 λk xk , where |λk | ¡ ε.
T xk
Consider the subspaces En  Spanpxk qnk1 ; then E1 € E2 €    is a sequence of
proper inclusions. Therefore we can choose vectors

P En , }yn }  1, distpyn , En
1 q ¥ 12 .

yn

(Why? Think about En {En
1 .)

Figure 4.1. Construction of subspaces En and vectors yn

We will show that the sequence pT yn q8

n1 contains no Cauchy subsequences,
which will contradict compactness of T . To this end, we express yn as a linear
combination
¸
n
pnq
yn  a k xk  apnnq xn un
1 , where un
1 P En
1 .

k 1

Then
T yn  λn apnnq xn vn
1 ,  T un
1 P En
1 .
where vn
1
Now we are ready to estimate }T yn
T ym } for n ¡ m. Since T ym P Em „ En
1 ,
we obtain
}T yn
T ym }  }λn apnnq xn wn
1 } where wn
1 P En
1
 }λn yn zn
1 } where wn
1 P En
1
¥ |λn | distpyn , En
1 q ¥ 2ε .
It follows that pT yn q8
n1 contains no Cauchy subsequences as claimed. The proof
is complete. 
Proposition 4.4.16 (Classification of spectrum of compact operators). Let
T P K pX, X q be a compact operator on a Banach space X. Then
σ pT q  σp pT q Y t0u.
Proof. As we already noticed in the beginning of this section, 0 P σ pT q. Let
now λ P σ pT q, λ  0. Fredholm alternative (Theorem 4.2.2) states that either
T
λI is not injective (in which case λ P σp pT q) or T
λI is both injective and
surjective. In the latter case T
λI is invertible by the inverse mapping theorem,
which means that λ R σ pT q. The proof is complete. 
 4.4. PROPERTIES OF SPECTRUM. SPECTRUM OF COMPACT OPERATORS. 108

4.4.6. Spectrum of unitary operators. In this subsection, H denotes a

Hilbert space.

Definition 4.4.17. An operator U P LpH, H q is called a unitary operator if U

is a bijective isometry on H. The latter means that U is bijective and

}U x}  }x} for all x P H.

Example 4.4.18. Examples of unitary operators include:

(i) operators on Cn and Rn given by n  n unitary complex matrices and orthog-
onal real matrices; in particular rotations, symmetries, and permutations of
coordinates in Cn and Rn ;
(ii) right shift R on `2 (but not left – why?)
(iii) an isometry between any pair of separable Hilbert spaces established in The-
orem 1.6.30.

Remark 4.4.19. A unitary operator U preserves all pairwise distances, i.e.

}U x
U y}  }x
y}.
Moreover, by polarization identity 1.4.19, U also preserves
the inner products:

xU x, U yy  xx, yy for all x, y P H.

Proposition 4.4.20. An operator U P LpH, H q is unitary if and only if
U  U  U U   I,

i.e. if and only if U is invertible and U
1  U  .

This identity is analogous to the one for unit complex numbers, zz  zz  1.

Proof. Necessity. If U is unitary then xU  U x, y y  xU x, U y y  xx, y y. Since

this holds for arbitrary x, y P H, it follows that U  U  I. Similarly one proves
that U U   I.
Sufficiency. Since U is invertible it is bijective. The isometry property follows
because }U x}2  xU x, U xy  xU  U x, xy  xx, xy  }x}2 . The proof is complete.


Proposition 4.4.21 (Spectrum of unitary operators). The spectrum of a uni-

tary operator U P LpH, H q lies on the unit circle:

σ pU q „ tλ P C : |λ|  1u.

Proof. The isometry property implies that }U }  }U
1 }  1. Therefore the

spectral radius is rpU q ¤ }U } ¤ 1 by Proposition spectrum bounded. On the
other hand, if |λ|   1 then the operator U
1 pU
λI q  I
λU
1 is invertible by
von Neumann’s Lemma 4.4.3 since }λU
1 }  |λ|   1. It follows that U
λI is
invertible. 

Exercise 4.4.22. Show that eigenvectors of a unitary operator U that

correspond to distinct eigenvalues are orthogonal.
 4.4. PROPERTIES OF SPECTRUM. SPECTRUM OF COMPACT OPERATORS. 109

4.4.7. Additional exercises. In the following two exercises, one can work
over R. Similar results hold over C. The only difference is that for Hilbert spaces,
one has to take complex conjugation in appropriate places (which ones?), see Re-
mark 2.4.26.
Exercise 4.4.23. [Spectrum of adjoint I] Let T P LpX, X q. Prove that
σ pT  q  σ pT q. Here the bar stands for complex conjugation rather than
for closure.

Exercise 4.4.24. [Spectrum of adjoint II] Let T P LpX, X q

(i) Prove that if λ P σp pT q and λ R σp pT  q then λ P σr pT  q. (Hint: use
the duality relations from Proposition 2.4.32 and Exercise 2.4.34
for the operator T
λI.)
(ii) Prove that
σr pT q „ σp pT  q „ σr pT q Y σp pT q.
Deduce that if X is reflexive, then σr pT  q € σp pT q. Deduce that
self-adjoint bounded linear operators in Hilbert space do not have
residual spectrum.

Exercise 4.4.25. [General multiplication operator on L2 ] Consider a

general multiplication operator T acting on L2 r0, 1s as
pT f qptq  gptqf ptq
where g P L8 r0, 1s is some fixed multiplier function.
Exercise 4.4.26. [Invertibility of a composition] Let S, T P LpX, X q.
Prove that the operator ST is invertible if and only if both S and T are
invertible.

Exercise 4.4.27. [Partial case of spectral mapping theorem] Let T P

LpX, X q. Prove that λ P σ pT q implies λn P σ pT n q. (Hint: (i) factor T n

λn I  S pT
λI q for some S P LpX, X q. (ii) Show that for U, V P LpX, X q,
the operator U V is invertible if and only if both U and V are invertible.)

Exercise 4.4.28. [Spectrum of projections] Compute the spectrum of

a projection P P LpX, X q on a Banach space X.
 CHAPTER 5

Self-adjoint operators on Hilbert space

Lec. 36: 12/03
Throughout this chapter, H will denote a Hilbert space, and we will study
bounded self-adjoint operators T on H.

5.1. Spectrum of self-adjoint operators

5.1.1. Definition and examples. Let T be a bounded linear operator on a
Hilbert space, i.e. T P LpH, H q. Recall from Section 2.4.9 that the adjoint operator
T  P LpH, H q is defined by xT  x, y y  xx, T y y for x, y P H.
Definition 5.1.1. An operator T P LpH, H q is called self-adjoint if T   T ,
i.e.
xT x, yy  xx, T yy, x, y P H.
Example 5.1.2. Examples of self-adjoint operators include:
(i) linear operators on Cn given by Hermitian matrices paij q, i.e. such that aij 
aji ;
³1
(ii) integral operators pT f qptq  0 k ps, tqf psq ds on L2 r0, 1s with Hermitian sym-
metric kernels, i.e. such that k ps, tq  k pt, sq;
(iii) orthogonal projections P on H. (Why?)
Every bounded linear operator can be decomposed into two self-adjoint opera-
tors:
Lemma 5.1.3. Every operator A P LpH, H q can be uniquely represented as
AT iS
where T, S P LpH, H q are self-adjoint operators.
Proof. If A  T iS then A  T
iS. Solving these two equations we see
 
that the lemma holds with T  A 2A and S  A
2iA . 
Exercise 5.1.4. Prove that the set of self-adjoint operators forms a
closed linear subspace in LpH, H q.

5.1.2. The quadratic form and the norm of a self-adjoint operator.

It is convenient to study self-adjoint operators T P LpH, H q through the quadratic
form
f pxq  xT x, xy, x P H.
One sees immediately that this quadratic form is real-valued, i.e. f pxq P R for
all x P H.1

x y  xx, T xy  xT x, xy.
1This follows from the identity T x, x

110
 5.1. SPECTRUM OF SELF-ADJOINT OPERATORS 111

Furthermore, the quadratic form f pT q determines the operator T uniquely.

This follows from the polarization identity that relates bilinear and quadratic forms:

 
(5.1) xT x, yy  41 f px y q
f px
y q if px iy q
if px
iy q
This is a slight generalization of the polarization identity of Proposition 1.4.19,
where T  I. (Prove it!)
The norm of T can be conveniently computed from the quadratic form f pT q:
Proposition 5.1.5 (Norm of a self-adjoint operator). For every self-adjoint
operator T P LpH, H q, one has
}T }  sup |xT x, xy|.
P
x SH

Proof. The lower bound follows by definition of the operator norm:

}T }  sup }T x}  sup |xT x, yy| ¥ sup |xT x, xy| : M.
P
x SH P
x,y SH P
x SH

It remains to show that the inequality here is actually the identity. To this end, we
note that
sup |xT x, y y|  sup RexT x, y y
P
x,y SH P
x,y SH
and use the real part of polarization identity (5.1):
1 
RexT x, y y  xT px y q, x y y
xT px
y q, x
y y
4
 
¤ M4 }x y}2 }x
y}2 (by the definition of M )
 
¤ M4 2}x}2 2}x}2 (by the parallelogram law)
¤ M (as }x}  }y}  1).
This completes the proof. 
5.1.3. Criterion of spectrum points. We would like to study the spectrum
of self-adjoint operators T P LpH, H q. An easy observation is that all eigenvalues
of T must be real, that is
σp pT q „ R.
Indeed, if λ is an eigenvalue with an eigenvector x then xT x, xy  xλx, xy  λxx, xy
which must be the same as xx, T xy  xx, λxy  λxx, xy. This shows that λ  λ, so
λ P R.
We will soon prove that the whole spectrum of T is real, i.e. σ pT q „ R, and
moreover we will compute the smallest interval containing σ pT q. Let us start with
ruling out the residual spectrum:
Proposition 5.1.6 (No residual spectrum). Let T P LpH, H q be a self-adjoint
operator. Then σr pT q  H.
Proof. Let λ P σr pT q. This means that kerpT
λI q  0 while ImpT
λI q is
not dense in H. Since λ is not an eigenvalue, λ is not an eigenvalue either (recall
that all eigenvalues of T must be real). Using this and the duality relation (2.4.32),
we obtain that pImpT
λI qqK  kerpT
λI q  kerpT
λI q  0. It follows that
ImpT
λI q is dense in H, which is a contradiction. 
 5.1. SPECTRUM OF SELF-ADJOINT OPERATORS 112

Proposition 5.1.7 (Invertibility criterion). Let T P LpH, H q be a self-adjoint

operator. T is invertible2 if and only if T is bounded below, i.e. there exists c ¡ 0
such that
}T x} ¥ c}x} for all x P H.
Proof. If T is invertible then T is bounded below with c  1{}T
1 }. To prove
the converse, assume that T is bounded below. Then by the criterion of isomorphic
embedding (Proposition 3.1.12), T is injective and Im T is closed in H. On the
other hand, since 0 R σr pT q  H, injectivity of T implies that Im T is dense in H.
It follows that Im T  H. So T is injective and surjective, thus T is invertible by
the inverse mapping theorem. 
Applying this result for the operator T
λI, we immediately obtain
Corollary 5.1.8 (Criterion of spectrum points). Let T P LpH, H q be a self-
adjoint operator. Then λ P σ pT q if and only if the operator T
λI is not bounded
below.
Remark 5.1.9 (Approximate point spectrum). A number λ P σ pT q for which
T
λI is not bounded below is called an approximate eigenvalue of T , and the
set of all approximate eigenvalues is called the approximate point spectrum of T .
Corollary 5.1.8 states that for self-adjoint operators, the whole spectrum is the
approximate point spectrum.
The reason for the name “approximate” is the following. If λ is an eigenvalue
then pT
λI qx  0 for some x P SH . If λ is an approximate eigenvalue then
pT
λI qx can be made arbitrarily close to zero for some x P SH . So, eigenvalues
of T form the point spectrum σp pT q while the approximate eigenvalues of T form
the continuous spectrum σc pT q.
5.1.4. The spectrum interval. Now we compute the tightest interval that
contains the spectrum of a self-adjoint operator T . This interval can be computed
from the quadratic form of T :
Theorem 5.1.10 (Spectrum interval). Let T P LpH, H q be a self-adjoint oper-
ator. Then
(i) The spectrum of T is real, and moreover σ pT q „ rm, M s where
m  inf xT x, xy, M  sup xT x, xy.
P
x SH P
x SH

(ii) The endpoints m, M P σpT q.

Proof. (i) Let λ P Czrm, M s; since the interval is closed we have
d : distpλ, rm, M sq ¡ 0.
Given x P SH , we use Cauchy-Schwarz to obtain the lower bound
   
}pT
λI qx} ¥ xpT
λI qx, xy  xT x, xy
λ ¥ d
where the last inequality follows because xT x, xy P rm, M s by definition. We have
shown that T
λI is bounded below. By the criterion of spectrum points (Propo-
sition 5.1.8), we conclude that λ R σ pT q.
(ii) Let us show that M P σ pT q; the claim for m can be proved similarly. (Do
this!) Without loss of generality we can assume that 0 ¤ m ¤ M (This follows by
2As usual, by being invertible we mean continuously invertible, i.e. T 1
P LpH, H q.
 5.2. SPECTRAL THEOREM FOR COMPACT SELF-ADJOINT OPERATORS 113

Figure 5.1. Distance from λ to the spectrum interval rm, M s is d.

a translation argument, namely by considering T
mI instead of T . Check this!)

So }T }  M .
Let us choose a sequence of vectors xn P SH so that xT xn , xn y Ñ M . Then
@ D
}pT
M I qxn }2  pT
M I qxn , pT
M I qxn
 }T xn }2
2M xT xn , xn y M 2 }xn }2 .
Now, }T xn }2 ¤ }T }2  M 2 , xT xn , xn y Ñ M and }xn }2  1. It follows that
lim sup }pT
M I qxn }2 ¤ M 2
2M 2 M 2  0
so T
M I is not bounded below. Therefore M P σ pT q. The proof is complete. 

As a consequence of this result, the spectral radius rpT q of a self-adjoint oper-

ator equals }T }, so Proposition 4.4.3 is tight and Gelfand’s formula is useless for
self-adjoint operators:
Corollary 5.1.11 (Spectral radius). Let T P LpH, H q be a self-adjoint oper-
ator. Then
rpT q  max |λ|  }T }.
Pp q
λ σ T

Proof. By the properties of the spectrum interval in Theorem 5.1.10,

rpT q  maxp|m|, |M |q  }T }
as claimed. 

5.1.5. Additional Exercises.

Exercise 5.1.12. Consider a self-adjoint operator P P LpH, H q such
that P 2  P . Prove that P is an orthogonal projection.

5.2. Spectral theorem for compact self-adjoint operators

Lec.37: 12/06
Compact self-adjoint operators on a Hilbert space H are proxies of Hermitian
matrices. As we know from linear algebra, every Hermitian matrix has diagonal
form in some orthonormal basis of Cn . Equivalently, such for such a matrix there
exists an orthonormal basis of Cn consisting of the eigenvectors. In this section, we
generalize this fact to infinite dimensions, for all compact self-adjoint operators on
H.
 5.2. SPECTRAL THEOREM FOR COMPACT SELF-ADJOINT OPERATORS 114

5.2.1. Invariant subspaces.

Proposition 5.2.1 (Eigenvectors orthogonal). Let T P LpH, H q be a self-
adjoint operator. Then its eigenvectors corresponding to distinct eigenvalues are
orthogonal.
Proof. If T x1  λ1 x1 and T x2  λ2 x2 then
λ1 xx1 , x2 y  xT x1 , x2 y  xx1 , T x2 y  λ2 xx1 , x2 y.
(In the last identity we used that λ2 is always real, so there is no conjugation). It
follows that if λ1  λ2 then xx1 , x2 y  0 as claimed. 

Definition 5.2.2 (Invariant subspace). A subspace E of H is called an invari-

ant subspace of T if T pE q „ E.
Example 5.2.3. Every eigenspace of T is invariant. More generally, the linear
span of any subset of eigenvectors of T is an invariant subspace.
One of the most well known open problems in functional analysis is the invari-
ant subspace problem. It asks whether every operator T P LpH, H q has a proper
invariant subspace (i.e. different from t0u and H).
Proposition 5.2.4. Let T P LpH, H q be self-adjoint. If E „ H is an invariant
subspace of T then E K is also an invariant subspace of T .
Proof. Let x P E K ; we need check that T x P E K . So let us choose y PE
arbitrarily. Then xT x, y y  xx, T y y  0 since x P E K and y P E so T y P E. 

5.2.2. Spectral theorem. The following result is known as the Hilbert-Schmidt

theorem.
Theorem 5.2.5 (Spectral theorem for compact self-adjoint operators). Let T
be a compact self-adjoint linear operator on a separable Hilbert space H. Then there
exists an orthonormal basis of H consisting of eigenvectors of H.
Proof. Let us first prove that T has at least one eigenvector.
By Proposition 4.4.16,
σ pT q  σp pT q Y t0u.
If σ pT q  0 then σp pT q  H, so T has an eigenvector. If σ pT q  0 then by
Corollary 5.1.11 we have }T }  rpT q  0, so T  0 and every vector in H is an
eigenvector of T .
We will complete the proof by induction. Consider the family of all orthonormal
sets in H consisting of eigenvectors of T . All such sets are at most countable since
H is separable. By Zorn’s lemma, this family has a maximal element pφk q8 k 1 .
(Check!) It remains to show that E : Spanpφk q  H.
Suppose E  H. Since E is an invariant subspace of T (check!), E K  t0u is
also an invariant subspace of T by Proposition 5.2.4. So we can use the first part of
the proof for the restriction T |E K which is a compact self-adoint operator on E K .
It follows that T |E K (and thus T itself) has an eigenvector in E K . This contradicts
the maximality of pφk q. The proof is complete. 
 5.2. SPECTRAL THEOREM FOR COMPACT SELF-ADJOINT OPERATORS 115

5.2.3. Diagonalization. Spectral Theorem 5.2.5 allows us to always repre-

sent compact self-adoint operators T P LpH, H q in a diagonal form, similarly to the
one for Hermitian matrices.
Let pφk q be an orthonormal basis of eigenvectors of T . Then T φk  λφk where
λn are the eigenvalues. We can identify the space H with `2 by identifying pφk q
with the canonical basis pek q of `2 (recall Section 1.6.7). With this identification,
T becomes a multiplication operator acting on `2 as T ek  λk ek ; equivalently
T ppxk q8 8
k1 q  pλk xk qk1 .

We see that T now has a quite simple form, which we studied in Example 4.3.5.
In literature, one comes across various forms of spectral Theorem 5.2.5. We
mention two of them. Let as before pφk q denote an orthonormal basis of eigenvectors
of T with corresponding eigenvalues λk . Orthogonal basis expansion gives
¸
x xx, φk yφk , x P H.
k

Applying operator T for both sides and using that T φk  λk φk we obtain that
¸
Tx  λn xx, φn yφn , x P H.
n

This is sometimes written as

¸
(5.2) T  λ k φk b φk
k

where φk b φk P LpH, H q denotes the elementary tensor, which is the rank-one

orthogonal projection in H onto the span of the vector φk .
Furthermore, by Theorem 4.4.15 all eigenvalues λk have finite multiplicity. So,
grouping in (5.2) the tensors φk b φk that correspond to the same λk , we obtain
the following decomposition of T :
¸
(5.3) T  λ k Pk
k

where Pk denotes the orthogonal projection in H onto the (finite-dimensional)

eigenspace corresponding to the eigenvalue λk .
Remark 5.2.6 (Normal operators). One can show that spectral Theorem 5.2.5
holds also for compact normal linear operators T on H. Recall that T is normal
if T  T  T T  . The only difference between the spectral properties of self-adjoint
and normal operators is that the eigenvalues of normal operators need not be real.
5.2.4. Separation of variables. We illustrate Spectral Theorem 5.2.5 with
a purely analytic consequence. The following result shows how one can separate
variables of a general function k pt, sq. It is also due to Hilbert and Schmidt.
Theorem 5.2.7 (Separation of variables). Consider a function k pt, sq P L2 pr0, 1s2 q
such that k pt, sq  k ps, tq. There exists an orthonormal basis pφn q of L2 r0, 1s and
numbers λk Ñ 0 such thath
8̧
k pt, sq  λn φn ptqφn psq.

n 1

The convergence of this series is understood in L2 pr0, 1s2 q.

 5.3. POSITIVE OPERATORS. CONTINUOUS FUNCTIONAL CALCULUS 116

³1
Proof. Consider the integral operator pT f qptq  0 k pt, sqf psq ds on L2 r0, 1s.
Let pφn q be an orthonormal basis of its eigenvectors. Then the functions
 φn ptqφm psq, n, m  1, 2, . . .
ψnm
form an orthonormal basis of L2 pr0, 1s2 q. (Check!)
Let us write the basis expansion of our function in L2 pr0, 1s2 q:
¸
k xk, ψnm yψnm .
n,m

Now we compute the coefficients

»1»1
xk, ψnm y  k pt, sqφn ptqφm psq dt ds
0 0
»1
» 1
 k pt, sqφm psq ds φn ptq dt (by Fubini theorem)
0 0
»1
 pT φm qptqφn ptq dt  xT φm , φn y  λm xφm , φn y
#0
nm
 λm ,
0, n  m.
Therefore ¸ ¸
k  xk, ψnn yψnn  λn φn ptqφn psq
n n
as claimed. 

5.2.5. Additional exercises.

Exercise 5.2.8. [Spectral theorem for general compact operators] Let
T be a compact linear operator on a separable Hilbert space H. Show
that there exist an orthonormal basis pφk q of H, an orthonormal system
pψk q in H, and a sequence of numbers λk ¥ 0, λk Ñ 0 such that
¸
T  λn φk b ψk .
n
The numbers λk are called singular values of T and the vectors φk and
ψk are called left (resp. right) singular vectors of T .
(Hint: Choose pψk q to be an orthonormal basis of eigenvectors of
T  T . Write the basis expansion of x P H and apply T to both sides.)

5.3. Positive operators. Continuous functional calculus

Lec. 38: 12/8
It is a powerful idea to work with linear operators by analogy with complex
numbers. For example, both numbers and operators can be added and multiplied,
so both C and LpH, H q are algebras (over C). Moreover, complex conjugate corre-
sponds to operator adjoint, which makes LpH, H q a  -algebra. Operator algebras
is a rich topic which we omit in this course.
In this section, we develop the analogy between numbers and operators in the
following way. We introduce a partial order on the set of self-adjoint operators
on T P LpH, H q, and we define an operator f pT q P LpH, H q for every continuous
 5.3. POSITIVE OPERATORS. CONTINUOUS FUNCTIONAL CALCULUS 117

function f : C Ñ C. In other words, we develop what is called the functional

calculus of operators.

5.3.1. Positive operators.

Definition 5.3.1. A self-adjoint operator T P LpH, H q is called positive3 if
xT x, xy ¥ 0 for all x P H.
Positive operators are generalizations of non-negative numbers (which corre-
spond to operators on one-dimensional space C1 ).
Example 5.3.2. Examples of positive operators include:
(i) T 2 for every self-adjoint T P LpH, H q, as xT 2 x, xy  xT x, T xy ¥ 0;
(ii) Hermitian matrices with non-negative eigenvalues;
(iii) More generally, compact self-adjoint operators on H with non-negative eigen-
values. (Why?)
Definition 5.3.3 (Partial order). For self-adjoint operators S, T P LpH, H q,
we shall say that S ¤ T if T
S ¥ 0.
This defines a partial order on LpH, H q.
Let us restate Theorem 5.1.10 on the spectrum interval in these new terms:
Theorem 5.3.4 (Spectrum interval). Let T P LpH, H q be a self-adjoint opera-
tor. Let m, M be the smallest and the largest numbers such that
¤ T ¤ M I.
mI
Then σ pT q „ rm, M s and m, M P σ pT q. 
As an immediate corollary, T is positive if and only if its spectrum is positive:
Corollary 5.3.5. Let T P LpH, H q be a self-adjoint operator. Then T ¥ 0 if
and only if σ pT q „ r0, 8q. 
5.3.2. Polynomials of an operator. We start to develop a functional calcu-
lus for self-adjoint operators T P LpH, H q. We begin by defining polynomials of T ,
then we extend the definition to continuous functions of T by approximation. Work-
ing with polynomials is straightforward, and the result of this subsection remain
valid for every bounded linear operator T on a general Banach space X.
Definition 5.3.6 (Polynomials of an operator). Consider a polynomial pptq 
a0 a1 t    an tn . For an operator T P LpH, H q, we define
ppT q  a0 I a1 T    an T n .
If T is self-adjoint operator then ppT q is also self-adjoint. (Check!) Moreover,
for two polynomials f and g, one has
paf bg qpT q  a  f pT q b  g pT q, pf gqpT q  f pT qgpT q, f pT q  f pT  q.
This last property states in other words that for a fixed T P LpH, H q, the map
p ÞÑ ppT q is an -algebra homomorphism from P rts into LpH, H q.
The following example may serve us as a test case for many future results.

3In linear algebra, positive operators are called positive semidefinite.

 5.3. POSITIVE OPERATORS. CONTINUOUS FUNCTIONAL CALCULUS 118

Example 5.3.7. Let T be a self-adjoint linear operator on an n-dimensional

Hilbert space. In an orthonormal basis of eigenvectors, T can be identified with the
n  n diagonal matrix
T  diagpλ1 , . . . , λn q
where λk are the eigenvalues of T . Then for every polynomial pptq we have
ppT q  diagpppλ1 q, . . . , ppλn qq.
(Check!) This example can be generalized for all compact self-adjoint operators T
on a general Hilbert space H. (Do this!)
5.3.3. Spectral mapping theorem for polynomials.
Lemma 5.3.8 (Invertibility). Let pptq be a polynomial and T P LpH, H q. Then
the operator ppT q is invertible if and only if pptq  0 for all t P σ pT q.
Proof. Let us factorize pptq  an pt
t1 q    pt
tn q where tk are the roots of
pptq. Then ppT q  an pT
t1 I q    pT
tn I q. By Exercise 4.4.26, ppT q is invertible
if and only if all factors T
tk I are invertible. This in turn is equivalent to the fact
that all roots tk R σ pT q. 

The spectrum of a polynomial ppT q can be easily computed from the spectrum
of T :
Theorem 5.3.9 (Spectral mapping theorem). Let pptq be a polynomial and
T P LpH, H q. Then4
σ pppT qq  ppσ pT qq.
Proof. For every complex number λ, we have λ P σ pppT qq if and only if the
operator ppT q
λI  pp
λqpT q is not invertible. By the invertibility Lemma 5.3.8,
this is equivalent to the condition that pp
λqptq  0 for some t P σ pT q, which
means that pptq  λ for some t P σ pT q. The latter is equivalent to λ P ppσ pT qq. 

Using the spectral mapping theorem, one can in particular easily compute the
norms of operator polynomials:
Corollary 5.3.10 (Operator norm of polynomials). Let pptq be a polynomial
and T P LpH, H q be a self-adjoint operator. Then
}ppT q}  tmax
PσpT q
|pptq|.
This result generalizes the identity rpT q  }T } for the spectral radius of self-
adjoint operators T proved in Corollary 5.1.11.

Proof. Let us apply Corollary 5.1.11 for the operator ppT q. Then spectral
mapping theorem yields
}ppT q}  rpppT qq  tPσmax
pppT qq
|t|  tPpmax
pσpT qq
|t|  smax
PσpT q
|ppsq|
as claimed. 

p p qq : tpptq : t P σpT qu.

4Here we use the notation p σ T
 5.3. POSITIVE OPERATORS. CONTINUOUS FUNCTIONAL CALCULUS 119

5.3.4. Continuous functions of an operator. Let T P LpH, H q be a self-

adjoint operator, and f ptq be a continuous function on σ pT q. We would like to
define f pT q P LpH, H q. To this end, we use Weierstrass approximation theorem,5
and we find polynomials pn ptq such that
(5.4) pn ptq Ñ f ptq uniformly on σ pT q.
This suggests us to define f pT q as the limit of operator polynomials pn pT q:
Definition/Proposition 5.3.11 (Continuous functions of an operator).
(i) The sequence pn pT q converges in LpH, H q to a limit that we call f pT q.
(ii) The operator f pT q P LpH, H q is self-adjoint, and it does not depend on the
choice of the approximating polynomials pn .
Proof. (i) By completeness of LpH, H q it suffices to check that ppn pT qq is a
Cauchy sequence in the operator norm. Computing the operator norm with the
help of Corollary 5.3.10 and using (5.4), we obtain the desired conclusion that
}pn pT q
pm pT q}  }ppn
pm qpT q}  tmax |pp
p qptq| Ñ 0
PσpT q n m
as n, m Ñ 8.

(ii) Since the operators pn pT q are self-adjoint, and the self-adjoint operators
form a closed subset of LpH, H q (Exercise 5.1.4), f pT q is also self-adjoint. Further-
more, repeating the estimate in part (i), one sees that for any other approximating
sequence of polynomials qn ptq one has }pn pT q
qn pT q} Ñ 0 as n Ñ 8. It follows
that the limit f pT q must be the same whether one chooses pn pT q or qn pT q as an
approximating sequence. 

By passing to the limit in the corresponding properties for polynomials, one

sees that for two polynomials f and g we have
paf bg qpT q  a  f pT q b  g pT q, pf gqpT q  f pT qgpT q, f pT q  f pT  q.
(Check!) This property states in other words that for a fixed T P LpH, H q, the map
f ÞÑ f pT q is an -algebra homomorphism from C pσ pT qq into LpH, H q.
Example 5.3.12. Consider an invertible self-adjoint operator T P LpH, H q;
then σ pT q P rm, M s with m ¡ 0. Consider the function f ptq  1{t, which is
continuous on rm, M s. Then f pT q  T
1 (Check!) In other words, we have the
remarkable identity of reciprocal and inverse:
1
T
 T
1 .
Lec. 39: 12/10

5.3.5. Spectral mapping theorem. We will now generalize the spectral

mapping Theorem 5.3.9 from polynomials to continuous functions of an operator.
It is based on the straightforward generalization of the invertibility Lemma 5.3.8:
Lemma 5.3.13 (Invertibility). Let T P LpH, H q be a self-adjoint operator and
f P C pσ pT qq. Then the operator f pT q is invertible if and only if f ptq  0 for all
t P σ pT q.

5Formally, since Weierstrass theorem is typically stated for functions on an interval, we first
pq
extend f t to a continuous function on some interval m, M r s… p q
σ T (for example, the spectral
interval). Such extension can be done e.g. by Tietze extension theorem.
 5.3. POSITIVE OPERATORS. CONTINUOUS FUNCTIONAL CALCULUS 120

Proof. Sufficiency. If f ptq  0 for all t P σ pT q then 1{f P C pσ pT qq, so 1{f pT q

is the inverse of T (see Example 5.3.12).
Necessity. Assume that f pt0 q  0 for some t0 P σ pT q. Choose a sequence
of polynomials pn ptq such that pn ptq Ñ f ptq uniformly on spT q. Without loss
of generality, we can assume that pn pt0 q  f pt0 q  0. (Justify this by a slight
translation!) Then pn pT q Ñ f pT q in LpH, H q.
On the other hand, the invertibility Lemma 5.3.8 for polynomials, the operators
pn pT q are not invertible. Since the non-invertible operators form a closed subset of
LpH, H q (see Exercise 2.4.44), it follows that f pT q is not invertible. This completes
the proof. 
Now the spectral mapping theorem follows from invertibility Lemma 5.3.13 by
the same argument as the corresponding result for polynomials, Theorem 5.3.9:
Theorem 5.3.14 (Spectral mapping theorem). Let T P LpH, H q be a self-
adjoint operator and f P C pσ pT qq. Then
σ pf pT qq  f pσ pT qq. 
This gives a simple way to create positive operators:
Corollary 5.3.15. Let T P LpH, H q be a self-adjoint operator and f P C pσpT qq.
If f ptq ¥ 0 for all t P σ pT q then f pT q ¥ 0.
Proof. By Corollary 5.3.5 it suffices to check that σ pf pT qq „ r0, 8q. The
latter follows from the spectral mapping theorem, as σ pf pT qq  f pσ pT qq „ r0, 8q.

Exercise 5.3.16. Prove the converse statement in Corollary 5.3.15.
It follows that the algebra homomorphism from C pσ pT qq into LpH, H q preserves
not only addition and multiplication, but also the order:
f ptq ¤ g ptq for all t P σ pT q implies f pT q ¤ g pT q.
Exercise 5.3.17. [Further properties of continuous functions of an op-
erator] Let T, S P LpH, H q be self-adjoint operators and f, g P C pσ pT qq.
Prove that:
(i) f pT q  f pT q ;
(ii) }f pT q}  suptPσpT q |f ptq| (this generalizes Corollary 5.3.10, and follows
by the same argument);
(iii) If operators S and T commute then operators f pS q and g pT q com-
mute. (Check this for polynomials and pass to a limit.)

? self-adjoint operator
5.3.6. Square root of an operator. Consider a positive
T P LpH, H q. Then σ pT q?
„ r0, 8q. The function f ptq  t is continuous on r0, 8q,
? 2 f pT q  T .
so we can define ?
? tq  t, the algebra homomorphism
Since p ? property implies that p T q2  T .
Since t ¥ 0, Corollary 5.3.15 implies that T is a positive self-adjoint operator.
Summarizing, we have proved the following (except uniqueness):
Proposition 5.3.18 (Square root of an operator). For every positive self-
?
adjoint operator T P LpH, H q, there exists a unique positive self-adjoint operator
T P LpH, H q such that ?
p T q2  T.
 5.3. POSITIVE OPERATORS. CONTINUOUS FUNCTIONAL CALCULUS 121

?
Exercise 5.3.19. Prove the uniqueness of T.
As an application of operator square root, we will now prove the following
result, whose formulation has nothing to do with functional calculus
Proposition 5.3.20. Let S, T P LpH, H q be a pair of commuting self-adjoint
operators. If S ¥ 0 and T ¥ 0 then ST ¥ 0.
For compact operators S, T this would follow because in this case one can find
a common basis of eigenvectors in H. (Do this!) In general, I don’t know any proof
that won’t involve functional calculus.
?
? Proof. We want to show that xST x, xy ¥ 0 for every x P H.?Since ?S ? and
T are communing self-adjoint operators (see Exercise 5.3.17), and ST  S T .
(Why?) So we have
? ? ? ? ? ? ? ? ? ? ? ?
xST x, xy  x S S T T x, xy  x T S  S T x, xy  x S T x, S T xy ¥ 0
as required. 

5.3.7. Modulus of an operator. Now consider an arbitrary operator T P

LpH, H q. Then T  T is a positive self-adjoint operator (check!) so it has a unique
positive square root. Hence we define
?
|T | : T T , T P LpH, H q
which we call the modulus of T . This generalizes the concept of modulus of complex
numbers,
?
|z|  z z, z P C.
Example 5.3.21. (i) For a diagonal operator T  diagpλ1 , . . . , λn q on Cn ,
one has |T |  diagp|λ1 |, . . . , |λn |q. (Check!)
(ii) For the multiplication operator pT f qptq  g ptqf ptq on L2 r0, 1s, one has p|T |f qptq 
|gptq|f ptq. (Check!)
(iii) For the right shift operator R on `2 , we saw that R  L (the left shift), so
|R|  I.
Lemma 5.3.22. For every operator T P LpH, H q and vector x P H, one has
}|T |x}  }T x}.
Proof. This is a matter of straightforward computation:
}|T |x}2  x|T |x, |T |xy  x|T |2 x, xy  xT  T x, xy  xT x, T xy  }T x}2 . 
Lec. 40: 12/13

5.3.8. Polar decomposition. Lemma 5.3.22 motivates us to consider a map

U : |T |x ÞÑ T x, x P H.
The following properties are obvious:
(i) U is well defined;
(ii) U is a linear operator.;
(iii) U is an isometry, i.e. }U y }  }y } for all y P Imp|T |q (by Lemma 5.3.22);
(iv) ImpU q  ImpT q.
We have proved the following result (except uniqueness):
5.4. BOREL FUNCTIONAL CALCULUS. SPECTRAL THEOREM FOR SELF-ADJOINT OPERATORS
122

Theorem 5.3.23 (Polar decomposition). For every operator T P LpH, H q,

there exists a unique bijective linear isometry U P LpImp|T |q, ImpT qq such that
 U |T |.
T
The uniqueness of U follows easily: T x  U |T |x means that U takes |T |x to
T x, thus U is uniquely determined on Imp|T |q.
Theorem 5.3.23 generalizes the polar decomposition in the complex plane. The
latter states that every z P C can be represented as
z  ei Argpzq |z|.
Here ei Argpzq is a unit scalar (generalized by U ), and |z | is the modulus of z (gen-
eralized by |T |).
Remark 5.3.24. In general, U can not be extended to a bijective linear isometry
on the whole space H. Indeed, if T is the right shift on `2 then |T |  I, so the
polar decomposition yields U  T . Although U  T is defined on the whole `2 , its
image is not even dense in `2 , so U is not bijective on `2 .
However, for invertible operators T , it is possible to extend U to a bijective
isometry on the whole space:
Theorem 5.3.25 (Polar decomposition for invertible operators). For every op-
erator T P LpH, H q, there exists a unique unitary operator U P LpH, H q such that
T  U |T |.
?  Since T is invertible, T  is invertible, so T  T is invertible, and finally
Proof.
|T |  T T is invertible. (Why? Use Example 4.4.26.) Therefore ImpT q 
Imp|T |q  H, and the claim follows from Theorem 5.3.23. 
Remark 5.3.26. This second form of the polar decomposition holds also for
all normal operators (those with T  T  T T  ), compact scalar operators, and
generally for all operators for which dim kerpT q  dim kerpT  q.

5.4. Borel functional calculus. Spectral theorem for self-adjoint

operators
In this section, we extend functional calculus to bounded Borel functions of
operators. This is done primarily to define the spectral projections, which are
indicator functions of an operator. Once we have spectral projections, we formulate
the spectral theorem for general (not necessarily compact) self-adjoint operators.
As usual, T P LpH, H q will denote a fixed self-adjoint operator on a Hilbert
space H. Let us fix an interval rm, M s which contains the spectrum σ pT q, e.g. the
spectrum interval.
5.4.1. Borel functional calculus. We consider the linear space of bounded
Borel complex-valued functions on rm, M s; denote this space B rm, M s. We would
like to define f pT q for f P B rm, M s, so that this extends the definition of f pT q for
continuous functions f P C rm, M s. The difficulty is that Borel functions are only
pointwise (but not uniform) limits of continuous functions.
Theorem 5.4.1 (Borel functional calculus). Let T P LpH, H q be a self-adjoint
operator on a Hilbert space H. For each f P B rm, M s one can define a self-adjoint
operator f pT q P LpH, H q such that
5.4. BOREL FUNCTIONAL CALCULUS. SPECTRAL THEOREM FOR SELF-ADJOINT OPERATORS
123

(i) paf bg qpT q  a  f pT q b  g pT q, pf g qpT q  f pT qg pT q, f pT q  f pT  q. In

other words, the map f ÞÑ f pT q is an -algebra homomorphism from B rm, M s
into LpH, H q.
(ii) If fn , f P B rm, M s satisfy supn }fn }8   8 and fn Ñ f pointwise then
fn pT q Ñ f pT q pointwise in LpH, H q.
(iii) }f pT q} ¤ }f }8 .
(iv) If T and S commute then f pT q and g psq commute for f, g P B rm, M s.
Proof. Construction of f pT q. For a fixed pair of vectors x, y P H, define a
linear functional Fx,y on C rm, M s by
Fx,y pf q : xf pT qx, y y, f P C rm, M s.
This functional is bounded:
|Fx,y pf q| ¤ }f pT q}}x}}y} ¤ }f }8 }x}}y},
where the last inequality follows by the spectral mapping Theorem 5.3.14. We have
shown that
Fx,y P pC rm, M sq and }Fx,y } ¤ }x}}y }.
By representation Theorem 2.2.8 for C pK q , there exists a unique Borel regular
signed measure µx,y on rm, M s with total variation |µx,y |prm, M sq ¤ }x}}y }, and
such that » M
xf pT qx, yy  f pλq dµx,y pλq, f P C rm, M s.
m
We extend this to bounded Borel functions by defining
»M
pBf qpx, yq : f pλq dµx,y pλq, f P Brm, M s.
m
(Since µx,y is a Borel measure, the integral is defined.) One quickly checks using
the definition of µx,y that pBf qpx, y q is linear in x and conjugate linear in y. So B
is what is called a sesquilinear form. Moreover, this form is bounded:
|pBf qpx, yq| ¤ }f }8  |µx,y |prm, M sq ¤ }f }8 }x}}y}.
It follows (see Exercise 2.4.45) that B is given by a linear operator that we shall
call f pT q P LpH, H q acting as
pBf qpx, yq  xf pT qx, yy, f P Brm, M s, x, y P H,
and that }f pT q} ¤ }f }8 . We thus defined f pT q for all bounded Borel functions f ,
and we verified (iii).
(ii) with weak convergence. Let us try to prove (ii). By the dominated conver-
gence theorem, for every x, y P H we have
»M »M
xfn pT qx, yy  fn pλq dµx,y pλq Ñ f pλq dµx,y pλq  xf pT qx, y y.
m m
This establishes a somewhat weaker conclusion than in (ii), namely that fn pT qx Ñ
f pT qx weakly for all x P H. We will prove the actual statement (ii) in a minute.
(i). All these identities hold for continuous functions f , and they are preserved
under pointwise limits. As every Borel function is a pointwise limit of continuous
functions, the conclusion follows by applying (ii) with weak convergence. (Check!)
The same argument proves (iv), and it also establishes xthat f pT q is a self-adjoint
operator.
5.4. BOREL FUNCTIONAL CALCULUS. SPECTRAL THEOREM FOR SELF-ADJOINT OPERATORS
124

(ii) completed. Now we complete the proof of (ii) by showing that fn pT qx Ñ

fn pT qx in H. To this end, we bound
@ D
}pfn
f qpT qx}2  pfn
f qpT qx, pfn
f qpT qx
@ D
 ppfn
f qpT qq2 x, x
@ D
 ppfn
f q2 pT qqx, x (by part (i), we have pg2 qpT q  pgpT qq2 )
Ñ0
where the last line follows by part (ii) with weak convergence, as pfn
f q2 Ñ 0
pointwise. This completes the proof. 
Exercise 5.4.2. Show that if f ptq ¥ 0 for all t then f pT q ¥ 0.

5.4.2. Spectral measures. Let us fix a self-adjoint operator T P LpH, H q.

In the proof of Borel functional calculus, we constructed Borel regular measures
µx,y pT q for every pair of vectors x, y P H. They are called spectral measures for T .
We showed that spectral measures satisfy the identity
»M
(5.5) xf pT qx, yy  f pλq dµx,y pλq
m
valid for every bounded Borel function f P B rm, M s. In particular, the bilinear
form of T can be reproduced using spectral measures as
»M
(5.6) xT x, yy  λ dµx,y pλq.
m

Exercise 5.4.3. Compute the spectral measures for the diagonal ma-
trix T  diagpλ1 , . . . , λn q acting as an operator on Cn .

Exercise 5.4.4. Let T be a multiplication operator in L2 r0, 1s by a

function g P L8 r0, 1s. Show that for f P B r0, 1s, the operator f pT q is the
multiplication operator in L2 r0, 1s by the function f pg ptqq.

5.4.3. Spectral projections. Let E be a Borel subset of rm, M s, and we

consider the indicator function
#
1, t P E
1E ptq 
0, t R E.
We shall consider the operator PE  1E pT q P LpH, H q.
Since 1E ptq2  1E ptq and 1E ptq is real valued, the homomorphism property
(iii) in Theorem 5.4.1 yields that PE is an orthogonal projection in H (see Exer-
cise 5.1.12).
The projections PE are called spectral projections associated with T .
Exercise 5.4.5. Show that if E1 „ E2 then PE1 ¤ PE and ImpE1 q „
ImpE2 q.
2

Proposition 5.4.6 (Projection-valued measure). The spectral projections PE

associated with an operator T have the following properties:
(i) PH  0, Prm,M s  I;
5.4. BOREL FUNCTIONAL CALCULUS. SPECTRAL THEOREM FOR SELF-ADJOINT OPERATORS
125

(ii) For every decomposition rm, M s  ”8k1 Ek into disjoint sets Borel Ek , one
has
8̧
I  PE n ,

n 1
where the pointwise convergence of the series.
These properties are strikingly similar to the axioms of a Borel probability
measure on rm, M s. However, the usual probability measures are functions λ : E ÞÑ
λpE q with values in r0, 1s, while PE takes values in LpH, H q, more specifically in the
operator interval between 0 and I. For this reason, the assignment Pλ : E ÞÑ PE
for Borel subsets E „ rm, M s is called a projection-valued measure for T .
The projection-valued measure can be reduced to the usual measure in a simple
way. For x P H, the assignment E ÞÑ xPE x, xy is clearly a usual Borel measure
on rm, M s. This follows from Proposition 5.4.6. Moreover, (5.5) shows that this is
nothing else than the spectral measure µx,x :
»M
xPλ pE qx, xy  xPE x, xy  1E ptq dµx,x pλq  µx,x pE q.
m
For this reason, the projection-valued measure Pλ itself, rather than µx,x , is often
called the spectral measure associated with the operator T .
5.4.4. Spectral theorem for self-adjoint operators.
Theorem 5.4.7 (Spectral theorem). Let T P LpH, H q be a self-adjoint operator
on a Hilbert space H. Then » 8
T  λ dPλ

8
where Pλ is the projection-valued measure associated with T . The integral in fact
can be taken over a finite interval rm, M s containing the spectrum of T .
The integration with respect to the operator-valued measure Pλ is understood
in the sense that » M
xT x, xy  λ xdPλ x, xy;
m
As we noted, xdPλ x, xy is just the spectral measure µx,x , so the last integral is the
usual Lebesgue integral.
Proof. With this remark, Theorem 5.4.7 is a reformulation of a partial case
of (5.6):
»M
xT x, xy  λ dµx,x pλq. 
m
Theorem 5.4.7 should be compared to the spectral Theorem 5.3 for compact
self-adjoint operators T . According to this theorem, T can be decomposed into the
sum ¸
T  λ k Pk
k
where λk are the eigenvalues of T and Pk denotes the orthogonal projection in H
onto the eigenspace corresponding to the eigenvalue λk , see (5.3).
As we know, for general operators T (not necessarily compact), the spectrum
of T may no longer be countable. So the sum is replaced by the integral in Theo-
rem 5.4.7.
 Bibliography

[1] K. Ball, An elementary introduction to modern convex geometry. Flavors of geometry, 1–58,
Math. Sci. Res. Inst. Publ., 31, Cambridge Univ. Press, Cambridge, 1997.
[2] L. Carleson, On convergence and growth of partial sums of Fourier series, Acta Mathe-
matica 116 (1966), 135–157.
[3] P. Enflo, A counterexample to the approximation problem in Banach spaces., Acta Math.
130 (1973), 309–317.
[4] E. Gluskin, The diameter of the Minkowski compactum is roughly equal to n, Funktsional.
Anal. i Prilozhen. 15 (1981), 72–73.
[5] W. T. Gowers, B. Maurey, The unconditional basic sequence problem, J. Amer. Math. Soc.
6 (1993), 851–874
[6] J. Lindenstrauss, Joram, On complemented subspaces of m, Israel J. Math. 5 (1967), 153–
156.
[7] J. Lindenstrauss, L. Tzafriri, On the complemented subspaces problem, Israel J. Math. 9
(1971), 263–269.
[8] J. Lindenstrauss, L. Tzafriri, Classical Banach spaces. I and II. Springer-Verlag, , 1977,
1979.
[9] A. Pelczkynski, Projections in certain Banach spaces, Studia Math. 19 (1960), 209–228.
[10] R. S. Phillips, On linear transformations, Trans. Amer. Math. Soc. 48 (1940), 516–541.

126
 Index

Absorbing set, 52 Essentially bounded, 5

Adjoint operator, 63 Extension
Angle between vectors, 21 of bounded linear functionals, 47
Annihilator, 64 of bounded linear operators, 61
approximate, 112
Fourier
Banach space, 15 basis, 29, 80
Banach-Alaoglu’s theorem, 91 coefficients, 31
Banach-Mazur distance, 73 series, 28, 31
Basis Fourier basis, 80
constant, 81 Frame, 36
Hamel, 2 Fredholm
orthonormal, 32 alternative, 99
Schauder, 79 integral equation, 59
Bessel’s inequality, 31 theory, 98
Biorthogonal functionals, 82
Gelfand’s formula, 105
Cauchy-Schwarz inequality, 20 Gram-Schmidt orthogonalization, 33
Closed graph theorem, 73 Graph of an operator, 73
Codimension, 4
Compact Haar system, 36, 80
operator, 94 Hellinger-Toeplitz theorem, 75
set, 83 Hermite polynomials, 37
compact operators, 106 Hilbert cube, 85
Completion, 18 Hilbert space, 25
Convex Hilbert-Schmidt
combination, 14 operator, 23, 58
hull, 14 theorem, 114
Convexity, 9 Hyperplane, 41

Differential operator, 59, 74 Image, 5

Dirac delta function, 39, 91 closed, 72, 98
Direct sum of normed spaces, 13, 71 Injectivization, 6
Dirichlet kernel, 60 Inner product, 19
Dual space, 41 Integral operator, 58, 94, 97, 100
of `p , 45 Inverse mapping theorem, 70
p q
of C K , 46 Isometry, 61
of Lp , 45 Isomorphic embedding, 72
of a Hilbert space, 42 Isomorphism, 61, 70
second dual, 51
Kernel, 5
Embedding, 6
Equicontinuity, 85 Legendre polynomials, 34
Ergodic theory, 64 Linear functional, 39
Ergodic transformation, 66 bounded, 40

127
 INDEX 128

Linear operator, 5 Separation of convex sets, 53

bounded, 56 Series in normed spaces, 16
norm, 56 Sesquilinear form, 67, 123
Shift operator, 6, 61, 103
Measure-preserving transformation, 66 Singular values and vectors, 116
Minkowski functional, 14, 52 Spectral
Minkowski inequality, 11 measure, 124
Modulus of an operator, 121 projection, 124
radius, 105, 113
Net, 83
theorem, 114, 125
Norm, 7
Spectral mapping theorem, 118, 120
equivalent, 71
Spectrum
Normed space, 7
continuous, 102
Open map, 68 interval, 112, 117
Open mapping theorem, 68 of a bounded linear operator, 101
Operator norm, 56 point, 102
Orthogonal complement, 26 residual, 102
orthogonal decomposition, 28 Square root of an operator, 120
orthogonal projection, 28 Strong
Orthogonal series, 30 topology, 86
Orthogonal system, 29 Sublinear function, 52
Orthogonality principle, 26 Subspace, 2, 6, 12, 15
complementary, 6
Parallelogram law, 24 complemented, 62
Parseval’s identity, 32 invariant, 114
Perfectly convex set, 69 Supporting functional, 50
Point evaluation functional, 39
Pointwise Triangle inequality, 7
bounded family of operators, 76 Tychonoff’s theorem, 92
convergence of operators, 84 Uniform
Polar decomposition, 122 convergence of operators, 84
Polarization identity, 24, 111 Uniformly
Positive operator, 117 bounded family of operators, 76
Precompact set, 83 Unitary operator, 61, 108
Principle of uniform boundedness, 76 p q
Universal space C K , 92
Projection, 62
onto convex sets, 28 Volterra operator, 59, 95
orthogonal, 26, 28, 60
Projection-valued measure, 125 Walsh system, 37
Pythagorean inequality, 20 Weak
boundedness, 77
Quotient space, 4, 6, 12, 15 closedness, 90
convergence, 86
Rademacher system, 37 topology, 89
Radon-Nikodym theorem, 43 Weak
Reflexive spaces, 51 convergence, 90
Regular point, 101 topology, 91
Resolvent
identity, 104 Young’s inequality, 22
operator, 103
Zorn’s lemma, 3
set, 101
Riesz representation theorem, 42

Schauder basis, 79
Schauder system, 80
Schur property, 89
Self-adjoint operator, 110
Semi-norm, 14
Separable space, 35