12.

Inner Product Spaces

12.1. Vector spaces

A real vector space is a set of objects that you can do two things with: you can add two of them
together to get another such object, and you can multiply one of them by any real number to get
another such object. There’s a set of axioms that a vector space must satisfy; you can find these
in other textbooks. Similarly, a complex vector space is a set of objects that you can do two things
with: you can add two of them together to get another such object, and you can multiply one of
them by any complex number to get another such object. If you are speaking about a real vector
space, you call any real number a scalar. If you are speaking about a complex vector space, you
call any complex number a scalar. If you have a (finite) list of vectors v1 , v2 , v3 , . . . , vn , the most
general way that you can combine these vectors together to get another vector is to choose a list
of scalars α1 , α2 , α3 , . . . , αn and form the linear combination
α1 v1 + α2 v2 + α3 v3 + · · · + αn vn
Examples of real vector spaces include RN , where the vectors are N -tuples of real numbers
(most familiarly, R2 – the plane – which is the set of ordered pairs of real numbers and R3 – three-
dimensional space – which is the set of ordered triples of real numbers). The basic operations look
like this: if x = (x0 , x1 , . . . , xN −1 ), y = (y0 , y1 , . . . , yN −1 ), and α is any real number, then
x + y = (x0 + y0 , x1 + y1 , . . . , xN −1 + yN −1 ) and αx = (αx0 , αx1 , . . . , αxN −1 ).
The zero vector is (0, 0, . . . , 0).Other examples include many different varieties of vector spaces
whose members are real valued functions on a certain domain. When you deal with such function
spaces, you have to be careful of a few things. One is that each function must have the same
domain – each function must be defined “everywhere” (actually, for spaces where membership is
determined by some kind of integrability condition, “almost everywhere” is good enough). As an
example, let’s consider a set of real-valued functions suitable for feeding into the Laplace transform:
the set of “locally integrable” functions on the interval [0, ∞) that “don’t grow too fast at ∞ ”.
Is f (t) = ln t a member of this set? Yes, although its domain isn’t p quite all of [0, ∞). It blows up
at 0, but not so badly as to spoil its integrability there. Is f (t) = 1 − t2 a member of this set?
No – because we need to know what it does between 1 and ∞. As such, it would be silly to ask
what the Laplace transform of this function is. You also must make sure that your set of functions
really is a vector space. This primarily means that you have to check that the sum of any two
such functions is also such a function, and that a constant times any such function is still such a
function. Examples of function spaces include the following:
Description Symbol
Continuous functions on [a, b] C[a, b]
Periodic continuous functions C(TP )
Continuous functions on the whole real line C(R)
Functions with one continuous derivative on a set S (S can be [a, b], TP , R, etc.) C 1 (S)
Bounded (actually, “essentially bounded”), functions on a set S L∞ (S)
Absolutely integrable functions on S L1 (S)
Square integrable functions on S L2 (S)

Examples of complex vector spaces include CN , the set of N -tuples of complex numbers, which
is just like RN , except that both the entries (components) of the vectors and the scalars are allowed

82
to be complex numbers, and various spaces of complex valued functions. Be sure to understand
that we are talking about having the values of the function being complex – we are not assuming
that these are functions of a complex variable. As an example, einx is a complex-valued function
of the real variable x. Our list of complex function spaces that we are likely to encounter is exactly
the same as the list above, and when we say exactly, we mean that we are using the exact same
symbols to name these spaces. When we use a symbol like C(TP ), we are not committing ourselves
as to whether we mean the real-valued functions or the complex-valued functions – we either have
to make that clear in the surrounding context, or else admit that in the particular case we have in
mind it doesn’t much matter which one we mean.
The study of vector spaces in general falls under the label linear algebra. In that study, sets
of vectors (that is, sets of elements of some vector space) are probed as to what their span is and
as to whether or not they are linearly independent (independent for short). For further details, see
any reasonably-written textbook on the subject. Here are two definitions out of all of that study:
if you have a function (or mapping, or map, etc. – the same idea has many names) T that has as
its domain – the things that you can feed this mapping – all of one vector space, and that has as its
output elements of some other (well, possibly other – no rule says it can’t be the same one) vector
space, then T is a linear transformation (or linear mapping or linear whatzit or just plain linear) if:

T (αu + βv) = αT (u) + βT (v) for all vectors u and v and all scalars α and β. (12.1)

A linear functional is something whose domain is some vector space – which takes vectors as
input – that gives scalars (real numbers if it is a real vector space and complex numbers if it is a
complex vector space) as output, and that satisfies equation (12.1).
By now you should be familiar with the principle that “∧” – the thing that takes in functions
and gives back Fourier coefficients – is, by any reasonable definition, linear.

12.2. Real inner product spaces

The vector space RN comes equipped with a very special geometrically-inspired algebraic op-
eration called the dot product. The dot product takes in two vectors and gives you back a real
number – a scalar. Synonyms for the words dot product include inner product and scalar product.
We will write the dot product of the two vectors u and v in at least two different notations: as
u · v or as hu, vi. The dot product on RN is defined as follows: if x = (x0 , x1 , . . . , xN −1 ) and
y = (y0 , y1 , . . . , yN −1 ), then
N
X −1
x · y = hx, yi = x0 y0 + x1 y1 + · · · + xN −1 yN −1 = xk yk (12.2)
k=0

The geometric description of the dot product is the following: if θ is the angle between x and
y, then  
−1 x·y
θ = cos (12.3)
kxk kyk
q √
where kxk = kxk2 = x20 + x21 + · · · + x2N −1 = x · x (12.4)
is the length or norm of the vector x. If the dot product of two vectors is zero, then the angle
between them is a right angle and we call the two vectors orthogonal. We also notice in (12.4) that
computations of lengths can use the dot product, as there seems to be a close relationship. (There

83
is also the familiar idea that in any problem involving lengths and the Pythagorean Theorem, it is
frequently easier to work with the square of a distance than it is to work with the distance itself.)
What if we have a real vector space that is not RN – such as one of the function spaces? We
may, under some circumstances, be able to define something called an inner product on that vector
space. An inner product on a real vector space is something that has all of the vital properties
of the dot product on RN – if we can figure out what those properties are. The accepted line of
jargon for a real inner product is that it is a positive-definite symmetric bilinear form. What does
this mean? It means that it is an object that takes as input two elements of the vector space and
gives back a real number - for vectors u and v, let us write the inner product as hu, vi. The other
words in this description have the following meanings:

Symmetric:
hu, vi = hv, ui for all vectors u and v (12.5)
Bilinear:
hαu + βv, wi = αhu, wi + βhv, wi and
hu, αv + βwi = αhu, vi + βhu, wi for all u, v, w, α and β (12.6)
Positive definite:
hu, ui ≥ 0 always and hu, ui > 0 for u 6= 0 (12.7)

The best example that we will have of something thatZsatisfies all of these properties is to take
a space whose elements are functions and to let hf, gi = f g, or something very much like that.
For instance, for functions periodic of period P, let us define the standard real inner product of
f and g as:
1 P
Z
hf, gi = f (x)g(x) dx (12.8)
P 0
It is not hard to show that this satisfies properties (12.5), (12.6), and (12.7) - except possibly for
a little fudging on the second part of (12.7), but let’s not worry too much about that now.
Any real vector space on which a real inner product has been defined is a real inner product
space.

12.3. Complex inner product spaces

We’d like to do this for complex vector spaces - but we realize that there are going to have to
be some subtle modifications of the detail. In particular, if we want something positive definite –
something like (12.7) – we are going to have to use something with a lot of complex conjugates and
absolute values in it. Conventional wisdom has settled on just what we need, and it is the following:
a complex inner product is something that takes as input two vectors from a complex vector space
and gives as output a complex number, and it is a positive-definite Hermitian sesquilinear form.
Actually, “sesquilinear” is a part of “Hermitian”, but I threw it in to make the phrase sound more
impressive. “Sesquilinear” means linear in one factor and “conjugate-linear” in the other factor -
that’s something like linear, but with some stray complex conjugates hanging around. It makes
no earthly difference which factor is which, but we have to come to some choice and stick to it.
As accident would have it, mathematicians have fallen into the rut of always putting the complex
conjugate on the second factor, while physicists have fallen into the rut of putting the complex
conjugate on the first factor. In what follows, we will follow the mathematician’s convention - we
suppose that a physicist will just have to read it in a mirror. Here’s what these words mean:

84
Hermitian:
hu, vi = hu, vi for all vectors u and v (12.9)
Sesquilinear:
hαu + βv, wi = αhu, wi + βhv, wi and
hu, αv + βwi = αhu, vi + βhu, wi for all u, v, w, α and β (12.10)
Positive definite:
hu, ui ≥ 0 always and hu, ui > 0 for u 6= 0 (12.11)
N
We will give two examples: the standard inner product on C is defined as follows: if z =
(z0 , z1 , . . . , zN −1 ) and w = (w0 , w1 , . . . , wN −1 ), then
N
X −1
hz, wi = z0 w0 + z1 w1 + · · · + zN −1 wN −1 = z k wk (12.12)
k=0
Z
On a function space the inner product of f and g will be hf, gi = f g, or to make it specific
to functions periodic of period P, the standard complex inner product of f and g is:

1 P
Z
hf, gi = f (x)g(x) dx (12.13)
P 0
Any complex vector space on which a complex inner product as been defined is called a complex
inner product space.

12.4. The real Cauchy-Buniakovski-Schwarz inequality

Theorem – the Cauchy-Schwarz inequality: in any real inner product space, for any two
vectors u and v,
hu, vi2 ≤ hu, uihv, vi (12.14)
with equality holding if and only if one of these vectors is a scalar multiple of the other.

The proof uses nothing but the properties of an inner product – that is, (12.5), (12.6), and
(12.7) – and the technique of completing the square. If v = 0, there is nothing to prove, both sides
of the inequality being zero, so we assume that v 6= 0. Consider the vector u + λv for any real
number λ. By property (12.7), the inner product of this vector with itself is always greater or equal
to zero (with equality only if it is the zero vector). Thus:

0 ≤ hu − λv, u − λvi

= hu, ui − hu, λvi − hλv, ui + hλv, λvi by (12.6)

= hu, ui − 2λhu, vi + λ2 hv, vi by (12.5) and (12.6)
Since we know that hv, vi is positive (by (12.7) again), we can divide both sides of this inequality
by it, getting:
2hu, vi hu, ui
λ2 − λ+ ≥0
hv, vi hv, vi
2hu, vi hu, ui
λ2 − λ≥−
hv, vi hv, vi

85
Now complete the square – that is, use formula (11.1) from the previous chapter:

hu, vi 2 hu, vi 2 hu, ui

   
2 2hu, vi
λ − λ+ ≥ −
hv, vi hv, vi hv, vi hv, vi

hu, vi 2 hu, vi2 − hu, uihv, vi

 
λ− ≥
hv, vi hv, vi2
This is true for all real λ. If we let λ be that value which minimizes the left hand side – that is,
hu, vi
if we let λ be , then the left hand side is zero and the right hand side must be less than or
hv, vi
equal to zero. Hence,
hu, vi2 − hu, uihv, vi
0≥
hv, vi2
or hu, vi2 ≤ hu, uihv, vi

As it turns out, there is another notation in which this fact is usually expressed. From property
(12.7), the inner product of a vector with itself is positive and hence has a real, positive square
root. We call this square root the norm of the vector - by analogy to (12.4). That is, in any real
inner product space we define the norm of a vector u to be
p
kuk = hu, ui (12.15)

Using this terminology, we take the square root of both sides of the inequality (12.14) to get:

The Cauchy-Schwarz inequality (norm version):

|hu, vi| ≤ kuk kvk (12.16)

hu, vi
One side effect of this is that we can form the fraction and be assured that it is between
kuk kvk
−1 and 1. Hence it has an arccosine, and we call that arccosine the angle between the two vectors.

12.5. The complex Cauchy-Buniakovski-Schwarz inequality

Theorem – the Cauchy-Schwarz inequality: in any real inner product space, for any two
vectors u and v,
|hu, vi|2 ≤ hu, uihv, vi (12.17)
with equality holding if and only if one of these vectors is a scalar multiple of the other.

What’s the difference between this and equation (12.14)? Two things: the fact that the vectors
involved belong to a complex rather than a real vector space, and the way that we had to make it
the square of the absolute value of something on the left hand side – we can’t put just the square
of a complex number into an inequality and have it be meaningful.
The proof uses nothing but the properties of an inner product – that is, (12.9), (12.10), and
(12.11) – and the technique of completing the square for a complex variable. If v = 0, there is
nothing to prove, both sides of the inequality being zero, so we assume that v 6= 0. Consider the
vector u + λv for any complex number λ. By property (12.1), the inner product of this vector with
itself is always greater or equal to zero (with equality only if it is the zero vector). Thus:

0 ≤ hu − λv, u − λvi

86
 = hu, ui − hu, λvi − hλv, ui + hλv, λvi by (12.6)
= hu, ui − λhu, vi − λhv, ui + λλhv, vi

= hu, ui − λhu, vi − λhu, vi + λλhv, vi by (12.9) and (12.10)

Since we know that hv, vi is positive (by (12.11) again), we can divide both sides of this inequality
by it, getting:
hu, vi hu, vi hu, ui
|λ|2 − λ −λ + ≥0
hv, vi hv, vi hv, vi
Next, employ equation (11.7) of the “Completing the Square” chapter:

hu, vi hu, vi hu, vi hu, vi hu, vi hu, vi hu, ui

|λ|2 − λ −λ + − + ≥0
hv, vi hv, vi hv, vi hv, vi hv, v hv, v hv, vi
! 
 hu, vi 2 hu, ui
  
hu, v hu, v
λ− λ− −
  + ≥0
hv, vi hv, vi hv, vi  hv, vi
 2 2
λ − hu, vi  + hu, uihv, vi − |hu, vi| ≥ 0
 
 hv, vi  hv, vi2
This is true for all possible values of λ, especially including that value which minimizes the left
hu, vi
hand side – namely, λ = . If we choose that value of λ, then the numerator of the second
hv, vi
fraction must be greater or equal to zero - that is, (12.17) must be true.
We now repeat the way we finished off the previous section: we let the norm of the vector be
the square root of its inner product with itself – that is, in a complex inner product space,
p
kuk = hu, ui (12.18)

Given this convention, we take the square root of both sides of (12.17) to get

The Cauchy-Schwarz inequality (norm version, complex vector space):

|hu, vi| ≤ kuk kvk (12.19)

12.6. Orthonormal sets in a real inner product space

The driving principle of the vast majority of mathematical work in inner product spaces is
orthogonality. The definition of orthogonal is the same in both real and complex inner product
spaces: two vectors are orthogonal if and only if their inner product is zero. With that in mind,
we define the concepts of an orthogonal set and an orthonormal set of vectors:

Definition: Let {ej } be a set of vectors in a (real or complex) inner product space. The variable
j – the index to this list of vectors – runs through some set of possible values. We are at the
moment being deliberately vague as to whether that index set is finite or infinite. Then {ej } is an
orthogonal set if and only if: (
= 0 if j 6= k
hej , ek i is (12.20)
> 0 if j = k

87
The same set is an orthonormal set if and only if:
(
0 if j 6= k
hej , ek i = (12.21)
1 if j = k

An orthonormal set is clearly a special case of an orthogonal set. On the other hand, any
orthogonal set may be turned into an orthonormal set by merely dividing each element by its own
norm. Note also that, although the zero vector is always orthogonal to everything, we don’t want
it as a member of anything that we are willing to call an orthogonal set. Our central problem is
this: suppose we have a finite orthonormal set {ej } (now we are specifying that the set of indices
– the set of possible values for j – be a finite set, although we are willing to let it be a very large
finite set.) Let v be any vector in the inner product space. How closely can we approximate v by
a linear combination of the elements of the orthonormal set? More specifically, how can we choose
the coefficients {αj } so that
X
αj ej is as close as possible to v?
j

But what do we mean by “as close as possible”? Surely we mean that the size of the difference is
as small as possible. But what do we mean by “size”? Well, let’s see – every inner product space,
real or complex, has a built-in notion of size: the norm, as defined in (12.15) and (12.18). That is,
our problem is to find αj so that


X
v − αj ej
is minimized.
j

To minimize this, it is sufficient to minimize its square. This isn’t a new idea, of course – it is
used in nearly every calculus problem that asks that a distance be maximized or minimized. The
Pythagorean theorem just makes working with the squares of distances easier than working with
the distances themselves. So here we go:
2 * +

X X X
v −
= v−
αj ej αj ej , v − αk ej (12.22)
j j k

We used two different names – j and k – for the index variable because we had to. If you multiply
a sum of, for instance, seven terms by itself, the result will be a sum with 49 terms – you’ve got to
take all possible products of any of the terms with any of the other terms. So far, we are assuming
that we are working in a real inner product space.Use (12.5) and (12.6) to simplify (12.22):
2
X X X X

v − αj ej
= hv, vi − hv, αj ej i − hαj ej , vi + hαj ej , αk ek i

j j j j,k
X X
= hv, vi − 2 αj hv, ej i + αj αk hej , ek i
j j,k
X X
= hv, vi − 2 αj hv, ej i + αj2 by 12.21)
j j

88
 X
αj2 − 2αj hv, ej i + kvk2


=
j

We complete the square, using formula (11.1):

X
αj2 − 2αj hv, ej i + hv, ej i2 − hv, ej i2 + kvk2


=
j
X X
= (αj − hv, ej i)2 − hv, ej i2 + kvk2 (12.23)
j j

We’re trying to minimize this quantity, and we get to choose the αj any way we want to. It
is clear that the first summation on the right hand side of (12.23), being the sum of squares, is
always greater than or equal to zero – but we can make it zero if we just choose the αj ’s right.
The right choice is to let αj be equal to hv, ej i. These values for the coefficients are called (for
reasons which will eventually become clear) the generalized Fourier coefficients for v with respect
to this orthonormal set. Since the left hand side of (12.22) is the square of a norm, it must always
be greater than or equal to zero. If we let αj be equal to hv, ej i for each j, the remaining portion
of the right hand side of (12.23) must be nonnegative. That is, we have the following inequality,
generally known as Bessel’s inequality:
X
hv, ej i2 ≤ kvk2 (12.24)
j

If we are able to write v as a linear combination of the ej’s, then the minimum value of the left
hand side of (12.22) would be zero and the inequality in (12.24) would actually be an equality.
What if the orthonormal set is actually an infinite set rather than a finite set? Then such sums
as appear in (12.22) and (12.24) would have to be interpreted as infinite series. Fortunately, the
very workings of this problem – notably Bessel’s inequality – help to assure us that these series
converge in some appropriate sense. (Actually, to get everything that we might ask for in terms
of these series being meaningful, we’ll have to have our inner product space be a complete metric
space, which makes it a Hilbert space – unfortunately, this is not the same meaning of the word
complete as we are about to use below.) Bessel’s inequality will always be true, and the norm of
the difference between v and our linear combination of the ej’s will always be minimized if we
choose our coefficients to be the generalized Fourier coefficients. A new issue now arises: does our
orthonormal set have enough elements in it that we can write any vector v as the limit of linear
combinations of – that is, as an infinite series based on – that set? If so, we call the orthonormal set
complete. (This is also known as having an orthonormal basis.) It happens – and the calculations
above provide the framework for this argument, too – that an orthonormal set is complete if and
only if the only vector orthogonal to every element in it is the zero vector.
Let’s summarize our findings:

Fourier coefficients:


X
v −
is minimized if each αj = hv, ej i
αj ej (12.25)
j

Bessel’s inequality: X
|hv, ej i|2 ≤ kvk2 (12.26)
j

89
If the orthonormal set is also complete, then:

Generalized Fourier series: X

v= hv, ej iej (12.27)
k
Generalized Parseval’s identity:
X
|hv, ej i|2 = kvk2 (12.28)
j

12.7. Orthonormal sets in a complex inner product space

Now we suppose that {ej } is a (finite, for the time being) orthonormal set in a complex inner
product space. Without further ado, let’s repeat what was in the last section:
2 * +
X X X

v −
= v−
αj ej αj ej , v − αk ej
j j k
X X X
= hv, vi − hv, αj ej i − hαj ej , vi − hαj ej , αk ek i
j j j,k
X X X
= hv, vi − αj hv, ej i − αj hv, ej i + αj αk hej , ek i
j j j,k
X X X
= hv, vi − αj hv, ej i − αj hv, ej i + |αj |2
j j j
X 
= |αj |2 − αj hv, ej i − αj hv, ej i + kvk2
j

We complete the square, using formula (11.7):

X 
= |αj |2 − αj hv, ej i − αj hv, ej i + |hv, ej i|2 − |hv, ej i|2 + kvk2
j
X X
= |αj − hv, ej i|2 − |hv, ej i|2 + kvk2 (12.29)
j j

Now we are ready to draw conclusions from this, exactly as before.

Formulas (12.25), (12.26), (12.27), and (12.28) are also valid without change in a complex inner
product space.

Of course, we were clever enough to include some well-chosen absolute values in these statements!

12.8. Square integrable functions on TP

Consider a function f (real or complex valued) that is defined on the line so as to be periodic
of period P. We call such a function square integrable, and say that it belongs to L2 (TP ),provided
the following integral converges:
1 P
Z
|f (x)|2 dx < ∞
P 0

90
 Just to give you a taste of this condition – it doesn’t require that the function be bounded, but
it is harder to satisfy than mere integrability, or even absolute integrability.As an example,
 take
1 P P P
the function f (x) = p for 0 < |x| ≤ . This function is integrable on − , (and hence
|x| 2 2 2
1
absolutely integrable since it is positive), but if we square it, we get , which is not integrable.
|x|
We say that this function belongs to L1 but not to L2 .
We now define the inner product. Please recognize that our decision to divide out front by P
represents one of several possible notational choices, and may not necessarily be reflected in other
works.

The inner product for real-valued functions:

Z w+P
2 1
If f, g ∈ L (TP ), then hf, gi = f (x)g(x) dx (12.30)
P w

The inner product for complex-valued functions:

Z w+P
1
If f, g ∈ L2 (TP ), then hf, gi = f (x)g(x) dx (12.31)
P w

In either case, the norm is as follows:

 Z w+P  21
1 2
kf k2 = hf, f i = |f (x)| dx (12.32)
P w

The Cauchy-Schwarz inequality in this case reads as follows:

 Z w+P   Z w+P  12  Z w+P  21
1 1 1
|f (x)|2 dx |g(x)|2 dx


P f (x)g(x) dx ≤ = kf k2 kgk2 (12.33)
w P w P w

A quick corollary of this is that

Z w+P  Z w+P  12  Z w+P  21
1 1 2 1
kf k1 = |f (x)| dx ≤ |f (x)| dx 1 dx ≤ kf k2 (12.34)
P w P w P w

12.9. Fourier series and orthogonal expansions

             
2πx 2πx 2 · 2πx 2 · 2πx 3 · 2πx 3 · 2πx
1, cos , sin , cos , sin , cos , sin ,...
P P P P P P
is an orthogonal set in the real inner product space L2 (TP ), and almost – but not quite – an
orthonormal set. The specific problem with being orthonormal is the normalization: the function
1
1 has norm equal to 1, but all of the other functions in this set have norm equal to √ . We could
2
chase through all of the consequences of that factor, but rather than give you the details, we’ll just
give the results.
If we want to approximate a square integrable function f by an N th degree trigonometric
polynomial, the the closest we can come in the L2 norm – the least squares or least root mean square
approximation – is to let the coefficients of this polynomial be exactly the Fourier coefficients. That

91
is, we let this trigonometric polynomial be the N th partial sum of the Fourier series for f. One
consequence of this is – by (12.26) – Bessel’s inequality:
N
a[0]2 1 X
a[k]2 + b[k]2 ≤ kf k22


+ (12.35)
4 2
k=1

which can also be written as

N w+P
a[0]2 X
Z
2
a[k]2 + b[k]2 ≤ |f (x)|2 dx


+ (12.36)
2 P w
k=1

But then, it turns out that this orthogonal set is complete. We won’t prove that, but basically,
our previous convergence theorems for Fourier series make this inevitable. That being the case, we
can say that the Fourier series of any square integrable function always converges to that function
in the L2 sense. Furthermore, by (12.28), we have Parseval’s identity:
∞
a[0]2 1 X
a[k]2 + b[k]2 = kf k22


+ (12.37)
4 2
k=1

∞ w+P
a[0]2 X
Z
2
a[k]2 + b[k]2 = |f (x)|2 dx


+ (12.38)
2 P w
k=1

Let’s try to repeat this for the complex case. This time, consider the set {e2πikx/P }∞ k=−∞ of
complex valued functions on TP . This turns out to be an orthonormal set – in fact, demonstrating
that fact is far easier for this case than for the real case. This is also a complete orthonormal set
– we couldn’t possibly have a different result for the complex case than for the real case – so we
have both a Bessel’s inequality and a Parseval’s identity for this case, too.

Bessel’s inequality:
N
X
|fb[k]|2 ≤ kf k22 (12.39)
k=−N

Parseval’s identity:
∞
X
|fb[k]|2 = kf k22 (12.40)
k=−∞

12.10. The trigonometric orthogonal set on PN

On the polygon PN , the set of functions ek [n] = e2πikn/N form an orthogonal set with respect
to the usual complex inner product. That is,
N −1 N −1
(
X X
2πijn/N −2πikn/N N if j = k
hej , ek i = ej [n]ek [n] = e e = (12.41)
n=0 n=0
0 otherwise

Equation (12.41) is just a restatement of equations (4.1) and (4.6). The factor of N that appears
in it means that the vectors are orthogonal but not quite orthonormal. Let’ s put this another way.
Suppose we have an N -dimensional complex inner product space with the standard inner product
with respect to that basis. (The set of all complex-valued functions on PN is just such a space.) We

92
can then write any vector as an N × 1 column matrix. Suppose we have a set of M such vectors.
Create an N × M matrix A whose M columns are these M vectors. If M = N we will have a
square matrix. How could we tell if these M vectors were independent? This would be a question
about the rank of the matrix A. If its rank is M, we have an independent set. In the M = N square
matrix case, the N vectors are independent if and only if the determinant of A is not zero. Now
how can we tell if the set of vectors is orthogonal ? To do this, let A∗ be the complex conjugate of
the transpose of A, and compute the matrix product A∗ A. This will be the M × M matrix whose
(j, k)th entry is precisely the inner product of the jth and kth vectors of our set. The statement
that the set is orthogonal is precisely the statement that A ∗ A is diagonal: that all of its entries
are zero except those on the main diagonal which are not zero. The statement that our set is
orthonormal is precisely the statement that A∗ A = I, the identity matrix. If A is a square matrix
such that A∗ A = I, we call A a unitary matrix.
Use this language to express (12.41). Let F be the N × N matrix whose columns are the vectors
ej named above. In other words, the (j, k)th entry of F is e2πijk/N = ζ jk . Then F is almost but
not quite a unitary matrix: F∗ F = N I.
So we have an orthogonal but not quite orthonormal set. What can we say about this in general?
Suppose that {ej } is a finite orthogonal (but not necessarily orthonormal) set in a complex inner
product space, and that
v is an arbitrary
vector in that space. We wish to find the coefficients αj
X

such that the norm v − α j ej
is minimized. This minimum is achieved if and only if

j

hv, ej i
αj = (12.42)
hej , ej i
Bessel’ s inequality in this case turns out to be
X |hv, ej i|2
≤ kvk2 (12.43)
hej , ej i
j

with equality (Parseval’ s identity) in the case of the orthogonal set being complete.
We may easily apply (12.42) and (12.43) to the case of the trigonometric basis for functions in
PN . The coefficients in (12.42) turn out to be:
N −1
hf, ej i 1 X
= f [n]e−2πijn/N = fb[j] (12.44)
hej , ej i N
n=0

The set of all N of these trigonometric functions is a set of N orthogonal, hence independent,
vectors. Therefore, they span the N -dimensional vector space of all functions on the polygon and
are a complete orthogonal set. Equality necessarily holds in (12.43). A careful working out of the
consequences of (12.43) leads us to Parseval’ s identity for this instance:
N
X −1 N
X −1
N kfbk2 = N |fb[k]|2 = |f [n]|2 = kf k22 (12.45)
k=0 n=0

12.11. Exercises

12.1. By imitating the derivations around equation (12.29), prove (12.42) and (12.43).

93
12.2. Which of the following Fourier series represent square-integrable functions on T2π , also known
as [−π, π]? What else can you say about the functions they represent?
∞ ∞
X sin kx X eikx
(a) =
k 2ik
k=1 k=−∞
k6=0
∞ ∞ p
X sin kx X |k| eikx
(b) √ =
k 2ik
k=1 k=−∞
k6=0
∞ ∞
X cos kx X eikx
(c) =
k 2|k|
k=1 k=−∞
k6=0
∞
X
(d) r|k| eikx , where 0 ≤ r < 1
k=−∞

Supplemental exercises:

12.3. Use the Fourier series of piecewise-polynomial functions on T and either Parseval’s identity
∞ ∞ ∞
X 1 X 1 X 1
or the synthesis equation at well-chosen points to calculate , , and . One
k2 k4 k6
k=1 k=1 k=1
possibility: Start with f1 (x) = 1−2x on the interval (0, 1), extended to be periodic of period 1.
0
Build a sequence of functions fn such that fn+1 (x) = fn (x). (That is, fn+1 is an antiderivative
Z 1
of fn .) Choose the constant of integration so that fn+1 [0] =
b fn+1 (x) dx = 0.
0

12.4. Find a way to automate the calculations in exercise 12.3, using a computer to help. Among
other things, you’ll need the ability to work with arbitrary precision rational numbers (hence
arbitrarily long integers). DERIVE, MAPLE, and MATHEMATICA have this capability.
∞
X 1
Calculate the exact value of . (Why 26? Because that was as far as Euler got, working
k 26
k=1
the problem by hand.)

94