i i “main”

2007/2/16
page 234

i i

CHAPTER

4
Vector Spaces
To criticize mathematics for its abstraction is to miss the point entirely. Abstraction
is what makes mathematics work. — Ian Stewart

The main aim of this text is to study linear mathematics. In Chapter 2 we studied
systems of linear equations, and the theory underlying the solution of a system of linear
equations can be considered as a special case of a general mathematical framework for
linear problems. To illustrate this framework, we discuss an example.
Consider the homogeneous linear system Ax = 0, where
 
1 −1 2
A =  2 −2 4  .
3 −3 6
It is straightforward to show that this system has solution set
S = {(r − 2s, r, s) : r, s ∈ R}.
Geometrically we can interpret each solution as defining the coordinates of a point in
space or, equivalently, as the geometric vector with components
v = (r − 2s, r, s).
Using the standard operations of vector addition and multiplication of a vector by a real
number, it follows that v can be written in the form
v = r(1, 1, 0) + s(−2, 0, 1).
We see that every solution to the given linear problem can be expressed as a linear
combination of the two basic solutions (see Figure 4.0.1):
v1 = (1, 1, 0) and v2 = (−2, 0, 1).

234

i i

i i
 i i “main”
2007/2/16
page 235

i i

4.1 Vectors in Rn 235

x3

v2
(2, 0, 1)

v
rv1 + sv2

x2

v1
(1, 1, 0)
x1

Figure 4.0.1: Two basic solutions to Ax = 0 and an example of an arbitrary solution to the
system.

We will observe a similar phenomenon in Chapter 6, when we establish that every

solution to the homogeneous second-order linear differential equation

y

+ a1 y
+ a2 y = 0

can be written in the form

y(x) = c1 y1 (x) + c2 y2 (x),

where y1 (x) and y2 (x) are two nonproportional solutions to the differential equation on
the interval of interest.
In each of these problems, we have a set of “vectors” V (in the first problem the
vectors are ordered triples of numbers, whereas in the second, they are functions that
are at least twice differentiable on an interval I ) and a linear vector equation. Further, in
both cases, all solutions to the given equation can be expressed as a linear combination
of two particular solutions.
In the next two chapters we develop this way of formulating linear problems in terms
of an abstract set of vectors, V , and a linear vector equation with solutions in V . We will
find that many problems fit into this framework and that the solutions to these problems
can be expressed as linear combinations of a certain number (not necessarily two) of basic
solutions. The importance of this result cannot be overemphasized. It reduces the search
for all solutions to a given problem to that of finding a finite number of solutions. As
specific applications, we will derive the theory underlying linear differential equations
and linear systems of differential equations as special cases of the general framework.
Before proceeding further, we give a word of encouragement to the more application-
oriented reader. It will probably seem at times that the ideas we are introducing are rather
esoteric and that the formalism is pure mathematical abstraction. However, in addition
to its inherent mathematical beauty, the formalism incorporates ideas that pervade many
areas of applied mathematics, particularly engineering mathematics and mathematical
physics, where the problems under investigation are very often linear in nature. Indeed,
the linear algebra introduced in the next two chapters should be considered an extremely
important addition to one’s mathematical repertoire, certainly on a par with the ideas of
elementary calculus.

4.1 Vectors in Rn
In this section, we use some familiar ideas about geometric vectors to motivate the more
general and abstract idea of a vector space, which will be introduced in the next sec-
tion. We begin by recalling that a geometric vector can be considered mathematically
as a directed line segment (or arrow) that has both a magnitude (length) and a direction
attached to it. In calculus courses, we define vector addition according to the parallel-
ogram law (see Figure 4.1.1); namely, the sum of the vectors x and y is the diagonal of

i i

i i
 i i “main”
2007/2/16
page 236

i i

236 CHAPTER 4 Vector Spaces

the parallelogram formed by x and y. We denote the sum by x + y. It can then be shown
geometrically that for all vectors x, y, z,

x+y = y+x (4.1.1)

xy and
y
x + (y + z) = (x + y) + z. (4.1.2)
x
These are the statements that the vector addition operation is commutative and associa-
Figure 4.1.1: Parallelogram law tive. The zero vector, denoted 0, is defined as the vector satisfying
of vector addition.
x + 0 = x, (4.1.3)

for all vectors x. We consider the zero vector as having zero magnitude and arbitrary
direction. Geometrically, we picture the zero vector as corresponding to a point in space.
Let −x denote the vector that has the same magnitude as x, but the opposite direction.
Then according to the parallelogram law of addition,

x + (−x) = 0. (4.1.4)

The vector −x is called the additive inverse of x. Properties (4.1.1)–(4.1.4) are the
fundamental properties of vector addition.
The basic algebra of vectors is completed when we also define the operation of
kx, k  0 multiplication of a vector by a real number. Geometrically, if x is a vector and k is
a real number, then kx is defined to be the vector whose magnitude is |k| times the
magnitude of x and whose direction is the same as x if k > 0, and opposite to x if
x
k < 0. (See Figure 4.1.2.) If k = 0, then kx = 0. This scalar multiplication operation
kx, k  0 has several important properties that we now list. Once more, each of these can be
established geometrically using only the foregoing definitions of vector addition and
Figure 4.1.2: Scalar
scalar multiplication.
multiplication of x by k.
For all vectors x and y, and all real numbers r, s and t,

1x = x, (4.1.5)
(st)x = s(tx), (4.1.6)
r(x + y) = rx + ry, (4.1.7)
(s + t)x = sx + tx. (4.1.8)

It is important to realize that, in the foregoing development, we have not defined a

“multiplication of vectors.” In Chapter 3 we discussed the idea of a dot product and cross
product of two vectors in space (see Equations (3.1.4) and (3.1.5)), but for the purposes
of discussing abstract vector spaces we will essentially ignore the dot product and cross
product. We will revisit the dot product in Section 4.11, when we develop inner product
spaces.
We will see in the next section how the concept of a vector space arises as a direct
generalization of the ideas associated with geometric vectors. Before performing this
abstraction, we want to recall some further features of geometric vectors and give one
specific and important extension.
We begin by considering vectors in the plane. Recall that R2 denotes the set of all
ordered pairs of real numbers; thus,

R2 = {(x, y) : x ∈ R, y ∈ R}.

The elements of this set are called vectors in R2 , and we use the usual vector notation
to denote these elements. Geometrically we identify the vector v = (x, y) in R2 with

i i

i i
 i i “main”
2007/2/16
page 237

i i

4.1 Vectors in Rn 237

y

the geometric vector v directed from the origin of a Cartesian coordinate system to
(x, y) the point with coordinates (x, y). This identification is illustrated in Figure 4.1.3. The
(0, y) numbers x and y are called the components of the geometric vector v. The geometric
v
vector addition and scalar multiplication operations are consistent with the addition and
x scalar multiplication operations defined in Chapter 2 via the correspondence with row
(x, 0)
(or column) vectors for R2 :
Figure 4.1.3: Identifying If v = (x1 , y1 ) and w = (x2 , y2 ), and k is an arbitrary real number, then
vectors in R2 with geometric
vectors in the plane.
v + w = (x1 , y1 ) + (x2 , y2 ) = (x1 + x2 , y1 + y2 ), (4.1.9)
kv = k(x1 , y1 ) = (kx1 , ky1 ). (4.1.10)

These are the algebraic statements of the parallelogram law of vector addition and
the scalar multiplication law, respectively. (See Figure 4.1.4.) Using the parallelogram
law of vector addition and Equations (4.1.9) and (4.1.10), it follows that any vector
v = (x, y) can be written as

v = xi + yj = x(1, 0) + y(0, 1),

where i = (1, 0) and j = (0, 1) are the unit vectors pointing along the positive x- and
y-coordinate axes, respectively.

y
(x1  x2, y1  y2)

(x2, y2)

vw
w
kv (kx1, ky1)
(x1, y1)
v
x

Figure 4.1.4: Vector addition and scalar multiplication in R2 .

The properties (4.1.1)–(4.1.8) are now easily verified for vectors in R2 . In particular,
the zero vector in R2 is the vector

0 = (0, 0).

Furthermore, Equation (4.1.9) implies that

(x, y) + (−x, −y) = (0, 0) = 0,

so that the additive inverse of the general vector v = (x, y) is −v = (−x, −y).
It is straightforward to extend these ideas to vectors in 3-space. We recall that

R3 = {(x, y, z) : x ∈ R, y ∈ R, z ∈ R}.

As illustrated in Figure 4.1.5, each vector v = (x, y, z) in R3 can be identified with the
geometric vector v that joins the origin of a Cartesian coordinate system to the point
with coordinates (x, y, z). We call x, y, and z the components of v.

i i

i i
 i i “main”
2007/2/16
page 238

i i

238 CHAPTER 4 Vector Spaces

(0, 0, z)

(x, y, z)

v
(0, y, 0)
y

(x, 0, 0) (x, y, 0)
x

Figure 4.1.5: Identifying vectors in R3 with geometric vectors in space.

Recall that if v = (x1 , y1 , z1 ), w = (x2 , y2 , z2 ), and k is an arbitrary real number,
then addition and scalar multiplication were given in Chapter 2 by

v + w = (x1 , y1 , z1 ) + (x2 , y2 , z2 ) = (x1 + x2 , y1 + y2 , z1 + z2 ), (4.1.11)

kv = k(x1 , y1 , z1 ) = (kx1 , ky1 , kz1 ). (4.1.12)

Once more, these are, respectively, the component forms of the laws of vector
addition and scalar multiplication for geometric vectors. It follows that an arbitrary
vector v = (x, y, z) can be written as

v = xi + yj + zk = x(1, 0, 0) + y(0, 1, 0) + z(0, 0, 1),

where i = (1, 0, 0), j = (0, 1, 0), and k = (0, 0, 1) denote the unit vectors which point
along the positive x-, y-, and z-coordinate axes, respectively.
We leave it as an exercise to check that the properties (4.1.1)–(4.1.8) are satisfied
by vectors in R3 , where
0 = (0, 0, 0),
and the additive inverse of v = (x, y, z) is −v = (−x, −y, −z).
We now come to our first major abstraction. Whereas the sets R2 and R3 and their
associated algebraic operations arise naturally from our experience with Cartesian ge-
ometry, the motivation behind the algebraic operations in Rn for larger values of n does
not come from geometry. Rather, we can view the addition and scalar multiplication
operations in Rn for n > 3 as the natural extension of the component forms of addition
and scalar multiplication in R2 and R3 in (4.1.9)–(4.1.12). Therefore, in Rn we have that
if v = (x1 , x2 , . . . , xn ), w = (y1 , y2 , . . . , yn ), and k is an arbitrary real number, then

v + w = (x1 + y1 , x2 + y2 , . . . , xn + yn ), (4.1.13)
kv = (kx1 , kx2 , . . . , kxn ). (4.1.14)

Again, these definitions are direct generalizations of the algebraic operations defined
in R2 and R3 , but there is no geometric analogy when n > 3. It is easily established that
these operations satisfy properties (4.1.1)–(4.1.8), where the zero vector in Rn is

0 = (0, 0, . . . , 0),

and the additive inverse of the vector v = (x1 , x2 , . . . , xn ) is

−v = (−x1 , −x2 , . . . , −xn ).

The verification of this is left as an exercise.

i i

i i
 i i “main”
2007/2/16
page 239

i i

4.1 Vectors in Rn 239

Example 4.1.1 If v = (1.2, 3.5, 2, 0) and w = (12.23, 19.65, 23.22, 9.76), then

v + w = (1.2, 3.5, 2, 0) + (12.23, 19.65, 23.22, 9.76) = (13.43, 23.15, 25.22, 9.76)

and

2.35v = (2.82, 8.225, 4.7, 0).

Exercises for 4.1

Key Terms 6. If s and t are scalars and x and y are vectors in Rn ,
then (s + t)(x + y) = sx + ty.
Vectors in Rn , Vector addition, Scalar multiplication, Zero
vector, Additive inverse, Components of a vector. 7. For every vector x in Rn , the vector 0x is the zero
vector of Rn .
Skills
8. The parallelogram whose sides are determined by vec-
tors x and y in R2 have diagonals determined by the
• Be able to perform vector addition and scalar multi-
vectors x + y and x − y.
plication for vectors in Rn given in component form.
9. If x is a vector in the first quadrant of R2 , then any
• Understand the geometric perspective on vector addi- scalar multiple kx of x is still a vector in the first quad-
tion and scalar multiplication in the cases of R2 and rant of R2 .
R3 . √
√ 5i − 6j + 2k in R is the same as
10. The vector 3
• Be able to formally verify the axioms (4.1.1)–(4.1.8) (5, −6, 2).
for vectors in Rn .
11. Three vectors x, y, and z in R3 always determine a
True-False Review 3-dimensional solid region in R3 .

For Questions 1–12, decide if the given statement is true or 12. If x and y are vectors in R2 whose components are even
false, and give a brief justification for your answer. If true, integers and k is a scalar, then x + y and kx are also
you can quote a relevant definition or theorem from the text. vectors in R2 whose components are even integers.
If false, provide an example, illustration, or brief explanation
of why the statement is false. Problems
1. If x = (3, 1), y = (−1, 2), determine the vectors
1. The vector (x, y) in R2 is the same as the vector v1 = 2x, v2 = 3y, v3 = 2x + 3y. Sketch the cor-
(x, y, 0) in R3 . responding points in the xy-plane and the equivalent
geometric vectors.
2. Each vector (x, y, z) in R3 has exactly one additive
inverse. 2. If x = (−1, −4) and y = (−5, 1), determine the vec-
tors v1 = 3x, v2 = −4y, v3 = 3x + (−4)y. Sketch the
3. The solution set to a linear system of 4 equations and corresponding points in the xy-plane and the equiva-
6 unknowns consists of a collection of vectors in R6 . lent geometric vectors.
4. For every vector (x1 , x2 , . . . , xn ) in Rn , the vector 3. If x = (3, −1, 2, 5), y = (−1, 2, 9, −2), determine
(−1) · (x1 , x2 , . . . , xn ) is an additive inverse. v = 5x + (−7)y and its additive inverse.

5. A vector whose components are all positive is called 4. If x = (1, 2, 3, 4, 5) and z = (−1, 0, −4, 1, 2), find y
a “positive vector.” in R5 such that 2x + (−3)y = −z.

i i

i i
 i i “main”
2007/2/16
page 240

i i

240 CHAPTER 4 Vector Spaces

5. Verify the commutative law of addition for vectors in 8. Show with examples that if x is a vector in the first
R4 . quadrant of R2 (i.e., both coordinates of x are posi-
tive) and y is a vector in the third quadrant of R2 (i.e.,
6. Verify the associative law of addition for vectors in
both coordinates of y are negative), then the sum x + y
R4 .
could occur in any of the four quadrants.
7. Verify properties (4.1.5)–(4.1.8) for vectors in R3 .

4.2 Definition of a Vector Space

In the previous section, we showed how the set Rn of all ordered n-tuples of real num-
bers, together with the addition and scalar multiplication operations defined on it, has
the same algebraic properties as the familiar algebra of geometric vectors. We now push
this abstraction one step further and introduce the idea of a vector space. Such an ab-
straction will enable us to develop a mathematical framework for studying a broad class
of linear problems, such as systems of linear equations, linear differential equations, and
systems of linear differential equations, which have far-reaching applications in all areas
of applied mathematics, science, and engineering.

Let V be a nonempty set. For our purposes, it is useful to call the elements of V
vectors and use the usual vector notation u, v, . . . , to denote these elements. For example,
if V is the set of all 2 × 2 matrices, then the vectors in V are 2 × 2 matrices, whereas
if V is the set of all positive integers, then the vectors in V are positive integers. We
will be interested only in the case when the set V has an addition operation and a scalar
multiplication operation defined on its elements in the following senses:

Vector Addition: A rule for combining any two vectors in V . We will use the usual
+ sign to denote an addition operation, and the result of adding the vectors u and v will
be denoted u + v.

Real (or Complex) Scalar Multiplication: A rule for combining each vector in V
with any real (or complex) number. We will use the usual notation kv to denote the result
of scalar multiplying the vector v by the real (or complex) number k.

To combine the two types of scalar multiplication, we let F denote the set of scalars
for which the operation is defined. Thus, for us, F is either the set of all real numbers or
the set of all complex numbers. For example, if V is the set of all 2 × 2 matrices with
complex elements and F denotes the set of all complex numbers, then the usual operation
of matrix addition is an addition operation on V , and the usual method of multiplying
a matrix by a scalar is a scalar multiplication operation on V . Notice that the result of
applying either of these operations is always another vector (2 × 2 matrix) in V .

As a further example, let V be the set of positive integers, and let F be the set of all
real numbers. Then the usual operations of addition and multiplication within the real
numbers define addition and scalar multiplication operations on V . Note in this case,
however, that the scalar multiplication operation, in general, will not yield another vector
in V , since when we multiply a positive integer by a real number, the result is not, in
general, a positive integer.

We are now in a position to give a precise definition of a vector space.

i i

i i