Topic 3
Topic 3
95
3.1 Vector spaces
Vector spaces
The concepts of addition, scalar multiplication and linear
combinations of vectors can be defined in a very general context,
which turns out to capture a wide range of phenomena.
96
Definition (Vector space)
Fix a field of scalars F. Then a vector space over F is a non-empty
set V with two operations: addition and scalar multiplication. These
operations are required to satisfy the following 10 rules (axioms).
~ 2 V:
For all ~u , ~v , w
Addition behaves well:
A1 ~u + ~v 2 V . (closure of vector addition)
A2 (~u + ~v ) + w
~ = ~u + (~v + w
~ ). (associativity)
A3 ~u + ~v = ~v + ~u . (commutativity)
There must be a zero and inverses:
A4 There exists a vector ~0 2 V such that
~v + ~0 = ~v for all ~v 2 V . (existence of zero vector)
A5 For all ~v 2 V , there exists a vector ~v
such that ~v + ( ~v ) = ~0. (additive inverse)
97
Definition (Vector space continued)
For all ~u , ~v , 2 V and ↵, 2 F:
1. Rn
Our definition of vector spaces was based on algebraic properties of
Rn , so it is easy to check that Rn is a vector space with scalars R.
Definition
Denote by Mm,n (or Mm,n (R)) the set of all m ⇥ n matrices with real
entries.
Mm,n is a real vector space with the usual matrix addition and scalar
multiplication operations. (These are useful for studying digital
images.)
99
3. Vector space of polynomials
Definition
For a fixed integer n > 0, denote by Pn (or Pn (R)) the set of all
polynomials with degree at most n:
Pn = {a0 + a1 x + a2 x 2 + · · · + an x n | a0 , a1 , . . . , an 2 R}
Two polynomials are equal if and only if their coefficients are equal.
Definition
Let
F(S, R) = { all functions f : S ! R }
denote the set of all functions from S to R, where S is a non-empty
set.
Note: Here the zero vector is the function which is identically zero,
i.e., ~0 : S ! R such that ~0(x) = 0 for all x 2 S.
101
Example:
If f , g 2 F(R, R) are given by
f (x) = sin(x) and g (x) = x for all x 2 R,
then
(f + g )(x) = sin(x) + x and (3f )(x) = 3 sin(x) for all x 2 R.
10 10
5 5
-5 5 -5 5
-5 -5
-10 -10
F2 = {0, 1},
Addition Multiplication
0+0=0 0⇥0=0
1+0=1 1⇥0=0
0+1=1 0⇥1=0
1+1=0 1⇥1=1
Y = {s 2 S | s is red}.
106
Applications of general vector spaces
The vector space formalism gives us an object-oriented language to
model objects of interest, for instance, speaking about the
polynomials directly, but with mathematical precision, when we do
curve fitting.
Regarding matrices and functions as vector spaces allows geometrical
notions such as distance, angles and projection to be extended to
these objects. We have already seen how the Frobenius distance
between matrices is useful to decide on the quality of data
compression in the context of digital pictures.
A similar notion is needed when deciding the quality of a compressed
audio file.
Orthogonal projections are used to find best approximations in a
range of settings.
107
3.2 Linear combinations
Let V be a vector space with scalars F.
where each ↵i 2 F.
Example:
1 1 5 1
In R : 3
2 +2 = is a linear combination of
1 2 1 1
1
and .
2
108
Example: Computer graphics application
Linear combinations are used to generate colours in computer
graphics.
Three primary colours are represented as unit vectors in R3 .
2 3 2 3 2 3
1 0 0
Red := 405 Green := 415 Blue := 405
0 0 1
110
111
General method for studying linear combinations in Rn
w~ is a linear combination of ~v 1 , . . . , ~v k
,
the equation A~x = w~ has a solution ~x 2 Rk
,
the system with augmented matrix [~v 1 , . . . , ~v k | w
~ ] has a solution
Note: 2 3
x1
6 7
If ~x = 4 ... 5 is a solution, then we can write x1~v 1 + . . . + xk ~v k = w
~.
xk
112
Example:
In the real vector space P2 , determine whether ~q = 1 2x x 2 is a
linear combination of ~p 1 = 1 + x + x 2 and ~p 2 = 3 + x 2 .
Solution:
113
114
3.3 Linear transformations
115
Definition (Linear transformation)
Let V and W be vector spaces over the same field of scalars F.
A linear transformation from V to W is a function T : V ! W such
that for all ~u , ~v 2 V and all ↵ 2 F:
1. T (~u + ~v ) = T (~u ) + T (~v ) (T preserves vector addition)
Note:
1. Conditions 1 and 2 are equivalent to the condition that T
preserves linear combinations: i.e. for all ~v 1 , ~v 2 2 V and all
↵1 , ↵2 2 F:
117
2. D preserves scalar multiplication
118
Theorem (Any matrix gives a linear transformation)
Let A be an m ⇥ n matrix with real entries. Then matrix
multiplication by A gives a linear transformation TA : Rn ! Rm
defined by
TA (~v ) = A~v .
119
Example: Determinant
Show that the determinant function
det : M2,2 ! R
a b
7 ! ad bc
c d
is not a linear transformation.
Solution:
Find a specific counterexample
120
Example: Translation
Is the function T : R2 ! R2 given by
✓ ◆
x1 x +1
T = 1
x2 x2 + 1
a linear transformation?
Solution:
121
Example: Sampling functions
Let V be the vector space of all real valued functions f : [a, b] ! R.
Then sampling functions at n points a1 , a2 , . . . , an defines a linear
transformation
2 3
f (a1 )
6f (a2 )7
6 7
S : V ! Rn , f 7 ! S(f ) = 6 . 7
4 .. 5
f (an )
y
2 3
y=f(x) 1.25
6 1.5 7
6 7
6 2.5 7
6 7
S(f ) = 6
6 2.5 77
6 2.25 7
6 7
4 2.5 5
x 4
3 2 1 1 2 3
122
Prove that S is a linear transformation.
Solution: 10 10
5 5
-5 5 -5 5
-5 -5
-10 -10
123
Theorem (Linear transformations are given by matrices)
Every linear transformation T : Fn ! Fm is given by multiplication by
a matrix:
T (~v ) = [T ] ~v ,
where
[T ] = [T (~e 1 ) · · · T (~e n )]
and ~e i is the ith standard basis vector for Rn .
Note:
Knowing the e↵ect of T on the standard basis vectors ~e 1 , . . .~e n
completely determines [T ] and T .
124
Proof:
125
Composition of linear transformations
Recall:
If S : U ! V and R : V ! W are functions, then the composition
R S :U!W
Solution:
We have already
found the matrices of the shear and the reflection.
x
The image of under their composition is
y
127
3.4 Subspaces
Definition (Subspace)
A subspace W of a vector space V is a subset W ✓ V that is itself a
vector space using the addition and scalar multiplication operations
defined on V .
Example: ⇢
x
Is the line 2 R2 y = 2x + 1 a subspace of R2 ?
y
Solution:
129
94
94
94
Theorem (Null Space)
Let A be an m ⇥ n matrix with entries from a field F. Then the
solution set
S = {~x 2 Rn | A~x = ~0}
is a subspace of Fn . This is called the null space or solution space of
A.
Proof: Exercise. (Check the conditions of the subspace theorem.)
Example: 2 3
x
Show that S = {4y 5 2 R3 | x + y + z = 0} is a subspace of R3 .
z
Solution:
130
Theorem
Let A be an m ⇥ n matrix with real entries. Then
Proof:
131
Example:
One way of coding data is to use a Hamming code so that errors can
not only be detected but also corrected efficiently.
1. Using modular 2 arithmetic is well-suited to describing digital
information.
2. Data is stored in the form of a string of 1s and 0s. These 1s and
0s are called binary numbers.
132
94
Try playing around with Hamming’s List of Words – check the
subspace property in some examples. You receive the corrupted
message 0 1 0 0 1 0 1 0. Which Hamming word do you
think it was?
2 3 2 3 2 3 2 3 2 3 2 3 2 3 2 3
0 1 0 1 0 1 0 1
6 0 7 6 1 7 6 0 7 6 1 7 6 0 7 6 1 7 6 1 7 6 0 7
6 7 6 7 6 7 6 7 6 7 6 7 6 7 6 7
6 7 6 7 6 7 6 7 6 7 6 7 6 7 6 7
6 0 7 6 1 7 6 0 7 6 1 7 6 1 7 6 0 7 6 0 7 6 1 7
6 7 6 7 6 7 6 7 6 7 6 7 6 7 6 7
6 0 7 6 1 7 6 0 7 6 1 7 6 1 7 6 0 7 6 1 7 6 0 7
6 7 6 7 6 7 6 7 6 7 6 7 6 7 6 7
6 0 7 6 1 7 6 1 7 6 0 7 6 0 7 6 1 7 6 0 7 6 1 7
6 7 6 7 6 7 6 7 6 7 6 7 6 7 6 7
6 7 6 7 6 7 6 7 6 7 6 7 6 7 6 7
6 0 7 6 1 7 6 1 7 6 0 7 6 0 7 6 1 7 6 1 7 6 0 7
6 7 6 7 6 7 6 7 6 7 6 7 6 7 6 7
4 0 5 4 1 5 4 1 5 4 0 5 4 1 5 4 0 5 4 0 5 4 1 5
0 1 1 0 1 0 1 0
2 3 2 3 2 3 2 3 2 3 2 3 2 3 2 3
1 0 0 1 0 1 1 0
6 0 7 6 1 7 6 0 7 6 1 7 6 1 7 6 0 7 6 0 7 6 1 7
6 7 6 7 6 7 6 7 6 7 6 7 6 7 6 7
6 7 6 7 6 7 6 7 6 7 6 7 6 7 6 7
6 0 7 6 1 7 6 1 7 6 0 7 6 0 7 6 1 7 6 0 7 6 1 7
6 7 6 7 6 7 6 7 6 7 6 7 6 7 6 7
6 1 7 6 0 7 6 1 7 6 0 7 6 1 7 6 0 7 6 1 7 6 0 7
6 7 6 7 6 7 6 7 6 7 6 7 6 7 6 7
6 1 7 6 0 7 6 1 7 6 0 7 6 1 7 6 0 7 6 0 7 6 1 7
6 7 6 7 6 7 6 7 6 7 6 7 6 7 6 7
6 7 6 7 6 7 6 7 6 7 6 7 6 7 6 7
6 0 7 6 1 7 6 1 7 6 0 7 6 0 7 6 1 7 6 1 7 6 0 7
6 7 6 7 6 7 6 7 6 7 6 7 6 7 6 7
4 0 5 4 1 5 4 0 5 4 1 5 4 1 5 4 0 5 4 1 5 4 0 5
1 0 0 1 0 1 0 1
133
Null space (kernel) of a linear transformation
The notion of null space makes sense for general a linear
transformation.
Example:
Let W be the real 2 ⇥ 2 matrices whose trace is equal to 0. Show
that W is a subspace of M2,2 .
134
Solution: We claim that Tr : M2,2 ! R is a linear transformation.
Once the claim is proved, we recognize W as the null space of Tr .
135
Example:
Let W be the set of real 2 ⇥ 2 matrices with determinant equal to 0:
⇢
a b
W = | a, b, c, d 2 R and ad bc = 0
c d
Show that W is not a subspace of M2,2 .
Solution:
Find a specific counterexample
136
Example:
Consider the set
Is H a subspace of R2 ?
Solution:
137
The set of inhomogeneous solutions
The solutions for T (~x ) = ~b where ~b 6= ~0 do not form a subspace.
The general solution (set of all solutions) of a vector equation
T (~x ) = ~b is related to the general solution of the homogeneous
system T (~x ) = ~0 by a parallel shift.
{~x = ~p + ~h | ~h 2 Null(T )}
138
Picture:
y
p
2x + 3y = 4.5
2x + 3y = 0
x
Ax = b
Ax = 0
d t
Determine a particular solution ~p and the general solution ~h to the
homogeneous system.
Solution:
140
94
Note The solution set of a linear system in three variables is
represented by either
I the empty set if the system is inconsistent
I a point in R3 if the solution is unique or
I a line in R3 if we have one free parameter
I a plane in R3 if we have two free parameters
I all of R3 .
In each case, the solution set is obtained as a parallel shift of the
null-space. Historically, much of linear algebra evolved around
describing planes and lines in R3 , identifying their intersections and
orthogonal complements.
141
Geometrically, an inconsistent system in 3 variables has no common
point of intersection for the planes determined by the system.
142
94
Geometrically, a consistent system in 3 variables has a common point,
line or plane of intersection for the planes determined by the system.
143
Example: (integration constant)
Let T = D be di↵erentiation as on Page 117.
144
Definition (Injective, surjective, bijective, inverses)
Let T : V ! W be a linear transformation. If for any ~b 2 W
T (~x ) = ~b
I has at least one solution then T is called surjective.
T surjective ()
T injective ()
145
94
94
Example:
Is the rotation T : R2 ! R2 around the origin anticlockwise by an
angle ✓, with standard matrix
cos ✓ sin ✓
[T ] = ,
sin ✓ cos ✓
Solution:
146
Example:
Consider the linear transformation T : R2 ! R2 for projection onto
the x-axis, with standard matrix
1 0
[T ] = .
0 0
(a) Is T surjective?
(b) Is T injective?
(c) Is T invertible?
Solution:
147
148
Example: Give an example of a linear transformation that is injective
but not surjective.
149
Let A be an m ⇥ n-matrix.
1. Under which condition is left-multiplication with A injective?
2. Under which condition is it surjective?
3. Under which condition is it bijective?
150
94
94
94
94
2 3
1 1
Example: A = 41 25
1 3
Example: AT A =
151
Example: Let A be an m ⇥ n matrix of rank n with entries from R.
Prove
1. multiplication with A is injective,
2. the matrix AT A is invertible, and
3. multiplication with AT is surjective.
152
94
3.5 Matrix inverses
A 1
= 4 ~c1 ~c2 ... ~cn 5,
A~cj = ~ej 1 6 j 6 n.
154
Example: 2 3
1 2 1
Find the inverse of A = 4 1 1 1 5.
0 1 3
Solution:
Form the augmented matrix [A | I3 ] and perform row operations so
that A is in reduced row-echelon form.
155
94
Linear systems revisited
Theorem
If A is an invertible n ⇥ n matrix, then every linear system of the form
Ax = b has a unique solution, given by x = A 1 b.
Example:
Use a matrix inverse to solve the linear system
x + 2y + z = 4
x y + z = 11
y + 3z = 21
Solution:
Step 1. Write the system in the form A~x = ~b
156
Step 2. Find A 1.
157
Step 3.
The solution is ~x = A 1~b.
158
Properties of the matrix inverse
If A and B are n ⇥ n invertible matrices and ↵ is a non-zero scalar,
then
1 1 1
1. (↵A) = ↵ A
2. (AB) 1 =B 1A 1
n
3. (An ) 1 = A 1 (for all integers n > 1)
⇣ ⌘ 1
T
4. AT = A 1
159
Theorem (Inverse of a 2 ⇥ 2 matrix)
a b
Let A= . Then
c d
1. A is invertible () ad bc 6= 0.
1 d b
2. If ad 6 0, then A 1 =
bc = .
ad bc c a
Example:
Find the inverses, if they exist, of the geometric examples from Pages
39 to 47.
Solution:
160
161
94
Example: Encrypting messages
A common way of sending an encrypted message is to assign an
integer value to each letter of the alphabet and send the message as
a string of integers.
For example the message
SEND MONEY
might be sent as
5, 8, 10, 21, 7, 2, 10, 8, 3
Here the S is represented by a 5, the E by an 8, and so on. Ciphers
like this are generally easy to break.
A more secure way is to use matrix multiplication and matrix inverses.
162
Step 1: Encrypting the message
We put the message (SEND MONEY) in a matrix by entering the
integers corresponding to each letter down the columns
2 3
5 21 10
B=4 8 7 8 5
10 2 3