CS 231A Computer Vision Midterm

Out: 12:30pm, February 25, 2015

Due: 12:30pm, February 27, 2015
Solution Set

Transformations

In this question, we will explore some interesting low-level properties of transformations.

(a) In the lectures, we showed that a 3D rotation can be formed as the product of three matrices
R = Rx ()Ry ()Rz (),
where

cos sin 0
cos 0 sin
1
0
0
0
1
0 , Rz () = sin cos 0
Rx () = 0 cos sin , Ry () =
0
0
1
sin 0 cos
0 sin cos

are rotations around the x, y, and z axis, respectively. This is called the X-Y-Z Euler angle
representation, indicating the order of matrix multiplication.
However, representing rotations in this form can lead to ambiguities, since the result depends on the order in which the transforms are performed.
Let p0 be the point obtained by rotating a point p with a rotation matrix R, so p0 = Rp.
Give an expression for p0 obtained by rotating p in the following two ways:
First rotate around x axis, then rotate around z axis.
First rotate around z axis, then rotate around x axis.
Show that these rotations produce different values of p0 .

Solution:
No, the result is dependent on the order of rotation.

p0 = Rx ()Rz ()p

1
0
0
cos sin 0
= 0 cos sin sin cos 0 p
0 sin cos
0
0
1

cos
sin
0

= cos sin cos cos sin p

sin sin sin cos cos

p0 = Rz ()Rx ()p

cos sin 0
1
0
0
= sin cos 0 0 cos sin p
0
0
1
0 sin cos

cos sin cos sin sin

= sin cos cos cos sin p
0
sin
cos

(b) To avoid the representation ambiguity introduced in part a, we can fix the rotation order.
However, even if the order of rotation is fixed, this rotation system may still result in a
degenerate representation.
For instance, let , , and be three different angles. We can represent an arbitrary
rotation with respect to the X-Y-X axes as the product Rx ()Ry ()Rx () (note that the
axes here are X-Y-X, not X-Y-Z). This representation of a rotation matrix should have
three degrees of freedom. However if we set = 0, something unusual happens. How many
degrees of freedom are left? Hint: some trigonometric identities may be useful.

Solution:

1
0
0
1 0 0
1
0
0

0 1 0
0 cos sin
Rx ()Ry (0)Rx () = 0 cos sin
0 sin cos
0 0 1
0 sin cos

1
0
0
= 0 cos cos sin sin cos sin sin cos
0 sin cos + cos sin sin sin + cos cos

1
0
0
= 0 cos( + ) sin( + )
0 sin( + ) cos( + )

Since it is parameterized by ( + ), it only has one degree of freedom.

(c) To avoid the loss of a degree of freedom mentioned above, we can use an alternative rotation
representation defined as follows:
R(
n, ) = I + sin [
n] + (1 cos )[
n]2 ,
where n
is a unit vector and is the angle of rotation. Recall that [] is the matrix cross
product operator (see Lecture 5).
Any valid rotation representation must satisfy some properties of rotation. Generally, for
any rotation matrix R, we must have that RT R = I. Prove that this equality holds for the
rotation matrix R defined above. (Hint: Consider the rotation given by RT .)

Solution:
[
n]T = [
n] because

0
nz ny
0
nz ny
0
nx = nz
0
nx = [
[
n]T = nz
n].
ny nx
0
ny nx
0

Therefore, we have
R(
n, )T = (I + sin [
n] + (1 cos )[
n]2 )T
= I + sin [
n]T + (1 cos )([
n]2 )T
= I sin [
n] + (1 cos )[
n]2
= R(
n, )
3

Therefore, applying RT R is equivalent to not rotating at all, so RT R = I.

Alternatively, students can also write out RT R explicitly. First, [
n]T = [
n] because

T
0
nz ny
0
nz ny
0
nx = [
0
nx = nz
n].
[
n]T = nz
ny nx
0
ny nx
0

Also observe that [

n]2 = n
n
T I, so we have
[
n]4 = (
nn
T I)(
nn
T I)
=n
n
T n
n
T 2
nn
T + I
=n
n
T 2
nn
T + I
= [
n]2 .
Thus,
RT R = (I + sin [
n] + (1 cos )[
n]2 )T (I + sin [
n] + (1 cos )[
n]2 )
= (I sin [
n] + (1 cos )[
n]2 )(I + sin [
n] + (1 cos )[
n]2 )
= I + sin [
n] + (1 cos )[
n]2 sin [
n] sin2 [
n]2 sin (1 cos )[
n]3
+ (1 cos )[
n]2 + sin (1 cos )[
n]3 + (1 cos )2 [
n]4
=I
(d) Let us now consider points in 1-D. In homogeneous coordinates, a point in 1-D can be
represented as (x, 1)T , where x R. More generally, such a point can be represented as
(kx, k)T or generally as (x1 , x2 )T .
A projective transformation of a point in 1D is represented by a 2 2 matrix,
x0 = H22 x,
where H22 has a non-zero determinant.
Given 4 points on a line, the cross ratio is defined as
Cross(x1 , x2 , x3 , x4 ) =
where


|xi xj | = det

|x1 x2 ||x3 x4 |
,
|x1 x3 ||x2 x4 |

xi1 xj1
xi2 xj2

with point xi given by (xi1 , xi2 )T and xj given by (xj1 , xj2 )T .

Prove that the value of the cross ratio is invariant under any projective transformation of
the line: ie show that if x0 = H22 x, then
Cross(x01 , x02 , x03 , x04 ) = Cross(x1 , x2 , x3 , x4 ).
Equivalently, the cross ratio is invariant to the projective coordinate frame chosen for the
line.
4

Solution:
|x0i x0j | = |i Hxi j Hxj |


i H11 xi1 + i H12 xi2 j H11 xj1 + j H12 xj2
= det
i H21 xi1 + i H22 xi2 j H21 xj1 + j H22 xj2


H11 H12
= i j (xi1 xj2 xi2 xj1 ) det
H21 H22
= i j |xi xj | det H

Cross(x01 , x02 , x03 , x04 ) =

|x1 x2 ||x3 x4 |
|x01 x02 ||x03 x04 |
1 2 |x1 x2 | det H3 4 |x3 x4 | det H
=
.
=
|x01 x03 ||x02 x04 |
1 3 |x1 x3 | det H2 4 |x2 x4 | det H
|x1 x3 ||x2 x4 |

(e) Given the previous result, does the cross ratio remain invariant under rotation? Give a
brief explanation.

Solution:
Yes, because rotation matrix is a special case of projective transformation.
(f) Extra Credit
Show that the result in (??) holds even when x0i = i H22 xi , where points are scaled by
arbitrary scale factors i .

Multiview geometry

Assume that we have two cameras with camera matrices M1 and M2 . Suppose that the cameras
generate image I1 and I2 , respectively, where p1 , p2 , p3 , p4 and p01 , p02 , p03 , p04 are corresponding
points across the two images.
(a) Suppose


M2 = K2 R2 T2



M1 = K1 R1 T1

(1)

where
K1 = K2 = K
R1 = I = Identity matrix
R2 = R

(2)

T1 = T2 = [0, 0, 0]T
Prove that the homography H defined by
p0i = Hpi where i = 1, 2, 3, 4

(3)

can be expressed as H = KRK 1 .

Solution:
Consider p and p0 are corresponding points in two images whose world coordinate is P . We
have




p=K I 0 P
p0 = K R 0 P
Therefore, we can have


p0 = KR I 0 P


= KR(K 1 K) I 0 P


= (KRK 1 )(K I 0 P )
= (KRK 1 )p = Hp
(b) Continuing with our setup in part (a). Do N > 3 corresponding points necessarily
need to belong to the same planar surface in 3D in order for the above result in part (a) to be
true? Justify your answer.
Solution:
No. Notice that in part (a), we did not use any coplanarity constraints.

(c) Now we change our setup of two cameras M1 and M2 as follows.





M2 = K2 R2 T2
M1 = K1 R1 T1

(4)

where
K1 = K2 = K
R1 = I = Identity matrix
R2 = R

T
T1 = 0 0 0
T

T2 = t = tx ty tz

(5)

All four image correspondences pi p0i are projections of four points Pi which lie on the same
planar surface in 3D. is defined as the set of points X such that Z T X = 0, where Z =
 T
T
v
1 (v is a 3 1 vector). An illustration of this step can be found in Figure ??. Express
the homography H (defined by p0 = Hp) in terms of K, R, t, v and explain your derivation.

Figure 1: Camera setup used in part (c) (d) (e) and (f).

Solution:
To compute H we back-project a point p in the first view and determine the intersection point
P of this ray with the plane X. The 3D point P is then projectedinto the
 second view.


K 1 p
For the first view p = K I 0 P and so any point on the ray P =
projects to p, where
d
d parameterizes the point on the ray. Since the 3D point P is on plane X it satisfies Z T P = 0.


K 1 p
This determines d, and P =
. The 3D point P projects into the second view as
v T K 1 p




p0 = K R t P




K 1 p
= KR Kt
v T K 1 p
= K(R tv T )K 1 p
Therefore, we have H = K(R tv T )K 1 .
(d) Continuing with our setup in part (c). Prove that the fundamental matrix F which
satisfies pT F p0 = 0 can be expressed as
F = ([e2 ] H)T
where e2 (see Figure ??) is the epipole in the second image plane.
Solution:
By the definition of fundamental matrix, the epipolar line l0 in the second image plane should
be
l0 = F T p
On the other hand, we can also express l0 as
l0 = e2 p0 = [e2 ] p0
Since p0 = Hp, we have F T = [e2 ] H.
(e) Continuing with our setup in part (c). In addition to the four correspondences on
the plane, we are given two additional pairs of corresponding points (p5 p05 , p6 p06 ) that do NOT
belong to the same planar surface . Show how to find epipole e2 and explain your derivation.
Hint: e2 can be expressed as the intersection of two lines.
Solution:
Suppose the two additional correspondences are {p5 , p05 } and {p6 , p06 }. e2 can be expressed as
the intersection of two lines in the second image plane, where these two lines are p05 Hp5 and
p06 Hp6 .

(f) Continuing with our setup in part (c). From what you have shown in the previous parts,
write a function computeF.m that estimates the fundamental matrix F using 6 correpondences
(4 of them belong to the same 3D plane).

(a) Image 1

(b) Image 2

Figure 2: Two images from camera M1 and M2

We provide
A dataset of six corresponding points from two images (Figure ??) under the camera
setup shown in Figure ??. p1.mat includes six points from image 1 and p2.mat includes
six corresponding points from image 2. The first four corresponding points are from the
same 3D plane, and the last two corresponding points are out of that plane. You can run
loadData.m to get familiar with the data.
Function computeH.m that estimates a homography from at least 4 points.
For this part, you need to turn in
An outline of the algorithm that you are using to compute F .
Your code for function computeF.m.
Your numerical results of F .
Hint:
You dont need to (and should not) label additional points.
You can verify your results by computing pT F p0 .
Solution:
function F = computeF( pt1, pt2 )
if (size(pt1, 1) ~= 6 || size(pt2, 1) ~= 6),
error(Wrong number of points.);
end
9

H = computeH(pt1(1:4, :), pt2(1:4, :));

x5
x5
x6
x6
l1
l2

=
=
=
=
=
=

H * [pt1(5,:); 1];
x5 / x5(3);
H * [pt1(6,:); 1];
x6 / x6(3);
cross(x5, [pt2(5,:); 1]);
cross(x6, [pt2(6,:); 1]);

e = cross(l1, l2);
e = e / e(3);
ex = zeros(3,3);
ex(1, 2) = -e(3);
ex(2, 1) = e(3);
ex(1, 3) = e(2);
ex(3, 1) = -e(2);
ex(2, 3) = -e(1);
ex(3, 2) = e(1);
F = (ex * H);
end

0.0000 0.0000 0.0008

0.0899
F = 104 0.0001 0.0000
0.0139 0.0800 5.2679

10

Hough Transform and Vanishing Points

For this problem, we wish to find the vanishing points of a scene by finding the intersection
point of a set of parallel lines. In Problem Set 1, we selected a pair of parallel lines using 4
points and then extrapolated to find their intersection. This process is prone to noise and is
sensitive to the selection of points (as many of you complained).
To improve this process, we will densely sample many points along a set of parallel lines and
use the Hough Transform to better fit lines in the scene. If we parameterize a line using r and
r
as presented in class, we can represent a line as y = cos()
sin() x + sin() . As a result, each edge
point will be mapped to a curve in the polar parameter space by the Hough Transform.
(a) We will begin by building some intuition about the Hough transform. For the given
Cartesian points in the left plot, sketch the corresponding curves in the Hough space. A
rough sketch is ok; ensure your sketch captures the fact that there are 10 points that all
lie along the same line.

Solution:

(b) Once we identify peaks in the Hough space, we can map these back to the Cartesian
space. For the given peaks in the Hough space (left plot below), sketch the corresponding
lines in the Cartesian space. The sketches can be approximate.

Solution:
(c) Assuming we densely sample the parallel lines in the scene that correspond to the same
vanishing point, we can use the Hough Transform to identify the most likely set of lines.
Since there could be multiple parallel lines in the scene, there could be multiple peaks in
the Hough space.
Let the vanishing point lie at x0 , y0 . The lines in the image that intersect the vanishing
point will form peaks in Hough space. Describe the curve in (, r) Hough space that all
peaks will lie along. Express your answer as a function of the form ri = r(i , x0 , y0 ).
11

Solution:
Since all of the points correspond to lines that should intersect at the same vanishing point, all of the peaks in the Hough transform should lie along the curve r() =
x0 cos()+ y0 sin(), which is the equation in Hough space of the set of all lines that go
through the vanishing point.
(d) If the points selected were from lines that corresponded to more than one vanishing point,
what would the peaks in the Hough space look like? Give your answer qualitatively,
referencing your solution to (c).

Solution:
The intersection points/peaks would represent a point wise sampling of multiple sinusoids,
of the form r() = xi cos()+ yi sin() for each unique vanishing point, with the xi , y + i
being the image coordinates of the vanishing point. .
(e) If we took the locations of the peaks described in part (d) and transformed them back
into Cartesian space, how could we use a procedure similar to the Hough transform to
find the location of the vanishing points?

Solution:
Plot lines in Cartesian space corresponding to detected peaks in the Hough parameter
space. Then we can quantize the Cartesian space and using the lines found by the hough
transform to cast votes for intersection points to find the vanishing points.
(f) In the previous parts of this problem we overlooked the size of the bins in the Hough space.
If the Hough space had bins of finite size to act as accumulators for the transformed points,
describe the effects of bin size on the procedure described in parts (d) and (e), assuming
that the original point sampling of the parallel lines described in part (a) was subject to
noise. Specifically discuss the trade offs that must be considered when selecting bin size.
12

Solution:
Assuming that the thresholds for the peak detection is proportional to the size of the
bins:
Smaller bin sizes more susceptible to noise, since the intersection of the curves or lines in
parts e/f could be spread by noise, so there could be spurious peaks detected in neighboring
cells. For part e, this would case spurious lines to be found, which would cause even more
spread in part f when finding vanishing points.
Larger bin sizes. are less susceptible to noise causing spurious peaks, however they could
result in more coarse detection of peaks since more noisy points could be incorporated in
each bin. Also, if lines/points are close together in the parameter spaces, they night get
binned together, so some lines and vanishing points may be merged.

Figure 3: Scene with two planes containing parallel lines

(g) Suppose the sets of parallel lines described thus far in this problem belong to N planes,
where the number of planes, N , is unknown. Each plane contains multiple sets of parallel
lines with different directions. A simple illustration of this setup can be seen in Figure
?? with N=2 planes. How might we find N , the number of planes in the scene and their
normal directions using techniques developed in earlier parts of this question? What
additional information might you need? Hint: What properties do vanishing points that
belong to the same plane have? Are they geometrically related?

Solution:
We can use Hough transform as described by parts d and e to find the vanishing points,
some might be spurious because there are many more intersections. All vanishing points
for a plane should fall on the the same horizon line (vanishing line). So we can use a

13

Hough transform on these vanishing points to find the most likely set of horizon lines.
The number of peaks in the Hough transform is an estimate for the number of planes.
The horizon lines for each plane can be used to find the normal from the expression

l = K T
n . We would need the camera matrix to find the normal.
(h) Extra Credit
One way to estimate the position (x0 , y0 ) of the vanishing point is to fit a model of the
form found in (c) to the n observed Hough peaks that lie at (ri , i ). We could then use
least squares to minimize the sum of squared errors between the measured ri and that
predicted by the model r(i , x0 , y0 ).
min

n
X

x0 ,y0

(ri r(i , x0 , y0 ))2

i=1

This is an elegant way to find a vanishing point just from the point measurements of the
parallel lines in a scene.
Formulate an appropriate objective to describe this problem and solve it using least
squares. For this part, you can assume that all peaks in Hough space correspond to lines
intersecting at a single vanishing point (x0 , y0 ) in the image. What is the expression for
the optimal vanishing point? Note: it is OK to leave matrix inverses in your solution,
but try to simplify other terms as much as possible.

Solution:
n
X

cos(i ) (ri x0 cos(i ) y0 sin(i )) = 0

i=1
n
X

sin(i ) (ri x0 cos(i ) y0 sin(i )) = 0

i=1

Which can be reformed to

n
X
i=1
n
X

!
2

cos(i )

x0 +

n
X

i=1

n
P

n
X

!
cos(i )sin(i ) x0 +

!
sin(i )2

y0 =

i=1

n
P


cos(i
cos(i )sin(i )

i=1
i=1



n
n
P
P
2
cos(i )sin(i )
sin(i )
)2

i=1

cos(i )ri

i=1

i=1

cos(i )sin(i ) y0 =

sin(i )ri

i=1

xo
yo

cos(i )ri

P
= i=1

sin(i )ri

i=1

i=1

n
P

i=1

n
P




cos(i
cos(i )sin(i )

xo
i=1
i=1



=
n
n
P
P
yo
2
cos(i )sin(i )
sin(i )
)2

i=1

Alternate formulations using SVD are also possible.

14

cos(i )ri

i=1
P

sin(i )ri

i=1

Blob detection

In lecture we learned how we can use a convolution with a Laplacian of a Gaussian to find edges
in an image. In this question we will explore this idea in more detail.
(a) Suppose that you have the following black-and-white image. In our image encoding, white
is represented by the value of 1 and black the value of 0.

Sketch a one-dimensional slice of the intensity (i.e. brightness) of this image in the xdirection. Then sketch the result of convolving this signal with the Laplacian of a Gaussian
and mark where the edges are located in the plot of the convolution. Your sketches do not
need to be scaled correctly because we have not specified the width of the Gaussian. Please
ignore border-effects, i.e. ignore the effect of convolving the kernel with the borders of the
image.
Draw your answer here:

Solution:

15

The signal starts at a high intensity / brightness (white), goes to low values (black), and
then goes back to high values (white). We therefore draw the signal as starting at 1, going
to 0, and going back to 1 (the shape of the graph is the important part here - the scale is
arbitrary).

After convolving the signal with the Laplacian of a Gaussian, the response looks like this
(ignoring border-effects):

Note that this is the reverse of the response from the lecture slides, since the signal in the
lecture slides went from low to high to low, whereas the signal here goes from high to low
to high.
(b) Given the following formula for a 2D Gaussian centered at (0, 0):
g=

1 (x2 +y2 )/(22 )

e
2 2

take derivatives and compute the formula for the 2D Laplacian of a Gaussian.

Solution:
First, we have the formula for a 2D Gaussian:
g=

1 (x2 +y2 )/(22 )

e
2 2

Taking the first partial derivative:

g
xg
= 2
x

The second partial derivative is then:

2g
g x2 g
=
+ 4
x2
2

Similarly:
2g
g y 2 g
=
+ 4
y 2
2

16

So we can write the sum of the partial derivatives as:

2g
2g
(x2 + y 2 2 2 )g
+
=
x2 y 2
4

Therefore:
2g
2g
+ 2)
2
x
y
2
2
(x + y 2 2 )g
=
2
1 (x2 +y2 )/(22 ) 2
(x + y 2 2 2 )
e
=
2 4

2norm g = 2 (

(c) Suppose that you convolve a 2D Laplacian of a Gaussian with the following image, where
the circle has radius r. Using your answer from parts ?? and ??, find the characteristic
scale of this image, in terms of the radius of the circle r.
Hint: At the characteristic scale, the convolution will have the same zeros as the Laplacian
convolution kernel.

Solution:
From part b, the zeros of the Laplacian occur when
x2 + y 2 2 2 = 0
Because the convolution has the same zeros as the Laplacian convolution kernel at the
characteristic scale (given in the hint), we want the zeros of the kernel to align with the
edges of the image. The edges of the image occur at
x2 + y 2 = r 2
Combining the above equations, we get that the characteristic scale occurs at
r2 2 2 = 0
or

= r/ 2

17