University of Waterloo

Department of Electrical and Computer Engineering

Winter, 2017

ECE 602: Introduction to Optimization

FINAL EXAMINATION

m
er as
co
eH w
Surname

o.
rs e
ou urc
Legal Given Name(s)

UW Student ID Number
o
aC s
vi y re

Instruction:
ed d

1. There are 100 points in total.

ar stu

2. This is a written, open-book exam (only unannotated lecture slides are allowed). Please turn
o↵ all electronic media and store them under your desk.
3. Be neat. Poor presentation will be penalized.
sh is

4. No questions will be answered during the exam. If in doubt, state your assumption(s)
Th

and continue.
5. Do not leave during the examination period without permission.
6. Do not stand up until all the exams have been picked up.

Do well!

https://www.coursehero.com/file/58351395/exam2017pdf/
1
 Question 1 (15 points)
Consider the non-linear optimization problem

min exp (2x1 ) + exp ( 2x2 )

s.t. x21 + x22 3x1 4x2  0,
x1 + x2 = 1,
x1 0,
x2 0.

a) Is the problem a convex optimization problem or not?

b) Apply Slater’s constraint qualification to show if the problem admits strong duality.
c) Give the Lagrange function and the Lagrange dual for this problem.

m
d) Derive the KKT conditions for the given problem.

er as
co
eH w
Question 2 (15 points)

o.
rs e
You have to design a 3-D block as a water storage. The volume of the block has to be at least
ou urc
9 cubic meters. The base area of the block is at most 6 square meters, while the height of the
block is at least 1 and at most 2 meters. Your task is to design the block that satisfies the above
conditions and the di↵erence between the base area and the height of the block is maximal.
o
aC s
vi y re
ed d
ar stu

a) Model the above problem as a constrained (non-linear) optimization problem.

b) Is this a convex optimization problem? Prove if it is convex; OR give an evidence that it is
sh is

not convex.
Th

c) Give the KKT conditions of your optimization problem.

d) Check if the height=1.5 meters, base-area = 6 square meters corresponds to an optimal
solution.

Question 3 (10 points)

Derive the conjugate function f ⇤ (y) for each of the following functions:

a) f (x) = 3x2 + 4x
b) f (x) = log x + 2

https://www.coursehero.com/file/58351395/exam2017pdf/
2
 Question 4 (20 points)
Consider the function f (x) = f (x1 , x2 ) = (x1 + x2 )2 .

a) Derive the gradient of f (x).

b) At the point x0 = (0, 1)T , consider the search direction d = (1, 1)T . Show that d is a
descent direction.
c) Find the stepsize ↵ that minimizes f (x0 + ↵d); that is, what is the result of this exact line
search? Provide the value of f (x0 + ↵d).
d) Derive the Hessian of f (x).
e) Perform one Newton step with ↵ = 1 starting at x0 = (0, 1)T to compute x1 . What are x1
and f (x1 )?

m
Question 5 (15 points)

er as
co
The null space of a matrix A 2 Rm⇥n (with m < n) is defined to be the set of all x 2 Rn such

eH w
that Ax = 0. The projection of a point z onto a convex set S is defined as the point x⇤ 2 S which

o.
is closest in Euclidean distance to z. Find a closed form solution for the projection of z onto the
rs e
convex set {x | Ax = 0}. (You can assume A is full rank, i.e., rank A = m.) To solve the problem,
ou urc
set it up as a constrained optimization, write out the Lagrangian, and derive the KKT conditions.
o

Question 6 (10 points)

aC s
vi y re

You are the manager of a large company where you face the decision of selecting the right projects
to maximize the total returns. There are n possible projects {Pk }nk=1 . Each project Pk runs for 3
years and has an overall return of ck dollars. The financial constraints are that, in year t, there
are only a total of ft dollars available for these projects, whereas project Pk requires at least ak,t
ed d

dollars (for t = 1, 2, 3 and k = 1, . . . , n). Formulate this problem as an integer program (i.e., a
ar stu

problem with integer variables/constraints).

Hint: Define variables xk so that xk = 1 means selecting Pk and xk = 0 means not selecting
Pk . Formulate your integer program in terms of these variables.
sh is
Th

Question 7 (15 points)

Consider the linear program

min x1 + 2x2
s.t. x1 + x2  1, x1 x2  1, x1 0, x2 0

a) Draw the region of feasibility.

b) Solve the LP and verify the optimality of your optimal solution.

https://www.coursehero.com/file/58351395/exam2017pdf/
3
 Solutions to Final Exam Winter 2017

Solutions to Question 1

m
er as
a) Yes, the objective function and all the constraints are convex, and the

co
equality constraint is linear.

eH w
b) (0.5, 0.5)T is an ideal Slater point because none of the inequality con-

o.
straints are active.
rs e
ou urc
c) The Lagrangian is
k
X
L(x, , µ) =f (x) + j gj (x) + µT (b Ax)
o

j=1
aC s

2x1 2x2
=e + e + 1 (x21 + x22 3x1 4x2 )
vi y re

+ 3 ( x1 ) + 4 ( x2 ) + µ(x1 + x2 1) .
Denote g( , µ) = inf x {L(x, , µ)}.
ed d

The Lagrange dual of this problem is

ar stu

sup g( , µ)
,µ

s.t. 0.
sh is

d) The KKT condition for this problem is

Th

g(x)  0
Ax = b
x 0
0
rx L(x, , µ) = 0
j gj (x) = 0, j = 1, · · · , k ,

https://www.coursehero.com/file/58351395/exam2017pdf/
 where

2e2x1 + 1 (2x1 3) 3 +µ
rx L(x, , µ) = .
2e 2x2 + 1 (2x2 4) 4+µ

Solutions to Question 2
a) The problem can be formulated as follows

max |d · l h|
l,d,h

m
s.t. h·d·l 9

er as
d·l 6

co
eH w
h 1

o.
h2
rs e l 0
ou urc
d 0.

b) Rewriting the first constraint in the standard form h · d · l + 9  0 and

o

computing the Hessian of the function g1 (l, d, h) = h · d · l + 9, we get:

aC s
vi y re

2 3
0 h d
r2 g1 (l, d, h) = 4 h 0 l 5.
d l 0
ed d

As the determinant of the Hessian is negative for l, d, h 0, the function

ar stu

g1 (l, d, h) is not convex. So, the first constraint is not convex and this is
not a convex optimization problem.
sh is

c) Noting that l · d 4.5 h, the Lagrange function of the optimization

problem is:
Th

L(l, d, h) = h d·l+y1 (9 h·d·l)+y2 (d·l 6)+y3 (1 h)+y4 (h 2) y5 l y6 d, y 0, y 2 R6 .

https://www.coursehero.com/file/58351395/exam2017pdf/
 Consequently, the KKT conditions of the NLO formulation are:

9 h·d·l 0
d·l 60
1 h0
h 20
l0
d0
d y1 (h · d) + y2 d y5 = 0

m
er as
l y1 (h · l) + y2 l y6 = 0

co
1 y1 (l · d) y3 + y4 = 0

eH w
y1 (9 h · d · l) = 0

o.
y2 (d · l 6) = 0
rs e
ou urc
y3 (1 h) = 0
y4 (h 2) = 0
y5 l = 0
o

y6 d = 0
aC s

y 0.
vi y re

d) If h = 1.5 and l · d = 6, then the problem is feasible as all the constraints

are satisfied. For 1  h  1.5 the problem gets infeasible. For 1.5 < h  2
ed d

the objective function value is smaller than for h = 1.5. So, h = 1.5 and
ar stu

l · d = 6 is the optimal solution.

Solutions to Question 3
sh is

y 4
a) f ⇤ (y) = maxx x · y (3x2 + 4x). By stationarity y 6x 4 = 0 ) x = 6
.
Th

2
Thus f ⇤ (y) = (y 124) .

b) f ⇤ (y) = maxx x·y+ln(x) 2. If y 0, then clearlyf ⇤ (y) = 1. Otherwise,

by stationarity y + x1 = 0 ) x = y1 . Thus f ⇤ (y) = 3 + ln( y1 ) =
3 ln( y).

https://www.coursehero.com/file/58351395/exam2017pdf/
 Solutions to Question 4
1. 
2(x1 + x2 )
rf (x) = .
2(x1 + x2 )

2. rf (x0 )T d = 0. Therefore, d is not a descent direction.

3. f (x0 + ↵d) = 1 for all ↵.

4. 
2 2

m
2
r f (x) = .

er as
2 2

co
eH w
5. r2 f (x) is not invertible. No Newton steps are defined.

o.
rs e
Solutions to Question 5
ou urc
minx 12 kx zk22 such that Ax = 0. Lagrangian L(x, ) = 12 kx zk22 + T Ax.
KKT conditions give Ax⇤ = 0 and x⇤ z + AT ⇤ = 0. The second condition
(on multiplying by A) yields Az = AAT ⇤ , implying ⇤ = (AAT ) 1 Az,
o

yielding x⇤ = z AT (AAT ) 1 Az.

aC s
vi y re

Solutions to Question 6
The problem can be formulated as follows
ed d

X
ar stu

max ck · xk
x
k
X
s.t. akt · xk  ft , t = 1, 2, 3
sh is

k
xk 2 {0, 1}, k = 1, · · · , n .
Th

Solutions to Question 7
a) Clear.

b) (0, 0)T is the optimal point, which can be verified by the Rockafellar-
Pshenichnyi condition.

https://www.coursehero.com/file/58351395/exam2017pdf/

Powered by TCPDF (www.tcpdf.org)