MA1521 CALCULUS FOR COMPUTING

Wang Fei

matwf@nus.edu.sg

Department of Mathematics
Office: S17-06-16
Tel: 6516-2937

Chapter 5: Optimization 2
Extreme Values . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3
Local Extreme Values. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5
Critical Point . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8
Examples . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12
Absolute Extreme on Closed Bounded Region . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15
Extreme Values with Restriction. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 22
Lagrange Multiplier. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 24
Lagrange Multiplier of More Variables . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 28
Extreme Values with Two Constraints . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 31

1
Chapter 5: Optimization 2 / 33

Extreme Values
• One-variable: Let y = f (x) be a function with domain D .
◦ f has a global (absolute) maximum at c ∈ D
⇔ f (c) ≥ f (x) for all x ∈ D .
◦ f has a global (absolute) minimum at c ∈ D
⇔ f (c) ≤ f (x) for all x ∈ D .
• Maximum and Minimum of Two-Variable Function.
Let z = f (x, y) be a function with domain D ⊆ R2 .
◦ f has a global (absolute) maximum at (a, b) ∈ D
⇔ f (a, b) ≥ f (x, y) for all (x, y) ∈ D .
◦ f has a global (absolute) minimum at (a, b) ∈ D
⇔ f (a, b) ≤ f (x, y) for all (x, y) ∈ D .

3 / 33

Extreme Values
• Extreme Value Theorem for Two-Variable Function:
Let z = f (x, y) be a continuous function defined on a closed, bounded domain D ⊆ R2 .
◦ Then f attains the (absolute) extreme values, i.e.,
There exist points (a, b) ∈ D and (c, d) ∈ D such that
• f (a, b) ≤ f (x, y) ≤ f (c, d) for all (x, y) ∈ D .
• Question. Suppose z = f (x, y) is continuous on a closed, bounded domain D .
◦ What are the (absolute) extreme values?
• It may be obtained at the boundary point of the domain; or
• It may be obtained in the interior of the domain.

4 / 33

2
Local Extreme Values
• Local Extreme Values for Two-Variable Functions.
Let z = f (x, y) be a function with domain D ⊆ R2 .
◦ f has a local (relative) maximum at (a, b) ∈ D
⇔ f (a, b) ≥ f (x, y) for all (x, y) in an open region containing (a, b).
◦ f has a local (relative) minimum at (a, b) ∈ D
⇔ f (a, b) ≤ f (x, y) for all (x, y) in an open region containing (a, b).

5 / 33

Local Extreme Values

• Recall the Fermat’s Theorem of one-variable function:
If y = f (x) has a local extreme value at c, and if f ′ (c) exists, then f ′ (c) = 0.
b
y

local max

O x

local min
b

◦ If f has a local extreme value at c, then the tangent line to y = f (x) at c, if exists, must be
horizontal.

6 / 33

3
Local Extreme Values
• Suppose y = f (x, y) has a local extreme value at (a, b).
◦ It is expected that
• The tangent plane to z = f (x, y) at (a, b), if exists, must be horizontal.
◦ Recall the tangent plane at (a, b):
• z = f (a, b) + fx (a, b)(x − a) + fy (a, b)(y − b).
It is horizontal ⇔ fx (a, b) = fy (a, b) = 0.

• First Derivative Test for Local Extreme Values.

Suppose z = f (x, y) has a local extreme value at (a, b).
◦ If fx (a, b) and fy (a, b) exist, then

fx (a, b) = fy (a, b) = 0.

7 / 33

Critical Point
• Definition. Let z = f (x, y) be a function with domain D . Then (a, b) ∈ D is called a critical
point if
◦ fx (a, b) = fy (a, b) = 0, or
◦ at least one of fx (a, b) and fy (a, b) does not exist.

• Therefore, if z = f (x, y) at a local extreme value at (a, b), then (a, b) is a critical point of f .
• Example. f (x, y) = x3 − y 3 − 2xy + 6.
◦ Let fx = 3x2 − 2y = 0. Then 3x2 = 2y .
◦ Let fy = −3y 2 − 2x = 0. Then 3y 2 = −2x.
3 2 2 27 4 8
−2x = 3y 2 = 3 x ⇒ x4 = − 27


2
x = 4
x.
2 4
◦ x = 0 ⇒ y = 0; x = −3 ⇒ y = 3.
points (0, 0) and − 23 , 23 .


Hence, f has two critical

8 / 33

4
Second Derivative Test
• Question. If z = f (x, y) has a local extremal value at (a, b), then (a, b) is a critical point.
◦ Suppose (a, b) is a critical point of z = f (x, y). How can we determine whether f has a local
maximum or minimum at (a, b)?

• Definition. Let (a, b) be a critical point of z = f (a, b).

◦ f is said to have a saddle point at (a, b) if f does not a local extremal value at (a, b).

9 / 33

Second Derivative Test

• Consider a one-variable function y = f (x):
Suppose f ′′ > 0 on interval I . Then f is concave up.

y = f (x)

(c, f (c))
b

◦ If f ′ (c) = 0 at some c,
then the tangent line of f at c is y = f (c).
•Since f is concave up,
the graph of f lies above y = f (c).
• In other words, f (x) > f (c) for all x 6= c.
∴ f has the minimum at c.

10 / 33

5
Second Derivative Test
• Second Derivative Test for One-Variable Function.
Suppose f ′ (c) = 0.
◦ f ′′ (c) > 0 ⇒ f has a local minimum at c;
◦ f ′′ (c) < 0 ⇒ f has a local maximum at c.

• Definition. The Hessian of z = f (x, y) is

fxx fxy 2
◦ H(x, y) = = fxx fyy − fxy .
fxy fyy
• Second Derivative Test for Two-Variable Function.
Suppose fx (a, b) = fy (a, b) = 0.
◦ H(a, b) > 0 and fxx (a, b) > 0 ⇒ local min at (a, b);
◦ H(a, b) > 0 and fxx (a, b) < 0 ⇒ local max at (a, b);
◦ H(a, b) < 0 ⇒ saddle point at (a, b).

11 / 33

Examples
• f (x, y) = x3 − y 3 − 2xy + 6.
◦ It has two critical numbers (0, 0) and − 23 , 23 .

◦ fx = 3x2 − 2y , fy = −3y 2 − 2x.

• fxx = 6x, fxy = −2, fyy = −6y .
• H(x, y) = (6x)(−6y) − (−2)2 = −36xy − 4.
◦ H(0, 0) =
−4 < 0 ⇒ saddle point at (0, 0).
◦ H − 23 , 23 = 12 > 0
fxx − 23 , 32 = −4 < 0 ⇒ local max at − 23 , 32 .



•

12 / 33

6
Examples
• f (x, y) = xy + 2x − ln x2 y , x > 0, y > 0.
◦ fx = y + 2 − x2 , fy = x − y1 .
1


• fx = fy = 0 ⇒ (x, y) = 2
,2 .
2 1
◦ fxx = fxy = 1, fyy =
x2
, y2
.

2

 1 
• H(x, y) =
x2 y2
− 12 = x22y2 − 1.

◦ H 12 , 2 = 1 > 0, fxx 12 , 2 = 2 > 0.

1


It follows that f has a local minimum 2 + ln 2 at 2
,2 .

13 / 33

Examples
• f (x, y) = 3x2 − 2xy + y 2 − 8y + 7.
◦ fx = 6x − 2y , fy = −2x + 2y − 8.
Let fx = fy = 0. Then
( (
0 = 6x − 2y x=2
• ⇒
0 = −2x + 2y − 8 y=6
fxx = 6, fxy = −2, fyy = 2.
• H(x, y) = (6)(2) − (−2)2 = 8 > 0.
• fxx = 6 > 0 ⇒ local minimum at (2, 6).
• Remark. Suppose z = f (x, y) has a critical point (a, b).
◦ H(x, y) > 0 & fxx > 0 on D ⇒ global min at (a, b);
◦ H(x, y) > 0 & fxx < 0 on D ⇒ global max at (a, b).

14 / 33

7
 Absolute Extreme on Closed Bounded Region
• Absolute Extreme on Closed Bounded Region.
Suppose z = f (x, y) is continuous on a closed and bounded region D .
Step 1. Find the critical points of f on the interior of D .

◦ (a, b) ∈ D such that fx (a, b) = fy (a, b) = 0, or at least one of fx (a, b) and fy (a, b) does
not exist.

Step 2. Find the extreme values of f on the boundary of D .

◦ Suppose y = y(x) on the boundary of D . Then

• f (x, y(x)) is a function in x.
Find its absolute extreme values.

Step 3. Compare the values of f (x, y) at the points obtained in Steps 1 and 2.

15 / 33

Examples
• T (x, y) = x2 + 2y 2 − x on D = {(x, y) | x2 + y 2 ≤ 1}.
Step 1: Find all critical points on x2 + y 2 < 1.
◦ Tx = 2x − 1 and Ty = 4y .
◦ Tx = Ty = 0 ⇒ (x, y) = 21 , 0 .

Step 2: Find the extreme values on x2 + y 2 = 1.

◦ f (x) = x2 + 2(1 − x2 ) − x = −x2 − x + 2, |x| ≤ 1.
• f ′ (x) = −2x − 1. f ′ (x) = 0 ⇒ x = − 12 .
◦ f (−1) = 2, f (1) = 0, f (− 12 ) = 94 .
Step 3: Compare to find absolute extreme values.
◦ T ( 12 , 0) = − 14 , f (− 21 ) = 94 , f (1) = 0.
Conclusion.
√
T (x, y) has the absolute minimum − 14 at ( 12 , 0), and the absolute maximum 9
4
at
(− 12 , ± 23 ).

16 / 33

8
Examples
• f (x, y) = 2 + 2x + 2y − x2 − y 2 on the region unclosed by x = 0, y = 0 and x + y = 9.
y

O 9 x

i) Find critical points in the interior.

ii) Boundary points.

◦ On x-axis;
◦ On y -axis;
◦ On x + y = 9.

17 / 33

Examples
• f (x, y) = 2 + 2x + 2y − x2 − y 2 .
◦ Find critical points in the interior.
• fx = 2 − 2x, fy = 2 − 2y .
• fx = fy = 0 ⇒ (x, y) = (1, 1).
• f (1, 1) = 4.
◦ On the segment (0, 0) to (9, 0).
• f (x, 0) = 2 + 2x − x2 , f ′ (x, 0) = 2 − 2x.
• f ′ (x, 0) = 0 ⇒ x = 1.
• f (0, 0) = 2, f (9, 0) = −61, f (1, 0) = 3.
◦ On the segment (0, 0) to (0, 9).
• f (0, y) = 2 + 2y − y 2 , f ′ (0, y) = 2 − 2y .
• f ′ (0, y) = 0 ⇒ y = 1.
• f (0, 0) = 2, f (0, 9) = −61, f (0, 1) = 3.

18 / 33

9
Examples
• f (x, y) = 2 + 2x + 2y − x2 − y 2 .
◦ On the segment y = 9 − x, 0 ≤ x ≤ 9.
• f (x, 9 − x) = −61 + 18x − 2x2 .
• f ′ (x, 9 − x) = 18 − 4x = 0 ⇒ x =
29 .
• f (9, 0) = f (0, 9) = −61, f 92 , 92 = − 41
2
.
y
b
9

9, 9)
(2 2
b

(1, 1)
b b
1
b b b

O 1 9 x

• Maximum: f (1, 1) = 4,
Minimum: f (0, 9) = f (9, 0) = −61.

19 / 33

Examples
x+y+z √
• Show that ≥ 3 xyz for all x, y, z ≥ 0,
3
Proof. Let A = x + y + z . Then z = A − x − y .
Maximize f (x, y) = xy(A − x − y) on the region unclosed by x = 0, y = 0 and x + y = A.
◦ Critical points on the interior.
• fx = y(A − 2x − y), fy = x(A − x − 2y).
• fx = fy = 0 ⇒ x = y = A3 . (x > 0, y > 0)
3
• f ( A3 , A3 ) = A27 .
◦ Boundary points.
• f (x, y) = xy(A − x − y).
• It is identically 0 on x = 0, y = 0, x + y = A.

20 / 33

10
Examples
x+y+z √
• Show that ≥ 3 xyz for all x, y, z ≥ 0,
3
Proof. Let A = x + y + z . Then z = A − x − y .
Maximize f (x, y) = xy(A − x − y) on the region unclosed by x = 0, y = 0 and x + y = A.
A3
◦ f (x, y) ≤ 27
for all x, y in the region.
(x + y + z)3 √ x+y+z
Then xyz ≤ 3
, i.e., 3 xyz ≤ .
3 3
• Remark. This is a special case of the Arithmetic-Geometric Mean Inequality for n = 3.
◦ Let n ∈ Z+ . For any x1 , x2 , . . . , xn ≥ 0,
x1 + x2 + · · · + xn √
• ≥ n x1 x2 · · · xn .
n
◦ Can you prove it?

21 / 33

Extreme Values with Restriction

• T (x, y) = x2 + 2y 2 − x subject to x2 + y 2 = 1.
◦ Draw level curves T (x, y) = c.
• Increase c until T (x, y) = c touches x2 + y 2 = 1.
T (x, y) has a minimum at the intersection.
• Increase c until T (x, y) = c leaves x2 + y 2 = 1.
T (x, y) has a maximum at the intersection.

22 / 33

11
 Extreme Values with Restriction
• T (x, y) = x2 + 2y 2 − x subject to x2 + y 2 = 1.
◦ Suppose T (x, y) has an extreme value at (x0 , y0 ).
T (x, y) = c and x2 + y 2 − 1 = 0:
• are tangent to each other;
• have the same tangent/normal line;
• ∇T (x0 , y0) k ∇g(x0 , y0), g(x, y) = x2 + y 2 − 1;
• ∇T (x0 , y0) = λ∇g(x0 , y0) for some λ ∈ R.
◦ Tx = 2x − 1, Ty = 4y ; gx = 2x, gy = 2y .
• 2x − 1 = λ2x, 4y = λ2y , x2 + y 2 = 1.
• y = 0 ⇒ x = ±1; √
y 6= 0 ⇒ λ = 2 ⇒ x = − 12 ⇒ y = ± 23 .
√
◦ T (1, 0) = 0, T (−1, 0) = 2, T (− 12 , ± 2
3
) = 94 .
√
• Max: T (− 12 , ± 2
3
) = 94 ; Min: T (1, 0) = 0.

23 / 33

Lagrange Multiplier
• The Method of Lagrange Multipliers
Find the local maximum and minimum values of z = f (x, y) subject to the constraint
g(x, y) = 0.
◦ Evaluate x, y and λ that simultaneously satisfy
• fx = λgx , fy = λgy and g(x, y) = 0.
• Absolute Extreme Values with Bounded Restriction
Maximize/Minimize f (x, y) subject to g(x, y) = 0,
where g(x, y) = 0 is a bounded curve.

Step 1. Check the end points of g(x, y) = 0, if any.

Step 2. Use Lagrange multiplier on interior of g(x, y) = 0.
Step 3. Compare the values of f at points obtained in 1) & 2).

24 / 33

12
Examples
x2 y 2
• Find the area of the largest rectangle inscribed in the ellipse + 2 = 1 (a, b > 0).
y a2 b
b
(x, y)

O a x

◦ Maximize f (x, y) = 4xy subject to

x2 y 2
• g(x, y) = + 2 − 1 = 0, x, y ≥ 0.
a2 b
◦ End points: (x, y) = (a, 0), (0, b).

25 / 33

Examples
x2 y 2
• Find the area of the largest rectangle inscribed in the ellipse + 2 = 1 (a, b > 0).
a2 b
◦ Maximize f (x, y) = 4xy subject to
x2 y 2
• g(x, y) = + 2 − 1 = 0, x, y ≥ 0.
a2 b
◦ Suppose x > 0, y > 0. Apply Lagrange multipliers:
)
fx = λgx ⇒ 4y = λ 2xa2 y x/y x a
2y
⇒ = 2 2 ⇒ = .
fy = λgy ⇒ 4x = λ b2 x a /b y b
a2 k 2 b2 k 2 √1 .
• x = ak & y = bk ⇒ a2
+ b2
=1⇒k = 2
 
• (x, y) = √a2 , √b2 .
 
◦ Min: f (a, 0) = f (0, b) = 0; Max: f √a , √b = 2ab.
2 2

26 / 33

13
 Examples
• Find the shortest distance from point P (x0 , y0 ) to straight line ax + by = c. (Assume the minimum
distance exists.)
p
Solution. d(x, y) = (x − x0 )2 + (y − y0 )2 .
◦ Minimize f (x, y) = (x − x0 )2 + (y − y0 )2
• subject to g(x, y) = ax + by − c = 0.
) (
fx = λgx ⇒ 2(x − x0 ) = λa x = x0 + λ2 a
⇒
fy = λgy ⇒ 2(y − y0 ) = λb y = y0 + λ2 b
λ c − ax0 − by0
a(x0 + λ2 a) + b(y0 + λ2 b) = c ⇒ =
2 a2 + b2
a(c − ax0 − by0 ) b(c − ax0 − by0 )
◦ x = x0 + , y = y0 + .
a2 + b 2 a2 + b 2
◦ The distance is minimized at this point:
r
a(c−ax0 −by0 )
2 
b(c−ax0 −by0 )
2 |ax0 + by0 − c|
a2 +b2
+ a2 +b2
= √
a2 + b2
27 / 33

Lagrange Multiplier of More Variables

• The Method of Lagrange Multipliers of Three-Variables
Find the local maximum and minimum values of w = f (x, y, z) subject to the constraint
g(x, y, z) = 0.
◦ Evaluate x, y and λ that simultaneously satisfy
• fx = λgx , fy = λgy , fz = λgz , g(x, y, z) = 0.
• Absolute Extreme Values with Bounded Restriction
Maximize/Minimize f (x, y, z) subject to g(x, y, z) = 0,
where g(x, y, z) = 0 is a bounded surface.

Step 1. Check max/min on the boundary of g(x, y, z) = 0.

Step 2. Use Lagrange multiplier on interior of g(x, y, z) = 0.
Step 3. Compare the values of f at points obtained in 1) & 2).

28 / 33

14
 Examples
x+y+z √
• Show that ≥ 3 xyz for all x, y, z ≥ 0.
3
Solution. Let A = x + y + z .
◦ Maximize f (x, y, z) = xyz subject to
• g(x, y, z) = x + y + z − A, x, y, z ≥ 0.
◦ Boundary points: x = 0, or y = 0, or z = 0.
• f (x, y, z) is identically zero on the boundary.
◦ Suppose x, y, z > 0 and use Lagrange multiplier.

fx = λgx ⇒ yz = λ   A
fy = λgy ⇒ zx = λ ⇒ x = y = z = .
 3
fz = λgz ⇒ xy = λ

A3
◦ f ( A3 , A3 , A3 ) = 27 is the maximum.
r
√ p
3 3 A3 x+y+z
Therefore, 3 xyz = f (x, y, z) ≤ = .
27 3
29 / 33

Examples
• Find the max/min of f (x, y, z) = ax + by + cz on the unit sphere x2 + y 2 + z 2 = 1, where
a, b, c > 0.
Solution. Let g(x, y, z) = x2 + y 2 + z 2 − 1.
◦ The sphere x2 + y 2 + z 2 = 1 has no boundary.
◦ Apply the Lagrange multipliers
 method:
a
fx = λgx ⇒ a = λ2x   x = 2λ

b
fy = λgy ⇒ b = λ2y ⇒ y = 2λ
  z= c

fz = λgz ⇒ c = λ2z

2λ
√
2 2 2
1 = ( 2λ ) + (2λ ) + ( 2λ ) ⇒ λ = ± a +b2 +c 
a 2 b 2 c 2
.
a
◦ (x, y, z) = ± √
a2 +b2 +c2
, √a2 +bb 2 +c2 , √a2 +bc 2 +c2

a
 √
Max: f √
a2 +b2 +c2
, √a2 +bb 2 +c2 , √a2 +bc 2 +c2 = a2 + b2 + c2
  √
Min: f √ −a
a2 +b2 +c2
−b
, √a2 +b2 +c2 ,
√ −c
a2 +b2 +c2
= − a2 + b2 + c2

30 / 33

15
Extreme Values with Two Constraints
• Lagrange Multipliers Method with Two Constraints
Find the local maximum and minimum values of w = f (x, y, z) subject to the constraints
◦ g(x, y, z) = 0 and h(x, y, z) = 0.
Evaluate x, y, z and λ, µ that simultaneously satisfy
◦ fx = λgx + µhx , fy = λgy + µhy , fz = λgz + µhz .
◦ g(x, y, z) = 0 and h(x, y, z) = 0.

• Idea of the Lagrange multipliers method.

◦ g(x, y, z) = h(x, y, z) = 0 defines a curve, say C .
◦ ∇g and ∇h are normal to C .
◦ We seek for points at which ∇f is normal to C .
• ∇f lies in the plane defined by ∇g and ∇h,
• i.e., ∇f = λ∇g + µ∇h for some λ, µ ∈ R.

31 / 33

Examples
• Suppose x + y + z = 1 and x2 + y 2 + z 2 = 1.
Find the extreme values of f (x, y, z) = x3 + y 3 + z 3 .
Solution. Two restrictions g(x, y, z) = x + y + z − 1 and h(x, y, z) = x2 + y 2 + z 2 − 1.
fx = λgx + µhx ⇒ 3x2 = λ + µ2x
fy = λgy + µhy ⇒ 3y 2 = λ + µ2y
fz = λgz + µhz ⇒ 3z 2 = λ + µ2z
◦ The equation 3α2 = λ + µ2α has at most 2 real roots.
• So x, y, z cannot be all distinct.
◦ Suppose
( y = z . Then( (
1 = x + 2y x=1 x = − 13
⇒ or
1 = x2 + 2y 2 y=z=0 y=z= 2
3

32 / 33

16
Examples
• Suppose x + y + z = 1 and x2 + y 2 + z 2 = 1.
Find the extreme values of f (x, y, z) = x3 + y 3 + z 3 .
Solution. Two restrictions g(x, y, z) = x + y + z − 1 and h(x, y, z) = x2 + y 2 + z 2 − 1.
◦ Suppose
( y = z . Then( (
1 = x + 2y x=1 x = − 13
⇒ or
1 = x2 + 2y 2 y=z=0 y=z= 2
3
2 2
◦ If x = y then (x, y, z) = (0, 0, 1), 3 , 3 , − 13
.

◦ If x = z then (x, y, z) = (0, 1, 0), 23 , − 13 , 23 .

Compare the values of f (x, y, z) at these 6 points.
Max: f (1, 0, 0) =
f (0, 1, 0) = f (0,

0, 1) 2= 21; 1
5
1 2 2 2 1 2
Min: f − 3 , 3 , 3 = f 3 , − 3 , 3 = f 3 , 3 , − 3 = 9 .

33 / 33

17