Session 2 – Constrained and

Unconstrained Optimization using the

Hessian and Bordered Hessian

Slide 1
 Session Overview
By the end of this section, you should be able to
• Distinguish between concave and convex functions
• Demonstrate how to use the Hessian and Bordered Hessian
determinant in solving unconstrained and constrained
optimization problems respectively
• Distinguish between constrained and unconstrained
optimization problems
• Understand the role of the Lagrangean multiplier in
constrained optimization
Slide 2
 Session Outline
The key topics to be covered in the session are as
follows:
• Concave and Convex Functions
• Unconstrained optimization using the Hessian
• Constrained Optimization using the Bordered
Hessian

Slide 3
 Reading List
• Dowling, E. T. Introduction to Mathematical
Economics (Second Edition), Schaum’s Outline
Series, McGRAW-HILL. ISBN 0-07-017674-4
• A. C., Chiang. (1984) Fundamental Method of
Mathematical Economics, McGraw Hill
international
• Raymond A. B., Ziegler, M. R, and Byleen K. E.
(2000). Applied Mathematic for Business,
Economics, Life Sciences and Social Sciences.
(Seventh edition).Prentice Hall. ISBN 0-13-
083120-4.
Slide 4
 Topic One
What are Concave and Convex Functions?
• A single variable function is concave (convex) at a
point , if in some region very close to the point
{a, f(a)}, the graph of thefunction lies completely below
(above) the tangent.

Nature of
Concave
Functions

Slide 5
 What are Concave and Convex
Functions? Cont’d

• For Case C, the graph is an increasing function, thus

• If , then the function is convex at Point a.
• The condition determines the convexity of a single
variable function.
• The convexity is associated with the minimum of a function.
Slide 6
 Example 1
• Investigate the concavity or convexity of the function
at the point x = 2
Solution
• Taking the first differential, we obtain
• Taking the second differential, we obtain

• Thus, at x = 2,

• This follows that the function is strictly

convex at the point x = 2.
Slide 7
 Activity 1
• Investigate the concavity or convexity of the function

1.
at the point x = 3.

2. at the point x = 0.5

Slide 8
Concavity and Convexity of a Multivariable Function
• For multivariable functions we use the Hessian determinant to
determine the concavity and convexity.
• Let be a real valued function which is twice differentiable
• Let be the Hessian determinant where

• fij is the second order partial derivative with respect ti i and j

• Then at a point, if the principal minors of the Hessian evaluated at
this point satisfies the condition:
• , then the function is concave
• , then the function is strictly concave
• , then the function is convex
• , then the function is strictly convex
Slide 9
 Example 2
• Investigate the function for concavity or convexity
Solution
• Given
• Taking the 1st order partial derivatives, we obtain:
and
• The 2nd order partial derivatives give:
, and
• Forming the Hessian, we obtain:
• and
• Since the principal minors are all positive, by our definition
is a convex function.
Slide 10
 Activity 2
• Determine the concavity and convexity of the following
functions:
1.
2. 𝑧 = 𝑥0.5𝑦0.3 at the points x = 1 and y = 2

Slide 11
 Topic 2
Unconstrained Optimization using the Hessian
• Conditions for an Extremum (Relative maxima and minima)
Condition Relative Maximum Relative Minimum
First-order zx = 0 and z y = 0 zx = 0 and z y = 0
(Necessary)
Second-order z xx , z yy < 0 and z xx , z yy > 0
(Sufficient) z xx . z yy - ( z xy ) > 0
2
z xx . z yy - ( z xy ) > 0
2

z xx . z yy > ( z xy ) z xx . z yy > ( z xy )
2 2

• It is possible that the 1st and 2nd conditions would be

satisfied but the 3rd would be violated. There are two
possibilities in that case:
• If is neither maximum or minimum
(inconclusive)
Slide 12
 Conditions for Extremum
• If , two situations can arise:
• and both and have the same signs
(either positive or negative), then at those critical points,
the function is said to be at an inflection point.
• and and have different signs (positive
and negative), then the function is said to be at a saddle
point.
• The last conditions have very little economic implication
at our level so we will focus lees on them
• Our emphasis is on the maximum and minimum at this
point Slide 13
 The Hessian Determinant
• The Hessian determinant is a special determinant formed
by arranging the 2nd order partial derivatives in such a
way that the 2nd order direct partial derivatives are on the
principal diagonal and the 2nd order cross partial
derivatives are off the principal diagonal.
• Thus for, , the Hessian will be
• The Hessian provides a convenient test for the 2nd order
condition for optimization.

Slide 14
 Conditions for an Extrema of Multivariable
Function of two variables
• The following condition holds using the Hessian

Condition Relative Maximum Relative Minimum

First-order zx = 0 and z y = 0 zx = 0 and z y = 0
(Necessary)
Second-order H1 <0 H1 >0
(Sufficient) H 2 >0 H 2 >0
The Hessian is said to The Hessian is said to
be negatively definite positively defined

Slide 15
 Example 3
• Find the maximum or minimum points of the function
using the Hessian for the 2nd order condition. What
is the maximum or minimum point?
Solution
• Finding critical values by partially differentiating, we have

• By solving Equations (1) and (2) simultaneously, x = 2 and y = ½

• Thus, the critical values are (2, ½)
• From Equations (1) and (2), we obtain 2nd order partial derivatives

Slide 16
 Solution to Example 3 Cont’d
• We obtain the Hessian determinant as follows:
z xx z xy -2 -2
H2 =
z yx z yy
= -2 -4
• Evaluating the minors of the principal diagonal at the critical points,
H 1 = - 2 < 0 and H 2 =-2(-4)-(-2)(-2) = 4 > 0
• is said to be negatively definite and thus, the function z has a
relatively maximum at the critical point (2, ½).
• To find the maximum point, substitute critical values into the
function . We obtain
! ! ! !#
Z =5 2 + 6 − 2" − 2 2 ( ) −2( )"=
" " " "
• Thus, z = 6.5 Slide 17
 Generalizing the Conditions for the Extremum
unconstrained optimization
• For n independent variables, the Hessian of is
usually referred to as a higher-order Hessian and expressed as:

where etc.

For the principal minors,

Slide 18
 Generalizing the Conditions for the Extremum of
unconstrained optimization –cont’d
Condition Relative Maximum Relative Minimum
First-order f1 = f 2 = ... = f n = 0 f1 = f 2 = ... = f n = 0
(necessary)
Second-order H1 < 0 , H 2 > 0 , H3 < 0 H1 > 0 , H 2 > 0 , H3 > 0
(sufficient) ,….. ,.... .,
….., (±) H n > (<)0 …, H n > 0
H is said to be H is said to be
negatively definite positively definite
Example 4
a. Find the extreme values of the function
and indicate whether it is a maximum or minimum.
b. Find the maximum or minimum value
Slide 19
 Solution to Example 4
a. We find critical values by setting the first partial derivatives to zero

• We solve the system of equations by using the elimination method.

Thus, we we multiply equation (2) by 4 and subtract it from
equation (1) and simplify as follows:

Slide 20
 Solution to Example 4 contd
• Substitute Equation (5) into (3), • Now we substitute into
we obtain: Equation (7) to find

• Now that we know the value of we • Thus the function

substitute it in equation (5) to obtain

Has a critical point at:

Slide 21
 Solution to Example 4 cont’d
• To find maximum or minimum points, we use the Hessian

with Hessian determinant

• Also, we evaluate the terms on the principal minors

• Since all the principal minors have a positive sign, the Hessian
satisfies the conditions for a relative minimum.
• The function z attains a relative minimum at the points

Slide 22
 Solution to Example 4 Cont’d
b. We find the relative minimum value by inserting the
critical points into the function and evaluating as follows:

• Thus, the minimum point z = 1.78

Slide 23
 Activity 3
1. (a) Find the extreme values of the function
and indicate
whether it is a maximum or a minimum.
(b) Find the maximum or minimum value

2. (a) Find the extreme values of the function

and indicate whether it is
a maximum or minimum.
(b) Find the maximum or minimum value
Slide 24
 Topic 3
Constrained Optimization Using the Bordered Hessian
Determinant
• We use the Bordered Hessian determinant to solve
optimization problems with a constraint.
• A classical example is utility maximization of a consumer
with fixed income and prices of commodities.
• For instance, optimizing a function subject to a
constraint
, we form the Lagrangean as:

• The 1st order condition requires that to find critical

points.
• To test the 2nd order condition we use the Bordered Hessian
Slide 25
 The Bordered Hessian
• The Bordered Hessian can be expressed in two ways:

or

• A closer look at the Bordered Hessian shows that it is the

Hessian bordered on the left or right by the 1st order
partial derivatives of the constraint with zero on the
principal diagonal.
• The order of the bordered Hessian is determined by the
Hessian that it bordered.
Slide 26
 Conditions for Extrema of Multivariable
Function with Two Variables under Constraints
• Using the denotations for the bordered Hessian, we formulate
the necessary and sufficient conditions for optimization
Condition Relative Relative Minimum
Maximum
First-order Lx = L y = Ll = 0 Lx = L y = Ll = 0
Second- H2 > 0 H2 < 0
order

• Once the critical points have been obtained, the next point is
to find the maximum value of the function by inserting the
values in the objective function and simplifying.
Slide 27
 Importance of the Lagrangean Multiplier
• The Lagrangean multiplier approximates a change in
the value of the objective function resulting from a
change in the constant of the constraint.
Example 5
a. Find the critical point(s) of the function
subject to the condition
a. What is the maximum/minimum value of the function.

Slide 28
 Solution to Example 5
• We set the Lagrangean function (L) as follows:

• We determine the critical points by setting the 1st order partial

derivatives to zero
• …………………..(1)
• …………………..(2)
• …………………...(3)
• Eliminating by subtracting (2) from (1), we get

Slide 29
 Solution to Example 5 Cont’d
• Thus, subject to the constraint
• Substituting (4) into (3) and
, the function
solving for y, we get
has (42, 30) as
the critical point.

• Put y = 30 into (4) • To determine whether the

point obtained is a maximum
or minimum, we evaluate the
• We substitute the values of x and
bordered Hessian determinant
y into either (1) and (2) to find
the value of

Slide 30
 Solution to Example 5 Cont’d
• Thus,

• With , is a negative definite and the function

attains a relative maximum at the critical point (42, 30).
• We put the values of x and y to find the maximum value

Slide 31
 Generalizing the Conditions for the Extremum
constrained optimization -1
• Suppose we optimize the multivariable function of n independent
variables subject to the constraint
.
• The Hessian of thus multivariable function is usually referred to as
a higher-order bordered Hessian and expressed as:

• Where etc.
Slide 32
 Generalizing the Conditions for the Extremum
constrained optimization Cont’d
• Let us denote the following to be the principal minors

, ……………,

• We can then formulate the necessary and sufficient

conditions for the maximum and minimum of a
multivariable function.
• The Hessian evaluated at the critical values must satisfy
the conditions for a maximum or a minimum as specified
as follows;
Slide 33
Conditions for Extremum using Bordered Hessian

• Condition Relative Maximum Relative Minimum

First-order fl = f1 = f2 = ... = fn = 0 fl = f1 = f2 = ... = fn = 0
Second- H 2 > 0; H 3 < 0; H 4 > 0;... H 2 , H 3 , H 4 ,..., H n < 0
order
, Hn >0 Bordered Hessian is
Bordered Hessian is positive definite
negative definite

Slide 34
 Activity 4
1. (a) Find the critical point(s) of the function
subject to the condition that
(b) What is the maximum/minimum value of the function.
(c) Estimate the value of the maximum value if the constant is
increased by 1.
2. Find the critical point(s) of the function subject to the
condition that
3. Determine if the Cobb Douglas production function is
convex or concave at and
4. Determine the nature of the extremum of the function

Slide 35
 Activity 4
5. Optimize the function subject to the constraint

Slide 36