Scribd
Upload
84 views12 pages

Dynamic Programming and Applications: Daniil Kashkarov, CERGE-EI

This document summarizes a lecture on dynamic programming and applications. It provides an example of a cake-eating problem that is solved using both a direct attack method by setting up a multi-period optimization problem, as well as a dynamic programming approach that breaks the problem down into single-period subproblems. The direct attack yields Euler equations that characterize the optimal consumption path. The dynamic programming approach uses the principle of optimality and value functions to recursively solve for the optimal policy.

Uploaded by

Martin Vallejos
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
84 views12 pages

Dynamic Programming and Applications: Daniil Kashkarov, CERGE-EI

This document summarizes a lecture on dynamic programming and applications. It provides an example of a cake-eating problem that is solved using both a direct attack method by setting up a multi-period optimization problem, as well as a dynamic programming approach that breaks the problem down into single-period subproblems. The direct attack yields Euler equations that characterize the optimal consumption path. The dynamic programming approach uses the principle of optimality and value functions to recursively solve for the optimal policy.

Uploaded by

Martin Vallejos
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
You are on page 1/ 12

Dynamic Programming and Applications

Lecture 3

Daniil Kashkarov, CERGE-EI

April 13, 2020

Daniil Kashkarov, CERGE-EI Dynamic Programming and Applications Lecture 3


Indirect Utility

In consumer choice theory:

V (I , p) = max u(c)
c
subject to pc = I

Where c is the vector of J consumption goods, p is a vector


of prices and I is income
u (c)
The first order conditions are: jpj = λ for j = 1, 2, ..., J,
where λ is the multiplier on the budget constraint and
uj (c) is the marginal utility from good j
V (I , p) is an indirect utility function
Indirect utility function shows the maximized level of utility
from the current state (I , p)

Daniil Kashkarov, CERGE-EI Dynamic Programming and Applications Lecture 3


Indirect Utility

Maximum utility which can be achieved in state (I , p) is given


by the indirect utility function
Give person income I and assume that the person acts
optimally (maximizes utility)
What if we give the consumer a bit more income?
Welfare (utility) goes up by VI (I , p) > 0
Can we predict where this additional income goes? No,
u (c)
since jpj = VI (I , p) ∀j, if consumer acts optimally
Indirect utility function:
summarizes the value of the consumer (household)
optimization problem
gives the marginal value of income without us knowing more
about consumption functions

Daniil Kashkarov, CERGE-EI Dynamic Programming and Applications Lecture 3


Indirect Profit Function

Firm must choose how many workers to hire at a wage of w


given its stock of capital k and product price p
The firm solves:

Π(w , p, k) = max p f (l, k) − w l


l

Solution of the problem gives labour demand that depends


on (w , p, k)
Indirect profit function Π(w , p, k) summarizes the value of the
firm given factor and product prices and capital stock
We can get the the marginal value of allowing the firm some
additional capital as Πk (w , p, k) = p fk (l, k)
We do not need how firm will adjust its labour in response to
additional capital

Daniil Kashkarov, CERGE-EI Dynamic Programming and Applications Lecture 3


Dynamic Optimization: A Cake-Eating Problem

You have cake of size W1


At each point of time (t = 1, 2, 3, ..., T ) you can eat some
part, but must save the rest
Your lifetime utility can be represented as:
T
X
β (t−1) u(ct )
t=1

where u(·) is real valued, differentiable, strictly increasing, and


strictly concave, and limc→0 u 0 (c) → ∞.
β is the discount factor
Evolution of the cake over time (discrete-time system) is:

Wt+1 = Wt − ct

Daniil Kashkarov, CERGE-EI Dynamic Programming and Applications Lecture 3


A Cake-Eating Problem: Direct Attack

Set up the sequence problem:


T
X
max β (t−1) u(ct )
T +1
{ct }T
1 ,{Wt }2 t=1
subject to Wt+1 = Wt − ct ;
ct ≥ 0; Wt ≥ 0, ∀t.

We can rewrite the first constraint as:


T
X
ct + WT +1 = W1
t=1

Let λ be the multiplier on the last constraint. Then the first


order conditions are given by:

β t−1 u 0 (ct ) = λ ∀t

Daniil Kashkarov, CERGE-EI Dynamic Programming and Applications Lecture 3


A Cake-Eating Problem: Direct Attack

Combining β t−1 u 0 (ct ) = λ and β t u 0 (ct ) = λ gives Euler


equation:

u 0 (ct ) = βu 0 (ct+1 )

Euler equation links consumption across any two periods and


is necessary condition for optimality for any t
If Euler equation does not hold for any two periods, we can
increase our lifetime utility by moving consumption from one
period to another

Daniil Kashkarov, CERGE-EI Dynamic Programming and Applications Lecture 3


A Cake-Eating Problem: Direct Attack

Satisfying the Euler equations is necessary, but not


sufficient
What if Euler equations hold, but WT > cT , i.e., there is
some cake left after T.
The non-negativity constraint on WT +1 in the solution
must bind, i.e., optimal sequence of consumption choices
across periods must imply that WT +1 = 0
So, for this problem there is an initial condition (W1 is given)
and a terminal condition (WT +1 = 0)
Then Euler equations determine the time path of consumption

Daniil Kashkarov, CERGE-EI Dynamic Programming and Applications Lecture 3


A Cake-Eating Problem: Direct Attack

Let the solution to the cake-eating problem be denoted as


VT (W1 )
VT (W1 ) is called a value function and represents the
maximum utility which we can get from a T-period problem
given a cake of W1 size
Value function is analogous to the indirect utility functions
An increase in the size of the cake leads to an increase in the
lifetime utility:

VT0 (W1 ) = λ = β t−1 u 0 (ct ), ∀t

It does not matter when the extra cake is eaten given that
consumer is acting optimally (similar to the case with different
consumption goods)

Daniil Kashkarov, CERGE-EI Dynamic Programming and Applications Lecture 3


A Cake-Eating Problem: Dynamic Programming Approach
Add a period 0 and give initial cake of size W0
Recall, dynamic programming approach allows us to convert a
T-period problem into a two period problem (now vs. future)
Suppose we already have VT (W1 ), i.e., the solution of the
T-period problem given W1
Given W0 we consider the problem:
max u(c0 ) + βVT (W1 ),
c0
where W1 = W0 − c0 , W0 is given.
We determine the size of the cake which will be available in
period 1, W1
Instead of choosing a sequence of consumption levels we just
find c0
And then VT (W1 ) completely summarizes optimal behaviour
from period 1
Daniil Kashkarov, CERGE-EI Dynamic Programming and Applications Lecture 3
A Cake-Eating Problem: Dynamic Programming Approach

For dynamic programming it does not matter how the cake


will be consumed after initial period
All that matters is that the agent will be acting optimally and
generating utility VT (W1 )
This is the principle of optimality
The first order condition is given by:

u 0 (c0 ) = βVT0 (W1 )

Marginal gain from reducing c0 is summarized by the


derivative of the value function
But how to construct VT (W1 )? By either solving the
sequence problem (direct attack) or using recursive approach

Daniil Kashkarov, CERGE-EI Dynamic Programming and Applications Lecture 3


A Cake-Eating Problem: Recursive Approach

Suppose that u(c) = ln(c)

1 Start with the single-period problem with V1 (W1 )


2 Add one more period and solve the two-period problem, using
V1 (W1 ) from the previous step to get V2 (W1 )
3 Given V2 (W1 ) move to the three-period problem...
4 Keep iterating to get VT (W1 )

Daniil Kashkarov, CERGE-EI Dynamic Programming and Applications Lecture 3

You might also like

pFad - Phonifier reborn

Pfad - The Proxy pFad of © 2024 Garber Painting. All rights reserved.

Note: This service is not intended for secure transactions such as banking, social media, email, or purchasing. Use at your own risk. We assume no liability whatsoever for broken pages.


Alternative Proxies:

Alternative Proxy

pFad Proxy

pFad v3 Proxy

pFad v4 Proxy