Albert N.

SHIRYAEV
Steklov Mathematical Institute,
Lomonosov Moscow State University

LECTURES on
Financial
Statistics, Stochastics, and Optimization

e-mail: albertsh@mi.ras.ru

1
TOPIC I. Financial models and innovations in stochastic
economics
1. The classical and neoclassical models of the dynamics of
the prices driven by Brownian motion and Lévy processes
2. Stylized facts
3. Constructions based on the change of time, stochastic
volatility

TOPIC II. Technical Analysis. I

1. Kagi and Renko charts
2. Prediction of time of maximum value of prices observable
on time interval [0, T ]
3. Quickest detection of time of appearing of arbitrage
4. Drawdowns as the characteristics of risk

TOPIC III. Technical Analysis. II

1. Buy and Hold
2. Stochastic Buy an Hold (rebalancing of the portfolio)

TOPIC IV. General Theory of Optimal Stopping 2

TOPIC I. Financial models and innovations
in stochastic economics
1. The classical and neoclassical models of the
dynamics of the prices driven by Brownian
motion and Lévy processes
2. Stylized facts

3. Constructions based on the change of time,

stochastic volatility

I-1
The construction of the right probability-statistical models of the
dynamics of prices of the basic financial instruments (bank account,
bonds, stocks, etc.) is undoubtedly one of important steps for successful
application of the results of mathematical finance and financial
engineering.

Without adequate models for prices there is no successful risk management

portfolio optimization, allocation of funds, derivative pricing, etc.

The main accent in this lecture is made on the construction of the

HYPERBOLIC LÉVY PROCESSES,

which are widely used in econometric models of the dynamics of the
financial indexes.

I-2
 THE FIRST CLASSICAL MODELS FOR
PRICE DYNAMICS

In the sequel,

S = (St )t≥0 is the price of (for simplicity) one asset

L. Bachelier (1900). Théorie de la spéculation:

St = S0 + µt + σBt ,

where B = (Bt )t≥0 is a standard Brownian motion, i.e.,

a Gaussian process with independent
increments and continuous trajectories,
B0 = 0, EBt = 0, E(Bt − Bs)2 = t − s.

I-3
M. Kendall (1953). The analysis of economic time series.
Part 1. Prices (J. Roy. Statist. Soc., 96, 11–25):

The empirical analysis of prices S = (Sn)n≥0 for

– wheat (monthly average prices on the Chicago market, 1883–1934),
– cotton (the New York Mercantile Exchange, 1816–1951)
did not reveal (contrary to common expectations) neither rhythms,
nor cycles. The observed data look as if

“...the Demon of Chance drew a random number... and

added it to the current price to determine the next... price” :

Sn = S0eHn , where Hn = h1 + · · · + hn is the sum of

independent random variables

(“ random walk hypothesis”)

I-4
M. F. M. Osborne (1959). Brownian motion in the stock market.
Operation Research, 7, 145–153: St = S0 + µt + σBt .

P. A. Samuelson (1965). Proof that properly anticipated prices

fluctuate randomly. Industrial Management Rev., 6, 41–49:

! σ2 "
St = S0eHt , Ht = µ − t + σBt ,
2

S = (St)t≥0 is an economic (geometric) Brownian motion;

dSt = St(µ dt + σ dBt )

#This model underlies the Black–Scholes theory of option pricing.$

I-5
MARTINGALE APPROACH TO STUDYING THE MODELS
S = (Sn)n≥0, Sn = S0eHn , Hn = h1 + · · · + hn
(hn = log(Sn/Sn−1) is a “return”, “logarithmic return”)

Doob’s decomposition. Assume that “stochastics” of the market

is described by a filtered probability space (Ω, F , (Fn )n≥0, P) and
E|Hn| < ∞, n ≥ 0.
n
# n !
# "
Hn = E(hk | Fk−1) + hk − E(hk | Fk−1) , or
k=1 k=1

hn = E(hn | Fn−1) + (hn − E(hn | Fn−1))

$ %& ' $ %& '
µn δn
are Fn−1 -measurable are Fn-measurable,
E(δn | Fn−1 ) = 0,
(δn) is a martingale difference
I-6
 1970s : linear models like AR, MA, ARMA with
#large time intervals –
year, quarter, month$ hn = µn + σn εn (i.e., δn = σn εn),

где µn and σn are Fn−1-measurable,

εn ∼ N (0, 1) are independent, n ≥ 1.

AR(p) model:
µn = a0 + a1hn−1 + · · · + aphn−p, σn = const

MA(q) model:
µn = b0 + b1εn−1 + · · · + bq εn−q , σn = const

ARMA(p, q) model:
( )
µn = a0 + a1hn−1 + · · · + aphn−p
( )
+ b0 + b1εn−1 + · · · + bq εn−q , σn = const
I-7
 1980s : nonlinear models ARCH, GARCH, CRR
#analysis of day data$
Sn = S0 exp{h1 + · · · + hn}

ARCH(p) model – AutoRegressive Conditional Heteroskedastic

model; P. Engle (1982):
*
+ p
+ #
hn = σnεn , σn = ,α0 + αih2
n−i is random (!).
i=1
GARCH(p, q) model – Generalized ARCH model; T. Bollerslev
(1986):
*
+- p . - q .
+ # #
+ 2 2
hn = σnεn , σn = , α0 + αihn−i + βj σn−j .
i=1 j=1
Binary CRR-model – Cox, Ross, Rubinstein (1979):
hn = log(1 + ρn), ρn takes two values, ρn > −1.

I-8
 1990s : (a) Stochastic processes with discrete
#intraday data analysis$ intervention of chance (piecewise
constant trajectories with jumps at
“close” times τ1, τ2, . . .):
#
Ht = hk I(τk ≤ t)

(b) Data come almost continuously.

I-9
WEAKNESS of the MODEL St = S0eHt , Ht = (µ−σ 2/2)t+σBt ,
based on a Brownian motion, i.e., dSt = St(µ dt + σ dBt ) with a
constant volatility σ.

Really observable smile effect says that the volatility σ is NOT a

constant.

Consider a call (buyer) option with pay-off function (ST − K)+:

( )
+
C(t, x) = EP̃ (ST − K) | St = x .
By the Black–Scholes formula we find C(t, x) = CBS(t, x; T, K, σ).
On option market there exist real prices
/
C(t, x; T, K).
/
From CBS(t, x; T, K, σ) ≈ C(t, x; T, K) we calculate the implied volatility
σ̃ = σ̃(t, x; T, K). Fix t, x, T . It turns out that σ̃(K) has a U -form
(with Kmin ≈ x) – smile effect.
I-10
1st CORRECTION
(R. Merton, 1973)

σ −→ σ(t)
σ(t) −→ σ(t, St)

2nd CORRECTION
(B. Dupire, 1994)
Pricing with a smile,
RISK, 7, 18–20

I-11
One- and two-dimensional distributions of H = (Ht )t≥0.
(∆)
The observable properties of ht = log(St/St−∆)

A. The behavior of empirical densities p̂(∆) (x), constructed upon

(∆) (∆)
h∆ , h2∆ , . . ., is different from that of normal distribution. In
a neighborhood of the central value, the densities p̂(∆)(x) are
peak-like, and “heavy tails” are observed as x → ±∞.
B. The empirical estimator of autocorrelation (t = k∆)
0 1203 1
(∆) (∆) (∆) (∆)
ρ(n∆) = Eht ht+n∆ − Eht Eht+n∆ Dh(∆)
t D h
(∆)
t+n∆

shows that for small n∆ the value ρ(n∆) is negative, while most
of the values of ρ(n∆) are close to zero (noncorrelatedness).
C. Analogous estimators for autocorrelation of absolute values
(∆) (∆)
|ht | and |ht+n∆| show that for small n∆ the autocorrelation
is positive (clustering effect).
I-12
HERE THE PICTURE

I-13
Searching for adequate statistical models which describe
dynamics of the prices S = (St)t≥0 led to

LÉVY PROCESSES.
Now these processes take the central place in modelling the
prices of financial indexes, the latter displaying the jump
character of changes.

I-14
 MAIN MODELS
based on a Brownian motion

Exponential
Brownian model
St = S0 exp{µt + σBt}

↓ ↓
Exponential
Exponential
INTEGRAL
TIME-CHANGED
Brownian model:
45 t 5 t 6 Brownian model:
7 8
St = S0 exp µs ds + σs dBs St = S0 exp µT (t) + BT (t)
0 0

I-15
Assuming that µ = 0, one can rewrite these models in a brief form:

S = S0eσB

↓ ↓
S = S0eσ·B S = S0eB◦T
5 ·
where • σ · B is the stochastic integral ( σs dBs ),
0
• B ◦ T is a time change in Brownian motion (BT (t) ).

I-16
A generalization of these “Brownian” models, which have being
predominating in financial modelling for a long time, is based on
the idea to replace

BROWNIAN MOTION by LÉVY PROCESSES :

B = (Bt )t≥0 L = (Lt )t≥0

S = S0eσL

↓ ↓
S = S0eσ·L S = S0eL◦T

I-17
LÉVY PROCESS L = (Lt )t≥0 is a process with stationary
increments, L0 = 0, which is continuous in probability.

Such processes have modifications whose trajectories

• are right-continuous (for t ≥ 0) and
• have limits from the left (for t > 0).

Kolmogorov-Lévy-Khinchin’s formula for characteristic functions:

9 0 5 ! 1:
λ2 "
EeiλLt = exp t iλb − c+ eiλx − 1 − iλh(x) F (dx) ,
2

where: h(x) = xI(|x| ≤ 1) (classical “truncation” function),

F (dx) is a σ-finite measure on R \ {0}
;
such that min(1, x2) F (dx) < ∞,
b ∈ R and c ≥ 0;
(b, c, F ) =: T is a triplet of local characteristics of L.
I-18
The Lévy–Itô representation for trajectories of L = (Lt )t≥0:
5 t5 5 t5
Lt = Bt + Lct + h(x) d(µ − ν) + (x − h(x)) dµ ,
0 0

• Bt = bt; • Lct is a continuous component of L

√
(Lct = c Wt, where Wt is a Wiener process);
• µ is the measure of jumps: for A ∈ B(R \ {0})
#
µ(ω; (0, t] × A) = I (∆Ls )
0<s≤t A
(∆Ls = Ls − Ls−);
• ν is the compensator of the measure of jumps µ:
;
ν((0, t] × A) = tF (A), F (A) = A F (dx).

The measure µ is a Poissonian measure with

4 # 6 4# 6
iλk
E exp i λ µ(Gk ) = exp
k≤n k k≤n
(e − 1)ν(Gk ) , n ≥ 1,

where Gk are sets from R+ × R and ν(dt, dx) = dt F (dx).

I-19
EXAMPLES of LÉVY PROCESSES :

• Brownian motion,
• Poisson process,
Nt
#
• compound Poisson process Lt = ξk , where
k=1
(Nt )t≥0 is a Poisson process,
(ξk )k≥1 is a sequence of independent and identically
distributed random variables

I-20
In connection with financial econometrics, these are

HYPERBOLIC Lévy processes,

that are of a great interest, because they model well the really
observable processes H = (Ht)t≥0 for many underlying financial
instruments (rate of exchange, stocks, etc.).

The credit of developing the theory of such processes and their

applications is due to E. Halphen, O. Barndorff-Nielsen, E. Eberlein.

We will construct these processes, following mostly Chapters 9

and 12 of the monograph: O. Barndorff-Nielsen, A. N. Shiryaev,
Change of Time and Change of Measures, World Scientific (in
print).

I-21
For a Lévy process (Ht)t≥0 we have

EeiλHt = (EeiλH1 )t.

The properties of Lévy’s processes imply that the random variable
h = H1 is infinitely divisible, i.e., for any n one can find i.i.d. r.v.’s
ξ1, . . . , ξn such that

Law(h) = Law(ξ1 + · · · + ξn).

We will look for the infinitely divisible r.v.’s h having the form

h = µ + βσ 2 + σε,
where ε is a standard Gaussian random variable, ε ∼ N (0, 1),
σ = σ(ω) is the “volatility” (which does not depend on ε), for
whose square, σ 2, we will construct the special distribution

GIG – Generalized Inverse Gaussian distribution.

I-22
Strikingly, this distribution (on R+ ) is infinitely divisible and the
distribution of h = µ + βσ 2 + σε (on R) is infinitely divisible as well.
Hence there exist Lévy processes T = (T (t))t≥0 and H ∗ = (Ht∗)t≥0
such that

Law(T (1)) = Law(σ 2) and Law(H1∗) = Law(h).

As a realization of H ∗ = (Ht∗)t≥0 one can take

Ht = µt + βT (t) + BT (t) ,
where the “time change” T = (T (t))t≥0 and the Brownian motion
B = (Bt )t≥0 are independent.

In the sequel, we do not distinguish between the processes H and H ∗.

This process H, remarkable in many respect, bears the name

L(GH) – Generalized Hyperbolic Lévy process.

I-23
Let discuss the details of construction of GIG-distributions for σ 2.

Let W = (Wt)t≥0 be a Wiener process (standard Brownian motion).

For A ≥ 0, B > 0 introduce

T A (B) = inf{s ≥ 0 : As + Ws ≥ B}.

HERE MUST BE A PICTURE

I-24
The formula for the density pT A(B) (s) = dP(T A (B) ≤ s)/ds is well
known:
B 1 2
pT A (B)(s) = ϕs(B − As), где ϕs(x) = √ e−x /(2s). (1)
s 2πs
Herefrom we find the Laplace transform:
7 < 8
−λT A (B) 2
Ee = exp AB(1 − 1 + 2λ/A ) .

Letting b = B 2 > 0 and a = A2 ≥ 0, we find from (1) the following

formula for p(s; a, b) = pT √a(√b)(s):
=
b √ab
p(s; a, b) = c1(a, b)s−3/2e−(as+b/s)/2 , где c1(a, b) = e .
2π

The distribution with density p(s; a, b) is named

IG = IG(a, b) – Inverse Gaussian distribution.

I-25
Next important step: one define ad hoc the function

p(s; a, b, ν) = c2(a, b, ν) sν−1e−(as+b/s)/2, (2)

where parameters a, b, ν ∈ R are chosen in such a way that p(s; a, b, ν)
be probability density on R+ :
a ≥ 0, b > 0, ν<0 > 5 ∞ ?
a > 0, b > 0, ν=0 ⇒ sν−1e−(as+b/s)/2 ds < ∞ .
0
a > 0, b ≥ 0, ν>0
5 ∞
ν−1 −y(s+1/s)/2
It is well known that Kν (y) ≡ 1
2 0 s e ds is the modified
third-kind Bessel function of order ν, which for y > 0 solves

y 2f 22 (y) + yf 2(y) − (y 2 + ν 2)f (y) = 0.

(a/b)ν/2
The constant in (2) has the form c2(a, b, ν) = √ .
2Kν ( ab)

I-26
The distribution on R+ with density

(a/b)ν/2 ν−1 −(as+b/s)/2

p(s; a, b, ν) = √ s e
2Kν ( ab)
bears the name

GIG = GIG(a, b) – Generalized Inverse Gaussian distribution.

I-27
IMPORTANT PROPERTIES of GIG-DISTRIBUTION (for σ 2):

A. This distribution is infinitely divisible.

B. The density p(s; a, b, ν) is unimodal with the mode

 ( )

b / 2(1 − ν) , if a = 0,
m= ( < )
 2
 (ν − 1) + ab + (ν − 1) /a, if a > 0.
; ∞ −λs
C. Laplace’s transform L(λ) = 0 e p(s; a, b, ν) ds is given by
<
D E
2λ −ν/2 Kν ( ab(1 + 2λ/a))
L(λ) = 1 + √ .
a Kν ( ab)
As a by-product, one deduces the representation for the density f (y)
;
of Lévy measure F (dy). (Note: L(λ) = exp{ 0∞(e−λy − 1)f (y) dy}.)

I-28
Particularly important SPECIAL CASES of GIG-distributions:

I. a ! 0, b > 0, ν = −1/2: in this case GIG(a, b, −1/2) = IG(a, b)

– Inverse Gaussian distribution. < √
b e ab.
Density: p(s; a, b) = c1(a, b)s−3/2e−(as+b/s)/2, c1(a, b) = 2π
<
Density of Lévy’s measure: f (y) = b e−ay/2 .
2π y 3/2

II. a > 0, b = 0, ν > 0: in this case GIG(a, 0, ν) = Gamma(a/2, ν)

(a/2)ν ν−1 −as/2
– Gamma distribution. Density: p(s; a, 0, ν) = Γ(ν) s e .
Density of Lévy’s measure: f (y) = y −1νe−ay/2.
<
a/b
III. a > 0, b > 0, ν = 1: p(s; a, b, 1) = √ e−(as+b/s)/2
2K1( ab)
– PH – Positive Hyperbolic distribution, or H+-distribution.

I-29
Since GIG-distribution is infinitely divisible, if one take it as the
distribution of squared volatility σ 2,

Law(σ 2) = GIG,
then one can construct a nonnegative nonincreasing Lévy process
T = (T (t))t≥0 (a subordinator) such that

Law(T (1)) = Law(σ 2) = GIG.

In the subsequent constructions, this process plays the role of

change of time, operational time,

business time.

As was explained above, the next step in construction of the return

process H = (Ht)t≥0, consists in the following.

I-30
Form the variable h = µ+βσ 2 +σε, where Law(σ 2) = GIG, Law(ε) =
N (0, 1), σ 2 and ε are independent. It is clear that

Law(h) = Eσ 2 N (µ + βσ 2, σ 2)
is a mixture of normal distributions, i.e., the density ph(x) of h is of
the form
5 ∞ 9 :
1 (x − (µ + βy))2
ph(x) = √ exp − pGIG (y) dy.
0 2πy 2y
Integrating and denoting ph(x) by p∗(x; a, b, µ, β, ν), we find that
! < "
∗
Kν−1/2 α b + (x − µ)2
p (x; a, b, µ, β, ν) = c3(a, b, β, ν) !< "1/2−ν eβ(x−µ) ,
b + (x − µ) 2

< 1
(a/b)ν/2 α 2 −ν
where α = a + β 2 and c3(a, b, β, ν) = √ √ .
2π Kν ( ab)
I-31
The obtained distribution Law(h) with density p∗(x; a, b, µ, β, ν) bears
the name

Generalized Hyperbolic distribution, GH = GH(a, b, µ, β, ν)).

In the case ν = 1 the graph of the function

<
log p∗(x; a, b, µ, β, 1) = log c∗(a, b, β) − α b + (x − µ)2 + β(x − µ)
is a hyperbola with asymptotes α|x − µ| + β(x − µ).

This is why the distribution for h in the case ν = 1 is called

hyperbolic, which explains the name “generalized hyperbolic distribution”
in the case of arbitrary ν.

I-32
SOME PROPERTIES of GH-distribution (for h):

A*. This distribution is infinitely divisible.

B*. If β = 0, then the distribution is unimodal with mode m = µ.

(In the general case m is determined as a root
of a certain transcendental equation.)
5 ∞
C*. Laplace’s transform L∗(λ) = eλx p(x; a, b, µ, β, ν) dx
0 <
(for complex λ such that |β + λ| < α, α = a + β 2)
is given by
<
0 1ν/2
a Kν ( b[α2 − (β + λ)2])
L∗(λ) = eλµ √ .
α2 − (β + λ)2 Kν ( ab)

I-33
THREE important SPECIAL CASES of GH-distributions:

I*. a ! 0, b > 0, ν = −1/2: in this case

GIG(a, b, −1/2) = IG(a, b) is Inverse Gaussian distribution.

The corresponding GH-distribution is commonly named

Normal Inverse Gaussian

and denoted by N ◦ IG. The density has the form
<
∗ 1 αb √ab K1(α b + (x−µ)2) β(x−µ)
p (x; a, b, µ, β, − 2 ) = e < e , x ∈ R.
π b + (x−µ)2
Laplace’s transform:
7 √ √ < 8
∗ 2
L (λ) = exp λµ + b( a − a − 2βλ − λ ) .

I-34
II*. a > 0, b = 0, ν > 0: in this case

GIG(a, 0, ν) = Gamma(a/2, ν) – Gamma distribution.

The corresponding GH-distribution is named

Normal Gamma distribution

(notation: N ◦ Gamma) or

VG-distribution
(notation: VG [Variance Gamma]). Density:

∗ aν ν−1/2
p (x; a, 0, µ, β, ν) = √ |x − µ|
πΓ(ν)(2α)ν−1/2 .
× Kν−1/2(α|x − µ|) eβ(x−µ)

Laplace’s transform: L∗(λ) = eµλ(a/[a − 2βλ − λ2])ν .

I-35
III*. a > 0, b > 0, ν = 1: in this case

GIG(a, b, 1) = H+(a, b) – Positive hyperbolic distribution.

The corresponding GH-distribution for h is the hyperbolic distribution
H called also

“ NORMAL positive hyperbolic distribution”

(notation: H or N ◦ H+ ). Density:
4 < 6
a
p∗(x; a, b, µ, β, 1) = √ exp −α b + (x − µ)2 + β(x − µ) ,
2bαK1( ab)
<
where α = a + β 2.

I-36
Having GIG-distributions for σ 2 and GH-distributions for h, we turn
to construction of the Lévy process H = (Ht )t≥0 used in representation
of the prices St = S0eHt , t ≥ 0.
TWO POSSIBILITIES
↓ ↓
the fact that h has infinitely using the already constructed
divisible distribution allows process T = (Tt)t≥0, one forms the
one to construct, using the process H = (Ht )t≥0:
general theory, the Lévy
∗ ∗ Ht = µt + βT (t) + BT (t) ,
process H = (Ht )t≥0 such
that where Brownian motion B and
∗
process T are taken to be
Law(H1) = Law(h)
independent.

The process H = (Ht)t≥0 bears the name

L(GH) – “ GENERALIZED hyperbolic Lévy distribution”.

I-37
In the cases I*, II*, and III* mentioned above the corresponding
Lévy processes have the special names:

L(N ◦ IG)-process,
L(N ◦ H+ )- or L(H)-process,
L(N ◦ Gamma)- or L(VG)-process.
It is interesting to note that L(N ◦ IG)- and L(N ◦ Gamma)-processes
have an important property:

Law(Ht ) belongs to the same type of distributions as Law(H1)

(this follows immediately from the formulae for Laplace’s transforms).

I-38
CONCLUDING NOTES.

Densities of distributions of h (= H1) are determined by FIVE

parameters (a, b, µ, β, ν), that gives a great freedom in determining
parameters which would fit well the empirical data.

In this connection it is appropriate to recall that in statistics, there

exists a well-known method of “Pearson’s curves”, which is widely
used to construct (one-dimensional) densities of distributions upon
independent observations over a random variable ξ. K. Pearson itself
(1894) constructed such densities as solutions f (x) of the system
of nonlinear equations
(x − a)f (x)
f 2 (x) = 2
.
b0 + b1x + b2x

I-39
These densities are determined by FOUR parameters (a, b0, b1, b2).

The density p∗(x; a, b, µ, β, ν) of GH-distributions of (constructively

built) variables h = µ + βσ 2 + σε is determined by FIVE parameters.
(It is known that these densities lie between Pearson’s curves of
type III and type V.)

The essential advantage of GH-distributions consists in their

infinite divisibility
(this is not the case for distributions from the Pearson system),
which enables us to construct processes H = (Ht )t≥0 which describe
adequately the time dynamics of logarithmic return of the prices
S = (St)t≥0.

I-40
TOPIC II. Technical Analysis. I
1. Kagi and Renko charts
2. Prediction of time of maximum value of prices
observable on time interval [0, T ]
3. Quickest detection of time of appearing of
arbitrage
4. Drawdowns as the characteristics of risk

II-1
The main motivation of this lecture is based on idea to obtain a
mathematical explanation of some practical methods (“when to buy,
when to sell”, etc.) of the Technical Analysis which have as usual
only a descriptive character.

As is well known, the “fundamentalists” are trying to explain

WHY the stock price moves;

they make their decisions by looking at the state of the “economy

at large”; they define a stock value and calculate proper stock prices
in view of its estimated future values; they build their analysis upon
the assumption that the actions of market operators are “rational”.

II-2
As to the “technicians” they concentrate on the local peculiarities
of the markets, they emphasize “mass behavior”, “market moods”;
they start their analysis from an idea that stock price movement
is “the product of supply and demand”; their basic concept is the
following: the analysis of past stock prices helps us to see future
prices because past prices take future prices into account; they try
to explain

HOW the stock prices move.

II-3
 1. Kagi and Renko charts

Let X = (Xt )t≥0 be a stock price.

The Japanese “Kagi chart” and “Renko chart” (also called the price
ranges) give methods to forecast price trends from price changes
which exceed either a certain range H or a certain rate H. The price
range or rate H is determined in advance. (In Japan, popular price
ranges are "5, 10, 20, 50, 100, 200.) Greater price ranges are used
for stocks with higher prices because their upward and downward
movements are larger.

II-4
 R → |X|, K → max X − X
RENKO construction: Step I: We construct (ρ/i ):

ρ/0 = 0,
7 8
ρ/n+1 = inf t > ρ/n : |Xt − Xρ/n | = H , n ≥ 1.

!
X ρ/7,,
-
4H !

ρ/2 ρ/6 ,, ρ/8

#
'
# -
3H ! ! !

2H !
(
)
! ! !

&% ρ/3 (( #
$
# ρ/9
% ρ/1 ρ/5
H ! !
#
$ +
*
#
ρ/0 ρ/4 *
t"
II-5
Step II: Construction (ρ/n) −→ (ρm, ρ∗m).
We look at all ρ/n such that
! "! "
Xρ/n − Xρ/n−1 Xρ/n−1 − Xρ/n−2 < 0.

ρm is a Markov time
X ρ∗m−1 ρ∗m is a non-Markov time

ρm

ρm−1

ρ∗m
t
II-6
 4 6
KAGI construction: κ0 = inf u > 0 : max X − min X = H
[0,u] [0,u]
 4 6


inf u ∈ [0, κ0] : Xu = min X
 if Xκ0 = max X
[0,κ0] [0,κ0]
κ0∗ = 4 6


inf u ∈ [0, κ0] : Xu = max X
 if Xκ0 = min X
[0,κ0] [0,κ0]

X X

H H

t t
κ0∗ κ0 κ0∗ κ0
II-7
Next step: we define by induction
 4 6





inf u > κn : max X − Xu = H if Xκn − Xκn∗ = H
[xn,u]
κn+1 = 4 6



inf u > κn : max X − Xu = H
 if Xκn − Xκn∗ = −H
[xn,u]

 4 6



 inf u ∈ [κn, κn+1] : Xu = max X if Xκn − Xκn∗ = H
 [κn,κn+1]
∗
κn+1 = 4 6


inf u ∈ [κn, κn+1] : Xu = [κ min if Xκn − Xκn∗ = −H

 X
n,κn+1]

II-8
Kagi and Renko variation (on [0, T ]):
N
#
KT (X; H) = |Xκn∗ −Xκ ∗ |, N = NT (X; H),
n−1
n=1
#M
RT (X; H) = |Xρ∗n −Xρ∗ |, M = MT (X; H).
n−1
n=1

Kagi and Renko volatilities (on [0, T ]):

KT (X; H)
kT (X; H) = ,
MT (X; H)
R (X; H)
rT (X; H) = T .
MT (X; H)

II-9
THEOREM. If X = σB, then
P
1) kT (σB; H) ∼ 2H, 2) rT (σB; H) ∼ 2H,
T σ2 T σ2
NT ∼ (P-a.s.), MT ∼ (P-a.s.),
H2 2H 2
2 2
P 2T σ P Tσ
KT = kT NT ∼ 2
; RT = rT MT ∼ .
H H

II-10
 Results of the statistical analysis
of some stock prices

X = (Xt )t≥0 — Future on Index SP500 (Emini-SP500 Futures)

1 point = $ 50

2002-2003 (471 trading days)

∆ = 1 sec, Xt is the value of the last transaction at time t.

H 1 1.25 1.5 2 2.25 2.5 3 4

rT (X;H) RENKO
H
1.83 1.84 1.86 1.88 1.86 1.88 1.80 1.69

H 1 1.25 1.5 2 2.25 2.5 3 4

kT (X;H) KAGI
H
1.83 1.85 1.85 1.89 1.91 1.93 1.92 1.87

Almost the same results are valid for Futures on Index Nasdaq 100
(Emini-Nasdag100 Futures), 1 point = $ 20
II-11
 3

2.5
Kagi
2

1.5
Renko
1

0.5

0.75 1.25 1.75 2.25 2.75 3.25 3.75 4.25

1 1.5 2 2.5 3 3.5 4 4.5

For EESR (United Energy System of

Russia)
rT (X; H) k (X; H)
∼ 1.99 ∼ T .
H H II-12
Let us say that X-market has

r(H)-property if E rT (X; H) ∼ r(H) · H

k(H)-property if E kT (X; H) ∼ k(H) · H

(For a Brownian motion r(H) = k(H) = 2.)

II-13
Define Renko strategy γ R = (γtR )t≥0 with
# ! "
γtR = sgn Xρn−1 − Xρ∗ I[ρn−1,ρ∗ (t)
n−1 n−1 )
n≥1
! "
× I(k(H) ≥ 2) − I(k(H) < 2) , t ≥ 0,
and the corresponding capital
5 t 5 t
γR
Ct = γuR dXu − λ |dγuR |.
0 0

Then
γR
C
lim E t = |r(H) − 2| · H − 2λ.
t→∞ Mt

The similar result is valid for the Kagi strategy γ K = (γtK )t≥0.

II-14
 X ρ∗m−1

ρm

ρm−1

ρ∗m
t

If R(H) > 2, then If R(H) < 2, then

we buy at ρm−2, ρm, . . . we buy at ρm−1, ρm+1, . . .
we sell at ρm−1, ρm+1, . . . we sell at ρm−2, ρm, . . .
(↑↑) (↓↓) (↑↓) (↓↑)

II-15
2. Prediction of time of maximum value of prices observable
on time interval [0, T ]

We would like to present now several our probability and statistical

approaches to solving some other problems of the technical analysis.

Problem. When to sell stock optimally?

!
We shall describe prices
"
10
by a Brownian motion Bθ 1 0
1 0
1 *# !
0*
B = (Bt)0≤t≤1; θ is a 1 # *%
#* %
1
%
point of maximum of B: ! 1 %
/0 1
/ 0 1
Bθ = max Bt . / 01
0≤t≤1 /
/
..#
#/
.
.
.
τ τ t"
θ 1
II-16
Suppose that we begin to observe this process at time t = 0
(“morning time”), and, using only past observations, we stop at
time τ declaring “alarm” about selling. It is very natural to try to
solve the following problem: to find “optimal” times τ ∗ and τ ∗∗ such
that either
inf E |Bθ − Bτ |2 = E |Bθ − Bτ ∗ |2
0≤τ ≤1
or
inf E |θ − τ | = E |θ − τ ∗∗|.
0≤τ ≤1

For us it was a little bit surprising that here the optimal stopping
times coincide: τ ∗∗ = τ ∗. The solution shows that
4 √ 6
τ ∗ = inf t ≤ 1 : max Bs − Bt ≥ z∗ 1 − t ,
s≤t
where z∗ is a certain (known) constant (z∗ = 1.12 . . .).
II-17
This problem belongs to the theory of optimal stopping and method
of its solution is based on reducing to the special free-boundary
problem.

1
t z √1- t
*

t
τ* 1

It is interesting to note that

E τ ∗ = 0.55 . . . , Dτ ∗ = 0.05 . . . .
II-18
 µ
The cases Bt = µt + Bt instead of Bt are more complicated.
If µ > 0 and µ is away from 0, then
µ µ
τ ∗ = inf{t ≤ 1 : b1(t) ≤ St − Bt ≤ b2(t)}
µ µ µ
where • Bt = µt + Bt , St = maxu≤t Bu ,
• b1(t) and b2(t) have the following form:

γ2
#Here C is
the area of
b2
C C
continuation of
D
b1 observations, D
γ1 is the stopping
area.$
0 u t T
* *

II-19
If µ > 0 and µ is close to 0, then the corresponding picture has the
following form:

C
γ2
C
b2

D
γ1 b1
C

u s 0 T
* *

II-20
For µ < 0 and if µ is far from 0, the picture is as follows:

D
b1
C

γ1

0 T

II-21
In the considered problem, the time θ is a “change point” of the
changing of the directions of trend
Solution of the problem

Bt “ inf E |Bτ − Bθ |2 ”
τ
or the problem “ inf τ E |τ −θ|”
depends, of course, on the
construction at any time t
a “good” prediction of the
change point θ. The natural
θ 1 t estimate of θ should be
based on the a posteriori
probability πt = P(θ ≤ t | FtB ), where FtB = σ(Bs, s ≤ t).

II-22
Stochastic analysis shows that
> ?
St − Bt
πt = 2ϕ √ − 1, St = max Bu ,
1−t u≤t
that explains appearing of the expression
St − Bt
√
1−t
which is involved above in the definition of optimal stopping time
9 :
St − Bt
τ ∗ = inf t ≤ 1 : √ ≥ z∗ .
1−t
Statistics St − Bt is appearing in many problems of the financial
mathematics and financial engineering (and, generally, in the mathematical
statistics under name CUSUM statistics).

Now we are going to tackle the following problem, which is interesting,

e.g., from the point of view of the quickest detection of arbitrage.
II-23
 3. Quickest detection of time of appearing of arbitrage

Problem. Suppose we observe the prices

Xt = r(t−θ)+ +σBt
or

σ dB , t ≤ θ,
t
dXt =
r dt + σ dBt , t > θ.
θ
Here a “change point” θ is considered as a time of appearing of
arbitrage. (Brownian motion’s prices correspond to the non-arbitrage
situation. Brownian motion with drift corresponds to a case of
arbitrage.)

One very difficult question here is “what is θ?”. There are two
approaches. In the first one we assume that θ is a random variable.
II-24
Suppose that τ is time of “alarm” θ. Consider two events
{τ < θ} and {τ ≥ θ}.
The set {τ < θ} is the event of a false alarm with a (false alarm)
probability P(τ < θ).
From a financial point of view
{τ <θ} an interesting characteristic of
& '$ %
# #
θ # the event {τ ≥ θ} is a delay
$ %& 'τ
τ time E(τ −θ | τ ≥ θ) or E(τ −θ)+.
{τ ≥θ}

These considerations lead to the following problem: in the class

Mα = {τ : P(τ < θ) ≤ α}, i.e., in the class of stopping times with
the probability of false alarm P(τ < θ) which less or equal the fixed
level α, one need to find optimal stopping τα∗ ∈ Mα such that
inf E(τ − θ)+ = E(τα∗ − θ)+.
τ ∈Mα

II-25
It turned out that it is not a simple problem if we consider an
arbitrary distribution for θ. However, there exists one case when we
may solve this problem in implicit form. This case is the following.

Assume that θ has the exponential distribution:

P(θ = 0) = π and P(θ > t | θ > 0) = e−λt,
where λ is a given positive constant and π ∈ [0, 1). This assumption
is very reasonable. Indeed, for A < a < b < B
! " |b − a|
lim P θ ∈ (a, b) | θ ∈ (A, B) = .
λ→0 |B − A|
It means that in limit (λ → 0) the conditional distribution of θ is
uniform, that is, in some sense the worst possible case from point
of view of uncertainty of time of appearing of a change point θ.

We describe now the results about structure of the optimal stopping

time τα∗ .
II-26
Denote πt = P(θ ≤ t | FtX ), where FtX = σ(Xs , s ≤ t). This process
satisfies the following nonlinear stochastic differential equation:
D E
r2 2 r
dπt = λ − 2 πt (1 − πt) dt + 2 πt(1 − πt) dXτ
σ σ
with π0 = π.

Then it turns out that an optimal stopping time τα∗ is given by

τα∗ = inf{t : πt ≥ Bα
∗
},
where (for case π = 0, for simplicity)
∗
Bα = 1 − α.
Second formulation of the quickest detection of arbitrage assumes
that θ is simply a parameter from [0, ∞). In this case we denote by
Pθ the distribution of the process X under the assumption that a
change point is occurred at time θ.
II-27
By P∞ we denote the distribution of X under assumption that there
is no change point at all. Denote for given T > 0
MT = {τ : E∞ τ ≤ T }
the class of stopping time for which the mean time E∞ τ before
(false) alarm is less or equal to T .

Put also
( )
+
C(T ) = inf sup ess sup Eθ (τ − θ) | Fθ (ω).
τ ∈MT θ ω
We proved that for each T > 0 in the class MT there exists an
optimal strategy with the following structure: declare alarm at time
4 6
τT∗ = inf t : max Xu − Xt ≥ a∗(T ) ,
u≤t
where a∗(T ) is a certain constant. It is interesting to note that (if
r2/(2σ 2) = 1)
C(T ) ∼ log T, T → ∞.
II-28
The given method, based on the

“CUSUM statistics max X − X”,

also is asymptotically optimal for more tractable criteria

D(T ) = inf sup Eθ (τ − θ | τ ≥ θ)

τ ∈M T θ
(We don’t know what is an optimal method for D(T )-criterion.)
Asymptotically, again

D(T ) ∼ log T, T → ∞.

II-29
 4. Drawdowns as the characteristics of risk

From given above exposition we observe importance of the “max X −

X”-characteristics for taking optimal decisions. Now we would like to
discuss that statistics and related ones in the problems of measure
of risk. There is a special terminology for such an object which
related to words “drawdown”, or “downfall”.

In practice a “drawdown” on time interval [0, t] is defined as the

percent change in a manager’s net asset value

– from any newly established peak to

a subsequent through,
– from a high “water mark” to the next
low “water mark”.

II-30
From the theoretical point of view,
Drawdown is a statistical measure of risk for investments; a
competitor to the standard measure of risk such
as return probability, VaR, Sharpe ration, etc.
There are various definitions of drawdown’s characteristics, which
measure the decline in net asset value from the historic high point.
In one financial paper we read that
. . . Measuring risk through extreme losses is a very
appealing idea. This is indeed how financial companies
perceive risks. This explains the popularity of loss
statistics as the maximum drawdown and maximum
loss. . .
and
. . . it does not seem possible to derive exact results for
the expected maximum drawdown.
II-31
Looking forward:

• What kinds of drawdowns should we expect over any given investment

horizon?
• How many drawdowns should be experienced?
• How big?

Under the
Commodity Futures Trading Commission’s
(CFTC)
mandatory disclosure regime managed futures advisors are obliged
to disclose, as part of their capsule performance records, their
“ worst peak-to-valley drawdown”.
We shall demonstrate here some our theoretical calculations related
to drawdowns.
II-32
 µ
Let Bt = µt + σWt be a Brownian motion with drift, W0 = 0.
There are several interesting characteristics related to

Range,
Drawdowns, !

Downfalls,. . . max Bs
µ
s≤t !

Range: µ
TH
"
µ
Rt = max Bsµ − min Bsµ
s≤t s≤t H
µ
Statistics TH for B µ: 2

4 6
µ µ
TH = inf t ≥ 0 : max Bsµ − Bt ≥ H
s≤t II-33
If µ = 0:
> ?2
0 = H
E TH , E max Bt0 = H,
σ 0
t≤TH
> ?4 > ?
0 2 H 0
−λTH 1 H√
DTH = , Ee = 2λ .
3 σ cosh σ

If µ 4= 0:
- 4 6 .
µ σ2 2µ 2µ
E TH = exp H −1− 2H ,
2µ2 σ2 σ
- 4 6 .
σ 2 2µ
µ
E max µ
Bt = exp 2
H −1 .
t≤TH 2µ σ

II-34
Towards a problem from Kolmogorov’s diary (1944):
...For free (or not) random walk: How Xt drops when Xt falls for
the first time (on (t − ∆, t)) from above to some level ξ? To all
appearance, certainly very steeply!..
(H) (H)
Bt = H + Bt , B0 = H, Bt = Bt0 !
H
(H) (H)
τ = inf{u : Bu = 0}
! "
(H)
F (t) = P τ ≤ t | mins≤T Bs ≤ 0

dF (t)
f (t) =
dt √
H T 2
= √ t−3/2e−H /2t , t ≤ T, 0 !
T
"
2G(H/ T )
5 ∞ τ (H)
−u2 /2
G(x) = e du
x II-35
The following three characteristics of drawdowns are the most important:

Maximum
1 drawdown
−→ Dt = max (Bs − Bs2 )
0≤s≤s2≤t

(cf. Rt = max0≤s,s2≤t(Bs − Bs2 ); so Dt ≤ Rt).

B
Dt

" II-36

t
 Drawdown from Dt = Bσt − min Bs2
high “water σt≤s2≤t
2 mark” to the −→
next low “water = max Bs − min Bs2
mark” 0≤s≤t σt≤s2≤t

(where σt = inf{s ≤ t : Bs = max Bu}.

u≤t

II-37
 Drawdown from
previous high
3 “water mark” −→ D t = max Bs − Bσ 2
to the lowest 0≤s≤σt2 t
“water mark”

(where σt2 = inf{s ≤ t : Bs = minu≤t Bu}.

!
B

Dt

" II-38

σt2 t
General results on D , D , D for B :
(1) Dt = Dt
(2) Dt = max(D t, Dt)
law
(3) Dt = max |Bs|
s≤t
law
(4) Dt = max |Bs|
gt≤s≤t
where gt = sup{s ≤ t : Bs = 0}.

II-39
Distributional results on D1, D 1 for a standard Brownian motion
B = B ◦:
D ∞ E 4 6
4 # (−1)n π 2(2n+1)2
(5) P(D1≤x) = P max |Bs|≤x = exp −
s≤1 π n=0 2n+1 8x2
<
E D1 = E max |Bs| = π = 1.2533 . . .
s≤1 2

√ <π
E Dt = σ t 2 (for σB ◦ on [0, t])

D E
(6) P(D 1 ≤ x) = P max |Bs| ≤ x = FD (x)
g1≤s≤1 1

dFD (x) < ∞

#
1 8 k−1 −1 2 2
2k x
fD (x) = = π (−1) ke
1 dx k=1
<
E D1 = 8 log 2 = 1.1061
π
II-40
Note that
∞
8 # 2 2
fR1 (x) = √ (−1)k−1k2e−k x /2, x > 0,
2π k=1
D E
FK1 (x) = P max |bs| ≤ x ,
s≤1
dFK1 (x) ∞
#
k−1 2 −k2 x2
fK1 (x) = = 8x (−1) k e ,
dx k=1

where b = (bs)s≤1 is a Brownian bridge (bs = Bs − sB1).

< <
Since fR1 (x) = 2 1 f (x), we have E R1 = 8 = 1.5957 . . . and
π x K1 π

E D1 ≤ ≤ E R1
E D1
= 3 =
8 π 8
log 2 ≤ ≤
π 2 π
1.1061 . . . ≤ 1.2533 . . . ≤ 1.5957 . . .

II-41
LEMMA.
law
(1) Dt = Dt

 Dt = Dt on {σt ≤ σt2 }
(2) Dt =

max(Dt, Dt) on {σt > σt2 }

(3) max(Dt, Dt) = Dt ≤ Rt

Known results about Rt and Td

R = R1: t = 1, µ = 0, σ = 1

dP(R ≤ x)
W. Feller (1951) got for fR (x) = , x > 0, the following
dx
formula:
∞ k 2 x2
8 # k−1 2 −
fR (x) = √ (−1) k e 2 .
2π k=1

II-42
REMARK. If b(t) = Bt − tB1, t ≤ 1, !is a Brownian" bridge, then for
Kolmogorov’s distribution FK(x) = P sup |b(t)| ≤ x we have
t≤1
∞ √ ∞
# 2π #
−2k2 x2 −(2k−1)2 π 2 /x2
FK(x) = 1 − 2 (−1)k−1e = e
k=1
x k=1

⇓ (θ-function)
∞
# 2 2
fK(x) = 8x (−1)k−1k2e−2k x .
k=1
∞ k 2 x2
8 # k−1 2 −
Since fR (x) = √ (−1) k e 2 , we get
2π k=1
=
21
fR(x) = fK(x),
π x
so
=
8
ER = (= 1.5957691216 . . .)
π
II-43
THEOREM. (t = 1, µ = 0, σ = 1)

law
(a) D1 = max |Bt|
0≤t≤1

(b) If FD1 (x) = P(D1 ≤ x) then (it is well known)

#∞ x 5
(y+4kx)2
0 1
(y+2x+4kx)2
1
FD1 (x) = 1 − √ e− 2 − e− 2 dy
2π k=−∞ −x
∞ 2 (2n+1)2
4 # (−1)n − π
= e 8x2
π n=0 2n + 1

<
(c) E D1 = E max |Bt| = π
0≤t≤1 2
√ < π)
(E DT = σ T 2

II-44
Proof. (a): Denote
5t
1
Mt = max Bs, Lt = lim I(|Bs | ≤ ε) ds.
s≤t ε↓0 2ε
0

By Lévy’s theorem
law
(Mt − Bt, Mt; t ≤ 1) = (|Bt|, Lt ; t ≤ 1).

Hence
! "
D1 = max (Bs −Bs2 ) = max max Bs − Bs2
0≤s≤s2≤1 0≤s2≤1 0≤s≤s2
law
= max (Ms2 − Bs2 ) = max |Bt|.
0≤s2≤1 0≤t≤1

II-45
Proof. (c): We give two proofs. Let β = (βt)t≥0 be a Brownian
motion. From self-similarity
law
(βat; t ≥ 0) = (a1/2βt; t ≥ 0).

So if s1 = inf{t ≥ 0 : |βt| = 1}, then

! " ! " ! "
P sup |βt| ≤ x = P sup |βt/x2 | ≤ 1 = P sup |βt| ≤ 1
0≤t≤1 t≤1 t≤1/x2
D E D E
1 1
= P s1 ≥ 2 =P √ ≤x ,
x s1
i.e.,
law 1
sup |βt| = √ .
t≤1 s1

II-46
 The normal distribution property
=
5∞ 2
2 − x2
e 2σ dx = σ
π
0
⇓
=
5∞ x2 s1
1 2
E D = E sup |βt| = E √ = E e− 2 dx
0≤t≤1 s1 π
0

II-47
 1
We have E e−λs1 = √ . Hence
cosh 2λ
= =
5 5∞
2 ∞ dx 2 ex dx
ED= =2
π 0 cosh x π e2x + 1
0
= = F∞
5∞ F
2 dy 2 F
=2 2
=2 arctan(x)F
π 1+y π F
1 1
= 3 3
2π π π
=2 = ⇒ ED=
π4 2 2

II-48
 3
π
Second proof of the equality E D = is based on the fact that
2
5 1
law 1 du
sup |βt| = ,
t≤1 2 0 (2)
Ru
(2)
where Rs is a Bessel-2:
5 s
(2) 1 du
Rs = βGs + .
2 0 Ru(2)

Thus,
< 3
π
E D = E sup |βt| = E R1(2) = E ξ12 + ξ22 = ,
2
ξ1⊥
⊥ ξ2, ξi ∼ N (0, 1).

II-49
THEOREM. (t = 1, µ = 0, σ = 1, D1 = Bσ1 − minσ1≤s2≤1 Bs2 )

law
(a) D1 = sup |Bs|, where g1 = sup{t ≤ 1 : Bt = 0}.
g1 ≤s≤1
=
∞ 2 2
8 # k−1 − k 2x
(b) fD (x) = (−1) ke , x > 0.
1 π k=1
=
8
(c) E D1 = log 2 (= 1.1061 . . .),
π

E D1 ≤ E D1 ≤ ER
< < <
8 log 2 ≤ π ≤ 8
π 2 π

1.1061... ≤ 1.2533... ≤ 1.5957...

II-50
Proof: By Lévy’s theorem

> ? > ?
Mt − Bt , Mt, Bt ; law |Bt|, Lt, Lt − |Bt |;
=
t≤1 t≤1
⇓
> ? > ?
Mt − Bt , Mt, Bt ; law |Bt|, Lt, Lt − |Bt |;
=
σ1 ≤ t ≤ 1 g1 ≤ t ≤ 1

7 8
where σ1 = min s ≤ 1 : Bs = max Bu .
u≤1

II-51
Therefore
! "
Bσ1 , max (Mt − Bt − Mt)
σ1≤t≤1
law ! "
= Lg1 − |Bg1 |, max (|Bt| − Lt)
g1≤t≤1
! "
= Lg1 , max |Bt| − Lg1
g1≤t≤1

(since Bg1 = 0 and Lt = Lg1 for g1 ≤ t ≤ 1).

Finally,

D1 = Bσ1 − min Bt = Bσ1 + max (−Bt)

σ1 ≤t≤1 σ1 ≤t≤1

= Bσ1 + max (Mt − Bt − Mt)

σ1 ≤t≤1
law
= Lg1 + max |Bt| − Lg1 = max |Bt|.
g1 ≤t≤1 g1≤t≤1

II-52
TOPIC III. Technical Analysis. II
1. Buy and Hold

2. Stochastic Buy an Hold

(rebalancing of the portfolio)

III-1-1
TOPIC III. 1: BUY & HOLD

Consider (B, S)-market. For example: dB = rB dt

dS = S(µ dt + σ dWt)
St
Pt = , MT = max Pt, mT = min Pt.
Bt t≤T t≤T

PROBLEMS:
D E D E
Sτ Sτ
Buying: (1) inf E U Selling: (1) sup E U
τ ≤T MT τ ≤T MT
D E D E
Sτ Sτ
(2) inf E U (2) sup E U
τ ≤T mT τ ≤T mT
where U = U (x) is a “utility function”.

III-1-2
Interesting cases: U (x) = x, U (x) = log x

For U (x) = x :

D E D E
Sτ M T − Sτ
(1): inf EU ∼ sup EU
τ ≤T MT τ ≤T MT

↑
Maximization of the expected relative
error between the buying price and the
highest possible stock price by choosing
a proper time to buy

(2): Maximization of the expected relative error between

the buying price and the lowest possible stock price

III-1-3
 For U (x) = log x :

Pτ Pτ ∗
T
E log = E log Pτ − E log MT , E log = E log Pτ ∗ − E log MT .
MT MT T
4D E 6
σ2
Take Pt = exp µ−r− t + σWt . Then
2
0D E 1
σ2
E log Pτ =E µ−r− τ + σWτ ,
2
0D E 1
σ2
sup E log Pτ = sup E µ − r − τ + σWτ
τ ≤T τ ≤T 2
0D E 1
σ2
= sup E µ − r − τ .
τ ≤T 2

0, µ − r − σ 2/2 ≤ 0,
Thus, τT∗ = (∗)
T, µ − r − σ 2/2 > 0.

III-1-4
Problem (1) for U (x) = x is more difficult.
However, it is remarkable that answer is the same:

Pτ Pτ ∗
sup E =E T, where τT∗ is given by (∗).
τ ≤T MT MT

Pτ
The first step of solving the nonstandard problem sup E is its
τ ≤T M T
reduction to the standard problem of the type

V = sup EG(Xτ )
τ ≤T
for some Markov process X and Markov time τ (with respect to
(FtX )t≥0, FtX = σ(Xs , s ≤ t)).

III-1-5
Taking E( · | Fτ ), Fτ = FτS , where FtS = σ(Su, u ≤ t), we find
D F E
Pτ Pτ FF ν − W ν ),
E = EE Fτ = E G(τ, M
MT MT F τ τ

where

ν = µ − r − σ 2/2, Wtν = νt + σWt, Mtν = max Wu,

0≤u≤t
( )
−x −MTν −t
G(t, x) = E e ∨e , t ∈ [0, T ], x ∈ [0, ∞).
For a given ν, the process Xt = Mtν − Wtν is a Markov process.

III-1-6
So the problem sup E(Pτ /MT ) is reduced to sup EG(τ, Xτ ).
τ ≤T τ ≤T

For G(t, x) we can find explicit expressions:

D E
2(ν − 1) −(ν−1/2)(T −t) −x + (ν − 1)(T − t)
G(t, x) = e Φ √
2ν − 1 T −t
D E
1 −x − ν(T − t)
+ e−(1−2ν)xΦ √
2ν − 1 T −t
D E
−x x − ν(T − t) 1
+e Φ √ , if ν %= ,
T −t 2
D E D E
T −t −x − (T − t)/2
G(t, x) = 1 + x + Φ √
=
2 T −t
4 6
T −t (x + (T − t)/2)2
− exp −
2π 2(T − t)
D E
x − (T − t)/2 1
+ e−xΦ √ , if ν = .
T −t 2
III-1-7
Solution of the problem “ supt≤T EG(τ, Xτ )”:

By dynamic programming methods we must find for 0 ≤ t ≤ T

and x ∈ [0, ∞)
x
V (t, x) = sup ExG(t + τ, Xt+τ ), where Xtx = x ∨ Mtν − Wtν .
τ ≤T −t
It is clear that V (0, 0) = supτ ≤T E(Pτ /MT ).

Introduce the sets

7 8
D = (t, x) ∈ [0, T ] × [0, ∞) : V (t, x) = G(t, x) ,
7 8
C = (t, x) ∈ [0, T ) × [0, ∞) : V (t, x) > G(t, x) ;
the set D is a set of stopping observation and
the set C is a continuation area.

III-1-8
To find V = V (t, x) and sets C and D we usually consider the Stefan
(free-boundary) problem: to find VG = VG (t, x), C,
G and DH such that

LxVG (t, x) = 0, G
(t, x) ∈ C, VG (t, x) = G(t, x), H
(t, x) ∈ D,
where Lx is the infinitesimal operator of the process X x = (Xtx )t≥0,
x ≥ 0. We know that

Law(Xtx, t ≤ T ) = Law(|Ytx|, t ≤ T ),
where Y x is a “bang-bang” process:

dYtx = −ν sgn Ytx dt + dW

I,
t Y0x = x.

III-1-9
By the Tanaka formula
5 t 5 t
|Ytx| = |x|+ sgn Ysx dYsx +Lt(Y x) = |x|−νt+ sgn Ysx dW
I +L (Y x),
s t
0 0
where Lt(Y x) is the local time of Y x at zero over time interval [0, t].

Since Law(X u ) = Law(|Y x|), from the previous formula we find that

∂f ∂f 1 ∂ 2f
Lxf (t, x) = −ν + 2
, t ∈ (0, T ), x ∈ (0, ∞),
∂t ∂x 2 ∂x
1,2 ∂f
for f ∈ C with reflection condition (t, 0+) = 0.
∂x

Free-boundary method can be applied here. However, it is more

convenient to use the following “direct” method.

III-1-10
If G(t, x) ∈ C 1,2, then, by the Itô formula,
5 s
G(t + s, |Ysx|) = G(t, s) + LxG(t + u, |Yux|) du
0
5 s 5 s
∂G ∂G
+ (t + u, |Yux|) dW
I +
u (t + u, |Yux|) dLu (Y x) .
0 ∂x 0 ∂x
$ %& ' $ %& '
is a martingale, = 0,
where −1 ≤ ∂G
∂x ≤ 0 since ∂f
∂x (t, 0+) = 0

So, V (t, x) = sup ExG(t + τ, Xτx) = sup ExG(t + τ, |Yτx|)

τ ≤T −t τ ≤T −t
5 τ
= G(t, x) + sup Ex H(t + u, |Yux|) du
τ ≤T −t 0
5 τ
= G(t, x) + sup Ex H(t + u, Xux) du,
τ ≤T −t 0

∂G ∂G 1 ∂ 2G
where H(t, x) ≡ LxG(t, x) = −ν + 2
.
∂t ∂x 2 ∂x
III-1-11
From explicit formulae for G = G(t, x) we find that
D E
1 ∂G
H(t, x) = ν − G(t, x) − (t, x).
2 ∂x
If ν ≥ 1/2, then ∂G/∂x ≤ 0 and H(t, x) ≥ 0.
5 τ
From V (t, x) = G(t, x)+ sup Ex H(t+u, Xux) du we conclude that
τ ≤T −t 0

if ν ≥ 1/2, then V (t, x) ≥ G(t, x)

if ν > 1/2, then V (t, x) > G(t, x).

III-1-12
As a result we see that
if ν > 1/2, then the optimal τT∗ equals T ;
if ν = 1/2, then any time τT∗ = t is optimal.

For ν < −1/2 we find that

D E D E
1 ∂G
H(t, x) = ν + G(t, x) − G(t, x) + (t, x) .
2 ∂x
$ %& '
>0
So, in this case V (t, x) ≤ G(t, x). From here we conclude that

if ν < −1/2, then the optimal τT∗ equals zero

The case −1/2 < ν < 1/2 is more complicated and can be investigated
by the “free-boundary methods”.
III-1-13
So, we have the following result:

the optimal stopping time τT∗ is DETERMINISTIC:


0, if ν ≡ µ − r − σ 2/2 ≤ 0,
τT∗ =
T, if ν ≡ µ − r − σ 2/2 > 0.

III-1-14
POPULAR STOCK MODELS IN FINANCE:

St = S0eµ(t)+σ(t)Bt (exponential Brownian model);

;t
µ(t)+ 0 σ(s)dBs
St = S0 e (stochastic volatility model);

St = S0eµ(t)+BT (t) (change-of-time model).

Instead of bT we can take a Lévy process Lt .

The similar model can be considered for case of discrete time.

III-1-15
For example, let’s take Gaussian-Inverse Gaussian process

Ht = µt + βT (t) + BT (t) ,
√
where / + √as ≥ bt},
T (t) = inf{s > 0 : Bs
(B/ ) and (B ) are independent Brownian motions.
s s

For case
4 6
St
sup E , St = eHt , MT = exp sup Ht
τ ≤T M T t≤T
we find that

if µ ≥ 0, β ≥ 0, then the optimal stopping time is τT∗ = T

III-1-16
TOPIC III. 2: STOCHASTIC “BUY & HOLD”
(PORTFOLIO REBALANCING)

III-2-1
TOPIC IV. General Theory of Optimal Stopping
Lecture 1. Introduction, pp. 2–46.

Lectures 2-3. Theory of optimal stopping for discrete time

(finite and infinite horizons)
A) Martingale approach B) Markovian approach
(pp. 47–85) (pp. 86–104)

Lectures 4-5. Theory of optimal stopping for continuous time

(finite and infinite horizons)
A) Martingale approach B) Markovian approach
(pp. 105–119) (pp. 120–146)

Essential references p. 147.

IV-1-1
Lecture 1. INTRODUCTION.

1. Connections of the Optimal stopping theory and the Mathematical

analysis (especially PDE-theory) are as well illustrated by the

Dirichlet problem for the Laplace equation:

to find a harmonic function u = u(x) in the class C 2 in the
bounded open domain C ⊆ Rd, i.e., to find a function u ∈ C 2
that satisfies the equation

∆u = 0, x ∈ C, (∗)
and the boundary condition

u(x) = G(x), x ∈ ∂D, where D = Rd \ C. (∗∗)

IV-1-2
Let
τD = inf{t : Btx ∈ D},
where
Btx = x + Bt
and B = (Bt )t≥0 is a d-dimensional standard Brownian motion.

Then the probabilistic solution of the Dirichlet problem

∆u = 0, x ∈ C,
u(x) = G(x), x ∈ ∂D,
is given by the formula

u(x) = EG(BτxD ), x ∈ C ∪ ∂D
D E
u(x) = ExG(BτD ) .

IV-1-3
The optimal stopping theory operates with the optimization
problems, where
7 8
• we have a set of domains C = C : C ⊆ Rd and
• we want to find the function

U (x) = sup ExG(BτD ) , where G = G(x) is given for all x ∈ Rd,

τD J K
D ∈ D = D = C̄ : C ∈ C
or, generally, to find the function

V (x) = sup ExG(Bτ ) , where τ is an arbitrary finite

τ
stopping time defined by the
process B.

IV-1-4
2. The following scheme illustrates the kind of concrete problems
of general interest that will be studied in the courses of lectures:

A. Theory of probability B. Mathematical statistics

sharp inequalities sequential analysis

C. Financial mathematics
stochastic equilibria

The solution method for problems A, B, C consists in reformulation

to an optimal stopping problem and reduction to a free-boundary
problem as stated in the diagram:

IV-1-5
 A, B, C
#$ #$
" $
1 4
!" !"
#!
Optimal stopping problems
#$ #$
" $
2 3
!" !"
#!
Free-boundary problems

IV-1-6
3. To get some idea of the character of problems A, B, C that will
be studied, let us begin with the following remarks.

(A) Let B = (Bt )t≥0 be a standard Brownian motion. Then

√
EBT = 0 and EBτ = 0 if E τ < ∞
Wald identities:
EBT2 = T and EBτ2 = Eτ if Eτ < ∞

From Jensen’s inequality and E|Bτ |2 = Eτ we get

E|Bτ |p ≤ (Eτ )p/2 for 0<p≤2

E|Bτ |p ≥ (Eτ )p/2 for 2≤p<∞

√
B. Davis (1976): E|Bτ | ≤ z1∗ E τ , z1∗ = 1.30693 . . .

IV-1-7
Now our main interest relates with the estimation of the expectations

E max Bt and E max |Bt|.

t≤τ t≤τ
We have
law
max B = |B|.
So,
=
2
E max Bt = E|BT | = T
t≤T π
and
 √
 Eτ ,
E max Bt = E|Bτ | ≤
 ∗ √
t≤τ z1 E τ , z1∗ = 1.30993 . . .

IV-1-8
The case of max |B| is more difficult. We know that

> ? ∞ > ?
4 # (−1)n π 2(2n + 1)2
P max |Bt| ≤ x = exp − .
t≤T π n=0 2n + 1 8x2

From here it is possible to obtain (but it is not easy!) that

3 ! "
π
E max |Bt| = T = 1.25331 . . . .
t≤T 2

<
(Recall that E|BT | = 2T (= 0.79788 . . .).)
π

IV-1-9
SIMPLE PROOF:
law √
(Bat ; t ≥ 0) = ( aBt; t ≥ 0).

Take σ = inf {t > 0 : |Bt| = 1}. Then

D E D E
P sup |Bt| ≤ x = P sup |Bt/x2 | ≤ 1
0≤t≤1 0≤t≤1
D E D E > ?
1 1
=P sup |Bt| ≤ 1 = P σ ≥ 2 =P √ ≤x ,
0≤t≤1/x2 x σ
that is,
law 1
sup |Bt | = √
0≤t≤1 σ

IV-1-10
The normal distribution property:
=
5
2 ∞ − x22
Ee 2a dx = a , a > 0. (∗)
π 0
So,
=
5
1 (∗) 2 ∞ − x2 σ
E sup |Bt| = E √ = Ee 2 dx.
0≤t≤1 σ π 0

Since Ee−λσ = 1√ , we get

cosh 2λ
= = =
5 5 5
2 ∞ dx 2 ∞ ex dx 2 ∞ dy
E sup |Bt| = = 2 =
0≤t≤1 π 0 cosh x π 0 e2 x + 1 π 1 1 + y2
= F∞ = 3
2 F 2 π π
F
= 2 arctan(x)F = 2 · = .
π F π 4 2
1

IV-1-11
 3 3
π π
E sup |Bt| = E sup |Bt| = T
0≤t≤1 2 0≤t≤T 2

In connection with MAX the following can be interesting. In his

speech delivered in 1856 before a grand meeting at the St.-Petersburg
University the great mathematician
P. L. Chebyshev (1821–1894)
has formulated some statements about the “unity of theory and
practice” . In particular he emphasized that

“a large portion of the practical questions can be stated in the

form of problems of MAXIMUM and MINIMUM... Only the
solution of these problems can satisfy the requests of practice
which is always in search of the best and the most efficient.”
IV-1-12
4. Suppose that instead of maxt≤T |Bt|, where, as already known,
3
π
E max |Bt| = T ,
0≤t≤T 2
we have some random time τ and we want to find

E max |Bt| = ?
0≤t≤τ

It is clear that it is virtually impossible

• to compute this expectation for every stopping time τ of B.

Thus, as the second best thing, one can try

• to bound it with a quantity which is easier computed.

A natural candidate for the latter is Eτ at least when finite.
In this way a P ROBLEM A has appeared.
IV-1-13
Problem A leads to the following maximal inequality:
D E√
E max |Bt| ≤ C Eτ (3)
0≤t≤τ

which is valid
√ for all stopping times τ of B with the best constant
C equal to 2.

We will see that the problem A can be solved in the form (3) by
REFORMULATION to the following optimal stopping problem:
D E
V∗ = sup E max |Bt| − cτ , (4)
τ 0≤t≤τ

where
• the supremum is taken over all stopping times τ of B
satisfying Eτ < ∞, and
• the constant c > 0 is given and fixed.
It constitutes Step 1 in the diagram above.
IV-1-14
If V∗ = V∗(c) can be computed, then from (4) we get
D E
E max |Bt| ≤ V∗(c) + c Eτ (5)
0≤t≤τ

for all stopping times τ of B and all c > 0. Hence we find

D E ! "
E max |Bt| ≤ inf V∗(c) + c Eτ . (6)
0≤t≤τ c>0
for all stopping times τ of B. The RHS in (6) defines a function of
Eτ that, in view of (4), provides a sharp bound of the LHS.

Our lectures demonstrate that the

optimal stopping free-boundary
can be reduced to a
problem (4) problem

This constitutes Step 2 in the diagram above.

IV-1-15
Solving the free-boundary problem one finds that V∗(c) = 1/2c.
Inserting this into (6) yields
! " √
inf E V∗(c) + c Eτ = 2 Eτ (7)
c>0
so that the inequality (6) reads as follows:
D E √
E max |Bt| ≤ 2 Eτ (8)
0≤t≤τ

for all stopping times τ of B.

√
This is exactly the inequality (3) above with C = 2.
√
The constant 2 is the best possible in (8).

IV-1-16
In the lectures we consider similar sharp inequalities for other stochastic
processes using ramifications of the method just exposed.

Apart from being able to

• derive sharp versions of known inequalities

the method can also be used to

• derive some new inequalities.

IV-1-17
(B) Classic examples of problems in SEQUENTIAL ANALYSIS:

• WALD’s problem (“Sequential analysis”, 1947) of sequential

testing of two statistical hypotheses

H0 : µ = µ0 and H1 : µ = µ1 (9)
about the drift parameter µ ∈ R of the observed process

Xt = µt + Bt , t ≥ 0, where B = (Bt )t≥0 is (10)

a standard Brownian
motion.

• The problem of sequential testing of two statistical hypotheses

H0 : λ = λ0 and H1 : λ = λ1 (11)

IV-1-18
about the intensity parameter λ > 0 of the observed process

Xt = Ntλ , t ≥ 0, where N = (Nt )t≥0 is a (12)

standard Poisson process.
The basic problem in both cases seeks to find the

optimal decision rule (τ∗, d∗)

in the class ∆(α, β) consisting of decision rules

(d, τ ), where τ is the time of stopping and

accepting H1 if d = d1 or
accepting H0 if d = d0,

such that the probability errors of the first and second kind satisfy:

P(accept H1 | true H0 ) ≤ α (13)

P(accept H0 | true H1 ) ≤ β (14)
and the mean times of observation E0τ and E1τ are as small as
possible.
It is assumed that α > 0 and β > 0 with α + β < 1.
IV-1-19
It turns out that with this (variational) problem

one may associate an optimal stopping (Bayesian) problem

which in turn can be reduced to a free-boundary problem .

This constitutes Steps 1 and 2 in the diagram above.

Solving the free-boundary problem leads to an optimal decision rule

(τ∗, d∗) in the class ∆(α, β) satisfying (13) and (14) as well as the
following two identities:

E0τ = inf E0τ (15)

(τ,d)
E1τ = inf E1τ (16)
(τ,d)
where the infimum is taken over all decision rules (τ, d) in ∆(α, β).
This constitutes Steps 3 and 4 in the diagram above.

IV-1-20
In our lectures we study these as well as closely related problems of

QUICKEST DETECTION.
(The story of creating of the quickest detection problem of randomly
appearing signal, its mathematical formulation, and the route of
solving the problem (1961) are also interesting.)

Two of the prime findings, which also reflect the historical development
of these ideas, are the

principles of SMOOTH and CONTINUOUS FIT

respectively.

IV-1-21
C) One of the best-known specific problems of

MATHEMATICAL FINANCE,
that has a direct connection with optimal stopping problems, is the
problem of determining the

arbitrage-free price of the American put option.

Consider the Black–Scholes model, where the stock price X =

(Xt )t≥0 is assumed to follow a geometric Brownian motion:
D E
Xt = x exp σBt + (r − σ 2/2) t , (17)

where x > 0, σ > 0, r > 0 and B = (Bt )t≥0 is a standard Brownian

motion. By Itô’s formula one finds that the process X solves

dXt = rXt dt + σXt dBt with X0 = x. (18)

IV-1-22
General theory of financial mathematics makes it clear that the
initial problem of determining the arbitrage-free price of the American
put option can be reformulated as the following optimal stopping
problem:

V∗ = sup Ee−rτ (K − Xτ )+ (19)

τ

where the supremum is taken over all stopping times τ of X.

This constitutes Step 1 in the diagram above.

The constant K > 0 is called the strike price. It has a certain

financial meaning which we set aside for now.

IV-1-23
It turns out that the optimal stopping problem (19):

V∗ = sup Ee−rτ (K − Xτ )+
τ
can be reduced again to a free-boundary problem which can be
solved explicitly. It yields the existence of a constant b∗ such that
the stopping time

τ∗ = inf { t ≥ 0 | Xt ≤ b∗ } (20)
is optimal in (19).

This constitutes Steps 2 and 3 in the diagram above.

Both the optimal stopping point b∗ and the arbitrage-free price V∗

can be expressed explicitly in terms of the other parameters in the
problem. A financial interpretation of these expressions constitutes
Step 4 in the diagram above.
IV-1-24
In the formulation of the problem (19) above:

V∗ = sup Ee−rτ (K − Xτ )+
τ
no restriction was imposed on the class of admissible stopping
times, i.e. for certain reasons of simplicity it was assumed there
that
τ belongs to the class of stopping times

M={τ |0≤τ <∞} (21)

without any restriction on their size.

IV-1-25
A more realistic requirement on a stopping time in search for the
arbitrage-free price leads to the following optimal stopping problem:

V∗T = sup Ee−rτ (K − Xτ )+ (22)

τ ∈MT

where the supremum is taken over all τ belonging to the class of

stopping times
MT = { τ | 0 ≤ τ ≤ T } (23)
with the horizon T being finite.

The optimal stopping problem (22) can be also reduced to a free-

boundary problem that apparently cannot be solved explicitly.

IV-1-26
Its study yields that the stopping time

τ∗ = inf { 0 ≤ t ≤ T | Xt ≤ b∗(t) } (24)

is optimal in (22), where b∗ : [0, T ] → R is an increasing continuous
function.

A nonlinear Volterra integral equation can be derived which characterizes

the optimal stopping boundary t :→ b∗(t) and can be used to compute
its values numerically as accurate as desired.

The comments on Steps 1–4 in the diagram above made in the

infinite horizon case carry over to the finite horizon case without
any change.

In our lectures we study these and other similar problems that arise
from various financial interpretations of options.
IV-1-27
5. So far we have only discussed problems A, B, C and their reformulations
as optimal stopping problems. Now we want to address the methods
of solution of optimal stopping problems and their reduction to free-
boundary problems.

There are essentially two equivalent approaches to finding a solution

of the optimal stopping problem. The first one deals with the problem

V∗ = sup EGτ in the case of infinite horizon, (25)

τ ∈M

or the problem

V∗T = sup EGτ in the case of finite horizon, (26)

τ ∈MT

where M = { τ | 0 ≤ τ ≤ ∞}, and MT = { τ | 0 ≤ τ ≤ T }.

IV-1-28
In this formulation it is important to realize that

G = (Gt)t≥0 is an arbitrary stochastic process defined on

a filtered probability space (Ω, F , (Ft )t≥0, P), where it is
assumed that G is adapted to the filtration (Ft )t≥0 which
in turn makes each τ from M or MT a stopping time.

Since the method of solution to the problems (25) and (26) is based
on results from the theory of martingales (Snell’s envelope, 1952),
the method itself is often referred to as the

MARTINGALE METHOD.

IV-1-29
On the other hand, if we are to take a state space (E, B) large
enough, then one obtains the

“Markov representation” Gt = G(Xt )

for some measurable function G, where X = (Xt )t≥0 is a Markov
process with values in E. Moreover, following the contemporary
theory of Markov processes it is convenient to adopt the definition
of a Markov process X as the family of Markov processes

((Xt )t≥0, (Ft )t≥0, (Px )x∈E ) (27)

where Px(X0 = x) = 1, which means that the process X starts at
x under Px. Such a point of view is convenient, for example, when
dealing with the Kolmogorov forward or backward equations, which
presuppose that the process can start at any point in the state
space.

IV-1-30
Likewise, it is a profound attempt, developed in stages, to study
optimal stopping problems through functions of initial points in the
state space.

In this way we have arrived to the second approach which deals with
the problem
V (x) = sup ExG(Xτ ) (28)
τ
where the supremum is taken over M or MT as above (Dynkin’s
formulation, 1963).

Thus, if the Markov representation of the initial problem is valid, we

will refer to the

MARKOVIAN METHOD of solution.

IV-1-31
6. To make the exposed facts more transparent, let us consider the
optimal stopping problem
D E
V∗ = sup E max |Bt| − cτ
τ 0≤t≤τ
in more detail.

Denote
Xt = |x + Bt| (29)
for x ≥ 0, and enable the maximum process to start at any point by
setting for s ≥ x
D E
St = s ∨ max Xr . (30)
0≤r≤t

IV-1-32
 D E
St = s ∨ max Xr
0≤r≤t

The process S = (St)t≥0 is not Markov, but

the pair (X, S) = (Xt, St)t≥0 forms a Markov process
with the state space
E = { (x, s) ∈ R2 | 0 ≤ x ≤ s }.

D E
The value V∗ from (4) above: V∗ = sup E max |Bt| − cτ coincides
τ 0≤t≤τ
with the value function
! "
V∗(x, s) = sup Ex,s Sτ − cτ (31)
τ
when x = s = 0. The problem thus needs to be solved in this more
general form.
IV-1-33
The general theory of optimal stopping for Markov processes makes
it clear that the optimal stopping time in (31) can be written in the
form
τ∗ = inf { t ≥ 0 | (Xt , St) ∈ D∗ } (32)

where D∗ is a stopping set, and

C∗ = E \ D∗ is the continuation set.

In other words,

• if the observation of X was not stopped before time t

since Xs ∈ C∗ for all 0 ≤ s < t, and we have that Xt ∈ D∗,
then it is optimal to stop the observation at time t,
• if it happens that Xt ∈ C∗ as well, then the observation
of X should be continued.

IV-1-34
 "
s=x
s Heuristic considerations about the
$
$
D∗ C∗ $ shape of the sets C∗ and D∗
$
$
$ makes it plausible to guess that
$
$
$ there exist a point s∗ ≥ 0 and
$
$
$ a continuous increasing function
% # $
$
%
$
s :→ g∗(s) with g∗(s∗) = 0 such
(xt, st)
$
$ that
$
$
D∗ = { (x, s) ∈ R2 | 0 ≤ x ≤ g∗(s) , s ≥ s∗ } (33)
$
$
$
$ #

x
Note that such a guess about the shape of the set D∗ can be made
using the following intuitive arguments. If the process (X, S) starts
from a point (x, s) with small x and large s, then it is reasonable to
stop immediately because to increase the value s one needs a large
time τ which in the formula (31) appears with a minus sign.
IV-1-35
At the same time it is easy to see that

if x is close or equal to s then it is reasonable to continue

the observation, at least for small time ∆, because s will
√
increase for the value ∆ while the cost for using this
√
time will be c∆, and thus ∆ − c∆ > 0 if ∆ is small
enough.

Such an a priori analysis of the shape of the boundary between the

stopping set C∗ and the continuation set D∗ is typical to the act of
finding a solution to the optimal stopping problem. The

art of GUESSING
in this context very often plays a crucial role in solving the problem.

IV-1-36
Having guessed that the stopping set D∗ in the optimal stopping
problem V∗(x, s) = supτ Ex,s(Sτ − cτ ) takes the form

D∗ = { (x, s) ∈ R2 | 0 ≤ x ≤ g∗(s) , s ≥ s∗ },
it follows that τ∗ attains the supremum, i.e.,
! "
V∗(x, s) = Ex,s Sτ∗ − cτ∗ for all (x, s) ∈ E. (34)
Consider V∗(x, s) for (x, s) in the continuation set

C∗ = C∗1 ∪ C∗2 (35)

where the two subsets are defined as follows:

C∗1 = { (x, s) ∈ R2 | 0 ≤ x ≤ s < s∗ } (36)

C∗2 = { (x, s) ∈ R2 | g∗(s) < x ≤ s , s ≥ s∗ }. (37)

IV-1-37
Denote by
1 ∂2
LX =
2 ∂x2
the infinitesimal operator of the process X. By the strong Markov
property one finds that V∗ solves

LX V∗(x, s) = c for (x, s) in C∗. (38)

If the process (X, S) starts at a point (x, s) with x < s, then during
a positive time interval the second component S of the process
remains equal to s.

This explains why the infinitesimal operator of the process (X, S)

reduces to the infinitesimal operator of the process X in the interior
of C∗.
IV-1-38
On the other hand, from the structure of the process (X, S) it follows
that at the diagonal in R2
+
• the condition of normal reflection holds:
F
∂V∗ F
F
(x, s)F = 0. (39)
∂s F
x=s−

Moreover, it is clear that for (x, s) ∈ D∗

• the condition of instantaneous stopping holds:
V∗(x, s) = s. (40)

Finally, either by guessing or providing rigorous arguments, it is

found that at the optimal boundary g∗
• the condition of smooth fit holds:
F
∂V∗ F
F
(x, s)F = 0. (41)
∂x F
x=g∗(s)+

IV-1-39
This analysis indicates that the value function V∗ and the optimal
stopping boundary g∗ can be obtained by searching for the pair of
functions (V, g) solving the following free-boundary problem:

LX V (x, s) = c for (x, s) in Cg (42)

F
∂V F
(x, s)FF = 0 (normal reflection) (43)
∂s x=s−
V (x, s) = s for (x, s) in Dg (instantaneous stopping) (44)
F
∂V F
(x, s)FF = 0 (smooth fit) (45)
∂x x=g(s)+
where the two sets are defined as follows (g(s0) = 0):

Cg = { (x, s) ∈ R2 | 0 ≤ x ≤ s < s0 or g(s) < x ≤ s, s ≥ s0 } (46)

Dg = { (x, s) ∈ R2 | 0 ≤ x ≤ g(s) , s ≥ s0 } (47)
It turns out that this system does not have a unique solution so
that an additional criterion is needed to make it unique in general.
IV-1-40
Let us show how to solve the free-boundary problem (42)–(45) by
picking the right solution (more details will be given in the lectures).

From (42) one finds that for (x, s) in Cg we have

V (x, s) = cx2 + A(s) x + B(s) (48)
where A and B are some functions of s. To determine A and B as
well as g we can use the three conditions
F
∂V F
(x, s)FF =0 (normal reflection)
∂s x=s−
V (x, s) = s for (x, s) in Dg (instantaneous stopping)
F
∂V F
(x, s)FF =0 (smooth fit)
∂x x=g(s)+

which yield
1
g 2(s) = , for s ≥ s0. (49)
2(s − g(s))
IV-1-41
It is easily verified that the linear function
1
g(s) = s − (50)
2c
solves (49). In this way a candidate for the optimal stopping boundary
g∗ is obtained.

1 one can determine V (x, s) explicitly using

For (x, s) ∈ E with s ≥ 2c

V (x, s) = cx2 + A(s) x + B(s)

and
1
g(s) = s − .
2c

This in particular gives that V (1/2c, 1/2c) = 3/4c.

IV-1-42
For other points (x, s) ∈ E when s < 1/2c one can determine V (x, s)
using that the observation must be continued. In particular for x =
s = 0 this yields that

V (0, 0) = V (1/2c, 1/2c) − c E0,0(σ) (51)

where σ is the first hitting time of the process (X, S) to the point
(1/2c, 1/2c).

Because E0,0(σ) = E0,0(Xσ2) = (1/2c)2 and V (1/2c, 1/2c) = 3/4c,

we find that
1
V (0, 0) = (52)
2c
as already indicated prior to (7) above. In this way a candidate for
the value function V∗ is obtained.

IV-1-43
The key role in the proof of the fact that

V = V∗ and g = g∗
is played by

Itô’s formula (stochastic calculus) and the

optional sampling theorem (martingale theory).

This step forms a VERIFICATION THEOREM that makes it

clear that

the solution of the free-boundary problem coincides

with the solution of the optimal stopping problem

IV-1-44
7. The important point to be made in this context is that the
verification theorem is usually not difficult to prove in the cases
when a candidate solution to the free-boundary problem is obtained
explicitly.

This is quite typical for one-dimensional problems with infinite

horizon, or some simpler two-dimensional problems, as the one just
discussed.

In the case of problems with finite horizon, however, or other

multidimensional problems, the situation can be radically different.

In these cases, in a manner quite opposite to the previous ones,

the general results of optimal stopping can be used to prove the
existence of a solution to the free-boundary problem, thus providing
an alternative to analytic methods.

IV-1-45
8. From the material exposed above it is clear that our basic interest
concerns the case of continuous time.

The theory of optimal stopping in the case of continuous time is

considerably more complicated than in the case of discrete time.

However, since the former theory uses many basic ideas from the
latter, we have chosen to present the case of discrete time first, both
in the martingale and Markovian setting, which is then likewise
followed by the case of continuous time. The two theories form
several my lectures.

IV-1-46
LECTURE 2–3:
Theory of optimal stopping for discrete time.
A. Martingale approach.
1. Definitions
(Ω, F , (Fn)n≥0, P), F0 ⊆ F1 ⊆ · · · ⊆ Fn ⊆ · · · ⊆ F , G = (Gn)n≥0.
Gain Gn is Fn -measurable
Stopping (Markov ) time τ = τ (ω):
τ : Ω → {0, 1, . . . , ∞}, {τ ≤ n} ∈ Fn for all n ≥ 0.

M is the family of all finite stopping times

M is the family of all stopping times
MN
n = {τ ∈ M | n ≤ τ ≤ N }
For simplicity we will set MN = MN
0 and Mn = M∞
n .

IV-2/3-1
The optimal stopping problem to be studied seeks to solve

V∗ = sup E Gτ . (53)
τ

For the existence of E Gτ suppose (for simplicity) that

E sup |Gk | < ∞ (54)
0≤k<∞

(then E Gτ is well defined for all τ ∈ MN

n , n ≤ N < ∞).

In the class MN
n we consider

VnN = sup E Gτ , 0 ≤ n ≤ N. (55)

τ ∈MN
n

Sometimes we admit that τ in (53) takes the value ∞ (P(τ = ∞) >

0), so that τ ∈ M. We put Gτ = 0 on {τ = ∞}.

Sometimes it is useful to set G∞ = lim sup Gn.

n→∞
IV-2/3-2
2. The method of backward induction.

VnN = sup E Gτ
n≤τ ≤N

To solve this problem we introduce (by backward induction) a special

N , SN
stochastic sequence SN , . . . , S N:
N −1 0
N =G ,
SN SnN = max{Gn, E(Sn+1
N | Fn )},
N
n = N − 1, . . . , 0.
N , equals G .
If n = N we have to stop and our stochastic gain SN N

Stop at time N
*+ ,,
*
** ,,
,,
&
** ,,
&'(
& ,, * -!
,
&&& ( ,,
, *
&& ( *+
* -*
,
&& ( **
(
( ** #
( **
()*
(
0 1 2 N −2 N −1 N
IV-2/3-3
For n = N − 1 we can either stop or continue. If we stop, our gain
N N
SN −1, equals GN −1, and if we continue our gain SN −1 will be equal
to E(SNN |F
N −1).

either stop at time N − 1 &

.
/
. "
.
&' ,,
& .
&& ,, .
&& ,, .
&.
.
+
* -!
,,
,
**
' ,,
& *
&& ,, *
&&& ,, **
& ,, *
&& -*
, or continue and stop at time N
#

0 1 2 N −2 N −1 N

So,
N N
SN −1 = max{GN −1, E(SN | FN −1)}
and optimal stopping time is
N N
τN −1 = min{N − 1 ≤ k ≤ N : S k = Gk }.

IV-2/3-4
Define now a sequence (SnN )0≤n≤N recursively as follows:

SnN = GN , n = N,
SnN = max{Gn, E(Sn+1
N
| Fn)}, n = N −1, . . . , 0.

The described method suggests to consider the following stopping

time:
τnN = inf{n ≤ k ≤ N : SkN = Gk } for 0 ≤ n ≤ N .

The first part of the following theorem shows that SnN and τnN solve
the problem in a stochastic sense.

The second part of the theorem shows that this leads also to a
solution of the initial problem
VnN = sup E Gτ for each n = 0, 1, . . . , N.
n≤τ ≤N

IV-2/3-5
Theorem 1. (Finite horizon)
I. For all 0 ≤ n ≤ N we have:

(a) SnN ≥ E(Gτ | Fn), ∀τ ∈ MN

n;
(b) SnN = E(Gτ N | Fn).
n

II. Moreover, if 0 ≤ n ≤ N is given and fixed, then we have:

(c) τnN is optimal in VnN = sup E Gτ ;

n≤τ ≤N
(d) if τ∗ is also optimal then τnN ≤ τ∗;
(e) the sequence (SkN )n≤k≤N is the smallest
supermartingale which dominates (Gk )n≤k≤N
(Snell’s envelope)
(f) the stopped sequence N
(Sk∧τ is a
N )n≤k≤N
n
martingale.

IV-2/3-6
Proof of Theorem 1.

I. Induction over n = N, N −1, . . . , 0.

Conditions

(a) SnN ≥ E(Gτ | Fn), ∀τ ∈ MN

n,

and

(b) SnN = E(Gτ N | Fn)

n
are trivially satisfied for n = N .

Suppose that (a) and (b) are satisfied for n = N, N −1, . . . , k, where
k ≥ 1, and let us show that they must then also hold for n = k−1.

IV-2/3-7
 ! "
(a) SnN ≥ E(Gτ | Fn), ∀τ ∈ MN
n : Take τ ∈ MN
k−1 and set τ̄ = τ ∨ k;
then τ̄ ∈ MN
k , and since {τ ≥ k} ∈ Fk−1 it follows that

E(Gτ | Fk−1) = E[I(τ = k−1)Gk−1 | Fk−1] + E[I(τ ≥ k)Gτ̄ | Fk−1]

= I(τ = k−1)Gk−1 + I(τ ≥ k) E[E(Gτ̄ | Fk ) | Fk−1].

By the induction hypothesis, (a) holds for n = k. Since τ̄ ∈ MN

k this
implies that
E(Gτ̄ | Fk ) ≤ SkN . (56)

From SnN = max(Gn, E(Sn+1

N | Fn )) for n = k − 1 we have
N ,
Gk−1 ≤ Sk−1 (57)
E(SkN | Fk−1) ≤ Sk−1
N . (58)

IV-2/3-8
Using (56)–(58) in (??) we get
N
E(Gτ | Fk−1) ≤ I(τ = k−1) Sk−1 + I(τ ≥ k) E(SkN | Fk−1)
N + I(τ ≥ k) S N
≤ I(τ = k−1) Sk−1 N (59)
k−1 = Sk−1.

This shows that

SnN ≥ E(Gτ | Fn), ∀τ ∈ MN

n
holds for n = k − 1 as claimed.
! "
(b) SnN = E(Gτ N | Fn) : To prove (b) for n = k − 1 it is enough to
n
check that all inequalities in (??) and (59) remain equalities when
N . For this, note that
τ = τk−1
N
τk−1 = τkN N
on {τk−1 ≥ k};
N N
Gk−1 = Sk−1 on {τk−1 = k − 1};
E(SkN | Fk−1) = Sk−1
N N ≥ k}.
on {τk−1

IV-2/3-9
Then we get
( )
N = k − 1) G
E Gτ N | Fk−1 = I(τk−1 k−1
k−1
( )
N ≥ k)
+ I(τk−1 E E(Gτ N | Fk ) | Fk−1
k

N = k − 1) G
= I(τk−1 + N ≥ k) E(S N | F
k−1 I(τk−1 k k−1)

N N N N N
= I(τk−1 = k − 1) Sk−1 + I(τk−1 ≥ k) Sk−1 = Sk−1 .

Thus
! "
SnN = E Gτ N | Fn
n
holds for n = k − 1. (We supposed by induction that (b) holds for
n = N, . . . , k.)

IV-2/3-10
 ! "
(c) τnN is optimal in VnN = sup E Gτ :
n≤τ ≤N
Take expectation E in SnN ≥ E(Gτ | Fn), τ ∈ Mn
n. Then

E SnN ≥ E Gτ for all τ ∈ MN

n
and by taking the supremum over all τ ∈ MN
n we see that
D E
E SnN ≥ VnN = sup E Gτ .
τ ∈MN
n

On the other hand, taking the expectation in SnN = E(Gτ N | Fn ) we

n
get
E SnN = E Gτ N
n
which shows that
D E
E SnN ≤ VnN = sup E Gτ .
τ ∈MN
n

IV-2/3-11
So,
E SnN = VnN
and since E SnN = E Gτ N , we see that
n

VnN = E Gτ N
n

implying the claim (c): “The stopping time τnN is optimal”.

! "
(d) N
if τ∗ is also optimal then τn ≤ τ∗ :
If we suppose that τ∗ is also optimal then τnN ≤ τ∗. We claim that
the optimality of τ∗ implies that SτN∗ = Gτ∗ (P-a.s.). Indeed,

for all n ≤ k ≤ N SkN ≥ Gk , thus SτN∗ ≥ Gτ∗ .

If SτN∗ 4= Gτ∗ (P-a.s.) then

P(SτN∗ > Gτ∗ ) > 0.

IV-2/3-12
It thus follows that
(α) (β)
E Gτ∗ < E SτN∗ ≤ E Sn = VnN ,
N

where

(α) follows by the supermartingale property of

(SkN )n≤k≤N (see (e)) and the optional sampling
theorem, and
(β) was obtained in (c).

The strict inequality E Gτ∗ < VnN , however, contradicts the fact that
τ∗ is optimal.

Hence SτN∗ = Gτ∗ (P-a.s.) and the fact that τnN ≤ τ∗ (P-a.s.) follows
from the definition

τnN = inf{n ≤ k ≤ N : SkN = Gk }.

IV-2/3-13
 !
(e) the sequence (SkN )n≤k≤N is the smallest supermartingale
"
which dominates (Gk )n≤k≤N :

From

SkN = max{Gk , E(Sk+1

N | Fk )}, k = N − 1, . . . , n,
we see that (SkN )n≤k≤N is a supermartingale:

SkN ≥ E(Sk+1
N | Fk ).

Also we have SkN ≥ Gk . It means that (SkN )n≤k≤N is a supermartingale

which dominates (Gk )n≤k≤N .

Suppose that (S/k )n≤k≤N is another supermartingale which dominates

(Gk )n≤k≤N , then the claim that S/k ≥ SkN (P-a.s.) can be verified by
induction over k = N, N − 1, . . . , l.
IV-2/3-14
Indeed, if k = N then the claim follows by SnN = GN for n = N .

Assuming that S/k ≥ SkN for k = N, N − 1, . . . , l with l ≥ n + 1 it

follows that
N = max(G N
Sl−1 l−1, E(Sl | Fl−1))
≤ max(Gl−1, E(S/l | Fl−1)) ≤ S/l−1 (P-a.s.)
using the supermartingale property of (S̃k )n≤k≤N . So, (SkN )n≤k≤N is
the smallest supermartingale which dominates (Gk )n≤k≤N
(Snell’s envelop).

IV-2/3-15
 ! "
(f) the stopped sequence (S N ) is a martingale :
k∧τnN n≤k≤N
To verify the martingale property
( )
N N
E S(k+1)∧τ N | Fk = Sk∧τ N
n n
with n ≤ k ≤ N − 1 given and fixed, note that
( ) ( )
E N
S(k+1)∧τ N | Fk = E N N
I(τn ≤ k) Sk∧τ N | Fk
n ( n )
N N
+ E I(τn ≥ k + 1) Sk+1 | Fk

= I(τnN ≤ k)Sk∧τ
N N N
N + I(τn ≥ k + 1) E(Sk+1 | Fk )
n

= I(τnN ≤ k)Sk∧τ
N N N N
N + I(τn ≥ k + 1)Sk = Sk∧τ N
n n

where we used that

SkN = E(Sk+1
N
| Fk ) on { τnN ≥ k + 1 }
and { τnN ≥ k + 1 } ∈ Fk since τnN is a stopping time.
IV-2/3-16
 Summary

1) The optimal stopping problem

V0N = sup E Gτ
τ ∈MN
0
is solved inductively by solving the problems

VnN = sup E Gτ for n = N, N − 1, . . . , 0.

τ ∈MN
n

2) The optimal stopping rule τnN for VnN satisfies

τnN = τkN on {τnN ≥ k}

for 0 ≤ n ≤ k ≤ N when τkN is the optimal stopping rule for VkN . In
other words, this means that if it was not optimal to stop within
the time set {n, n + 1, . . . , k − 1} then the same optimality rule for
VnN applies in the time set {k, k + 1, . . . , N }.
IV-2/3-17
3) In particular, when specialized to the problem V0N , the following
general principle (of dynamic programming) is obtained:

if the stopping rule τ0N is optimal for V0N and it was not
optimal to stop within the time set {0, 1, . . . , n − 1}, then
starting the observation at time n and being based on the
information Fn, the same stopping rule is still optimal for
the problem VnN .

" 4
$
$$ $
$ $ $$ $$ $ $$ $$ $$ $$ $
$ $ $ $ $ $ $ D
$ $ $
$
$
$ $ $ $ $ $ $ $ $ $$
$ $ $ $ 000
122
222 $
000 3
2$
+
*
**
**
0000
1222
0 000
1222
00 1*
00 C
00 2 300
22 2 300
22

$ $ $ $
$
$ $ $ $ $
$ $ $ $ $
$
$
$
$
$
$
$
$
$
$
$
$
$
$
D
$
$
$
$
$
$
$$
$ $ $ $
$ $ $ $ $ $ $ $ $ $ $$ $ $ $$ $ $ $
$ $ $ $ $ $ $ $ $ $ $ $ $ $ $ $ $ #

0 n−1 n
IV-2/3-18
3. The method of ESSENTIAL SUPREMUM

The method of backward induction by its nature requires that the

horizon N be FINITE so that the case of infinite horizon remains
uncovered.

It turns out, however, that the random variables SnN defined by the
recurrent relations

SnN = GN , n = N,
SnN = max{Gn, E(Sn+1
N
| Fn)}, n = N −1, . . . , 0,
admit a different characterization which can be directly extended to
the case of infinite horizon N .

This characterization forms the base of the SECOND method that

will now be presented.
IV-2/3-19
Note that the relations

(a) SnN ≥ E(Gτ | Fn) ∀τ ∈ MN

n;
(b) SnN = E(Gτ N | Fn)
n
from Theorem 1 suggest that the following identity should hold:

SnN = sup E(Gτ | Fn) .

τ ∈MN
n

(!) Difficulty: supτ ∈MN E(Gτ | Fn) need not define a

n
measurable function.

To overcome this difficulty it turns out that the concept of

ESSENTIAL SUPREMUM

proves useful.
IV-2/3-20
Lemma (about Essential Supremum).

Let {Zα, α ∈ A} be a family of random variables defined on (Ω, F , P)

where the index set A can be arbitrary.

I. Then there exists a countable subset J of A such that the random

variable Z ∗ : Ω → R defined by

Z ∗ = sup Zα
α∈J
satisfies the following two properties:

(a) P(Zα ≤ Z ∗) = 1, ∀α ∈ A;
(b) If Z̃ : Ω → R is another random variable
satisfying P(Zα ≤ Z ∗) = 1, ∀α ∈ A, then
P(Z ∗ ≤ Z̃) = 1.

IV-2/3-21
II. Moreover, if the family {Zα, α ∈ A} is upwards directed in the
sense that
for any α and β in A there exists γ in A
such that max(Zα, Zβ ) ≤ Zγ (P-a.s.),
then the countable set J = {αn, n ≥ 1} can be chosen so that
Z ∗ = lim Zαn (P-a.s.)
n→∞
where Zα1 ≤ Zα2 ≤ · · · (P-a.s.).

Proof. (1) Since x :→ π2 arctan(x) is a strictly increasing function

from R to [−1, 1], it is no restriction to assume that |Zα| ≤ 1.

(2) Let C denote the family of all countable subsets C of A. Choose

an increasing sequence {Cn, n ≥ 1} in C such that
D E D E
def
a = sup E sup Zα = sup E sup Zα .
C∈C α∈C n≥1 α∈Cn

IV-2/3-22
 def L∞
Then J = n=1 Cn is a countable subset of A and we claim that
def
Z ∗ = sup Zα
α∈J
satisfies the properties (a) and (b).

(3) To verify these claims take α ∈ A arbitrarily.

(a): If α ∈ J then Zα ≤ Z ∗ so that (a) holds. If α ∈

/ J and we assume
that P(Zα > Z ∗) > 0, then

a < E(Z ∗ ∨ Zα) ≤ a

since a = E Z ∗ ∈ [−1, 1] (by the monotone convergence
theorem) and J ∪ {α} belongs to C. As the strict inequality
is impossible, we see that P(Zα ≤ Z ∗) = 1, ∀α ∈ A as claimed.

(b): follows from Z ∗ = supα∈J Zα and (a): P(Zα ≤ Z ∗) = 1, ∀α ∈ A,

since J is countable.
IV-2/3-23
Finally, assume that the condition in II is satisfied. Then the initial
countable set
J = {α1, α2, . . .}
can be replaced by a new countable set J ◦ = {α◦1, α◦2, . . .} if we
initially set α◦1 = α1, and then inductively choose α◦n+1 ≥ α◦n ∨ αn+1
for n ≥ 1, where γ ≥ α ∨ β corresponds to Zα, Zβ and Zγ such that
Zγ ≥ Zα ∨ Zβ (P-a.s.). The concluding claim Z ∗ = limn→∞ Zαn in II
is then obvious, and the proof of the lemma is complete.

With the concept of essential supremum we may now rewrite

SnN ≥ E(Gτ | Fn ) ∀τ ∈ MN
n; SnN = E(Gτ N | Fn)
n
in Theorem 53 above as follows:

SnN = ess sup E(Gτ | Fn) for all 0 ≤ n ≤ N .

n≤τ ≤N

IV-2/3-24
This ess sup identity provides an additional characterization of the
sequence of r.v.’s (SnN )0≤n≤N introduced initially by means of the
recurrent relations
SnN = GN , n = N,
SnN = max{Gn, E(Sn+1
N
| Fn)}, n = N −1, . . . , 0.

Its advantage in comparison with these recurrent relations lies in

the fact that the identity
SnN = ess sup E(Gτ | Fn )
n≤τ ≤N
can naturally be extended to the case of INFINITE horizon N . This
programme will now be described.

Consider (instead of VnN = supτ ∈MN E Gτ )

n

Vn = sup E Gτ .
τ ∈M∞
n

IV-2/3-25
To solve this problem we will consider the sequence of r.v.’s (Sn)n≥0
defined as follows:
Sn = ess sup E(Gτ | Fn)
τ ≥n
as well as the following stopping time:
τn = inf{k ≥ n | Sk = Gk } for n ≥ 0,
where inf ∅ = ∞ by definition.
The first part (I) of the next theorem shows that (Sn)n≥0 satisfies
the same recurrent relations as (SnN )0≤n≤N .
The second part (II) of the theorem shows that Sn and τn solve the
problem in a stochastic sense.
The third part (III) shows that this leads to a solution of the initial
problem Vn = supτ ≥n E Gτ .
The fourth part (IV) provides a supermartingale characterization of
the solution.
IV-2/3-26
Theorem 2 (Infinite horizon).

Consider the optimal stopping problems

Vn = sup E Gτ , τ ∈ M∞
n , n≥0
τ ≥n
assuming that the condition E sup |Gk | < ∞ holds.
0≤k<∞

I. The following recurrent relations hold:

Sn = max{Gn, E(Sn+1 | Fn )}, ∀n ≥ 0.

II. Assume moreover if required below that

P(τn < ∞) = 1.
Then for all n ≥ 0 we have:

Sn ≥ E(Gτ | Fn ) ∀τ ∈ Mn, Sn = E(Gτn | Fn).

IV-2/3-27
III. Moreover, if n ≥ 0 is given and fixed, then we have:

The stopping time τn = inf{k ≥ n : Sk = Gk } is

optimal in Vn = supτ ≥n E Gτ .
If τ∗ is an optimal stopping time for Vn = supτ ≥n E Gτ
then τn ≤ τ∗ (P-a.s.).

IV. The sequence (Sk )k≥n is the smallest supermartingale

which dominates (Gk )k≥n (Snell’s envelope).

The stopped sequence (Sk∧τn )k≥n is a martingale.

Finally, if the condition P(τn < ∞) = 1 fails so that

P(τn = ∞) > 0, then there is NO optimal stopping time
in Vn = supτ ≥n E Gτ .

IV-2/3-28
Proof. I. We need prove the recurrent relations

Sn = max{Gn, E(Sn+1 | Fk )}, n ≥ 0.

Let us first show that

Sn ≤ max{Gn, E(Sn+1 | Fk )}.

For this, take τ ∈ Mn and set τ̄ = τ ∨ (n + 1).
Then τ̄ ∈ Mn+1, and since {τ ≥ n + 1} ∈ Fn we have

E(Gτ | Fn) = E[I(τ = n)Gn | Fn] + E[I(τ ≥ n + 1)Gτ̄ | Fn]

= I(τ = n)Gn + I(τ ≥ n + 1) E(Gτ̄ | Fn)
= I(τ = n)Gn + I(τ ≥ n + 1) E[E(Gτ̄ | Fn+1) | Fn ]
≤ I(τ = n)Gn + I(τ ≥ n + 1) E Sn+1 | Fn )
≤ max{Gn, E(Sn+1 | Fn )}.

IV-2/3-29
From this inequality it follows that

Sn = ess sup E(Gτ | Fn) ≤ max{Gn, E(Sn+1 | Fn)}

τ ≥n
which is the desired inequality.

For the reverse inequality, let us first note that Sn ≥ Gn (P-a.s.)

by the definition of Sn, so that it is enough to show (and it is the
most difficult part of the proof) that

Sn ≥ E(Sn+1 | Fn )
which is the supermartingale property of (Sn)n≥0. To verify this
inequality, let us first show that the family {E(Gτ | Fn+1); τ ∈ Mn+1}
is upwards directed in the sense that
for any α and β in A there exists γ in A
(∗)
such that Zα ∨ Zβ ≤ Zγ .

IV-2/3-30
For this, note that if σ1 and σ2 are from Mn+1 and we set σ3 =
σ1IA + σ2IĀ where

A = {E(Gσ1 | Fn+1) ≥ E(Gσ2 | Fn+1)},

then σ3 ∈ Mn+1 and we have

E(Gσ3 | Fn+1) = E(Gσ1 IA + Gσ2 IĀ | Fn+1)

= IA E(Gσ1 | Fn+1) + IĀ E(Gσ2 | Fn+1)
= E(Gσ1 | Fn+1) ∨ E(Gσ2 | Fn+1)
implying (∗) as claimed. Hence by Lemma there exists a sequence
{σk , k ≥ 1} in Mn+1 such that

ess sup E(Gτ | Fn+1) = lim E(Gσk | Fn+1)

τ ≥n+1 k→∞
where
E(Gσ1 | Fn+1) ≤ E(Gσ2 | Fn+1) ≤ · · · (P-a.s.).

IV-2/3-31
Since
Sn+1 = ess sup E(Gτ | Fn+1),
τ ≥n+1
by the conditional monotone convergence theorem we get
( )
E(Sn+1 | Fn) = E lim E(Gσk | Fn+1) | Fn
k→∞( )
= lim E E(Gσk | Fn+1) | Fn
k→∞
= lim E(Gσk | Fn ) ≤ Sn.
k→∞
So, Sn = max{Gn, E(Sn+1 | Fn)} and the proof if I is complete.

II. The inequality Sn ≥ E(Gτ | Fn ), ∀τ ∈ Mn , follows from the definition

Sn = ess supτ ≥n E(Gτ | Fn).
For the proof of the equality Sn = E(Gτn | Fn) we use the fact stated
below in IV that the stopped sequence (Sk∧τn )k≥n is a martingale.
IV-2/3-32
Setting G∗n = supk≥n |Gk | we have
! "
|Sk | ≤ ess sup E |Gτ | | Fk ≤ E(G∗n | Fk ) (∗)
τ ≥k
for all k ≥ n. Since G∗n is integrable due to E supk≥n |Gk | < ∞, it
follows from (∗) that (Sk )k≥n is uniformly integrable.

Thus the optional sampling theorem can be applied to the martingale

(Mk )k≥n = (Sk∧τn )k≥n and we get

Mn = E(Mτn | Fn). (∗∗)

Since Mn = Sn and Mτn = Sτn we see that (∗∗) is the same as Sn =
E(Gτn | Fn).

III: “The stopping time τn is optimal in Vn = supτ ≥n E Gτ .”

The proof uses II and is similar to the corresponding proof in Theorem

1 (N < ∞).
IV-2/3-33
IV. “The sequence (Sk )k≥n is the smallest supermartingale which
dominates (Gk )k≥n” (Snell’s envelop).

We proved in I that (Sk )k≥n is a supermartingale. Moreover, from

the definition
Sn = ess sup E(Gτ | Fn)
τ ≥n
it follows that Sk ≥ Gk , k ≥ n, which means that (Sk )k≥n dominates
(Gk )k≥n. Finally, if (S̃k )k≥n is another supermartingale which dominates
(Gk )k≥n, then from Sn = E(Gτn | Fn ) (Part II) we find

Sk = E(Gτk | Fk ) ≤ E(S̃τk | Fk ) ≤ S̃k , ∀k ≥ n.

(The last inequality follows by the optional sampling theorem being
applicable since S̃k− ≤ G− ∗ ∗
k ≤ Gn (= supk≥n |Gk |) with Gn integrable.)

IV-2/3-34
The statement

“The stopped sequence (Sk∧τn )k≥n is a martingale”

is proved in exactly the same way as for case N < ∞.

Finally, note that the final claim

“If the condition P(τn < ∞) = 1 fails so that P(τn = ∞) >

0, then there is NO optimal stopping time in the problem
Vn = supτ ≥n E Gτ ”

follows directly from III (“If τn is optimal stopping tome then τn ≤ τ∗

(P-a.s.) for the problem Vn = supτ ≥n E Gτ ”).

IV-2/3-35
Remark. From the definition

Sn = ess sup E(Gτ | Fn)

n≤τ ≤N
it follows that
N :→ SnN and N :→ τnN
are increasing. So,

Sn∞ = lim SnN and τn∞ = lim τnN

N →∞ N →∞
exist P-a.s. for each n ≥ 0.

IV-2/3-36
Note also that from
VnN = sup E Gτ
n≤τ ≤N

it follows that N :→ VnN is increasing, so that Vn∞ = limN →∞ VnN

exists for each n ≥ 0.

From SnN = ess supn≤τ ≤N E(Gτ | Fn ) and Sn = ess supτ ≥n E(Gτ | Fn)
we see that
Sn∞ ≤ Sn and τn∞ ≤ τn. (∗)
Similarly,
! "
Vn∞ ≤ Vn = sup E Gτ . (∗∗)
τ ≥n

If condition E supn≤k<∞ |Gk | < ∞ does not hold then the inequalities
in (∗) and (∗∗) can be strict.
IV-2/3-37
Theorem 3 (From finite to infinite horizon).

If E sup0≤k<∞ |Gk | < ∞ then in Sn∞ ≤ Sn, τn∞ ≤ τn and Vn∞ ≤ Vn we

have equalities for all n ≥ 0.

Proof. From

SnN = max{Gn, E(Sn+1

N
| Fn)}, n ≥ 0,
we get
Sn∞ = max{Gn, E(Sn+1
∞ | F )},
n n ≥ 0.
So, (Sn∞)n≥0 is a supermartingale.

Since Sn∞ ≥ Gn we see that

(Sn∞)− ≤ G− −
n ≤ sup Gn , n ≥ 0.
n≥0
So, ((Sn∞ )−)n≥0 is uniformly integrable.
IV-2/3-38
Then by the optional sampling theorem we get

Sn∞ ≥ E(Sτ∞ | Fn ) for all τ ∈ Mn. (∗)

Moreover, since Sk∞ ≥ Gk , k ≥ n, it follows that Sτ∞ ≥ Gτ for all

τ ∈ Mn, and hence

E(Sτ∞ | Fn) ≥ E(Gτ | Fn) (∗∗)

for all τ ∈ Mn . From (∗), (∗∗) and

Sn = ess sup E(Gτ | Fn)

τ ≥n
we see that Sn∞ ≥ Sn.

Since the reverse inequality holds in general as shown above, this

establishes that Sn∞ = Sn (P-a.s.) for all n ≥ 0. From this it also
follows that τn∞ = τn (P-a.s.), n ≥ 0. Finally, the third identity
Vn∞ = Vn follows by the monotone convergence theorem.
IV-2/3-39
B. Markovian approach.

We will present basic results of optimal stopping when

the time is discrete and the process is Markovian.

1. We consider a time-homogeneous Markov chain X = (Xn)n≥0

• defined on a filtered probability space (Ω, F , (Fn )n≥0, Px)

• taking values in a measurable space (E, B)
where for simplicity we will assume that

(a) E = Rd for some d ≥ 1

(b) B = B(Rd ) is the Borel σ-algebra on Rd.

IV-2/3-40
It is assumed that the chain X starts at x under Px for x ∈ E.

It is also assumed that the mapping x :→ Px(F ) is measurable for

each F ∈ F .

It follows that the mapping x :→ Ex(Z) is measurable for each random

variable Z.

Finally, without loss of generality we will assume that (Ω, F ) equals

the canonical space (E N0 , BN0 ) so that the shift operator θn : Ω → Ω
is well defined by

θn(ω)(k) = ω(n+k) for ω = (ω(k))k≥0 ∈ Ω and n, k ≥ 0.

(Recall that N0 stands for N ∪ {0}.)

IV-2/3-41
Given a measurable function G : E → R satisfying the following
condition (with G(XN ) = 0 if N = ∞):
D E
Ex sup |G(Xn)| < ∞
0≤n≤N
for all x ∈ E, we consider the optimal stopping problem

V N (x) = sup Ex G(Xτ )

0≤τ ≤N
where x ∈ E and the supremum is taken over all stopping times τ
of X. The latter means that τ is a stopping time w.r.t. the natural
filtration of X given by

FnX = σ(Xk ; 0 ≤ k ≤ n) for n ≥ 0.

IV-2/3-42
Since the same results remain valid if we take the supremum in

V N (x) = sup Ex G(Xτ ) (∗)

0≤τ ≤N
over stopping times τ w.r.t. (Fn)n≥0, and this assumption makes
final conclusions more powerful (at least formally), we will assume
in the sequel that the supremum in (∗) is taken over this larger class
of stopping times.

Note also that in (∗) we admit that N can be +∞ as well.

In this case, however, we still assume that the supremum is taken

over stopping times τ , i.e. over Markov times τ satisfying 0 ≤ τ < ∞.
In this way any specification of G(X∞) becomes irrelevant for the
problem (∗).

IV-2/3-43
To solve
V N (x) = sup Ex G(Xτ ) (∗)
0≤τ ≤N
when N < ∞, we may note that by setting Gn = G(Xn) for n ≥ 0
the problem reduces to the problem
VnN = sup Ex Gτ . (∗∗)
n≤τ ≤N

Having identified (∗) as (∗∗), we can apply the method of back-

ward induction which leads to a sequence of r.v.’s (SnN )0≤n≤N and
a stopping time τnN = inf{n ≤ k ≤ N : SkN = Gk }.

The key identity is

SnN = V N −n(Xn ) for 0 ≤ n ≤ N , Px-a.s.; x ∈ E (∗∗∗)
Once (∗∗∗) is known to hold, the results of the Theorem 1 (finite
horizon) from the Martingale theory translate immediately into the
present Markovian setting and get a more transparent form.
IV-2/3-44
To get formulation, let us define

CnN = { x ∈ E : V N −n(x) > G(x) }

N
Dn = { x ∈ E : V N −n(x) = G(x) }
for 0 ≤ n ≤ N . We also define stopping time
N }.
τD = inf { 0 ≤ n ≤ N : Xn ∈ Dn
and the transition operator T of X

T F (x) = Ex F (X1)
for x ∈ E whenever F : E → R is a measurable function so that
F (X1 ) is integrable w.r.t. Px for all x ∈ E.

IV-2/3-45
Theorem 4 (Finite horizon: The time-homogeneous case)

Consider the optimal stopping problems

V n(x) = sup Ex G(Xτ ) (∗)

0≤τ ≤n
assuming that Ex sup |G(Xk )| < ∞. Then
0≤k≤N

I. Value functions V n satisfy the “Wald–Bellman equation”

V n(x) = max(G(x), T V n−1(x)) (x ∈ E)

for n = 1, . . . , N where V 0 = G.
II. N } is
The stopping time τD = inf { 0 ≤ n ≤ N : Xn ∈ Dn
optimal in (∗) for n = N .
III. If τ∗ is an optimal stopping time in (∗) then τD ≤ τ∗ (Px-a.s.)
for every x ∈ E.

IV-2/3-46
 IV. The sequence (V N −n(Xn))0≤n≤N is the smallest
supermartingale which dominates (G(Xn ))0≤n≤N under Px
for x ∈ E given and fixed.
V. The stopped sequence (V N −n(Xn∧τD ))0≤n≤N is a
martingale under Px for every x ∈ E.

Proof. To verify the equality SnN = V N −n(Xn ) recall that

SnN = Ex(G(Xτ N ) | Fn ) (i)

n

for 0 ≤ n ≤ N . Since SkN −n ◦ θn = Sn+k

N we get that τnN satisfies

τnN = inf{n ≤ k ≤ N : SkN = G(Xk )} = n + τ0N −n ◦ θn (ii)

for 0 ≤ n ≤ N (θn ω(k) = ω(k + n)).

IV-2/3-47
Inserting (ii) into (i) and using the Markov property we obtain
( ) ( )
SnN = Ex G(Xn+τ N −n◦θ ) | Fn = Ex G(Xτ N −n ) ◦ θn | Fn
0 n 0
(α)
(iii)
= EXn G(Xτ N −n ) = V N −n(Xn )
0

where (α) follows by (i): SnN = Ex(G(Xτ N ) | Fn ), which imply

n

Ex S0N −n = Ex G(Xτ N −n ) = sup Ex G(Xτ ) = V N −n(x) (iv)

0 0≤τ ≤N −n
for 0 ≤ n ≤ N and x ∈ E.

Thus SnN = V N −n(Xn ) holds as claimed.

IV-2/3-48
To verify the “Wald–Bellman equation”, note that the equality

SnN = max{Gn, E(Sn+1

N | Fn)},
using the Markov property, reads as follows:
7 ( )8
V N −n (Xn) = max G(Xn), Ex V N −n−1 (Xn+1) | Fn
7 ( )8
N −n−1
= max G(Xn), Ex V (X1) ◦ θn | Fn
7 8 (∗)
= max G(Xn), EXn V N −n−1 (X1)
7 8
N −n−1
= max G(Xn), T V (Xn )
for all 0 ≤ n ≤ N . Letting n = 0 and using that X0 = x under Px we
see that (∗) yields V n(x) = max{G(x), T V n−1(x)}.

The remaining statements of the theorem follow directly from the

Martingale Theorem (1). The proof is complete.

IV-2/3-49
The “Wald–Bellman equation” can be written in a more compact
form as follows. Introduce the operator Q by setting

QF (x) = max(G(x), T F (x))

for x ∈ E where F : E → R is a measurable function for which
F (X1 ) ∈ L1(Px) for x ∈ E. Then the “Wald–Bellman equation” reads
as follows:
V n(x) = QnG(x)
for 1 ≤ n ≤ N where Qn denotes the n-th power of Q. These
recursive relations form a constructive method for finding V N when
Law(X1 | Px) is known for x ∈ E.

IV-2/3-50
TIME-INHOMOGENEOUS MARKOV CHAINS X = (Xn )n≥0

Put Zn = (n, Xn).

Z = (Zn)n≥0 is a time-homogeneous Markov chain.

Optimal stopping problem:

(∗) V N (n, x) = sup En,x G(n+τ, Xn+τ ) , 0 ≤ n ≤ N.

0≤τ ≤N −n

We assume
D E
(∗∗) En,x sup |G(n + k, Xn+k )| < ∞, 0 ≤ n ≤ N.
0≤k≤N −n

IV-2/3-51
Theorem 5 (Finite horizon: The time-inhomogeneous case)

Consider the optimal stopping problem (∗) upon assuming that the
condition (∗∗) holds. Then:

I. The function V n satisfies the “Wald–Bellman equation”

V N (n, x) = max(G(n, x), T V N (n, x))

for n = N −1, . . . , 0 where

T V N (n, x) = En,x V N (n + 1, Xn+1), n = N − 1, . . . , 0,

and
T V N (N −1, x) = EN −1,x G(N, XN );

IV-2/3-52
 II. The stopping time
N
τD = inf{n ≤ k ≤ N : (n + k, Xn+k ) ∈ D}
with
7 8
D = (n, x) ∈ {0, 1, . . . , N }×E : V (n, x) = G(n, x)
is optimal in the problem (∗):

V N (n, x) = sup En,x G(n+τ, Xn+τ );

0≤τ ≤N −n

III. If τ∗N is an optimal stopping time in (∗) then τD N ≤ τN

∗
(Pn,x-a.s.) for every (n, x) ∈ {0, 1, . . . , N }×E;

IV-2/3-53
 IV. The value function V N is the smallest superharmonic
function which dominates the gain function G on
{0, . . . , N }×E,

T V N (n, x) ≤ V N (n, x), V N (n, x) ≥ G(n, x);

V. The stopped sequence

! "
N N
V ((n+k) ∧ τD , X(n+k)∧τ N )
D 0≤k≤N −n
is a martingale under Pn,x for every (n, x) ∈ {0, 1, . . . , N }×E;

The proof is carried out in exactly the same way as the proof of
Theorem 4.

IV-2/3-54
 Optimal stopping for infinite horizon (N = ∞):

V (x) = sup Ex G(Xτ )

τ

Theorem 6

Assume Ex supn≥0 |G(Xn )| < ∞, x ∈ E.

I. The value function V satisfies the “Wald–Bellman equation”

V (x) = max(G(x), T V (x)), x ∈ E.

II. Assume moreover when required below that Px(τD < ∞) = 1

for all x ∈ E, where

τD = inf{t ≥ 0 : Xt ∈ D}
with D = {x ∈ E : V (x) = G(x)}. Then the stopping time τD
is optimal.
IV-2/3-55
III. If τ∗ is an optimal stopping time then τD ≤ τ∗ (Px-a.s. for
every x ∈ E).

IV. The value function V is the smallest superharmonic function

(Dynkin’s characterization) (T V ≤ V ) which dominates the
gain function G on E, or, equivalently, (V (Xn))n≥0 is the
smallest supermartingale (under Px, x ∈ E) which dominates
(G(Xn ))n≥0.

V. The stopped sequence (V (Xn∧τD ))n≥0 is a martingale under

Px for every x ∈ E.
VI. If the condition Px(τD < ∞) = 1 fails so that Px(τD = ∞) >
0 for some x ∈ E, then there is no optimal stopping time in
the problem V (x) = supτ Ex G(Xτ ) for all x ∈ E.

IV-2/3-56
Corollary (Iterative method). We have

V (x) = lim QnG(x)

n→∞

(a constructive method for finding the value function V ).

Uniqueness in the Wald–Bellman equation

F (x) = max(G(x), T F (x))

Suppose E supn≥0 F (Xn ) < ∞.

Then F equals the value function V if and only if the following

“ boundary condition at infinity ” holds:

lim sup F (Xn ) = lim sup G(Xn) Px-a.s. ∀x ∈ E.

n→∞ n→∞

IV-2/3-57
2. Given α ∈ (0, 1] and bounded g : E → R and c : E → R+ , consider
the optimal stopping problem
D τ
# E
V (x) = sup Ex ατ g(Xτ ) − αk−1c(Xk−1) .
τ
k=1

I = (X
Let X I )
n n≥0 denote the Markov chain X killed at rate α. It
means that
T/ F (x) = α T F (x).
Then
D τ
# E
I )−
V (x) = sup Ex g(X I
τ c(Xk−1) .
τ
k=1

The “Wald–Bellman equation” takes the following form:

7 8
V (x) = max g(x), αT V (x) − c(x) .

IV-2/3-58
LECTURES 4–5.
Theory of optimal stopping for continuous time
A. Martingale approach
Let (Ω, F , (Ft )t≥0, P) be a stochastic basis (a filtered probability
space with right-continuous family (Ft )t≥0 where each Ft contains
all P-null sets from F .

Let G = (Gt)t≥0 be a gain process. (We interpret Gt as the gain if

the observation of G is stopped at time t.)

DEFINITION.
A random variable τ : Ω → [0, ∞] is called a Markov time
if {τ ≤ t} ∈ Ft for all t ≥ 0.
A Markov time is called a stopping time if τ < ∞ P-a.s.
IV-4/5-1
We assume that G = (Gt)t≥0 is right-continuous and left-continuous
over stopping times (if τn ↑ τ then Gτn → Gτ P-a.s.).

We also assume that

! "
E sup |Gt| < ∞ (GT = 0 if T = ∞).
0≤t≤T

BASIC OPTIMAL STOPPING PROBLEM:

VtT = sup E Gτ .
t≤τ ≤T

We shall admit that T = ∞. In this case the supremum is still taken

over stopping times τ , i.e. over Markov times τ satisfying t ≤ τ < ∞.

IV-4/5-2
Two ways to tackle the problem VtT = supt≤τ ≤T E Gτ :

(1) Discrete time approximation

7 8
(n) (n) (n)
[0, T ] −→ T(n) = t0 , t1 , . . . , tn ↑ T is a dense
subset of [0, T ]
G −→ G(n) = (G (n) )
ti
with applying previous discrete-time results and then
passing to the limit n → ∞;

(2) Straightforward extension of the method of essential

supremum. This programme will now be addressed.

We denote for simplicity of the notation

Vt = VtT (T < ∞ or T = ∞).

IV-4/5-3
Consider the process S = (St)t≥0 defined as follows:

St = ess sup E (Gτ |Ft).

τ ≥t
The process S is the Snell’s envelope of G.

Introduce
τt = inf {u ≥ t | Su = Gu} where inf ∅ = ∞ by definition.

We shall see below that

St ≥ max{Gt, E (Su | Ft )} for u ≥ t.

The reverse inequality is not true generally.

However,
St = max{Gt, E (Sσ∧τt | Ft)}
for every stopping time σ ≥ t and τt given above.
IV-4/5-4
Theorem 1. Consider the optimal stopping problem

Vt = sup E Gτ , t ≥ 0,
τ ≥t
upon assuming E supt≥0 |Gt| < ∞. Assume moreover when required
below that
P(τt < ∞) = 1, t ≥ 0.
(Note that this condition is automatically satisfied when the horizon
T is finite.) Then:

I. For all t ≥ 0 we have

St ≥ E(Gτ | Ft) for each τ ∈ Mt

St = E(Gτt | Ft)
where Mt = {τ : τ ≤ T } if T < ∞,
Mt = {τ : τ < ∞} if T = ∞.

IV-4/5-5
 II. The stopping time τt = inf{u ≥ t : Su = Gu} is
optimal (for the problem Vt = supτ ≥t E Gτ ).
III. If τt∗ is an optimal stopping time as well then
τt ≤ τt∗ P-a.s.
IV. The process (Su)u≥t is the smallest right-
continuous supermartingale which dominates
(Gs)s≥t.
V. The stopped process (Su∧τt )u≥t is a right-
continuous martingale.
VI. If the condition P(τt < ∞) = 1 fails so that
P(τt = ∞) > 0, then there is no optimal stopping
time.

IV-4/5-6
Proof. 1◦. Let us first prove that S = (St)t≥0 defined by

St = ess sup E (Gτ | Ft)

τ ≥t
is a supermartingale.

Show that the family {E (Gτ | Ft) : τ ∈ Mt } is upwards directed in the

sense that if σ1 and σ2 are from Mt then there exists σ3 ∈ Mt such
that
E (Gσ1 | Ft) ∨ E (Gσ2 | Ft) ≤ E (Gσ3 | Ft).
Put σ3 = σ1IA + σ2IĀ where

A = {E (Gσ1 |Ft) ≥ E (Gσ2 | Ft)}.

Then σ3 ∈ Mt and

E(Gσ3 | Ft) = E (Gσ1 IA + Gσ2 IĀ | Ft) = IA E (Gσ1 | Ft) + IĀ E (Gσ2 | Ft)
= E (Gσ1 | Ft ) ∨ E (Gσ2 | Ft ).

IV-4/5-7
Hence there exists a sequence {σk ; k ≥ 1} in Mt such that

(∗) ess sup E (Gτ | Ft) = lim E (Gσk | Ft )

τ ∈Mt k→∞
where
E(Gσ1 | Ft) ≤ E (Gσ2 | Ft) ≤ · · · P-a.s.
From (∗) and the conditional monotone convergence theorem (using
E supt≥0 |Gt| < ∞) we find that for 0 ≤ s < t
! "
E(St | Fs) = E lim E (Gσk | Ft) | Fs
k→∞
= lim E [ E (Gσk | Ft) | Fs]
k→∞ ! "
= lim E (Gσk | Fs) ≤ Ss = ess sup E (Gτ | Fs) .
k→∞ τ ≥s
Thus (St)t≥0 is a supermartingale as claimed.

IV-4/5-8
Note that from E supt≥0 |Gt| < ∞ and

St = ess sup E(Gτ | Ft),

τ ≥t
ess sup E(Gτ | Ft) = lim E (Gσk | Ft )
τ ≥t k→∞
it follows that

E St = sup E Gτ .
τ ≥t

2◦. Let us next show that the supermartingale S admits a right-

continuous modification S/ = (S/t)t≥0.

From the general martingale theory it follows that it suffices to

check that

t % E St is right-continuous on R+ .

IV-4/5-9
By the supermartingale property of S

E S t ≥ · · · ≥ E S t2 ≥ E S t1 , tn ↑ t.
So, L := limn→∞ E Stn exists and

E St ≥ L.
To prove the reverse inequality, fix ε > 0 and by means of E St =
supτ ≥t E Gτ choose σ ∈ Mt such that

E Gσ ≥ E St − ε.
Fix δ > 0 and note that there is no restriction to assume that
tn ∈ [t, t + δ] for all n ≥ 1. Define

σ if σ > tn,
σn =
t+σ if σ ≤ tn.

IV-4/5-10
Then for all n ≥ 1 we have

(∗) E Gσn = E Gσ I(σ > tn) + E Gt+δ I(σ ≤ tn) ≤ E Stn

since σn ∈ Mtn and E St = supτ ≥t E Gτ . Letting n → ∞ in (∗) and

assuming that E sup0≤t≤T |Gt| < ∞ we get

E Gσ I(σ > t) + E Gt+δ I(σ = t) ≤ L (= lim E Stn ).

n
Letting now δ ↓ 0 and using that G is right-continuous we obtain

E Gσ I(σ > t) + E GtI(σ = t) = E Gσ ≤ L.

From here and E Gσ ≥ E St − ε we see that L ≥ E St − ε for all ε > 0.

Hence L ≥ E St and thus

lim E Stn = L = E St, tn ↑ t,

n→∞
showing that S admits a right-continuous modification S/ = (S/t)t≥0
which we also denote by S throughout.
IV-4/5-11
Let us prove property IV:

The process (Su)u≥t is the smallest right-continuous

supermartingale which dominates (Gs)s≥t.
For this, let SG = (SGu)u≥t be another right-continuous supermartingale
which dominates G = (Gu)u≥t. Then by the optional sampling theorem
(using E supt≥0 |Gt| < ∞) we have

SGu ≥ E (SGτ | Fu) ≥ E (Gτ | Fu)

for all τ ∈ Mu when u ≥ t. Hence by the definition Su = ess sup E(Gτ | Fu)
τ ≥u
we find that Su ≤ SGu (P-a.s.) for all u ≥ t. By the right-continuity of
S and SG this further implies that

P(Su ≤ SGu for all u ≥ t) = 1

as claimed.
IV-4/5-12
Property I: for all t ≥ 0

(∗) St ≥ E(Gτ | Ft) for each τ ∈ Mt,

(∗∗) St = E(Gτt | Ft ).

The inequality (∗) follows from the definition St = ess supτ ≥t E(Gτ | Ft).
The proof of (∗∗) is the most difficult part of the proof of the
Theorem.

The sketch of the proof is as follows.

IV-4/5-13
Assume that Gt ≥ 0 for all t ≥ 0.

(α) Introduce, for λ ∈ (0, 1), the stopping time

τtλ = inf{s ≥ t : λSs ≤ Gs}

λ = τ .)
(Then λSτ λ ≤ Gτ λ , τt+ t
t t

(β) We show that

St = E (Sτ λ | Ft) for all λ ∈ (0, 1).

t
So St ≤ (1/λ) E (Gτ λ | Ft ) and letting λ ↑ 1 we get
t

St ≤ E (Gτ 1 | Ft)
t

where τt1 = limλ↑1 τtλ (τtλ ↑ when λ ↑).

(γ) Verify that τt1 = τt. Then St ≤ E (Gτt | Ft ) and evidently
St ≥ E (Gτt | Ft ). Thus St = E (Gτt | Ft).

IV-4/5-14
For the proof of property V:

The stopped process (Su∧τt )u≥t is a right-

continuous martingale

it is enough to prove that

E Sσ∧τt = E St
for all bounded stopping times σ ≥ t.

The optional sampling theorem implies

E Sσ∧τt ≤ E St. (60)

On the other hand, from St = E (Gτt | Ft) and Sτt = Gτt we see that

E St = E Gτt = E Sτt ≤ E Sσ∧τt .

Thus, E Sσ∧τt = E St and (Su∧τt )u≥t is a martingale.
IV-4/5-15
B. Markovian approach

Let X = (Xt)t≥0 be a strong Markov process defined on a filtered

probability space
(Ω, F , (Ft )t≥0, Px)

where x ∈ E (= Rd), Px(X0 = x) = 1,

x → Px(A) is measurable for each A ∈ F .

Without loss of generality we will assume that

(Ω, F ) = (E [0,∞), B[0,∞)) (canonical space)

Shift operator θt = θt(ω) : Ω → Ω is well defined by

θt(ω)(s) = ω(t + s) for ω = (ω(s))s≥0 ∈ Ω and t, s ≥ 0.

IV-4/5-16
We consider the optimal stopping problem

V (x) = sup Ex G(Xτ )

0≤τ ≤T

G(XT ) = 0 if T < ∞; Ex sup |G(Xt)| < ∞.

0≤t≤T

Here τ = τ (ω) is a stopping time w.r.t.

(Ft )t≥0 (FtX ⊆ Ft, FtX = σ(Xs ; 0 ≤ s ≤ t)).

G is called the gain function,

V is called the value function.

IV-4/5-17
 V (x) = sup Ex G(Xτ )
CASE T = ∞: τ
Px(X0 = x) = 1

Introduce

the continuation set C = {x ∈ E : V (x) > G(x)} and

the stopping set D = {x ∈ E : V (x) = G(x)}

NOTICE! If
V is lsc (lower semicontinuous) G is usc (upper semicontinuous)
$
& %
&
% &

then
C is open and D is closed

IV-4/5-18
The first entry time

τD = inf{t ≥ 0 : Xt ∈ D}
for closed D is a stopping time since both X and (Ft )t≥0 are right-
continuous.

DEFINITION. A measurable function F = F (x) is said to be

superharmonic (for X) if

Ex F (Xσ ) ≤ F (x)
for all stopping times σ and all x ∈ E. (It is assumed that F (Xσ ) ∈
L1(Px) for all x ∈ E whenever σ is a stopping time.)

We have:
(F (Xt ))t≥0 is a supermartingale
F is superharmonic iff
under Px for every x ∈ E.

IV-4/5-19
The following theorem presents

NECESSARY CONDITIONS
for the existence of an optimal stopping time.

Theorem. Let us assume that there exists an optimal stopping time

τ∗ in the problem
V (x) = sup Ex G(Xτ )
τ
i.e. V (x) = Ex F (Xτ∗ ). Then

(I) The value function V is the smallest superharmonic

function (Dynkin’s characterization) which dominates
the gain function G on E.

IV-4/5-20
Let us in addition to “ V (x) = Ex F (Xτ∗ )” assume that

V is lsc and G is usc.

Then

(II) The stopping time τD = inf{t ≥ 0 : Xt ∈ D} satisfies

τD ≤ τ∗ (Px -a.s., x ∈ E )
and is optimal;
(III) The stopped process (V (Xt∧τD ))t≥0 is a right-continuous
martingale under Px for every x ∈ E.

IV-4/5-21
Now we formulate

SUFFICIENT CONDITIONS
for the existence of an optimal stopping time.

Theorem. Consider the optimal stopping problem

V (x) = sup Ex G(Xτ )

τ
upon assuming that the condition

Ex sup |G(Xt)| < ∞, x ∈ E,

t≥0
is satisfied.

IV-4/5-22
Let us assume that there exists the smallest superharmonic function
VG which dominates the gain function G on E.

Let us in addition assume that

VG is lsc and G is usc.

Set D = {x ∈ E : VG (x) = G(x)} and let τD = inf{t : Xt ∈ D}.

We then have:

(a) If Px(τD < ∞) = 1 for all x ∈ E, then VG = V and τD is

optimal in V (x) = supτ Ex G(Xτ );
(b) If Px(τD < ∞) < 1 for some x ∈ E, then there is no
optimal stopping time in V (x) = supτ Ex G(Xτ ).

IV-4/5-23
Corollary (The existence of an optimal stopping time).

Infinite horizon (T = ∞). Suppose that V is lsc and G is usc. If

Px(τD < ∞) = 1 for all x ∈ E, then τD is optimal. If Px(τD < ∞) < 1
for some x ∈ E, then there is no optimal stopping time.

Finite horizon (T < ∞). Suppose that V is lsc and G is usc. Then
τD is optimal.

Proof for T = ∞. (The case T < ∞ can be proved in exactly the

same way as the case T = ∞ if the process (Xt ) is replaced by the
process (t, Xt ).)

The key is to show that V is SUPERHARMONIC.

IV-4/5-24
If so, then evidently V is the smallest superharmonic function
which dominates G on E. Then the claims of the corollary follow
directly from the Theorem (on sufficient conditions) above.

For this, note that V is measurable (since it is lsc) and thus so is

the mapping

(∗) V (Xσ ) = sup EXσ G(Xτ )

τ
for any stopping time σ which is given and fixed.

On the other hand, by the strong Markov property we have

(∗∗) EXσ G(Xτ ) = Ex [ G(Xσ+τ ◦θσ ) | Fσ ]

for every stopping time τ and x ∈ E. From (∗) and (∗∗) we see that

V (xσ ) = ess sup Ex [ G(Xσ+τ ◦θσ ) | Fσ ]

τ
under Px where x ∈ E is given and fixed.
IV-4/5-25
We can show that the family
7 8
E [Xσ+τ ◦θσ | Fσ ] : τ is a stopping time

is upwards directed: if ρ1 = σ + τ1 ◦ θσ and ρ2 = σ + τ2 ◦ θσ then there

is ρ = σ + τ ◦ θσ such that

E [G(Xρ) | Fσ ] = E [G(Xρ1 ) | Fσ ] ∨ E [G(Xρ2 ) | Fσ ].

From here we can conclude that there exists a sequence of stopping

times {τn; n ≥ 1} such that

V (Xσ ) = lim Ex [G(Xσ+τn◦θσ ) | Fn ]

n
where the sequence {Ex [ G(Xσ+τn◦θσ ) | Fn]} is increasing Px-a.s.

IV-4/5-26
By the monotone convergence theorem using E supt≥0 |Gt| < ∞ we
can conclude

Ex V (Xσ ) = lim
n
Ex G(Xσ+τn◦θσ ) ≤ V (x)
for all stopping times σ and all x ∈ E. This proves that V is
superharmonic.

REMARK 1. If the function

x :→ Ex G(Xτ )
is continuous (or lsc) for every stopping time τ , then x :→ V (x) is lsc
and the results of the Corollary are applicable. This yields a powerful
existence result by simple means.

IV-4/5-27
REMARK 2. The above results have shown that the optimal stopping
problem
V (x) = sup Ex G(Xτ )
τ
is equivalent to the problem of finding the smallest superharmonic
function VG which dominates G on E. Once VG is found it follows
that V = VG and τD = inf{t : G(Xt) = VG (Xt )} is optimal.

There are two traditional ways for finding VG :

(i) Iterative procedure (constructive but non-explicit)

(ii) Free-boundary problem (explicit or non-explicit).

IV-4/5-28
For (i), e.g., it is known that if G is lsc and

Ex inf G(Xt) > −∞ for all x ∈ E,

t≥0

then VG can be computed as follows:

VG (x) = lim lim QN

n G(x)
n→∞ N →∞
where
QnG(x) := G(x) ∨ Ex G(X1/2n )

and QN
n is the N -th power of Qn .

IV-4/5-29
The basic idea (ii) is that

VG and C (or D)
should solve the free-boundary problem:

(∗) LX VG ≤ 0
(∗∗) VG ≥ G (VG > G on C & VG = G on D)
where LX is the characteristic (infinitesimal) operator of X.

Assuming that G is smooth in a neighborhood of ∂C the following

“rule of thumb” is valid.

IV-4/5-30
If X after starting at ∂C enters immediately into int(D) (e.g. when X
is a diffusion process and ∂C is sufficiently nice) then the condition
LX VG ≤ 0 under (∗∗) splits into the two conditions:

LX VG = 0 in C
F F
∂ VG FF ∂G FF
F = F (smooth fit).
∂x ∂C ∂x ∂C

On the other hand, if X after starting at ∂C does not enter immediately

into int(D) (e.g. when X has jumps and no diffusion component
while ∂C may still be sufficiently nice) then the condition LX VG ≤ 0
(i.e. (∗)) under (∗∗) splits into the two conditions:

LX VG = 0 in C
F F
G F F
VF = GF (continuous fit).
∂C ∂C

IV-4/5-31
 Proof of the Theorem on NECESSARY conditions
Basic lines

(I) The value function V is the smallest superharmonic

function which dominated the gain function G on E.

We have by the strong Markov property:

Ex V (Xσ ) = Ex EXσ G(Xτ∗ ) = Ex Ex[G(Xτ∗ ) ◦ θσ | Fσ ]

= Ex G(Xσ+τ∗◦θσ ) ≤ sup Ex G(Xτ ) = V (x)
τ
for each stopping time σ and all x ∈ E.

Thus V is superharmonic.

IV-4/5-32
Let F be a superharmonic function which dominates G on E. Then

Ex G(Xτ ) ≤ Ex F (Xτ ) ≤ F (x)

for each stopping time τ and all x ∈ E. Taking the supremum over all
τ we find that V (x) ≤ F (x) for all x ∈ E. Since V is superharmonic
itself, this proves that V is the smallest superharmonic function
which dominated G.

(II) Let us show that the stopping time

τD = inf{t : V (Xt ) = G(Xt)}

is optimal (if V is lsc and G is usc).

We assume that there exists an optimal stopping time τ∗:

V (x) = Ex G(Xτ∗ ), x ∈ E.

IV-4/5-33
We claim that V (Xτ∗ ) = G(Xτ∗ ) Px-a.s. for all x ∈ E.

Indeed, if Px{V (Xτ∗ ) > G(Xτ∗ )} > 0 for some x ∈ E, then

Ex G(Xτ∗ ) < Ex V (Xτ∗ ) ≤ V (x)

since V is superharmonic, leading to a contradiction with the fact
that τ∗ is optimal. From the identity just verified it follows that

τD ≤ τ∗ Px-a.s. for all x ∈ E.

IV-4/5-34
By (I) the value function V is the superharmonic (Ex V (Xσ ) ≤ V (X)
for all stopping time σ and x ∈ E). Setting σ ≡ s and using the
Markov property we get for all t, s ≥ 0 and all x ∈ E

V (Xt) ≥ EXt V (Xs ) = Ex [V (Xt+s) | Ft].

This shows that

The process (V (Xt))t≥0 is a supermartingale

under Px for each x ∈ E.

Suppose for the moment that V is continuous. Then obviously it

follows that (V (Xt ))t≥0 is right-continuous. Thus, by the optional
sampling theorem (using E supt≥0 |G(Xt)| < ∞), we see that

Ex V (Xτ ) ≤ Ex V (Xσ ) for σ ≤ τ .

IV-4/5-35
In particular, since τD ≤ τ∗ we get

V (x) = Ex G(Xτ∗ ) = Ex V (Xτ∗ )

≤ Ex V (XτD ) = Ex G(XτD ) ≤ V (x),
where we used that
V (XτD ) = G(XτD )
Now it is easy to show that τD is optimal if V is continuous.

IV-4/5-36
If V is only lsc, then again (see the lemma below) the process
(V (Xt))t≥0 is right-continuous (Px-a.s. for each x ∈ E), and the
proof can be completed as above.

This shows that τD is optimal if V is lsc as claimed.

Lemma. If a superharmonic function F : E → R is lsc, then

the supermartingale (F (Xt ))t≥0 is right-continuous
(Px-a.s. for each x ∈ E).

We omit the proof.

IV-4/5-37
(III) The stopped process (V (Xt∧τD ))t≥0 is a right-continuous
martingale under Px for every x ∈ E.

PROOF. By the strong Markov property we have

( )
Ex [ V (Xt∧τD ) | Fs∧τD ] = Ex EXt∧τ G(XτD ) | Fs∧τD
D
! "
= Ex Ex [ G(XτD ) ◦ θt∧τD | Ft∧τD ] | Fs∧τD
! "
= Ex Ex [ G(XτD ) | Ft∧τD ] | Fs∧τD = Ex [ G(XτD ) | Fs∧τD ]
= EXs∧τ G(XτD ) = V (Xs∧τD )
D

for all 0 ≤ s ≤ t and! all x ∈ E "proving the martingale property. The

right-continuity of V (Xt∧τD ) follows from the right-continuity
t≥0
of (V (Xt ))t≥0 that we proved above.

The proof of the theorem on necessary conditions is complete.

IV-4/5-38
REMARK. The result and proof of the Theorem extend in exactly
the same form (by slightly changing the notation only) to the finite
horizon problem
VT (X) = sup Ex G(Xτ ).
0≤τ ≤T

Now we formulate the theorem which provides

sufficient condition
for the existence of an optimal stopping time.

IV-4/5-39
THEOREM. Consider the optimal stopping problem

V (x) = sup Ex G(Xτ )

τ
upon assuming that Ex supt≥0 |G(Xt )| < ∞, x ∈ E. Let us assume
that

(a) there exists the smallest superharmonic function VG which

dominates the gain function G on E;
(b) VG is lsc and G is usc.

Set D = {x ∈ E : VG (x) = G(x)} and τD = inf{t : Xt ∈ D}.

We then have:

(I) If Px(τD < ∞) = 1 for all x ∈ E, then VG = V and τD is

optimal;
(II) If Px(τD < ∞) < 1 for some x ∈ E, then there is no
optimal stopping time.
IV-4/5-40
SKETCH OF THE PROOF.

(I) Since VG is superharmonic majorant for G, we have

Ex G(Xτ ) ≤ Ex VG (Xτ ) ≤ V (x)

for all stopping times τ and all x ∈ E. So

G(x) ≤ V (x) = sup Ex G(Xτ ) ≤ VG (x)

τ
for all x ∈ E.

Next step (difficult!): assuming that Px(τD < ∞) = 1 for all x ∈ E,

we prove the inequality

VG (x) ≤ V (x)
and optimality of time τD .

IV-4/5-41
(II) If Px(τD < ∞) < 1 for some x ∈ E then there is no optimal
stopping time.

Indeed, by “necessary-condition theorem” if there exists optimal

optimal τ∗ then τD ≤ τ∗.

But τD takes value ∞ with positive probability for some x ∈ E.

So, for this state x we have Px(τ∗ = ∞) > 0 and τ∗ cannot be

optimal (in the class M = {τ : τ < ∞}).

IV-4/5-42