Introduction To Markov Chain

Introduction to Markov Analysis
Important Concepts, Properties and Applications

Armin Paul D. Allado
Quantitative Methods and Information Technology
Ateneo de Manila University
QUANT 164: OR Modelling Applications| Armin Paul D. Allado | 1st Semester SY 2021-2022
1 | Motivation (Finance)
Useful in answering:
1. What is the probability that an AAA-rated bond will
become AA in the next period?
2. In the long-term, what % of loans will end up being bad
loans?
QUANT 164: OR Modelling Applications| Armin Paul D. Allado | 1st Semester SY 2021-2022 2
1 | Motivation (Epidemiology)
1. What is the probability that
someone susceptible from a virus
will be infected?
2. Over the next week, what is the
expected % of people in a city that
will be infected?
1 | Motivation (Marketing)
1. What is the probability that brand
awareness will lead to consumer purchase?
2. On average, what is the expected % of
target market who will actually purchase the
product after awareness phase?
1 | Markov Chain
Markov Chain
 Models a stochastic (random) process {Xt} values in a countable set {0, 1, 2, …}
 Elements in the countable set are called states (can be finite or infinite)
 E.g. Model the behavior of a dynamic inventory system with possible ending
inventory levels (states) contained in the set {0, 1, 2, 3, 4}
1 | Markov Chain
Markov Chain
 Markov chain “jumps” from state to state to model the behavior of a dynamic
system over time t = 0, 1, 2,…
 E.g. Ending inventory level can jump from 2 to 4 in the next time t.
1 | Markov Chain
Finite State Illustration: Machine Maintenance
1 | Markov Chain
Infinite State Illustration: Job Shop
2 | Transition Matrix
Transition Matrix
 A matrix that shows the probability of transitioning from state i to state j in next
time t + 1
 E.g. Transition matrix of ending inventory level
State: Possible Ending Inventory Level {0, 1, 2, 3, 4}
0 1 2 3 4
0 0.25 0.10 0.25 0.15 0.25
1 0.40 0.30 0.15 0.10 0.05
2 0.05 0.10 0.35 0.25 0.25
3 0.05 0.05 0.20 0.45 0.25
4 0.15 0.15 0.20 0.20 0.30
3 | Illustration # 1
Consider the following model for the value of a stock. At the end of a given day,
the price is recorded. If the stock has gone up, the probability that it will go up
tomorrow is 0.7. If the stock has gone down, the probability that it will go up
tomorrow is only 0.5.
Construct a transition matrix.
Define the states as follows:

Let state 0 represent the stock’s going up
Let state 1 represent the stock’s going down
Transition matrix:
Suppose now that the stock market model is changed so that the stock’s going up
tomorrow depends upon whether it increased today and yesterday.
If the stock has increased for the past two days, it will increase tomorrow with
probability 0.9. If the stock increased today but decreased yesterday, then it will
increase tomorrow with probability 0.6. If the stock decreased today but increased
yesterday, then it will increase tomorrow with probability 0.5. Finally, if the stock
decreased for the past two days, then it will increase tomorrow with probability
0.3.
Construct a transition matrix.
Transform to a Markov Chain by saying that the state at any time n is determined
by the behavior of the stock during both that day and the previous day.
Define the states as follows:

State 0: The stock increased both today and yesterday.
State 1: The stock increased today and decreased yesterday.
State 2: The stock decreased today and increased yesterday.
State 3: The stock decreased both today and yesterday.
State 0: The stock increased both today and yesterday.

State 1: The stock increased today and decreased yesterday.
State 2: The stock decreased today and increased yesterday.
State 3: The stock decreased both today and yesterday.
Transition matrix:
4 | Properties of Markov Chain
Markovian (Memoryless) Property

 Pij = P{Xn+1 = j | Xn = i, Xn-1 = in-1, … , X0 = i0}
= P{Xn+1 = j | Xn = i}
 Pij = P{Xn+1 = j | Xn = i}
 These are called transition probabilities or one-step probabilities
 Future states are dependent only on the present state, and are independent of
past history
One-step Transition Probabilities

 Pij = P{Xn+1 = j | Xn = i, Xn-1 = in-1, … , X0 = i0}
= P{Xn+1 = j | Xn = i}
 Pij = P{Xn+1 = j | Xn = i}
n-step Transition Probabilities

 Pijn = P{Xn+m = j | Xm = i}, n,m ≥ 0, i,j ≥ 0
Transition Probability Matrix

 P = [Pij]
Property of Transition Probabilities

 Pij ≥ 0, = 1
5 | n-Step Transition Probabilities

Introduction To Markov Chain

Uploaded by

Copyright:

Available Formats

Introduction To Markov Chain

Uploaded by

Document Information

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Introduction To Markov Chain

Uploaded by

Copyright:

Available Formats

Introduction to Markov Analysis

Important Concepts, Properties and Applications

Finite State Illustration: Machine Maintenance

Infinite State Illustration: Job Shop

Construct a transition matrix.

Define the states as follows:

Construct a transition matrix.

Define the states as follows:

State 0: The stock increased both today and yesterday.

Markovian (Memoryless) Property

One-step Transition Probabilities

n-step Transition Probabilities

Transition Probability Matrix

Property of Transition Probabilities

You might also like

Pfad - The Proxy pFad of © 2024 Garber Painting. All rights reserved.