Markov and Mr. Monopoly Make Millions: Probability
Markov and Mr. Monopoly Make Millions: Probability
Markov and Mr. Monopoly Make Millions: Probability
Probability
Sets
A set is a collection of items. An example of a set can be all the Course XIV
classes offered at MIT: {14.01, 14.02, 14.04, 14.05, 14.32, 14.33, 14.36 . . . }. For
the following definitions and examples, let A and S be arbitrary sets.
A subset is a set contained within another set, in other words, if all members
of a set belongs to another set. A is a subset of S if all members of A belong
to S, and we write A ⊆ S. Note that:
1
SP.268 Monopoly
The cardinality of a set, denoted |A| here, is the number of elements in that
set. If A ⊆ S, then |A| ≤ |S|. If A ⊂ S, then |A| < |S|.
Now that we have defined sets generally, let’s look at how sets are used when
applied to probability. The ‘things’ or ‘items’ that we’re concerned with are
outcomes–outcomes from flipping coins, dealing hands or cards, etc. The
sample space is the set of all possible outcomes, denoted Ω. A subset of a
sample space consists of the outcomes that we’re interested in, called events.
• Commutative:
A∪B =B∪A
A∩B =B∩A
2
SP.268 Monopoly
• Associative:
(A ∪ B) ∪ C = A ∪ (B ∪ C)
(A ∩ B) ∩ C = A ∩ (B ∩ C)
• Distributive:
(A ∪ B) ∩ C = (A ∩ C) ∪ (B ∩ C)
(A ∩ B) ∪ C = (A ∪ C) ∩ (B ∪ C)
The probability of an event corresponds to the map from the set of events to
the interval [0,1]. When we talk about the probability of some event A in Ω,
it will always conform to the following axioms:
2. P (A) ≥ 0.
3
SP.268 Monopoly
For most problems, the goal will be to find the likelihood that an event E
happens, or P (E), out of the set of possible outcomes S. When all the
outcomes are equally likely,
|E|
P (E) = .
|S|
We’re adding up all the elements in E and all the elements in S, then dividing
them. This leads us to the topic of counting, which is used when dealing with
discrite, finite sample spaces.
Counting
We make the assumption that all the outcomes are equally likely, also known
as the assumption of uniform probability. All that needs to be done then is
add up the number of outcomes that we care about and divide that by the
number of all possible outcomes. The trickiest part is defining the event and
sample space and making sure that we count everything the right number of
times.
Counting Rules
We’ve seen the Sum Rule already, just not labeled with the name. If
A1 , A2 , . . . , An are disjoint sets, then
|A1 ∪ A2 ∪ . . . ∪ An | = |A1 | + |A2 | + . . . + |An |.
What’s the probability version of the Sum Rule?
4
SP.268 Monopoly
Example 1
Excuse this somewhat lame example, but its purpose is to show the sum rule
at work. In a group of 150 students, 15 use Internet Explorer as their web
browser of choice, 80 use Firefox, 15 use Safari, and 40 use Chrome. If being
“cool” means you use Firefox or Chrome as your main web browser, what is
the probability that we pick one student who is “cool?”
Recall the definition of set intersections, when one event OR the other hap-
pens. Let the set C be the set of “cool” students; there are 80 + 40 students
in C, by the sum rule. Let S be the set of all students; there are 150 students
in total, as stated in the problem. Therefore, the probability of picking a cool
student is:
|C| 120 4
P (picking a cool student) = = = .
|S| 150 5
The multiplication rule states that for a length-k sequence, where the first
term is chosen out of set S1 , the 2nd term is chosen out of set S2 . . . the last
term is chosen out of Sk , then
Example 2
The athena combination lock just got changed again. You’re far from any
Quickstation and there’s no one else nearby. Suppose you wanted to try
your luck at guessing the combo (and SIPB’s hint board is not there). As
you try the different combos that pop up in your head, you wonder, how
many possible combination of buttons are there?
The athena door locks have 6 buttons, 3 on the top row and 3 on the bottom
row. The athena passcode is 5 digits. Let Di , for i = 1, 2, 345, represent
the set of buttons possible for each digit; each digit can be any of the first 5
5
SP.268 Monopoly
buttons.
Permutations
a) What is the probability that two people in the room have the same
birthday? Ignore complications with leap years and assume there are
365 days in a year. We also assume that birthdays are random (not
exactly true).
This problem is best approached the other way around, because the
probability that no two people have the same birthday would be easier
to find.
Let A be the event that two people have the same birthday. Then Ac
is the event that no two people have the same birthday. Note that
P (A) = 1 − P (Ac ). We start with person 1; this person can have any 1
of 365 days out of the year. A second person can only have a birthday
on the 364 days out of the year that hasn’t been ‘taken.’ By assumption
of random birthdays, and of uniform probability, the chance that this
6
SP.268 Monopoly
The answers to part a) and part b) are quite different, but the way the ques-
tions were phrased were only slightly different. Half the work in probability
questions is usually figuring out what the question wants from you...
What happens if n > 365? You can answer part a) without doing any math,
by the Pigeonhole Principle. The Pigeonhole Principle states that in a
mapping from set X to set Y , if |X| > |Y |, then more than one element of
X map to soem element in Y .
Combinations
We will also want to deal with collections that are unordered. How many
ways are there to take r objects out of a set of n objects?
For the first object, we have n to choose from. For the 2nd object, we have
n − 1 to choose from. For the rth object, we have n − r + 1 to choose from.
But note that once we’ve selected r objects this way, they are in some kind
n!
of order, and the answer n(n − 1)(n − 2) · · · (n − r + 1) = (n−r)! is not correct.
We must divide by r!, which is the number of ways you can order (permute)
r objects.
7
SP.268 Monopoly
The number of ways that we can take r objects out of a set of n objects is
therefore
n! n
= .
r!(n − r)! r
b) How many ways are there to deal a flush, a 5-card hand with all cards
the same suit?
There are 41 ways to choose the suit, and 13
5
ways to choose the 5
cards out of that suit.
13
# of ways to deal a flush = 4 = 5148
5
c) How many ways are there to deal a 5-card hand with 1 pair?
There are 13 4
1
ways to choose the card value of the pair, and 2
ways
50
to choose the suits of the pair; then there are 3 ways to choose the
remaining 3 cards of the 5-card hand.
4 50
# of ways to deal a hand with 1 pair = 13 = 1528800
2 3
d) How many ways are there to deal a 5-card hand with only 1 pair?
As before, there are 13
1
ways to choose the card value of the pair, and
4
2
ways to choose the suits of the pair. But the problem specifies only
1 pair. The remaining 3 cards in the hand cannot contain a pair. So
8
SP.268 Monopoly
4 12 3
# of ways to deal a hand with only 1 pair = 13 4 = 1098240
2 3
f) How many ways are there to deal a full house, a 5-card hand with 3 of
one kind and 2 of another?
13
As before, there are 1
ways to choose the card value of the 3-of-a-
kind, and 3 ways to choose the suits of the pair.Then there are 12
4
1
ways to choose the value of the 2-of-a-kind and 42 ways to choose the
suit.
4 12 4
# of ways to deal a full house = 13 = 3744
3 1 2
Conditional Probability
If we’re interested in the probability that some event A occurs given that
some event B has already occurred, the sample space becomes B. The prob-
ability of A conditioned on B becomes a probability on the space B.
9
SP.268 Monopoly
So we get that
P (A ∩ B)
P (A|B) = , for P (B) 6= 0.
P (B)
The Law of Total Probability gives us the ability to isolate the probability
of one event on a partitioned probability space. Given a space Ω that is
partitioned by Bn : n = 1, 2, . . . , and an event A,
n
X
P (A) = P (A ∩ Bi )
i=1
Example 5
Melissa and I are going to assign your P/F grades for this seminar by picking
them out of a hat. We take 100 slips of paper and mark P on half of them,
F on the other half. Then we put the slips of paper in two hats, and pick
a slip of paper from one of the hats. Whatever we pick will be your grade.
But, being as merciful and fair as we so obviously are, and curious how much
you got out of this class, we’ll leave it up to you to place the slips of paper
into the two hats any way you want. How will you do it?
Monopoly
The game of MONOPOLY
R came about during the Great Depression, orig-
inating from Charles Darrow of Germantown, Pennsylvania. It started out
as handmade sets sold in a shop in Philadelphia, and as people grew to love
10
SP.268 Monopoly
• The MONOPOLY
R man isn’t a Parker Brother. His name is Mr.
Monopoly.
• Parker Brothers once sent an armored car with a million MONOPOLY
R dollars
to Pittsburgh because a marathon game there had run out of money.
• MONOPOLY
R comes in a Braille version.
• The four most-landed-on squares are Jail, Illinois Avenue, “Go”, and
the B&O Railroad.
The last in the list of fun facts above is more than meets the eye. What
makes certain game squares more likely to be landed-on than others? Illinois
Avenue doesn’t seem to be special compared to other properties. . . It turns
out that we can model the MONOPOLY
R game board to calculate the exact
probabilities of landing on a certain square.
Rules
The objective of the game is to bankrupt all opponents, though most games
played with family and friends end when it is apparent that someone will
win. A typical game of MONOPOLY
R uses the following items:
• 1 game board
• 2 dice
11
SP.268 Monopoly
• 32 houses
• 12 hotels
• 16 Chance cards
• $15140 in MONOPOLY
R money
The Chance and Community Chest cards are placed face down on the game
board, and a player must pick one of the cards when he lands on the Chance
or the Community Chest game squares. Each player is given $1500 to be-
gin the game. All remaining money, game piece, houses, hotels, and deeds
of unsold property go to the Bank. The Bank collects all taxes, fines,
loans, and interest. The Bank never goes ‘broke.’ If the Bank runs out
of MONOPOLY
R money, then more can be issued (see fun fact above).
Players begin on the Go square, roll two dice, and advance as many steps as
dots displayed on the the two dice. A player can buy any property, utility,
or railroad that isn’t already owned by another player, or must to draw
Chance/Community Chest cards, pay rent, fines, or go to Jail as dictated
by the square he lands on. If a player throws a double, then he moves his
token the number of steps, is subject to whatever privileges or penalties of
the square he lands on, and then tosses the dice again. If a player tosses
three doubles in a single turn, he must go to Jail.
Landing on the Jail square is just ‘visiting Jail’, while landing on the ‘Go to
Jail’ square, drawing a ‘Go to Jail’ card, and tossing doubles 3 times during
a turn are actual Jail sentences. Any Jail term lasts 3 turns. A player tosses
dice at each turn, and if he tosses a double, then he is free to get out of jail
and advances the number of steps as his double shows. That player does not
take another turn. A player gets out of Jail if he has a ‘Get out of Jail Free’
card, or if another player is willing to sell him a ‘Get out of Jail Free’ card
at a negotiated price, or if the player pays a $50 fine.
12
SP.268 Monopoly
More details of the rules of the game will unfurl as we analyze the game.
First, a bit of linear algebra.
13
SP.268 Monopoly
Example 6
1 2
Find the eigenvalues and eigenvectors of A = 2 4 .
1−λ 2
A − λI = 2 4−λ
λ2 − 5λ = 0 gives λ1 = 0 and λ2 = 5.
1 2 x1 0 2
(A − 0I) = x2 = gives eigenvector −1
12 24 x1 00 1
(A − 5I) = 2 4 x2 = 0 gives eigenvector 2
As a side note, because the vectors that make up A are constant multiples of
each other, we know that A itself is a singular matrix. The determinant of
a matrix can be found by taking the product of all its eigenvalues, so if the
determinant is zero, then we know one of the eigenvalues must be zero.
Markov Chains
14
SP.268 Monopoly
probability that a player ends his turn in that state. Our state, H, belongs
to the set S of the state space of size 40. The Markov chain is described
in terms of its transition probabilities pij , which is the probability that
we’ll go from state i to state j at a time step. The transition probabilities
sum to 1.
pij = P (Hn+1 = j|Hn = i), i, j ∈ S
X
pij = 1
j
The probability that we’re in a certain state at time step n depends only
on our state at time step n − 1, and is independent of all states besides the
previous state:
The transition matrix captures all the transition probabilities and operates
on our state distribution vector. Such a matrix is called a Markov matrix,
and it is also a square matrix.
p00 p01 . . . p0m
p10 p11 . . . p1m
.. .. . . ..
. . . .
pm0 pm1 . . . pmm
Special matrices will have special eigenvalues and eigenvectors, and for Markov
matrices, all entries are positive and every row adds to 1. Can you see why
they must add to 1? The largest eigenvalue is 1, and the corresponding
eigenvector is the state that comes out at the end. The eigenvectors of other
eigenvalues fall to 0 over time.
Example 7
The ESG elevator has two states: FAIL and MEGAFAIL (we pretend that
those dark days of a completely broken elevator never happened). If it is
FAIL today, then the probability that it becomes MEGAFAIL tomorrow
15
SP.268 Monopoly
x1
(A − I)x = 0 gives us eigenvector x2 wherex1 = x2
So, surprisingly, although it seems more likely that we would end up with a
MEGAFAIL elevator, the odds are actually even between FAIL and MEGAFAIL.
MONOPOLY
R
Initial Analysis
16
SP.268 Monopoly
The 1st row of our 40 × 40 Markov transition matrix M would be H10 . The
second row of the matrix would be this vector with 3 zeros before the begin-
ning of the fractions and 27 zeros after, in essence, the same vector ‘shifted’
over by 1. The third row would be shifted over again, so on until we’ve
completed all 40 rows of the matrix. We won’t enumerate all 1600 entries.
To get our state probability vector at time step n, we would just calculate
An Hn . For sufficiently large n, our state probability vector would be the
eigenvector of A that has eigenvalue 1. Our distribution slowly approaches
1
one with all entries equal to 40
To deal with the rule about tossing doubles, we can modify X1 to have 35
nonzero elements. The maximum number of spaces a player can move is
35, (if a player rolls a {(6,6),(6,6),(6,5)}). Note that the ‘Go to Jail’ square
17
SP.268 Monopoly
has probability 0 because our state distribution is “the probability that the
player ends a turn on that square”. If a player lands on ‘Go to Jail’, then he
would go immediately to Jail and end his turn there.
Jail is a little more complicated. The player has several options when he
goes to Jail: (1) stay there for three turns, (2) roll doubles and get out of
Jail early, (3) pay $50 fine and get out of Jail early, and (4) use a ‘Get out
Jail Free’ card. For a single transition matrix, we can only take one of the
cases. Then we can switch transition matrices depending on what part of
the game we’re in. Early on in the game, it is beneficial to move around as
much as possible to buy up property; later in the game, ‘Jail’ is like ‘Free
Parking’–you don’t have to pay sky-high rents for a bunch of turns while
other players duke it out. If we’re dealing with the case where the player
chooses to stay in Jail, then we’ll have to break up the ‘Jail’ state into three
separate states: ‘Entered Jail on Last Turn’, ‘Entered Jail Two Turns Ago’,
‘Will Leave Jail on Next Turn’. On any of those states, it is possible to roll a
double and transition to another game square. The transition matrix should
reflect this.
18
SP.268 Monopoly
Rent-collecting is when things actually start to get interesting. After all, the
whole point of the game is to bankrupt the other players. What strategy
should we take in building houses and hotels? What can we use from our
probabilistic analysis? If we take the actual decimal values of the probabil-
ities and analyze the time of the break-event point (total cost of buildings
divided by expected earnings from property per turn; how would you calcu-
late the expected earnings?) which is when rents collected becomes greater
than the cost of building the houses and hotels, we find that with 2 houses
or fewer, it typically takes 20 moves or more to break even. With 3 houses,
the chances are significantly better. It is even better than building 4 houses
or a hotel. This is preferable strategy because one of the principle strategies
of MONOPOLY
R is to deplete accounts of other players fast while accumu-
lating fast yourself (so that you can purchase more property and build more
buildings). If the break-even point takes too long, then we are wasting valu-
able resources that could have been allocated to buildings on other properties
and raising the rents of those properties.
Remarks
Other similar board games can be modeled in the same way. The premesis of
Markov chains is that the next state is independent of all previous states–it
ony depends on the current state.
19