Conditional Probability - Ch2
Conditional Probability - Ch2
Conditional Probability - Ch2
Conditional probability
They toss the coin once and the result is heads; call this event E. How should
we now reassess their chances? We have
E = {HHH, HHT, HT H, HT T },
and if we are given the information that the result of the first toss is heads, then E
now becomes the sample space of the experiment, since the outcomes not in E are
no longer possible. In the new experiment, the outcomes ‘Alice pays’ and ‘Bob
pays’ are
23
24 CHAPTER 2. CONDITIONAL PROBABILITY
Thus the new probabilities that Alice and Bob pay for dinner are 3/4 and 1/4
respectively.
In general, suppose that we are given that an event E has occurred, and we
want to compute the probability that another event A occurs. In general, we can no
longer count, since the outcomes may not be equally likely. The correct definition
is as follows.
Let E be an event with non-zero probability, and let A be any event. The
conditional probability of A given E is defined as
P(A ∩ E)
P(A | E) = .
P(E)
Again I emphasise that this is the definition. If you are asked for the definition
of conditional probability, it is not enough to say “the probability of A given that
E has occurred”, although this is the best way to understand it. There is no reason
why event E should occur before event A!
Note the vertical bar in the notation. This is P(A | E), not P(A/E) or P(A \ E).
Note also that the definition only applies in the case where P(E) is not equal
to zero, since we have to divide by it, and this would make no sense if P(E) = 0.
To check the formula in our example:
P(A ∩ E) 3/8 3
P(A | E) = = = ,
P(E) 1/2 4
P(B ∩ E) 1/8 1
P(B | E) = = = .
P(E) 1/2 4
It may seem like a small matter, but you should be familiar enough with this
formula that you can write it down without stopping to think about the names of
the events. Thus, for example,
P(A ∩ B)
P(A | B) =
P(B)
if P(B) 6= 0.
Example A random car is chosen among all those passing through Trafalgar
Square on a certain day. The probability that the car is yellow is 3/100: the
probability that the driver is blonde is 1/5; and the probability that the car is
yellow and the driver is blonde is 1/50.
Find the conditional probability that the driver is blonde given that the car is
yellow.
2.2. GENETICS 25
Solution: If Y is the event ‘the car is yellow’ and B the event ‘the driver is blonde’,
then we are given that P(Y ) = 0.03, P(B) = 0.2, and P(Y ∩ B) = 0.02. So
P(B ∩Y ) 0.02
P(B | Y ) = = = 0.667
P(Y ) 0.03
to 3 d.p. Note that we haven’t used all the information given.
There is a connection between conditional probability and independence:
Proposition 2.1 Let A and B be events with P(B) 6= 0. Then A and B are indepen-
dent if and only if P(A | B) = P(A).
Proof The words ‘if and only if’ tell us that we have two jobs to do: we have to
show that if A and B are independent, then P(A | B) = P(A); and that if P(A | B) =
P(A), then A and B are independent.
So first suppose that A and B are independent. Remember that this means that
P(A ∩ B) = P(A) · P(B). Then
P(A ∩ B) P(A) · P(B)
P(A | B) = = = P(A),
P(B) P(B)
that is, P(A | B) = P(A), as we had to prove.
Now suppose that P(A | B) = P(A). In other words,
P(A ∩ B)
= P(A),
P(B)
using the definition of conditional probability. Now clearing fractions gives
P(A ∩ B) = P(A) · P(B),
which is just what the statement ‘A and B are independent’ means.
This proposition is most likely what people have in mind when they say ‘A
and B are independent means that B has no effect on A’.
2.2 Genetics
Here is a simplified version of how genes code eye colour, assuming only two
colours of eyes.
Each person has two genes for eye colour. Each gene is either B or b. A child
receives one gene from each of its parents. The gene it receives from its father
is one of its father’s two genes, each with probability 1/2; and similarly for its
mother. The genes received from father and mother are independent.
If your genes are BB or Bb or bB, you have brown eyes; if your genes are bb,
you have blue eyes.
26 CHAPTER 2. CONDITIONAL PROBABILITY
Example Suppose that John has brown eyes. So do both of John’s parents. His
sister has blue eyes. What is the probability that John’s genes are BB?
Solution John’s sister has genes bb, so one b must have come from each parent.
Thus each of John’s parents is Bb or bB; we may assume Bb. So the possibilities
for John are (writing the gene from his father first)
each with probability 1/4. (For example, John gets his father’s B gene with prob-
ability 1/2 and his mother’s B gene with probability 1/2, and these are indepen-
dent, so the probability that he gets BB is 1/4. Similarly for the other combina-
tions.)
Let X be the event ‘John has BB genes’ and Y the event ‘John has brown
eyes’. Then X = {BB} and Y = {BB, Bb, bB}. The question asks us to calculate
P(X | Y ). This is given by
P(X ∩Y ) 1/4
P(X | Y ) = = = 1/3.
P(Y ) 3/4
Example An ice-cream seller has to decide whether to order more stock for the
Bank Holiday weekend. He estimates that, if the weather is sunny, he has a 90%
chance of selling all his stock; if it is cloudy, his chance is 60%; and if it rains, his
chance is only 20%. According to the weather forecast, the probability of sunshine
is 30%, the probability of cloud is 45%, and the probability of rain is 25%. (We
assume that these are all the possible outcomes, so that their probabilities must
add up to 100%.) What is the overall probability that the salesman will sell all his
stock?
This problem is answered by the Theorem of Total Probability, which we now
state. First we need a definition. The events A1 , A2 , . . . , An form a partition of the
sample space if the following two conditions hold:
(a) the events are pairwise disjoint, that is, Ai ∩ A j = 0/ for any pair of events Ai
and A j ;
(b) A1 ∪ A2 ∪ · · · ∪ An = S .
2.3. THE THEOREM OF TOTAL PROBABILITY 27
Another way of saying the same thing is that every outcome in the sample space
lies in exactly one of the events A1 , A2 , . . . , An . The picture shows the idea of a
partition.
A1 A2 . . . An
Theorem 2.2 Let A1 , A2 , . . . , An form a partition of the sample space with P(Ai ) 6=
0 for all i, and let B be any event. Then
n
P(B) = ∑ P(B | Ai ) · P(Ai ).
i=1
B
A1 A2 . . . An
Consider the ice-cream salesman at the start of this section. Let A1 be the
event ‘it is sunny’, A2 the event ‘it is cloudy’, and A3 the event ‘it is rainy’. Then
A1 , A2 and A3 form a partition of the sample space, and we are given that
Let B be the event ‘the salesman sells all his stock’. The other information we are
given is that
You will now realise that the Theorem of Total Probability is really being used
when you calculate probabilities by tree diagrams. It is better to get into the habit
of using it directly, since it avoids any accidental assumptions of independence.
One special case of the Theorem of Total Probability is very commonly used,
and is worth stating in its own right. For any event A, the events A and A0 form a
partition of S . To say that both A and A0 have non-zero probability is just to say
that P(A) 6= 0, 1. Thus we have the following corollary:
Corollary 2.3 Let A and B be events, and suppose that P(A) 6= 0, 1. Then
Example I have two red pens, one green pen, and one blue pen. I select two
pens without replacement.
(a) What is the probability that the first pen chosen is red?
(b) What is the probability that the second pen chosen is red?
For the first pen, there are four pens of which two are red, so the chance of
selecting a red pen is 2/4 = 1/2.
For the second pen, we must separate cases. Let A1 be the event ‘first pen red’,
A2 the event ‘first pen green’ and A3 the event ‘first pen blue’. Then P(A1 ) = 1/2,
P(A2 ) = P(A3 ) = 1/4 (arguing as above). Let B be the event ‘second pen red’.
If the first pen is red, then only one of the three remaining pens is red, so that
P(B | A1 ) = 1/3. On the other hand, if the first pen is green or blue, then two of
the remaining pens are red, so P(B | A2 ) = P(B | A3 ) = 2/3.
2.5. BAYES’ THEOREM 29
using the definition of conditional probability twice. (Note that we need both A
and B to have non-zero probability here.) Now divide this equation by P(B) to get
the result.
30 CHAPTER 2. CONDITIONAL PROBABILITY
Example Consider the ice-cream salesman from Section 2.3. Given that he sold
all his stock of ice-cream, what is the probability that the weather was sunny?
(This question might be asked by the warehouse manager who doesn’t know what
the weather was actually like.) Using the same notation that we used before, A1
is the event ‘it is sunny’ and B the event ‘the salesman sells all his stock’. We are
asked for P(A1 | B). We were given that P(B | A1 ) = 0.9 and that P(A1 ) = 0.3, and
we calculated that P(B) = 0.59. So by Bayes’ Theorem,
P(B | A1 )P(A1 ) 0.9 × 0.3
P(A1 | B) = = = 0.46
P(B) 0.59
to 2 d.p.
Example Consider the clinical test described at the start of this section. Suppose
that 1 in 1000 of the population is a carrier of the disease. Suppose also that the
probability that a carrier tests negative is 1%, while the probability that a non-
carrier tests positive is 5%. (A test achieving these values would be regarded as
very successful.) Let A be the event ‘the patient is a carrier’, and B the event ‘the
test result is positive’. We are given that P(A) = 0.001 (so that P(A0 ) = 0.999),
and that
P(B | A) = 0.99, P(B | A0 ) = 0.05.
(a) A patient has just had a positive test result. What is the probability that the
patient is a carrier? The answer is
P(B | A)P(A)
P(A | B) =
P(B | A)P(A) + P(B | A0 )P(A0 )
0.99 × 0.001
=
(0.99 × 0.001) + (0.05 × 0.999)
0.00099
= = 0.0194.
0.05094
(b) A patient has just had a negative test result. What is the probability that the
patient is a carrier? The answer is
P(B0 | A)P(A)
P(A | B0 ) =
P(B0 | A)P(A) + P(B0 | A0 )P(A0 )
2.6. ITERATED CONDITIONAL PROBABILITY 31
0.01 × 0.001
=
(0.01 × 0.001) + (0.95 × 0.999)
0.00001
= = 0.00001.
0.94095
So a patient with a negative test result can be reassured; but a patient with a posi-
tive test result still has less than 2% chance of being a carrier, so is likely to worry
unnecessarily.
Of course, these calculations assume that the patient has been selected at ran-
dom from the population. If the patient has a family history of the disease, the
calculations would be quite different.
so
P(A ∩ B ∩C) = P(C | A, B)P(A ∩ B).
Now we also have
P(A ∩ B) = P(B | A)P(A),
so finally (assuming that P(A ∩ B) 6= 0), we have
What is A2 ∩ A3 ? It is simply the event that all three people have birthdays on
different days.
Now this process extends. If Ai denotes the event ‘pi ’s birthday is not on the
same day as any of p1 , . . . , pi−1 ’, then
Call this number qi ; it is the probability that all of the people p1 , . . . , pi have
their birthdays on different days.
The numbers qi decrease, since at each step we multiply by a factor less than 1.
So there will be some value of n such that
that is, n is the smallest number of people for which the probability that they all
have different birthdays is less than 1/2, that is, the probability of at least one
coincidence is greater than 1/2.
By calculation, we find that q22 = 0.5243, q23 = 0.4927 (to 4 d.p.); so 23
people are enough for the probability of coincidence to be greater than 1/2.
Now return to a question we left open before. What is the probability of the
event A3 ? (This is the event that p3 has a different birthday from both p1 and p2 .)
2
If p1 and p2 have different birthdays, the probability is 1 − 365 : this is the
calculation we already did. On the other hand, if p1 and p2 have the same birthday,
1
then the probability is 1 − 365 . These two numbers are P(A3 | A2 ) and P(A3 | A02 )
respectively. So, by the Theorem of Total Probability,
to 4 d.p.
Problem How many people would you need to pick at random to ensure that
the chance of two of them being born in the same month are better than even?
Assuming all months equally likely, if Bi is the event that pi is born in a dif-
ferent month from any of p1 , . . . , pi−1 , then as before we find that
so
1 2
P(B1 ∩ · · · ∩ Bi ) = (1 − 12 )(1 − 12 )(1 − i−1
12 ).
We calculate that this probability is
for i = 4 and
for i = 5. So, with five people, it is more likely that two will have the same birth
month.
A true story. Some years ago, in a probability class with only ten students, the
lecturer started discussing the Birthday Paradox. He said to the class, “I bet that
no two people in the room have the same birthday”. He should have been on safe
ground, since q11 = 0.859. (Remember that there are eleven people in the room!)
However, a student in the back said “I’ll take the bet”, and after a moment all the
other students realised that the lecturer would certainly lose his wager. Why?
(Answer in the next chapter.)
(a) Neither of Sally’s parents has cystic fibrosis. Nor does she. However, Sally’s
sister Hannah does have cystic fibrosis. Find the probability that Sally has
at least one C gene (given that she does not have cystic fibrosis).
(c) Harry and Sally plan to have a child. Find the probability that the child will
have cystic fibrosis (given that neither Harry nor Sally has it).
Solution During this solution, we will use a number of times the following prin-
ciple. Let A and B be events with A ⊆ B. Then A ∩ B = A, and so
P(A ∩ B) P(A)
P(A | B) = = .
P(B) P(B)
(a) This is the same as the eye colour example discussed earlier. We are given
that Sally’s sister has genes CC, and one gene must come from each parent. But
2.7. WORKED EXAMPLES 35
neither parent is CC, so each parent is CN or NC. Now by the basic rules of
genetics, all the four combinations of genes for a child of these parents, namely
CC,CN, NC, NN, will have probability 1/4.
If S1 is the event ‘Sally has at least one C gene’, then S1 = {CN, NC,CC}; and
if S2 is the event ‘Sally does not have cystic fibrosis’, then S2 = {CN, NC, NN}.
Then
P(S1 ∩ S2 ) 2/4 2
P(S1 | S2 ) = = = .
P(S2 ) 3/4 3
(b) We know nothing specific about Harry, so we assume that his genes are
randomly and independently selected from the population. We are given that the
probability of a random gene being C or N is 1/50 and 49/50 respectively. Then
the probabilities of Harry having genes CC, CN, NC, NN are respectively (1/50)2 ,
(1/50) · (49/50), (49/50) · (1/50), and (49/50)2 , respectively. So, if H1 is the
event ‘Harry has at least one C gene’, and H2 is the event ‘Harry does not have
cystic fibrosis’, then
(c) Let X be the event that Harry’s and Sally’s child has cystic fibrosis. As in
(a), this can only occur if Harry and Sally both have CN or NC genes. That is,
X ⊆ S3 ∩ H3 , where S3 = S1 ∩ S2 and H3 = H1 ∩ H2 . Now if Harry and Sally are
both CN or NC, these genes pass independently to the baby, and so
P(X) 1
P(X | S3 ∩ H3 ) = = .
P(S3 ∩ H3 ) 4
Thus,
1 P(S1 ∩ S2 ) P(H1 ∩ H2 )
= · ·
4 P(S2 ) P(H2 )
1
= · P(S1 | S2 ) · P(H1 | H2 )
4
1 2 2
= · ·
4 3 51
1
= .
153
I thank Eduardo Mendes for pointing out a mistake in my previous solution to
this problem.
Question The Land of Nod lies in the monsoon zone, and has just two seasons,
Wet and Dry. The Wet season lasts for 1/3 of the year, and the Dry season for 2/3
of the year. During the Wet season, the probability that it is raining is 3/4; during
the Dry season, the probability that it is raining is 1/6.
(a) I visit the capital city, Oneirabad, on a random day of the year. What is the
probability that it is raining when I arrive?
(b) I visit Oneirabad on a random day, and it is raining when I arrive. Given this
information, what is the probability that my visit is during the Wet season?
(c) I visit Oneirabad on a random day, and it is raining when I arrive. Given this
information, what is the probability that it will be raining when I return to
Oneirabad in a year’s time?
(You may assume that in a year’s time the season will be the same as today but,
given the season, whether or not it is raining is independent of today’s weather.)
Solution (a) Let W be the event ‘it is the wet season’, D the event ‘it is the dry
season’, and R the event ‘it is raining when I arrive’. We are given that P(W ) =
1/3, P(D) = 2/3, P(R | W ) = 3/4, P(R | D) = 1/6. By the ToTP,
(c) Let R0 be the event ‘it is raining in a year’s time’. The information we are
given is that P(R ∩ R0 | W ) = P(R | W )P(R0 | W ) and similarly for D. Thus