Elements of probability

Elena Ballante,
Department of Political and Social Sciences
elena.ballante@unipv.it
Table of contents

• Bayes’ theorem
• Returning to De Mere's bet
• The birthday paradox
• A few exercises to conclude
Bayes’ theorem
Law of total probability

Events B1 and B2 are disjoint (B1 ∩ B2 = ∅) and exhaustive (B1 ∪ B2 = Ω).

The sets A ∩ B1 e A ∩ B2 are disjoint, then P(A) = P(A ∩ B1) + P(A ∩ B2)
Applying the multiplication rule we obtain:
P(A) = P(B1) P(A| B1) + P(B2) P(A| B2)
This formula expresses the rule of total probability considering 2 events,
can be generalized to n events B1, B2, . . . ,Bn disjointed and exhaustive
Law of total probability

Let A be an event and {B1, B2, . . . ,Bn} a family of events in the sample space Ω
incompatible, exhaustive with P(Bi)≠0 for every i.

Then it can be shown that:

P(A) = P(A|B1) P(B1)+P(A| B2) P(B2)+ … +P(A| Bn) P(Bn) =
= σ𝑛𝑖=1 P(A|Bi ) P(Bi ) (compact notation of the sum)
Bayes’ theorem

Let A be an event with P(A) > 0 and let {B1, B2, . . . ,Bn} be events of the
sample space Ω that satisfy the hypotheses of the total probability theorem.
Then for every k

P(A|𝐵𝑘) P(𝐵𝑘)
P(𝐵𝑘 |A) = σ𝑛 =
𝑖=1 P(A |𝐵 𝑖 ) P( 𝐵 𝑖 )

P(A|𝐵𝑘) P(𝐵𝑘)
= =
P(A|𝐵1 ) P(𝐵1 )+P(A|𝐵2) P(𝐵2)+⋯.+P(A|𝐵𝑛 ) P(𝐵𝑛 )

P(A|𝐵𝑘) P(𝐵𝑘)
=
𝑃(𝐴)
Bayes’ theorem

Example
An industry has installed a system for quality control, which guarantees that, if a
part is defective, it is disposed of with probability 0.999.
There is a probability of 0.002 that even a non-defective part will be discarded.
It also knows that the probability of a part being defective is 0.05.
Calculate the probability that a piece that has not been eliminated at the quality
control is defective.
Bayes’ theorem

Example
An industry has installed a system for quality control, which
guarantees that, if a part is defective, it is disposed of with probability 0.999.
There is a probability of 0.002 that even a non-defective part will be discarded.
It also knows that the probability of a part being defective is 0.05.
Calculate the probability that a part that has not been eliminated at the quality control is
defective.

Event E: “The part is discarded”.

Event D: “the part is defective”

We know that: P(E|D) = 0.999 ഥ) = 0.002

P(E| D P(D) = 0.05

ഥ)
We need to evaluate P(D| E
Bayes’ theorem

Example
Event E: “The part is discarded”.
Event D: “the part is defective”

We know that: P(E|D) = 0.999 ഥ) = 0.002

P(E| D P(D) = 0.05

ഥ).
We need to evaluate P(D| E
P( ഥ
E|D)P(D)
Applying the Bayes: theorem: P(D| E ഥ) = ഥ ഥ D)P(
ഥ ഥ)
P(E|D)P(D)+P(E| D
We have that:
ഥ|D) = 1 - P(E|D) = 1 – 0.999 = 0.001
P(E P(Eഥ |Dഥ ) = 1 - P(E|D
ഥ ) = 1 – 0.002 = 0.998
ഥ ) = 1 - P(D) = 1 – 0.05 = 0.95
P(D
0.001 ∗ 0.05
Using the theorem we obtain: P(D| E ഥ) = = 0.0000527
0.001∗ 0.05+0.998 ∗0.95
Bayes’ theorem

Example
We know that the percentage of people who have red hair in Piedmont, Sardinia
and Marche is 5%, 1% and 2%, respectively, and that the three regions have 4.5,
2 e 1.5 million inhabitants, respectively.
Chosen a person at random from among the inhabitants of the three regions,
calculate the probability that the person's region of origin be the Sardinia,
supposing that:
1 they has red hair
2 they does not have red hair
Teorema di Bayes
Esempio We know that the percentage of people who have red hair in Piedmont,
Sardinia and Marche is 5%, 1% and 2%, respectively, and that the three regions
have 4.5, 2 e 1.5 million inhabitants, respectively.
Chosen a person at random from among the inhabitants of the three regions,
calculate the probability that the person's region of origin be the Sardinia,
supposing that:
1 they has red hair

R: red hair; P: Piedmont region of origin; S: region of origin Sardinia;

M: region of origin Marche.
Therefore: P(R|P) = 0.05, P(R|S) = 0.01, P(R|M) = 0.02, P(P) = 4.5/8 = 9/16,
P(S) = 2/8 = 4/16, P(M) = 1.5/8 = 3/16
Using the theorem we obtain :P(S|R) =
P(R|S)P(S) 0.01 4/16
= = 0.0727
P(R|S)P(S)+P(R|P)P(P)+P(R|M)P(M) 0.01 4/16+0.05 9/16+0.02 3/16
Bayes’ theorem
Example We know that the percentage of people who have red hair in Piedmont,
Sardinia and Marche is 5%, 1% and 2%, respectively, and that the three regions have
4.5, 2 e 1.5 million inhabitants, respectively.
Chosen a person at random from among the inhabitants of the three regions, calculate
the probability that the person's region of origin be the Sardinia, supposing that:
2 they does not have red hair
R: red hair; P: Piedmont region of origin; S: region of origin Sardinia;
M: region of origin Marche.
Therefore : P(R|P) = 0.05, P(R|S) = 0.01, P(R|M) = 0.02, P(P) = 4.5/8 = 9/16,
P(S) = 2/8 = 4/16, P(M) = 1.5/8 = 3/16
P(Rത |P) = 0.95, P(Rത | S) = 0.99 e P(Rത | M) = 0.98
Using the theorem we obtain:
P( ഥ
R|S)P(S) 0.99 4/16
ത
P(S| R) = ഥ ഥ ഥ = = 0.2563
P(R|S)P(S)+P(R|P)P(P)+P( R|M)P(M) 0.99 4/16+0.95 9/16+0.98 3/16
Returning to De Mere's bet
De Mere's bet

In 1654 the Chevalier de Méré wanted to settle a question about a

gambling game. To solve this problem, the field of probability was
developed by famous mathematicians Blaise Pascal and Pierre de
Fermat.

The question was:

"Is it easier to make at least one 6 in 4 rolls of a dice or at least a
double 6 in 24 rolls of a pair of dice?"
De Mere's bet

"Is it easier to make at least one 6 in 4 rolls of a dice or at least a

double 6 in 24 rolls of a pair of dice?"

We define the following events:

E: “at least one 6 in 4 rolls of a dice”
F: “at least a double 6 in 24 rolls of a pair of dice”
A: “a 6 in a roll of a dice”
B: “a double 6 in a roll of a pair of dice”
De Mere's bet

E: “at least one 6 in 4 rolls of a dice”

F: “at least a double 6 in 24 rolls of a pair of dice”
A: “a 6 in a roll of a dice”
B: “a double 6 in a roll of a pair of dice”
We already saw that: P(A) = 1/6 and P(B) = 1/36

Calculate P(Eത ) and P(Fത ). We can exploit the property of independence of the results
in the roll of the dice:
ഥ) P(A
P(Eത) = P(A ഥ) P(Aഥ) P(A
ഥ) = 5/6 * 5/6 * 5/6 * 5/6=(5/6)4=0.482
P(E) = 1 - P(Eത ) = 1 - (5/6)4 = 0.518

P(Fത ) = P(Bത ) P(Bത ) P(Bത ) ... P(Bത ) =35/36 * 35/36 * 35/36 * … * 35/36= (35/36)24

P(F) = 1 - P(Fത ) = 1-(35/36)24 = 0.491

De Mere's bet

Therefore P(F) < P(E) . . . That's why it's important to have

"good" friends!

Blaise Pascal e Pierre de Fermat

The birthday paradox
The birthday paradox

How likely is it to find two people born on the same day

of the year in this classroom? High or low?

The birthday paradox (1939, Richard von Mises) states that the
probability of at least two people in a group having their birthdays on
the same day is far greater than intuition might say, even if to arrive at
the certain event it is necessary to consider a group of at least. . . 366
people (non-leap year).
The birthday paradox

Let's do the math with these hypotheses, resorting to what we have learned about
conditional probability.
To make the calculation easier, we assume that:
• the years have 365 days
• It is equiprobable to be born every day of the year, even if in reality it is not true.

Some data. . .
Define An = “at least two between n people born on the same date” and we focus on
its complement P(Aഥn).
The simplified reasoning is. . .
Given any person in a group, there are 364 out of 365 cases in which a second person's
birthday occurs on a different day; If you consider a third person, there are 363 out of
365 cases in which they has a birthday on a different day than the first two people . . .
The birthday paradox

Define An = “at least two between n people born on the same date” and
we focus on its complement P(Aഥn).
Possible cases to the complement:
Birthday1 Birthday2 Birthday3 … Birthdayn
365 365 365 … 365 = (365)n = D(365,n(r ))
Favorable cases to the complement:
Birthday1 Birthday2 Birthday3 ... Birthdayn
365 364 363 … (365-n+1) = D(365,n)

365 364 363 … (365−n+1)

P(An) = 1 -
(365)n
The birthday paradox

365 364 363 … (365−n+1)

P(An) = 1 -
(365)n

0.027
A few exercises to conclude
An exercise on probability

From a survey carried out on 100 customers of a company, it emerged that

the number of satisfied customers is 90.

• Supposing you extract with replacement 2 clients, calculate the

probability that both are not satisfied
• Supposing you extract 2 different customers, calculate the probability
that both are satisfied
An exercise on probability

From a survey carried out on 100 customers of a company, it

emerged that the number of satisfied customers is 90.

• Supposing you extract with replacement 2 clients, calculate the

probability that both are not satisfied
An exercise on probability

From a survey carried out on 100 customers of a company, it emerged that

the number of satisfied customers is 90.

• Supposing you extract with replacement 2 clients, calculate the probability

that both are not satisfied

E: “The client is not satisfied"

P(E) = 10/100 = 0.10

Draws are with replacement, so I can use the multiplication rule for
independent events:
The probability is: 10/100 * 10/100= 0.01
An exercise on probability

From a survey carried out on 100 customers of a company, it

emerged that the number of satisfied customers is 90.

• Supposing you extract 2 different customers, calculate the

probability that both are satisfied
An exercise on probability

From a survey carried out on 100 customers of a company, it emerged that

the number of satisfied customers is 90.

• Supposing you extract 2 different customers, calculate the probability that

both are satisfied

E1: “The first client extract is satisfied"

P(E1) = 90/100=0.90

Draws are not with replacement.

E2: “The second client extract is satisfied"
P(E2|E1) = 89/99= 0.899
The probability requested is: 90/100 * 89/99=0.809
An exercise on probability

From a survey carried out on 100 customers of a company, it emerged that the
number of satisfied customers is 90.
• Supposing you extract 2 different customers, calculate the probability that
both are satisfied.

The number of possible draws, i.e. all the ways to choose 2 customers from a
group of 100: 100
2
[combination, n=100, k=2]

The number of favorable draws, i.e. all simple combinations of groups of 2

satisfied customers chosen from the 90 satisfied customers: 90
2
[combination,
n=90, k=2]
An exercise on probability

Possible cases: 100

2
Favorable cases: 90
2
90 90!
2 2! 88! 90! 2! 98! 90 89
The probability requested is: 100 = 100! = = =
2 2! 88! 100! 100 99
2!98!
0.809
An exercise on probability

An urn contains 20 coloured balls: 8 red, 3 white and 9 black balls. Three balls
are randomly drawn, without putting them back into the urn after the
extraction.
Determine the probability that:
1. all three are red;
2. at least one is white;
3. one for each color, without considering the order of extraction;
4. one red, one white and one black, in this order.
An exercise on probability

An urn contains 20 coloured balls: 8 red, 3 white and 9 black balls. Three
balls are randomly drawn, without putting them back into the urn after the
extraction.
Determine the probability that all three are red.

Solution 1: combinatorics
There are 83 ways to choose, without considering the order, 3 red balls
among the 8 available (favorable cases) and 20
3
ways to choose 3 balls of
any color (Possible cases).
8
3 8! 3!17! 876
P(«three red balls are drawn»)= ൘ 20 = = =0.0491
3 3!5! 20! 20 19 18
An exercise on probability

An urn contains 20 coloured balls: 8 red, 3 white and 9 black balls. Three balls are
randomly drawn, without putting them back into the urn after the extraction.
Determine the probability that all three are red.

Solution 2: Conditional Probability

Ei=«The i-th ball drawn is red» i=1,2,3
For the first draw we have 8 red balls out of 20 so P(E1)=8/20
In the second draw, knowing that we have drawn a red ball in the first, we have
P(E2|E1)=7/19
Similarly, P(E3|E2∩E1)=6/18
Then the probability that the three balls drawn are red is
6 7 8
P(E3 ∩ E2∩E1)= P(E3|E2∩E1)P(E2∩E1)= P(E3|E2∩E1)P(E2|E1)P(E1)= = 0.0491
18 19 20
An exercise on probability

An urn contains 20 coloured balls: 8 red, 3 white and 9 black balls. Three balls are
randomly drawn, without putting them back into the urn after the extraction.
Determine the probability that at least one white ball will be drawn.

ത
Let E=«at least one white ball is drawn», consider 𝐸=«none of the balls drawn is
white».
The probability that no white ball will be drawn is calculated by using combinatorics.
The favorable cases will be all the ways of extracting the 17 non-white balls, therefore
17 20
3
, while the possible cases are as before 3
.

17
ത
Then P(E)=1-P(𝐸)=1- 3
൘ 20 =1-0.596=0.404
3
An exercise on probability

An urn contains 20 coloured balls: 8 red, 3 white and 9 black balls. Three
balls are randomly drawn, without putting them back into the urn after the
extraction.
Determine the probability that one of each colour will be drawn, regardless
of the order.

8 3 9
1 1 1 839321
P(«one ball per color»)= ൘ 20 = 20 19 18 =0.189
3
An exercise on probability

An urn contains 20 coloured balls: 8 red, 3 white and 9 black balls. Three balls are
randomly drawn, without putting them back into the urn after the extraction.
Determine the probability that a red, white, and black will be drawn, in this order.

There are 8 * 3 * 9 ways to choose a red ball, a white ball and a black ball.
The ways of choosing any three balls considering the order are D20,3
839 839
P(red, white, black)= = =0.0316
D20,3 20 19 18

Instead, with conditional probability we write:

P(N3 ∩ B2 ∩ R1)=P(N3|B2 ∩ R1)P(B2 ∩ R1)= P(N3|B2 ∩ R1)P(B2 |R1) P(R1) =
9 3 8
= = 0.0316
18 19 20