Elements of probability

Elena Ballante,
Department of Political and Social Sciences
elena.ballante@unipv.it
Table of contents

• Conditional probability
• Indipendence
• Bayes’ theorem
Conditional probability

3
Conditional probability

Let A and B be any two events in the sample space and let P(A)≠ 0.
The probability of event B, assuming that event A has already occurred, is called the
probability of B conditional on A and is defined by:
P(A ∩ B)
P(B|A) =
P(A)

Similarly, if P(B)≠0, the probability of A conditional on B is defined by

P(A ∩ B)
P(A|B) =
P(B)

An immediate consequence of the definition of conditional probability is the

Multiplication Rule:
P(A ∩ B) = P(A) P(B|A) if P(A) ≠ 0.
P(A ∩ B) = P(B) P(A|B) if P(B) ≠ 0.
Conditional probability

Example
Given an urn containing 15 white balls and 5 black balls, we indicate with A the event
"white ball extraction" and with B the event "black ball extraction".
Calculate the probability of getting in two consecutive draws a white ball and then a
black ball, assuming that the first ball drawn does not be put back into the urn.
Conditional probability

20 balls in the urn, 15 white and 5 black.

The probability of drawing a white ball on the first draw is P(A) = 15/20 = 0.75
The probability of drawing a black ball after you have already drawn a white ball,
without replacement, is 5/19, because there are only 19 more balls in the urn.
Therefore, the conditional probability is P(B|A) = 5/19 = 0.2632.

The probability P(A ∩ B) of getting a white ball and then a black one in two consecutive
draws, without putting the white one already drawn back into the urn is
P(A ∩ B) = P(A) * P(B|A) = 0.1974
Conditional probability

What if the drawn ball was put back into the urn?

5
The probability of drawing a black ball on the second drawn would be .
20
Then the probability of getting first a white ball and then a black one in two
consecutive draws would be:
15 15
P (A ∩ B) = P(A) * P(B|A) = P(A) * P(B) = = 0.1875
20 20
Conditional probability

Students who have followed the course and students who have not attended it
go to have a university exam.
The teacher estimates that 65% of the students have taken the course.
The probability that a student will pass the exam given that he has taken the
course is 0.75, while the probability that a student will pass the exam since he
has not taken the course is 0.40.

What is the probability that a student will pass the exam?

What is the probability that a student has taken the course knowing that they
has passed the exam?
Probabilità condizionata
The teacher estimates that 65% of the students have taken the course.
The probability that a student will pass the exam given that he has taken the course is
0.75, while the probability that a student will pass the exam since he has not taken the
course is 0.40.
What is the probability that a student will pass the exam?

We indicate with A and B the events: A = "the student passes the exam"; B="the
student has taken the course".
P(B) = 0.65 and consequently: P(Bത ) = 1 − P(B) = 1 − 0.65 = 0.35
P(A|B) = 0.75 and P(A| Bത ) = 0.40. Event A can be represented as the union of two
incompatible events: A = (A ∩ B) ∪ (A ∩ Bത ) therefore P(A) = P(A ∩ B) + P(A ∩ Bത )
P(A ∩ B) = P(A|B) ∗ P(B) = 0.75 ∗ 0.65 = 0.4875
P(A ∩ Bത ) = P(A| Bത ) ∗ P(Bത ) = 0.40 ∗ 0.35 = 0.1400.
Therefore P(A) = 0.4875 + 0.1400 = 0.6275
Probabilità condizionata

The teacher estimates that 65% of the students have taken the course.
The probability that a student will pass the exam given that he has taken the course is
0.75, while the probability that a student will pass the exam since he has not taken the
course is 0.40.
What is the probability that a student has taken the course knowing that they has
passed the exam?

We indicate with A and B the events: A = "the student passes the exam"; B="the
student has taken the course".
From the first point we have: P(A ∩ B) = P(A|B) ∗ P(B) = 0.75 ∗ 0.65 = 0.4875 and
P(A)= 0.6275.
The probability required is P(B|A) = P(A ∩ B) /P(A) = 0.4875 / 0.6275 = 0.7769
Indipendence
Conditional probability

It may happen that the conditional probability P(B|A) is equal to the simple
probability P(B); this condition intuitively means that knowing that A has occurred
does not change the assessment of the probability of B.
Two events A and B are said to be independent if:
P(B|A) = P(B)
in this case we also have
P(A|B) = P(A)

In the case of two independent events, the Multiplication Rule, seen above,
becomes:
P(A ∩ B) = P(A|B) * P(B)= P(B|A) * P(A)= P(A) * P(B)
This rule can be used to determine whether two events are independent.
Conditional probability

Example
What is the probability of getting tail twice in two successive flips of a
coin?

Since the probability of getting tails is P(T) = 1/2 for each toss and the two
tosses are independent, the probability of getting tail twice is:
P(T;T) = 1/2 * 1/2 = 0.25
Conditional probability
Example
A dice is rolled twice. Calculate the probability of getting 4, 5 or 6 on the first roll
and 1, 2, 3 or 4 on the second.
Example
The probability of two independent events occurring is 0.8 and 0.9 respectively
Find the probability that:
1 both are realized;
2 neither of them is realized;
3 at least one of the two is realized.
Example
Two cards are drawn from a deck of 52 cards. Calculate the probability of
drawing two queens if:
1 The first card is returned to the deck before the second card
extraction;
2 The top card is not returned to the deck before the
second draw.
Conditional probability

Example
A dice is thrown twice. Calculate the probability of getting 4, 5
or 6 on the first roll and 1, 2, 3 or 4 on the second.

Let A = {4; 5; 6} and B = {1; 2; 3; 4}.

Calculate the probability P(A ∩ B)

The result of the second toss is independent of the first, i.e. the
two events A and B are independent, so
34
P(A ∩ B) = P(A) * P(B) = = 0.3333
66
Conditional probability

Example
The probabilities of two independent events occurring is 0.8 and 0.9, respectively.
Find the probability that:
1) both are realized;
2) neither of the two is realized;
3) at least one of the two is realized.

Event A: “A occurs“. P(A) = 0.8

Event B: “B occurs “. P(B) = 0.9
Since A and B are independent events:1
1) P(both are realized) = P(A ∩ B) = P(A) * P(B) = 0.8 * 0.9 = 0.72
2) P(neither of the two is realized) = P(𝐴ҧ ∩ 𝐵ത ) = P(𝐴ҧ) * P(𝐵ത ) = 0.2 * 0.1 = 0.02
3) P(at least one of the two is realized) = 1 - P(neither of them is realized) = 1 – 0.02 = 0.98
Conditional probability

Example
Two cards are drawn from a deck of 52 cards. Calculate the probability of
drawing two queens if:

1) the first card is returned to the deck before the second card extraction.
In this case the events are independent there are 4 queens in the deck, so:
P = (4/52)*(4/52) = 0.0059

2) The first card is not returned to the deck before the second card
extraction.
In this case, the events are not independent. Among the 51 cards
left after the extraction of the first queen there are only 3
queens left, so the probability of drawing one of these is 3/51.
The requested probability is P = 4/52 * 3/51 = 0.0045
Conditional probability

Example
In which of the following cases events A and B
are independent?

1) P(A) = 0.2, P(B) = 0.6, P(A⋃B) = 0.68

2) P(A) = 0.1, P(B) = 0.99, P(A⋃B) = 0.80
3) P(A) = 0.6, P(B) = 0.4, P(A⋃B) = 0.76
4) P(A) = 0.8, P(B) = 0.8, P(A⋃B) = 0.96
Conditional probability

Example
In which of the following cases events A and B
are independent?
1) P(A) = 0.2, P(B) = 0.6, P(A⋃B) = 0.68
2) P(A) = 0.1, P(B) = 0.99, P(A⋃B) = 0.80
3) P(A) = 0.6, P(B) = 0.4, P(A⋃B) = 0.76
4) P(A) = 0.8, P(B) = 0.8, P(A⋃B) = 0.96

Remembering that P(A⋃B) = P(A) + P(B) - P(A ∩ B)

then P(A ∩ B) = P(A) + P(B) - P(A⋃B).
If independent: P(A ∩ B) = P(A) * P(B)
1 P(A ∩ B) = 0.12; P(A) * P(B) = 0.12: sì
2 P(A ∩ B) = 0.29; P(A) * P(B) = 0.10: no
3 P(A ∩ B) = 0.24; P(A) * P(B) = 0.24: sì
4 P(A ∩ B) = 0.64; P(A) * P(B) = 0.64: sì
Bayes’ theorem

20
Law of total probability

Events B1 and B2 are disjoint (B1 ∩ B2 = ∅) and exhaustive (B1 ∪ B2 = Ω).

The sets A ∩ B1 e A ∩ B2 are disjoint, then P(A) = P(A ∩ B1) + P(A ∩ B2)
Applying the multiplication rule we obtain:
P(A) = P(B1) P(A| B1) + P(B2) P(A| B2)
This formula expresses the rule of total probability considering 2 events,
can be generalized to n events B1, B2, . . . ,Bn disjointed and exhaustive
Law of total probability

Let A be an event and {B1, B2, . . . ,Bn} a family of events in the sample space Ω
incompatible, exhaustive with P(Bi)≠0 for every i.

Then it can be shown that:

P(A) = P(A|B1) P(B1)+P(A| B2) P(B2)+ … +P(A| Bn) P(Bn) =
= σ𝑛𝑖=1 P(A|Bi ) P(Bi ) (compact notation of the sum)
Law of total probability

Example
Considering two urns containing respectively:
U1: 2 red balls and 1 black ball
U2: 3 red balls and 2 black balls

We randomly choose an urn and randomly draw a ball from the urn
chosen. What is the probability of drawing a black ball?
Law of total probability

Example
Considering two urns containing respectively:
U1: 2 red balls and 1 black ball
U2: 3 red and 2 black balls
We randomly choose an urn and randomly draw a ball from the urn
choice. What is the probability of drawing a black ball?

Event B1: “U1 has been chosen” Event B2: “U2 has been chosen”
B1 ∩ B2 = ∅; e B1 ∪ B2 = Ω
Event A: “a black ball has been drawn”

P(B1) = 1/2 P(B2) = 1/2 P(A| B1) = 1/3 P(A|B2) = 2/5

Law of total probability
Example
Considering two urns containing respectively:
U1: 2 red balls and 1 black ball
U2: 3 red and 2 black balls
We randomly choose an urn and randomly draw a ball from the urn
choice. What is the probability of drawing a black ball?

Event B1: “U1 has been chosen” Event B2: “U2 has been chosen”
B1 ∩ B2 = ∅; e B1 ∪ B2 = Ω
Event A: “a black ball has been drawn”
P(B1) = 1/2 P(B2) = 1/2 P(A| B1) = 1/3 P(A|B2) = 2/5
Applying the total probability theorem:
P(A)=P(B1) P(A| B1) + P(B2) P(A| B2)
P(A) = P(A|B1) P(B1)+P(A| B2) P(B2) = 1/3 1/2+2/5 1/2 = 0.367
Bayes’ theorem

Let A be an event with P(A) > 0 and let {B1, B2, . . . ,Bn} be events of the sample
space Ω that satisfy the hypotheses of the total probability theorem.
Then for every k

P(A|𝐵𝑘) P(𝐵𝑘) P(A|𝐵𝑘) P(𝐵𝑘)

P(𝐵𝑘 |A) = σ𝑛 = =
𝑖=1 P(A|𝐵𝑖 ) P(𝐵𝑖 ) P(A|𝐵1 ) P(𝐵1 )+P(A|𝐵2 ) P(𝐵2 )+⋯.+P(A|𝐵𝑛 ) P(𝐵𝑛 )
P(A|𝐵𝑘) P(𝐵𝑘)
=
𝑃(𝐴)

Example
Returning to the previous example, we can now answer to the following
question: if a black ball has been drawn, what is the probability of having
chosen the first urn?
Bayes’ theorem

Example
Returning to the previous example, we can now answer to the following
question: if a black ball has been drawn, what is the probability of having
chosen the first urn?

Event B1: “U1 has been chosen” Event B2: “U2 has been chosen”
B1 ∩ B2 = ∅; e B1 ∪ B2 = Ω
Event A: “a black ball has been drawn”

To answer this question, we need to calculate the probability P(B1|A)

P(A|B1) P(B1) 1/3 1/2

P(B1|A) = = = 0.455
P(A) 11/30
Bayes’ theorem

Example
An industry has installed a system for quality control, which guarantees that, if a
part is defective, it is disposed of with probability 0.999.
There is a probability of 0.002 that even a non-defective part will be discarded.
It also knows that the probability of a part being defective is 0.05.
Calculate the probability that a piece that has not been eliminated at the quality
control is defective.
Bayes’ theorem

Example
An industry has installed a system for quality control, which
guarantees that, if a part is defective, it is disposed of with probability 0.999.
There is a probability of 0.002 that even a non-defective part will be discarded.
It also knows that the probability of a part being defective is 0.05.
Calculate the probability that a part that has not been eliminated at the quality control is
defective.

Event E: “The part is discarded”.

Event D: “the part is defective”

We know that: P(E|D) = 0.999 ഥ) = 0.002

P(E| D P(D) = 0.05

ഥ)
We need to evaluate P(D| E
Bayes’ theorem

Example
Event E: “The part is discarded”.
Event D: “the part is defective”

We know that: P(E|D) = 0.999 ഥ) = 0.002

P(E| D P(D) = 0.05

ഥ).
We need to evaluate P(D| E
P( ഥ
E|D)P(D)
Applying the Bayes: theorem: P(D| E ഥ) = ഥ ഥ D)P(
ഥ ഥ)
P(E|D)P(D)+P(E| D
We have that:
ഥ|D) = 1 - P(E|D) = 1 – 0.999 = 0.001
P(E P(Eഥ |Dഥ ) = 1 - P(E|D
ഥ ) = 1 – 0.002 = 0.998
ഥ ) = 1 - P(D) = 1 – 0.05 = 0.95
P(D
0.001 ∗ 0.05
Using the theorem we obtain: P(D| E ഥ) = = 0.0000527
0.001∗ 0.05+0.998 ∗0.95
Bayes’ theorem

Example
We know that the percentage of people who have red hair in Piedmont, Sardinia
and Marche is 5%, 1% and 2%, respectively, and that the three regions have 4.5,
2 e 1.5 million inhabitants, respectively.
Chosen a person at random from among the inhabitants of the three regions,
calculate the probability that the person's region of origin be the Sardinia,
supposing that:
1 they has red hair
2 they does not have red hair
Teorema di Bayes
Esempio We know that the percentage of people who have red hair in Piedmont,
Sardinia and Marche is 5%, 1% and 2%, respectively, and that the three regions
have 4.5, 2 e 1.5 million inhabitants, respectively.
Chosen a person at random from among the inhabitants of the three regions,
calculate the probability that the person's region of origin be the Sardinia,
supposing that:
1 they has red hair

R: red hair; P: Piedmont region of origin; S: region of origin Sardinia;

M: region of origin Marche.
Therefore: P(R|P) = 0.05, P(R|S) = 0.01, P(R|M) = 0.02, P(P) = 4.5/8 = 9/16,
P(S) = 2/8 = 4/16, P(M) = 1.5/8 = 3/16
Using the theorem we obtain :P(S|R) =
P(R|S)P(S) 0.01 4/16
= = 0.0727
P(R|S)P(S)+P(R|P)P(P)+P(R|M)P(M) 0.01 4/16+0.05 9/16+0.02 3/16
Bayes’ theorem
Example We know that the percentage of people who have red hair in Piedmont,
Sardinia and Marche is 5%, 1% and 2%, respectively, and that the three regions have
4.5, 2 e 1.5 million inhabitants, respectively.
Chosen a person at random from among the inhabitants of the three regions, calculate
the probability that the person's region of origin be the Sardinia, supposing that:
2 they does not have red hair
R: red hair; P: Piedmont region of origin; S: region of origin Sardinia;
M: region of origin Marche.
Therefore : P(R|P) = 0.05, P(R|S) = 0.01, P(R|M) = 0.02, P(P) = 4.5/8 = 9/16,
P(S) = 2/8 = 4/16, P(M) = 1.5/8 = 3/16
P(Rത |P) = 0.95, P(Rത | S) = 0.99 e P(Rത | M) = 0.98
Using the theorem we obtain:
P( ഥ
R|S)P(S) 0.99 4/16
ത
P(S| R) = ഥ ഥ ഥ = = 0.2563
P(R|S)P(S)+P(R|P)P(P)+P( R|M)P(M) 0.99 4/16+0.95 9/16+0.98 3/16