STAT 112

INTRODUCTION TO STATISTICS AND

PROBABILITY II

SESSION 1B- Basic Concepts in Probability 2

Department of Statistics and Actuarial Science

University of Ghana

STAT 112 1 / 33
Session Outline

1 Conditional Probability and Bayes’ Rule

Conditional Probability
Independent Events
Total Probability Rule
Bayes’ Theorem
Applications of Bayes’ Theorem and TPR

STAT 112 2 / 33
Session Overview

Probability plays a major role in everyday life and decision making

depends on one or more events of uncertainty.
In this session, students will be introduced to conditional events and
probabilities, independent events and probability, total probability rule and
Bayes Theorem.

STAT 112 3 / 33
Learning Outcomes

At the end of the session, the student will:

1 Understand dependent and independent events.
2 Calculate conditional probabilities of random events.
3 Apply total probability rule and Bayes theorem for solving real life
problem.

STAT 112 4 / 33
Conditional Probability

Suppose S is the sample space for an experiment with events A and

B.
Then for experiments with equally likely outcomes, the conditional
probability of A given B is the fraction of outcomes in B that are also
in A .
Since only outcomes in B are taken into consideration, in effect, B
becomes the sample space, so we have:

n(event A occurs given that event B occurs)

P (A |B ) =
n(event B occurs)
P (A ∩ B )
P (A |B ) = , where P (B ) > 0
P (B )

STAT 112 5 / 33
Conditional Probability Cont’d.

Similarly, we can define P (B |A ) as:

P (A ∩ B )
P (B |A ) = , where P (A ) > 0.
P (A )

Also a very useful formula in computing the joint probability of events

is
P (A ∩ B ) = P (A |B )P (B ) = P (B |A )P (A )
In general, for events Ai (i = 1, 2, 3, . . . )

P (A1 ∩ A2 ∩ A3 ∩ · · · ∩ An )

= P (A1 )P (A2 |A1 )P (A3 |A1 , A2 )P (A4 |A1 , A2 , A3 ) . . . P (An |A1 , A2 , . . . , An−1 )
The above formula is known as Multiplication Rule.

STAT 112 6 / 33
Example 1

A survey of attitudes about one’s job yields the data in the following table

Jobs Happy Unhappy Total

Bus Drivers 50 75 125
Lawyers 40 35 75
Total 90 110 200

A person from this group is selected at random. Given that the selected
person is a bus driver, find the probability that he or she is happy.

STAT 112 7 / 33
Example Cont’d: Solution

Let H denote the event that a person is happy and B, the event that a bus
driver is selected. The required probability is P (H |B )

Now, using the conditional probability formula we have:

P (H ∩ B )
P (H |B ) =
P (B )
50 125
From the table, P (H ∩ B ) = and P (B ) =
200 200
Therefore,
50/200 50
P (H |B ) = = = 0.4
125/200 125

STAT 112 8 / 33
Example 2

There are 8 boxes, 3 of type A and 5 of type B. Type A boxes each

contain 3 green marble and 4 red marbles. Type B boxes each contain 2
green marble and 4 red marbles. A box is selected at random and a
marble is drawn at random from the selected box.
i Find the probability that a red marble is selected.
ii A marble was selected and was found to be red, what is the
probability that it came from a type A box.
Solution:
Let R denote the event that a red marble is selected, let A represent the
event that a type A box is selected, and let B denote the event that a type
B box is selected.

STAT 112 9 / 33
Solution

i The required probability is P (R ).

Note: a red marble may either come from box A or box B but not
both, hence R = (R ∩ A ) ∪ (R ∩ B ), since (R ∩ A ) and (R ∩ B ) are
mutually exclusive events,

=⇒ P (R ) = P (R ∩ A ) + P (R ∩ B )
Using the multiplication formula, we have:

P (R ∩ A ) = P (R |A )P (A )
Given that 3 of the 8 boxes are of type A, P (A ) = 38 .
Next, type A box contains 7 marbles, of which 4 are red.

STAT 112 10 / 33
Solution Cont’d.

i Continued:
Therefore, the conditional probability is; P (R |A ) = 47 .
By the multiplicative rule,

4 3 3
P (R ∩ A ) = P (R |A )P (A ) = × = .
7 8 14
5 4 2
However, P (B ) = 8 and P (R |B ) = 6 = 3 and multiplicative rule yield
the result;
2 5 5
P (R ∩ B ) = P (R |B )P (B ) = × = .
3 8 12
3 5
∴ P (R ) = P (R ∩ A ) + P (R ∩ B ) = + = 0.63
14 12

STAT 112 11 / 33
Solution

ii The required probability is P (A |R ).

Using the conditional probability formula, we have:

P (A ∩ R ) 0.21
P (A |R ) = = = 0.33
P (R ) 0.63
From (i) above.

STAT 112 12 / 33
Independents Events

Two events, A and B that are related in such a way that the
occurrence of either one has no bearing on the occurrence of the
other are said to be independent.

For instance, if a coin is tossed repeatedly, the result of one toss does
not affect the result of any other toss.

Alternatively; Two events A and B are said to be independent if

knowledge of the occurrence of A, in no ways influences the
probability of the occurrence of B and vice versa.

Statistically, events A and B are said to be independent, if and only if

P (A ∩ B ) = P (A ) × P (B ).

STAT 112 13 / 33
Independent Events Cont’d.

It follows from the definition of conditional probability that if A and B

are independent, then

P (A |B ) = P (A ) and P (B |A ) = P (B ).

We say that three events A, B, and C are jointly or mutually

independent if and only if the following conditions hold:
1 P (A ∩ B ) = P (A ) × P (B )
2 P (A ∩ C ) = P (A ) × P (C )
3 P (B ∩ C ) = P (B ) × P (C )
4 P (A ∩ B ∩ C ) = P (A ) × P (B ) × P (C )

STAT 112 14 / 33
Example 3

A ball is drawn at random from an open box containing four balls

numbered 1, 2, 3, 4. Let A = {1, 2}, B = {1, 3}, C = {1, 4}.
Examine whether or not A , B , C are independent.

Solution:
For events A, B, C to be independent then, the conditions stated above
must hold
A ∩ B = {1} =⇒ P (A ∩ B ) = 14
A ∩ C = {1} =⇒ P (A ∩ C ) = 41
B ∩ C = {1} =⇒ P (B ∩ C ) = 14
2 2 1
1 P (A ∩ B ) = P (A ) × P (B ) = 4 × 4 = 4
2 2 1
2 P (A ∩ C ) = P (A ) × P (C ) = 4 × 4 = 4
2 2 1
3 P (B ∩ C ) = P (B ) × P (C ) = 4 × 4 = 4

STAT 112 15 / 33
Solution Cont’d.

1
Also; A ∩ B ∩ C = {1} =⇒ P (A ∩ B ∩ C ) = 4

2 2 2 1
But P (A ) × P (B ) × P (C ) = 4 · 4 · 4 = 8 , P (A ∩ B ∩ C ).

Hence A, B, C are pairwise independent but not jointly or mutually

independent.

STAT 112 16 / 33
Theorem 3:

If A and B are independent events, then

i. A ′ and B ′ are independent.
ii. A ′ and B are independent.
iii. A and B ′ are independent.
The proofs are left as an Exercise.

STAT 112 17 / 33
Total Probability Rule

Suppose that the sample space, S of a random experiment, can be

partitioned into a set of n mutually exclusive and exhaustive events,
say A1 , A2 , . . . , An (Figure 1).
Let B be another event in the sample space with a strictly positive
probability of taking place.

Figure: 1

STAT 112 18 / 33
Total Probability Rule Cont’d

Then
n
X
P (B ) = P (B |Ai )P (Ai ), where P (Ai ) > 0.
i =1

Sketch of the Proof:

From Figure 1 above, we can write

B = (B ∩ A1 ) ∪ (B ∩ A2 ) ∪ (B ∩ A3 ) ∪ · · · ∪ (B ∩ An )
P (B ) = P (B ∩ A1 ) + P (B ∩ A2 ) + P (B ∩ A3 ) + · · · + (B ∩ An )
n
X
⇒ P (B ) = P (B ∩ Ai )
i =1

STAT 112 19 / 33
Total Probability Rule (Cont..)

Since, (B ∩ Ai ), where i = 1, 2, . . . , k are mutually exclusive events.

And the;
P (B ∩ Ai )
P ( B |A i ) = , where P (Ai ) > 0
P (Ai )

=⇒ P (B ∩ Ai ) = P (B |Ai )P (Ai ), where i = 1, 2, . . . , n

Therefore
n
X
P (B ) = P (B |Ai )P (Ai ) (1)
i =1

STAT 112 20 / 33
Bayes’ Theorem

Suppose that the events A1 , A2 , . . . , An , is a partition of S, sample

space and B is any event. Then for any i,

P (Ai )P (B |Ai )
P (Ai |B ) =
P (A1 )P (B |A1 ) + P (A2 )P (B |A2 ) + · · · + P (An )P (B |An )

Sketch Proof:
Now,
P (Ai ∩ B )
P (Ai |B ) = , where P (B ) > 0
P (B )
P (B ∩ Ai )
Also, P (B |Ai ) = , where P (Ai ) > 0
P (Ai )

⇒ P (B ∩ Ai ) = P (B |Ai )P (Ai ).

STAT 112 21 / 33
Bayes’ Theorem Cont’d.

Hence,
P (B |Ai )P (Ai )
P (Ai |B ) = Pn (2)
i =1 P (B |Ai )P (Ai )

Equations (1) and (2) are known as Total Probability Rule and Bayes’
Theorem, respectively.

STAT 112 22 / 33
Example 1:

A company producing electric relays has three manufacturing plants

producing 50, 30 and 20 percent, respectively of its products. Suppose
that the probabilities that a relay manufactured by these plants is defective
are 0.02, 0.05 and 0.01, respectively.

a. If a relay is selected at random from the output of the company, what

is the probability that it is defective?

b. If a relay selected at random is found to be defective, what is the

probability that it was manufactured by plant 2?

STAT 112 23 / 33
Solution

(a) Let B be the event that the relay is defective, and let Ai be the
event that the relay is manufactured by plant i , where(i = 1, 2, 3). The
desired probability is P (B ).

Therefore, we have
P3
P (B ) = i =1 P (B |Ai )P (Ai )

P (B ) = 0.02(0.5) + (0.05)(0.3) + (0.01)(0.2) = 0.027

(b) The desired probability is

P (B |A2 )P (A2 ) (0.05)(0.3)

P (A2 |B ) = = = 0.556
P (B ) 0.027

STAT 112 24 / 33
Example 2:

It is estimated that 50% of emails are spam emails. Some software has
been applied to filter these spam emails before they reach your inbox.
A certain brand of software claims that it can detect 99% of spam emails,
and the probability for a false positive (a non-spam email detected as
spam) is 5%.
Now if an email is detected as spam, then what is the probability that it is
in fact a non-spam email?

STAT 112 25 / 33
Example 2:

It is estimated that 50% of emails are spam emails. Some software has
been applied to filter these spam emails before they reach your inbox.
A certain brand of software claims that it can detect 99% of spam emails,
and the probability for a false positive (a non-spam email detected as
spam) is 5%.
Now if an email is detected as spam, then what is the probability that it is
in fact a non-spam email?

Solution:
Let define the following events:
A = event that an email is detected as spam,
B =event that an email is spam,
B̄ = event that an email is not spam.

Given that, P (B ) = P (B̄ ) = 0.5, P (A |B ) = 0.99, P (A |B̄ ) = 0.05

STAT 112 25 / 33
Solution Cont’d.

Using Bayes’ theorem or formula, we have:

P (A |B̄ )P (B̄ )
P (B̄ |A ) =
P (A |B )P (B ) + P (A |B̄ )P (B̄ )
0.05 × 0.5
= = 0.048
0.99 × 0.5 + 0.05 × 0.5

STAT 112 26 / 33
Example 3:
Suppose we have 3 cards identical in form except that both sides of the
first card are coloured red, both sides of the second card are coloured
black, and one side of the third card is coloured red and the other side is
coloured black.The 3 cards are mixed up in a hat, and 1 card is randomly
selected and put down on the ground.
If the upper side of the chosen card is coloured red, what is the probability
that the other side is coloured black?

STAT 112 27 / 33
Example 3:
Suppose we have 3 cards identical in form except that both sides of the
first card are coloured red, both sides of the second card are coloured
black, and one side of the third card is coloured red and the other side is
coloured black.The 3 cards are mixed up in a hat, and 1 card is randomly
selected and put down on the ground.
If the upper side of the chosen card is coloured red, what is the probability
that the other side is coloured black?

Solution:
Let RR, BB, and RB denote, respectively, the events that the chosen card
is the red-red, the black-black, or the red-black card. Letting R be the
event that the upturned side of the chosen card is red, we have that the
desired probability is obtained by

P (RB ∩ R )
P (RB |R ) =
P (R )

STAT 112 27 / 33
Solution Cont’d.

P (R |RB )P (RB )
P (RB |R ) =
P (R |RB )P (RB ) + P (R |RR )P (RR ) + P (R |BB )P (BB )

1 1
2 · 3 1
= 1 1 1 1
= .
2 · 3 +1· 3 +0· 3
3

Note that there were 3 black sides and 3 red sides, yielding the
P (RB ) = 13 = P (RR ) = P (BB )

STAT 112 28 / 33
Example 4:

In certain juice production company there are three production machines

involved in the production. Suppose that machine M1 produces 30% of the
day’s production of the product, while machines M2 and M2 produce 25%
and 45%, respectively. Suppose further that, machines M1 , M2 and M3
turn out 2%, 3% and 4% defective products, respectively, each day. If an
item is picked at random and found to be defective, what is the probability
that it did not come from machine M1 ?

STAT 112 29 / 33
Example 4:

In certain juice production company there are three production machines

involved in the production. Suppose that machine M1 produces 30% of the
day’s production of the product, while machines M2 and M2 produce 25%
and 45%, respectively. Suppose further that, machines M1 , M2 and M3
turn out 2%, 3% and 4% defective products, respectively, each day. If an
item is picked at random and found to be defective, what is the probability
that it did not come from machine M1 ?

Solution:
Let M1 , M2 , M3 denote the sets of products produced by three machines of
the company, respectively. Let D denote the event that an item is
′
defective. The required probability is P [(M1 |D ) ].

Now, we have P (M1 ) = 0.30, P (M2 ) = 0.25, P (M3 ) = 0.45,

P (D |M1 ) = 0.02, P (D |M2 ) = 0.03 and P (D |M3 ) = 0.04.

STAT 112 29 / 33
Solution Cont’d.

By Bayes’ formula, we have

P (D |M1 )P (M1 )
P (M1 |D ) =
P (D |M1 )P (M1 ) + P (D |M2 )P (M2 ) + P (D |M3 )P (M3 )

(0.02)(0.30)
=
(0.02)(0.30) + (0.03)(0.25) + (0.04)(0.45)
= 0.19

′
∴ P [(M1 |D ) ] = 1 − P (M1 |D ) = 0.81

STAT 112 30 / 33
Trial Questions:

1 In a study, physicians were asked what the odds of breast cancer

would be in a woman who was initially thought to have a 1% risk of
cancer but who ended up with a positive mammogram result (a
mammogram accurately classifies about 80% of cancerous tumors
and 90% of benign tumors). Ninety-five (95) out of a hundred
physicians estimated the probability of cancer to be about 75%. Do
you agree?

STAT 112 31 / 33
Trial Questions:

2 Three manufactures M1 , M2 and M3 , supply all of the electronic

components to a manufacturer of VCR’s. Each VCR uses one unit of
the components. Suppose M1 supplies 41 units, 15% of which are
defective; that M2 supplies 63 units, 10% of which are defective; and
M3 supplies 54 units, 12% of which are defective. A VCR is selected
at random and is found to be defective. Which manufacturer is most
likely to have provided the components for the defective VCR?
Explain your answer.

STAT 112 32 / 33
End of Session

THANK YOU!!!

STAT 112 33 / 33