Edwin Moyo (emoyo@mu.ac.

zm)

February 18, 2022

Edwin Moyo (emoyo@mu.ac.zm) February 18, 2022 1 / 29

Lecture 3-Probability and Probability Distributions

This lecture develops work on probability and probability distributions . At

the end of this session, students will be able to;
Outcomes
1 Basic probability concepts
2 Conditional probability
3 Bayes’ theorem to revise probabilities
4 Various counting rules
5 The properties of a probability distribution
6 to compute the expected value and variance of a probability
distribution
7 To compute probabilities from the binomial, Poisson, and
hypergeometric distributions
8 To compute probabilities from the normal, exponential distributions.

Edwin Moyo (emoyo@mu.ac.zm) February 18, 2022 2 / 29

Definitions and Axioms of Probability
Definitions and Notation

Probability: The mathematics of chance or the study of random or

non-deterministic experiments and their outcomes.
Random experiment: An experiment whose outcome depends on
chance (e.g., the tossing of a coin, the selection of balls from a bag,
the throwing of a die, a football match).
Trial: A single performance of an experiment (e.g., one toss of a coin,
one throw of a die, the selection of one ball from a bag, one football
match).
Sample space (Ω): The non-empty set of all possible outcomes of a
random experiment. For a coin toss, Ω = {H, T }, for a toss of two
coins, Ω = {HH, HT , TH, TT }, for a single football match,
Ω = {W , L, D } (where W = win, L = loss and D = draw) and for a
roll of one die, Ω = {1, 2, 3, 4, 5, 6}.

Edwin Moyo (emoyo@mu.ac.zm) February 18, 2022 3 / 29

Definitions and Axioms of Probability
Definitions and Notation

Sample point: Each possible outcome ω ∈ A, where A ⊂ Ω.

Event: This is a subset A of Ω having one or more sample points as
its elements.
σ-algebra: A non-empty collection F of subsets of Ω is called a
σ-algebra (or σ-field) if it satisfies the following properties:
1 ∅, Ω ∈ F .
2 If A ∈ F , then A0 ∈ F , i.e., F must contain both A and its

complement. We can also say that a σ-algebra is closed under

complementation.
∞
[ ∞
\
3 If A1 , A2 , . . . ∈ F , then Ak ∈ F and Ak ∈ F . In other words,
k =1 k =1
a σ-algebra F is closed under countable unions and intersections.

Edwin Moyo (emoyo@mu.ac.zm) February 18, 2022 4 / 29

Definitions and Axioms of Probability
Definitions and Notation

Probability measure: Let F be a σ-algebra of subsets of Ω. Then the

probability measure is a function P that assigns real numbers to the
events defined in a sample space Ω. It can also be defined as a set
function P → [0, 1] defined on F and satisfying the following
properties:
1 P( ∅ ) = 0 and P( Ω ) = 1.
2 0 ≤ P(A) ≤ 1 for any event A ∈ Ω ⊂ F

3 If A , A , . . . is a sequence of disjoint sets in Ω ⊂ F , then

1 2
∞ ∞
P( ∑ P(Ak ).
[
Ak ) = Narrowing this down, if two sets A and B
k =1 k =1
are disjoint, then the probability of their union is the sum of their
respective probabilities, i.e., P(A ∪ B ) = P(A) + P(B ). It follows that
if A, B ∈ Ω, then A ⊂ B implies that P(A) ≤ P(B ).
Marginal probability: This is the probability of a single event A
occurring, expressed simply as P(A). A single event is an event that
describes the outcomes of only one random variable.
Edwin Moyo (emoyo@mu.ac.zm) February 18, 2022 5 / 29
Definitions and Axioms of Probability
Definitions and Notation

Joint probability: This is the probability of two or more events

occurring simultaneously, e.g., if two dice are rolled, the outcomes
from the two dice occur with joint probability.
Mutually exclusive events: Two events are mutually exclusive if they
cannot occur simultaneously (e.g., the outcomes of a football match
are mutually exclusive, since one team cannot win, lose or draw all at
the same time). In mathematical terms, two events A and B are said
to be mutually exclusive iff

P(A or B ) = P(A ∪ B ) = P(A) + P(B )

or iff A and B are disjoint (i.e., if P(A ∩ B ) = 0). This is the

addition rule for mutually exclusive events.

Edwin Moyo (emoyo@mu.ac.zm) February 18, 2022 6 / 29

Definitions and Axioms of Probability
Definitions and Notation

Independent events: Two events A and B are said to be independent

if the occurrence of A has no bearing on the occurrence of B, i.e., iff

P(A and B ) = P(A ∩ B ) = P(A)P(B )

This is the multiplication rule for independent events.

Equally likely events: Two events A and B are equally likely if they
have the same probability of occurrence. For example, in a coin toss
the probability of getting heads (H ) is the same as that of obtaining
tails (T ).0 In this case, P(H ) = P(T ) = 0.5.
Theoretical and empirical probability: Probability that is computed
purely from the facts of a given situation (without conducting an
experiment) is called theoretical probability. However, probability
computed from the outcomes of an actual experiment is called
empirical probability.
Edwin Moyo (emoyo@mu.ac.zm) February 18, 2022 7 / 29
Definitions and Axioms of Probability
Connection between Set Theory and Probability Theory

Set theory Probability theory

Universal set Ω Sample space Ω (certain event)
Empty set Impossible event
Members of Ω are sets A, B, ... Members of Ω are events A, B, ...
Elements a, b, . . . Sample points (or simple events) a, b, . . .
A Event A occurs
A0 (or A or Ac ) Event A does not occur
A∪B At least one of the events A and B occurs
(i.e., A occurs or B occurs or both occur)
A ∩ B (or AB) Both events A and B occur
A⊂B A is a sub-event of B (i.e., occurrence of A
necessarily implies occurrence of B)
A ∩ B = ∅ (disjoint sets A and B) A and B are mutually exclusive events
(i.e., they cannot occur simultaneously)

Edwin Moyo (emoyo@mu.ac.zm) February 18, 2022 8 / 29

Definitions and Axioms of Probability
Axioms

1 Non-negativity: 0 ≤ p (A) ≤ 1 for every event A ∈ Ω. This means

that the probability of any event A is a number between 0 and 1. The
numbers 0 and 1 are called probability limits.
2 Additivity: If A and B are any two disjoint (or mutually exclusive)
events, then the probability of their union satisfies
P(A ∪ B ) = P(A) + P(B )
3 Normalization: P(Ω) = 1, that is, the probability of the entire
sample space Ω is equal to 1. This also means that if A, B and C are
the exhaustive events of an experiment, then
P(A) + P(B ) + P(C ) = 1. Exhaustive events refer to the only
outcomes of an experiment (e.g., in a coin toss, H and T are the only
events and they are exhaustive; the only outcomes of a football match
are win, loss or draw, so they are exhaustive outcomes).
4 Complementarity: P(A) = 1 − P(A) for every event A ∈ Ω.
Edwin Moyo (emoyo@mu.ac.zm) February 18, 2022 9 / 29
Definitions and Axioms of Probability
Elementary Properties

1 P(∅) = 0, i.e., the probability of an impossible event is zero. An

impossible event is an event that can never occur. An example is a
football team winning and losing a match at the same time. This is
impossible. Another example is obtaining a 7 when you roll one die.
This is an impossible event since a die only has numbers 1 − 6.
2 If A ⊂ B, then P(A) ≤ P(B ).
3 P(B ∩ A) = P(B ) − P(B ∩ A).
4 Given any non-mutually exclusive events A and B,
P(A or B ) = P(A ∪ B ) = P(A) + P(B ) − P(A ∩ B ).

Edwin Moyo (emoyo@mu.ac.zm) February 18, 2022 10 / 29

Definitions and Axioms of Probability
Examples

Example: A government agency employs 100 clerk-typists, classified by

sex and marital status as shown in the table below:

If an employee is picked at random from the 100 employees, find the

probability that he or she is
(a) Single (X )
(b) Male or single (M or X )
(c) Female and married (F and Y )
(d) Male or married (M or Y )
Edwin Moyo (emoyo@mu.ac.zm) February 18, 2022 11 / 29
Definitions and Axioms of Probability
Examples

Solution
(a) P(X ) = 40
= 0.4
100
(b) P(M or X ) = P(M ∪ X ) = P(M ) + P(X ) − P(M ∩ X ) =
40 40 16 64
100 + 100 − 100 = 100 = 0.64
(c) P(F and Y ) = P(F ∩ Y ) = P(F )P(Y ) = 100 60 60
× 100 36
= 100 = 0.36.
Alternatively, you can see from the table that there are 36 employees
who are both female (event F ) and married (event Y ), so P(F and
36
Y ) = 100 = 0.36
(d) P(M or Y ) = P(M ∪ Y ) = P(M ) + P(Y ) − P(M ∩ Y ) =
40 60 24 76
100 + 100 − 100 = 100 = 0.76

Edwin Moyo (emoyo@mu.ac.zm) February 18, 2022 12 / 29

Definitions and Axioms of Probability
Examples

Example : A business person carries fire insurance on both his home and
store. During a given year the probability of fire at home (event A) is 0.06
and the probability of a store fire (event B) is 0.13. If A and B are
independent, find (a) P(A and B ) (b) P(A or B ) (c) P(only one fire will
occur)
Solution
(a) P(A and B ) = P(A)P(B ) = 0.06 × 0.13 = 0.0078
(b) P(A or B ) = P(A) + P(B ) − P(A and
B) = 0.06 + 0.13 − 0.0078 = 0.1822
(c) Here we need P(A) = 1 − P(A) = 1 − 0.06 = 0.94 and
P(B ) = 1 − P(B ) = 1 − 0.13 = 0.87, so that P(only one fire will
occur) = P(AB or
AB ) = P(AB ) + P(AB ) = 0.06 × 0.87 + 0.94 × 0.13 = 0.1744.

Edwin Moyo (emoyo@mu.ac.zm) February 18, 2022 13 / 29

Definitions and Axioms of Probability
Examples

Example: Let A and B be events such that P(A ∪ B ) = 0.875,

P(A ∩ B ) = 0.25 and P(A) = 0.625. Find P(A), P(B ) and P(A ∩ B ).
Solution Since P(A) = 0.625, P(A) = 1 − P(A) = 1 − 0.625 = 0.375.
Since P(A ∪ B ) = P(A) + P(B ) − P(A ∩ B ), we have
P(B ) = P(A ∪ B ) − P(A) + P(A ∩ B ), i.e.,
P(B ) = 0.875 − 0.375 + 0.25 = 0.75.
Since P(B ) = 0.75, it follows that P(B ) = 0.25. Finally,
P(A ∩ B ) = P(A) − P(A ∩ B ) = 0.375 − 0.25 = 0.125.

Try this: A and B are events such that P(A) = 0.125, P(A ∪ B ) = 0.25
and P(B ) = p. Find p if
(a) A and B are mutually exclusive.
(b) A and B are independent.
(c) A ⊂ B.

Edwin Moyo (emoyo@mu.ac.zm) February 18, 2022 14 / 29

Conditional Probability and Bayes’ Theorem
Conditional Probability

A conditional probability is the probability of one event A occurring given

information about the occurrence of another event B. Thus, the
conditional probability of an event A given that an event B has occurred,
denoted by P(A|B ), is defined as

P(A ∩ B )
P(A|B ) = , provided P(B ) 6= 0 (1)
P(B )

where P(A ∩ B ) is the joint probability of A and B. This means that if B

is an impossible event, then the occurrence of A cannot be conditioned on
that of B. Similarly,

P(A ∩ B )
P(B |A) = , provided P(A) 6= 0 (2)
P(A)

is the conditional probability of an event B occurring given that an event

A has already occurred.
Edwin Moyo (emoyo@mu.ac.zm) February 18, 2022 15 / 29
Conditional Probability and Bayes’ Theorem
Bayes’ Theorem

The essential feature of conditional probability is that the sample space is

reduced to the outcomes describing the given prior event only, not all
possible outcomes as for marginal and joint probabilities. Combining (1)
and (2) gives the generalized multiplication rule:

P(A ∩ B ) = P(A|B )P(B ) = P(B |A)P(A) (3)

From (3), we obtain Bayes’ Theorem which states that

P(B |A)P(A)
P(A|B ) = , for P(B ) 6= 0 (4)
P(B )

If A and B are independent, then P(A|B ) = P(A) and P(B |A) = P(B ),
so that (as before) P(A ∩ B ) = P(A)P(B ), which is the ordinary
multiplication rule for independent events.

Edwin Moyo (emoyo@mu.ac.zm) February 18, 2022 16 / 29

Conditional Probability and Bayes’ Theorem
Bayes’ Theorem

Bayes’ Theorem (Two event case)

P (B | A)P (A)
P (A | B ) =
P (B | A)P (A) + P (B | Ac ) P (Ac )

and

P (A | Bi ) P (Bi )
P (Bi | A) =
P (A | B1 ) P (B1 ) + · · · + P (A | Bk ) P (Bk )
where Bi is the i th event out of k mutually exclusive and collectively
exhaustive events.

Edwin Moyo (emoyo@mu.ac.zm) February 18, 2022 17 / 29

Conditional Probability and Bayes’ Theorem
Example

Example : Let A and B be events such that P(A) = 0.5, P(B ) = 0.333
and P(A ∩ B ) = 0.25. Find (a) P(A|B ) (b) P(B |A) (c) P(A ∪ B )
(d) P(A|B ) (e) P(B |A). Solution
P(A∩B )
1 P(A|B ) = P(B )
= 0.25
0.333 = 0.75
P(A∩B )
2 P(B |A) = P(A)
= 0.25
0.5 = 0.5
3 P(A ∪ B ) = P(A) + P(B ) − P(A ∩ B ) = 0.5 + 0.333 − 0.25 = 0.583
P(A∩B )
4 Since P(A|B ) = P(B )
(by definition of conditional probability), we
first find: P(A) = 1 − P(A) = 1 − 0.5 = 0.5 and
P(B ) = 1 − P(B ) = 1 − 0.333 = 0.667. By De Morgan’s law (from
set theory), (A ∪ B ) = A ∩ B. Taking probabilities on both sides, we
have P(A ∩ B ) = P((A ∪ B )) = 1 − P(A ∪ B ) = 1 − 0.583 = 0.417.
Therefore, P(A|B ) = 0.417
0.667 = 0.625
P(A∩B )
5 P(B |A) = P(A)
= 0.417
0.5 = 0.834
Edwin Moyo (emoyo@mu.ac.zm) February 18, 2022 18 / 29
Example
In the past, 40% of the new-model televisions have been successful, and
60% have been unsuccessful. Before introducing the new model television,
the marketing research department conducts an extensive study and
releases a report, either favorable or unfavorable. In the past, 80% of the
successful new-model television(s) had received favorable market research
reports, and 30% of the unsuccessful new-model television(s) had received
favorable reports. For the new model of television under consideration, the
marketing research department has issued a favorable report. What is the
probability that the television will be successful?

Edwin Moyo (emoyo@mu.ac.zm) February 18, 2022 19 / 29

Example
In the past, 40% of the new-model televisions have been successful, and
60% have been unsuccessful. Before introducing the new model television,
the marketing research department conducts an extensive study and
releases a report, either favorable or unfavorable. In the past, 80% of the
successful new-model television(s) had received favorable market research
reports, and 30% of the unsuccessful new-model television(s) had received
favorable reports. For the new model of television under consideration, the
marketing research department has issued a favorable report. What is the
probability that the television will be successful?

Solution
Let, event S = successful television, event S 0 = unsuccessful television,
event F = favourable report, F 0 = unfavourable report. And,

P(S ) = 0.40, P(F |S ) = 0.8, P(| S 0 ) = 0.6, P (F | | S 0 ) = 0.3

Edwin Moyo (emoyo@mu.ac.zm) February 18, 2022 19 / 29

Solution Cont’
Then,
P (F | S )P (S )
P (S | F ) =
P (F | S )P (S ) + P (F | S 0 ) P (S 0 )
(0.80)(0.40)
=
(0.80)(0.40) + (0.30)(0.60)
0.32 0.32
= =
0.32 + 0.18 0.50
= 0.64
The probability of a successful television, given that a favorable report was
received, is 0.64. Thus, the probability of an unsuccessful television, given
that a favorable report was received, is 1 − 0.64 = 0.36.

Edwin Moyo (emoyo@mu.ac.zm) February 18, 2022 20 / 29

Table below summarizes the computation of the probabilities and Figure
presents the decision tree.

Edwin Moyo (emoyo@mu.ac.zm) February 18, 2022 21 / 29

Task
Chipego and Sons produces engine fuses by five production lines in a
manufacturing operation. The fuses are costly, are quite reliable and
shipped to suppliers in 1000-unit lots. Because testing is destructive, most
buyers of the fuses test only a small number of fuses before deciding to
accept or reject lots of incoming fuses. All five production lines produce
fuses at the same rate and normally produce only 2% defective fuses,
which are randomly dispersed in the output. Unfortunately, production line
1 suffered mechanical difficulty and produced 5% defectives during the
month of May. This situation became known to the manufacturer after
the fuses had been shipped. Miss Elizabeth received a lot produced in May
and tested three fuses. One failed.
(a) What is the probability that the lot was produced on line 1?
(b) What is the probability that the lot came from one of the four other
lines?

Edwin Moyo (emoyo@mu.ac.zm) February 18, 2022 22 / 29

Counting Techniques and Applications
Multiplication Rule of Counting
Suppose that there are k different tasks where the i-th task can be
completed in ni ways, i = 1, 2, . . . , k. Then the total number of ways these
k tasks can be completed is given by n1 n2 · · · nk = ∏ki=1 ni .

Example
How many 7-character license plates are possible if the first three
characters must be letters, the last four must be digits 0 − 9, and repeated
characters are allowed?Solution:

26 × 26 × 26 × 9 × 10 × 10 × 10 = 158, 184, 000

Task: Many garage doors have remote-access keypads outside the door.
Let’s suppose a thief approaches a particular garage and notices that four
particular numbers are well-used. If we assume the code uses all four
numbers exactly once, how many 4-digit codes does the thief have to try?
Edwin Moyo (emoyo@mu.ac.zm) February 18, 2022 23 / 29
Counting Techniques and Applications
Combinations
A combination is a selection of distinct objects without regard to the order
in which they are arranged. The number of different combinations of r
objects out of n distinct objects is denoted by nCr (or C (n, r ) or (nr)) and
calculated as
n!
nCr = (5)
r ! (n − r ) !
where n!, called the factorial of n, is defined as
n! = n (n − 1)(n − 2) · · · 1 = ∏ni=1 i. If a + b = n, then (na) = (bn ).

Example
Let’s consider the board of trustees with 30 members. In how many ways
could the board elect four members for the finance committee? Solution:
 
30 30!
= = 27405
4 (30 − 4)! × 4!
Edwin Moyo (emoyo@mu.ac.zm) February 18, 2022 24 / 29
Counting Techniques and Applications
Permutations
A permutation is a combination arranged in a particular order. The
number of different permutations of r objects from n distinct objects is
denoted by nPr (or P(n,r)) and calculated as

n!
nPr = (6)
(n − r ) !

Note that n! = n (n − 1)! = n (n − 1)(n − 2)! = · · · . Also, 0! = 1.

Example
Suppose an organization elects its officers from a board of trustees. If
there are 30 trustees, how many possible ways could the board elect a
president, vice-president, secretary, and treasurer? Solution:
n! 30!
nPr = = = 657720
(n − r ) ! (30 − 4)!
Edwin Moyo (emoyo@mu.ac.zm) February 18, 2022 25 / 29
Counting Techniques and Applications

Note: Permutations are, therefore, nothing more than arrangements,

while combinations are selections.
Permutations with Non-distinct Items
The number of permutations of n objects, where there are n1 of the first
type, n2 of the second type, etc, is
n!
n1 ! · n2 ! · · · · · · nk !

Example
In how many ways can the word REARRANGE be rearranged?
Solution:
9!
= 15, 120
3! × 2! × 2!

Edwin Moyo (emoyo@mu.ac.zm) February 18, 2022 26 / 29

Counting Techniques and Applications

Example
11!
Example : Evaluate (a) 7! (b) 8! (c) 10C 6 (d) 10P6
Solution
(a) 7! = 7 × 6 × 5 × 4 × 3 × 2 × 1 = 5040
11! 11×10×9×8!
(b) 8! = 8! = 11 × 10 × 9 = 990
10 ×9×8×7
(c) 10C 6 = ( 6 ) = 6!4! = 10
10!
4×3×2×1 = 210
10×9×8×7×6×5×4!
(d) 10P6 = 10!
4! = 4! = 10 × 9 × 8 × 7 × 6 × 5 = 151200

Example: A committee of 5 persons is to be selected randomly from a

group of 5 men and 10 women. Find the probability that the committee
(a) consists of 2 men and 3 women
(b) consists only of women

Edwin Moyo (emoyo@mu.ac.zm) February 18, 2022 27 / 29

Counting Techniques and Applications
Examples

Solution
(a) The total number of outcomes is the total number of ways of
choosing 5 people out of 15. This is n (Ω) = (15
5 ) = 3003. But 2 men
can be chosen from 5 in (52) = 10 ways and 3 women can be selected
from 10 in (10
3 ) = 120 ways. Thus, the committee of 2 men and 3
women can be selected in a total of 10 × 120 = 1200 different ways,
so that P(committee consists of 2 men and 3 women) = 12003003 = 0.4.
(b) Since the committee of 5 will consist only of women, these can be
selected in (10
5 ) = 252 ways, so that P(committee consists only of
252
women) = 3003 = 0.084.

Edwin Moyo (emoyo@mu.ac.zm) February 18, 2022 28 / 29

Edwin Moyo (emoyo@mu.ac.zm) February 18, 2022 29 / 29