Statistical Methods

By
Wilhemina Adoma Pels

Department of Statistics and Actuarial Science

KNUST

Probability

February 13, 2023

1 / 46
Basic Concepts of Probability

Introduction
A cynical person once said, “The only two sure things are death
and taxes.” This philosophy no doubt arose because so much in
people’s lives is affected by chance.

From the time you awake until you go to bed, you make decisions
regarding the possible events that are governed at least in part
by chance.

For example, should you carry an umbrella to work today?

Will your phone battery last until tomorrow?

Probability as a general concept can be defined as the chance

of an event occurring.

2 / 46
Basic Concepts of Probability

Introduction
Probability forms the basis of inferential statistics. We can
think of the probability of an outcome as the likelihood of
observing that outcome. If something has a high likelihood of
happening, it has a high probability (close to 1). If something
has a small chance of happening, it has a low probability (close
to 0). If something occurs that has a low probability, we
investigate to find out ”what’s up”.

Probability
Is a measure of the likelihood of a random phenomenon or
chance behavior. Probability describes the long-term proportion
with which a certain outcome will occur in situations with
short-term uncertainty.

3 / 46
Basic Concepts of Probability

The theory of probability grew out of the study of various games

of chance using coins, dice and card. Since these lend themselves
well to the application of concepts of probability

4 / 46
Basic Concepts of Probability Cont’d
Illustration
Consider an experiment in which only one of two possible
outcomes can occur. For example, the result of treatment with
an antibiotic is that an infection is either cured or not cured.

Definitions
a. Experiment: An experiment is any process that generates
a set of data or well-defined outcomes. There are two types
of experiments, namely Deterministic and Random (or
Chance) Experiment. In the deterministic experiments the
observed results are not subject to chance while the
outcomes of random experiments cannot be predicted with
certainty. A random experiment could be as simple as
tossing a coin or die and observing an outcome or complex
as choosing 50 people from a population and testing them
for the AIDS disease.
5 / 46
Definitions Cont’d
b. Trial
Each repetition of an experiment is called a trial. That is, a
trial is a single performance of an experiment.

c. Outcome
The possible result of each trial of an experiment is called an
outcome. When an outcome of an experiment has equal chance
of occurring as the others the outcomes are said to be equally
likely. For example, the toss of a coin and a die yield the
possible outcomes in the sets, {H, T} and {1, 2, 3, 4, 5, 6} and
a play of a football match yields {win (W), loss (L), draw (D)}.

Sample Space
Sample space is the collection of all possible outcomes at a
probability experiment. We use the notation S for sample
space. Each element or outcome of the experiment is called
sample point.
6 / 46
Example of Sample Spaces
1 The results of two and three tosses of a coin give the
following sample spaces:
S = {HH, HT, T H, T T }
S = {HHH, HHT, HT H, HT T, T HH, T HT, T T H, T T T }
2 Some sample spaces for various probability experiments are
shown here:

7 / 46
Example of Sample Spaces
Find the sample space for rolling two dice

8 / 46
Definitions Cont’d

Event:
An event is a collection of one or more outcomes from an
experiment. That is, it is a subset of a sample space. It is
denoted by a capital letter. For example we may have:
The event of observing a head (H) in three tosses of a coin,
A = {HTT, TTH}
Consider a newly married couple planning to have three
children. The event of the family having two girls is: D =
{BGG, GBG, GGB}

An event can be one outcome or more than one outcome.

For example, if a die is rolled and a 6 shows, this result is
called an outcome, since it is a result of a single trial.

9 / 46
Definitions Cont’d

An event with one outcome is called a simple event. The

event of getting an odd number when a die is rolled is
called a compound event, since it consists of three
outcomes or three simple events

In general, a compound event consists of two or more

outcomes or simple events

Tree Diagram:
The tree diagram represents pictorially the outcomes of random
experiment. The probability of an outcome which is a sequence
of trials, is represented by any path of the tree. For example,

10 / 46
Definitions Cont’d
Consider a couple planning to have three children, assuming each
child born is equally likely to be a boy (B) or girl (G).

11 / 46
Determination of Probability of an Event

The probability of an event A, denoted, P(A), gives the

numerical measure of the likelihood of the occurrence of
event A which is such that 0 ≤ P (A) ≤ 1.

If P (A) = 0, the event A is said to be impossible to occur

and if P (A) = 1, A is said to be certain. If A′ is the
complement of the event A, then P (A′ ) = 1 − P (A), called
the probability that event A will not occur.

There are three main schools of thought in defining and

interpreting the probability of an event. These are the
Classical Definition, Empirical Concept and the
Subjective Approach. The first two are referred to as
the Objective Approach.

12 / 46
Probability of an Event
a. The Classical Definition
This is based on the assumption that the outcomes of an
experiment are equally likely. For example, if an experiment
can lead to n mutually exclusive and equally likely outcomes,
then the probability of the event A is defined by

n(A) N umber of successf ul outcomes

P (A) = =
n(S) N umber of possible outcomes

The classical definition of probability of event A is referred to

as priori probability because it is determined before any
experiment is performed to observe the outcomes of event A.

The Empirical Concept:

This concept uses the relative frequencies of past occurrences to
develop probabilities for future. The probability of an event A
happening in future is determined by observing what fraction of
the time similar events happened in the past. That is,
13 / 46
Probability of an Event
The Empirical Concept Cont’d:
number of times A occured in the past
P (A) =
T otal number of observation
The relative frequency of the occurrence of the event A used to
estimate P(A) becomes more accurate if trials are largely
repeated. The relative frequency approach of defining P(A) is
sometimes called posteriori probability because P(A) is
determined only after event A is observed.

The Subjective Definition:

The subjective concept of probability is based on the degree of
belief through the evidence available. The probability of an
event A may therefore be assessed through experience, intuitive,
judgement or expertise. For example, determining the
probability of getting a cure of a disease or going to rain today.
This approach to probability has been developed relatively
recently and is related to Bayesian Decision Analysis.
14 / 46
Probability of an Event

The difference between classical and empirical probability is

that classical probability assumes that certain outcomes are equally
likely (e.g. when a die is rolled), while empirical probability relies
on actual experience to determine the likelihood of outcomes.

In empirical probability, one might actually roll a given die 6000

times, observe the various frequencies, and use these frequencies
to determine the probability of an outcome.

15 / 46
Probability of an Event
Example 1
Consider the problem of a couple planning to have three
children, assuming each child born is equally likely to be a boy
(B) or a girl (G).
a. List the possible outcomes in this experiment
b. What is the probability of the couple having exactly two
girls?

Solution:
The sample space for this experiment is
S = {BBB, BBG, BGB, BGG, GBG, GGB, GGG}
Let A be the event of the couple having exactly two girls.
Then, A = {BGG, GBG, GGB}
n(A) 3
P (A) = =
n(S) 8
16 / 46
Probability of Compound Events

Two or more events are combined to form a single event using

the set operations, ∪ and ∩ . The event
(A ∪ B) occurs if either A or B both occur(s).
(A ∩ B) occurs if both A and B occur.
Definitions:
Mutually Exclusive Events
Two or more events which have no common outcome(s) (i.e.
never occur at the same time) are said to be mutually exclusive.
If A and B are mutually exclusive events of an experiment, then
A ∩ B = ø and P (A ∪ B) = P (A) + P (B), since P (A ∩ B) = 0

17 / 46
Probability of Events
Independent Events:
Two or more events are said to be independent if the
probability of occurrence of one is not influenced by the
occurrence or non- occurrence of the other(s). Mathematically,
the two events, A and B are said to be independent, if and only
if P (A ∩ B) = P (A) · P (B). However, if A and B are such that,
P (A ∩ B) = P (A) · P (B|A), they are said to be conditionally
independent.
Conditional Probability:
Let A and B be two events in the sample space, S with
P (B) > 0. The probability that an event A occurs given that
event B has already occurred, denoted P (A|B), is called the
conditional probability of A given B. The conditional
probability of A given B is defined as.
P (A|B) = P P(A∩B)
(B) , P (B) > 0. In particular, if S is a finite
n(A∩B) n(B)
equiprobable space, then P (A ∩ B) = n(S) , P (B) = n(S)
18 / 46
Probability of Events
Exhaustive Events:
Two or more events defined on the same sample space are said
to be exhaustive if their union is equal to the sample space S
(thus, if they partition the sample space mutually exclusively).

Eg: if A1 , A2 , A3 ∈ S A1 ∪ A2 ∪ A3 = S
Definition (partition of sample space):
A1 , A2 , A3 · · · An form a partition of the same sample space S if
the following hold:
1 Ai ̸= ø for all i = 1, 2, 3, · · · , n
2 Ai ∩ Aj = ø for all i ̸= j, i, j = 1, 2, 3, · · · , n
Pn
i=1 Ai = S
3

In other words, the n - events A1 , A2 , A3 · · · An form a partition

of the sample space S if the n - events are (a) nonempty, (b)
mutually exclusive and (c) collectively exhaustive.
19 / 46
Probability of Events
Example
a. In a certain population of women, 40% have had breast
cancer, 20% are smokers and 13% are smokers and have
had breast cancer. If a women is selected at random from
the population, what is the probability that she had breast
cancer, smokes or both?
b. Let A and B be events such that P (A) = 0.6, P (B) = 0.5
and (A ∪ B) 0.8.
Find P (A|B)
Are A and B independent?
Solution
a. Let B be the event of women with breast cancer and W the
event of women who smoke. Then,
P (B) = 0.4, P (W ) = 0.2, P (B ∩ W ) = 0.13
= P (B ∪ W ) = P (B) + P (W ) − P (B ∩ W )
= 0.4 + 0.20 − 0.13 = 0.47
20 / 46
Probability of Events

b. Given that P (A) = 0.6, P (B) = 0.5, P (A ∪ B) = 0.8

(i)P (A ∩ B) = P (A) + P (B) − P (A ∪ B)

= 0.6 + 0.5 − 0.8 = 0.3

P (A ∩ B)
P (A|B) = , P (B) > 0
P (B)
0.3 3
= = = 0.6
0.5 5
ii. A and B are independent if P (A) · P (B) = P (A ∩ B)
P (A) · P (B) = (0.6)(0.5) = 0.3 = P (A ∩ B)
Which means that A and B are independent.

21 / 46
The Multiplication Rule for P (A ∩ B)
The definition of conditional probability yields the following
result, obtained by multiplying both sides of the conditional
probability equation by P(B).

P (A ∩ B)
P (A|B) =
P (B)
P (A ∩ B)
P (A|B) ∗ P (B) = ∗ P (B)
P (B)
P (A|B) ∗ P (B) = P (A ∩ B)
This rule is important because it is often the case that P (A ∩ B)
is desired, whereas both P(B) and P (A|B) can be specified from
the problem description.

22 / 46
The Law of Total Probability
Let A1 , · · · , Ak be mutually exclusive and exhaustive events.
Then for any other event B,

P (B) = P (B|A1 ) ∗ P (A1 ) + · · · + P (B|Ak ) ∗ P (Ak )

k
X
= P (B|Ai ) ∗ P (Ai )
i=1

Bayes’ Rule
The power of Bayes’ rule is that in many situations where we
want to compute P (A|B) it turns out that it is difficult to do so
directly, yet we might have direct information about P (B|A).
Bayes’ rule enables us to compute P (A|B) in terms of P (B|A).

P (A ∩ B) P (B|A)P (A)
P (A|B) = =
P (B) P (B)

23 / 46
Bayes’ Theorem
Let A and Ac constitute a partition of the sample space S such
that with P (A) > 0 and P (Ac ) > 0, then for any event B in S
such that P (B) > 0,

P (B|A)P (A)
P (A|B) =
P (B|A)P (A) + P (B|Ac )P (Ac )

Example
A paint-store chain produces and sells latex and semigloss
paint. Based on long-range sales, the probability that a
customer will purchase latex paint is 0.75. Of those that
purchase latex paint, 60% also purchase rollers. But only 30%
of semigloss pain buyers purchase rollers. A randomly selected
buyer purchases a roller and a can of paint. What is the
probability that the paint is latex?

24 / 46
Solution
L = {The customer purchases latex paint.}, P(L) = 0.75
S = {The customer purchases semigloss paint.}, P(S) = 0.25
R = {The customer purchases roller.}
P (R|L) = 0.6; P (R|S) = 0.3

P (R) = P (R|L)P (L)+P (R|S)P (S) = (0.6×0.75)+(0.3×0.25) = 0.525

P (L ∩ R)
P (L|R) =
P (R)
P (R|L)P (L)
=
P (R)
0.6 × 0.75
=
(0.6 × 0.75) + (0.3 × 0.25)
≈ 0.857

25 / 46
Axioms of Probability
Given an experiment and a sample space, S , the objective of
probability is to assign to each event A a number P(A), called
the probability of the event A, which will give a precise measure
of the chance that A will occur. To ensure that the probability
assignments will be consistent with our intuitive notions of prob-
ability, all assignments should satisfy the following axioms (basic
properties) of probability
A.1: For every event A, 0 ≤ P (A) ≤ 1
A.2: P(S) = 1
A.3: If A and B are mutually exclusive events, i.e A ∩ B = øthen
P (A ∪ B) = P (A) + P (B)
A.4: If A1 , A2 , · · · , An is a sequence of n mutually exclusive
events, then,

P (A1 ∪A2 ∪A3 ∪· · ·∪An ) = P (A1 )+P (A2 )+P (A3 )+· · ·+P (An )
26 / 46
 Theorems

The following theorems arise directly from the above axioms:

Theorem 1 : If ø is the empty set, then P (ø) = 0
Theorem 2 : If A′ is the complement of an event A , then

P (A′ ) = 1 − P (A)

27 / 46
Cont’d Probability

Conditional Probability
If A and B are any two events defined on the same sample space
S, the conditional probability of A given B is defined by

P (A ∩ B)
P (A|B) = , P (B > 0) (1)
P (B)

Note: If P (B) = 0, then P (A|B) is undefined

Example: In a certain community, 36 percent of the families

own a dog, 22 percent of the families that own a dog also own a
cat, and 30 percent of the families own a cat. A family is selected
at random.
(a) Compute the probability that the family owns both a cat and
dog.
(b) Compute the probability that the family owns a dog, given
that it owns a cat.
28 / 46
Cont’d Probability

Solution: Let C = {family owns a cat} and D = {family owns a

dog}. From the problem, we are given that P (D) = 0.36, P (C|D) =
0.22 and P (C) = 0.30.
In (a), we want P (C ∩ D). We have

P (C ∩ D) P (C ∩ D)
0.22 = P (C|D) = =
P (D) 0.36

Thus, P (C ∩ D) = 0.36 × 0.22 = 0.0792

For (b), we want P (D|C) Simply use the definition of conditional
probability

P (C ∩ D) 0.0792
P (D|C) = = = 0.264
P (C) 0.30

29 / 46
Some Axioms

It is interesting to note that conditional probability P (A|B) sat-

isfies the axioms for a probability set function when P (B) > 0.
In particular,
1 P (A|B) ≥ 0
2 P (B|B) = 1
3 If A1 , A2 , · · · is a countable sequence of pairwise mutually
exclusive events (i.e. Ai ∩ Aj = ∅, i ̸= j) in S then
  ∞
X
P ∪∞
i=1 Ai |B = P (Ai |B) (2)
i=1

30 / 46
MULTIPLICATION LAW OF PROBABILITY

1.The multiplication rule

If we multiply each side of Equation (2) by P (B), we obtain the
following multiplication rule which enables us to calculate the
probability that two event will both occur.

If in an experiment, the events A and B can both occur,then

P (B)P (A|B) = P (A ∩ B) (3)

31 / 46
Since the events A ∩ B and B ∩ A are equivalent, it follows from
Equation (4) that we can also write
P (A ∩ B) = P (B ∩ A) = P (A)P (B|A) (4)
In order words, it does not matter which event is referred to as
A and which event is referred to as B.

Example: Suppose that we have a basket containing 20 oranges,

of which 5 are spoilt. If 2 oranges are selected at random and
removed from the basket in succession without replacing the the
first, what is the probability both oranges are spoilt.

Solution: Let A denote the event that the first orange is spoilt
and B the event that the second orange is spoilt, then we wish to
5
find P (A ∩ B). The probability of removing a spoilt orange is 20 .
If the first orange is spoilt, then probability of removing a second
4
spoilt orange from the remaining 4 is 19 . By the multiplication
rule,  5  4  1
P (A ∩ B) = P (A)P (B|A) = =
20 19 19
32 / 46
MULTIPLICATION LAW OF PROBABILITY
The multiplication rule can be applied to two or more events. For
three events, A, B, C, the multiplication rule takes the following
form.
Theorem 1
P (A ∩ B ∩ C) = P (A)P (B|A)P (C|A ∩ B), where P (A) ̸= 0
and P (A ∩ B) ̸= ∅
Proof:By associative law,
A ∩ B ∩ C = (A ∩ B) ∩ C
∴ P (A ∩ B ∩ C) = P [(A ∩ B) ∩ C]
= P (A ∩ B)P (C|A ∩ B)
= P (A)P (B|A)P (C|A ∩ B)
Theorem 1 can be extended by mathematical induction to the
following theorem
Theorem 2
For any events A1 , A2 , · · · , An (n > 2)
P (A1 ∩ A2 ∩ · · · ∩ An ) =
P (A1 )P (A2 |A1 )P (A3 |A1 ∩ A2 ) · · · P (An |A1 ∩ A2 · · · ∩ An−1 ) 33 / 46
Independent Events
When the occurrence or non-occurrence of A has no effect on
whether or not B occurs, and vice versa, we say that the events
A and B are independent.
Conditions
Two events with nonzero probabilities are independent iff, any
one of the following equivalent statement is true
P (A|B) = P (A)
P (B|A) = P (B)
P (A ∩ B) = P (A)P (B)
Otherwise, A and B are called dependent events.

Note that if A and B are independent,

P (A ∩ B) P (A)P (B)
P (A|B) = = = P (A)
P (B) P (B)
and P (A ∩ B) P (B)P (A)
P (B|A) = = = P (B)
P (A) P (A)
34 / 46
Independent Events
Example
A pair of dice is thrown twice. Find the probability of getting
totals of 7 and 11
Solution: Let A1 , A2 , B1 , B2 be the respective events that a total
of 7 occurs on the first throw, a total of 7 occurs on the second
throw, a total of 11 occurs on the first throw and a total of 11
occurs on the second throw. We are interested in the probability
of the event (A1 ∩ B2 ) ∪ (A2 ∩ B1 ). it is clear that all the events
are independent. Moreover, A1 ∩ B2 and (A2 ∩ B1 ) are mutually
exclusive events. Hence

P [(A1 ∩ B2 ) ∪ (A2 ∩ B1 )] = P (A1 ∩ B2 ) + P (A2 ∩ B1 )

= P (A1 )P (B2 ) + P (A2 )P (B1 )
 6  2   6  2  1
= + =
36 36 36 36 54
35 / 46
Tutorials

Conditional Probability, Multiplication rule and independent events-

Tutorial.

https://bit.ly/3bfKbhv

36 / 46
Total Probability Rule

Theorem
If A1 , A2 , · · · , An form a partition of a Sample space S, then for
any event B defined on S s.t. P (B) > 0, the total probability
rule is
Xn
P (B) = P (Ai )P (B|Ai )
i=1

Suppose A and B are events in a nonempty sample space S. We

can express the event A as follows ,A = (A ∩ B) ∪ (A ∩ B̄), union
of disjoint events.
By the third Kolmolgorov axiom

P (A) = P (A ∩ B) + P (A ∩ B̄)

= P (A|B)P (B) + P (A|B̄)P (B̄)

37 / 46
Total Probability Rule

where the last step follows from the multiplication law of proba-
bility. This is called the Law of Total Probability (LOTP).
The LOTP is helpful. Sometimes P (A|B), P (A|B̄), and P (B)
may be easily computed with available information whereas com-
puting P (A) directly may be difficult. Note : The LOTP follows
from the fact that B and B̄ partition S ; that is,
B and B̄ are disjoint, and
B ∩ B̄ = S
Example: An insurance company classifies people as “accident-
prone” and “non accident-prone.” For a fixed year, the probabil-
ity that an accident-prone person has an accident is 0.4, and the
probability that a non-accident-prone person has an accident is
0.2. The population is estimated to be 30 percent accident-prone.
(a) What is the probability that a new policy-holder will have an
accident?

38 / 46
Total Probability Rule
Solution
Define A = {policy holder has an accident} and B = {policy
holder is accident-prone}. Then,
P (B) = 0.3; P (A|B) = 0.4; P (B̄) = 0.7; and P (A|B̄) = 0.2.

By the LOTP,

P (A) = P (A|B)P (B) + P (A|B̄)P (B̄)

= (0.4)(0.3) + (0.2)(0.7) = 0.26

b. Now suppose that the policy-holder does have an accident.

What is the probability that he was “accident-prone?
We want P (B|A). Note that

P (A ∩ B) P (A|B)P (B) (0.4)(0.3)

P (B|A) = = = = 0.46
P (B) P (A) 0.26
39 / 46
Bayes Rule
We see that, in general,

P (A ∩ B P (A|B)P (B) P (A|B)P (B)

P (B|A) = = =
P (A) P (A) P (A|B)P (B) + P (A|B̄)P (B̄)

This is a form of Bayes Rule.

EXAMPLE
A KCCR test is 95 percent effective at detecting COVID’19
when it is present (sensitivity). When the virus is not present,
the test is 99 percent effective at declaring the subject negative
(specificity). If 8 percent of the population has the disease
(prevalence), what is the probability that a subject has the
disease given that
his test is positive?
his test is negative?

40 / 46
Solution:
Let D = {disease is present} and A = {test is positive}.
We are given that P(D) = 0.08 (prevalence), P (A|D) = 0.95
(sensitivity), and P (Ā|D̄) = 0.99(specificity).
In part (a), we want to compute P (A|D). By Bayes Rule,

P (A|D)P (D)
P (D|A) =
P (A|D)P (D) + P (A|D̄)P (D̄)

(0.95)(0.08)
= ≡ 0.892
(0.95)(0.08) + (0.01)(0.92)
In part (b), we want P (D|Ā). By Bayes Rule,

P (Ā|D)P (D)
P (D|Ā) =
P Ā|D)P (D) + P (Ā|D̄)P (D̄)

(0.05)(0.08)
= ≡ 0.004
(0.05)(0.08) + (0.99)(0.92)
41 / 46
Bayes’ Theorem, Screening Tests, Sensitivity,
Specificity, and Predictive Value Positive and Negative:
There are two states regarding the disease and two states regard-
ing the result of the screening test:

We define the following events of interest:

D: the individual has the disease (presence of the disease)
D̄: the individual does not have the disease (absence of the dis-
ease)
T : the individual has a positive screening test result
T̄ : the individual has a negative screening test result

42 / 46
Definitions
There are two false results:
1. A false positive result:
This result happens when a test indicates a positive status
when the true status is negative. Its probability is:
P (T |D̄) = P( positive result | absence of the disease )

2. A false negative result

This result happens when a test indicates a negative status
when the true status is positive. Its probability is:
P (T̄ |D) = P( negative result | presence of the disease )

The Sensitivity:
The sensitivity of a test is the probability of a positive test
result given the presence of the disease. P (T |D) = P( positive
result of the test | presence of the disease )

43 / 46
Definitions Cont’d

The specificity:
The specificity of a test is the probability of a negative test
result given the absence of the disease. P (T̄ |D̄) = P( negative
result of the test | absence of the disease)

To clarify these concepts, suppose we have a sample of (n) sub-

jects who are cross-classified according to Disease Status and
Screening Test Result as follows:

Disease
Test Result Present(D) Absent (D̄) Total
Positive(T) a b a + b = n(T )
Negative (T̄ ) c d c + d = n(T̄ )
Total a + c = n(D) b + d = n(D̄) n

44 / 46
Tutorials

The Total Probability Rule and Bayes Theorem-Tutorial.

https://www.youtube.com/watch?v=gf2tm7SAOgs

45 / 46
46 / 46