PROBABILITY AND STATISTICS:

STAT 253

By
Jonathan Kwaku Afriyie

Department of Statistics and Actuarial Science

KNUST

archimedes09.jak@gmail.com
jonathan.afriyie@knust.edu.gh

1 / 49
Introduction to Probability

Why learn Probability?

Nothing in life is certain.
In everything we do, we gauge the chances of successful
outcomes, from business to medicine to the weather.
A probability provides a quantitative description of the
chances or likelihoods associated with various outcomes.

Probability
Probability is a measure of the likelihood that an event in the
future will happen

2 / 49
INTRODUCTION TO PROBABILITY

Experiment
An experiment is any process that generates well-defined
outcomes. An experiment that can result in different outcomes,
even though it is repeated in the same manner every time, is
called a random experiment.

Trial
A trial is a single performance of an experiment (that is, a
repetition of an experiment.)

3 / 49
INTRODUCTION TO PROBABILITY

Outcome
The possible results of each trial of an experiment is called
outcome. When an outcome of an experiment has equal chance
of occurring as the others the outcomes are said to be equally
likely. For example, the toss of a coin and a die yield the
possible outcomes in the set, {H,T} and {1, 2, 3, 4, 5, 6} and a
play of a football match yields {win(W ), loss(L), draw(D)}.

4 / 49
INTRODUCTION TO PROBABILITY

Sample Space
Sample Space It is the set of all possible outcomes of an
experiment. The letter S denotes it. Each element or outcome
of the experiment is called sample point or outcome. For
example: The results of two and three tosses of a coin give the
sample spaces:
S = {HH, HT, T H, T T }
S = {HHH, HHT, HT H, HT T, T HH, T HT, T T T }

5 / 49
INTRODUCTION TO PROBABILITY

Events
An event is a collection of one or more outcomes from an
experiment which is a subset of a sample space. It is denoted by
a sample space. For example we may have: The event of
observing exactly two heads (H’s) in three tosses of a coin

A = {HHT, HT H, T HH}
The event of observing a total score of 8 on two tosses of a die

B = {(2, 6), (3, 5), (4, 4), (5, 3), (6, 2)}

6 / 49
INTRODUCTION TO PROBABILITY

Operations on Events
Two or more events are combined to form a single event using
the two set operations, ∪ and ∩. The event
(A ∪ B) occurs if either A or B or both occur(s)
(A ∩ B) occurs if both A and B occur.
A′ (Ac or Ā) denotes the event which occurs if and only if
and only if A does not occur.

7 / 49
INTRODUCTION TO PROBABILITY

Mutually Exclusive and Independent Event

8 / 49
INTRODUCTION TO PROBABILITY

Mutually Exclusive Events

Two events A and B are said to be mutually exclusive if
P (A ∩ B) = 0
Hence P (A ∪ B) = P (A) + P (B)

9 / 49
INTRODUCTION TO PROBABILITY

Independent Events
Two events are said to be independent if the occurrence of one
has no effect on the occurrence of the other.
If A and B are independent events then

P (A and B) = P (A ∩ B) = P (A).P (B)

10 / 49
INTRODUCTION TO PROBABILITY

De Morgan’s Law
For two events A and B in the same sample space,
(A ∪ B)′ = A′ ∩ B ′
(A ∩ B)′ = A′ ∪ B ′
If A, B and C are events defined on the same sample space then
(A ∪ B ∪ C)′ = A′ ∩ B ′ ∩ C ′
(A ∩ B ∩ C)′ = A′ ∪ B ′ ∪ C ′

11 / 49
INTRODUCTION TO PROBABILITY
Probability of an event
The classical definition of probability uses the idea of equally
likely outcome to compute the probability of an event. Thus, if
the events in a sample space are equally likely, then

n(E)
P (E) =
n(S)
where n(E) is the number of elements in E

The Axiomatic definition of Probability

For any event E, 0 ≤ P (E) ≤ 1
P (S) = 1 for the sure event S.
If A and B are mutually exclusive events of (subsets of S),
then

P (A ∪ B) = P (A) + P (B)
12 / 49
INTRODUCTION TO PROBABILITY
Example
E 1 2 3 4 5
P(E) 0.05 0.25 0.3 0.15 0.25
a. Calculate P (E ≥ 2)
b. Find P (1 ≤ E < 5)
c. Find P (E ≥ 3)

Solution
a. P (E ≥ 2) = P (E = 2) + P (E = 3) + P (E = 4) + P (E = 5)
= 0.25 + 0.3 + 0.15 + 0.25 = 0.95
b. Find P (1 ≤ E < 5) = P (E = 1) + P (E = 2) + P (E =
3) + P (E = 4)
= 0.05 + 0.25 + 0.3 + 0.15 = 0.75
c. Find P (E ≥ 3) = P (E = 3) + P (E = 4) + P (E = 5) =
0.3 + 0.15 + 0.25
13 / 49
INTRODUCTION TO PROBABILITY

Example
Events A and B are such that P (A) = 0.4, P (B) = 0.5 and
P (A ∩ B) = 0.2. Find a. P (A ∪ B) b. P (A ∩ B ′ ) c. P (A′ ∩ B)
d. P (A′ ∩ B ′ )

Solution
a. P (A ∪ B) = P (A) + P (B) − P (A ∩ B) = 0.4 + 0.5 − 0.2
b. P (A ∩ B ′ ) = P (A) − P (A ∩ B) = 0.4 − 0.2 = 0.2
c. P (A′ ∩ B) = P (B) − P (A ∩ B) = 0.5 − 0.2 = 0.3
d. P (A′ ∩ B ′ ) = 1 − P (A ∪ B) = 1 − 0.7 = 0.3

14 / 49
INTRODUCTION TO PROBABILITY

Conditional Probability

15 / 49
INTRODUCTION TO PROBABILITY

Example

16 / 49
INTRODUCTION TO PROBABILITY

The Total Probability Rule

The events A1 , A2 , A3 , A4 , A5 are mutually exclusive and
S = A1 ∪ A2 ∪ A3 ∪ A4 ∪ A5 . These events partition the sample
space S. It can be seen that if B is an event defined on the
sample space S such that P (B) > 0, then

B = (A1 ∩ B) ∪ (A2 ∩ B) ∪ (A3 ∩ B) ∪ (A4 ∩ B) ∪ (A5 ∩ B)

Since (A1 ∩ B), (A2 ∩ B), ..., (A5 ∩ B) are mutually exclusive
events,
17 / 49
INTRODUCTION TO PROBABILITY

Total Probability Rule Cont’d

P (B) = P (A1 ∩B)+P (A2 ∩B)+P (A3 ∩B)+(A4 ∩B)+(A5 ∩B)
P (B) = P (B | A1 )P (A1 ) + P (B | A2 )P (A2 ) + P (B | A3 )P (A3 )
+ P(B| A4 )P (A4 ) + P (B | A5 )P (A5 )

Theorem
If A1 , A2 , ..., An form a partition of a sample space S, then for
any eventPB defined on S such that P (B) > 0 ,
P (B) = ni=1 P (B | Ai )P (Ai )

18 / 49
INTRODUCTION TO PROBABILITY

Example
In a certain assembly plant, three machines A, B, and C make
30%, 45% and 25%, respectively, of the products. It is known
from past experience that 2% of the products made by machine
A, 3% of the products made by machine B and 2% of the
products made by machine C are defective. If a finished
product is selected at random, what is the probability that it is
defective?

Solution
Let A1 denote the event “the finished product was made by
machine A”, A2 denote the event “the finished product was
made by machine B”, A3 denote the event “the finished product
was made by machine C”, and let D denote the event “the
finished product is defective”

19 / 49
INTRODUCTION TO PROBABILITY

Solution
We wish to find P(D)
P (A1 ) = 0.3, P (A2 ) = 0.45, P (A3 ) = 0.25, P (D | A1 ) = 0.02,
P (D | A2 ) = 0.03, P (D | A3 ) = 0.02

P (D) = P (D | A1 )P (A1 ) + P (D | A2 )P (A2 ) + P (D | A3 )P (A3 )

= 0.30.02 + 0.450.03 + 0.250.02
= 0.0245

The probability that a finished product selected at random is

defective is 0.0245

20 / 49
INTRODUCTION TO PROBABILITY

Bayes’ Theorem
In the Example above, if a finished product is found to be
defective, what is the probability that it was made by machine
A1 ?
1 ∩D)
P (A1 | D) = P (A
P (D) = P (D|A 1 )P (A1 )
P (D) = (0.02)(0.3)
0.0245 = 0.2449

Bayes’ Theorem
Let A1 , A2 , ..., An be a collection of events which partition a
sample space S. Let B be an event defined on S such that
P (B) ̸= 0 Then for any of the events Aj , (j = 1, 2, ..., n)
P (B|Aj )P (Aj )
P (Aj | B) = P (B) ,j = 1, 2, ..., n
Pn
P (B) = i=1 P (B | Ai )P (Ai )

21 / 49
INTRODUCTION TO PROBABILITY

Example
A consulting firm rents cars from three agencies: 30% from
agency A, 20% from agency B and 50% from agency C. 15% of
the cars from A, 10% of the cars from B and 6% of the cars
from C have bad tyres. If a car rented by the firm has bad
tyres, find the probability that
a it came from agency C
b it came from agency A
c it came from agency B
d the car had a bad tyre

22 / 49
INTRODUCTION TO PROBABILITY

Solution
Let A1 denote the event “the car came from agency A”, A2
denote the event “the car came from agency B”, A3 denote the
event “the car came from agency C”, and let T denote the event
“a car rented by the firm has bad tyres”. P (A1 ) = 0.3,
P (A2 ) = 0.2, P (A3 ) = 0.5, P (T | A1 ) = 0.15,
P (T | A2 ) = 0.1 and P (T | A3 ) = 0.064
a.
P (T |A3 )P (A3 )
P (A3 | T ) = P (T |A1 )P (A1 )+P (T |A2 )P (A2 )+P (T |A3 )P (A3 )

0.060.5
= = 0.3158
0.150.3 + 0.10.2 + 0.060.5

23 / 49
INTRODUCTION TO PROBABILITY

Solution Cont’d
P (T |A1 )P (A1 )
b. P (A1 | T ) = P (T |A1 )P (A1 )+P (T |A2 )P (A2 )+P (T |A3 )P (A3 )

0.150.3
= = 0.4737
0.150.3 + 0.10.2 + 0.060.5
P (T |A1 )P (A1 )
c. P (A1 | T ) = P (T |A1 )P (A1 )+P (T |A2 )P (A2 )+P (T |A3 )P (A3 )

0.10.2
= = 0.2105
0.150.3 + 0.10.2 + 0.060.5
d. P (T ) = P (T | A1 )P (A1 ) + P (T | A2 )P (A2 ) + P (T |
A3 )P (A3 )

P (T ) = 0.150.3 + 0.10.2 + 0.060.5 = 0.095

24 / 49
COUNTING TECHNIQUES

In this section we are going to consider two possibilities of

selecting k elements from a set consisting of n distinct choices.
These are:
Ordered, without replacement
Unordered, without replacement

25 / 49
COUNTING TECHNIQUES

Ordered, without replacement(Permutation)

n!
No. of Selections =n Pr =
(n − k)!
Example
As an illustration, suppose one is given the four letters a, b, c,
d. Find the number of permutations of these four letters taken
two at a time.
Solution
4!
No. of Selections =4 P2 = = 4 × 3 = 12
(4 − 2)!
These permutations are easily enumerated as follows:
ab, ba, ac, ca, ad, da, bc, cb, bd, db, cd, dc

26 / 49
COUNTING TECHNIQUES

Ordered, without replacement(Permutation)

Example
There are 50 students in a certain school. In how many ways
can they elect a President, Vice President and Treasurer if no
student is assigned more than 1 position?

Solution
No. of ways =50 P3
50!
=
(50 − 3)!
= 50 × 49 × 48
= 117600

27 / 49
Permutation
What happens if the objects to permute are not distinct? Ex-
ample :
Consider the word PEPPER. How many permutations of the
letters are possible?
Trick: Initially, treat all letters as distinct objects y writing, say,
P1 E1 P2 P3 E2 R :
There are 6! = 720 different orderings of these distinct objects.
Now, there are
3! ways to permute the P’s
2! ways to permute the E’s
1! ways to permute the Rs.
So, 6! is actually 3! × 2! × 1! times too large. That is, there are
6!
= 60 possible permutations
3! × 2! × 1!
n!
f ormula =
n1 !n2 ! · · · nk !
28 / 49
COUNTING TECHNIQUES

Unordered, without replacement(Combination)

 
n n n!
No. of selections = Cr = = k!(n−k)!
r
Example
As an illustration, the number of combinations of two letters
selected from
  the letters a, b, c, d is given by
4 4! 4!
=4 C2 = = 2!(4−2)! = =6
2 2!2!
The actual combinations are ab, ac, ad, bc, bd, cd
In combinations, if two letters are chosen from the four letters
a, b, c, d, the combination ab is the same as ba, but of course it
differs from the combination ac. Primarily, order does not
matter in combination.

29 / 49
COUNTING TECHNIQUES

Unordered, without replacement(Combination)

Example
From a group of 5 friends, you must select 3 friends to join you
on a camping trip. In how many different ways can you select
three friends.
Solution
5! 5! 5×4×3×2×1
No. of ways =5 C3 = 3!(5−3)! = = = 10
3!2! (3 × 2 × 1)(2 × 1)

30 / 49
COUNTING TECHNIQUES

Unordered, without replacement(Combination)

Example
From a group of 5 women and 7 men, how many different
committees consisting of 2 women and 3 men can be formed?

Solution
   
5 7
×
2 3
5! 7!
2!(5−2)! × 3!(7−3)!
10 × 35
350

31 / 49
Random variables

Definition
A random variable is a function that assigns a real number to
each element in the Sample Space of a random experiment.

A random variable is denoted by an uppercase letter, such as X,

and a corresponding lowercase letter such as x, is used to denote
a possible value of X.

Example: Suppose that our experiment consists of tossing 3 fair

coins. If we let Y denote the number of heads that appear, then
Y is a random variable taking on one of the values 0, 1, 2, and 3
with respective probabilities
P {Y = 0} = P {(T, T, T )} = 18
P {Y = 1} = P {(T, T, H), (T, H, T ), (H, T, T )} = 38
P {Y = 2} = P {(T, H, H), (H, T, H), (H, H, T )} = 38
P {Y = 3} = P {(H, H, H)} = 81

32 / 49
Random variables
DISCRETE RANDOM VARIABLES
A random variable (r.v) that can take on at most a countable
number of possible values is said to be discrete. In other words,
a real valued function defined on a discrete sample space is
called a discrete random variable.
PROBABILITY MASS FUNCTION (PMF)
A function f (x) is the probability mass function of a discrete
random variable X if it has the following properties:

f (x) ≥ 0 ∀x

∞
X
f (xi ) = 1
i=1

Also, the notation p(x) or P (X = x) is sometimes used

instead of f (x)
33 / 49
PROBABILITY MASS FUNCTION (PMF)

Example
Determine whether each of the following can serve as a
probability mass function of a discrete random variable:
a. f (x) = 12 (x − 2), x = 1, 2, 3, 4
1
b. g(x) = 10 (x + 1), x = 0, 1, 2, 3
1 2
c. h(x) = 20 x , x = 0, 1, 2, 3, 4

34 / 49
RANDOM VARIABLES
Properties of Expectation and Variance of a Random Variable
If X is a random variable and a is any constant, then
E(aX) = aE(X)
E(a) = a
E(X + a) = E(X) + E(a) = E(X) + a
V ar(aX) = a2 V ar(X)
V ar(a) = 0
V ar(X + a) = V ar(X) + V ar(a) = V ar(X)

Expectation and Variance of a Discrete Random Variable

1 The Mean of the random variable X is
P
E(X) = xf (x)
2 Variance of the random variable X is
V ar(x) = E(X 2 ) − [E(X)]2 = x2 f (x) − [ xf (x)]2
P P

35 / 49
Example:If X is a discrete random variable having the proba-
X=x 1 2 3
bility distribution
P (X = x) k 2k k
Find P (X ≤ 2)
Solution: We know that
X
P (X = x) = 1
⇒ 4k = 1
1
∴k=
4
Hence
P (X ≤ 2) = P (X = 1) + P (X = 2)
= k + 2k = 3k
1 3
⇒3× =
4 4

36 / 49
RANDOM VARIABLE

CONTINUOUS RANDOM VARIABLE

If the range of a random variable X contains an interval (either
finite or infinite) of real numbers, then X is a continuous
random variable

Note: The sample space of the continuous random variable must

be continuous and cannot be discrete. In most of the practical
problems, continuous random variable represent measured data,
such as all possible heights, weights, temperature, etc. whereas
discrete random variables represent count data such as the num-
ber of defectives in a sample and so on

37 / 49
R.V
Other examples: the time that a train arrives at a specified stop
and the lifetime of a transistor. Let X be such a random variable.
We say that X is a continuous random variable if there exists a
nonnegative function f , defined for all real x ∈ (−∞, ∞) that,
for any set B of real numbers
Z
P (X ∈ B) = f (x)dx
B
The function f is called the probability density function of the
random variable X.
PROPERTIES OF PROBABILITY DENSITY FUNCTION
(PDF)
fR (x) ≥ 0, ∀x ∈ R
∞
−∞ f (x)dx = 1
Rb
P (a < X < b) = a f (x)dx
Note: In case of continuous random variables, the probability
at a point is always zero, i.e.P (X = a) = 0 for all possible values
of a. 38 / 49
R.V
CUMULATIVE DISTRIBUTION FUNCTION (CDF)
The cumulative distribution F (x) of a continuous random
variable X with PDF f (x) is given by
Z x
F (x) = P (X ≤ x) = f (t)dt, −∞ < x < ∞
−∞

d
F (x)
f (x) =
dx
If X is continuous random variable with PDF f (x) , then
Z ∞
mean = E(X) = xf (x)dx
−∞

V ar(X) = E(X 2 ) − [E(X)]2

R∞
E(X 2 ) = −∞ x
2 f (x)dx

39 / 49
R.V

Example
(
1 2
Show that f (x) = 9x , 0<x<3
0, elsewhere
is the p.d.f of a continuous random variable X. Hence find
a. P (x ≤ 2)
b. P (x > 1)
c. P (2 ≤ x ≤ 3)

40 / 49
R.V

Example
1 A random variable X has the p.d.f given by
( √
c x, 0<x<1
f (x) =
0, elsewhere
a Find the value of the constant c
b calculate P (X < 41 )
2 A random
( variable X has p.d.f. given by
1
, 0 < x < 10
f (x) = 10
0, elsewhere
Find
a P (X > 8 | X > 5)
b P (X > 7 | X < 9)

41 / 49
R.V

Example
(
x, −0.5 < x < 0.5
a. f (x) =
0, elsewhere
(
1
, −1 < x < 1
b. f (x) = 2
0, elsewhere
(
1
, 0<x<1
c. f (x) = x
0, elsewhere
(
1
, 0<x<1
d. f (x) = 32
3, 2<x<3

42 / 49
R.V

Example
Show that the following functions are probability density
functions for
( some value of c and determine c
cxe−4x , x≥0
a. f (x) = .
0, elsewhere
(
cx2 , −1 < x < 10
b. f (x) = .
0, elsewhere
(
c(1 + 2x), 0<x<2
c. f (x) =
0, elsewhere
(
1 cx
( 2 )e , x≥0
d. f (x) = .
0, elsewhere

43 / 49
R.V

Example
Let X be  a continuous random variable with p.d.f.
−4x,
 −0.5 < x < 0
f (x) = 4x, 0 < x < 0.5

0, elsewhere

Find:
i.P (X ≤ 0.3),
ii. P(X ≤ 0.3),
iii. P(0.2 ≤ X ≤ 0.2)

44 / 49
R.V

Example
Let X denote the length in minutes of a long-distance telephone
conversation. Assume that the p.d.f of X is given by
1
1 − 10
f (x) = 10 e x, x > 0
a. Verify that f is a p.d.f. of a continuous random variable.
b. Find the probability that a randomly selected call will last:
i. at most 7 minutes
ii. at least 7 minutes
iii. exactly 7 minutes

45 / 49
R.V
Example
A random( variable X has the PDF f (x) given by
cxe−x , x>0
f (x) = . Find the value of c and CDF of x
0, x≤0

If f (x) is a PDF, then

Z ∞
f (x)dx = 1
−∞
Z ∞
cxe−x dx = 1
0
" ! !#∞
e −x e−x
c x −1 =1
−1 −1
0
h i
c 0+1
c=1
f (x) = xe−x , x>0 46 / 49
R.V

.
47 / 49
R.V

Exercise
x2
If the CDF of a random variable X is given by 16 for 0 ≤ x < 4.
Find P (X > 1|X < 3)

48 / 49
THANK YOU

49 / 49