Learning UNIT 4: INTRODUCTION TO PROBABILITY

LEARNING OBJECTIVES
• Understand and define different types of probabilities
• Describe properties of probabilities
• Apply rules of probabilities to solve problems
• Use counting rules to count the number of outcomes in a sample or an event

Textbook reference: Chapter 2

4.1. Introduction
Inferential statistics is the branch of statistics that uses sample information to make estimates and decisions
concerning the entire population. Probability theory is used to assess how good such estimates are in the face
of uncertainty as it deals with the chance, likelihood or possibility that an event may occur. The science of
measuring uncertainty is called probability. It gives an indication of how likely a particular outcome is and
plays an important role in the decision-making process.

4.2. Notation And Terminology

Probability
A probability is between 0 and 1 inclusive, i.e., it cannot be negative, and it cannot exceed one.

Random experiment
A procedure that results in an uncertain outcome.

Sample space
The set of all possible outcomes of a random experiment, denoted by S. The probability of the sample space
is P ( S ) = 1 .

Event
A subset or portion of the sample space. The probability of an event A is denoted by P ( A)

Certain event
An event that is sure to occur. The probability of a certain event is 1.

Impossible event
An event that has no chance of occurring. The probability of an impossible event is 0.

1
Empty set
A set that does not contain any of the outcomes of the sample space is called the empty set, denoted by  =  
and its probability is P ( ) = 0 , i.e., the impossible event.

Elementary event
An event that has only one possible outcome.

Venn diagram
Venn diagrams are used to visually represent events and how they co-exist in the sample space. The sample
space is represented by a rectangular box and circles are typically used to describe the events. The sizes of the
circles do not necessarily correspond to the sizes of the probabilities. Probability calculations can become
difficult using Venn diagrams. These diagrams are best used to visualise events rather than to calculate
probabilities of events. Other methods to represent probabilities are discussed later in the unit. The following
concepts/terminology are illustrated using Venn diagrams.

Complement of an event
The set of all outcomes of the sample space excluding the event itself.

Intersection of events
Intersection refers to where events occur together. The intersection of two events A and B is denoted by A  B
and describes all the outcomes that are common to both A and B, i.e., both A and B occurred.

2
Union of events
Union refers to the combination of events. The union of two events A and B is denoted by A  B and contains
the outcomes of event A, or the outcomes of event B, or the outcomes of both A and B, i.e., either A or B or
both events occurred.

Mutually exclusive events

Two events A and B are mutually exclusive if the occurrence of one event means that of the other event cannot
occur at the same time. The intersection of mutually exclusive events A and B yields the empty set, i.e.,
A  B =  and therefore P ( A  B ) = 0 .

Exhaustive events
Events are exhaustive if they fill up the sample space, i.e., the union of events is the sample space.

3
Partitioning
Events that are mutually exclusive and exhaustive form a partitioning of the sample space.

4.3. Type Of Probability

Objective Probability
The probability of an event is referred to as an objective probability if it can be calculated from experiments
or through empirical observation. The objective probability of event A is defined as:
number of outcomes in A r
P ( A) = =
total number of outcomes in S n

For example, an experiment consisting of a single roll of a fair six-side die, i.e., each of the six outcomes are
equally likely and the probability of each outcome is known in advance, or a survey of Grade 12 learners
shows that 17% wants to study at UJ.

Objective probabilities are classified into three different types: a marginal probability is the probability that a
single event A occurs; a joint probability is the probability that both events A and B occur; a conditional
probability is the probability that event A occurs, given that event B already occurred.

Subjective Probability
Subjective probabilities differ from person to person and are based on a person’s past experience, judgment
or opinion, and are used in situations where objective probabilities cannot be determined. For example, the
probability that the traffic lights at Kingsway Road and University Road will be working today is perceived
differently depending on a person’s own experience.

4.4. Probability Rules

Probability concepts are expressed in mathematical terms as a set of probability rules. All the rules are
underpinned by the basic facts that 0  P ( A)  1 and that the sum of the probabilities of all events in the
sample space is equal to 1.

4
Complement Rule
The complement of an event A implies that A does not occur. That includes all the outcomes in the sample
space that do not belong to event A.
P ( A) + P ( A ) = 1
 P ( A) = 1 − P ( A )
 P ( A ) = 1 − P ( A)

Addition Rule
The addition rule expresses the probability of the union of two events in terms of marginal and joint
probabilities.
P ( A  B ) = P ( A) + P ( B ) − P ( A  B )

P ( A  B ) = P ( A) + P ( B ) , if and only if A and B are mutually exclusive

5
Statistical Independence
Two events are statistically independent if the occurrence of one does not change the probability of the next
one occurring. If events A and B are statistically independent, then the probability of the intersection between
A and B is equal to the product of the two marginal probabilities.
P ( A  B ) = P ( A)  P ( B )

Consider an experiment of flipping a fair coin once, followed by a single roll of a fair six-sided die. The sample
space is then:
S = {H1, H2, H3, H4, H5, H6, T1, T2, T3, T4, T5, T6}

Conditional Probability
If the occurrence of one event affects the occurrence of a second event, the events are dependent. For two
events A and B, if event B has occurred, the sample space is reduced to event B. Then the probability of A
given B is equal to the probability of both A and B, divided by the probability of B.
P ( A  B)
P ( A | B) =
P ( B)

Multiplication Rule
The intersection of two dependent events can be expressed in terms of a marginal and a conditional event.
P ( A  B)
Since P ( A | B ) =  P ( A  B) = P ( B) P ( A | B)
P ( B)
P ( B  A)
Since P ( B | A ) =  P ( A  B ) = P ( A) P ( B | A)
P ( A)

6
Bayes’ Theorem
Bayes’ Theorem was developed to revise probability calculations in light of new information and calculate
posterior probabilities. This theorem is a special application of conditional probabilities and makes use of the
law of total probability. Let A1 , A2 , , Ak be a collection of mutually exclusive and exhaustive events with

probabilities P ( Ai ) , with i = 1, 2, , k . Then for any other event B for which P ( B )  0 , the posterior

probability of Aj (for any j = 1, 2, , k ) given that B has occurred is:

P ( Aj  B ) P ( B | Aj ) P ( Aj )
P ( Aj | B ) = =
P ( B) k

P(B | A ) P( A )
i =1
i i

4.5. Probability Representation

Objective probabilities can be represented in different formats.

Venn diagrams
As discussed previously in the lecture notes, Venn diagrams are used to visualise relationships and to
understand how events co-exist in the sample space.

Notation
Probabilities can be represented through the use of notation. This is very useful when the given probability
information can be easily substituted into specific probability formulae.

1) Let P ( A) = 0.7 , P ( B ) = 0.4 and P ( A  B ) = 0.3 . Find P ( A  B ) :

P ( A  B ) = P ( A) + P ( B ) − P ( A  B )
= 0.7 + 0.4 − 0.3 = 0.8

2) Let events A and B be independent events, where P ( A) = 0.7 and P ( B ) = 0.4 . Find P ( A  B ) and

P ( A  B) .

P ( A  B ) = P ( A)  P ( B ) = 0.7  0.4 = 0.28

P ( A  B ) = P ( A )  P ( B ) = 0.3  0.4 = 0.12

3) A device consists of three independent components. The probabilities of the three components
functioning correctly are 0.96, 0.92 and 0.95, respectively. The device can only function properly if all
three components function correctly. What is the probability that the device will not function properly,
i.e., at least 1 of the 3 components does not function correctly?

7
 Let Ci = the event that component i works
Then P(at least 1 component does not work) = 1 – P(all 3 components work)
= 1 − P ( C1  C2  C3 ) = 1 − P ( C1 )  P ( C2 )  P ( C3 )
= 1 − ( 0.96  0.95  0.95) = 0.161

4) Let P ( A ) = 0.6 , P ( B ) = 0.4 and P ( A  B ) = 0.3 . Find P ( A | B ) and P ( B | A ) .

P ( A  B ) 0.3
P ( A | B) = = = 0.75
P ( B) 0.4

P ( B  A) 0.3
P ( B | A) = = = 0.5
P ( A) 0.6

5) Let P ( A ) = 0.6 , P ( B ) = 0.4 , P ( A | B ) = 0.75 and P ( B | A ) = 0.5 . Find P ( A  B ) .

P ( A  B ) = P ( A) P ( B | A ) = 0.6  0.5 = 0.3

P ( A  B ) = P ( B ) P ( A | B ) = 0.4  0.75 = 0.3

Contingency tables
A contingency table represents probabilities or empirical counts of events in the rows and columns of a table.

1) Let P ( A ) = 0.25 , P ( B ) = 0.3 and P ( A  B ) = 0.85 . Construct a contingency table.

The event A  B includes 3 of the 4 cells in the table, namely A  B , A  B and A  B . The
probability of the union of these 3 cells is 0.85. It is not possible to determine the probabilities of each
of the 3 cells directly from the union. Using the complement rule, we can see that
P ( A  B ) = 1 − P ( A  B ) = 1 − 0.85 = 0.15 . We can now complete the table.

A A Marginal
B 0.1 0.6 0.7
B 0.15 0.15 0.3
Marginal 0.25 0.75 1

8
 2) A sample of two hundred and forty students is classified according to sex, male or female, and whether
they play sport or did not play sport. The sample consists of one hundred and forty females. One hundred
and sixty-nine students did not play sport. Thirty-five students are male and played sport. Construct a
contingency table with these empirical counts.
Male (M) Female (F) Total
Play sport (P) 35 36 71
Do not play sport (D) 65 104 169
Total 100 140 240

• What is the probability that a student is either female or play sport?

140 + 71 − 36
P ( F  P) = = 0.729 using the addition rule
240
35 + 36 + 104
P ( F  P) = = 0.729 using the individual cells
240

• What is the probability that a student is male and do not play sport?
65
P ( M  D) = = 0.271
240

• What proportion students do not play sport?

169
P ( D) = = 0.704
240

• If a student is female, what is the probability that the student will play sport?

P ( P  F ) 36 240 36
P(P | F ) = = = = 0.257
P(F ) 140 140
240

• Amongst students who play sport, what proportion is female?

P ( F  P ) 36 240 36
P ( F | P) = = = = 0.507
P ( P) 71 71
240

Tree diagrams
If an experiment consists of several stages, a tree diagram is useful to visualise probabilities at the different
stages, where the possible outcomes are represented by the branches of the tree. The first level consists of the
marginal/unconditional probabilities and subsequent levels consist of the conditional probabilities.

9
From the multiplication rule, the products of the respective marginal and conditional probabilities yield the
intersection probabilities of the events. For two events A and B the tree diagram is structured as follows:

1) A retailer noted that 20% of the time, customers made small purchases (under R50). The rest of the time
the purchase amount was R50 or more. The retailer’s customers either pay with cash or with a credit
card. If a customer made a small purchase, the probability of paying with credit card is 0.1. For larger
purchases, the probability of paying with credit card is 0.7. Construct a tree diagram.

• What is the probability that a customer, who just left the retailer, made a small purchase and paid for
it with cash?
P (  R50  C ) = P (  R50 )  P ( C | R50 ) = 0.2  0.9 = 0.18

• What is the probability that a customer will make a cash payment at the retailer?
P ( C ) = P (  R50  C ) + P ( R50 + C )
= P (  R50 )  P ( C | R50 ) + P ( R50 + )  P ( C | R50 + )
= 0.2  0.9 + 0.8  0.3 = 0.18 + 0.24 = 0.42

10
• A customer just made a cash payment. What is the probability that the amount paid was R50 or more?
P ( R50 + C )
P ( R50+ | C ) =
P (C )
0.8  0.3
=
0.2  0.9 + 0.8  0.3
0.24
=
0.42
= 0.571

2) A batch of 15 parts contains 4 defective items. Two parts are chosen one at a time without replacement.
For this example, we can use a tree diagram to represent the probabilities. The marginal portion of the
tree lists the probabilities associated with the first part selected. The conditional portion of the tree lists
the probabilities associated with the second part selected, given the first part selected.

• What is the probability that both parts selected are defective?

4 3
P ( D1  D2 ) =  = 0.057
15 14

• What is the probability that exactly one of the parts selected is defective?

P ( D1  D2 ) + P ( D1  D2 ) =
4 11 11 4
 +  = 0.419
15 14 15 14

• What is the probability that at least one of the parts selected is defective?
P ( at least 1 defective ) = 1 − P ( no defectives )
= 1 − P ( D1  D2 )
11 10
= 1−  = 0.476
15 14

11
Exercise 4.1

1) Let P ( A) = 0.7 , P ( B ) = 0.4 and P ( A  B ) = 0.2

Construct a contingency table to represent all the probabilities in the sample space

• P ( A  B) =

• (
P A B = )

• P( A  B ) =

• P ( A | B) =

• P ( B | A) =

2) Let P ( B ) = 0.4 and P ( A | B ) = 0.5

P ( A  B) =

12
3) A courier company has two delivery vehicles. The probability that any vehicle will be available when
needed is 0.85. The availability of one vehicle is independent of the other. An order is placed.

• What is the probability that both vehicles will be available to make the delivery?

• What is the probability that neither vehicle will be available to make the delivery?

4) In a manufacturing plant machines A and B are used equally often to manufacture a mechanical part.
The probability that machine A produces a defective part is 0.1, while machine B has a 40% chance of
producing a defective part. Construct a tree diagram represent the given probabilities.

• What is the probability that a randomly selected part was manufactured by machine A and it was good?

• What proportion of parts is not defective?

13
 • A randomly selected part is tested and found to be defective. What is the probability that it was
produced by machine B?

4.6. Counting Rules

When the various outcomes of an experiment are equally likely (i.e., the same probability is assigned to each
elementary event), the task of computing probabilities reduces to counting. In particular, if N is the number of
outcomes in a sample space and N(A) is the number of outcomes contained in an event A, then:
N ( A)
P ( A) =
N

If a list of the outcomes is available or easy to construct and N is small, then the numerator and denominator
of the equation above can be obtained without the benefit of any general counting principles. However, for
many experiments N is large. In such cases counting methods are used to calculate the number of outcomes in
an event and in the sample space.

The product rule for ordered pairs

Consider a set consisting of ordered pairs of elements and we wish to count the number of such pairs (with
replacement). If the first element of an ordered pair can be selected in n1 ways, and for each of these n1 ways
the second element of the pair can be selected in n2 ways, then the number of pairs is n1 × n2. This can be
extended to the general product rule k objects.

For a set consisting of an ordered collection of k elements (k-tuples): there are n1 possible choices for the first
element, for each choice of the first element there are n2 possible choices of the second element, … , for each
possible choice of the first k – 1 elements there are nk choices of the kth element. Then there are
n1 × n2 × … × nk possible k-tuples.

14
Example
A fair coin is tossed once, followed by one roll of a fair six-sided die. How many possible outcomes are there
in this experiment, i.e., in the sample space?
• There are 2 possible outcomes on the coin
• There are 6 possible outcomes on the die
• Therefore, 6  2 = 12

Factorial (not in textbook)

Consider a group of n distinct items / individuals / objects. The total number of different ways that all n items
can be arranged is “n-factorial”, expressed mathematically as:
n ! = ( n )  ( n − 1)  ( n − 2 )  ( n − 3)  3  2 1 .

Example
How many different arrangements are there of the letters A, B and C?
• An arrangement of all 3 letters implies that order is important
• There are 3 choices for the first position
• There are 2 choices for the second position
• There is 1 choice for the third position
• Therefore, 3! = 3  2 1 = 6

Permutations
Consider a fixed set consisting of n distinct elements. A k-tuple is formed by selecting successively from this
set without replacement so that an element can appear in at most one of the k positions. Order is important.
Any ordered sequence of k objects taken from a set of n distinct objects is called a permutation of size k of

the objects. The number of permutations of size k that can be constructed from n objects is denoted by Pk ,n ,
read “n permutation k”, where:
n!
Pk ,n =
( n − k )!

Example
How many different three letter “words” can be created from our alphabet such that each letter is unique?
• Select 3 distinct letters out of the set of 26 letters
• Order is important, e.g., the word “CAT” is different from the work “ACT” – same letters but a
reordering of the letters yield a different word / outcome
• Therefore, P3,26 = 15600

15
Combinations
Often the objective is to count the number of unordered subsets, without replacement, of size k that can be
formed from a set consisting of n distinct objects. Given a set of n distinct objects, any unordered subset of
size k of the objects is called a combination. The number of combinations of size k that can be formed from n
n
distinct objects will be denoted by Ck ,n or   , read “n choose k”, where:
k 
n n!
 =
 k  k !( n − k )!

Example
How many subsets of two elements can be taken from the set {1, 2, 3, 4}?
• Select 2 numbers out of the set of 4 numbers
• Order is NOT important, e.g., the set {1, 2} is the same as the set {2, 1} – a reordering of numbers in
the subset does not yield a different word / outcome
 4
• Therefore,   = 6
 2

Exercise 4.2
1) Student numbers at a certain university consist of 7 digits followed by a single letter. The first two digits
are the year of first registration, followed by one digit, which is either a 0 or a 1, followed by four
randomly selected digits and then the randomly selected letter. How many student numbers are possible
between 2016 and 2019 inclusive?

2) In how many ways can ten people be assigned to ten different jobs?

3) There are ten workers and six jobsites, and the manager must assign one worker to each jobsite. How
many possible allocations are there?

4) Twelve students applied to be tutors for a statistics course, but the lecturer only requires five. How
many different ways are there to randomly select five of the twelve students?

16