Introduction to Probability

Applied Statistics and Probability

ASP611S
Lecture Notes
Learning outcomes

At the end of this unit students should be able to:

 Understand and use the terminology of probability.
 Determine whether two events are mutually exclusive, collectively
exhaustive and whether two events are independent.
 Calculate probabilities using the Addition Rules and Multiplication Rules.
 Calculate conditional probability
 Construct and interpret Tree Diagrams.
 Apply the law of total probability and Bayes Theorem to answer problems
2.1 SET THEORY
 KEYWORDS:

Set,
subset,
intersection set,
union set,
complement set,
empty set,
universal set,
Venn diagrams
 2.1.1 SET
A set, always denoted by a capital letter.

Is a collection of distinguishable objects or entities that can be

all listed or there exists a rule that determines whether or not a
given object belongs to the set.
Example..
 A set of positive integers that are greater than zero but less than 10

• Because 1 belongs to Set A, we write it as

• Since, 0 does not belong to Set A, we write it as .
 2.1.2 SUBSET
 Set B is a subset of C if every element of B also belongs to C.
 That is B is contained or included in C. B is a subset of C and C is superset of
B.
 This can be written mathematically as .
 If every element of B does not belong to C, we say B is not a subset of C
and we write Example:

Therefore,
2.1.3 IDENTICAL SET

 Two sets are identical if they contain the same elements, and
then we write

Example..

Therefore, means A and B are identical set.

 2.1.4 EMPTY SET

 An empty set is a set with no element.

 The empty set is denoted by , this simply means that

 Take note that the empty set is a subset of any set.

2.1.5 SIZE OF A SET

 is the number of elements in the set A, it is also called size of
A.
E.g Let
Then,
2.1.6 INTERSECTION SET ( )

The intersection of two or more sets is the set that

contains precisely those elements which belong to
both or all sets.
Example..
Let and
the intersection of set L and M is
Venn diagram representation of ( )
2.1.7 Union Set

The union of two sets A and B (or more) is the set that
contains the elements that belong to A or B or both.
Example:

 Let and
The union set of set A and B is
Venn Diagram representation of Union

The shaded are shows the set 𝐴 ∪ 𝐵.

2.1.8 Universal set , the sample space

 Universal set is a set that contains all objects under consideration.

 It is the totality of all the sets under consideration.

Example..

If, and then the totality

of these two sets is
2.1.9 Complement of a set A
 The complement of a set A, denoted by , is the set of all
those elements of that are not in A.

Example:
Consider Ω = {1,2,3,4,5,6,7,8,9,10,11,12,13,14} and A = 2,3,4,6 .
𝐴 = {1,5,7,8,9,10,11,12,13,14}
 2.2 Events
Keywords:

Random experiment,
 sample space,
trial,
elementary event,
certain event,
impossible event,
independent events,
mutually exclusive events.
 2.2.1 Random Experiments
 A random experiment is a procedure whose outcome in a
particular performance ( i.e a trial ) cannot be predetermined.
 But, we must be able to list the set of all possible outcomes of
the experiment.
Example:
• Tossing a coin
• Rolling a die,
• Recording the age of every second student met in the corridor for one
Monday morning.
2.2.2 Sample space (S)
The set of all possible outcomes of a random experiment is called the sample
space.

2.2.3 Trial
 Each repetition of the procedure of a random experiment is called a trial.

 A trial gives rise to and only one of the possible outcomes.

Experiment, outcome & Sample Space
 2.2.4 Event
 An event is a subset of a sample space.

Example:

Let (the set of all possible outcomes of throwing a die).

Define the following events:

A: “The face turns up it is an even number”

Set notation is A={2,4,6}

B: “The face that turns up is an odd number”

Set notation is B={1,3,5}

C: “The face that turns up is a number greater 6”

We know that there is no die with a face labeled with a number greater than six. It becomes obvious
that it is impossible to observe a face labeled with number greater than six. In other words, such event
will not take place. This event is called an impossible event.
 2.2.5 Impossible event and Certain event
 Recall that and are subsets of a sample space
 Therefore and are events.

 is called impossible event and is called a certain event.

Example:

Consider a “fake” coin (both faces are tails). The set of possible events consists of
one element only: tail. We are quite sure that every time we toss the coin, the
outcome turns to be a tail.

Thus, is certain event.

2.2.6 Elementary event

Elementary event: an event with exactly one element in it is called
elementary event.
2.2.2.7 Mutually exclusive, pairwise mutually exclusive and
collectively exhaustive
 Two events A and B are mutually exclusive if .
“Getting an odd number” and an “even number” after rolling a die are
two mutually exclusive events.
 If a family of sets (events) A1, A2,A3,…,An are such that any pair of
them is mutually exclusive, e. g 𝒊 𝒋 and if 𝟏 𝟐
𝒏 i.e. the union of the sets (events) exhausts the sample space,
then the family of the sets (events) 𝟏 𝟐 𝒏 are said to be pairwise
mutually exclusive and collectively exhaustive.
Collectively exhaustive

 Events are said to be collectively exhaustive when the union of

all passible event is equal to sample space.
 Venn Diagram below displays collectively exhaustive events A
and B .
 A and B occupy the entire space.
3. PROBABILITY
3.1 Definition of a probability:
Suppose that S is the sample space for a random experiment.
, we define probability of A, denoted by or , to
be a real number with the following properties:

 , and
 If , then
Axioms of Probability
3.1 Some useful theorems

Theorem 1:
Let Then

Theorem 2:
Additional rule:
For any arbitrary events A and B,
Example
If the probabilities are respectively 0.86, 0.35 and 0.29 that a family selected at
random from a sample survey in a large metropolitan area will own a colour
television set, a black and white set or both kinds of sets.
What is the probability that such a family will own either kind of set?
A = {family owns colour television set}
B = {family owns black and white television set}
AB= {family owns both kinds of sets}
We want to find P(AB) :
Using Addition Rule P(AB) = P(A) + P(B) – P(AB)

P(A) = 0.86 P(B) = 0.35 P(AB) = 0.29

P(AB) = 0.86 + 0.35 - 0.29 = 0.92
Therefore P(family owns either kinds of sets) = 0.92
Addition Rule Example 2

 If the probabilities are respectively 0.23, 0.24 and 0.38 that a car
stopped at a road block will have faulty brakes, badly worn tyres,
faulty brakes or badly worn tyres.

 What is the probability that such a car will have both faulty brakes
and badly worn tyres?
Solution

 B = {the event that car will have faulty brakes}

 T = {the event that car will have badly worn tyres}
 B  T = {the event that car will have badly worn tyres or faulty
brakes}
 We are want to find P(BT):
 Using Addition Rule P(BT) = P(B) + P(T) – P(BT)
 P(B) = 0.23 P(T) = 0.24 P(BT) = 0.38
 P(BT) = P(B) + P(T) - P(BT)
= 0.23 +0.24 – 0.38
= 0.09
 P(car will have badly worn tyres and faulty brakes) = 0.09
Theorem 3:

 If 𝐵 ⊂ 𝐴, then 𝑃 𝐵 < 𝑃 𝐴 , ∀𝐴, 𝐵 ⊂ 𝑆


 Theorem 4:
 If 𝐴 , 𝐴 , … , 𝐴 are pairwise mutually exclusive, i.e. 𝐴 ∩ 𝐴 = ∅ if 𝑖 ≠ 𝑗, then
𝑃 𝐴 ∪ 𝐴 ∪ ⋯ ∪ 𝐴 = 𝑃 𝐴 + 𝑃 𝐴 + ⋯ + 𝑃(𝐴 ) (Additional rule)
 This can be re-written as
 𝑷(⋃𝒏𝒊 𝟏 𝑨𝒊 ) = ∑𝒏𝒊 𝟏 𝑷(𝑨𝒊 )
Theorem 6:
If events A and B are independent, then

3.2 Equally probable elementary events

• If there are elementary events in and each one has the same
probability of occurring, then the probability of each other and every
event must be

• Let consist of the union of equiprobable elementary events. Then the

probability of occurrence of is

( )
= ( )
Class Example:

100 people bought tickets in a charity raffle. 60 of them bought the tickets
because they supported the charity. 75 bought the tickets because they
liked the prize. No one who neither supported the charity nor liked the prize
bought a ticket.

• What is the probability that the prize-winning ticket was bought by

someone who like the prize ?
𝑃 𝑙𝑖𝑘𝑒𝑑 𝑡ℎ𝑒 𝑝𝑟𝑖𝑧𝑒 = = 0.75

• What is the probability that the prize-winning ticket was bought by

someone who did not support the charity?

40
𝑃 𝑁𝑜𝑡 𝑠𝑢𝑝𝑝𝑜𝑟𝑡𝑖𝑛𝑔 𝑡ℎ𝑒 𝑐ℎ𝑎𝑟𝑖𝑡𝑦 = = 0.40
100

• What is the probability that the prize-winning ticket was bought by

someone who both supported charity and liked the prize?
Solution continue

 𝑙𝑒𝑡 𝑥 𝑏𝑒 𝑡ℎ𝑒 𝑝𝑟𝑜𝑏𝑎𝑏𝑖𝑙𝑖𝑡𝑦 𝑡ℎ𝑎𝑡 𝑡ℎ𝑒 𝑝𝑟𝑖𝑧𝑒 −

𝑤𝑖𝑛𝑛𝑖𝑛𝑔 𝑡𝑖𝑐𝑘𝑒𝑡 𝑤𝑎𝑠 𝑏𝑜𝑢𝑔ℎ𝑡 𝑏𝑦 𝑠𝑜𝑚𝑒𝑜𝑛𝑒 𝑤ℎ𝑜 𝑠𝑢𝑝𝑝𝑜𝑟𝑡𝑒𝑑 𝑏𝑜𝑡ℎ 𝑐ℎ𝑎𝑟𝑖𝑡𝑦 𝑎𝑛𝑑 𝑙𝑖𝑘𝑒𝑑 𝑡ℎ𝑒 𝑝𝑟𝑖𝑧𝑒

 0.75 + 0.6 − 𝑥 = 1
 −𝑥 = 1 − 1.35
 −𝑥 = −0.35
 𝑥 = 0.35
Joint probability

 The probability of the intersection of two events A and B is called

their joint probability

P (A  B )
Example
If a delivery is chosen at random what is the
probability that the delivery is early.
(this is an example of marginal probability)

P (e a r ly )  P (E a r ly  Jo n e s )  P (E a r ly  S m ith )  P (E a r ly  R o b in s o n )
20 10 0
P (E a r ly )=  
300 300 300
30

300
 Lesson 2
3.4 Conditional Probabilities

• You may register for Statistics 2B….

On condition that…
Provided that…
Given that…
If…
…You passed Statistic 2A
• The conditional probabilities is based on a part of a sample space, not
the whole sample space.

Let A and B be two events in a sample space S, the conditional

probability of event B given that event A occurred, denoted by
is:

𝑷(𝑨∩𝑩)
𝑷(𝑨)
, provided
Example 1

 Consider a bag containing 12 balls, of which 8 are green and 4 are

white. In an experiment 2 balls are drawn in succession, white
followed by green.
 Let A = {event of drawing a white ball}
 Let B = {event of drawing a green ball}
 Find P(A) , P(B)
we first need to know if the draw is
made:
With replacement Without replacement
 The probability of drawing a  Or without replacement (i.e.
specific coloured ball depends on drawing a ball, examining the
whether the balls are drawn with colour and keeping the ball out of
replacement (i.e. drawing a ball the bag before the next draw is
from the bag, examining the made).
colour and putting it back into the
bag before the next draw is
made)
Suppose the draws are made without
replacement
 The probability of drawing a white ball in the first draw

4 1
P ( A)  
12 3
 This ball is then put aside.

 The probability of drawing a green ball in the second draw

8
P ( B / A) 
11
 This is because we now have 11 balls altogether in the bag out of which 8
are green.
Example 2

 You, woman with medical background, are one of 198 applicants for an
M.B.A programme of whom 81will be selected. You hear, along the
grapevine, on good authority that there were 70 applicants, of whom 38
were selected.
 Assess your probability of being accepted before and after you receive
grapevine information.
Solutions
Exercise 1

 Suppose all 100 employees of a company were asked whether they

are in favour of or against paying high salaries for C.E.Os

a) Find the probability that an employee selected at random is in

favour of paying high salary to the C.E.Os given that he is male.
 Exercise cont…

 b) Find the probability that an employee selected at random is in favour of

paying high salary to the C.E.Os
19
𝑃 𝑖𝑛 𝑓𝑎𝑣𝑜𝑟 = = 0.19
100
c) Find the probability that an employee
selected at random is a female and against
paying high salary to the C.E.Os

Hint: the key word and means intersection

𝑃 𝑓𝑒𝑚𝑎𝑙𝑒 ∩ 𝑎𝑔𝑎𝑖𝑛𝑠𝑡 = = 0.36

Class exercise:

A manufacturer of airplane parts knows from the past experience

that the probability is 0.8 that an order will be ready for shipment on
time, and it is 0.72 that an order will be ready for shipment on time
and also be delivered on time.

What is the probability such that an order be delivered on time given

that it was ready for shipment?
Solution
3.5 Independent Events
• Intuitively, independent events have no effect on upon each other. Two
events A and B are independent if the occurrence of event A has
nothing to do with occurrence of event B.

• This implies that the conditional probabilities of B given A is the same as

the unconditional probability of B.
𝑷(𝑨∩𝑩)
i.e. This can be re written as follows 𝑷(𝑨)

Then,

Definition:
Two events A and B are independent if and only if
 Example 4:
Suppose a study of speeding violations and drivers who use car phones
produced the following fictional data:

Speeding No speeding violation Total

violation in the in the last year (𝐵)
last year (B)
Car phone user (A) 25 280 305
Not car phone user (𝑨) 45 405 450

Total 70 685 755

Calculate the probability using the table:

(a) P (Person had no violation in the last year and was a car phone
user)
(b) P (Person is car phone user or person had no violation in the last
year)
(c) P (Person is car phone user given that person had a violation in
the last year)
(d) Are the events car phone user and Speeding violations
Solutions
d) Let A be an event for car phone user, B be an event of Speeding violations
𝑟𝑒𝑐𝑎𝑙𝑙 𝑡ℎ𝑎𝑡 𝑖𝑓 𝑃 𝐴 ∩ 𝐵 = 𝑃 𝐴 𝑃 𝐵 , 𝑡ℎ𝑒 𝑒𝑣𝑒𝑛𝑡𝑠 𝐴 𝑎𝑛𝑑 𝐵 𝑎𝑟𝑒 𝑖𝑛𝑑𝑒𝑝𝑒𝑛𝑑𝑒𝑛𝑡
𝑃 𝐴∩𝐵 = = 0.0331

𝑃 𝐴 = = 0.4040

𝑃 𝐵 = = 0.0927

 𝑃 𝐴 𝑃 𝐵 = 0.4040 × 0.0927 = 0.0375

 Since 𝑃 𝐴 ∩ 𝐵 ≠ 𝑃 𝐴 𝑃 𝐵 , hence the two events are not independent.
Tree diagram

Recall the exc 1 above Tree diagram

Construct the tree diagram for the
data
Theorem on Total probability
Theorem cont.…

 The events A1 , A2 , and A3 constitute a partition of the sample space, S

 They are mutually exclusive and collectively exhaustive.

Theorem of Total probability
(Partition Theorem)
Theorem of Total probability
(Partition Theorem)
Example 1

 Suppose that we are concerned with the completion of a highway

construction job, which may be delayed because of a strike.
Suppose, furthermore, that the probabilities are 0.60 that there will
be a strike, 0.85 that the job will be completed on time if there is no
strike, and 0.35 that the job will be completed on time if there is a
strike.

 What is the probability that the job will be completed on time?

Let Ai be the partitioned sample space, S
such that P(A1) + P(A2) = 1

 Then A1 = {the event of a strike}

 and A2 = {the event that there will be no strike}

 Let B = {the event that the job will be completed on time}

Solution

Applying the Partition

Summary theorem
 P(A1) = 0.60
 P(A2) = 0.40

B = {the job will be completed on time}

 Then we are given:

 P(B/A1) = 0.35
 P(B/A2) = 0.85
Conclusion

 Therefore

P(job will be completed on time)=0.55

 Example 2

1. A family has two dogs (Rex and Rover) and a cat called Garfield. None of them is fond of
the postman. If they are outside, the probability that Rex, Rover and Garfield will attack the
postman are 30%, 40% and 15%, respectively. Only one is outside at time, with probabilities
10%, 20% and 70%, respectively.
a) What is the probability that the postman will be attacked?
b) What is the probability that Garfield was the culprit?
Solution:
a) Let us define our events: Let 𝐴 = {𝑤𝑖𝑙𝑙 𝑎𝑡𝑡𝑎𝑐𝑘 𝑡ℎ𝑒 𝑝𝑜𝑠𝑡𝑚𝑎𝑛}
Let B = {𝑂𝑛𝑙𝑦 𝑜𝑛𝑒 𝑖𝑠 𝑜𝑢𝑡𝑠𝑖𝑑𝑒 𝑎𝑡 𝑡𝑖𝑚𝑒}
Since, we have defined our events 𝐴 and 𝐵. Now, which one is partitioned…Is it 𝐴 or 𝐵?

Recall, 𝟏 𝟐 𝒌 and

Then, Event B is partitioned into 𝟏 𝟐 𝟑 : 𝐵 = {𝑅𝑒𝑥 𝑖𝑠 𝑜𝑢𝑡𝑠𝑖𝑑𝑒 𝑎𝑡 𝑡𝑖𝑚𝑒}

𝐵 = {𝑅𝑜𝑣𝑒𝑟 𝑖𝑠 𝑜𝑢𝑡𝑠𝑖𝑑𝑒 𝑎𝑡 𝑡𝑖𝑚𝑒}
𝐵 = {𝐺𝑎𝑟𝑓𝑖𝑒𝑙𝑑 𝑖𝑠 𝑜𝑢𝑡𝑠𝑖𝑑𝑒 𝑎𝑡 𝑡𝑖𝑚𝑒}
 Solution continues..
𝑃(𝐵 ) = 10% = 0.1
𝑃 𝐵 +𝑃 𝐵 +𝑃 𝐵 =1
𝑃(𝐵 ) = 20% = 0.2 0.1+0.2+0.7=1 (proved that B is partitioned)
𝑃(𝐵 ) = 70% = 0.7
Then, A is given by conditional: 𝑃 𝐴 𝐵 = 30% = 0.3
𝑃 𝐴𝐵 = 40% = 0.4

𝑃 𝐴 =? 𝑃 𝐴𝐵 = 15% = 0.15

𝑃(𝐵 ) 𝑃 𝐴𝐵 𝑃(𝐵 )𝑃 𝐴 𝐵
𝑃(𝐵 ) 0.1 0.3 𝟎. 𝟏 × 𝟎. 𝟑 = 𝟎. 𝟎𝟑 𝑃(𝐵 )𝑃 𝐴 𝐵
𝑃(𝐵 ) 0.2 0.4 𝟎. 𝟐 × 𝟎. 𝟒 = 𝟎. 𝟎𝟖 𝑃(𝐵 )𝑃 𝐴 𝐵
𝑃(𝐵 ) 0.7 0.15 𝟎. 𝟕.× 𝟎. 𝟏𝟓 = 𝟎. 𝟏𝟎𝟓 𝑃(𝐵 )𝑃 𝐴 𝐵
This the probability that the 𝑃 𝐴 = 0.215
postman was attacked
Bayes’ Theorem

 If the events A1, A2, A3,......., Ak constitute a partition of a sample space, S,

and P(Ai)0 for i=1, 2, 3,...., k then for any event B also
 Solution continues..(b)

Since, we want to find the probability “If the postman is attacked, what is the
probability that Garfield was the culprit?” ( We will make use of Bayes Theorem)
𝑃 𝐵 𝐴 =?

𝑃 𝐵 . 𝑃(𝐴|𝐵 )
𝑃 𝐵 𝐴 =
𝑃(𝐴)

Therefore, If the postman is attacked the probability that Garfield was the
culprit is 0.4884 or 48.84%
Exercise
A juice manufacturer has three machines, 1,2 and 3 for
manufacturing fruit juice, it is known that the machines
manufacture 25%, 35% and 40% of the total output respectively of
the outputs from machines 6%, 3% and 2% respectively are known
to be badly packed. If the packet of juice is chosen at random and
found to be badly packed, what is the probability that the packet
came from
a) Machine 1
b) Machine 2
c) Machine 3