AI121 – Discrete Mathematics

Introduction To Discrete Probability

Dr. Adeem Aslam

Assistant Professor

Department of Electrical Engineering

University of Engineering and Technology Lahore

Introduction To Discrete Probability (AI121 - Discrete Mathematics, Spring 2025) 1

Outline
• Introduction to Probability
• Axiomatic Definitions
• Conditional Probability
• Total Probability and Bayes’ Theorems
• Independence
• Random Variable
• Cumulative Distribution Function (CDF)
• Probability Density/Mass Function (PDF/PMF)
• Uniform Distribution, Bernoulli Distribution
• Combined Experiments
• Binomial and Geometric Distributions
Introduction To Discrete Probability (AI121 - Discrete Mathematics, Spring 2025) 2
Introduction To Probability
• Although, the theory of probability was originally invented to study gambling, it now plays an essential role in
a wide variety of disciplines of science and engineering.
The purpose of the
• For example, probability theory is extensively applied in: theory of probability is
to describe and predict
➢ the study of genetics, such averages in terms of
probabilities of events.
➢ computer science for study of algorithms,
➢ wireless communication,
➢ signal processing (there is a separate sub-discipline by the name of statistical signal processing),
➢ quantum mechanics (the position of an electron is given by probabilistic wavefunction),
to name a few.

• The theory of probability is concerned with the averages of mass phenomenon occurring either sequentially
or simultaneously, electron emission, arrival of telephone calls, birth and death, system failure etc.

• It has been observed that certain averages in these and other fields approach a constant value as the number
of observations increases, and this value remains the same if the averages are evaluated over any
subsequence specified before the experiment is performed.
➢ For instance, the percentage occurrence of ``heads" approaches 0.5 or some other constant, and the same average is
observed if, let's say, every fourth toss is considered.
Introduction To Discrete Probability (AI121 - Discrete Mathematics, Spring 2025) 3
Introduction To Probability
Relative Frequency Interpretation
• The probability of an event is a number , assigned to the event , and can be interpreted as follows:
➢ If the experiment is performed times and the event occurs times, then with a high degree of
certainty, the relative frequency, , of occurrence of is close to , i.e.,

provided is sufficiently large.

• The lack of precision in the above definition, due to the phrases highlighted, cannot be avoided, which is why, the theory
of probability, like any physical theory, is related to the physical phenomenon in inexact terms.

• In applying the theory of probability to real-world problems, following steps must be clearly distinguished:
➢ Determination of the probabilities of certain events by an inexact process.
❖ This process could be based on the relationship between probability and observation, as in (1).
❖ Or it could be based on some ``reasoning" making use of certain symmetries, i.e.,
According to the
Classical Definition of Probability classical definition, the
probability of an event
• If there are number of outcomes favorable to the event , out of a total ``equally likely"
is determined apriori,
i.e., without actual
outcomes, then experimentation.

Introduction To Discrete Probability (AI121 - Discrete Mathematics, Spring 2025) 4

Introduction To Probability
• For example, if a loaded die is rolled a times and the event shows times, then

• If the die is fair, then, because of its symmetry, .

• The lack of precision in the above definition, due to the phrases highlighted, cannot be avoided, which is why, the theory
of probability, like any physical theory, is related to the physical phenomenon in inexact terms.

• In applying the theory of probability to real-world problems, following steps must be clearly distinguished:
i. Determination of the probabilities of certain events by an inexact process.
❖ This process could be based on the relationship between probability and observation, as in (1).
❖ Or it could be based on some ``reasoning" making use of certain symmetries, i.e.,
According to the
Classical Definition of Probability classical definition, the
probability of an event
• If there are number of outcomes favorable to the event , out of a total ``equally likely"
is determined apriori,
i.e., without actual
outcomes, then experimentation.

Introduction To Discrete Probability (AI121 - Discrete Mathematics, Spring 2025) 5

Introduction To Probability
ii. Assuming the probabilities satisfy certain axioms/postulates, and by deductive reasoning,
determination, from probabilities of certain events , of probabilities of other events.

• For example, in the roll of a fair die experiment, we deduce that

because probability of each face is ( ) and axioms tell us to add the probabilities.

iii. Making a physical prediction based on the numbers so obtained.

• For example, if a fair die is rolled times, we predict that will show about times.

__________________________________________________________________________________________________________________________________________________________
The act of drawing specific conclusions from general information, as opposed to inductive reasoning, in which conclusions are drawn by going from specific observations to general principles.
Application of (1) in reverse.

Introduction To Discrete Probability (AI121 - Discrete Mathematics, Spring 2025) 6

Axiomatic Definitions
• The modern framework of applied probability is based on the principles of set theory!

Probability Space (Sample Space)

• Following the terminology used in set theory, the probability space is defined as follows:
i. Space , sometimes also denoted by , is called the certain event that occurs in every trial of the
experiment.
ii. Elements of are the experimental outcomes. A trial is an
iii. Subsets of are called events. instance of an
experiment.
iv. Empty set, denoted by or , is an impossible event.
v. Event , consisting of a single element , is called an elementary event.

• Furthermore:
➢ Union of two events and , denoted by , is the event that occurs when or or both occur.
➢ Intersection of two events and , denoted by , is the event that occurs when both and occur.
Axioms of Probability Two events are called
• The modern framework of probability is defined by the following axioms: mutually exclusive if
occurrence of one of
i. , them excludes the
occurrence of the
ii. , other.
iii. , if and are mutually exclusive.
Introduction To Discrete Probability (AI121 - Discrete Mathematics, Spring 2025) 7
Axiomatic Definitions – Borel Fields
• Instead of considering all possible subsets of the probability space as events, we consider only those that
form a Borel field.
The main reason for not
Field considering all subsets is that
it becomes impossible to
• A field is a non-empty class of sets such that: assign probabilities to all
subsets satisfying the axioms
when the experiment has
infinitely many outcomes.

• From above, it follows that:

i.
ii.
iii. sets that can be written as union and intersection of finitely many sets in are also in .
This is not
Borel Field necessarily true
• If the union and intersection of an infinite sequence of sets in F belongs to F , then F is for a sequence of
infinitely many
a Borel Field.
• The class of all subsets of the set is a Borel field.
_______________________________________________________________________________________________________________________________________________________________________________________________________
sets.

As an example of an experiment with infinitely many outcomes, consider the roll of a die that lands on a point on the table which is specified by the coordinates. Then, the outcome is not only the face value but also the infinitely many
possible ordered pairs of the form .

Introduction To Discrete Probability (AI121 - Discrete Mathematics, Spring 2025) 8

Axiomatic Definitions – Events and Experiment
• Events are certain subsets of that form a Borel field.
➢ Defining events like this enables us to assign probabilities not only to finite unions and intersections of
events but also to their limits.
Experiment
• The following make up what is called an experiment:
i. The probability of all experimental outcomes.
ii. The Borel field of all events of .
iii. The probabilities of these events.

• As an example, consider rolling a fair die.

• Then, the experiment consists of:
i. The probability space

ii. The Borel field of all events of , i.e.,

iii. The probabilities of these events, i.e., , and probabilities of other events given by
the axiom III and its generalization and extension, to be given later.
Introduction To Discrete Probability (AI121 - Discrete Mathematics, Spring 2025) 9
Probabilities of Events
• Repeated application of axiom III allows us to write

if the events , , are all (pairwise) mutually exclusive, because that renders the events and as
mutually exclusive, i.e.,

• Extending the above for a finite sequence of mutually exclusive events, we get the axiom IIIa as
• Generalizing (4), for an infinite sequence of mutually
exclusive events,

• Now, consider the set to be a partition of the product space , i.e.,

• Specializing (5), for an event ,
partitioned by , we note that
then,

Introduction To Discrete Probability (AI121 - Discrete Mathematics, Spring 2025) 10

Probabilities of Events
• As an example, consider the roll of a fair die.
• Then,

• For an event , the probability of even showing is given by

• Similarly,

• Let us now consider that the die is loaded in such a way that appear twice as often as each of the other
face values, but the other face values are equally likely. Then,

• Now,
Introduction To Discrete Probability (AI121 - Discrete Mathematics, Spring 2025) 11
Probabilities of Events
• A trivial partition of the probability space is given by an event and its complement, i.e., give a probability
space and an event , we have

• If an event is a subset of , as shown, then:

Figure 1: Venn diagram representation for
subset of a set.

• Finally, for two events and , not necessarily mutually exclusive, we note that

Figure 2: Venn diagram representation for

two arbitrary sets.
Introduction To Discrete Probability (AI121 - Discrete Mathematics, Spring 2025) 12
Probabilistic Reasoning – The Monty Hall Three-Door Puzzle
• A common task in statistics is to determine which of two events is more likely, the analysis of which can
sometimes be quite tricky.

• As an illustration of probabilistic reasoning regarding such problems, let us discuss a famous problem
originating with the 1960s television game show “Let’s Make a Deal”, and named after its host, Monty Hall.

• The Monty Hall Three-Door Puzzle:

• Suppose that you are a game show contestant, where you have a chance to win a large prize, and are asked to
select one of three doors to open; the large prize, such as a sports car, is behind one of the three doors and
the other two doors contain, let’s say, goats.

• Once you select a door, the game show host, who knows what is behind each door, does the following:
➢ First, whether or not you selected the winning door, he opens one of the other two doors that he knows
is a losing door (selecting at random if both are losing doors).
➢ Then he asks you whether you would like to switch doors.

• Should you change doors or keep your original selection, or does it not matter?
Introduction To Discrete Probability (AI121 - Discrete Mathematics, Spring 2025) 13
Probabilistic Reasoning – The Monty Hall Three-Door Puzzle
• Probabilistic Reason for The Monty Hall Three-Door Puzzle:

• The probability of selecting the correct door (before the host opens a door and asks you whether you want to
switch the doors) is 1/3, because the three doors are equally likely to be the correct door.

• Now, when the game show host opens one of the other doors, he will always open the door that the car is
not behind.
➢ As a result, the probability that the initially selected door is correct does not change once the game show host opens
one of the other doors.
➢ This means that the probability that you selected incorrectly, which is the probability that the car is behind one of
the two doors you did not select, which equals 2/3, does not change after the game show host opens one of the
other doors.

• So, if you selected incorrectly, when the game show host opens a door to show you that the car is not behind
it, the car is behind the other door.
➢ In this case, you will always win if your initial choice was incorrect and you change doors.
❖ So, by changing doors, the probability you win is 2/3.

• In other words, you should always change doors when given the chance to do so by the game show host.
➢ This doubles the probability that you will win.
Introduction To Discrete Probability (AI121 - Discrete Mathematics, Spring 2025) 14
Probabilistic Reasoning – The Monty Hall Three-Door Puzzle

• A simple solution for The Monty Hall Three-Door Puzzle:

Result if Result after

Behind Door 1 Behind Door 2 Behind Door 3 staying with switching from
door 1 door 1
Goat Goat Car Wins goat Wins car
Goat Car Goat Wins goat Wins car
Car Goat Goat Wins car Wins goat

Figure 3: Assuming that the contestant chooses door 1, the contestant wins in two out of three equally likely scenarios upon
switching, and wins in only one out of three scenarios upon staying with the initial choice of door 1. Hence, the probability of
wining the car upon switching has doubled to 2/3. The brown colored cell indicates the door the game host show opens for the
contestant. If the initially chosen door has car behind it, the game show is at liberty to choose any of the other two doors,
because both contain goat.

Introduction To Discrete Probability (AI121 - Discrete Mathematics, Spring 2025) 15

Conditional Probability
• Oftentimes there are cases where one is interested in finding the likelihood of an event based on some partial
information.
➢ For instance, we may want to find out that:
❖ How likely is a person to have a disease given that the medical test was negative?
❖ How likely is it that the spot on a radar screen is an enemy aircraft given that it shows on the screen?
❖ How likely is the outcome to be ``two" if ``even" shows in the roll of a die?}

• The above queries are answered by defining a new law of probability, called conditional probability, defined
as

where the event $(A\,|\,M)$ is interpreted as the occurrence of $A$ given that $M$ has occurred.
➢ This new construct can be proved to be in harmony with the axioms of probability and hence, is a
probability of new kind:
i. Axiom I:

ii. Axiom II:

iii. Axiom III:

Introduction To Discrete Probability (AI121 - Discrete Mathematics, Spring 2025) 16
Conditional Probability
• As an example, consider again the roll of a fair die with the probability space

• Now, if the event occurs, then the probability of two showing on the face increases
from to the following:

• As another example, consider all the 16 4-bit strings, generated at random, to be is equally likely.
➢ Then, what is the probability that a 4-bit string contains at least two consecutive 0s, given that its first bit
is a 0 ?, assuming that 0 bits and 1 bits are equally likely.
• The experiment is given by:

• The events of interest, and the conditional probability of the event in question, are:

Introduction To Discrete Probability (AI121 - Discrete Mathematics, Spring 2025) 17

Total Probability and Bayes’ Theorems
• Consider the partition of the probability space into events, and an arbitrary event
$B, as shown.

• Then:

Figure 4: Partitioning of the probability space

and an arbitrary event.

which is called the total probability theorem, and the final expression has been obtained by using axiom III.

• Moreover,

which is called the Bayes’ theorem.

Introduction To Discrete Probability (AI121 - Discrete Mathematics, Spring 2025) 18
Total Probability and Bayes’ Theorems
• As an example, consider four boxes containing electronic components.
➢ Box 1 contains 2000 components of which 5% are defective, box 2 contains 500 components of which 40% are
defective, and boxes 3 and 4 contain 1000 components each of which 10% are defective.

• A single component is selected at random from a randomly selected box.

➢ What is the probability that it is defective ?
➢ If the selected component is defective, what is the probability that it came from box 2 ?

• Let us denote the events as follows:

• Because the boxes are selected randomly, we note that

• As a result,

and

Introduction To Discrete Probability (AI121 - Discrete Mathematics, Spring 2025) 19

Independence
• The events are called pairwise
• Two events and are called independent if independent if

• They are called mutually independent if (14)

holds.
• When two events are independent, the occurrence of one of the events gives no information about the
probability that the other event occurs.
➢ Hence, independence of two events and can further be expressed as

which can also be seen as a consequence of the definitions in (12) and (9).

Independence of More Than Two Events

• Suppose that the independence of events has been established for every .
• Then, the events are independent if any of them are independent, and

• If events and are independent, then so are and , and , and .

Introduction To Discrete Probability (AI121 - Discrete Mathematics, Spring 2025) 20
Random Variable
• Given an experiment, specified by:
➢ the probability space ,
➢ Borel field of subsets of , called events,
➢ probabilities assigned to these events,

a random variable is defined as a function , with

domain as the set and range as a set of real
numbers, i.e.,

Figure 1: Illustration of the action of a random variable.

whose values are numbers assigned to every
outcome of the experiment.
Equation and figure counters
• As an example, consider the roll of a die experiment, specified as: have been reset to 1.

• Then, a random variable can be defined as

______________________________________________________________________________________________________
or
Since the random variable is a function, one-to-one and many-to-one mappings are allowed but one-to-many mapping isn't.
Introduction To Discrete Probability (AI121 - Discrete Mathematics, Spring 2025) 21
Random Variable
• Events generated by random variables are of the
following form:
➢ , which represents the subset of the
probability space that consists of all those
outcomes such that .

Figure 2: Meaning of the events generated by a random variable.

Introduction To Discrete Probability (AI121 - Discrete Mathematics, Spring 2025) 22

Random Variable
• Events generated by random variables are of the
following form:
➢ , which represents the subset of
the probability space that consists of all those
outcomes such that .

Figure 3: Meaning of the events generated by a random variable.

Introduction To Discrete Probability (AI121 - Discrete Mathematics, Spring 2025) 23

Random Variable
• Events generated by random variables are of the
following form:
➢ , which represents the subset of the
probability space that consists of all those
outcomes such that .

Figure 4: Meaning of the events generated by a random variable.

Introduction To Discrete Probability (AI121 - Discrete Mathematics, Spring 2025) 24

Random Variable
• Events generated by random variables are of the
following form:
➢ , which represents the subset of the
probability space that consists of all those
outcomes such that .

• As an example, consider the roll of a die experiment,

specified as:

Figure 5: Meaning of the events generated by a random variable.

• Then,
i. represents , Random Variable
ii. is an empty set, A random variable is defined as a process of assigning a number ,
iii. represents , to every outcome of an experiment, in such a way that the resulting
function satisfies the following two conditions:
iv. represents ,
i. The set is an event for every .
v. represents . ii. .
Introduction To Discrete Probability (AI121 - Discrete Mathematics, Spring 2025) 25
Cumulative Distribution Function (CDF)
• Because the constituency of the subset is a function of the real number , so is its probability, i.e.,
P(x <= x). .

Definition
The probability of the event depends on
the number and is called the Cumulative Distribution
Function (CDF) of the random variable, denoted by ,
i.e.,

• As an example, consider the roll of a die experiment,

specified as:

•
Figure 6: CDF of the random variable ,
Then, the CDF of the random variable is shown on defined for the roll-of-a-die experiment.

the right.
Introduction To Discrete Probability (AI121 - Discrete Mathematics, Spring 2025) 26
Properties of CDF
• Using the notation

we establish the following properties of the CDF .

1. , and .

2. is a non-decreasing function of , i.e., if , then , because:

3. As a consequence of the above two properties, .

4. , because the events and form a partition of the probability

space , hence,

Introduction To Discrete Probability (AI121 - Discrete Mathematics, Spring 2025) 27

Properties of CDF
• Using the notation

we establish the following properties of the CDF .

5. is continuous from the right, i.e.,

Proof:
Consider a decreasing sequence of event, given by , where and is small. Then,

Now, let . Then,

6. , and form a partition of the event,

given by , hence,
Introduction To Discrete Probability (AI121 - Discrete Mathematics, Spring 2025) 28
Properties of CDF
• Using the notation

This shows that

the only
we establish the following properties of the CDF . discontinuities
allowed in a CDF
are of jump type.
7. CDF may be discontinuous from the left i.e.,

which can be obtained from 6. by setting , and letting .

8. , which can be obtained from 6. and 7. using the fact that

6. , and form a partition of the event, given by ,

hence,
Introduction To Discrete Probability (AI121 - Discrete Mathematics, Spring 2025) 29
Probability Density Function (PDF)
• Probability Density Function (PDF) of a random variable , denoted by , is defined as the first-order
derivative of its CDF, i.e.,

PDF is non-negative
because CDF is
monotonically non-
• The above expression can be inverted to express decreasing.

• Since , we conclude from (5) that PDF of a random

variable is normalized,
i.e., the probability
mass under it is 1.

• From property 6. and eq. (5), we observe that

Introduction To Discrete Probability (AI121 - Discrete Mathematics, Spring 2025) 30

Types Of Random Variables
• A random variable is called continuous if its CDF and PDF are continuous functions.

• A random variable is called discrete if its CDF is a piece-wise constant function.

➢ As a result, its PDF is given by a sequence of impulses as

where the impulse, defined by unit area, i.e.,

is also called Dirac delta function, characterized by the following Sifting theorem
Figure 7: CDF of a random variable ,
defined for the roll-of-a-die experiment.

Introduction To Discrete Probability (AI121 - Discrete Mathematics, Spring 2025) Figure 8: Dirac delta function. 31
Types Of Random Variables
• A random variable is called continuous if its CDF and PDF are continuous functions.

• A random variable is called discrete if its CDF is a piece-wise constant function.

➢ As a result, its PDF, also called probability mass function (PMF) is given by a sequence of impulses as

where the impulse, defined by unit area, i.e.,

Figure 9: PDF of a random variable ,

defined for the roll-of-a-die experiment.
is also called Dirac delta function, characterized by the following Sifting theorem

• A random variable is called mixed if its CDF is neither continuous nor discrete,
but a mixture of both.
➢ The resulting PDF is continuous with impulses at points of discontinuity in the CDF.
Introduction To Discrete Probability (AI121 - Discrete Mathematics, Spring 2025) Figure 8: Dirac delta function. 32
Discrete Random Variable – Uniform Distribution
• The simplest discrete random variable is given by the following PDF/PMF

which is called the discrete uniform random variable.

• A discrete uniform random variable models the probability of equally likely outcomes of an experiment.

Figure 11: PDF of the CDF shown on the left.

Figure 10: CDF of a discrete uniform random variable with six

equally likely outcomes given by .
Introduction To Discrete Probability (AI121 - Discrete Mathematics, Spring 2025) 33
Bernoulli Random Variable
• Bernoulli trial is an instance of an experiment with only two outcomes, namely the success and the failure.

• It must be noted that the experiment does not actually have to have only two outcomes for it to be specified
by Bernoulli trials; the event success is the happening of any event and failure is then the complement of the
success.
➢ For instance, the roll-of-a-die experiment has six outcomes, however, with respect to the event

an instance of the experiment is a Bernoulli trial w.r.t. , in the sense that either occurs or it doesn’t.

• A random variable mapping the outcomes of a Bernoulli trial on the real line is called a Bernoulli random
variable.
➢ Representing the probability of success with and that of failure with , we observe that that the PMF of a Bernoulli
random variable is given by

➢ Oftentimes, binary values are used, i.e., and . Figure 12: PDF/PMF (blue) and CDF (maroon) of
Introduction To Discrete Probability (AI121 - Discrete Mathematics, Spring 2025) a Bernoulli random variable . 34
Combined Experiments
• So far, we have considered experiments of a single nature at a time, i.e., rolling of a die and tossing of a coin.

• However, if experiments of different kind are performed simultaneously, these can be considered as a single
trial of a combined experiment.

• As an example, consider rolling a fair die and tossing a fair coin at the same time, with the probability spaces,
given by

• Then, under the reasonable assumption that the outcomes of the two experiments are independent of each
other, we conclude that:

• However, the idea of independence used to reach the above conclusion does not agree with the definition of
independence of two events, as the events must belong to the same probability space.

Introduction To Discrete Probability (AI121 - Discrete Mathematics, Spring 2025) 35

Combined Experiments
Definition
• To employ the notion of independence in the case of multiple
experiments, a combined probability space is constructed as
the cartesian product of the individual probability spaces,

whose elements consist of all ordered pairs of the outcomes

of the individual experiments.

• Events of the combined space are cartesian products of the

form , where and , whose probability
is given by

Figure 13: Event in the combined probability space.

Introduction To Discrete Probability (AI121 - Discrete Mathematics, Spring 2025) 36

Combined Experiments
• For the case of die rolling and coin tossing experiments, the combined probability space is given by

• In this case, events ``two" and ``heads" are not elementary, rather subsets of with two and six elements
respectively, i.e.,

with probabilities, given by

• Hence,

Introduction To Discrete Probability (AI121 - Discrete Mathematics, Spring 2025) 37

Binomial Random Variable
• Now consider a sequence of independent Bernoulli trials of an experiment, in which the probability of
success in each trial is , the outcome being an -element set of the form ,
where each is either a success or a failure.

➢ By virtue of the trials being independent, occurrence of the event success is independent in all trials, i.e., the
probability of success in a given trial does not depend on the outcome of any other trial.
❖ As an example, consider a sequence of independent tosses of a fair coin, in which the event may be defined as

❖ Then, the probability of success in a given toss is independent of the outcome of any other toss, and is given by

• Then, the event occurs in mutually exclusive ways, i.e.,

Arranging successes in n
• For instance, in a sequence of independent tosses of a fair coin, the event independent trials is
equivalent to selecting k
objects out of without
paying attention to the
can occur in the following mutually exclusive ways order!

Introduction To Discrete Probability (AI121 - Discrete Mathematics, Spring 2025) 38

Binomial Random Variable
• Then, using the axiom of additivity, we write the probability as

For exactly k
successes in trials,
there have to be n-k
where, using independence of trials, we note (by extending (14)) that failures!

• Therefore,

• A random variable that gives the number of successes in a sequence of n independent Bernoulli trials is
called a binomial random variable, whose PMF is given by (12), i.e.,
Binomial distribution refers
to the staircase function in
which the jump
discontinuities at are
given by (15).

Introduction To Discrete Probability (AI121 - Discrete Mathematics, Spring 2025) 39

Geometric Random Variable
• Instead of finding the probability of successes in a sequence of n independent Bernoulli trials, one may be
interested in finding the probability of the first success after trials (out of ).

• Such a random variable that quantifies the number of trials to achieve first success is called a geometric
random variable, whose probability density function is given by

where

• Geometric distribution is the staircase plot of (17) vs. the variable .

Lecture 11 40