Probability and Statistics

EEF 271e

Ibraheem Shayea
Electronics and Communication Engineering Department,
Faculty of Electrical and Electronics Engineering
İstanbul Teknik Üniversitesi - İTÜ
 Syllabus 08

Discrete Probability Distributions

Part A
Bernoulli Distributions
Lecture Objectives
What you should Learn from this Lecture ?

❑ Understanding :
o Concept of Bernoulli Distributions
o PDF of Bernoulli Distribution Function
o CDF of Bernoulli Distribution Function
o Mean
o Median
o Variance
o Standard Deviation
o Practice some Examples
Outline
❖ Discrete Probability Distributions
❖ Concept of Bernoulli Distribution
❖ PMF of Bernoulli Distribution
❖ CDF of Bernoulli Distribution
❖ Mean of Bernoulli Distribution
❖ Median of Bernoulli Distribution
❖ Variance of Bernoulli Distribution
❖ Standard Deviation of Bernoulli Distribution
❖ Properties
❖ Examples
Discrete Distributions Functions
Discrete Distributions Functions
Discrete Random Variable

Discrete Random Variable is a random variable that has countable

values, such as a list of non-negative integers and non-fractional values.
That meaning its values can be listed in a sequence (e.g., 𝑥1 , 𝑥2 , 𝑥3 , …).

6
Discrete Distributions Functions
Discrete Random Variable
Example 01

❖ Discrete RandomVariable
During a given time interval, the number of telephone calls received is
an example of a discrete random variable.

❖ Continuous RandomVariable
The duration of a call is an example of a continuous random variable.

7
Discrete Distributions Functions
Discrete Random Variable

Example 02

❖ Discrete Random Variable

❖ Selecting Randomly a nonnegative integer out of a countably infinite
list, i.e. {0, 1, 2, …}.

❖ Continuous Random Variable

❖ Selecting Randomly a nonnegative real number out of an uncountably
infinite list, i.e. [0, ∞).

8
Discrete Distributions Functions
Discrete Random Variable

Discrete Random Variable is a random variable that can only take on

a countable number of different values, as opposed to a continuous
random variable which can take on any value within a given range.

9
Discrete Distributions Functions
Discrete Random Variable

Discrete Random Variable has a Probability Mass Function (PMF) and

Continuous RandomVariable has a Probability Density Function (PDF).

10
Discrete Distributions Functions
Discrete Distribution Function

❖ Discrete Distribution describes the probability of occurrence of

each value of a discrete random variable.

❖ Discrete Distribution is a distribution of data in statistics that has

discrete values. Discrete values are countable, finite, positive integers,
such as 1, 10, 15, etc.

❖ In a Discrete Probability Distribution, each possible value of the

discrete random variable can be associated with a non-zero
probability.
Discrete Distributions Functions
Discrete Distribution Function

❖ Discrete Distribution has a range of values that are countable.

❖ Discrete Probability Distribution is made up of discrete

variables.

❖ Discrete Probability Distribution describes a probability

distribution of a random variable X, in which X can only take on the
values of discrete integers.

12
Discrete Distributions Functions
Discrete Distribution Function

Discrete Distribution Function is a probability distribution that

describes the occurrence (event) chance of each discrete random
variable, which are countable values. It is represented by a Probability
Mass Function (PMF) and Cumulative Distribution Function (CDF).

13
Discrete Distributions Functions
Types of Discrete Probability Distribution Functions

Examples of a Discrete Distribution Functions:

1. Bernoulli Distribution
2. Binomial Distribution
3. Negative Binomial Distribution (Pascal Distribution)
4. Geometric Distribution
5. Poisson Distribution

6. Hypergeometric Distribution
7. …etc
Concept of Bernoulli Distribution
Concept of Bernoulli Distribution
Definition

❖ Bernoulli Distribution can be assumed as a model for the set of

possible outcomes of any single experiment that enquires a Yes/No
question.

❖ Bernoulli Distribution is also considered as a special case of

the Binomial Distribution

16
Concept of Bernoulli Distribution
Definition

Bernoulli Distribution is a probability distribution used to model a

single trial of a binary (two-valued) random experiment, where the
outcome can either be a success or a failure.

17
Concept of Bernoulli Distribution
Application of Bernoulli Distribution

Bernoulli Distribution has a variety of applications in different fields of

science and engineering.

18
Concept of Bernoulli Distribution
Application of Bernoulli Distribution
Some of the most common applications include:
❖ Binary Classification Problems ❖ Insurance

❖ Quality Control ❖ Polling and Surveys

❖ Marketing Research ❖ Web Traffic Analysis

❖ Genetics ❖ Financial Modelling

❖ Sports Analytics ❖ Risk Assessment

19
Concept of Bernoulli Distribution
Application of Bernoulli Distribution
Coin Flipping Experiments
Bernoulli Distribution is commonly used to model the outcomes of
coin flipping experiments, where the outcome can either be heads or tails.

20
Concept of Bernoulli Distribution
Application of Bernoulli Distribution

Binary Classification Problems

Bernoulli Distribution is often

used, in machine learning, to model
binary classification problems, where
the outcome can be classified as either
a positive or negative instance.

21
Concept of Bernoulli Distribution
Application of Bernoulli Distribution

Quality Control

Bernoulli Distribution can be used to model the outcomes of quality

control tests, where the outcome can be either acceptable or defective.

22
Concept of Bernoulli Distribution
Application of Bernoulli Distribution
Marketing Research

Bernoulli Distribution can be used to model the success or failure of a

marketing movement or the probability of a customer buying a product.

23
PMF of Bernoulli Distribution
PMF of Bernoulli Distribution
PMF for a Bernoulli Distribution
Probability Mass Function (PMF) of the Bernoulli distribution is
defined as follows:

𝟏 − 𝒑 𝒙=𝟎

𝒑𝑿 𝒙 = 𝒑 𝒙=𝟏

𝟎 𝑶𝒕𝒉𝒆𝒓𝒘𝒊𝒔𝒆
where
o 𝒑 Represents the probability of success
➢ Probability of getting a 1, if we assign 1 to success and 0 to failure.
➢ where 0 < 𝑝 < 1

o 1 − 𝑝 represents probability of failure (getting a 0). 25

PMF of Bernoulli Distribution
PMF for a Bernoulli Distribution
❖ Bernoulli distribution has a single parameter, usually denoted by
𝑝:
o 𝑝 represents the probability of success (getting a 1) .
o 1 − 𝑝 represents the probability of failure (getting a 0).

❖ Bernoulli Random Variable (BRV) 𝑿 takes:

o The value of 𝟏 with probability 𝑝 (known as the probability of success)
o The value of 0 with probability 𝟏 − 𝒑 (the probability of failure)

❖ Probability of Success
o Range of 𝑝: 0 < 𝑝 < 1
o A discrete random variable with the range {0, 1} 26
PMF of Bernoulli Distribution
Bernoulli Trial
❖ Bernoulli Trial is corresponds to sampling from the Bernoulli
distribution, it is a random experiment with exactly two possible
outcomes, in which the probability of each of the two outcomes
remains the same every time the experiment is conducted.

❖ Bernoulli Trial is equivalent to the tossing of a biased coin or

examining if a component is defective in a system or modeling any
other process that has only two possible outcomes, such as taking a
pass-fail exam.

❖ Bernoulli Trial is a basic building block for some well-known

discrete distributions, such as the binomial, geometric, and Pascal
distributions. 27
PMF of Bernoulli Distribution
Independent Trials

Independent Trials means that the outcome of one trial has no

influence over the outcome of another trial.

28
PMF of Bernoulli Distribution
Matlab for the PMF of Bernoulli Distribution
PDF of a Bernoulli Distribution is defined in Matlab by the following
function:

𝒑𝑿 𝒙 = binopdf(x,n,p);

𝑥 : RandomVariable
𝑛 : Bernoulli Trial , 𝑛 = 1
p : Probability of Success
29
PMF of Bernoulli Distribution
Example 01:
Matlab : PMF

clc
clear all
close all

p = 0.65;
x = 0:1;
n = 1';
y = binopdf(x,n,p);

figure
stem(x,y,'filled','LineWidth',3)
xlabel('Observation')
ylabel('Probability')
xlim([-1 2])

figure
bar(x,y,1)
xlabel('Observation')
ylabel('Probability') 30
CDF of Bernoulli Distribution
CDF of Bernoulli Distribution
CDF for a Bernoulli Distribution

???

32
CDF of Bernoulli Distribution
CDF for a Bernoulli Distribution
Cumulative Distribution Function (CDF) of the Bernoulli
distribution is a step function that gives the probability that the random
variable X is less than or equal to a given value x:

𝑭(𝒙) = 𝑷(𝑿 ≤ 𝒙)

CDF is derived from PMF

∞

𝐹(𝑥) = ෍ 𝑝𝑋 𝑥
𝑥=−∞

𝑃𝑋 𝑥 = 𝑃 𝑋 = 𝑥𝑘 , 𝑘 = 1,2 3, … ∞ 33
CDF of Bernoulli Distribution
CDF for a Bernoulli Distribution
For the Bernoulli Distribution, where the random variable X can only
take on the values of 0 or 1, the CDF is:

0 𝑥<0

𝐹𝑋 𝑥 = 1 − 𝑝 0 ≤𝑥 < 1

1 𝑥≥1
where
o 𝒑 represents the probability of success.
➢ probability of getting a 1, if we assign 1 to success and 0 to failure.
➢ where 0 < 𝑝 < 1
o 1 − 𝑝 represents probability of failure (getting a 0). 34
CDF of Bernoulli Distribution
CDF for a Bernoulli Distribution

❑ Thus, the CDF of the Bernoulli distribution is a step function that:

o Jumps from 0 𝑡𝑜 1 − 𝑝 at 𝑥 = 0 and
o Then remains constant at 1 − 𝑝 for 𝑟𝑎𝑛𝑔𝑒 𝑜𝑓 0 < 𝑥 < 1
o Then jumps from 1 − 𝑝 𝑡𝑜 1 𝑎𝑡 𝑥 = 1
o Then remains constant at 1 for all x greater than or equal to 1 (𝑥 ≥ 1).

Note that the CDF of the

Bernoulli distribution is very
simple, due to the fact that it
only has two possible
outcomes. 35
CDF of Bernoulli Distribution
PMF and CDF Plotting for a Bernoulli Distribution
❑ The PMF and CDF for the Bernoulli distribution.

PMF CDF

Bernoulli Distribution. 36
CDF of Bernoulli Distribution
Matlab for the CDF of Bernoulli Distribution
The CDF of a Bernoulli Distribution is defined in Matlab by the following
function:

𝒑𝑿 𝒙 = binocdf(x,n,p);

𝑥 : RandomVariable

𝑛 : Bernoulli Trial , 𝑛 = 1

p : Probability of Success 37
CDF of Bernoulli Distribution
Matlab

clc
clear all
close all

p = 0.65;
x = 0:1;
n = 1';
Z = binocdf(x,n,p);

figure
stairs(x, Z,'LineWidth',3)
xlabel('Observation')
ylabel('Cumulative Probability’)

38
Mean of Bernoulli Distribution
Mean of Bernoulli Distribution
How to obtain Mean of Bernoulli Distribution ?
The mean 𝝁𝑿 of the Bernoulli random variable X, which cannot be
independently set as it is function of p only is as follows:

𝑀𝑒𝑎𝑛 = 𝐸 𝑋 = ෍ 𝑥 𝑝𝑋 𝑥
𝑥=−∞

1 − 𝑝 𝑥=0

where, 𝑝𝑋 𝑥 = 𝑝 𝑥=1

0 𝑂𝑡ℎ𝑒𝑟𝑤𝑖𝑠𝑒 40
Mean of Bernoulli Distribution
How to obtain Mean of Bernoulli Distribution ?
The mean 𝝁𝑿 of the Bernoulli random variable X, which cannot be
independently set as it is function of p only is as follows:

𝑀𝑒𝑎𝑛 = 𝐸 𝑋 = 1 × 𝑝 + 0 × (1 − 𝑝)

𝑴𝒆𝒂𝒏 = 𝑬 𝑿 = 𝒑

𝑴𝒆𝒂𝒏 = 𝑬 𝑿 = 𝝁𝑿 = 𝒑
41
Median
Median
Median

In a Bernoulli distribution, the Median is the value that separates

the lower and upper halves of the distribution.

43
Median
Median
The median of a Bernoulli distribution depends on the probability of success,
denoted by p.

❖ If 𝒑 > 𝟎. 𝟓, then the median is 1.

❖ If 𝒑 < 𝟎. 𝟓, then the median is 0.

❖ If 𝒑 = 𝟎. 𝟓, then both 0 and 1 are equally likely, and the median

is arbitrary, often taken as either 0 or 1.
44
Median
Median

So, the median of a Bernoulli distribution is either 0 or 1, depending on

the value of p

𝟏
𝟎 𝒊𝒇 𝒑 <
𝟐

𝟏
𝑴𝒆𝒅𝒊𝒂𝒏 = [𝟎, 𝟏] 𝒊𝒇 𝒑 =
𝟐

𝟏
𝟏 𝒊𝒇 𝒑 >
𝟐

45
Variance
Variance
Variance
The variance of a r.v. X, denoted by 𝜎𝑋2 or 𝑉𝑎𝑟(𝑋),is defined by:

Thus

Note from definition that:

Variance
Variance
The variance 𝝈𝟐𝑿 of the Bernoulli random variable X, which cannot be
independently set as it is function of 𝑝 only is, respectively, as follows:

When the random variable is discrete the Variance is calculated by :

Bernoulli random variable X,

48
Variance
Variance

Variance for Bernoulli random variable X, can be calculated by :

Variance = 𝝈𝟐𝑿 = 𝒑 𝟏 − 𝒑

49
Standard Deviation
Standard Deviation

Standard Deviation

Standard Deviation is the square root of the Variance.

𝝈𝑿 = 𝝈𝟐𝑿 = 𝒑 𝟏 − 𝒑

51
Properties
Properties
Properties
The Bernoulli distribution has several important properties that are worth
knowing:

❖ Discrete Probability Distribution

❖ Bernoulli Distribution has a Single Parameter

❖ Special Case of the Binomial Distribution

❖ OnlyTwo Outcomes

❖ Symmetric Bernoulli
53
Properties
Discrete Probability Distribution

Bernoulli distribution is a discrete probability distribution, as it can only

take on the values of 0 or 1.

𝟏 − 𝒑 𝒙=𝟎

𝒑𝑿 𝒙 = 𝒑 𝒙=𝟏

𝟎 𝑶𝒕𝒉𝒆𝒓𝒘𝒊𝒔𝒆

54
Properties
Bernoulli Distribution has a Single Parameter

The Bernoulli distribution has a single parameter, usually denoted by p,

which represents the probability of success (i.e., the probability of getting a
1).

𝟏 − 𝒑 𝒙=𝟎

𝒑𝑿 𝒙 = 𝒑 𝒙=𝟏

𝟎 𝑶𝒕𝒉𝒆𝒓𝒘𝒊𝒔𝒆

55
Properties
Special Case of the binomial distribution

Bernoulli Distribution is a special case of the binomial distribution,

which models the number of successes in a fixed number of independent
Bernoulli trials.

𝑁𝑢𝑚𝑏𝑒𝑟 𝑜𝑓 𝑇𝑟𝑖𝑎𝑙𝑠 = 𝑛 = 1

56
Properties
Only Two Outcomes
o Bernoulli Distribution only has two outcomes

o Bernoulli RandomVariable has outcomes {0, 1}.

57
Properties
Symmetric Bernoulli
❖ Symmetric Bernoulli Random variable has outcomes {−1, 1}.

❖ This Figure shows the PMF for the symmetric Bernoulli example.

Figure Symmetric Bernoulli random variable ( PMF based on the

58
Kronecker delta function)
Examples
Examples
Example 01

Solution

60
Examples
Example 01
1. Write the mathematical expression for 𝑭𝑿 (𝒙).
Solution

61
Examples
Example 01
2. Calculate 𝑭𝑿 (−𝟏), 𝑭𝑿 (𝟎), 𝑭𝑿 (𝟎. 𝟓), and 𝑭𝑿 (𝟏)
Solution

62
Examples
Example 01
3. Plot the CDF for this Bernoulli random variable.
Solution

The CDF is a step function with:

o A jump from 0 𝑡𝑜 0.4, 𝑎𝑡 𝑥 = 0

o A jump from 0.4 𝑡𝑜 1 𝑎𝑡 𝑥 = 1.

63
Examples
Example 02
A factory produces light bulbs, and each bulb has a 95% chance of passing
quality control. Let 𝑋 be a random variable that represents the result of
inspecting a single bulb, where:
𝑋 = 1 if the bulb passes inspection
𝑋 = 0 if the bulb fails inspection
Assuming the result follows a Bernoulli distribution, calculate:
1) The probability that a randomly selected bulb passes inspection.
2) The probability that a randomly selected bulb fails inspection.
Solution
Examples
Example 02
Solution
Given:
o Success probability 𝑝 = 0.95
o Failure probability 1 − 𝑝 = 0.05
The End

Thank You
References 1
1) Yates, R. D. ve Goodman, D. (2005). Probability and Stochastic Processes: A Friendly Introduction for
Electrical and Computer Engineers. JohnWiley and Sons.

2) Hsu, H. (2010). Probability, RandomVariables, and Random Processes (Second Edition). McGraw- Hill
(Schaum's Outline Series).

3) Leon-Garcia, A. (2008). Probability, Statistics, and Random Processes For Electrical Engineering (Third
Edition). Prentice Hall.

4) Krishnan,V. (2006). Probability and Random Processes.Wiley-Interscienc.

5) Ali Grami (2020).“Probability, Random Variables, Statistics, and Random Processes Fundamentals &
Applications” , first edition, JohnWiley & Sons, Inc.

6) JOHN J. SHYNK (2013). Probability, RandomVariables, And Random Processes:Theory and Signal
Processing Applications”, University of California, Santa Barbara, CA, USA .

7) Papoulis, A. (1991). Probability, RandomVariables and Stochastic Processes (Third Edition). McGraw-
Hill.
References 3
1) https://online.stat.psu.edu/stat504/node/209/
2) https://support.minitab.com/en-us/minitab-express/1/help-and-how-to/basic-
statistics/probability-distributions/supporting-topics/basics/continuous-and-discrete-
probability-distributions/
3) https://online.stat.psu.edu/stat504/node/209/
4) https://support.minitab.com/en-us/minitab-express/1/help-and-how-to/basic-
statistics/probability-distributions/supporting-topics/basics/continuous-and-discrete-
probability-distributions/
5) https://www.investopedia.com/terms/p/probabilitydistribution.asp
6) https://opentextbc.ca/introbusinessstatopenstax/chapter/properties-of-continuous-
probability-density-functions/
7) https://sites.nicholas.duke.edu/statsreview/discrete-probability-distributions/