Axioms of Probability w-3

Now we introduce the probability for discrete sample spaces (recall that those
with only a finite set of outcomes).
The restriction to these sample spaces enables us to simplify the concepts and
the presentation without excessive mathematics.
Probability is used to quantify the likelihood, or chance, that an outcome of a
random experiment will occur. “The chance of rain today is 30%’’ is a
statement that quantifies ourfeeling about the possibility of rain.
The likelihood of an outcome is quantified by assigning a number from the
interval [0, 1] to the outcome (or a percentage from 0 to 100%).
Higher numbers indicate that the outcome is more likely than lower numbers.
A probability of 0 indicates an outcome will not occur. A probability of 1
indicates an outcome will occur with certainty.
Axioms of Probability
The probability of an outcome can be interpreted as our subjective probability, or
degree of belief, that the outcome will occur. Different individuals may assign different
probabilities to the same outcomes.
Another interpretation of probability is based on the conceptual model of repeated
replications of the random experiment. The probability of an outcome is interpreted as
the limiting value of the proportion of times the outcome occurs in n repetitions of the
random experiment as n increases beyond all bounds. For example, if we assign
probability 0.2 to the outcome that there is a corrupted pulse in a digital signal, we
might interpret this assignment as implying that, if we analyze many pulses,
approximately 20% of them will be corrupted. This example provides a relative
frequency interpretation of probability.
The proportion, or relative frequency, of replications of the experiment that result in
the outcome is 0.2. Probabilities are chosen so that the sum of the probabilities of all
outcomes in an experiment adds up to 1. This convention facilitates the relative
frequency interpretation of probability.
Axioms of Probability
Probabilities for a random experiment are often assigned on the
basis of a reasonable model of the system under study.
One approach is to base probability assignments on the simple
concept of equally likely outcomes.
For example, suppose that we will select one laser diode randomly
from a batch of 100. Randomly implies that it is reasonable to
assume that each diode in the batch has an equal chance of being
selected. Because the sum of the probabilities must equal 1, the
probability model for this experiment assigns probability of 0.01 to
each of the 100 outcomes. We can interpret the probability by
imagining many replications of the experiment.
Axioms of Probability
Equally Likely Outcomes: Whenever a sample space consists of N
possible outcomes that are equally likely, the probability of each
outcome is 1/N
Probability of en Event: For a discrete sample space, the probability
of an event E, denoted as P(E), equals the sum of the probabilities
of the outcomes in E
Axioms of Probability
Example:
A random experiment can result in one of the outcomes {a, b, c, d}
with probabilities 0.1, 0.3, 0.5, and 0.1, respectively. Let A denote
the event {a, b}, B the event {b, c, d}, and C the event {d}.Then,
P(A) = 0.1 + 0.3 = 0.4
P(B) = 0.3 + 0.5 + 0.1 = 0.9
P(C) = 0.1

P(A’)=?, P(A ∩ B)=?, P(A U B)=?, P(A ∩ C)=? 5 mins…

Axioms of Probability
Definition:
Probability is a number that is assigned to each member of a
collection of events from a random experiment that satisfies the
following properties: If S is the sample space and E is any event in a
random experiment,
(1) P(S)=1
(2) 0 ≤ P(E) ≤ 1
(3) For two events E1 and E2 with E1 ∩ E2 = Ø
P(E1 U E2) = P(E1) + P(E2)
Axioms of Probability
Definition:
Probability of Union: P(A U B) = P(A) + P(B) – P(A ∩ B)
If A and B are mutually exclusive events, P(A U B) = P(A) + P(B)
In general P(A U B U C) = P(A) + P(B) + P(C) - P(A ∩ B) - P(A ∩ C) - P(B
∩ C) + P(A ∩ B ∩ C)

If E1, E2, E3, …., Ek are Mutually Exclusive (that is each and every
pair has no intersection at all) events than
P(E1 U E2 U E3 U E4….. U Ek) = P(E1) + P(E2) + P(E3)+ P(E4)+ ….+
P(Ek)
Conditional Probability
Definition:
Sometimes probabilities need to be reevaluated as additional information
becomes available.
A useful way to incorporate additional information into a probability model is to
assume that the outcome that will be generated is a member of a given event.
This event, say A, defines the conditions that the outcome is known to satisfy.
Then probabilities can be revised to include this knowledge.
The probability of an event B under the knowledge that the outcome will be in
event A is denoted as and this is called the conditional probability of B given A
and written as P(B \ A) and calculated as P(A ∩ B) / P(A) where P(A)>0.
P(B \ A) = P(A ∩ B) / P(A) where P(A)>0
Conditional Probability
Example:
A day’s production of 850 manufactured parts contains 50 parts
that do not meet customer requirements. Two parts are selected
randomly without replacement from the batch. What is the
probability that the second part is defective given that the first part
is defective? Let A denote the event that the first part selected is
defective, and let B denote the event that the second part selected
is defective. The probability needed can be expressed as P(B \ A) . If
the first part is defective, prior to selecting the second part, the
batch contains 849 parts, of which 49 are defective; therefore,
P(B \ A) = 49 / 849
Multiplication and Total Probability Rules
Multiplication Rule
P(B \ A) = P(A ∩ B) / P(A) => P(A ∩ B) = P(A). P(B \ A) = P(B). P(A \ B)
Total Probability Rule
P(B) = P(B ∩ A) + P(B ∩ A’) = P(A). P(B \ A) = P(A’). P(B \ A’)

Mutually Exclusive and Exhaustive Events: If two sets are mutually exclesive, ie
no intersection, and they together form the sample space then we call them
Mutually Exclusive and Exhaustive Events.

Assume are E1,E2,E3,..,Ek mutually exclusive and exhaustive sets. Then

P(B) = P(B ∩ E1) + P(B ∩ E2) + P(B ∩ E3) + …+ P(B ∩ Ek)
= P(E1). P(B \ E1) + P(E2). P(B \ E2) + P(E3). P(B \ E3) + …..+ P(Ek). P(B \ Ek)
Independence of Events
Definition:
Two events are independent (that is the occurance of one does not
affect the occurance of other) if one of the following holds;
P(B \ A) = P(B)
P(A \ B) = P(A)
P(B ∩ A) = P(A). P(B)

This can be generalised to k independent events…

Bayes Theorem
Very important contribution to Conditional Probabilities and their calculation.

Remember P(A ∩ B) = P(A). P(B \ A) = P(B ∩ A) = P(B). P(A \ B)

From here we can write P(A). P(B \ A) = P(B). P(A \ B) which yields

P(A \ B) = P(A). P(B \ A) / P(B) for P(B) > 0.

Which is a very useful result that enables us to solve P(A \ B) in terms of P(B \ A)
Bayes Theorem
Definition:
If E1, E2, E3, ….. Ek are k mutually exclusice and exhaustive events
and B is any event then

𝑃 𝐵\E1 .𝑃(𝐸1)
P(E1 \ B) =
𝑃 𝐵\E1 .𝑃 𝐸1 +𝑃 𝐵\E2 .𝑃 𝐸2 +⋯………+𝑃 𝐵\Ek .𝑃(𝐸𝑘)

for P(B) > 0.

Random Variables
Definition:
A random variable is a function that assigns a real number to each outcome in the sample
space of a random experiment.
A random variable is denoted by an uppercase letter such as X. After an experiment is
conducted, the measured value of the random variable is denoted by a lowercase letter such
as x=70 milliamperes.
A discrete random variable is a random variable with a finite (or countably infinite) range.
A continuous random variable is a random variable with an interval (either finite or infinite) of
real numbers for its range.

Examples of continuous random variables: electrical current, length, pressure, temperature,

time, voltage, weight
Examples of discrete random variables: number of scratches on a surface, proportion of
defective parts among 1000 tested, number of transmitted bits received in error
Discrete Random Variables and
Discrete Probability Distributions
Random variables are so important in random experiments that
sometimes we essentially ignore the original sample space of the
experiment and focus on the probability distribution of the random
variable.
The probability distribution of a random variable X is a description
of the probabilities associated with the possible values of X.
For a discrete random variable, the distribution is often specified by
just a list of the possible values along with the probability of each.
In some cases, it is convenient to express the probability in terms of
a formula.
Discrete Random Variables and
Discrete Probability Distributions
Example:
There is a chance that a bit transmitted through a digital transmission channel is
received in error. Let X equal the number of bits in error in the next four bits
transmitted. (Here X is our random variable)
The possible values for X are {0, 1, 2, 3, 4}. Based on a model for the errors that is
presented in the following section, probabilities for these values will be determined.
Suppose that the probabilities are;
P(X=0) = 0.6561 P(X=1) = 0.2916 P(X=2) = 0.0486
P(X=3) = 0.0036 P(X=4) = 0.0001
The probabilities assigned to all possible values (ie sample space) of X form a
probability distribution. For discrete case we call this distribution as Probability Mass
Function.
A random experiment can often be summarised with a random variable and its
distribution.
Probability Mass Function

A probability belonging to a random variable must be in [0,1]

inclusive
Sum of probabilities in the sample space should add up to 1
Often probability and mass function related as P(X=xi) = f(xi)
Probability Mass Function
Example:
The sample space of a random experiment is {a, b, c, d, e, f }, and each
outcome is equally likely. A random variable is defined as follows:
Outcome a b c d e f
x 0 0 1.5 1.5 2 3

Determine the probability mass function of X. Use the probability mass

function to determine the following probabilities:
a) P(X = 1.5)=? b) P( 0.5 < X < 2.7)=? c) P(X > 3)=?
d) P(0 ≤ X < 2)=? e) P(X=0 or X=2)=?

Let’s do it together, next example will be your turn..

Probability Mass Function
Solution:
First let’s determine the probability mass function. Recall that probability mass
function maps sample space to real line with probabilities. What was the
definition of equally likely: All probabilities in the sample space are the same..
Outcome a b c d e f
x 0 0 1.5 1.5 2 3
f(x) 1/6 1/6 1/6 1/6 1/6 1/6

a) P(X = 1.5) = P(Event set {c,d}) = 1/6 + 1/6 = 2/6 = 1/3

b) P( 0.5 < X < 2.7) = P(Event set {c, d, e}) = 1/6 + 1/6 + 1/6 = 3/6 = 1/2
c) P(X > 3) = P(Event set {Ø}) = 0
d) P(0 ≤ X < 2) = P(Event set {a, b, c, d}) = 1/6 + 1/6 + 1/6 + 1/6 = 4/6 = 2/3
e) P(X=0 or X=2) = P(X=0) + P(X=2) = 1/6 + 1/6 = 2/6 = 1/3
Cumulative Distribution Function

Order the random variable from least to most

Instead of the form P(X=x), find P(X≤x) for each x which is F(x)
Cumulative Distribution Function
Example:
Let’s find the Cumulative Distribution Function F(x) of our previous example

Outcome a b c d e f
x 0 0 1.5 1.5 2 3
f(x) 1/6 1/6 1/6 1/6 1/6 1/6
x 0 1.5 2 3
F(x) 2/6 4/6 5/6 6/6
P(X≤x) P(X≤0) P(X≤1.5) P(X≤2) P(X≤3)
Mean and the Variance of a Discrete Random
Variable
Mean and the Variance of a Discrete RV
Example:
Let’s find the Mean and the Variance of our previous example
Outcome a b c d e f
x 0 0 1.5 1.5 2 3
f(x) 1/6 1/6 1/6 1/6 1/6 1/6
x 0 1.5 2 3
F(x) 2/6 4/6 5/6 6/6
P(X≤x) P(X≤0) P(X≤1.5) P(X≤2) P(X≤3)
Mean 1 1 1 1 1 1 8
µ=E(X) ෍ 𝑥. 𝑓 𝑥 = 0 + 0 + 1.5 + 1.5 + 2 + 3 = = 1.33
6 6 6 6 6 6 6
Variance
෍ 𝑥2. 𝑓 𝑥 − µ2 = 2.91
2

StDev 2 = 𝟏. 𝟕𝟏
Mean and the Variance of a function of RV
Recall that Mean = E(X) = µ

What about the expected value of a function h(X) of the RV X

𝐸(ℎ 𝑥 ) = ෍ ℎ 𝑥 𝑓(𝑥)
𝑥

Which is the generalization of an expected value, which is also a function of RV

X
Mean and the Variance of a function of RV
Example:
Let’s find the expected value of the function h(x) = 2x + 1 where x is our previous
random variable with the following mass and distribution functions

Outcome a b c d e f
x 0 0 1.5 1.5 2 3
f(x) 1/6 1/6 1/6 1/6 1/6 1/6
x 0 1.5 2 3
F(x) 2/6 4/6 5/6 6/6
P(X≤x) P(X≤0) P(X≤1.5) P(X≤2) P(X≤3)

10 mins class excercise..

Some Discrete Distributions
We will see some important discrete probability distributions associated with discrete random
variable of specific type

We will visit the following Special Type Distributions

Discrete Uniform Distribution
Bernoulli Trials
Binomial Distribution
Geometric Distribution
Negative Binomial Distribution
Hypergeometric Distribution
Poisson Distribution
Discrete Uniform Distribution
The simplest discrete random variable is one that assumes only a
finite number of possible values, each with equal probability. A
random variable X that assumes each of the values x1 ,x2 , x3 , …. xn
with equal probability 1/n is frequently of interest
Discrete Uniform Distribution µ and 
- Rolling a dice
- Write a number in 0-9 to a piece of paper, put them in a bag and select at
random
- Generate a random number in [20-30]
Discrete Uniform Distribution µ and 
Example:
Let the random variable X have a discrete uniform distribution on
the integers in [1,3]. Determine the P(X > 2) and find the mean and
variance of X.

If X have discrete uniform distribution, all outcomes should be equally likely

that is P(X=x)=1/n where n is the size of the sample space. Then
P(X>2)=P(X=3)=1/3

Recall that for discrete uniform distribution µ=E(X)=(b+a)/2 and σ2 =[(b-a+1)2 -

1]/ 12 from these formulas µ=(3+1)/2=2 and σ2=[(3-1+1)2 -1]/ 12=8/12=1/3
Bernoulli Trials – Bernoulli Distribution
• A trial with only two possible outcomes is used so frequently as a
building block of a random experiment that it is called a Bernoulli
trial
• Often one of the outcomes termed as Success and the other(s) as
Failure
• These terms are only for convinience. Their meaning depends on
our interested event in the random experiment
• For example tossing a coin is a Bernoulli trial with two possible
outcomes H or T. If we are interested in number of Heads we call it
Success and Tail as Failure
• Or in a multiple choice exam (4 choices) Success is giving the right
answer by random selection (without any knowledge about the
Bernoulli Trials – Bernoulli Distribution
• It is usually assumed that the trials that constitute the random
experiment are independent. This implies that the outcome from
one trial has no effect on the outcome to be obtained from any other
trial
• And that the probability of a success in each trial is constant. In the
multiple choice experiment, if the test taker has no knowledge of
the material and just guesses at each question, we might assume
that the probability of a correct answer is 1/4 for each question
• Most of the time the Binomial Distribution comprised of the
collection of Bernoulli Trials