CSD502 Probability and Statistics

Standard Probability Distributions

Bernoulli Distribution
Experiments with only two outcomes are called Bernoulli experiments.The outcomes are
called as success and failure. Let X be a r.v such that X = 1 if success occurs and X = 0 if failure
occurs. Let ‘p’ be the probability of success and ‘q = 1-p’ be the probability of failure. Then X
is said to follow a Bernoulli distribution. The p.m.f of X is given by
P(X= x) = px(1-p)1-x x = 0,1

X : 0 1
P(X=x): q p
Mean= E(X) = p , Var = V(X) = pq

Binomial distribution
When Bernoulli trials are repeated independently and if the chance of getting a success
remains same for the trials then the number of successes observed in these ‘n’ trials follows
Binomial distribution.
Let X denote the number of successes and ‘p’ be the probability of success in a single trial
then X follows Binomial distribution with parameters ‘n’ and ‘p’
X B( n , p)
The p.m.f of X is given by
P(X = x) = nCx px qn-x x = 0,1,2,…n
0 otherwise
Mean = E(X) = np
Var = V(X) = npq
Mean > Variance
If p < 0.5, the distribution is positively skewed. If p = 0.5, the distribution is symmetric.If p >
0.5, the distribution is negatively skewed

Additive property of Binomial distribution

Let X B( n1 , p) and Y B(n2, p)
If X and Y are independent r.v.s then X + Y B( n1+n2 , p )
If X B ( n , p ) then Y = n – X represents the number of failures
Y B(n,q)

R commands for Binomial Distribution

Let X → B(n, p)
• dbinom(r, n, p) : P(X = r)
• pbinom(r, n, p) : P(X ≤ r)
• qbinom(prob, n, p) : gives the value of c such that P(X ≤ c)= prob.
• rbinom(N, n, p): generates a sample of size N from B(n,p)
Poisson Distribution
Poisson Process
A Poisson Process is a model for a series of discrete event where
• The average time between events is known, but the exact timing of events is random.
• The arrival of an event is independent of the event before (waiting time between
events is memoryless).
• The average time between events is known but they are randomly spaced.
• Two events cannot occur at the same time.
For example, suppose we own a website which goes down on average once per 60 days, but
one failure doesn’t affect the probability of the next. All we know is the average time
between failures.
We might have back-to-back failures, but we could also go years between failures due to the
randomness of the process.

Examples of a Poisson Process:

• customers calling a help center,
• visitors to a website,
• radioactive decay in atoms,
• photons arriving at a space telescope,
• movements in a stock price.
• Poisson processes are generally associated with time, but they do not have
to be. In the stock case, we might know the average movements per day
(events per time), but we could also have a Poisson process for the number of
trees in an acre (events per area).

The Poisson Process is the model used for describing randomly occurring events over a given
area or a given time period.
The probability mass function gives the probability of observing X events in a Poisson
Process.
X P(λ)
The p.m.f of X is given by
−λ 𝑥
ⅇ λ
P( X = x) = 𝑥!
x = 0,1,2…
0 o.w
Mean = E(X) = λ Var = V(X) = λ
Poisson distribution gives probability of X events in an interval.

Additive Property of Poisson distribution

Let X P(λ1) and Y P(λ2). If X and Y are independent r.v.s then
X+Y P(λ1 + λ2 )
Poisson distribution as a limiting form of Binomial distribution
Let X B( n,p) . When ‘n’ is very large and ‘p’ is very small, we find λ = np and
Use Poisson distribution to find the probabilities.
Commands in R
Let X → P(𝛌).
• dpois(r, 𝛌) : P(X = r)
• ppois(r, 𝛌 ) : P(X ≤ r)
• qpois(prob, 𝛌) : gives the value of c such that P(X ≤ c)= prob.
• rpois(N, 𝛌): generates a sample of size N from P(𝛌)

Geometric Distribution
Consider a sequence of Bernoulli trials with constant probability of success ‘p’ in a single
trial.Let X represent number of failures before the first success. (X+1)th trial is a success. Then
X follows Geometric distribution
X G( p)
P.m.f of X is given by
P(X=x) = qxp x = 0,1,2,…
0 o.w
Where q = 1 - p
Mean = E(X) = q/p Variance = V(X) = q/p2
Real life Situations:
No. of bombs dropped until it hits the target.
No of persons to be interviewed for a post until a suitable candidate is found.
R Commands
Let X→ G(p)
dgeom(r, p) : P(X = r)
pgeom(r, p ) : P(X ≤ r)
qgeom(prob, p) : gives the value of c such that P(X ≤ c)= prob.
rgeom(N, p): generates a sample of size N from G(p)

Negative Binomial Distribution

Suppose a trial results into two outcomes a success and a failure with constant probability p
and q at each trial respectively. Suppose X denotes the number of failures before getting kth
success, i.e. no of trails required will be X+k.
P(X=x) = P (getting kth success at X+kth trial)
= P (Getting k-1 successes in X+k-1 trials and kth success at (X+k)th trial)
Since trials are independent:
P(X=x) = P (Getting k-1 successes in X+k-1 trials) *P (Getting kth success at
(X+k)th trial)
= x+k-1Ck-1 pk-1 qx p
= x+k-1Ck-1 pk qx
Thus, p.m.f. of negative binomial distribution is
P(X=x) = x+k-1Ck-1 pk qx , x= 0,1,2,…… 0<p<1, q=1-p, k≥0

= 0, otherwise.
If k=1, pmf of Negative Binomial distribution is the pmf of a Geometric distribution.
 If X → B(n,p), n the number of trials is fixed and X the number of successes is a variable.
On the other hand if X → NB(k,p), the number of successes k is fixed and the number of
trials is a variable.
Mean = kq/p Variance = kq/p2

• Variance > Mean

Additive Property
Suppose X and Y are independent random variables such that X→NB(k1,p) and Y→NB(k2,p),
then X+Y→NB(k1+k2,p)
Suppose X1,X2,…Xk are iid random variables having geometric distribution with parameter p,
then ΣXi follows negative binomial distribution with parameter k and p.
R commands
Let X→NB(k, p)
• dnbinom(r, k, p) : P(X = r)
• pnbinom(r, k, p) : P(X ≤ r)
• qnbinom(prob, k, p) : gives the value of c such that P(X ≤ c)= prob.
• rnbinom(N, k, p): generates a sample of size N from NB(k,p)

Uniform Distribution
Let X be a discrete r.v with ‘n’ values with equal probability 1/n. Then the p.m.f. of X can be
represented as
P(x) = 1/n x = 1,2,3…n
i.e
X :1 2 3 …. n
P(X=x):1/n 1/n 1/n 1/n
The above distribution is called as Uniform distribution.Parameter of the Uniform
distribution is ‘n’
Mean = E(X) = (n+1)/2
Var = V(X) = (n2-1)/12

General Form
P(x) = 1/ (b-a+1) ; x = a , a+1 , ….b
0 ; otherwise
Mean = (a+b)/2 Var = [(b-a+1)2 – 1]/12
R Commands
Download the package ‘extraDistr’
Let X→Discrete Uniform over a to b
• ddunif(r, a, b) : P(X = r)
• pdunif(r, a, b) : P(X ≤ r)
• qdunif(prob, a, b) : gives the value of c such that P(X ≤ c)= prob.
• rdunif(N, a, b): generates a sample of size N from Discrete Uniform Distribution over
a to b.
Uniform Distribution
X U(a,b)
The p.d.f of X is given by
f(x) = 1/ (b – a) a≤x≤b
0 o.w
This distribution is also known as rectangular distribution
Mean and Variance of X
E(X) = (b + a)/2 V(X) = (b-a)2/12

Distribution function of X
F(x) = 0 x<a
(x-a)/(b-a) a ≤ x ≤ b
1 x>b

The continuous uniform distribution represents a situation where all outcomes in a range
between a minimum and maximum value are equally likely.

• This distribution is widely used in risk analysis;

• Monte Carlo software algorithms use a sample from this distribution (between zero
and one) to generate random samples from other distributions.
There are only a few real-life processes that have this form of uncertainty. For example:
• the position of a particular air molecule in a room,
• the point on a car tyre where the next puncture will occur,
• the number of seconds past the minute that the current time is,
• the length of time that one may have to wait for a train.
•
R Commands
Let X → Uniform (a, b).
• dunif( r, a, b) : density at X=r
• punif(r,a,b) = P( X < r),
• qunif(p,a,b) = c, P(X < c) = p
• runif(n,a,b) Sample of size n from Uniform distribution over (a,b).

Exponential Distribution
The exponential distribution is one of the widely used continuous distributions. It is often
used to model the time elapsed between events. The exponential distribution is often
concerned with the amount of time until some specific event occurs.

For example
• the amount of time (beginning now) until an earthquake occurs.
• the length, in minutes, of long distance business telephone calls.
• the amount of time, in months, a car battery lasts.
It is used in the field of reliability.
For exponential distribution large values are few as compared to small values.
A continuous random variable X is said to have an exponential distribution with parameter
λ>0, shown as X Exponential(λ), if its p.d.f is given by
f (x) = λe−λx , x > 0

0 , otherwise

Mean = 1/λ V(X) = 1/λ2

Distribution function of X
−λ𝑥
𝐹 (𝑥) = 1 − ⅇ 𝑥≥0
0 𝑥<0
R Commands
Let X → exp( 𝛌).
dexp(r, λ) = density at X=r
pexp(r, λ) = P( X < r)
qexp(p, λ) = c, P(X < c) = p
rexp(n, λ) Sample of size n from Exponential Distribution with parameter λ.

Normal Distribution
A normal distribution, is a distribution that occurs naturally in many situations. For example,
it is seen in tests like the SAT and GRE. The bulk of students will score the average (C), while
smaller numbers of students will score a B or D. An even smaller percentage of students
score an F or an A. This creates a distribution that resembles a bell (hence the nickname).
The bell curve is symmetrical. Half of the data will fall to the left of the mean; half will fall to
the right.Many groups follow this type of pattern. That’s why it’s widely used in business,
statistics and in government bodies like the FDA.
Ex.Heights of people, Measurement errors, Blood pressure, IQ scores, Salaries.
The empirical rule tells you what percentage of your data falls within a certain number of
standard deviations from the mean:
• 68% of the data falls within one standard deviation of the mean.
• 95% of the data falls within two standard deviations of the mean.
• 99.7% of the data falls within three standard deviations of the mean.
The standard deviation controls the spread of the distribution. A smaller standard deviation
indicates that the data is tightly clustered around the mean; the normal distribution will be
taller. A larger standard deviation indicates that the data is spread out around the mean; the
normal distribution will be flatter and wider.

Properties of a normal distribution

● The mean, mode and median are all equal.
● The curve is symmetric at the center (i.e. around the mean, µ).
● Exactly half of the values are to the left of center and exactly half the values
are to the right.
● The total area under the curve is 1.
A continuous type random variable X is said to follow normal distribution if its pdf is given
by,
1 2
− (𝑥−µ)
1 2

f(x) = ⅇ 2σ
-∞<x<∞,-∞<μ<∞,σ>0
2πσ

X→ N( µ, σ2). In particular, let µ = 0, σ2=1, X→ N(0,1). This is a standard normal variable.

If X→ N( µ, σ2) then (X - µ)/ σ N(0,1)
Mean = Median = Mode = µ , Variance = σ2
Probability curve is bell shaped, symmetric, unimodal at X = µ and it extends indefinitely in
both the directions.

Additive property
a. Let X and Y be independent normal variates with X N (μ1 , σ12) and
Y N (μ2 , σ22) then
X+Y N(μ1+μ2 , σ12 + σ22) and X – Y N(μ1- μ2 , σ12 + σ22)
b. Let X N(μ , σ2). If Y = AX + B then Y N(Aμ+B , A2σ2)

Normal approximation to Binomial and Poisson distribution

i. X B(n,p) . For large n, Binomial distribution can be approximated to Normal
distribution
E(X) = np V(X) = npq
X N( np , npq )
𝑥−𝑛𝑝
→ 𝑁(0, 1)
𝑛𝑝𝑞

ii. X P( λ ) .For large values of λ , we use Normal distribution with

E(X) = λ and V(X) = λ
X N(λ , λ)
𝑥−λ
→ 𝑁(0, 1)
λ

R Commands
Let X → N( 𝛍, 𝛔2)
• dnorm(r, µ,σ) = density at X=r
 • pnorm (r, µ, σ) = P( X < r),
• qnorm (p, µ, σ) = c, P(X < c) = p
• rnorm (n, µ, σ) Sample of size n from Normal Distribution with parameters µ, σ.
Gamma Distribution
The gamma distribution is a family of right-skewed, continuous probability distributions.
Gamma distributions occur frequently in models used in
• engineering (such as time to failure of equipment and load levels for
telecommunication services),
• meteorology (rainfall),
• finance (insurance claims and loan defaults)
for which the variables are always positive and the results are skewed
(unbalanced).
Gamma distribution is used to predict the wait time until future events.
The exponential distribution predicts the wait time until the *very
first* event. The gamma distribution, on the other hand, predicts the wait time until the
*k-th* event occurs.

Let X be a continuous random variable denoting time required for α events to occur in a
Poisson process with mean arrival time of β, then the probability distribution function is:
Where,
● Γ(x) = the gamma function,
● α = The shape parameter.
● β (sometimes θ is used instead) = The rate parameter (the reciprocal of the
scale parameter).
α and β are both greater than 1.

When α = 1, this becomes the exponential distribution. When β = 1 this becomes the
standard gamma distribution. The effect of changing alpha and beta on the shape of the
gamma distribution.
α is the number of events you are waiting for and β is the time between the events.
• If α remains unchanged but β increases, the graph shifts to the right as waiting times
will lengthen.
• If β stays the same but the α increases, the graph will also shift to the right.
• As α approaches infinity, the gamma closely matches the normal distribution.

Mean: E(X) = αβ Variance: var(X) = αβ2

 • Additive Property of Gamma Distribution
• X Gamma(α1, β) and Y Gamma(α2, β), then X+Y Gamma( α1+ α2, β) if X and Y are
independent.
• If X→Gamma(α, β) then cX→Gamma (α, βc)
• If X is standard normal variable then Y = X2 has Gamma(1/2, 2) distribution
R Commands
Let X →Gamma(α, β)
• dgamma(r, 𝛂, 1/𝛃) : density at X=r
• pgamma(r, 𝛂, 1/𝛃) : P(X < r)
• qgamma(p, 𝛂, 1/𝛃) : returns c such that P( X < c) =p
• rgamma(n, 𝛂, 1/𝛃): A sample of size n from Gamma distribution with
parameters 𝛂, 𝛃.

Weibull Distribution
Weibull distribution is a continuous probability distribution, generally used in failure or
survival analysis in manufacturing, industrial engineering, electronic equipments,
mechanical devices, etc. to predict the length of life or proper functionality of a product
from a specified time until it fails.
Weibull analysis is the study life data analysis or the analysis of time to failure using the
Weibull distribution (and sometimes, the lognormal).
Lifetimes of medical and dental implants
• Components produced in a factory (like bearings, capacitors, or dialetrics)
• Warranty analysis,
 • Utility services,
• Other areas where time-to-failure is important.
The major advantages to using Weibull analysis is that it can be used for analyzing lifetimes
with very small samples. It also produces an easy-to- understand plot.
Definition:
A random variable X is said to have a Weibull distribution with parameters
α and β (α>0,β>0) if the pdf of X is

X W(α ,β)
α is the shape parameter and β is the scale parameter.
When α = 1 , it is pdf for an exponential distribution with parameter β.
Thus we see that the exponential distribution is a special case of both the gamma and
Weibull distributions.
The value for the shape parameter (α) determines the failure rates:
• If α is less than 1, then the failure rate decreases with time (i.e. the process
has a large number of infantile or early-life failures and fewer failures as time
passes).
• For α = 1: the failure rate is constant, which means it’s indicative of useful life
or random failures.
• α > 1: the failure rate increases with time (i.e. the distribution If models
wear-out failures, which tend to happen after some time has passed).
Mean and Variance of X
R commands
Let X → Weibull( 𝛂, 𝛃)
dweibull(x, shape =𝛂 , scale =𝛃 ): density at X=x
pweibull(r, shape =𝛂 , scale =𝛃): P( X ≤ r)
qweibull(p, shape =𝛂 , scale =𝛃) : Returns c such that P( X ≤ c) = p
rweibull(n, shape =𝛂 , scale =𝛃) : Sample of size n from Weibull(𝛂, 𝛃)
Lognormal Distribution
A random variable Y is said to follow a lognormal distribution with parameters 𝛍 and 𝛔 if
log(Y) follows a normal distribution with mean 𝛍 and standard deviation 𝛔.
A normal distribution can be converted to a log-normal distribution using logarithmic
mathematics. That is primarily the basis as log-normal distributions can only come from a
normally distributed set of random variables.
Y is lognormal → X = log(Y) is normal. X is normal → Y = eX is lognormal
Log-normal distributions are positively skewed with long right tail.
Applications of Lognormal Distribution
There are many mathematical similarities between the normal and lognormal distributions.
Normal distributions can allow for negative random variables while log- normal distributions
include all positive variables.
One of the most common applications where log-normal distributions are used in finance is
in the analysis of stock prices.
• The potential returns of a stock can be graphed in a normal distribution.
• The prices of the stock however can be graphed in a log-normal distribution.
• The log-normal distribution can be used to identify the compound return that the
stock can expect to achieve over a period of time.

Definition
A continuous random variable X is said to follow lognormal distribution with parameters µ
and σ if its pdf is given by,

If X follows lognormal with parameters µ and σ, then Y =aX follows lognormal with
parameters µ+ln(a), σ
 • The standard normal density curve is symmetric bell shaped curve, with mean and
median located at x = 0.
• The standard lognormal density (since it is derived from the standard normal
distribution) is located entirely over the positive x-axis.
• The lognormal density curve is not symmetric and is a uni-modal curve skewed
toward the right.
• All the standard normal values on the negative x-axis, when exponentiated,
are in the interval (0, 1). Hence the lower half of the lognormal probabilities
lie in the interval x = 0 to x = 1.The other half of the lognormal probabilities
lie in the interval (1, ∞). The distribution positively skewed.
• Though lognormal distribution is a skewed distribution, some are less
skewed than others. The lognormal distributions with larger parameter
value of 𝛔 tend to be more skewed.
R Commands
Let X → lognormal( 𝛍, 𝛔)
• dlnorm(x, µ, σ ): density at X=x
• plnorm(r, µ, σ ): P( X ≤ r)
• qlnorm (p, µ, σ ) : Returns a value c such that P( X ≤ c) = p.
• rlnorm(n, µ, σ ) : A random sample of size n from lognormal distribution
with parameters 𝛍, 𝛔

Pareto distribution
Definition: Pareto distribution is a skewed, heavy-tailed distribution that is sometimes
used to model the distribution of incomes. The basis of the distribution is that a high
proportion of a population has low income while only a few people have very high
incomes.

• All the density curves are skewed to the right and have a long tail.
• The curve with a higher value of 𝛂 approaches the x-axis faster.
• When the Pareto model is used as a model of lifetime of systems (machines or
devices), a larger value of the shape parameter 𝛂 would mean that less “lives”
surviving to old ages.
• If the Pareto model is used as a model of income or wealth of individuals, then a
higher 𝛂 would mean a smaller proportion of the people are in the higher income
brackets.
• The long tail contains 80% or more of the probabilities.
• Applied as a model to describe income and wealth of a country, for
• modelling extreme loss in insurance applications or in financial
• applications.
X Pa(α)
The p.d.f of X is given by
α
f(x) = α+1 x≥1 ,α>0
𝑥

α
Mean = E(X) = α−1
if α >1

α
Variance = V(X) = 2 if α >2
(α−1) (α−2)

Note: Mean and variance may not always exist.

R commands are not available directly for the calculation of probabilities. Use library
extraDistr. Let X → Paα)
dpareto(x, α ): density at X=x
ppareto(r, α ): P( X ≤ r)
qpareto(p, α) : Returns a value c such that P( X ≤ c) = p.
rpareto(n, α) : A random sample of size n from Pareto distribution