Module 2: Inferential Statistics

Lecture 3: Discrete Probability Distributions

Probability Distributions and Random Variables
● Discrete probability distribution: the distribution of a discrete random variable
● Discrete random variable: a r.v. that takes discrete values. Discrete r.v. typically
counts - Eg. number of kids in a household, number of successes in n trials
● Continuous random variable: a r.v. that takes values on (part of) the real line - Eg.
waiting time in a queue, height of soldiers, inflation rates

Discrete Probability Distributions

● Defined via the means of probability distribution function (pdf) which
assigns a probability within [0,1] to possible outcomes such that all
probabilities sum up to 1 (nothing else than the relative frequencies of
the population)
● pdf tells us how likely it is that discrete r.v. takes value/outcome x

Population Mean, Variance and SD

● Population mean/expectation

● Vaiance

● Standard deviation

Graphical representation
of PDF
● Pdf of r.v. X shown graphically as histogram

Cumulative Distribution Function (CDF)

● calculates the probability that the r.v. is smaller
than or equal to a certain value

Graphical representation of CDF

Special Cases of Distribution Functions
Binomial distribution: pdf
● A r.v. X taking value in (0,1,...,n) is said to follow the binomial distribution denoted by
X Bin(n , p)
● If it describes the (randome) number of successes out of n trials in a binomial
experiment. The probability distribution function (pdf) is:
With 2 parameter:
→ n: number of trials
→ p: probability of success p ϵ (0 , 1)

→ E(X) = np; VAR(X)=np(1-p)

Binomial Distribution: cdf

Bernoulli Distribution
● A r.v. X taking value in {0,1} (0,1) is said to follow the Bernoulli distribution, denoted
by X Ber (p)
● If it describes the binary outcome 0 (failure) or 1 (success), with probability of
success p or P(X=1)=p
● The probability distribution function (pdf) is P( X=x ; p)= p x ¿
- Where x=0 or 1
● The bernoulli distribution has one parameter
- P: probability of success p ϵ (0 , 1) (ie. 0 ≤ p ≤ 1 ¿
- E(X)=p; Var(X)=p(1-p)
- Apparently, bernoulli distribution is a special case of
binomial distribution where the number of trials n=1

Discrete Uniform distribution

● A r.v.X taking value in {a,a+1,...,b-1,b} is said to follow the discrete uniform
distribution X DUnif (a , b)
● If all potential outcomes/realisations between a and b are equally likely, the probability
distribution function (pdf) is →
- With 2 parameters
→ a: the minimum value that X can take
→ b: the maximum value that X can take; so there are b−a+1 potential outcomes
● If all potential outcomes (realisations) between a and b are equally likely
 ● Eg.

Lecture 4: Continuous Probability Distributions

● Discrete probability distribution: the distribution of a discrete random variable
● Discret
e

random variable: a r.v. that takes discrete values. Discrete r.v. typically counts - Eg.
number of kids in a household, number of successes in n trials
● Continuous random variable: a r.v. that takes values on (part of) the real line. - Eg.
height of soldiers, inflation rates, waiting time in a queue

2 different probability distribution functions (pdfs): Discrete, Continuous

→ Scores add up to 1

Probability
density
function
 ● continuous probability distribution for X is defined via the means of probability density
function (pdf) which assigns a positive value to possible outcomes of X such that the
density is integrated to 1 (this means that the area under the curve is 1). The
probability that X lies between two numbers is the area under pdf function between
those numbers

Discrete random variable

● P(X=x), where x is some specific value because P(X=x) =0 always
● A continuous r.v. has infinitely many outcomes. If a single outcome had positive
probability, the probabilities would add up to infinity and not 1
● Eg. What is the probability that a random person waits exactly
2.71285748634050284… minutes?
- The probability is 0. However the probability that a person waits in between
2.71…84 and 2.71…9 is strictly positive

Inequaltiies implications
●

Lecture 5: Sampling and Sampling Distributions

Sampling Distributions
Why sample and not use the entire population?
● It is costly - eg. cannot measure everyone’s height (7.7 billion individuals to measure)
● Time-consuming - eg. by the time measuring is finished, world population changed
● If there is true randomness involved, the population size can be infinity
● It risks killing entire population - measure average car crash;we need to crash all cars

● Statistical inference or statistical analysis uses information from a sample to infer

properties about the population
● Statistical inference goes from the sample to the population. We use information from
a sample to estimate parameters in the population
● Often we cannot compute the population parameters
● We can however compute sample statistics. We hope sample statistic can serve as
an estimate for population parameter. This is the case if the sample is collected
randomly
● Collecting different samples lead to different sample statistics. Sampling error means
the discrepancy between a sample statistic and corresponding population parameter
 ● A sample
statistic is
also
called
(serves
as) an
estimate
of the
corresponding population
● X is an estimate of μₓ (we collect a sample and compute X such that it reflects μₓ)
2 2
● s is an estimate of σ
x x
● CVsample is an estimate of CVpopulation
● For each sample, we can compute a sample statistic,
eg. X , so we will have a collection of such sample
statistics, ie. a collection of X ’s
● Beacuase the sample is chosen randomly, so the
sample statistic (the estimate) itself is a random variable, which means it follows
some certain distribution. The variability of this
distribution helps us understand how accurate our
estimate is

● When studying the (sampling) distribution of a

sample statistic or an estimate, we assume all
random variables involved are normally
distributed: X N (μ ₓ , σₓ)

Sampling Distribution of the average

Distribution of sample average: known std
● Assuming X N (μ ₓ , σₓ) and we hace a sample of size n, ie. we have x ₁ , x ₂ , ... , xₙ
Sampling Distribution: X

Sample distribution of mean: unknown std

● Assuming X N (μ ₓ , σₓ) and we hace a sample of size n, ie. we have x ₁ , x ₂ , ... , xₙ
● Student’s
t

distribution is a continuous dustribution characterised by one parameter: degree of

freedom (dof) v. It looks a lot like the standard normal distribution, symmeteric
around zero. But the smaller dof v is, the fatter tail (than standard normal distribution)

Sampling distribution of variance

2
Sampling distribution: s
x
● Assuming X N (μ ₓ , σₓ) and we hace a sample of size n, ie. we have x ₁ , x ₂ , ... , xₙ

2
x distribution
● x 2 distribution is a continuous distribution characterised by one parameter dof v. It is
asymmetric and takes only positive values (because variance is always positiv)
2
x
● PDF of x 2 for different dof v. Note that the mean of the r.v. (n−1) s is v=n-1
σ 2x
 Lecture 6: point and Interval Estimates
Confidence Interval
Statistical inference
● Given different samples, we have different sample statistics, so different point
estimates
● Sampling error is the discrepancy between sample statistics and (unobserved or
2
unmeasurable) population statistics (eg. difference between X and μ x or s and σ 2x )
x

Confidence interval
● Based on sampling distributions of a sample
statistic, we can construct an interval (an
upper and lower limit) called confidence
interval, to pin down a possible range into
which the true but unknown population
parameter may fall
● Based on sampling distribution, we construct confidence intervals for:
- μ x when the population standard deviation σ x is known; μ x when σ x is
unknown)
2
- σx
- σx

● Interpretation of confidence interval: if we keep resampling, all sample statistics form

an interval, within which the true population parameter falls with some percentage
probability
● Informal interpretation (theoretically incorrect): with some percentage of confidence,
the true population statistic is going to fall within this interval. The population mean is
just a number (which we do not observe/know). Once we constructed the confidence
interval, it either covers the true population mean or not

Confidence interval for mean (case of known variance/std)

● From the sampling distribution of X we know, if the population standard deviation σ x
is known, we have:

Width of confidence interval