Chapter #7 Point Estimation and Sampling Dist
Chapter #7 Point Estimation and Sampling Dist
Chapter 7
Point Estimation of
Parameters and Sampling
Distributions
Introduction
The field of statistical inference consists of those
methods used to make decisions or to draw conclusions
about a population.
These methods utilize the information contained in a
sample from the population in drawing conclusions.
Θ is the uppercase of θ
Example:
As an example,suppose the random variable X is normally distributed with
an unknown mean μ. The sample mean is a point estimator of the unknown
population mean μ. That is, μ = X . After the sample has been selected,
the numerical value x is the point estimate of μ.
Thus= =
if x1 25, =
x2 30, =
x3 29, and x4 31, the point estimate of μ is
25 + 30 + 29 + 31
=x = 28.75
4
Some Parameters & Their Statistics
Estimation problems occur frequently in engineering. We
often need to estimate
Parameter Measure Statistic
μ Mean of a single population x-bar 𝑋𝑋�
σ2 Variance of a single population s2
σ Standard deviation of a single population s
p Proportion ot items of a population that belong to class of interest p -hat 𝜌𝜌�
μ1 - μ2 Difference in means of two populations x bar1 - x bar2 𝑋𝑋�1 − 𝑋𝑋�2
p 1 - p 2 Difference in proportions of two populations p hat1 - p hat2 𝜌𝜌�1 − 𝜌𝜌�2
0.5 0.5
0.4 0.4
0.3
Series1 0.3 Series1
0.2
0.2
0.1
0.1
0
1 2
0
1 - Coin 1 1.5 2
2 - Coins
0.4 0.4
0.35 0.35
0.3 0.3
0.25 0.25
0.2 0.2
Series1 Series1
0.15 0.15
0.1 0.1
0.05 0.05
0 0
1 1.333 1.666 2 1 2 3 4 5
3- Coins 4 - Coins
7.2 Sampling Distributions and the Central
Limit Theorem
X is N(µ, σ𝟐𝟐/𝒏𝒏)
If the population is continuous, unimodal and symmetric, 𝑛𝑛 = 5 is
enough.
If 𝑛𝑛 ≥ 30, that will be enough regardless of the shape of the population.
If 𝑛𝑛 < 30, still it will be fine if the population distribution is not severely
non-normal
7.2 Sampling Distributions and the Central
Limit Theorem
Example 7-1
An electronic company manufactures resistors that have a mean
resistance of 100 ohms and a standard deviation of 10 ohms.
The distribution of resistance is normal.
Find the probability that a random sample of 𝑛𝑛 = 25 resistors will have
an average resistance less than 95 ohms.
𝑃𝑃 𝑋𝑋� < 95 = ? ?
Note that the sampling distribution of 𝑋𝑋� is normal, with mean µ=100
ohms and a standard deviation of:
7.2 Sampling Distributions and the Central
Limit Theorem
Example 7-1
Therefore, the desired probability, shaded area shown in the figure,
can be estimated by standardizing the point 𝑋𝑋� = 95:
And therefore
Probability for previous example
7.2 Sampling Distributions and the Central
Limit Theorem (CLT)
Example 7-2
Suppose that a random variable 𝑋𝑋 has a continuous uniform
distribution: 1 2, 4 ≤ x ≤ 6
f ( x) =
0, otherwise
Find the distribution of the sample mean of a random sample
of size n = 40.
By the CLT the distribution X is normal .
b+a 6+4
=µ = = 5
2 2
(=
b − a) ( 6 − 4)
2 2
σ2 13
1212
σ2 1 3 1
σ=
2
X
= =
n 40 120
Sampling Distribution of a Difference in Sample Means
If we have two independent populations with means μ1 and μ2, and
variances σ12 and σ22, and
If 𝑋𝑋�1 and 𝑋𝑋�2 are the sample means of two independent random samples
of sizes n1 and n2 from these populations:
Then the sampling distribution of:
(𝑋𝑋�2 −𝑋𝑋�1 ) − (𝜇𝜇2 − 𝜇𝜇1 )
𝑧𝑧 =
𝜎𝜎22 𝜎𝜎12
+
𝑛𝑛2 𝑛𝑛1
𝑃𝑃 𝑋𝑋�2 − 𝑋𝑋�1 ≥ 25 =?
19
General Concepts of Point Estimation
Unbiased Estimators
An estimate should be close to the true value of the unknown
parameter
Formally, we say that Θ is unbiased estimator of θ if the expected
value of Θ is equal to θ
� − θ is
If the estimator is not unbiased, then the difference E(Θ)
�
called the bias of the estimator Θ
20
7-3 General Concepts of Point Estimation
7-3.1 Unbiased Estimators
An estimate should be close to the true value of the unknown parameter.
( )
E Θ1 = [E ( X 1 ) + E ( X 2 ) + + E ( X 7 )] = (7 E ( X )) = (7 µ ) = µ
ˆ 1
7
1
7
1
7
( ˆ )
E Θ = [E (2 X ) − E ( X ) + E ( X
2
1
2
1 6 7 )] =
1
2
[2µ − µ + µ ] = µ
VΘ( )
ˆ = V X 1 + X 2 + ... + X 7 = 1 (V ( X ) + V ( X ) + + V ( X ) ) = 1 (7σ 2 ) = 1 σ 2
1 72 1 2 7
7 49 7
σ2
V (Θˆ1 ) =
7
( )
ˆ = V 2 X 1 − X 6 + X 4 = 1 (V (2 X ) + V (− X ) + V ( X ) ) = 1 (4V ( X ) + (−1) 2 V ( X ) + V ( X ))
VΘ 2 22 1 6 4 1 6 4
2 4
1
4
( 1
4
)3
V (θˆ2 ) = 4σ 2 + σ 2 + σ 2 = 6σ 2 = σ 2
2
Since both estimators are unbiased,
The variances can be compared to decide to select the better estimator.
� 𝟐𝟐
� 𝟏𝟏 is smaller than that of 𝜣𝜣
The variance of 𝜣𝜣
� 𝟏𝟏 is the better estimator.
Hence, 𝚯𝚯
7-3 General Concepts of Point Estimation
7-3.3 Standard Error of an Estimator (cont.)
Suppose we are sampling from a normal distribution 𝑋𝑋 with mean µ
and variance σ2 : N(µ , σ2 )
� is normal with mean µ and variance σ2 /𝑛𝑛 , so
The distribution of 𝑿𝑿
the standard error of 𝑋𝑋� is:
𝜎𝜎
𝜎𝜎𝑋𝑋� =
𝑛𝑛
𝑆𝑆
𝜎𝜎�𝑋𝑋� =
𝑛𝑛
Example 7-5: Thermal Conductivity
These observations are 10 measurements of
xi
thermal conductivity of Armco iron.
41.60
Since σ is not known, we use s to calculate 41.48
the standard error. 42.34
41.95
Since the standard error is about 0.2% of
41.86
the mean, the mean estimate is fairly 42.18
precise. 41.72
We can be very confident that the true 42.26
population mean is 41.924 ± 2(0.0898) or 41.81
42.04
in the range [41.744, 42.104]. 41.924 = Mean
A point estimate of the mean thermal conductivity is the
0.284 = Std dev (s )
sample mean 𝑥𝑥̅ = 41.925.
Assume conductivity in normally distributed and 2 times the
0.0898 = Std error
standard error is 2𝜎𝜎�𝑥𝑥 = 2(0.0898)
Method of Moment
7-4.1 Method of Moments
Let 𝑋𝑋1, 𝑋𝑋2, … , 𝑋𝑋𝑛𝑛 be a random sample from the probability
distribution 𝑓𝑓(𝑥𝑥), where 𝑓𝑓(𝑥𝑥) can be either a:
– Discrete probability mass function, or
– Continuous probability density function
34
Example 9
E(X) = �
X
Therefore, E(X) = �
X results in 1/λ = �
X,
λ = 1/�
X, is the moment estimator of λ
35
Example 10
Answer:
Using the method of moments we should equate the first population
∑ 𝑋𝑋𝑖𝑖
moment 𝐸𝐸[𝑋𝑋], to the first sample moment
𝑛𝑛
∑ 𝑋𝑋𝑖𝑖
𝐸𝐸[𝑋𝑋]= � 𝐸𝐸[𝑋𝑋]=(3𝑘𝑘 + 𝑘𝑘)/2=2𝑘𝑘,
=𝑋𝑋,
𝑛𝑛
𝑋𝑋�
𝑋𝑋� = 2k, k =
2
36
7-4 Methods of Point Estimation
Example 2:
Example 2: Normal Distribution Moment Estimators
Suppose that X1, X2, …, Xn is a random sample from a
normal distribution with parameter μ and σ2 where
E(X) = μ and E(X2) = μ2 + σ2.
n
µ = X = 1 1 n 2
=
∑ Xi
n i 1=
and µ +σ = ∑ Xi
2 2
ni1
2
n
1
n
1 n ∑ X i
2
− n n∑ i
X
i1
∑ i
2
σ
= =
X 2
− =
X 2 i 1 =
n i =1 n
n n
2
∑ Xi ∑( Xi − X )
2
n
= ∑ X i2 − i =1 =i =1
1
(biased)
n i =1 n n
38
Question: last year final exam
Let X1 , X2 , … … , X7 denote a random sample from a population having a mean
μ and variance σ2 .
1) Which one of the following estimators of 𝜇𝜇 is the best? (Justify your
𝑋𝑋 + 𝑋𝑋 + 𝑋𝑋 + 𝑋𝑋 6𝑋𝑋 −2 𝑋𝑋4 −2 𝑋𝑋6
answer) 𝜃𝜃̂1 = 1 3 5 7 𝜃𝜃̂2 = 2
4 2
2) Propose (with justification) an estimator of μ better than θ� 1 and θ� 2
3) Suppose that 𝑋𝑋1 , 𝑋𝑋2 , … , 𝑋𝑋𝑛𝑛 is a random sample from an exponential
distribution with an unknown parameter 𝜆𝜆. Using method of moment,
determine an estimator of 𝜆𝜆 (Hint: mean of the distribution).
Problem:
Assume that your task was to estimate the expected value of the age of the 18,500 students at
Qatar University. You were already told that the age of students at QU follows a Normal
Distribution 𝑵𝑵 𝝁𝝁, 𝝈𝝈𝟐𝟐 with a standard deviation of 4 years. The sample that you could get is
composed of 25 students. Answer the following questions:
1-Use the method of moment to find an estimator, 𝝁𝝁 � 𝟏𝟏 , for the average age of the students at the
university.
𝟏𝟏
One unknown => one equation : 𝑬𝑬 𝑿𝑿 = 𝝁𝝁 = ∑𝒙𝒙𝒊𝒊 � 𝝁𝝁
= 𝑿𝑿 �
�𝟏𝟏 = 𝑿𝑿
𝟐𝟐𝟐𝟐
� 𝟏𝟏 is a biased or unbiased estimator.
2-Check if 𝝁𝝁
𝟏𝟏 𝟏𝟏 𝟐𝟐𝟐𝟐𝟐𝟐
�𝟏𝟏 = 𝑬𝑬
𝑬𝑬 𝝁𝝁 ∑𝒙𝒙𝒊𝒊 = ∑𝑬𝑬 𝒙𝒙𝒊𝒊 = �𝟏𝟏 is unbiased ((bias =0)
= 𝝁𝝁 which shows that 𝝁𝝁
𝟐𝟐𝟐𝟐 𝟐𝟐𝟐𝟐 𝟐𝟐𝟐𝟐
3-Show if the below estimators of 𝝁𝝁 are biased or not and find the value of the bias for
unbiased ones (if any)?
𝟏𝟏
� 𝟐𝟐 =
𝝁𝝁 ∑𝟏𝟏𝟏𝟏 𝑿𝑿 (the sum of even numbered samples from X2 till X20),
𝟏𝟏𝟏𝟏 𝒊𝒊=𝟏𝟏 𝟐𝟐𝟐𝟐
𝟏𝟏
� 𝟑𝟑 =
𝝁𝝁 𝟒𝟒𝟒𝟒𝟑𝟑 + 𝟐𝟐𝑿𝑿𝟏𝟏𝟏𝟏 + 𝟏𝟏. 𝟓𝟓𝟓𝟓𝟏𝟏𝟏𝟏 + 𝟎𝟎. 𝟓𝟓𝑿𝑿𝟐𝟐𝟐𝟐 ,
𝟒𝟒
𝟏𝟏 𝟏𝟏𝟏𝟏 𝟏𝟏 𝟏𝟏𝟏𝟏𝟏𝟏
� 𝟐𝟐 = 𝑬𝑬
𝑬𝑬 𝝁𝝁 ∑ 𝑿𝑿 = (𝑬𝑬 𝑿𝑿𝟐𝟐 + 𝑿𝑿𝟒𝟒 + ⋯ + 𝑿𝑿𝟐𝟐𝟐𝟐 = � 𝟐𝟐 is unbiased estimator
= 𝝁𝝁 => 𝝁𝝁
𝟏𝟏𝟏𝟏 𝒊𝒊=𝟏𝟏 𝟐𝟐𝟐𝟐 𝟏𝟏𝟏𝟏 𝟏𝟏𝟏𝟏
𝟏𝟏 𝟖𝟖𝟖𝟖
𝑬𝑬[�
𝝁𝝁𝟑𝟑 ] = 𝑬𝑬 𝟒𝟒𝟒𝟒𝟑𝟑 + 𝟐𝟐𝑿𝑿𝟏𝟏𝟏𝟏 + 𝟏𝟏. 𝟓𝟓𝟓𝟓𝟏𝟏𝟏𝟏 + 𝟎𝟎. 𝟓𝟓𝑿𝑿𝟐𝟐𝟐𝟐 = � 𝟑𝟑 is biased estimator,
= 𝟐𝟐𝟐𝟐 ≠ 𝝁𝝁 𝝁𝝁
𝟒𝟒 𝟒𝟒
bias= 𝟐𝟐𝟐𝟐 − 𝝁𝝁 = 𝝁𝝁
Problem (cont.)
� 𝟏𝟏 , 𝝁𝝁
4-Compare the three estimators 𝝁𝝁 � 𝟐𝟐 and 𝝁𝝁
� 𝟑𝟑 using the Mean Squared Error as a
criteria for comparison and show which one is the best among the three estimators
𝑴𝑴𝑴𝑴𝑴𝑴 � 𝟏𝟏
𝝁𝝁 𝑴𝑴𝑴𝑴𝑴𝑴 � 𝟏𝟏
𝝁𝝁
� 𝟐𝟐
< 𝟏𝟏 𝒂𝒂𝒂𝒂𝒂𝒂 � 𝟑𝟑
𝝁𝝁𝟏𝟏 is the most efficient estimator
< 𝟏𝟏 �
𝑴𝑴𝑴𝑴𝑴𝑴 𝝁𝝁 𝑴𝑴𝑴𝑴𝑴𝑴 𝝁𝝁
5-Can you do better than the estimator that you found to be the best in part (iv),
given you have the same number of samples? Explain.
� 𝟏𝟏 because is the
Using the same sample, we cannot build a better estimator than 𝝁𝝁
MVUE