Scribd
Upload
6 views

lec1

Uploaded by

mail2vinaykk
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
6 views

lec1

Uploaded by

mail2vinaykk
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
You are on page 1/ 2

Lecture 1

Estimation theory.

1.1 Introduction
Let us consider a set X (probability space) which is the set of possible values that
some random variables (random object) may take. Usually X will be a subset of ,
for example {0, 1}, [0, 1], [0, ∞), , etc.
I. Parametric Statistics.
We will start by considering a family of distributions on X :
• { θ , θ ∈ Θ}, indexed by parameter θ. Here, Θ is a set of possible parameters


and probability θ describes chances of observing values from subset of X, i.e.




for A ⊆ X, θ (A) is a probability to observe a value from A.




• Typical ways to describe a distribution:


– probability density function (p.d.f.),
– probability function (p.f.),
– cumulative distribution function (c.d.f.).
For example, if we denote by N (α, σ 2 ) a normal distribution with mean α and variance
σ 2 , then θ = (α, σ 2 ) is a parameter for this family and Θ = × [0, ∞).
Next we will assume that we are given X = (X1 , · · · , Xn ) - independent identically
distributed (i.i.d.) random variables on X , drawn according to some distribution θ0 

from the above family, for some θ0 ∈ Θ, and suppose that θ0 is unknown. In this
setting we will study the following questions.
1. Estimation Theory.
Based on the observations X1 , · · · , Xn we would like to estimate unknown pa-
rameter θ0 , i.e. find θ̂ = θ̂(X1 , · · · , Xn ) such that θ̂ approximates θ0 . In this
case we also want to understand how well θ̂ approximates θ0 .

1
LECTURE 1. ESTIMATION THEORY. 2

2. Hypothesis Testing.
Decide which of the hypotheses about θ0 are likely or unlikely. Typical hypothe-
ses:
• θ0 = θ1 ? for some particular θn ?
• θ0 ≷ θ1
• θ0 6= θ1
Example: In a simple yes/no vote (or two candidate vote) our variable (vote)
can take two values, i.e. we can take the space X = {0, 1}. Then the distribution is
described by
(1) = p, (0) = 1 − p
 

for some parameter p ∈ Θ = [0, 1]. The true parameter p0 is unknown. If we conduct
a poll by picking n people randomly and if X1 , · · · , Xn are their votes then:
1.Estimation theory. What is a natural estimate of p0 ?
#(10 s among X1 , · · · , Xn )
p̂ = ∼ p0
n
How close is p̂ to p0 ?
2. Hypothesis testing. How likely or unlikely are the following:
1
• Hypothesis 1: p0 > 2
1
• Hypothesis 2: p0 < 2
II. Non-parametric Statistics
In the second part of the class the questions that we will study will be somewhat
different. We will still assume that the observations X = (X1 , · · · , Xn ) have unknown
distribution , but we won’t assume that comes from a certain parametric family
 

{ θ , θ ∈ Θ}. Examples of questions that may be asked in this case are the following:


• Does  come from some parametric family { 

θ, θ ∈ Θ}?
• Is  = 

0 for some specific 

0?

If we have another sample X 0 = (X10 , · · · , Xm


0
) then,
• Do X and X 0 have the same distribution?
If we have paired observations (X1 , Y1 ), · · · , (Xn , Yn ):
• Are X and Y independent of each other?
• Classification/regression problem: predict Y as a function of X; i.e.,
Y = f (X) + small error term .

You might also like

pFad - Phonifier reborn

Pfad - The Proxy pFad of © 2024 Garber Painting. All rights reserved.

Note: This service is not intended for secure transactions such as banking, social media, email, or purchasing. Use at your own risk. We assume no liability whatsoever for broken pages.


Alternative Proxies:

Alternative Proxy

pFad Proxy

pFad v3 Proxy

pFad v4 Proxy