Examples Consistency Asymptotic Normality Two-Step Estimators Estimating the Asymptotic Variance Hypothesis Testing

Estimadores Extremos
Introduo
Cristine Campos de Xavier Pinto
CEDEPLAR/UFMG
Maio/2010
Cristine Campos de Xavier Pinto Institute
Estimadores Extremos
Examples Consistency Asymptotic Normality Two-Step Estimators Estimating the Asymptotic Variance Hypothesis Testing
Sequences and Convergence
Deterministic Sequence vs Random Sequence
Convergence of Deterministic Sequences: A sequence of
nonrandom numbers a
N
: N = 1, 2, ... converges to a limit
a if for all > 0, there exists a N

such that if N > N

then
[a
N
a[ <
In this case,
a
N
a as N
Cristine Campos de Xavier Pinto Institute
Estimadores Extremos
Examples Consistency Asymptotic Normality Two-Step Estimators Estimating the Asymptotic Variance Hypothesis Testing
Convergence in Probability: A sequence of random
variables X
N
: N = 1, 2, ... converges in probability to a
constant a if for all > 0
Pr [[X
N
a[] 0 as N
In this case,
X
N

p
a , plimX
N
= a
Cristine Campos de Xavier Pinto Institute
Estimadores Extremos
Examples Consistency Asymptotic Normality Two-Step Estimators Estimating the Asymptotic Variance Hypothesis Testing
Bounded Deterministic Sequences: A sequence of
nonrandom numbers a
N
: N = 1, 2, ... is bounded if and
only if there is some b < such that
[a
N
[ _ b for all N = 1, 2, ...
Bounded in Probability: A sequence of random variables
X
N
: N = 1, 2, ... is bounded in probability if and only if for
every > 0, there exists a b

< and a integer N

such that
Pr [[X
N
[ _ b

] < for N _ N

Cristine Campos de Xavier Pinto Institute

Estimadores Extremos
Examples Consistency Asymptotic Normality Two-Step Estimators Estimating the Asymptotic Variance Hypothesis Testing
Notation
Deterministic Sequences:
A sequence of random numbers a
N
is O
_
N

_
if N

a
N
is
bounded.
If = 0, a
N
is bounded and we can write a
N
= O (1)
a
N
is o
_
N

_
if N

a
N
0.
When = 0, a
N
0, and we say a
N
= o (1)
Cristine Campos de Xavier Pinto Institute
Estimadores Extremos
Examples Consistency Asymptotic Normality Two-Step Estimators Estimating the Asymptotic Variance Hypothesis Testing
Random Sequences:
If X
N
is bounded in probability, we write X
N
= O
p
(1) .
If X
N

p
0, we write X
N
= o
p
(1)
Lemma: If X
N

p
a, then X
N
= O
p
(1) .
Cristine Campos de Xavier Pinto Institute
Estimadores Extremos
Examples Consistency Asymptotic Normality Two-Step Estimators Estimating the Asymptotic Variance Hypothesis Testing
Notation
Random Sequences:
A sequence of random numbers X
N
is O
p
(R
N
), with R
N

is a nonrandom, positive sequences, if

X
N
R
N
= O
p
(1)
We write
X
N
= O
p
(R
N
)
A random sequence X
N
, with R
N
is a nonrandom,
positive sequences, if
X
N
R
N
= o
p
(1)
We write
X
N
= o
p
(R
N
)
Cristine Campos de Xavier Pinto Institute
Estimadores Extremos
Examples Consistency Asymptotic Normality Two-Step Estimators Estimating the Asymptotic Variance Hypothesis Testing
Examples:
Lets get back to OLS estimator, under the classical linear
model assumptions
_
N
_

OLS

0
_
=
_
1
N

N
i =1
X
/
i
X
i
_
1
_
1
_
N

N
i =1
X
/
i

i
_
_
1
N

N
i =1
X
/
i
X
i
_
1

_
E
_
X
/
X
__
1
= o
p
(1) (using WLLN)
1
_
N

N
i =1
X
/
i

i
= O
p
(1) (by CLT)
At the end,
_
N
_

OLS

0
_
=
_
E
_
X
/
X
__
1
_
1
_
N
N

i =1
X
/
i

i
_
+o
p
(1)
= O
p
(1)
Cristine Campos de Xavier Pinto Institute
Estimadores Extremos
Examples Consistency Asymptotic Normality Two-Step Estimators Estimating the Asymptotic Variance Hypothesis Testing
However
1
N
N

i =1
X
/
i

i
E
_
X
/

_
. .
=0 by assumption
= o
p
(1)

OLS

0
=
_
E
_
X
/
X
__
1
0 +o
p
(1)
= o
p
(1)
Cristine Campos de Xavier Pinto Institute
Estimadores Extremos
Examples Consistency Asymptotic Normality Two-Step Estimators Estimating the Asymptotic Variance Hypothesis Testing
Properties:
1 o
p
(1) +o
p
(1) = o
p
(1)
2 O
p
(1) +O
p
(1) = O
p
(1)
3 o
p
(1) +O
p
(1) = O
p
(1)
4 O
p
(1) O
p
(1) = O
p
(1)
5 o
p
(1) O
p
(1) = o
p
(1)
6 o
p
(1) o
p
(1) = o
p
(1)
Cristine Campos de Xavier Pinto Institute
Estimadores Extremos
Examples Consistency Asymptotic Normality Two-Step Estimators Estimating the Asymptotic Variance Hypothesis Testing
Extremum estimators (M-estimators): estimators obtained
by either minimizing or maximizing a certain function dened
over a parameter space.
An estimator

is an extremum estimator if there is an
objective function

Q
N
() such that

maximizes

Q
N
() subject to
where is the set of possible parameter values.
In this course, we will work with four examples of extremum
estimators: MLE, NLS, GMM and CMD.
Cristine Campos de Xavier Pinto Institute
Estimadores Extremos
Examples Consistency Asymptotic Normality Two-Step Estimators Estimating the Asymptotic Variance Hypothesis Testing
Example 1: Maximum Likelihood Estimator (MLE)
Suppose we have a random sample (Z
1
, ..., Z
N
) with p.d.f
f (Z[
0
) equal to some member of family of p.d.fs f (Z[ ).
The MLE maximizes

Q
N
() =
1
N
N

i =1
ln f (Z
i
[ )
where

Q
N
() is the normalized likelihood function.
Cristine Campos de Xavier Pinto Institute
Estimadores Extremos
Examples Consistency Asymptotic Normality Two-Step Estimators Estimating the Asymptotic Variance Hypothesis Testing
Example 2: Nonlinear Least Squares (NLS)
We have a random sample of (Y
i
, X
i
)
N
i =1
with
E[ Y[ X] = h (X,
0
), the estimator maximizes

Q
N
() =
1
N
N

i =1
[Y
i
h (X
i
.)]
2
Cristine Campos de Xavier Pinto Institute
Estimadores Extremos
Examples Consistency Asymptotic Normality Two-Step Estimators Estimating the Asymptotic Variance Hypothesis Testing
Example 3: Generalized Method of Moments (GMM)
Suppose that there is a vector of moment functions g (Z, )
such that the population moments satisfy
E[g (Z,
0
)] = 0
The GMM estimator minimizes a squared Euclidean distance
of the sample moments from their populations analog (=zero).
Let

W be a positive semi-denite matrix so that
_
m
/

Wm
_1
2
is a measure of distance from m to zero. The GMM estimator
maximizes

Q
N
() =
_
1
N
N

i =1
g (Z
i
, )
_/

W
_
1
N
N

i =1
g (Z
i
, )
_
Cristine Campos de Xavier Pinto Institute
Estimadores Extremos
Examples Consistency Asymptotic Normality Two-Step Estimators Estimating the Asymptotic Variance Hypothesis Testing
Example 4: Classical Minimum Distance Estimator (CMD)
Suppose that there is a vector of estimators
p

0
and a
vector of functions h () with
0
= h (
0
).
An estimate of can be constructed by maximizing

Q
N
() = [ h ()]
/

W [ h ()]
Cristine Campos de Xavier Pinto Institute
Estimadores Extremos
Examples Consistency Asymptotic Normality Two-Step Estimators Estimating the Asymptotic Variance Hypothesis Testing
Remarks
There is a dierent framework that is the minimum distance
estimation.
The minimum distance estimation is the class of estimators
such that

maximizes

Q
N
() subject to
where

Q
N
() = g
N
()
/

W g
N
()
with g
N
() as a vector of data and parameters such that
g
N
(
0
)
p
0
and

W is positive denite,
GMM and CMD are special cases of minimum distance.
This framework is useful to get the asymptotic distribution of
GMM and CMD.
Cristine Campos de Xavier Pinto Institute
Estimadores Extremos
Examples Consistency Asymptotic Normality Two-Step Estimators Estimating the Asymptotic Variance Hypothesis Testing
General idea: If

Q
N
() converges in probability to Q
0
(),
for every and Q
0
() is maximized at the true parameter
0
,
then the limit of the maximum

should be the maximum of
the limit (
0
), under some regularity conditions.
To get consistency of an extremum estimator, we need to
dene uniform convergence in probability.
Uniform Convergence in Probability:

Q
N
() converges
uniformly in probability to Q
0
() if
sup

Q
N
() Q
0
()

p
0
Cristine Campos de Xavier Pinto Institute
Estimadores Extremos
Examples Consistency Asymptotic Normality Two-Step Estimators Estimating the Asymptotic Variance Hypothesis Testing
Theorem
If there is a function Q
0
() such that
(i ) Q
0
() is uniquely maximized at
0
(ii ) is compact
(iii ) Q
0
() is continuous
(iv)

Q
N
() converges uniformly to Q
0
(), then

0
Cristine Campos de Xavier Pinto Institute
Estimadores Extremos
Examples Consistency Asymptotic Normality Two-Step Estimators Estimating the Asymptotic Variance Hypothesis Testing
Comments on this theorem:
Condition (i ): Identication condition. This condition is
related to the general idea of identication: distribution of the
data at the true parameter is dierent than at any other
possible parameter value.
Condition (ii ) is very important, and strong. It requires that
bounds on the true parameter value are known. The practice
of ignoring the compactness restriction is justied for
estimators where compactness can be dropped without
aecting consistency. One nice result (next theorem) is when
the objective function is compact.
Conditions (iii ) and (iv) are the regularity conditions for
consistency. These assumptions are satised if the moments
of certain functions exist and there is some continuity in

Q
N
() or in the distribution of the data.
Cristine Campos de Xavier Pinto Institute
Estimadores Extremos
Examples Consistency Asymptotic Normality Two-Step Estimators Estimating the Asymptotic Variance Hypothesis Testing
Theorem
If there is a function Q
0
() such that
(i ) Q
0
() is uniquely maximized at
0
(ii )
0
is an element of the interior of convex set and

Q
N
() is concave
(iii )

Q
N
()
p
Q
0
() for all , then

exists with
probability approaching one and

0
Cristine Campos de Xavier Pinto Institute
Estimadores Extremos
Examples Consistency Asymptotic Normality Two-Step Estimators Estimating the Asymptotic Variance Hypothesis Testing
Idea: Large sample estimators are approximately equal to
linear combinations of sample averages, so we can CLT and
LLN.
Lets assume that
0
is an interior of , which means that
must have nonempty interior. Since


p

0
,

is in the
interior of with probability one. If

Q
N
() is continuously
dierentiable then (with probability one)

solves the FOC
s
N
_
Z,

_
= 0
where s
_
Z
i
,

_
is a vector of partial derivatives of

Q
N
() .
Cristine Campos de Xavier Pinto Institute
Estimadores Extremos
Examples Consistency Asymptotic Normality Two-Step Estimators Estimating the Asymptotic Variance Hypothesis Testing
If

Q
N
() is twice continuous dierentiable, then we can
expanded FOC around
0
s
N
_
Z
i
,

_
= s
N
(Z
i
,
0
) +H
N
_

, Z
_ _

0
_
where H
N
_

, Z
i
_
is a matrix with second derivatives
evaluated at a dierent mean value. Since this mean value are
between

and
0
, then it must converge in probability to
0
.
Combining the results above
0 =
1
_
N
N

i =1
s (Z
i
,
0
) +
1
N
N

i =1
H
N
_

, Z
i
_
_
N
_

0
_
Cristine Campos de Xavier Pinto Institute
Estimadores Extremos
Examples Consistency Asymptotic Normality Two-Step Estimators Estimating the Asymptotic Variance Hypothesis Testing
Lemma: If Z
i
is i.i.d, H (Z, ) is continuous at
0
with
probability one, and there is a neighborhood A of
0
such
that E[sup
A
|H (Z, )|] < , then for any


p

0
,
1
N
N

i =1
H
_
Z
i
,

_

p
E[H (Z,
0
)]
We use this lemma to show that
1
N
N

i =1
H
N
_

, Z
i
_

p
E[H (Z,
0
)]
If H
0
= E[H (Z,
0
)] is nonsingular, then
1
N

N
i =1
H
N
_

, Z
i
_
is nonsingular with probability one, and
_
N
_

0
_
=
_
1
N
N

i =1
H
N
_

, Z
i
_
_
1
_

1
_
N
N

i =1
s (Z
i
,
0
)
_
Cristine Campos de Xavier Pinto Institute
Estimadores Extremos
Examples Consistency Asymptotic Normality Two-Step Estimators Estimating the Asymptotic Variance Hypothesis Testing
If
1
_
N

N
i =1
s (Z
i
,
0
) is the average of i.i.d random vector with
mean zero, multiplied by
_
N, then we can apply the CLT to
this term.
At the end,
_
N
_

0
_

d
A
_
0, H
1
0
H
1
0
_
where = E
_
s (Z
i
,
0
) s (Z
i
,
0
)
/
_
.
Cristine Campos de Xavier Pinto Institute
Estimadores Extremos
Examples Consistency Asymptotic Normality Two-Step Estimators Estimating the Asymptotic Variance Hypothesis Testing
We will get the asymptotically linear representation of each
estimator
_
N
_

0
_
=

N
i =1
(Z
i
)
_
N
+o
p
(1)
where E[(Z)] = 0 and E
_
(Z) (Z)
/
_
exists.
Asymptotic normality of

results from CLT applied to

N
i =1
(Z
i
)
_
N
.
Inuence Function: (Z
i
) .
For asymptotic normality, we have two basic results. One for
extremum estimators, and another one for minimum distance
estimators.
Cristine Campos de Xavier Pinto Institute
Estimadores Extremos
Examples Consistency Asymptotic Normality Two-Step Estimators Estimating the Asymptotic Variance Hypothesis Testing
Theorem
Suppose that

is such that

maximizes

Q
N
() subject to ,


p

0
(i )
0
interior()
(ii )

Q
N
() is twice continuously dierentiable in a
neighborhood A of
0
.
(iii )
_
N\

Q
N
()
d
A (0, )
(iv) there is H () that is continuous at
0
and
sup
A
_
_
_\

Q
N
() H ()
_
_
_
p
0
(v) H = H (
0
) is nonsingular
then
_
N
_

0
_

d
A
_
0, H
1
0
H
1
0
_
Cristine Campos de Xavier Pinto Institute
Estimadores Extremos
Examples Consistency Asymptotic Normality Two-Step Estimators Estimating the Asymptotic Variance Hypothesis Testing
Theorem
Suppose that

is such that

maximizes

Q
N
() subject to
where

Q
N
() = g
N
()
/

W g
N
()

p

0

W
p
W, W is positive semi-denite
and
Cristine Campos de Xavier Pinto Institute
Estimadores Extremos
Examples Consistency Asymptotic Normality Two-Step Estimators Estimating the Asymptotic Variance Hypothesis Testing
Theorem
(i )
0
interior()
(ii ) g
N
() is continuously dierentiable in a
neighborhood of A of
0
(iii )
_
N g
N
(
0
)
d
A (0, )
(iv) There is G () that is continuous at
0
and
sup
A
|\

g
N
() G ()|
p
0
(v) for G = G (
0
), G
/
WG is nonsingular
then
_
N
_

0
_

d
A
_
0,
_
G
/
WG
_
1
G
/
WWG
_
G
/
WG
_
1
_
Cristine Campos de Xavier Pinto Institute
Estimadores Extremos
Examples Consistency Asymptotic Normality Two-Step Estimators Estimating the Asymptotic Variance Hypothesis Testing
Two-step estimator is one that depends on some preliminary,
"rst-step" estimator of a parameter vector.
Feasible GLS estimator and IV estimator are examples of
two-step estimators.
Question: Does the rst step aects the asymptotic variance
of the second? If it does, how?
A general type of estimator

that is one that, with probability
approaching one, solves an equation:
1
N
N

i =1
s (Z
i
, , ) = 0
where q is a vector of functions with the same dimension of
and is a rst-step estimator.
The rst question is: When

will be consistent for
0
?
Cristine Campos de Xavier Pinto Institute
Estimadores Extremos
Examples Consistency Asymptotic Normality Two-Step Estimators Estimating the Asymptotic Variance Hypothesis Testing
To get identication, we need to know about the asymptotic
behavior of .
General assumption:

p

+
Note that
+
does not need to converge to a parameter
indexing some interesting feature of the distribution.
Example: Two-stage least squares. We ask that

p

+
We did not ask

p

0
where
X =
0
Z +v
Cristine Campos de Xavier Pinto Institute
Estimadores Extremos
Examples Consistency Asymptotic Normality Two-Step Estimators Estimating the Asymptotic Variance Hypothesis Testing
Identication condition: Q
0
(,
+
) is uniquely maximized at

0
.
The consistent result is the same as before, but using
Q
0
(,
+
) .
Theorem
If there is a function Q
0
(,
+
) such that
(i ) Q
0
(,
+
) is uniquely maximized at
0
(ii ) is compact
(iii ) Q
0
(,
+
) is continuous
(iv)

Q
N
(,
+
) converges uniformly to Q
0
(,
+
), then

0
Cristine Campos de Xavier Pinto Institute
Estimadores Extremos
Examples Consistency Asymptotic Normality Two-Step Estimators Estimating the Asymptotic Variance Hypothesis Testing
Asymptotic Normality
Two cases can happen:
1 The asymptotic variance of
_
N
_

0
_
does not depend on
the asymptotic variance of
_
N (
+
) .
2 The asymptotic variance of
_
N
_

0
_
must adjusted to
account for the rst-stage estimation of
+
.
Question: When can we ignore the rst-stage estimation
error?
Cristine Campos de Xavier Pinto Institute
Estimadores Extremos
Examples Consistency Asymptotic Normality Two-Step Estimators Estimating the Asymptotic Variance Hypothesis Testing
Using arguments similar to the ones above,
_
N
_

0
_
= H
+
(
0
)
1
_

1
_
N
N

i =1
s (Z
i
,
0
, )
_
+o
p
(1)
where H
+
(
0
) = E[H (Z;
0
,
+
)]
Doing a mean value expansion for the second term
1
_
N
N

i =1
s (Z
i
,
0
, ) =
1
_
N
N

i =1
s (Z
i
,
0
,
+
) +F
0
_
N (
+
) +o
p
(1)
where
F
0
= E[\

s (Z,
0
,
+
)]
Cristine Campos de Xavier Pinto Institute
Estimadores Extremos
Examples Consistency Asymptotic Normality Two-Step Estimators Estimating the Asymptotic Variance Hypothesis Testing
If E[\

s (Z,
0
,
+
)] = 0, we can ignore the rst-stage
estimation error.
The asymptotic variance of
_
N
_

0
_
is the same as if
+
were plugged into.
When this condition fails, we need to adjust the variance of
_
N
_

0
_
.
To do the adjustment, we get the rst-order representation of
_
N (
+
)
_
N (
+
) =

N
i =1

(Z
i
)
_
N
+o
p
(1)
with E[

(Z
i
)] = 0
Cristine Campos de Xavier Pinto Institute
Estimadores Extremos
Examples Consistency Asymptotic Normality Two-Step Estimators Estimating the Asymptotic Variance Hypothesis Testing
Using this linear representation, we can write
_
N
_

0
_
= H
+
(
0
)
1
_

1
_
N
N

i =1
g (Z
i
,
0
,
+
)
_
+o
p
(1)
where g (Z
i
,
0
,
+
) = s (Z
i
,
0
,
+
) +F
0

(Z
i
) .
Note that
E[g (Z
i
,
0
,
+
)] = 0
In this case
Avar
_
N
_

0
_
= H
+1
0
DH
+1
0
where
D = E
_
g (Z
i
,
0
,
+
) g (Z
i
,
0
,
+
)
/
_
= Var [g (Z
i
,
0
,
+
)] .
Cristine Campos de Xavier Pinto Institute
Estimadores Extremos
Examples Consistency Asymptotic Normality Two-Step Estimators Estimating the Asymptotic Variance Hypothesis Testing
Lets consider the case when we do not have a nuisance
parameter () ,
_
N
_

0
_

d
A
_
0, H
1
0
H
1
0
_
In this case, we need to get an estimator for H and .
Under some regularity conditions that ensures uniform
convergence of the matrix of second derivatives
(Condition (iv))
1
N
N

i =1
H
_
Z
i
,

_

p
H
0
Advantage: Always available in problems with twice
continuously dierentiable functions.
Drawbacks: Requires calculation of second derivatives and it
is NOT guaranteed to be positive semidenite.
Cristine Campos de Xavier Pinto Institute
Estimadores Extremos
Examples Consistency Asymptotic Normality Two-Step Estimators Estimating the Asymptotic Variance Hypothesis Testing
If we have more information about the structure of our
problem, we can use a dierent estimator. Suppose that we
can partition Z into X and Y, and that
0
indexes some
feature of the distribution of Y given X. Dene
A(X,
0
) = E[ H (Z,
0
)[ X]
A(X,
0
) is a function of X, and by the law of iterated
expectation
E[A(X,
0
)] = E[H (Z,
0
)] = H
0
Under standard regularity conditions
1
N
N

i =1
A
_
X
i
,

_

p
H
0
Cristine Campos de Xavier Pinto Institute
Estimadores Extremos
Examples Consistency Asymptotic Normality Two-Step Estimators Estimating the Asymptotic Variance Hypothesis Testing
Getting an estimator for is easy
1
N
N

i =1
s
_
Z
i
,

_
s
_
Z
i
,

_/

p

Combining these estimators, we can consistently estimate
Avar
_
N
_

0
_
[
Avar
_
N
_

0
_
=

H
1

H
1
Cristine Campos de Xavier Pinto Institute
Estimadores Extremos
Examples Consistency Asymptotic Normality Two-Step Estimators Estimating the Asymptotic Variance Hypothesis Testing
The asymptotic standard error are obtained from a matrix
[
Avar
_

_
=

H
1

H
1
N
which can be expressed as
_
1
N
N

i =1
H
_
Z
i
,

_
_
1
_
1
N
N

i =1
s
_
Z
i
,

_
s
_
Z
i
,

_/
_
_
1
N
N

i =1
H
_
Z
i
,

_
_
1
or
_
1
N
N

i =1
A
_
X
i
,

_
_
1
_
1
N
N

i =1
s
_
Z
i
,

_
s
_
Z
i
,

_/
_
_
1
N
N

i =1
A
_
X
i
,

_
_
1
These expressions are both at least positive semidenite
Cristine Campos de Xavier Pinto Institute
Estimadores Extremos
Examples Consistency Asymptotic Normality Two-Step Estimators Estimating the Asymptotic Variance Hypothesis Testing
In the case of a two-step estimator, we may need to adjust for
the estimating error in the rst stage.
If E[\

s (Z,
0
,
+
)] = 0, the estimator will be the same as
above but with H
_
Z
i
,

,
_
or A
_
X
i
,

,
_
, and
s
_
Z
i
,

,
_
.
Cristine Campos de Xavier Pinto Institute
Estimadores Extremos
Examples Consistency Asymptotic Normality Two-Step Estimators Estimating the Asymptotic Variance Hypothesis Testing
If E[\

s (Z,
0
,
+
)] ,= 0, the asymptotic variance estimator
of

need to be adjusted, taking into account the asymptotic
variance of .
In this case, we can estimate H using H
_
Z
i
,

,
_
or
A
_
X
i
,

,
_
, but we need to estimate D.
To get an estimator for D, rst we need to estimate F
0
. We
can use

F =
1
N
N

i =1
\

s
_
Z
i
,

,
_
Then,

D =
1
N
N

i =1
g
_
Z
i
,

,
_
g
_
Z
i
,

,
_/
where g
_
Z
i
,

,
_
= s
_
Z
i
,

,
_
+

F

(Z
i
, ) .
Cristine Campos de Xavier Pinto Institute
Estimadores Extremos
Examples Consistency Asymptotic Normality Two-Step Estimators Estimating the Asymptotic Variance Hypothesis Testing
Wald Test
Wald test is easy if you know the form of asymptotic variance.
To test Q restrictions:
H
0
: c (
0
) = 0
we can form the Wald statistics
W = c
_

_/ _

C

V

C
/
_
1
c
_

_
where

V is an asymptotic variance estimator of

,
C = C
_

_
, where C
_

_
is the QxK matrix of rst
derivatives (Jacobian) of c () .
If

V is a robust estimator of the variance,
0
interior(),
C (
0
) = \

c (
0
) has full-rank
W ~ A
2
K
under H
0
Cristine Campos de Xavier Pinto Institute
Estimadores Extremos
Examples Consistency Asymptotic Normality Two-Step Estimators Estimating the Asymptotic Variance Hypothesis Testing
Lagrange Multiplier
Only requires estimation under the null. If the unrestricted
model is dicult to estimate, LM is a good option.
Assume that there Q continuously dierentiable restrictions
imposed on
0
under H
0
.
Assume that the restrictions dene a mapping from
h : R
KQ
R
Q
. Under the null

0
= h (
0
) , where
0
is (K Q) x1 and
0
is Kx1
We need to assume that
0
is in the interior of its parameter
space (), under H
0
.
Assume that h is twice continuously dierentiable on the
interior of .
Cristine Campos de Xavier Pinto Institute
Estimadores Extremos
Examples Consistency Asymptotic Normality Two-Step Estimators Estimating the Asymptotic Variance Hypothesis Testing
Let

be the solution of the constrained minimization problem
min

i =1
q (Z
i
, d ())
The constraint estimator of
0
is simply

= h
_

_
.
The LM statistics is based on the limiting distribution of

N
i =1
s
i
_

_
_
N
under H
0
If

is replaced by

, this statistics is equal to zero.
Cristine Campos de Xavier Pinto Institute
Estimadores Extremos
Examples Consistency Asymptotic Normality Two-Step Estimators Estimating the Asymptotic Variance Hypothesis Testing
Under the given assumptions,
_
N
_

0
_
= O
p
(1)
Using the Delta Method,
_
N
_

0
_
= O
p
(1)
A standard mean value expansion
1
_
N
N

i =1
s
i
_

_
=
1
_
N
N

i =1
s
i
(
0
) +H
_
N
_

0
_
+o
p
(1)
under H
0
.
Cristine Campos de Xavier Pinto Institute
Estimadores Extremos
Examples Consistency Asymptotic Normality Two-Step Estimators Estimating the Asymptotic Variance Hypothesis Testing
Lets play with the restrictions:
0 =
_
Nc
_

_
=
_
Nc (
0
) +C
+
_
N
_

0
_
where C
+
is the Jacobian matrix evaluated at mean value
between

and
0
.
Under H
0
,
c (
0
) = 0 and p limC
+
= C (
0
) = C
Under H
0
,
C
_
N
_

0
_
= o
p
(1)
and
CH
1
1
_
N
N

i =1
s
i
_

_
= CH
1
1
_
N
N

i =1
s
i
(
0
) +o
p
(1)
Cristine Campos de Xavier Pinto Institute
Estimadores Extremos
Examples Consistency Asymptotic Normality Two-Step Estimators Estimating the Asymptotic Variance Hypothesis Testing
We know that by the CLT
CH
1
1
_
N
N

i =1
s
i
(
0
)
d
A
_
0, CH
1
H
1
C
_
where
= E
_
s (Z
i
,
0
) s (Z
i
,
0
)
/
_
Cristine Campos de Xavier Pinto Institute
Estimadores Extremos
Examples Consistency Asymptotic Normality Two-Step Estimators Estimating the Asymptotic Variance Hypothesis Testing
Under the assumptions we impose, CH
1
H
1
C has full
rank,
_
N

i =1
s
i
_

_
_/
H
1
C
/
_
CH
1
H
1
C

1
CH
1
_
N

i =1
s
i
_

_
_

p
A
2
Q
The score and LM statistics is
LM =
1
N

_
N

i =1
s
i
_

_
_
/

H
1

C
/
_

C

H
1

H
1

C
_
1

C

H
1
_
N

i =1
s
i
_

_
_
where all the estimated values are evaluated at

.
Under H
0
, LM
p
A
2
Q
Cristine Campos de Xavier Pinto Institute
Estimadores Extremos
Examples Consistency Asymptotic Normality Two-Step Estimators Estimating the Asymptotic Variance Hypothesis Testing
Criterion Function Statistics
Both the restricted and unrestricted models are easy to
estimate.
In the case of the two-step estimators, we have to assume
that has no eect on the asymptotic distribution of the
M-estimator.
Lets consider the case in which
E
_
s (Z
i
,
0
) s (Z
i
,
0
)
/
_
= E[H (Z, )] .
Note that
Q
N
_

_
Q
N
_

_
=
N

i =1
q
_
Z
i
,

i =1
q
_
Z
i
,

_
Cristine Campos de Xavier Pinto Institute
Estimadores Extremos
Examples Consistency Asymptotic Normality Two-Step Estimators Estimating the Asymptotic Variance Hypothesis Testing
Doing a second order expansion,
N

i =1
q
_
Z
i
,

i =1
q
_
Z
i
,

_
=
N

i =1
s
_
Z
i
,

_ _

_
+
1
2
_

_/
_
N

i =1
H
++
i
_
_

_
where H
++
i
is a Hessian evaluated at means value between

and

.
Under H
0
,

N
i =1
H
++
i
N
= H +o
p
(1) and
_
N
_

_
= O
p
(1) ,
2
_
N

i =1
q
_
Z
i
,

i =1
q
_
Z
i
,

_
_
=
_
N
_

_/
H
_
N
_

_
+o
p
(1)
Cristine Campos de Xavier Pinto Institute
Estimadores Extremos
Examples Consistency Asymptotic Normality Two-Step Estimators Estimating the Asymptotic Variance Hypothesis Testing
From before, we know that
_
N
_

_
= H
1

N
i =1
s
_
Z
i
,

_
_
N
+o
p
(1)
Using these two equations,
QLR = 2
_
N

i =1
q
_
Z
i
,

i =1
q
_
Z
i
,

_
_
=
_
_

N
i =1
s
_
Z
i
,

_
_
N
_
_
/
H
1
_
_

N
i =1
s
_
Z
i
,

_
_
N
_
_
+o
p
(1)
Under H
0
, QLR A
2
Q
Cristine Campos de Xavier Pinto Institute
Estimadores Extremos
Examples Consistency Asymptotic Normality Two-Step Estimators Estimating the Asymptotic Variance Hypothesis Testing
Local Alternatives
So far, we only derive the limiting of the statistics under the
null hypothesis.
We need to know the behave under alternative hypothesis in
order to choose the test with the highest power.
Local Alternative: is a hypothesis under which we can
approximate the nite sample power of test statistics for
alternatives "close" to H
0
.
If H
0
: c (
0
) = 0, then a sequence of local alternatives is
H
N
1
= c (
0,N
) =

0
_
N
where
0
is a given Qx1 vector.
Each of the statistics have a well-dened limiting distribution
under the alternative that diers from the limiting under H
0
.
Cristine Campos de Xavier Pinto Institute
Estimadores Extremos
Examples Consistency Asymptotic Normality Two-Step Estimators Estimating the Asymptotic Variance Hypothesis Testing
Under the local alternative (under some regularity conditions),
Wald and LM statistics have a limiting noncentral chi-squared
distribution with Q degrees of freedom.
The noncentral parameter depends on C, H , and
0
.
For various
0
, we can estimate the asymptotic local power of
the test statistics.
We can compare the test statistics based, using the power
under local alternatives.
Cristine Campos de Xavier Pinto Institute
Estimadores Extremos
Examples Consistency Asymptotic Normality Two-Step Estimators Estimating the Asymptotic Variance Hypothesis Testing
References
Amemya: 4
Wooldridge, 12
Newey, W. and D. McFadden (1994). "Large Sample
Estimation and Hypothesis Testing", Handbook of
Econometrics, Volume IV, chapter 36.
Cristine Campos de Xavier Pinto Institute
Estimadores Extremos