Invariant density estimation.

Let us introduce the local time estimator

T
(x) =

T
(x)
T (x)
2
where the local time

T
(x) = |X
T
x| |X
0
x|
_
T
0
sgn(X
t
x) dX
t
We have
f

T
(x) f
S
(x) ,

T (f

T
(x) f
S
(x)) =N
_
0, I
f
(S, x)
1
_
1
Here
I
f
(S, x) =
_
4 f
S
(x)
2
E
S
_
1I
{>x}
F()
()f()
_
2
_
1
.
This asymptotic normality follows from the representation

T
_
f

T
(x) f
S
(x)
_
= 2
f(x)

T
_
X
T
X
0
_
1I
{v>x}
F(v)
(v)
2
f(v)
_
dv
2
f(x)

T
_
T
0
_
1I
{X
t
>x}
F(X
t
)
(X
t
) f(X
t
)
_
dW
t
.
2
We have the similar lower bound
lim
0
lim
T
sup
S()V

T E
S
_

f
T
(x) f
S
(x)
_
2
I
f
(S

, x)
1
.
The proof is based on the estimate
sup
S()V

E
S
_

f
T
(x) f
S
(x)
_
2
sup

f
T
(x) f

(x)
_
2
with the same parametric family but with dierent parametrization
providing f

(x) = +o (1). The minimization

= I
f
(S

, x)
1
3
As before we introduce the parametric family of functions
S

(x) = S

(x) + (

) (x)(x)
2
, I

= E

()
2
()
2
,
where = (

+), > 0 and the function () is

() + (

)()()
2
V

for all
.
V

=
_
S () : sup
x
|S (x) S

(x)|
_
.
Set
f

(x) = G()
1
(x)
2
exp
_
2
_
x
0
S

(y)
(y)
2
dy + 2(

)
_
x
0
(y) dy
_
.
4
and obtain
f

(x) = f
S

(x) + 2 (

)f
S

(x) E
S

_
x

(v) dv +o(

),
Set

= f
S

(x) and introduce the following class of functions:

_
x

(v) dv = (2 f
S

(x))
1
_
.
Then for () K we have the expansion f

(x) = +o(

).
Repeating the same arguments as befor we obtain the HajekLe Cam
inequality
lim
0
lim
T
inf

T
sup
|

|<
E

_
T
1/2
(

T
)
_
2
I
1

.
inf
()K
I

= I
f
(S

, x)
1
5
E
S

_
x

(v) dv =
_

f(y)
_
x
y
(v) dv dy
=
_
x

_
x
y
f(y) (v) dy dv +
_

x
_
x
y
f(y) (v) dv dy
=
_
x

_
x

f(y) (v) 1I
{v>y}
dy dv

_

x
_

x
f(y) (v) 1I
{vy}
dv dy
=
_
x

(v) F(v) dv
_

x
(v) (1 F(v)) dv
=
_

(v)[F(v) 1I
{v>x}
] dv.
6
Thus by the CauchySchwarz inequality
(2 f(x))
2
=
__

(v)[F(v) 1I
{v>x}
] dv
_
2

(v)
2
(v)
2
f(v) dv
_

_
F(v) 1I
{v>x}
_
2
(v)
2
f(v)
dv
= E()
2
()
2
E
_
F() 1I
{>x}
() f()
_
2
.
Therefore
I

_
4 f(x)
2
E
_
1I
{>x}
F()
() f()
_
2
_
1
= I
f
(S

, x)
for all () K .
7
Weak Convergence of the LTE
Introduce the random function
T
(x) =

T (f

T
(x) f(x)) , x IR
and denote by () = {(x), x IR} the Gaussian process with mean
0 and the covariance function
R
S
(x, y) = 4f
S
(x) f
S
(y) E
S
_
[
{>x}
F
S
()][
{>y}
F
S
()]
()
2
f
S
()
2
_
.
Suppose that
dX
t
= S (X
t
) dt + dW
t
, X
0
, 0 t T
Theorem 1 Let S () S

and S () P. Then the stochastic

(R) and put

(y) (y)
2
(x)
2
h(x)
.
Introduce the estimator
f

T
(x) =
1
T
_
T
0
R
x
(X
t
) dX
t
+
1
T
_
T
0
N
x
(X
t
) dt.
9
Then E
S
f

T
(x) = f
S
(x) and
L
S
_

T
_
f

T
(x) f
S
(x)
__
N
_
0, I
f
(S, x)
1
_
.
The proof is based on

T
_
f

T
(x) f
S
(x)
_
=

T
_
f

T
(x) f
S
(x)
_
+o (1).
In particular, if (x) 1 and h(x) = x
3
then
f

T
(x) =
2
Tx
3
_
T
0
1I
{X
t
<x}
X
3
t
dX
t
+
+
3
Tx
3
_
T
0
1I
{X
t
<x}
X
2
t
dt
is unbiased and asymptotically ecient estimator of the density.
10
Kernel-type Estimator.

f
T
(x) =
1

T
_
T
0
Q
_

T(X
t
x)
_
dt
where the kernel Q() is a bounded function with compact support
[A, B] and
_
B
A
Q(u) du = 1,
_
B
A
u Q(u) du = 0.
We have

T
_

f
T
(x) f(x)
_
=

T
_

T
(x)
T(x)
2
f(x)
_
+o (1) =
=

T (f

T
(x) f (x)) +o (1) .
11
Let

(L) be the class of dierentiable functions such that the

() is Holder of order , = 1 +.
C. The functions S() and () satisfy the global Lipschitz condition,
G(S) < and EX
2
0
< .
Theorem. Let the condition C be fullled, I
f
(S, x) > 0 and the
function f()

(L) then for every x IR

T
_

f
T
(x) f
S
(x)
__
N
_
0, I
f
(S, x)
1
_
.
12
Semiparametric Estimation
The observations are
dX
t
= S(X
t
) dt +(X
t
) dW
t
, X
0
, 0 t T
Put

S
= E
S
R() S () +E
S
N ()
and consider the problem of its estimation.
Fisher information
I

(S) =
_
_
_
E
S
_
R() ()
2
f
S
() + 2M
S
()
() f
S
()
_
2
_
_
_
1
where
M
S
(y) = E
__
F
S
(y)
{<y}
_
[R() S () +N ()]
_
13
Introduce the set
V

= {S() : sup
xIR
|S(x) S

(x)| }
Theorem. Let
sup
SV

G(S) < , I

(S

) > 0
then for all estimators

T
lim
0
lim
T
sup
S()V

T E
S
_

T

S
_
2
I

(S

)
1
.
14
Empirical estimator

T
=
1
T
_
T
0
R(X
t
) dX
t
+
1
T
_
T
0
N (X
t
) dt
under moments condition is asymptotically normal:
L
S
_

T
_

T

S
__
N
_
0, I

(S)
1
_
.
and asymptotically ecient:
lim
0
lim
T
sup
S()V

T E
S
_

T

S
_
2
= I

(S

)
1
.
15
Distribution function F (x): Put R(y) 0 and N (y) =
{y<x}
then = F (x) and

T
=

F
T
(x)
Density f (x): Put R(y) = (x)
2
sgn(x y) and N (y) 0 then
= f (x) and

T
=

f
T
(x) =
1
T (x)
2
_
T
0
sgn(x X
t
) dX
t
Moment = E
S

k
: Put R(y) = 0 and N (y) = y
k
then

T
=
1
T
_
T
0
X
k
t
dt
16
Integral-type Risk
Let us put
R
_

f
T
, f
S
_
= E
S
_
(

f
T
(x) f
S
(x))
2
dx
R
f
(S) =
_
I
f
(S

, x)
1
dx
Theorem. For all estimators

f
T
(x)
lim
0
lim
T
sup
S()V

T R
_

f
T
, f
S
_
R
f
(S

) .
The proof is based on the estimate
sup
S()V

R
_

f
T
, f
S
_
sup

R
_

f
T
, f

_
17
The parametric problem corresponds to
S (, x) = S

(x) +
k

i=1

i
(x) (x)
2
V

with least favorable

i
(y) =
1
(y)
_
IR
_
R(y, x) (y) +
2M
S

(y, x)
(y)f(y)
_

i
(x) dx,
and k .
All mentioned above estimators are asymptotically ecient,
i.e.,
lim
0
lim
T
sup
S()V

TR
_

f
T
, f
S
_
= R
f
(S

)
18
Second order ecient estimator
Let us study the quantity
_
TR
_

f
T
, f
S
_
R
f
(S)
_
. Note that for
LTE
T
1
2
_
TR
_
f

T
, f
S
_
R
f
(S)
_
Q = 0
Better: it can be shown that if S () is k 1 times dierentiable then
T
1
2k1
_
TR
_

f
T
, f
S
_
R
f
(S)
_
P < 0.
Why better? Because
TR
_

f
T
, f
S
_
R
f
(S) P T

1
2k1
19
The observed process is
dX
t
= S (X
t
) dt + dW
t
, 0 t T
where S ()

:
lim
|x|
sup
S

sgn(x) S (x) < 0

)
Suppose that the function S () is (k 1)-times dierentiable, x
S

(x) = x and introduce the set

) =
_
S () :
_
_
f
(k)
S
(x) f
(k)
S

(x)
_
2
dx R
_
.
20
Put

(k 1, R, S

)
Theorem. (Dalalyan-K) For all estimators

f
T
()
lim
T
sup
S()

T
1
2k1
_
TR
_

f
T
, f
S
_
R
f
(S)
_

(k, R)
where

(k, R) = 2 (2k 1)
_
4k
(k 1) (2k 1)
_ 2k
2k1
R

1
2k1
.
21
Introduce the localised orthonormal basis of L
2
(IR):
e
l,m
(x) =
1

2
T
exp
_
il(x a
m
+
T
)

T
_

{xI
m
}
.
Above i =

1 and I
m
is the interval of length 2
T
with center
a
m
= 2m
T
. The Fourier coecients of the LTE are

T,l,m
=
_
IR
e
l,m
(x)f

T
(x) dx =
1
T
_
T
0
e
l,m
(X
t
) dt.
Introduce the linear estimator

f
T
(x) =

l,mZ

T,l,m

T,l,m
e
l,m
(x)
22
Put

T,l,m
=
_
1

l

T

k+
T
_
+
,
where
T
= 1/

log T 0 and

T
=
T
_
8k
2(k1)
RT(k 1)(2k 1)
_

1
2k1

Theorem. (Dalalyan-K) Let k > 1. Then

T
1
2k1
_
TR
_

f
T
, f
S
_
R
f
(S)
_
=

(k, R).
23
This second order asymptotically ecient estimator can be written
(for x I
m
) as

f
T
(x) =
1
2T
T
_
T
0

T

l=
T
_
1

l

T

k
T
_
cos
_
l (x X
t
)

T
_

{X
t
I
m
}
dt
or

f
T
(x) =

T

l=
T
_
1

l

T

k
T
_
1
2
T
_
a
m
+
T
a
m

T
cos
_
l (x y)

T
_
f

T
(y) dy
Here k
T
= k +
T
.
24
Trend estimation
The observed process is ergodic diusion
dX
t
= S(X
t
) dt + dW
t
, X
0
, 0 t T
with unknown trend coecient S ().
Risk of an estimator

S
T
()
R
_

S
T
, S
_
= E
S
_
_

S
T
(x) S (x)
_
2
f
S
(x)
2
dx
Fix a central function S

() and dene its -vicinity

=
_
S () : sup
x
|S (x) S

(x)|
_
25
Conditions S

.
S
1
. The function S () P is such that
lim
|x|
sgn(x) S(x) < 0.
S
2
. The function S () C
k
(R) with some k 1 and belongs to the
set

=
_
S() V

:
_
IR
_
f
(k+1)
S
(x) f
(k+1)
S

(x)
_
2
dx 4 R
_
.
S
3
. The Fourier transform

() of the function f

() is such that
_
IR
||
2k+

()

2
d <
with some positive constant .
26
Let us put
(k, R) = (2k + 1)
_
k
(k + 1) (2k + 1)
_ 2k
2k+1
R
1
2k+1
Theorem. (Dalalyan-K) Let the conditions S

be fullled then
lim
T
inf

S
T
sup
S()

T
2k
2k+1
R
_

S
T
, S
_
(k, R) .
27
Remind that
S(x) =
f

S
(x)
2f
S
(x)
and introduce the estimator

S
T
(x) =

T
(x)
2 f

T
(x) +
T
e
l
T
|x|
where f

T
(x) is the local-time estimator of the density,

T
= T
(1)/2
, l
T
= [ln T]
1
, the constant < 1/ (2k + 1) and

T
(x) is asymptotically ecient estimator of the derivative f

S
(x).
28
The estimator of the derivative is

T
(x) =
2
T
T
_
T
0
K

T
(x X
t
)
_
dX
t
where the kernel
K

(x) =
1

_
1
0
(1 u
k+
T
) cos(ux) du
and

T
=
_
R (k + 1) (2k + 1)
4k
_ 1
2k+1
T
1
2k+1
.
Here
T
= (log T)
1/2
.
29
Theorem. (Dalalyan-K) Let the conditions S

be fullled then
lim
T
sup
S()

T
2k
2k+1
R
_

S
T
, S
_
= (k, R) .
If the values k 2 and R > 0 are unknown then it is possible to
construct an adaptive estimator

S
T
(), which has the same
asymptotic properties as

S
T
().
Theorem. (Dalalyan) Let the conditions S

be fullled then
lim
T
sup
S()

T
2k
2k+1
R
_

S
T
, S
_
= (k, R) .
30