Scribd Logo
Upload

Welcome to Scribd!

  • 22 views

    Asymptotic Variance

    Uploaded by

    SiyaB
    The document discusses the asymptotic variance of maximum likelihood estimators (MLEs). It states that for large sample sizes, the MLE will approximate a normal distribution centered around the true parameter value with variance equal to the inverse of the expected information. The expected information can be defined using first and second derivatives of the log-likelihood function. Two examples are then presented to demonstrate calculating the asymptotic variance of MLEs under different probability distributions.

    Copyright:

    © All Rights Reserved
    Download as PDF, TXT or read online from Scribd

    Asymptotic Variance

    Uploaded by

    SiyaB
    22 views6 pages
    The document discusses the asymptotic variance of maximum likelihood estimators (MLEs). It states that for large sample sizes, the MLE will approximate a normal distribution centered around the true parameter value with variance equal to the inverse of the expected information. The expected information can be defined using first and second derivatives of the log-likelihood function. Two examples are then presented to demonstrate calculating the asymptotic variance of MLEs under different probability distributions.

    Original Description:

    summary

    Copyright

    © © All Rights Reserved

    Available Formats

    PDF, TXT or read online from Scribd

    Did you find this document useful?

    Is this content inappropriate?

    The document discusses the asymptotic variance of maximum likelihood estimators (MLEs). It states that for large sample sizes, the MLE will approximate a normal distribution centered around the true parameter value with variance equal to the inverse of the expected information. The expected information can be defined using first and second derivatives of the log-likelihood function. Two examples are then presented to demonstrate calculating the asymptotic variance of MLEs under different probability distributions.

    Copyright:

    © All Rights Reserved
    Download as PDF, TXT or read online from Scribd
    Download as pdf or txt
    22 views6 pages

    Asymptotic Variance

    Uploaded by

    SiyaB
    The document discusses the asymptotic variance of maximum likelihood estimators (MLEs). It states that for large sample sizes, the MLE will approximate a normal distribution centered around the true parameter value with variance equal to the inverse of the expected information. The expected information can be defined using first and second derivatives of the log-likelihood function. Two examples are then presented to demonstrate calculating the asymptotic variance of MLEs under different probability distributions.

    Copyright:

    © All Rights Reserved
    Download as PDF, TXT or read online from Scribd
    Download as pdf or txt
    You are on page 1of 6

    lOMoARcPSD|7314911

    Asymptotic variance

    Statistical Inference (California State University, Long Beach)

    Studocu is not sponsored or endorsed by any college or university


    Downloaded by Siyabulela Banzana (ceeyabanz@gmail.com)
    lOMoARcPSD|7314911

    ASYMPTOTIC VARIANCE of the MLE


    Maximum likelihood estimators typically have good properties when the sample size is large.

    Suppose X1 , . . . , Xn are iid from some distribution Fθo with density fθo . We observe data x1 , . . . , xn .
    The Likelihood is:

    Y
    n
    L(θ) = fθ (xi )
    i=1

    and the log likelihood is:

    X
    n
    l(θ) = log[fθ (xi )]
    i=1

    The maximum likelhood estimator maximizes l(·), and is an estimator for θo . Under some
    regularity conditions of fθ , for n large

    1
    θ̂ ≈ N (θo , )
    nI(θo )
    where I(θ) is called the information, and is defined as

    ∂logfθ (X) 2
    I(θ) = E( )
    ∂θ
    Notice that X is capitalized above. It denotes that the expectation is beinf taken with respect
    to X and its distribution. Also note that the derivative is with repect to θ.
    It is often that case that the information also can be expressed as

    ∂ 2 logfθ (X)
    I(θ) = −E( )
    ∂θ2

    For θ̃ any unbiased estmator for θo , we have a lower bound on the variance of θ̃

    1
    V ar(θ̃) ≥
    nI(θo )

    Downloaded by Siyabulela Banzana (ceeyabanz@gmail.com)


    lOMoARcPSD|7314911

    Let’s work some examples.

    1
    Example: X1 , . . . , Xn iid with density fσ (x) = 2σ exp(−|x|/σ).

    Method of Moments
    1. E(X) = 0 by symmetry
    R∞
    E(X 2 ) = 2
    −∞ x fσ (x)dx =

    2. σ =

    3. σ̂ =

    Method of Moments
    1. Likelihood function: L(σ) =

    log likelihood l(σ) =

    2. Differentiate with respect to σ

    ∂logl(σ)
    =
    ∂σ

    3. Set to 0 and solve for σ̃

    Downloaded by Siyabulela Banzana (ceeyabanz@gmail.com)


    lOMoARcPSD|7314911

    Asymptotic Variance of σ̃
    ∂logfσ (x)
    ∂σ =

    ∂ 2 logfσ (x)
    ∂σ 2
    =

    2 logf (x)
    I(σ) = −E[ ∂ σ
    ∂σ 2
    ]=

    The variance of θ̃ is asymptotically equivalent to


    1
    nI(σ) =
    How would we make a confidence interval for σ using σ̂ and this asymptotic variance?

    Downloaded by Siyabulela Banzana (ceeyabanz@gmail.com)


    lOMoARcPSD|7314911

    Example: X1 , . . . , Xn iid with density fθ (x) = (θ + 1)xθ for 0 ≤ θ ≤ 1.

    Method of Moments
    R1
    1. E(X) = 0 (θ + 1)xθ+1 dx

    2. θ =

    3. θ̂ =

    Method of Moments
    1. Likelihood function: L(θ) =

    log likelihood l(θ) =

    2. Differentiate with respect to θ

    ∂logl(θ)
    =
    ∂θ

    3. Set to 0 and solve for θ̃

    Downloaded by Siyabulela Banzana (ceeyabanz@gmail.com)


    lOMoARcPSD|7314911

    Asymptotic Variance of θ̃
    ∂logfθ (x)
    ∂θ =

    ∂ 2 logfθ (x)
    ∂θ2
    =

    2 logf (x)
    I(θ) = −E[ ∂ ∂θ2
    θ
    ]=

    The variance of θ̃ is asymptotically equivalent to


    1
    nI(θ) =

    Downloaded by Siyabulela Banzana (ceeyabanz@gmail.com)

    You might also like

    pFad - Phonifier reborn

    Pfad - The Proxy pFad of © 2024 Garber Painting. All rights reserved.

    Note: This service is not intended for secure transactions such as banking, social media, email, or purchasing. Use at your own risk. We assume no liability whatsoever for broken pages.


    Alternative Proxies:

    Alternative Proxy

    pFad Proxy

    pFad v3 Proxy

    pFad v4 Proxy