Scribd
Upload
3 views

sml

The Acceptance-Rejection Method is a technique used to generate random variables from a difficult density function by using an easier proposal density. It involves finding a constant C to ensure the difficult density is bounded by C times the proposal density, and outputs a random variable based on acceptance criteria. The efficiency of the method is determined by the expected number of trials needed to achieve one acceptance, which is influenced by how closely the proposal density approximates the target density.

Uploaded by

yeonni42
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
3 views

sml

The Acceptance-Rejection Method is a technique used to generate random variables from a difficult density function by using an easier proposal density. It involves finding a constant C to ensure the difficult density is bounded by C times the proposal density, and outputs a random variable based on acceptance criteria. The efficiency of the method is determined by the expected number of trials needed to achieve one acceptance, which is influenced by how closely the proposal density approximates the target density.

Uploaded by

yeonni42
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
You are on page 1/ 49

IEOR 4404: Simulation

Lecture 4: Acceptance-Rejection
Method
Acceptance Rejection Method
A method to generate “difficult” r.v. from “easy” ones.

• We want to generate a r.v. with density 𝑓 𝑥 (difficult)

• We can generate a r.v. with density 𝑔(𝑥) (easy)


Acceptance Rejection Method
Acceptance rejection method:
• Find a constant 𝐶 such that
𝑓 𝑥
≤ 𝐶 for all possible 𝑥
𝑔 𝑥
• To generate one output,
– Generate 𝑌 ∼ 𝑔(𝑦)
𝑓 𝑌
– “Accept” 𝑌 with probability 𝐶𝑔 𝑌 ; otherwise “reject” and go back to the
beginning

• The r.v. from 𝑔(𝑥) is called the proposal r.v.


Acceptance Rejection Method
Acceptance rejection method:
• Find a constant 𝐶 such that
𝑓 𝑥
≤ 𝐶 for all possible 𝑥
𝑔 𝑥
• To generate one output,
– Generate 𝑌 ∼ 𝑔(𝑦)
– Generate 𝑈 ∼ 𝑈𝑛𝑖𝑓(0,1)
𝑓 𝑌
– If 𝑈 ≤ 𝐶𝑔 𝑌 , set 𝑋 = 𝑌 (accept); otherwise, go back to the beginning
(reject)
– Output 𝑋
Example
• Generate a r.v. with pdf
𝑓 𝑥 = 3𝑥 2 , 0≤𝑥≤1
• Use a 𝑈𝑛𝑖𝑓(0,1) proposal
𝑔 𝑥 = 1, 0≤𝑥≤1
Example
• Generate a r.v. with pdf
𝑓 𝑥 = 3𝑥 2 , 0≤𝑥≤1
• Use a 𝑈𝑛𝑖𝑓(0,1) proposal
𝑔 𝑥 = 1, 0≤𝑥≤1
• We have
𝑓 𝑥 3𝑥 2
= = 3𝑥 2 ≤ 3 for any 0 ≤ 𝑥 ≤ 1
𝑔 𝑥 1
• Thus we can choose 𝐶 = 3
Example
Acceptance-rejection method:
• Generate 𝑌 ∼ 𝑈𝑛𝑖𝑓(0,1)
• Generate 𝑈 ∼ 𝑈𝑛𝑖𝑓(0,1)
𝑓 𝑌
• If 𝑈 ≤ = 𝑌 2 , set 𝑋 = 𝑌 (accept); otherwise, go back to
𝐶𝑔 𝑌
the beginning (reject)
• Output 𝑋
Example
• Generate the standard normal r.v.

1 𝑥2
𝑓 𝑥 = 𝑒− 2 , −∞ < 𝑥 < ∞
2𝜋

• Use a proposal

1 −𝑥
𝑔 𝑥 = 𝑒 , −∞ < 𝑥 < ∞
2
Implementation
In the first example, suppose the sequence of Unif(0,1) generated by the
computer is:

0.10 0.32 0.50 0.49 0.23 0.37 0.58 0.32 0.39 0.99
Implementation
In the first example, suppose the sequence of Unif(0,1) generated by the
computer is:

0.10 0.32 0.50 0.49 0.23 0.37 0.58 0.32 0.39 0.99

The A-R algorithm runs as:


Y=0.10
U=0.32
Since 𝑈 = 0.32 > 𝑌 2 = 0.01, continue
Y=0.50
U=0.49
Since 𝑈 = 0.49 > 𝑌 2 = 0.25, continue
Y=0.23
U=0.37
Since 𝑈 = 0.37 > 𝑌 2 = 0.0529, continue
Y=0.58
U=0.32
Since 𝑈 = 0.32 ≤ 𝑌 2 = 0.3364, stop
Output = 0.58
Intuition
Motivation:
• Since 𝑓(𝑥) is difficult to sample from, we want to use
an easier density 𝑔(𝑥)
• However, we cannot just change the problem to
sampling from 𝑔(𝑥)

Mechanism:
• 𝐶𝑔(𝑥) is an “envelope” that covers 𝑓(𝑥)
• 𝐶 must be at least 1; if 𝐶 = 1, then 𝑓 ≡ 𝑔
• The acceptance probability 𝑓(𝑥)/𝐶𝑔(𝑥) is an
adjustment from sampling 𝑔(𝑥) to sampling 𝑓(𝑥)
Validity
The A-R algorithm indeed generates a random variable with
density 𝑓(𝑥)

First note that


𝑃(𝑎𝑐𝑐𝑒𝑝𝑡)
∞
= න 𝑃 𝑎𝑐𝑐𝑒𝑝𝑡|𝑌 = 𝑥 𝑔 𝑥 𝑑𝑥
−∞
∞
𝑓 𝑥
=න × 𝑔 𝑥 𝑑𝑥
−∞ 𝐶𝑔 𝑥
∞
‫׬‬−∞ 𝑓 𝑥 𝑑𝑥
=
𝐶
1
=
𝐶
where we use the law of total probability in the first equality
Validity
Next consider

𝑃 𝑋 ∈ 𝑥, 𝑥 + Δ𝑥 𝑎𝑐𝑐𝑒𝑝𝑡
𝑃 𝑋 ∈ 𝑥, 𝑥 + Δ𝑥 , 𝑎𝑐𝑐𝑒𝑝𝑡
=
𝑃 𝑎𝑐𝑐𝑒𝑝𝑡
𝑃 𝑌 ∈ 𝑥, 𝑥 + Δ𝑥 × 𝑃 𝑎𝑐𝑐𝑒𝑝𝑡|𝑌 ∈ 𝑥, 𝑥 + Δ𝑥
=
𝑃 𝑎𝑐𝑐𝑒𝑝𝑡
𝑓 𝑥
𝑔 𝑥 Δ𝑥 ×
𝐶𝑔 𝑥
=
1
𝐶
= 𝑓 𝑥 Δ𝑥
Efficiency
• The runtime of the algorithm depends on the number
of trials to obtain one acceptance
• Since the algorithm requires a random number of
trials, it makes sense to consider the expected number
of trials as an efficiency measurement.
• The number of trials to get one acceptance is Geo(p),
where p is the acceptance probability
• The acceptance probability is p=1/C, where C is the
constant that bounds 𝑓(𝑥)/𝑔(𝑥)
• In other words, the expected number of trials to get
one acceptance is 1/p=C
Efficiency
• If 𝑔(𝑥) is close to 𝑓(𝑥), then C is small, so p is large,
and expected number of trials is small
• If 𝑔 ≡ 𝑓, then C=1, so that p=1 and we finish the
random variable generation always in one trial
How to Choose Proposal
• Before we even consider efficiency, we have to make
sure the basic requirement of 𝑔(𝑥) holds
• Requirement 1: 𝑔(𝑥) should have at least the same
support as 𝑓(𝑥)
How to Choose Proposal
• Requirement 2: 𝑔(𝑥) should have a heavier tail than
𝑓(𝑥) in the case that 𝑓(𝑥) has unbounded support
How to Choose Proposal
Most generally, we must be able to find 𝐶 that is finite;
otherwise, the A-R procedure fails
How to Choose Proposal
Can we use the following to generate Exp(1), i.e., with
density 𝑓 𝑥 = 𝑒 −𝑥 for 𝑥 > 0 and 0 otherwise:
1 −𝑥
• Two-sided exponential, i.e., 𝑔 𝑥 = 2
𝑒 , −∞ < 𝑥 < ∞
• Unif(0,1), i.e., 𝑔 𝑥 = 1 for 0 < 𝑥 < 1 and 0 otherwise

Can we use the following to generate N(0,1), i.e., with density


2
𝑥
1 −2
𝑓 𝑥 = 2𝜋
𝑒 , −∞ < 𝑥 < ∞:
• Exp(1), i.e., 𝑔 𝑥 = 𝑒 −𝑥 for 𝑥 > 0 and 0 otherwise
− 𝑥 3
• Density 𝑔 𝑥 = 𝐾𝑒 for some 𝐾 > 0

You might also like

pFad - Phonifier reborn

Pfad - The Proxy pFad of © 2024 Garber Painting. All rights reserved.

Note: This service is not intended for secure transactions such as banking, social media, email, or purchasing. Use at your own risk. We assume no liability whatsoever for broken pages.


Alternative Proxies:

Alternative Proxy

pFad Proxy

pFad v3 Proxy

pFad v4 Proxy