Scribd
Upload
188 views

Pstat 160 A Homework 6

This document contains instructions for Homework 6 in PSTAT 160A, which is due on December 4, 2014. It includes 4 problems involving Markov chains. Problem 1 involves a random walk with reflecting barriers, computing expected return times. Problem 2 examines the interarrival times of buses and waiting times, computing limiting distributions. Problem 3 analyzes a random walk on non-negative integers with a reflecting barrier at 0. Problem 4 classifies states and computes invariant distributions and limiting averages for a Markov chain with 5 states. Students are instructed to show their work and include their name on submitted pages.

Uploaded by

lex7396
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
188 views

Pstat 160 A Homework 6

This document contains instructions for Homework 6 in PSTAT 160A, which is due on December 4, 2014. It includes 4 problems involving Markov chains. Problem 1 involves a random walk with reflecting barriers, computing expected return times. Problem 2 examines the interarrival times of buses and waiting times, computing limiting distributions. Problem 3 analyzes a random walk on non-negative integers with a reflecting barrier at 0. Problem 4 classifies states and computes invariant distributions and limiting averages for a Markov chain with 5 states. Students are instructed to show their work and include their name on submitted pages.

Uploaded by

lex7396
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
You are on page 1/ 2

Homework 6

PSTAT 160A FALL 2014

D UE T HURSDAY, D EC 4, 2014

(1) Let (Xn ) be a random walk on {0, 1, . . . , M } with two reflecting barrier at 0 and at M > 1. That is,
the transition matrix of (Xn ) is

Pi,j

1 p
=

if 0 < i < M and j = i + 1,


if 0 < i < M and j = i 1,
if i = M and j = M 1 or i = 0 and j = 1,
otherwise.

(a) Draw the transition graph for M = 3.


(b) Compute the invariant distribution for arbitrary M using the weighted graph method.
(c) Let T = inf {n 1 : Xn = 0} be the first return time to 0. Compute E [T | X0 = 0] and
E [T | X0 = 1].
(2) Let Z1 , Z2 , . . . be independent identically distributed, positive integer valued random variables.
These are the interarrival times of a bus service at a given bus stop. So, there is a bus at time 0,
the next bus arrives at Z1 , the next one at Z1 + Z2 and so on. Denote Un the time elapsed since the
last bus at time n and Vn the waiting time for the next bus at time n. The aim of this exercise is the
computation of the PMF Un + Vn for large n.
(a) What is the state space of the Markov chain Xn = (Un , Vn ), n 1?
(b) What are the transition probabilities of the Markov chain (Xn )n1 ?
(c) Is (Xn ) irreducible?
(d) What is the invariant distribution of (Xn ) in terms of the PMF of Z1 ?
(e) Compute
n
1X
P [Uk + Vk = j ] .
lim
n n
k=1
(f) If the law of Z1 is a so called non-lattice distribution, that is, the gratest common divisor of
{n : P [Z1 = n] > 0} is 1, then Xn is aperiodic. What is
lim P [Un + Vn = j ]

in this case?
(3) A particle performs a random walk on the non-negative integers {0, 1, . . .} as follows. At each step,
it moves right with probability p and left with probability q = 1 p, except that, if it is at 0 now,
it moves right with probability p and remains at 0 with probability q. This called a simple random
walk on D = {0, 1, 2, . . .} with a (not instanteneously) reflecting barrier at 0.
Let Xn be the particles position after the nth step. Then, (Xn )n0 is a Markov chain with state space
D.
(a) Write down the transition probabilities Pi,j = P [Xn+1 = j | Xn = i].
(b) Classify the states when p > q.
(c) Classify the states when p < q.
(d) Find the invariant distribution = (i )i0 when p < q.
(e) Let T = min {n 0 : Xn = 0}. Compute E [T | X0 = i] for i 1, when p < q.
Hint. What is the relationship between E [T | X0 = i] and E [T | X0 = 1]?

(4) Let (Xn )n0 be a Markov chain with state space D = {a, b, c, d, e} and transition matrix

0 0 0 0 1
1 0 0 0 0

1
1
P =
/2 /2 0 0 0
1/3 1/3 1/3 0 0
1/4 1/4 1/4 1/4 0
(a) Classify the states.
(b) Compute the invariant distribution = (a , b , c , d , e ).
(c) Let T = min {n 1 : Xn is b or c}. Compute

" T 1
#

X

E
f (Xn ) X0 = a .

n=0

for f with f (a) = 2, f (d) = 3, f (e) = 1.


(d) Define the reward function g by

2 if i = a and j = e,
g(i, j) = 3 if i = e and j = a,

0 otherwise.
Compute
n

1X
lim
g(Xk1 , Xk ).
n n
k=1

Each problem is worth 25 points. Write your name at the top right corner of each page you submit.
2

You might also like

pFad - Phonifier reborn

Pfad - The Proxy pFad of © 2024 Garber Painting. All rights reserved.

Note: This service is not intended for secure transactions such as banking, social media, email, or purchasing. Use at your own risk. We assume no liability whatsoever for broken pages.


Alternative Proxies:

Alternative Proxy

pFad Proxy

pFad v3 Proxy

pFad v4 Proxy