1. (15%) Let X(t) be a pure death process starting from X(0) = N .

Assume that the death

parameters are µ1 , µ2 , · · · , µN . Let T be an exponentially distributed random variable
with parameter θ. Show that
N
Y µi
P (X(T ) = 0) = .
i=1 µi + θ

Solution

P (X(T ) = 0) = P (WN ≤ T )
= Ee−θWN
= Ee−θ(S1 +S2 +···+SN )
N
Y
= Ee−θSi
i=1
YN Z ∞
= e−θs µi e−µi s ds
i=1 0
YN
µi
= .
i=1 µi + θ

2. (15%) Let X1 (t), X2 (t) and X3 (t) be independent two-state Markov chains having the
infinitesimal matrix ° °
° −λ λ °
° °
A=° °.
° µ −µ °

Determine the transition probabilities P00 (t) and P33 (t) for X1 (t) + X2 (t) + X3 (t).
Solution: We have the transition probability matrix of X1 (t)(or X2 (t), X3 (t)),
∞
X A n tn
P X (t) = I +
n=1 n!
A A −(λ+µ)t
= I+ − e ,
λ+µ λ+µ
that is,

X µ λ −(λ+µ)t
P00 (t) = + e ;
λ+µ λ+µ
X λ λ −(λ+µ)t
P01 (t) = − e ;
λ+µ λ+µ
X µ µ −(λ+µ)t
P10 (t) = − e ;
λ+µ λ+µ
X λ µ −(λ+µ)t
P11 (t) = + e ,
λ+µ λ+µ
So
Z
P00 (t) = P (Z(t) = 0|Z(0) = 0)
= P (X1 (t) = 0, X2 (t) = 0, X3 (t) = 0|X1 (0) = 0, X2 (0) = 0, X3 (0) = 0)
= P (X1 (t) = 0|X1 (0) = 0)P (X2 (t) = 0|X2 (0) = 0)P (X3 (t) = 0|X3 (0) = 0)
à !3
µ λ −(λ+µ)t
= + e
λ+µ λ+µ

2
 and

P3Z (t) = P (Z(t) = 3|Z(0) = 3)

= P (X1 (t) = 1, X2 (t) = 1, X3 (t) = 1|X1 (0) = 1, X2 (0) = 1, X3 (0) = 1)
= P (X1 (t) = 1|X1 (0) = 1)P (X2 (t) = 1|X2 (0) = 1)P (X3 (t) = 1|X3 (0) = 1)
à !3
λ µ −(λ+µ)t
= + e .
λ+µ λ+µ

3. (15%) Suppose that N (t) is a renewal process with interoccurrence times X1 , X2 , · · ·.

Suppose EX1 = µ and V ar(X1 ) = σ 2 . Let Wn be the waiting time for the n-th renewal.

(a) Find E(W2 W4 ).

(b) Find limt→∞ EWN (t)+1 /t.
(c) Find limt→∞ Eγt /t, where γt is the excess life at t.

Solution:

(a)

E(W2 W4 ) = E[(X1 + X2 )(X1 + X2 + X3 + X4 )]

= E[X12 + X22 + 2X1 X2 + X1 X3 + X1 X4 + X2 X3 + X2 X4 ]
= 2σ 2 + 8µ2 .

(b) because
EWN (t)+1 = EX1 [1 + M (t)]
and
M (t) 1
lim =
t→∞ t EX1
Thus
EWN (t)+1 EX1 [1 + M (t)]
lim = lim = 1.
t→∞ t t→∞ t
(c)
Eγt EWN (t)+1 − t
lim = lim = 0.
t→∞ t t→∞ t
4. (15%) A machine can work for Xi days before it breaks down. When breakdown, it will
be repaired immediately. The repair will take Yi days. After the repair, a battery has to
be recharged. The time for recharging is zero. Suppose the distributions for Xi and Yi
are respectively
P (Xi = 2) = 0.4, P (Xi = 3) = 0.6,
and
P (Yi = 1) = 0.8, P (Yi = 2) = 0.2.
Assume that Xi and Yi are independent. What is the expected number of times that the
battery is recharged within 5 days?
Solution Let Zi be the duration between two successive times that the battery is recharged.
Then
Zi = Xi + Yi . (interoccurrence time)

3
 The distribution is
P (Zi = 3) = 0.4 ∗ 0.8 = 0.32;
P (Zi = 4) = 0.8 ∗ 0.6 + 0.2 ∗ 0.4 = 0.56
P (Zi = 5) = 0.6 ∗ 0.2 = 0.12.

Let N (t) be the number of times that the battery is rechagred in time (0, t].
Let M (t) = E(N (t)). We have
M (1) = p1 = P (Zi = 1) = 0
M (2) = p1 + p2 + p1 ∗ M (1) = 0
M (3) = p1 + p2 + p3 + p1 ∗ M (2) + p2 ∗ M (1)
= 0.32
M (4) = p1 + p2 + p3 + p4
+p1 ∗ M (3) + p2 ∗ M (2) + p3 ∗ M (1)
= 0.88
M (5) = p1 + p2 + p3 + p4 + p5 + p1 ∗ M (4)
+p2 ∗ M (3) + p3 ∗ M (2) + p4 ∗ M (1)
= 1.

Method 2
P (N (5) = 1) = P (X1 = 2, Y1 = 1) + P (X1 = 2, Y1 = 2) + P (X1 = 3, Y1 = 2)
+P (X1 = 3, Y1 = 1)
= 0.4 × 0.8 + 0.4 × 0.2 + 0.6 × 0.8 + 0.6 × 0.2
= 1.
So EN (5) = 1 × 1 = 1.
5. (20%) Let {Zn : n = 1, 2, · · ·} be a Markov Chain with the transition probability matrix

0 1 2
0 0.2 0 0.8
P=1 0 1 0
2 0.8 0 0.2

We assume that the process begins in state Z0 = 0, and then the successive returns to
state 0 form a renewal process.

(a) Determine the mean duration of one of these renewal intervals, i.e. the expectation
of the interoccurrence time.
(b) Find P (W2 > 3|N (2) = 1), where W2 is the waiting time of the second renewal and
N (2) is the number of renewals up to time 2.

Solution:
(a) Let Xn be one of the duration of the renewal. Then
P (Xn = 1) = 0.2
P (Xn = 2) = 0.8 ∗ 0.8
P (Xn = k) = 0.8 ∗ 0.2k−2 ∗ 0.8, for k > 2

4
 We have the mean duration
E(Xn ) = 2.
(b) Note that
[
{N (2) = 1} = (W1 = 1, W2 > 2) (W1 = 2)
[
= (X1 = 1, X2 > 1) (X1 = 2)

we have

P {N (2) = 1}
= P (X1 = 1, X2 > 1) + P (X1 = 2)
= P (X1 = 1)P (X2 > 1) + P (X1 = 2)
= 0.2(1 − 0.2) + 0.82 = 0.8.

Note

P (W2 > 3, N (2) = 1)

\ [
= P {(W2 > 3) [(W1 = 1, W2 > 2) (W1 = 2)]}
= P (W1 = 1, W2 > 3) + P (W1 = 2, W2 > 3)
= P (X1 = 1, X2 > 2) + P (X1 = 2, X2 > 1)
= P (X1 = 1)P (X2 > 2) + P (X1 = 2)P (X2 > 1)
= 0.544.

Therefore
0.544
P (W2 > 3|N (2) = 1) = = 0.68.
0.8

6. (20%) Suppose that B(t) is a Brownian Motion with diffusion coefficient σ 2 = 2 and
starting point B(0) = 0.

(a) Define Z(t) = B(t) − tB(1) for t ∈ [0, 1]. Then Z(t) is a Brownian bridge process.
Find the covariance Cov(Z(s), Z(t)) for s, t ∈ [0, 1].
(b) Find E[B(2)B(4)], E[B(2)B(4)|B(1) = 1] and P (B(5) < 3|B(1) = 2, B(2) = 3).

Solution: (a).

EZ(t) = 0. EZ(s) = 0

The covariance

Cov(Z(s), Z(t)) = E(Z(s)Z(t))

= E[(B(s) − sB(1))(B(t) − tB(1))]
= E[B(s)B(t) − tB(1)B(s) − sB(1)B(t) + tsB(1)2 ]
= min(s, t)σ 2 − tsσ 2 − stσ 2 + stσ 2
= min(s, t)σ 2 − tsσ 2
= 2(min(s, t) − st).

(b).

E[B(2)B(4)] = min(2, 4)σ 2 = 4.

5
Let N1 = B(2) − B(1) and N2 = B(4) − B(2). Then N1 , N2 and B(1) are independent.

E[B(2)B(4)|B(1) = 1]
= E{[N1 + B(1)][N2 + N1 + B(1)]|B(1) = 1}
= E{[N1 + 1][N2 + N1 + 1]}
= EN1 N2 + EN12 + EN1 + E(N1 + N2 ) + 1
= 3.

By the Markov property of the Brownian motion,

P (B(5) < 3|B(1) = 2, B(2) = 3)

= P (B(5) < 3|B(2) = 3)
= P (B(5) − B(2) < 0|B(2) = 3)
= P (B(5) − B(2) < 0)
= 0.5.

(END OF PAPER)