Large deviations provide good approximation to

queueing system with dynamic routing

K. Duffy(1) , D. Malone(1) , E.A. Pechersky(2) ,

Y.M Suhov(2,3) and N.D. Vvedenskaya(2)

December 7, 2004

(1) Hamilton Institute, National University of Ireland, Maynooth, Co. Kildare, Ireland
(2)Institute for Information Transmission Problems, RAS, Moscow
and
(3) Statistical Labarotary, DPMMS, Cambridge University, Cambridge, UK
E-mail correspondence: ndv@iitp.ru

Abstract
We consider a system with two infinite-buffer FCFS servers (of speed
one). The arrivals processes are three independent Poisson flows Ξi , of
rates λi , i = 0, 1, 2, each with IID task service times. The tasks from Ξi
are directed to server i, i = 1, 2 (dedicated traffic). The tasks from Ξ0 are
directed to the server that has the shorter workload in the buffer at the
time of arrival (opportunistic traffic). We compare the analytical data for
the large deviation (LD) probabilities for the virtual waiting time in flow
Ξ0 and empercial delay freqencies from simulations.

1 Introduction
We consider queueing systems with dynamic routing. In particular large devia-
tion (LD) probabilities of long delay in the stationary regime. Various problems
of heavy load in systems with dynamic routing have already been investigated.
For example, see [12, 13, 14, 8] and references therein. In particular, in [12, 13]
a several-server system (with different speeds) and a single discretionary flow,
where all tasks are directed to the least busy server, was studied (a GI/GI/s/∞
load-balanced queue). In [14, 8] a system with two servers was considered, with
different speeds and three Poisson flows, two dedicated and one discretionary
Here the delay probabilities are analyzed in a system with non-exponential ser-
vice times. We consider a model with two servers and three Poisson flows,
each with a general distribution of service times (a M/GI/2/∞ load-balanced

1
queue). We only discuss systems with equal server speed and do not consider
LD probabilities for queue lengths.
We present a novel method to derive LD probabilities: we introduce an auxil-
iary system where the discretionary flow is partitioned between servers in some
proportion, regardless of workloads. We then determine a ‘genuine’ proportion
that occurs in the original system in the stationary LD regime.
We believe this procedure can be followed for more complicated queueing net-
works, where other approaches would prove too intricate. Even if the auxiliary
flow approach is not rigorously justifiable for these more complicated networks,
belief in its predictions can be gained through simulations such as those de-
scribed in this paper (where the LD limits can be rigorously justified for Poisson
processes).
The problem of the accuracy of probability estimates obtained by LD theory
was discussed by R.L. Dobrushin in one of his first works on large deviations in
queuing systems [3]. He wrote that in many real systems the system designer
sets a significance level, p; he wishes that the waiting times ω experienced in
the system exceed a given bound with probability smaller than p. Thus it is of
interest to study the function T (p) defined by

P(ω > T (p)) = p.

If the rate of decrease in the tail of the probability distribution describing the
input flow is exponential or faster, it is naturally to expect

T (p)
lim = α,
p→0 ln(1/p)

for some constant α. Thus T (p) ≈ α ln(1/p) for small p, and it is of interest to
find the constant α.
Is this approximation sufficient for applications? In the majority of cases only
rough estimates for the desired p are specified. It is usual to use a scale of the
type
p = 10−6 , 10−7 , 10−8 , . . .
Thus for LD approximations to be useful, they must be accurate with respect to
this scale. This work presents examples where the LP approximation is useful.
We compare our LD predictions with simulation data. The comparison shows
that for our system the LD probabilities provide good approximation, even for
relatively small delays.
In Section 2 we describe the system under investigation, its auxiliary system
and the main LD theorem is presented. In Section 3 we present simulation
results for Poisson flows for the initial system and compare them with the LD
predictions.

2
2 The initial and auxiliary systems

2.1 The initial model

We focus on a system with two infinite-buffer FCFS servers of speed one. The
arrivals to this system are formed by three independent Poisson flows Ξ0 , Ξ1
and Ξ2 , of rates λi , i = 0, 1, 2, each with IID service times. Flows Ξ1 and Ξ2
are dedicated: flow Ξ1 is directed to server 1; flow Ξ2 to server 2. Flow Ξ0 is
discretionary: its tasks join the queue with the smaller workload. We denote
by S (k) the random service time in flow Ξk and by ϕk the Laplace transform of
S (k) ,
(k)
ϕk (θ) = EeθS . (1)

We assume that the functions ϕk , k = 0, 1, 2, are defined on positive intervals

and on these intervals take all values in [1, ∞). Formally,
(0) (0) (1) (1) (2) (2)
Ξ0 = (τj , ξj )∞ ∞ ∞
j=−∞ , Ξ1 = (τj , ξj )j=−∞ , Ξ2 = (τj , ξj )j=−∞ , (2)

where for the j th message of the k th flow, τjk is its arrival time and ξjk is its
service time.
We make the following assumptions:

(k)
1. For k = 0, 1, 2, {ξj } forms an i.i.d. sequence taking values in R+ . There
(k)
exists θ+ > 0, k = 0, 1, 2 > 0, such that
(k)
ϕk (θ) = E exp{θξ (k) } < ∞ for θ < θ+ (3)

and lim ϕk (θ) = ∞.

(k)
θ↑θ+

(k) (k)
2. The sequences {τj } and {ξj } are independent.
(k)
3. Each sequence of random variables {τj } forms a stationary Poisson point
(k) (k) (k)
process on R, where τj < τj+1 and τ−1 < 0 ≤ t0 . The rate of the kth
process is denoted by λk .
4. There exists a unique stationary regime; this is guaranteed by the stability
condition X
λi ϕ0i (0) < 1, i = 1, 2, λi ϕ0i (0) < 2. (4)
i=0,1,2

3
We consider a Markov process (w 1 (t), w2 (t)) on R2+ defined by the generator

Lf (w1 , w2 ) =
∂f ∂f
− 1
I(w1 >0) − I(w2 >0) +
∂w 1 ∂w2
λ1 Ef (w + ξ 1 , w2 ) − f (w1 , w2 ) +


λ2 Ef (w1 , w2 + ξ 2 ) − f (w1 , w2 ) +
 
(5)
λ0 Ef (w1 + ξ 0 , w2 ) − f (w1 , w2 ) I(w1 <w2 ) (w1 , w2 )+
 

λ0 Ef (w1 , w2 + ξ 0 ) − f (w1 , w2 ) I(w1 >w2 ) (w1 , w2 ).

 

The vector (w 1 (t), w2 (t)) describes the servers’ workload at time t. On the other
side (w1 (t), w2 (t)) is the vector of the virtual waiting times at time t.
We analyze the LD probabilities for the delay of a virtual task (of zero length)
embedded into flow Ξ0 at a fixed time. Let w 1 (t), w2 (t) be the workloads at the
first and second servers at time t, say t = 0, and let

w0 (t) = min{w1 (t), w2 (t)}, (6)

the minimum workload. We consider the large deviation problem of existence

and identification of the limit
−1
I0 (d) = lim log P(w0 ≥ nd), (7)
n→∞ n
for d > 0.
In our problem the input flows to each server are neither independent nor Pois-
son, even though the source flows are. The process (w 1 , w2 ) takes values in
the quarter-plane R2+ = {(w1 , w2 ) : w1 , w2 ≥ 0} and has transition prob-
abilities that are homogeneous in each half of the quarter-plane: {(w 1 , w2 ) :
0 < w1 < w2 } and {(w1 , w2 ) : 0 < w2 < w1 }. However, on the diagonal
{(w1 , w2 ) : 0 < w1 = w2 } the transition probabilities are discontinuous. The
large deviation principal in the case of a two-dimensional Markov process with
discontinuous transition probabilities is considered in [5, 6], [7], [2], [1] (thus we
can call I0 (d) the rate function of the initial problem). We do not follow these
works. We introduce a new, different strategy to estimate the probability of
large delay.

2.2 The auxiliary system

To use a general large deviation theory adapted to the situation where the
transition probability are continuous we consider the auxiliary systems where
there are two servers and two independent Poisson flows to these servers. More
precisely, we introduce an auxiliary model where the discretionary flow Ξ0 is
divided into two independent Poisson sub-flows each directed to a particular

4
server with no per-message routing. Using the large deviation principal for
the Poisson processes (see [4]) of the auxiliary system , we find the optimal
LD trajectories for achieving large workloads in both servers and determine a
suitable proportion that divides flow Ξ0 in a way that the probability of large
workloads in the initial and auxiliary systems coincide. The idea of the auxiliary
system was proposed in [9, 10]
We first describe the auxiliary system informally. Consider a queueing system
with two servers and two Poisson flows ΞA A
1 and Ξ2 . The flows are directed to
the first and the second server respectively. The flows’ rates are

λA
1 = λ1 + αλ0 , and λA
2 = λ2 + (1 − α)λ0 , 0 ≤ α ≤ 1.

And the Laplace transforms of the service times are

1
ϕA
1 (θ) = (λ1 ϕ1 (θ) + αλ0 ϕ0 (θ)),
λ1 + αλ0
1
ϕA
2 (θ) = (λ2 ϕ2 (θ) + (1 − α)λ0 ϕ0 (θ)).
λ2 + (1 − α)λ0

Where all ϕi and λi , i = 0, 1, 2, are the same as in the initial problem. Later
we shall determine the value of α that makes our initial and auxiliary systems
equivalent with respect to LD probabilities.
In the auxiliary system arrivals of the discretionary flow Ξ0 are directed to server
1 with probability α and to server 2 with probability 1 − α, independently. It is
convenient to represent flow Ξ0 as a superposition of independent Poisson flows
ΞA 1 1 ∞ A 2 2 ∞
1 = {(τAj , ξAj )}j=−∞ and Ξ2 = {(τAj , ξAj )}j=−∞ of rates αλ0 and (1 − α)λ0
with the same service-time distribution as in original flow Ξ0 .
Consider the workloads w1A and w2A at two servers of the auxiliary system:

wiA = sup{aA
i (t) − t}, i = 1, 2, (8)
t≥0

where  P (i) P (i)


 ξj + ξAj , if t > 0,
 j: 0<τji ≤t (i)
j: 0<τAj ≤t
aA
i (t) = P (i) P (i) (9)
− ξj − ξAj , if t ≤ 0.


(i) (i)
j: τ <tj ≤0 j: τ <tAj ≤0

We study the following event

 
A= min {wiA } > nd , (10)
i=1,2

and determine the function

−1
I0A (d) = lim log P( min {wiA } > nd), d > 0. (11)
n→∞ n i=1,2

5
To use LD principle of [4] we re-scale our processes to take the LD limit. For
time t and space x the rescaling is

t → t/n = t(n) , and x → x/n = x(n)

(n)
thus the arrival processes are rescaled Ξi → Ξi in the following way:
(n) (n)
τj = τj /n, ξj = ξj /n, −∞ < j < ∞,
(n)
wi (t(n) ) = wi (t)/n, i = 0, 1, 2.
In the new notation, we have
 
(n) (n)
A= min {w1 , w2 } >d .
i=1,2

The rate function is defined as follows. Let the trajectory x(t) = (x1 (t), x2 (t)
be absolutely continuous on [0, ∞] then the value I(x) of the rate function on
x is
Z ∞
sup θ1 ẋ1 (t) + θ2 ẋ2 (t) − λA
  A  A
 A 
I(x) = 1 ϕ1 (θ1 ) − 1 − λ2 ϕ2 (θ2 ) − 1 dt.
0 θ1 ,θ2
(12)
We have to find inf x∈A I(x). Belonging x ∈ A means that
( )
min sup {x1 (t) − t}, sup {x2 (t) − t} > d.
i=1,2 t/geq0 t/geq0

At fixed α the problem is reduced to minimization of

   
T1 sup{v1 θ1 − λ[ϕ1 (θ1 ) − 1]} + T2 sup{v2 θ2 − λ[ϕ2 (θ2 ) − 1]} (13)
θ1 θ2

over T1 , T2 , v1 and v2 , where vi = TiT−d

i
, i = 1, 2. However, making the auxiliary
system approximating the original one, requires to chose α in a way such that
T1 = T2 and thus v1 = v2 .
We give an explicit identification for the mininum of expression (13) (this min-
imum is I0A (d) defined in equation (11)). It is determined in terms of solution
ϑ to (14) and the solutions θi , θ0,j , i = 1, 2, j = 3 − i, to (15),(16),
 
1 X
ϑ= λi (ϕi (ϑ) − 1) , (14)
2 i=0,1,2

and

θi + θ0,j = λi (ϕi (θi ) − 1) + λj (ϕj (θ0,j ) − 1) + λ0 (ϕ0 (θ0,j ) − 1), (15)

6
 λi ϕ0i (θi ) = λj ϕ0j (θ0,j ) + λ0 ϕ00 (θ0,j ), (16)
where
i = 1, 2, j = 3 − i.

Theorem 1.
Let ϑ be a solution to (14)
A. In the case
λ0 ϕ00 (ϑ) ≥ λ1 ϕ01 (ϑ) − λ2 ϕ02 (ϑ),
 
(17)

I0A (d) has the form

I0A (d) = 2dϑ, (18)

This case is called a balanced case. Here

1 λ2 ϕ02 (ϑ) − λ1 ϕ01 (ϑ)
α= + .
2 λ0 ϕ00 (ϑ)

B. In the case
λ2 ϕ02 (ϑ) > λ1 ϕ01 (ϑ) + λ0 ϕ00 (ϑ), (19)
I0A (d) has the form
I0A (d) = d(θ2 + θ0,1 ). (20)
where θ2 , θ01 are the solution to (15), (16) with i = 2, j = 1.
This case is called an unbalanced case. Here α = 1.
C. In the case
λ1 ϕ01 (ϑ) > λ2 ϕ02 (ϑ) + λ0 ϕ00 (ϑ). (21)
I A (d) has the form
I0A (d) = d(θ1 + θ0,2 ), (22)
where θ1 , θ0,2 are the solution to (15), (16) with i = 1, j = 2
This case is also an unbalanced case. Here α = 0.
Further, I0 (d) = I0A (d).

Remark 1 Observe that the expression for I0A in the balanced case is identical
to the system where all flows are fed into a single-server queue working at speed
2. (See, e.g. [11], [4] and the references there.)

Remark 2 In the unbalanced case, alpha is 0 or 1, corresponding to the whole

of the discretionary flow joining a specific server.

7
 Exponential message sizes (10,000,000 in experiment) Exponential message sizes (10,000,000 in experiment)
0 0
F1=(1,0.3), F2=(1,0.3), F0=(1,0.6) F1=(1,0.1), F2=(1,0.2), F0=(1,0.4)
LD prediction LD prediction
-2 -2

-4
-4

-6
-6
log freq W>n

log freq W>n

-8
-8
-10

-10
-12

-12
-14

-14 -16

-16 -18
0 1 2 3 4 5 6 7 8 0 0.5 1 1.5 2 2.5 3 3.5 4 4.5
n n

Constant message sizes (10,000,000 in experiment) Constant message sizes (10,000,000 in experiment)
0 0
F1=(1,0.4), F2=(1,0.4), F0=(1,0.5) F1=(1,0.5), F2=(1,0.5), F0=(1,0.4)
LD prediction LD prediction
-2 -2

-4 -4

-6 -6
log freq W>n

log freq W>n

-8 -8

-10 -10

-12 -12

-14 -14

-16 -16
0 0.5 1 1.5 2 2.5 3 3.5 0 0.5 1 1.5 2 2.5 3 3.5 4 4.5 5
n n

Figure 1: Balanced case.

Exponential message sizes (10,000,000 in experiment) Exponential message sizes (10,000,000 in experiment)
0 0
F1=(1,0.9), F2=(1,0.3), F0=(1,0.3) F1=(1,0.9), F2=(1,0.3), F0=(1,0.4)
LD prediction LD prediction
-2 -2

-4 -4

-6 -6
log freq W>n

log freq W>n

-8 -8

-10 -10

-12 -12

-14 -14

-16 -16
0 1 2 3 4 5 6 7 8 9 0 2 4 6 8 10 12 14 16
n n

Constant message sizes (10,000,000 in experiment) Constant message sizes (10,000,000 in experiment)
0 0
F1=(1,0.9), F2=(1,0.3), F0=(1,0.3) F1=(1,0.9), F2=(1,0.3), F0=(1,0.4)
LD prediction LD prediction
-2 -2

-4 -4

-6 -6
log freq W>n

log freq W>n

-8 -8

-10 -10

-12 -12

-14 -14

-16 -16

-18 -18
0 0.5 1 1.5 2 2.5 3 3.5 4 4.5 5 0 1 2 3 4 5 6 7
n n

Figure 2: Unbalanced case.

8
3 Simulation results
A series of simulations were performed to calculate log freq(w 0 ≥ d) empirically
and to compare these data with I0A (d). According to the LD theory as n → ∞
 
− log P(w0 ≥ nd) = nI0 (d) + o n . (23)

Fundamental to determining the merit of approximating

  P(w 0 ≥ nd) by exp(−nI(d))
is understanding the character of the error o n .

In simulation, the original system was modeled. Starting with an empty system,
10 million messages were generated between all three flows. Records were kept
of the waiting times experienced by messages from flow 0. Figures 1 and 2 show
the log empirical frequencies with which w 0 exceeds n vs. n. Also shown is the
LD approximations to these probabilities, −nI0 (1). The legends on the graphs
correspond to the flow parameters in the experiment; Fi (λ, c) represents the
Poisson flow Ξi , i = 0, 1, 2, of rate λi with mean task length ci . Constant and
exponential message sizes were investigated.
All the empirical traces show similar behavior: a non-zero intercept; initial
curvature; a straight line; and then noise. The noise is caused due to scarcity of
data. If the experiments are run for longer, the place at which the noise occurs
moves further to the right. The non-zero intercept and initial curvature are due
to the details of the process. The LD probabilities should, in theory, match the
slope of the straight line −nI0 (1). The LD prediction is plotted from the origin,
for convenience.
The graphs in Figure 1 show the balanced case and the graphs in Figure 2
show the unbalanced case, for exponential and constant message sizes. Good
agreement of slope is seen in all cases. Indeed, the approximation works well
for not only large values of n, but also quite small values. That the difference
between prediction and the linear LD approximation is approximately constant
suggests P(w 0 > n)
≈ exp(µ − nI0 (1)). That is the error in using the LD
approximation, o n , is constant.
Acknowledgment. The work was partly done at the Dublin Institute for
Advanced Studies, Ireland. The authors thank Tony Dorlas for his hospitality.
The work of E.A.P. and N.D.V. was partly supported by Russian RFFI grant
02-01-00068. K.D and D.M are supported by Science Foundation Ireland under
the national development plan.

References
[1] Murat Alanyali and Bruce Hajek, On large deviations in load sharing net-
works, Ann. Appl. Probab., 8, 67–97, 1998.

9
 [2] Blinovski V.M. and Dobrushin R.L. Process level large deviations for a class
of piecewise homogeneous random walks, In The Dynkin Festschrift: Progr.
Probab., 34, 1–59, Birkhuser Boston, Boston, MA, 1994.
[3] Dobrushin R.L., Pechersky E.A., Large deviations for tandem queueing
systems J. Appl. Math. Stochastic Anal. 7 (1994), no. 3, 301–330.
[4] Dobrushin, R.L. and Pechersky, E.A.Large deviations for random processes
with independent increments on infinite intervals, Probl. Inform. Transm.,
34,4, pp 354-382, 1998.
[5] Dupuis, P., Ellis, R.S. and Weiss, A., Large deviations for Markov processes
with discontinuous statistics, I: general upper bound. Ann. Probab., 19,
1280-1297, 1991.
[6] Dupuis, P. and Ellis, R.S., Large deviations for Markov processes with dis-
continuous statistics, II: random walks, Probab. Theory Related Fields, 91
(1992), 153-194.
[7] Ignatyuk, I.A., Malyshev, V.A. and Shcherbakov, V.V., The influence of
boundaries in problems on large deviations, Uspekhi Mat. Nauk 49, (1994),
43-102.
[8] McDonald, D.R. and Turner, S.R.E., Resource Pooling in Distributed
Queueing Networks, Fields Inst. Communications,28, 107-131, 2000.
[9] Pechersky E.A., Suhov Y.M. and Vvedenskaya N.D. Large deviation in two-
server system with dynamic routing, Preprint, Isaac Newton Institute for
Math. Sci., NI03075-IGS, 1 December 2003.
[10] Pechersky E.A., Suhov Y.M. and Vvedenskaya N.D. Large deviations in a
two-server system with dynamic routing’, Proceedings , 2004 IEEE Internat.
Symposiom on Inform. Theory , Chicago, USA, p.114.
[11] Shwartz, A., and Weiss, A., Large deviations for performance analysis:
queues, communications and computing, Chapman & Hall, 1995.
[12] Sadovski, J.S. and Szpankovski, W.,The probability of large queue lengths
and waiting times in a heterogeneous multiserver queue I: tight limits, Adv.
Appl. Prob., 27, 532-566, 1995.
[13] Sadovski, J.S. The probability of large queue lengths and waiting times in
a heterogeneous multiserver queue II:positive recurrence and logarithmic
limits, Adv. Appl. Prob., 27, 567-583, 1995.
[14] Turner, S.R.E., Large deviations for Join the Shortest Queue Fields Inst.
Communications,28, 95-106, 2000.

10