DELAY

PERFORMANCE
ANALYSIS
Queueing Models
Little’s theorem
QUEUING IN THE NETWORK LAYER AT A
ROUTER

Bertksekas Gallager, Data Networks

2
 QUEUEING DELAY (REVISITED)

average queueing
• R: link bandwidth (bps)

delay
• L: packet length (bits)
• a: average packet arrival
rate
traffic intensity
= La/R
§ La/R ~ 0: avg. queueing delay small La/R ~ 0

§ La/R -> 1: avg. queueing delay large

§ La/R > 1: more “work” arriving
than can be serviced, average delay infinite!

La/R -> 1 3
 BASIC SINGLE QUEUE MODEL

• Classical queuing theory can be applied to an output link in a router.

4
 BASIC SINGLE QUEUE MODEL

• For example, a 56 kbps transmission line can “serve” 1000-bit packets at a rate
of

56,000 bits/sec
= 56 packets/sec
1000 bits/packet

5
 APPLIC ATIONS OF QUEUING ANALYSIS
OUTSIDE OF NETWORKING

• Checkout line in a supermarket

• Waiting for a teller in a bank
• Batch jobs waiting to be processed by the CPU

6
THE POISSON ARRIVAL MODEL

A Poisson process is a sequence of events “randomly spaced in

time”
7
 THE POISSON ARRIVAL MODEL

• Examples
• Customers arriving to a bank
• Packets arriving to a buffer
• The rate λ of a Poisson process is the average number of events per unit time
(over a long time).

8
 PROPERTIES OF A POISSON PROCESS

• For a length of time t the probability of n arrivals in t units of time is

(l t ) - lt n
Pn (t ) = e
n!

9
 PROPERTIES OF A POISSON PROCESS

• For 2 disjoint (non-overlapping) intervals, (s1, s2) and (s3, s4), (i.e. s1 < s2 £
s3 < s4), the number of arrivals in (s1, s2) is independent of the number of
arrivals in (s3, s4)

10
 INTERARRIVAL TIMES OF A POISSON
PROCESS

• Pick an arbitrary starting point in time (call it 0).

• Let t 1= the time until the next arrival

- lt
P (t 1 > t ) = P0 (t ) = e

11
 INTERARRIVAL TIMES OF A POISSON
PROCESS

• So
- lt - lt
Ft1 (t ) = P (t 1 £ t ) = 1 - e and ft1 (t ) = le

t 1,the time until the first arrival,

Has an exponential distribution!

12
 INTERARRIVAL TIMES OF A POISSON
PROCESS

• Let t 2 = the length of time between the first and second arrival.
• We can show that

i.e. t 2 is exponential and independent of t 1!

P (t 2 > t | t 1 = s ) = P (t 2 > t ) = e - lt for any s, t > 0

13
 INTERARRIVAL TIMES OF A POISSON
PROCESS

• Similarly define t 3 as the time between the second and third arrival; t4 as
the time between the third and fourth arrival;…
• The random variables t 1 ,t 2 , t 3 ,… are called the interarrival times of the
Poisson process

14
 INTERARRIVAL TIMES OF A POISSON
PROCESS
• The interarrival time random variables, t 1, t 2t, 3 …
• Are (pair-wise) independent.
• Each has an exponential distribution with mean 1/λ.

15
 THE M/M/1 QUEUE

• An M/M/1 queue has

• Poisson arrivals (with rate λ)
• Exponential service times (with mean 1/μ, so μ is the “service rate”).
• One (1) server
• An infinite length buffer
• The M/M/1 queue is the most basic and important queuing model.

16
 QUEUING NOTATION

“M/M/1” is a special case of more general

(Kendall) notation: X/Y/m/k, where
• X is a symbol representing the interarrival
process
• M = Poisson (exponential interarrival times, t )
• D = Deterministic (constant t ).

17
 QUEUING NOTATION

• Y is a symbol representing the service distribution

M = exponential, D = deterministic
G = General (or arbitrary).
• m = number of servers
• k = number of buffer slots (omitted when k = ¥ )

18
 ASIDE: THE D/D/1 QUEUE

• The D/D/1 queue has

• Deterministic arrivals (periodic with period = 1/λ).
• Deterministic service times (each service takes exactly 1/μ).
• As well as 1 server and an infinite length buffer.

19
 ASIDE: THE D/D/1 QUEUE

• If λ < μ then there is no waiting in a D/D/1 queue.

Randomness is a major cause

of delay in a network node!

20
 STATE ANALYSIS OF AN M/M/1 QUEUE

• Let n be the state of the system = the number of packets in the system
(including the server).
• Let pn be the steady state probability of finding n customers waiting in the
system (including the server).

21
 STATE ANALYSIS OF AN M/M/1 QUEUE

• How to find pn? The state diagram:

22
 STATE ANALYSIS OF AN M/M/1 QUEUE

¹
• If the system is stable (i.e. pn 0 for each n), then in a steady state it will drift
back and forth across the dotted line. So,
• the number of transitions from left to right = the number of transitions from
right to left.

23
 STATE ANALYSIS OF AN M/M/1 QUEUE

• Thus we obtain the balance equations

pn l = pn +1µ for each n ³ 0

24
 STATE ANALYSIS OF AN M/M/1 QUEUE

• Lets solve the balance equations: pn l = pn +1µ

• For n = 0 we get
p1 = r p0
• If we let r =l/µ , this becomes

25
 STATE ANALYSIS OF AN M/M/1 QUEUE

• Similarly
p2 = r p1 = r 2 p0
• And in general

pn=rnp0

26
 STATE ANALYSIS OF AN M/M/1 QUEUE

• We have pn=rnp0 for n =1,2,3,...

¥
• We need to solve for p0 , so we need one more equation. Use S pn = 1
n =0

• We obtain

æ 1 ö
¥
1 = S r n p0 = p0 S r n =
n =0
¥

n =0
{ p0 ç ÷ for r < 1
è 1- r ø
¥ for r ³ 1

27
 STATE ANALYSIS OF AN M/M/1 QUEUE

• So we must have
p0 = 1 - r
and

pn = (1 - r ) r n
for n = 1, 2,3,...

28
 STATE ANALYSIS OF AN M/M/1 QUEUE

• Note that requiring ρ < 1 for stability (i.e. λ <µ ) makes intuitive sense.
• Also ρ=1-ρ0
= probability that the queuing system is NOT empty
= probability the server is working

29
 STATE ANALYSIS OF AN M/M/1 QUEUE

So ρ is sometimes called the “server utilization”

• Finally note that pn = (1- ρ)pn, n = 0,1,2,3,… is a geometric distribution

30
 HOW LONG IS THAT LINE?

• Lets look again at the M/M/1 queuing system.

• n = the number in the system (including the server)
• So the average number in the system is

¥ ¥
r r
E (n) = å npn = (1 - r )å np = (1 - r )
n
=
n =0 n =0 (1 - r ) 1 - r
2

31
 LITTLE’S FORMULA AND QUEUING
DELAY

• Let T = time spent by a customer in a

queuing system (waiting and being served).
• E(T) = the average delay for a customer.

32
 LITTLE’S FORMULA AND QUEUING
DELAY

• Little’s Formula says l E (T ) = E (n)

• where λ is the “arrival rate for customers

eventually served”

Little’s Formula holds for very general

queuing systems (not just M/M/1). Even
whole networks!
33
 LITTLE’S FORMULA AND QUEUING
DELAY

• Little’s Formula is either deep or obvious.

Intuition:
• Pick a “typical customer”
• When it arrives to the queuing system, it
should find E(n) customers waiting.

34
 LITTLE’S FORMULA AND QUEUING
DELAY

• When it leaves the system, it has been in the

system for E(T). Thus λE(T) customers should have
arrived during its time in the system.
• In steady state, the average number of customers
left behind on the departure should equal the
average number found on the arrival, i.e. λE(T) =
E(n)

35
 LITTLE’S FORMULA AND QUEUING DELAY

• Let’s apply Little to the M/M/1 queue

E ( n) r 1
E (T ) = = =
l l (1 - r ) µ - l
• E(T) is measured in units of time. Sometimes it
is more convenient to consider

µ E ( n) r 1
µ E (T ) = = =
l r (1 - r ) 1 - r
which is unitless
36
 LITTLE’S FORMULA AND QUEUING
DELAY

• Sometimes we consider the waiting time W,

i.e., the time spent waiting in the queue (not in
service). So,

1
E (W ) = E (T ) -
µ
37
 SINGLE LINK EXAMPLE

• Poisson packet arrivals with rate λ = 2000 p/s

• Fixed link capacity C = 1.544 Mb / s.

38
 SINGLE LINK EXAMPLE

• We approximate the packet length

distribution by an exponential with mean L
= 515 b/p
• Thus the service time is exponential with
mean
1 L 515 b / p
= = » 0.33ms / p
µ C 1.544 Mb / s
i.e. packets are served at a rate of
µ = 3000 p / s
39
 SINGLE LINK EXAMPLE

• Using our formulas for an M/M/1 queue

So l
r = = 0.67
and
µ
r
E ( n) = = 2.0 packets
1- r

E ( n)
E (T ) = = 1.0 ms
l
40
 Number of customers in the system at time T

Choose a time t when the system is empty, then shaded area is

Equally, the shaded area is composed of horizontal strips of height 1 and width T (i) (for the ith
customer).
Hence,