Chapter 4 Recurrences

1
 What are Recurrence
Relations
• Recurrence relations describe functions over the natural
numbers.
• They express the value of a function for a given integer
x > 0 as functions of the values of the function for
arguments less than x.
Examples:

2
 Motivation
• The running times of many algorithms (especially recursive
algorithms) are easily expressed using recurrence relations:

• Example: (Merge Sort)

• Problem: How do we compare functions expressed using recurrence

relations?
• We need to solve them, that is, derive closed expressions of the
functions expressed by recurrences using O-, Ω-, and Θ-notation.

3
 Introduction
• A recurrence is an equation or inequality that describes a
function in terms of its value on smaller inputs


 (1)
T (n)  2T ( n / 2)( n )
If n = 1
If n > 1
Θ(nlgn)
• Three methods for solving recurrences
– Substitution method
– Recursion-tree method
– Master method
• Notes – some technical details are neglected
– Assumption of integer arguments to functions: T(n)
– Boundary condition for small n  T(n) is constant for small n
– Floors, ceilings

4
Substitution Method

5
 The Substitution Method
• The three steps of the substitution method:

• 1. Make a good guess

• 2. Verify the guess, assuming that it can be verified for some

base case n = n0

• 3. Choose an n0 for which the guess works and such that

Step 2 for any n > n0 does not depend on the claim for some
n' < n0
6
 Basic concept
• Useful when it is easy to guess the form of the answer

• Can be used to establish either upper or lower bound on a

recurrence

• Example: T (n)  2T ( n / 2)  n

T(n) = O(n lg n) ?

7
 Determine an upper bound
on T (n)  2T ( n / 2 )  n
 
(Cont.)
– Induction : holds for n ? T ( n / 2)  c n / 2 lg n / 2)
T ( n)  2T ( n / 2)  n
 2(c n / 2 lg n / 2))  n
 cn lg n / 2  n
 cn(lg n  lg 2)  n
 cn lg n  cn  n
 cn lg n (holds as long as c 1)

8
 Determine an upper bound
on T (n)  2T ( n / 2)  n
• Guess T(n) = O(n lg n)  prove T(n)  cn lg n for some c
– By mathematical induction
– Inductive base: prove the inequality holds for some small n
• n = 1?  T(1)  c * 1 * log 1 = 0  but T(1) = 1
• start from T(2) = 4 or T(3) = 5  choose any c  2
– OK because asymptotic notation requires us to prove T(n) 
cn lg n for n  n0
• Trick: extend boundary conditions to make the inductive
assumption work for small n
– Induction assumption: assume the bound holds for n / 2
T ( n / 2)  c n / 2 lg n / 2) T(2)=2T(1) + 2 = 4
cn lg n = c * 2 * lg 2 = 2c

T(3)=2T(1) + 3 = 5
9
cn lg n = c * 3 * lg 3
 Making a good guess
• Need experience and creativity
• Use recursion trees to generate good guesses
• If a recurrence is similar to one you are familiar, then
guessing a similar solution is reasonable
– T ( n)  2T ( n / 2  17)  n T(n) = O(n lg n)  Why?
– The additional term (17) cannot substantially affect the solution to
the recurrence (when n is large)
• Prove loose upper and lower bounds and then reduce the
range of uncertainty

10
 Subtleties
• Sometimes you may guess right but the math does not
seem to work out in the induction
– Inductive assumption is not strong enough
– Revise the guess by subtracting a lower-order term
T (n)  T ( n / 2)  T ( n / 2)  1  O(n) ??? Show T(n)  cn
T (n)  c( n / 2)  c( n / 2)  1  cn  1 Seem wrong!!
New guess T(n)  cn-b
T (n)  c( n / 2  b)  c( n / 2  b)  1  cn  2b  1
 cn  b (as long as b 1)

11
 Avoiding Pitfalls
T (n)  2T ( n / 2)  n
– Falsely prove T(n)=O(n) by guessing T(n)  cn and…

T (n)  2(c n / 2)  n  cn  n  O(n)  Wrong

You have to prove the exact form of the inductive hypothesis

12
 Changing Variables

T (n)  2T ( n )  lg n m=lg n
m
T (2 )  2T (2 2 )  m
m S(m)=T(2m)

S ( m )  2 S ( m / 2)  m S(m)=O(m lg m)

T ( n)  T ( 2 m )  S ( m)
 O(m lg m)  O(lg n lg lg n)

13
Recursion-Tree Method

14
 Basic concept
• Recursion tree
– Each node represents the cost of a single subproblem somewhere in the set
of recursive function invocations
– Sum the costs within each level of the tree to obtain a set of per-level costs
– Sum all the per-level costs to determine the total cost of all levels of the
recursion
• Useful when the recurrence describes the running time of a divide-and-
conquer algorithm
• Useful for generating a good guess, which is then verified by the
substitution method
– Sloppiness are allowed
• Example: T ( n)  3T ( n / 4)  ( n 2 )
– Ignore the floors and write out the implicit coefficient c > 0 (sloppiness)

15
Recurrence tree for
T(n)=3T(n/4) + cn2

16
Recurrence tree for
T(n)=3T(n/4) + cn2

17
 T(n)=3T(n/4) + cn2
• The subproblem size for a node at depth i is n/4i
– When the subproblem size is 1  n/4i = 1i=log4n
– The tree has log4n+1 levels (0, 1, 2,.., log4n)
• The cost at each level of the tree (0, 1, 2,.., log4n-1)
– Number of nodes at depth i is 3i
– Each node at depth i has a cost of c(n/4i)2
– The total cost over all nodes at depth i is 3i c(n/4i)2=(3/16)icn2
• The cost at depth log4log
n4 n
3  n log 4 3
– Number of nodes is
– Each contributing
n logcost
43
T (1T(1)
)  (n log 4 3 )
– The total cost

18
 T(n)=3T(n/4) + cn2 (Cont.)
3 2 3 2 2 3 log 4 n 1 2
T (n)  cn  cn  ( ) cn  ...  ( )
2
cn  (n log 4 3 )
16 16 16
log 4 n 1
3 i 2
  ( ) cn  (n log 4 3 )
i 0 16

3 i 2
  ( ) cn  (n log 4 3 )
i  0 16

1 16 2
 cn  (n
2 log 4 3
)  cn  (n log 4 3 )
3 13
1
16
 O(n 2 )

19
 T(n)=3T(n/4) + cn2 (Cont.)
Prove T(n)=O(n2) is an upper bound by the substitution method
 T(n)  dn2 for some constant d > 0

T (n)  3T ( n / 4)  cn 2
2
 3d n / 4  cn 2
 3d (n / 4) 2  cn 2 n2 is also a tight bound. Why?
3 2
 dn  cn 2
16
 dn 2 (as long as d (16/13)c)

20
A recursion tree for
T(n)=T(n/3) + T(2n/3) + cn

2
( )x n  1
3
n  (3 / 2) x
x  log (3 / 2) n 21
 T(n)=T(n/3) + T(2n/3) + cn

• No. of levels * cost of each level = O(cn log3/2 n)=O(n lg n)

• Complication
2log3 / 2 n  n log3 / 2 2
– If the tree is complete binary tree, # of leaves =
– But the tree is not complete
– Go down from the root, more and more internal nodes are absent
• Verify O(n lg n) is an upper bound by the substitution
method

22
The Master Method

23
 Basic concept
• Cookbook for solving T (n)  aT (n / b)  f (n)
– Describe the running time of an algorithm that divides a problem of
size n into a subproblems, each of size n/b
– a  1 and b > 1 (constant), f(n): asymptotically positive
– Require memorization of three cases
– Floor and ceiling functions are omitted

24
 Master Theorem
• T (n)  aT (n / b)  f (n) can be bounded asymptotically as follows
– If f (n)  O (n logb a  ) ( > 0), then T (n)  (n logb a )
– If f (n)  (n b ) , then T (n)  (n b lg n)
log a log a

log a 
– If f (n)  (n b ) ( > 0), and if af(n/b)  cf(n) for some constant c
< 1 and all sufficiently large n, then T (n)  ( f (n))
log b a
• The solution is determined by the larger of f(n) and n
– Case 1: f(n) must be polynomially smaller than n log b a
• f(n) must be asymptotically smaller than n logb a by a factor of n
log a
– Case 3: f(n) must be polynomially larger than n b , and in
addition satisfy the regularity condition that af(n/b)  cf(n)
• The three cases do not cover all possibilities for f(n)

25
 Using the master method
• Example 1: T(n)=9T(n/3) + n (Case 1)
– a=9, b=3, f(n) = n
log 9 1
– n b  n 3  (n ), f (n)  O (n 3 )
log a log 9 2

– T(n) = (n2)
• Example 2: T(n) = T(2n/3) + 1 (Case 2)
– a=1, b=3/2, f(n)=1
– n b  n 3 / 2  1, f (n)  1  (1)
log a log 1

– T(n) = (lg n)

26
 Using the master method
(Cont.)
• Example 3: T(n) = 3T(n/4) + n lg n (Case 3)
– a=3, b=4, f(n) = n lg n
– n logb a  n log 4 3  O (n 0.793 ) f (n)  (n 4 ) ( 0.2)
log 3

– For sufficiently large n, af(n/b) = 3(n/4)lg(n/4)  (3/4)n lg n=cf(n) for

c=3/4
– T(n) = (n lg n)
• Example 4: T(n) = 2T(n/2) + n lg n
– a=2, b=2, f(n) = n lg n
– n logb a  n log 2 2  n
– f (n) / n log b a  n lg n / n  lg n (Asymptotically less than n for any  )
– Falls into the gap between case 2 and case 3

27
?

28