# Algorithms: mathemati al ba kground

Re urren es
 While sum play a role in analysis of iterations, re-
ursive programs requires analysis of re urren es.
 A re urren e is expressed by itself using smaller
parameteri values.
 Merge sort for example,

( ) = (1)
T n
, if n = 1
2T (n=2) + (n), if n  1
We know T (n) = (n log n).
 But how? { Substitution method involves
{ Make a good guess (?) then apply indu tion
to nd the onstants and show that solution
works.

" Department of C S E !
R. K. Ghosh Le ture Notes, January, 2003 19 of 395
# Algorithms: mathemati al ba kground

Re urren es
 Consider the re urren e T (n) = 2T (bn=2 )+ n.
Assume solution to be O(n log n). So
( )  2( bn=2 log(bn=2 ) + n
T n
 n log(n=2) + n
 n log n n log 2 + n
 n log n n + n
 n log n, for  1
 Use mathemati al indu tion to show that the so-
lution holds.

" Department of C S E !
R. K. Ghosh Le ture Notes, January, 2003 20 of 395
# Algorithms: mathemati al ba kground

Re urren es
 The onstant must satisfy the boundary on-
ditions.
 Suppose we hoose to be large, but how large?
 If T (1) = 1, then as 1 log 1 = 0, even if we let
to be very large the solution does not satisfy
boundary.
 Also annot be arbitrarily large.
 Sin e asymptoti time omplexity is needed, it
suÆ es to let onditions hold when n  n , 0
where n0 is a xed positive number.
 So, T (k) for some k  2 an be onsidered as
boundary onditions for indu tion.
 From re urren e T (2) = 4, T (3) = 5.
 Letting  2 implies
T (2)   2 log 2, and T (3)   3 log 3;

" thus, satis es boundary onditions.

Department of C S E !
R. K. Ghosh Le ture Notes, January, 2003 21 of 395
# Algorithms: mathemati al ba kground

What is a Good Guess

 Relies on intuition.
 For example,
( ) = 2T (bn=2 + 18) + n
T n

 We an again assume the solution to be

O (n log n).

{ Noti e that the problem is divided into

smaller problems ea h of size nearly half of
input size, assuming n to be large.
{ So ex ess of 18 in problem size ompared to
earlier re urren e does not have any e e t as
far as asymtoti running time.

" Department of C S E !
R. K. Ghosh Le ture Notes, January, 2003 22 of 395
# Algorithms: mathemati al ba kground

Proving Good Guess

 Consider
( ) = T (bn=2 ) + T (dn=2e) + 1
T n

 Guess is O(n). But when we try to proove or-

re tness
( )  bn=2 + dn=2e + 1 = n + 1
T n

does not imply that T (n)  n for any hoi e

of .

" Department of C S E !
R. K. Ghosh Le ture Notes, January, 2003 23 of 395
# Algorithms: mathemati al ba kground

Proving Good Guess

 We are o by a onstant fa tor. But indu tion

does not work if the indu tive hypothesis is not
proved exa tly.
 To over ome it, substra t a onstant from the
guess, say b, then try to prove solution T (n) 
n b.

( )  bn=2
T n b + dn=2e b+1 = n 2b+1
Implies
( )
T n n b, as long as b  1
 Now an be hosen to handle boundary ondi-
tions.
 Similar problems also arise when we try to prove
T (n) = 2T (bn=2 ) + n as O (n) as follows.

T (n)  2( bn=2 ) + n
T (n)  n + n = O (n)

" Department of C S E !
R. K. Ghosh Le ture Notes, January, 2003 24 of 395
# Algorithms: mathemati al ba kground

Changing Variables
p
 Consider T (n) = 2T (b n ) + log n.
 Change n = 2 , then T (2 ) = 2T (b2 )+ m.
m m m=2

 Rename T (2 ) as S (m), then the re urren e re-

m

lation be omes
S m( ) = 2S (m=2) + m
 So solution be omes S (m) = O(m log m) =
O (log n log log n).

" Department of C S E !
R. K. Ghosh Le ture Notes, January, 2003 25 of 395
# Algorithms: mathemati al ba kground

Method of Iteration
 Simply go on iterating the re urren e down to
base ase.
 Consider, the iteration
T (n) = n + 3T (bn=4 )
= n + 3(n=4 + 3T (bn=16 ))
= n + 3n=4 + 9(n=16 + 3T (bn=64 ))
= n + 3n=4 + 9n=16 + 27T (bn=64 )
 Obviously the iteration an ontinue till
bn=4 = 1 or when i ex eeds log n times.
i
4

 Using bn=4 < n=4 ,

i i

T (n)  n + 3n=4 + 9n=16 + : : : + 3 (1) log4 n

P1

 + (n )
i=0
3 i
4
log4 3

= 4n + o(n), as log 3 < 1; (n

4
log4 3
) = o(n)
= O(n)

" Department of C S E !
R. K. Ghosh Le ture Notes, January, 2003 26 of 395
# Algorithms: mathemati al ba kground

Re ursion Tree
 It helps to visualize progress of iteration.
 Consider the gure below for
T (n) = 2
T (n=2) + n
2

n2 n2

T(n/2) T(n/2) (n/2)

2
(n/2) 2

T(n/4) T(n/4) T(n/4) T(n/4)

n2 n2

(n/2) 2 (n/2) 2 (n/2) 2

2
(n/4) 2 (n/4) (n/4) 2 (n/4) 2 (n/4) 2

" !
Total Θ( n2 )

Department of C S E

R. K. Ghosh Le ture Notes, January, 2003 27 of 395

# Algorithms: mathemati al ba kground

Master Theorem
 In general a re urren e relation may be of the
form
( ) = aT (n=b) + f (n)
T n
for onstants a  1, b > 1 and f (n) is an aymp-
toti ally positive funtion.
 Master theorem states that
1. If f (n) = O(nlogb a ) for some onstant  >
0 then T (n) = (nlogb a).
2. If f (n) = O(nlogb a) then T (n) =
(nlogb a log n).
3. If f (n) = (nlogb a+) for some onstant  >
0, and if af (n=b)  f (n) for some onstant
< 1 and suÆ iently large n then T (n) =
(f (n)).

" Department of C S E !
R. K. Ghosh Le ture Notes, January, 2003 28 of 395
# Algorithms: mathemati al ba kground

Master Theorem
 Intuitively f (n) is ompared with fun tion nlogb a
and solution to re urren e is determined by larger
of the two fun tions.
 In ase 1 nlogb a is larger and the solution is of
(
O nlogb a ).
 In ase 2 both are same. In this ase fa tor log n
is multiplied.
 In ase 1 nlogb a is smaller. The omplexity is
determined by f (n).

" Department of C S E !
R. K. Ghosh Le ture Notes, January, 2003 29 of 395
# Algorithms: mathemati al ba kground

Example: Using Master Theorem

 Consider T (n) = 9T (n=3) + n. Here a = 9; b =

3,and f (n) = n. Sin e f (n) (
O nlog3 9 
)
where  = 1, ase 1 applies.
 So time omplexity will be T n( ) = (n log3 9
)=
(n ) 2

 Consider T (n) = T (2n=3) + 1. Here a = 1; b =

3=2, and f (n) = 1. Sin e n
log3=2 1
) = n = 1.0

So, ase 2 applies.

 Thus time omplexity is (n logb a
log n) =
(log n)

" Department of C S E !
R. K. Ghosh Le ture Notes, January, 2003 30 of 395
# Algorithms: mathemati al ba kground

Example: Using Master Theorem

 Consider T (n) = 3T (n=4) + n log n. Here a =

3; b = 4, and f (n) = n log n. Sin e n log4 3
=
n , and f (n)=n
0:793
= n log n, where 
0:793 
=
0:207. So, ase 3 applies.
 Thus time omplexity is (f (n)) = (n log n).
 Consider T (n) = 2T (n=2) + n log n. Here a =
2; b = 2, and f (n) = n log n.
 Sin e n = n, and learly n log n > n.
log2 2

But only logarithmi ally whi h is asymtoti ally

smaller than n for any  > 0. So, ase 3 does
not apply. It falls in the gap between ase 2 and
ase 3.

" Department of C S E !
R. K. Ghosh Le ture Notes, January, 2003 31 of 395
# Algorithms: mathemati al ba kground

Re urren e and Annihilators

 Let A = fa ; a ; a ; : : :g be a given sequen e on
0 1 2
whi h we use a left shift operator E . Then
E A
= fa ; a ; a ; : : :g
1 2 3

 For on retness let T = f2 ; 2 ; 2 ; 2 ; : : :g, then

0 1 2 3

2
T = f2 ; 2 ; 2 ; 2 ; : : :g
1 2 3 4

 Similarly
E T
= f2 ; 2 ; 2 ; 2 ; : : :g
1 2 3 4

 Thus
(E 2)
T = f0; 0; 0; 0; : : :g
 Then (E 2) is said to annihiliate A.
 In our rst proof we use the annihiliator opera-
tion to solve Master theorem by applying te h-
nique of domain transformation.

" Department of C S E !
R. K. Ghosh Le ture Notes, January, 2003 32 of 395
# Algorithms: mathemati al ba kground

Proof of Master Theorem

 Let n = b .
k

 Applying domain transformation generalized re-

urren e an be expressed as
( ) = a
T (n=b) + (n )
T n k

= a
T (b =b) + (b
m mk
)
= a
T (b =b) + (b
m mk
)
= a
T (b ) + (b
m 1 mk
)
 Setting S (m) = T (b ), m

S (m) = a
S (m 1) + (b )
mk

S (m) a
S (m 1) = (b )
mk

 Substituting m by m + 1,
S (m + 1) a
S (m) = (b )
(m+1)k

 As b and k are onstants is onstant, there-

bk
fore, b(m+1)k = bk
bmk = (bmk ).

"  Thus S (m + 1)
( ) = (b )

!
mk
a S m

Department of C S E

R. K. Ghosh Le ture Notes, January, 2003 33 of 395

# Algorithms: mathemati al ba kground

Proof of Master Theorem

 Now we apply annihilator (E a) on the equation
S (m + 1) a
S (m) = (b )
mk

to obtain
(E a)
S (m) = (b ) mk

 To eliminate (b ) apply annihilator (E

mk
bk )
to the last equation
(E b
k
)(E a )
S (m) = 0

" Department of C S E !
R. K. Ghosh Le ture Notes, January, 2003 34 of 395
# Algorithms: mathemati al ba kground

Proof of Master Theorem

 If a = b then we have a double root for har-
k

a teristi equation. So general solution should

be
( ) = ( + m)a
S m 1
m
2
T (2 ) = ( + m)a
m
1 2
m

T (n) = ( + log n)a

1 2 b
logb n

= ( + log n)n
1 2 b
logb a

 So this proves the ase 2 of master theorem

T (n) = (log n
nlogb a).

" Department of C S E !
R. K. Ghosh Le ture Notes, January, 2003 35 of 395
# Algorithms: mathemati al ba kground

Proof of Master Theorem

 If a 6= b
k
then
S m( )= a + b 1
m
2
km

T (2 ) =
m
a + b
1
logb n
2
k logb n

T (n) = n + b
1
logb a
2
logb nk

= n + n 1
logb a
2
k

= (n + n ) logb a k

 So depending on whether n dominates nlogb a we

k

have the orresponding result of master theorem

overing ases 1 and 3 respe tively.

" Department of C S E !
R. K. Ghosh Le ture Notes, January, 2003 36 of 395
# Algorithms: mathemati al ba kground

Dire t Proof
 Let b > 1 and n be an exa t power of b, i.e.,
n =b k
for k > 0. So

( ) = (1)
T n
aT (n=b) + f (n)
, if n = 1
, if n = bi
where i is a positive integer.
 Apply iteration method
( ) = f (n) + af (n=b) + : : : + a
T n
logb n
T (1)
 Sin e a = n a T (1) = n
logb n logb a logb n logb a
.
 Thus
logb n
X1
T n ( ) = (n logb a
)+ j
a f n=b( j
)
j =0

" Department of C S E !
R. K. Ghosh Le ture Notes, January, 2003 37 of 395
# Algorithms: mathemati al ba kground

Proof: Case 1
 For proving ase 1, f (n) = nlogb a 

logb n
X1 logb n
X1  n logb a

f (n=b ) =


j j j
a a
bj
0 0
X 1  a j
logb n

= nlogb a 
b

(b )
 j

sin e b > a
logb n logb n
X1 X1
a
j

f (n=b )  n
j logb a 
(b ) j

0 0

" Department of C S E !
R. K. Ghosh Le ture Notes, January, 2003 38 of 395
# Algorithms: mathemati al ba kground

Proof: Case 1 ( ontd.)

 Sin e the sum on RHS of above inequality is now

in form of GP

(b ) = (b )b 1
logb n
X1  logb n
 j

1
1
0
n
= b 1
 Thus
logb n
X1
a
j

f (n=b ) = O(n
j logb a 
)
0

" Department of C S E !
R. K. Ghosh Le ture Notes, January, 2003 39 of 395
# Algorithms: mathemati al ba kground

Proof: Case 2
 In ase 2,
f n( ) = (n logb a
) ) f (n=b ) = ((n=b)
j logb a
)
 Thus
logb n logb n
X1 X1
a
j

f (n=b ) =
j
a
j

(n=b ) j logb a

0 0
logb n
X1
= nlogb a (a=(blogb a
)) j


0
n logb a
log b n

" Department of C S E !
R. K. Ghosh Le ture Notes, January, 2003 40 of 395