Recursion

Sections 6.1 and 6.2 (Rosen)

 Outline
• Introduction, Motivating Example
• Recurrence Relations
– Definition, general form, initial conditions, terms
• Linear Homogeneous Recurrences
– Form, solution, characteristic equation, characteristic polynomial, roots
– Second order linear homogeneous recurrence
• Double roots, solution, examples
• Single root, example
– General linear homogeneous recurrences: distinct roots, any multiplicity
• Linear Nonhomogenous Recurrences
• Other Methods
– Backward substitution
– Recurrence trees
– Generating functions
2
 Motivating Example: Factorial
• Recall the factorial function:
1 if n= 0 or 1
n! =
n.(n-1)! if n > 1
• Consider the following (recursive) algorithm for computing n!
Factorial
Input: nN
Output: n!
1. If (n=1) or (n=0)
2. Then Return 1
3. Else Return n  Factorial(n-1)
4. Endif
5. End
3
 Factorial: Analysis
How many multiplications M(x) does factorial perform?
• When n=1 we does not perform any
• Otherwise, we perform one…
• … plus how ever many multiplications we perform in the
recursive call Factorial(n-1)
• The number of multiplications can be expressed as a formula
(similar to the definition of n!
M(0) = 0
M(n) = 1 + M(n-1)
• This relation is known as a recurrence relation

4
 Recurrence Relations
• Definition: A recurrence relation for a sequence
{an} is an equation that expresses an in terms of one
or more of the previous terms in the sequence:
a0, a1, a2, …, an-1
for all integers nn0 where n0 is a nonnegative
integer.
• A sequence is called a solution of a recurrence if its
terms satisfy the recurrence relation

5
 Recurrence Relations: Solutions
• Consider the recurrence relation an=2an-1-an-2
• It has the following sequences an as solutions
– an= 3n
– an= n+1
– an=5
• The initial conditions + recurrence relation
uniquely determine the sequence

6
 Recurrence Relations: Example
• The Fibonacci numbers are defined by the recurrence
F(n) = F(n-1) +F(n-2)
F(1) = 1
F(0) = 1
• The solution to the Fibonacci recurrence is
fn = 1/5 ((1+5)/2)n – 1/5 ((1-5)/2)n

(The solution is derived in your textbook.)

7
 Recurrence Relations: General Form
• More generally, recurrences can have the form
T(n) = T(n-) + f(n), T() = c
or
T(n) = T(n/) + f(n), T() = c

• Note that it may be necessary to define several T(),

which are the initial conditions

8
 Recurrence Relations: Initial Conditions

• The initial conditions specify the value of the first few

necessary terms in the sequence.
• In the Fibonacci numbers, we needed two initial conditions:
F(0)=F(1)=1 since F(n) is defined by the two previous terms in
the sequence
• Initial conditions are also known as boundary conditions (as
opposed to general conditions)
• From now on, we will use the subscript notation, so the
Fibonacci numbers are:
fn = fn-1 + fn-2
f1 = 1
f0 = 1
9
 Recurrence Relations: Terms
• Recurrence relations have two parts: recursive terms and non-
recursive terms
T(n) = 2T(n-2) + n2 -10
• Recursive terms come from when an algorithms calls itself
• Non-recursive terms correspond to the non-recursive cost of
the algorithm: work the algorithm performs within a function
• We will see examples later. First, we need to know how to
solve recurrences.

10
 Solving Recurrences
• There are several methods for solving
recurrences
– Characteristic Equations
– Forward Substitution
– Backward Substitution
– Recurrence Trees
– Generating functions …

11
 Linear Homogeneous Recurrences
• Definition: A linear homogeneous recurrence relation of
degree k with constant coefficients is a recurrence relation of
the form
an = c1an-1 + c2an-2 + … + ckan-k
with c1, c2, …, ckR, ck 0.
• Linear: RHS is a sum of multiples of previous terms of the
sequence (linear combination of previous terms). The
coefficients are all constants (not functions depending on n)
• Homogeneous: no terms occur that are not multiples of aj’s
• Degree k: an is expressed in terms of (n-k)th term of the
sequence
12
 Linear Homogeneous Recurrences: Examples

• The Fibonacci sequence is a linear homogeneous

recurrence relation
• So are the following relations:
an = 4an-1 + 5an-2 + 7an-3
an = 2an-2 + 4an-4 + 8an-8
How many initial conditions do we need to specify
for these relations?
As many as the degree k: k=3, 8 respectively
• So, how do solve linear homogeneous recurrences?
13
 Solving Linear Homogeneous Recurrences
• We want a solution of the form an=rn where r is some real constant
• We observe that an=rn is a solution to a linear homogeneous recurrence if
and only if
rn = c1rn-1 + c2rn-2 + … + ckrn-k
• We can now divide both sides by rn-k, collect terms and we get a k-degree
polynomial
rk - c1rk-1 - c2rk-2 - … - ck = 0
• This equation is called the characteristic equation of the recurrence
relation
• The roots of this polynomial are called the characteristics roots of the
recurrence relation. They can be used to find the solutions (if they exist) to
the recurrence relation. We will consider several cases.

14
Second Order Linear Homogeneous Recurrences

• A second order linear homogeneous recurrence is a

recurrence of the form
an = c1an-1+ c2an-2
• Theorem (Theorem 1): Let c1, c2R and suppose that r2-c1r-
c2=0 is the characteristic polynomial of a 2nd order linear
homogeneous recurrence which has two distinct* roots r1,r2,
then {an} is a solution if and only if
an= 1r1n + 2r2n
for n=0,1,2,… where 1, 2 are constants dependent upon the
initial conditions

* We discuss single root later

15
 Second Order Linear Homogeneous
Recurrences: Example A (1)
• Find a solution to
an = 5an-1 - 6an-2
with initial conditions a0=1, a1=4
• The characteristic equation is
r2 - 5r + 6 = 0
• The roots are r1=2, r2=3
r2 - 5r + 6 = (r-2)(r-3)
• Using the 2nd order theorem we have a solution
an = 12n + 23n
16
 Second Order Linear Homogeneous
Recurrences: Example A (2)
• Given the solution
an = 12n + 23n
• We plug in the two initial conditions to get a system
of linear equations
a0 = 120 + 230
a1 = 121 + 231
• Thus:
1 = 1 + 2
4 = 21 + 32
17
 Second Order Linear Homogeneous
Recurrences: Example A (3)
1 = 1 + 2
4 = 21 + 32
• Solving for 1 = (1 - 2), we get
4 = 21 + 32
4 = 2(1-2) + 32
4 = 2 - 22 + 32
2 = 2
• Substituting for 1: 1 = -1
• Putting it back together, we have
an = 12n + 23n
an = -12n + 23n
18
 Second Order Linear Homogeneous
Recurrences: Example B (1)
• Solve the recurrence
an = -2an-1 + 15an-2
with initial conditions a0= 0, a1= 1
• If we did it right, we have
an = 1/8 (3)n - 1/8 (-5)n

• To check ourselves, we verify a0, a1, we compute a3 with both

equations, then maybe a4, etc.

19
 Single Root Case
• We can apply the theorem if the roots are distincts, i.e. r1r2
• If the roots are not distinct (r1=r2), we say that one
characteristic root has multiplicity two. In this case, we apply
a different theorem
• Theorem (Theorem2)
Let c1, c2R and suppose that r2 - c1r - c2 = 0 has only one
distinct root, r0, then {an} is a solution to an = c1an-1+ c2an-2 if
and only if
an= 1r0n + 2nr0n
for n=0,1,2,… where 1, 2 are constants depending upon the
initial conditions
20
 Single Root Case: Example (1)
• What is the solution to the recurrence relation
an = 8an-1 - 16an-2
with initial conditions a0= 1, a1= 7?
• The characteristic equation is:
r2 – 8r + 16 = 0
• Factoring gives us:
r2 – 8r + 16 = (r-4)(r-4), so r0=4
• Applying the theorem we have the solution:
an= 1(4)n + 2n(4)n
21
 Single Root Case: Example (2)
• Given: an= 1(4)n + 2n(4)n
• Using the initial conditions, we get:
a0= 1 = 1(4)0 + 20(4)0 = 1
a1= 7 = 1(4) + 21(4)1 = 41 + 42
• Thus: = 1 = 1, 2 = 3/4
• The solution is
an= (4)n + ¾ n (4)n
• Always check yourself…
22
General Linear Homogeneous Recurrences
• There is a straightforward generalization of these cases to
higher-order linear homogeneous recurrences
• Essentially, we simply define higher degree polynomials
• The roots of these polynomials lead to a general solution
• The general solution contains coefficients that depend only on
the initial conditions
• In the general case, the coefficients form a system of linear
equalities

23
General Linear Homogeneous Recurrences:
Distinct Roots
• Theorem (Theorem 3)
Let c1,c2,..,ck R and suppose that the characteristic equation
rk - c1rk-1 - c2rk-2 - … - ck = 0
has k distinct roots r1,r2, …,rk. Then a sequence {an} is a
solution of the recurrence relation
an = c1an-1 + c2an-2 + … + ckan-k
if and only if
an = 1r1n + 2r2n + … + krkn
for n=0,1,2,… where 1,2,…,k are constants depending upon
the initial conditions

24
 Higher Order Linear Homogeneous
Recurrences: Example
• Find a solution to
an = 6an-1 - 11an-2 + 6an-3
with initial conditions a0=2, a1=5, a2= 15
• The characteristic equation is
r3 – 6r2 + 11r - 6 = 0
• The roots are r1=1, r2=2 and r3=3

• Using the theorem we have a solution

an = 11n + 22n + 33n
25
Higher Order Linear Homogeneous Recurrences:
Example
• Given the solution we plug in the three initial
conditions to get a system of linear equations
2 = 1 + 2 + 3
5 = 1 + 2.2+ 3.3
15 = 1 + 2.4 + 3.9
• Solving, we get 1 = 1, 2 = -1, and 3 = 2
• Putting it back together, we have
an = 1 - 2n + 23n

26
General Linear Homogeneous Recurrences:
Any Multiplicity
• Theorem (Theorem 4)
Let c1,c2,..,ck R and suppose that the characteristic equation
rk - c1rk-1 - c2rk-2 - … - ck = 0
has t roots with multiplicities m1,m2, …,mt. Then a sequence
{an} is a solution of the recurrence relation
an = c1an-1 + c2an-2 + … + ckan-k
if and only if an = (1,0 + 1,1n + … + 1,m1-1nm1-1) r1n +
(2,0 + 2,1n + … + 2,m2-1nm2-1) r2n + ...
(t,0 + t,1n + … + t,mt-1nmt-1) rtn
for n=0,1,2,… where i,j are constants for 1  i  t and
0  jmi-1 depending upon the initial conditions
27
General Linear Homogeneous Recurrences:
Any Multiplicity

• Suppose the characteristic equation has the roots

2, 2, 2, 5, 5, and 9

Then the solution is

an = (1,0 + 1,1n + 1,2n2) 2n + (2,0 + 2,1n) 5n + 3,0 9n

28
General Linear Homogeneous Recurrences:
Any Multiplicity
• Find a solution to an = -3an-1 - 3an-2 - an-3
with initial conditions a0=1, a1=-2, a2= -1
• The characteristic equation is r3 + 3r2 + 3r + 1 = 0 => (r + 1)3 = 0
• It has single root r = -1 with multiplicity 3.
• Hence the solution is an = (1,0 + 1,1n + 1,2n2) (-1)n
• Given the solution we plug in the three initial conditions to get a system of
linear equation 1 = 1,0
-2 = - 1,0 - 1,1 - 1,2
-1 = - 1,0 + 21,1 + 41,2
• Solving, we get 1,0 = 1, 1,1 = 3, and 1,2= -2
• Putting it back together, we have
an = (1 + 3n - 2n2)(-1)n

29
Linear NonHomogeneous Recurrences
• For recursive algorithms, cost function are often not homogeneous
because there is usually a non-recursive cost depending on the input size
• Such a recurrence relation is called a linear nonhomogeneous recurrence
relation
• Such functions are of the form
an = c1an-1 + c2an-2 + … + ckan-k + f(n)
• f(n) represents a non-recursive cost. If we chop it off, we are left with
an = c1an-1 + c2an-2 + … + ckan-k
which is the associated homogeneous recurrence relation
• Every solution of a linear nonhomogeneous recurrence relation is the sum
of
– a particular solution and
– a solution to the associated linear homogeneous recurrence relation

30
Solving Linear NonHomogeneous Recurrences (1)

• Theorem (Theorem 5)
If {an(p)} is a particular solution of the
nonhomogeneous linear recurrence relation with
constant coefficients
an = c1an-1 + c2an-2 + … + ckan-k + f(n)
then every solution is of the form {an(p) + an(h)} where
{an(h)} is a solution of the associated homogeneous
recurrence relation
an = c1an-1 + c2an-2 + … + ckan-k

31
Solving Linear NonHomogeneous Recurrences (2)

• There is no general method for solving such

relations.
• However, we can solve them for special cases
• In particular, if f(n) is
– a polynomial function
– exponential function, or
– the product of a polynomial and exponential
functions,
then there is a general solution
32
Solving Linear NonHomogeneous Recurrences (3)

• Theorem (Theorem 6)
Suppose {an} satisfies the linear nonhomogeneous
recurrence relation
an = c1an-1 + c2an-2 + … + ckan-k + f(n)
where c1,c2,..,ck R and
f(n) = (btnt + bt-1nt-1 + .. + b1n + b0) sn
where b0,b1,..,bn,s R
… continues

33
Solving Linear NonHomogeneous Recurrences (4)

• Theorem (Theorem 6)… continued

When s is not a root of the characteristic equation of the
associated linear homogeneous recurrence relation, there is a
particular solution of the form
(ptnt+ pt-1nt-1+ … +p1n + p0) sn
When s is a root of this characteristic equation and its
multiplicity is m, there is a particular solution of the form
nm(ptnt+ pt-1nt-1+ … +p1n + p0) sn

34
 Linear NonHomogeneous Recurrences: Examples

• The examples in the textbook are quite good and

illustrate how to solve simple nonhomogeneous
relations

35
 Other Methods
• When analyzing algorithms, linear homogeneous recurrences
of order greater than 2 hardly ever arise in practice
• We briefly describe two unfolding methods that work for a lot
of cases
– Backward substitution: this works exactly as its name suggests.
Starting from the equation itself, work backwards, substituting values
of the function for previous ones
– Recurrence trees: just as powerful, but perhaps more intuitive, this
method involves mapping out the recurrence tree for an equation.
Starting from the equation, you unfold each recursive call to the
function and calculate the non-recursive cost at each level of the tree.
Then, you find a general formula for each level and take a summation
over all such levels

36
Backward Substitution: Example (1)
• Give a solution to
T(n)= T(n-1) + 2n
where T(1)=5

• We begin by unfolding the recursion by a simple substitution

of the function values

• We observe that
T(n-1) = T((n-1) -1) + 2(n-1) = T(n-2) + 2 (n-1)

• Substituting into the original equation

T(n)=T(n-2)+2(n-1)+2n
37
Backward Substitution: Example (2)
• If we continue to do that we get
T(n) = T(n-2) + 2(n-1) + 2n
T(n) = T(n-3) + 2(n-2) + 2(n-1) + 2n
T(n) = T(n-4) + 2(n-3) + 2(n-2) + 2(n-1) + 2n
…..
T(n) = T(n-i) + j=0i-1 2(n - j) function’s value at the ith iteration
• Solving the sum we get
T(n) = T(n-i) + 2n(i-1) – 2(i-1)(i-1+1)/2 + 2n
T(n) = T(n-i) + 2n(i-1) – i2 + i + 2n

38
Backward Substitution: Example (3)
• We want to get rid of the recursive term
T(n) = T(n-i) + 2n(i-1) – i2 + i + 2n
• To do that, we need to know at what iteration we reach our
based case, i.e. for what value of i can we use the initial
condition T(1)=5?
• We get the base case when n-i=1 or i=n-1
• Substituting in the equation above we get
T(n) = 5 + 2n(n-1-1) – (n-1)2 + (n-1) + 2n
T(n) = 5 + 2n(n-2) – (n2-2n+1) + (n-1) + 2n = n2 + n + 3

39
 Recurrence Trees (1)
• When using recurrence trees, we graphically represent the
recursion
• Each node in the tree is an instance of the function. As we
progress downward, the size of the input decreases
• The contribution of each level to the function is equivalent to
the number of nodes at that level times the non-recursive
cost on the size of the input at that level
• The tree ends at the depth at which we reach the base case
• As an example, we consider a recursive function of the form
T(n) = T(n/) + f(n), T() = c

40
 Recurrence Trees (2)
Iteration
Cost
0 TT(n)
f(n)

1
T(n/) T(n/)    T(n/) f(n/)

2 T(n/2)      

T(n/2) T(n/2) T(n/2) 2f(n/2)
.
.
.
.
i
i f(n/i)
.
.
.
.
log n

41
 Recurrence Trees (3)
• The total value of the function is the
summation over all levels of the tree

• Consider the following concrete example

T(n) = 2T(n/2) + n, T(1)= 4

42
 Recurrence Tree: Example (2)
Iteration
Cost
0 T(n)
n

1 T(n/2) T(n/2)
n/2 +n/2

T(n/4) T(n/4) T(n/4) T(n/4) 4. n/4

2
8.n/8
. T(n/8) T(n/8) T(n/8) T(n/8) T(n/8) T(n/8) T(n/8) T(n/8)
.
.
.
i
2i(n/2i)
.
.
.
.
log2 n

43
 Recurrence Trees: Example (3)
• The value of the function is the summation of
the value of all levels.
• We treat the last level as a special case since
its non-recursive cost is different

44
 Smoothness Rule
• In the previous example, we make the following assumption
n has a power of two (n=2k)
This assumption is necessary to get a nice depth of log(n) and
a full tree
• We can restrict consideration to certain powers because of
the smoothness rule, which is not studied in this course.
• For more information about that rule, consult textbooks on
Algorithms (The Design & Analysis of Algorithms by A. Levitin)

45