Introduction To Algorithms

CS 445
Spring 2005
Discussion Session 1
Instructor : Dr Alon Efrat
TA : Pooja Vaswani
02/07/2005
 Important Correction
• There is no unique set of values for n o and c in proving the asymptotic
bounds but for whatever value of n o you take, it should satisfy for all n
> no.( Note again, satisfy for all n > no.)

Example : To prove f(n) = O(g(n)

Suppose we get values c = 100 and n o = 5. This means the asymptotic

bound is valid for c = 100 and all n > 5. For the same question, we
could also have had c = 105 and n o = 1 which means, the asymptotic
bound is valid for c = 100 and all n > 1. Please note, that all the
values of n equal to and as well as greater than n o that should be
satisfied.

The mistake made during the discussion session was that we just kept
in mind the equality that n = n o = 5 and didn’t mention anything about
the greater than sign. ( Sorry about that !)

2
 Algorithm Analysis: Example
• Alg.: MIN (a[1], …, a[n])
m ← a[1];
for i ← 2 to n
if a[i] < m
then m ← a[i];
• Running time:
– the number of primitive operations (steps) executed
before termination
T(n) =1 [first step] + (n) [for loop] + (n-1) [if condition] +
(n-1) [the assignment in then] = 3n - 1
• Order (rate) of growth:
– The leading term of the formula
– Expresses the asymptotic behavior of the algorithm

3
 Typical Running Time Functions
• 1 (constant running time):
– Instructions are executed once or a few times

• logN (logarithmic)
– A big problem is solved by cutting the original problem in smaller
sizes, by a constant fraction at each step

• N (linear)
– A small amount of processing is done on each input element

• N logN
– A problem is solved by dividing it into smaller problems, solving
them independently and combining the solution

4
 Typical Running Time Functions
• N2 (quadratic)
– Typical for algorithms that process all pairs of data items (double
nested loops)

• N3 (cubic)
– Processing of triples of data (triple nested loops)

• NK (polynomial)

• 2N (exponential)
– Few exponential algorithms are appropriate for practical use

5
Why Faster Algorithms?

6
 Asymptotic Notations
• A way to describe behavior of functions in the limit
– Abstracts away low-order terms and constant factors

– How we indicate running times of algorithms

– Describe the running time of an algorithm as n grows to 

• O notation: asymptotic “less than”: f(n) “≤” g(n)

  notation: asymptotic “greater than”: f(n) “≥” g(n)

  notation: asymptotic “equality”: f(n) “=” g(n)

7
 Examples

– 2n = O(n ):
2 3 2n 2
≤ cn 3
 2 ≤ cn  c = 1 and n0= 2

– n2 = O(n2): n ≤ cn  c ≥ 1  c = 1 and n0= 1

2 2

– 1000n2+1000n = O(n2):

1000n2+1000n ≤ cn2 ≤ cn2+ 1000n  c=1001 and n0 = 1

– n = O(n2): n ≤ cn 2
 cn ≥ 1  c = 1 and n0= 1

8
 Examples

– 100n + 5 ≠ (n2)

To find c, n0 such that: 0  cn2  100n + 5

100n + 5  100n + 5n (for n  1) = 105n
cn2  105n  n(cn – 105)  0
Since n is positive  cn – 105  0  n  105/c
 contradiction: n cannot be smaller than a constant

9
 Examples
– 100n + 5 ≠ (n2)
For the above question, we had a solution proposed as follows to
prove 100n + 5 = (n2) :-
cn2 < 100n + 5
Let no = 100 . That gives us,
c x 10000 < 100 x 100 + 5
c < 10005 / 10000
So the contant c = 1.0005
But does this work when c = 1.0005 and n = 200 ( remember, it should
satisfy for all n > no . Here n is 200 and no is 100)
No, it doesn’t work. Hence you cannot prove that
100n + 5 = (n2)

10
 Examples

• Prove that 100n + 5 = O(n2)

– 100n + 5 ≤ 100n + n = 101n ≤ 101n2

for all n ≥ 5

n0 = 5 and c = 101 is a solution

– 100n + 5 ≤ 100n + 5n = 105n ≤ 105n2

for all n ≥ 1

n0 = 1 and c = 105 is also a solution

Must find SOME constants c and n0 that satisfy the asymptotic notation relation

11
 Mathematical Induction
• Used to prove a sequence of statements (S(1), S(2), …

S(n)) indexed by positive integers

• Proof:

– Basis step: prove that the statement is true for n = 1

– Inductive step: assume that S(n) is true and prove that S(n+1) is

true for all n ≥ 1

• The key to proving mathematical induction is to find case n

“within” case n+1

12
 Example
For example on how to solve using
Mathematical Induction, Refer to Text Book,
Appendix A, A.2 .

Also revise Summation formulas and properties

from Appendix A.

13
 Recurrences
Def.: Recurrence = an equation or inequality that
describes a function in terms of its value on
smaller inputs, and one or more base cases
• E.g.: T(n) = T(n-1) + n

• Useful for analyzing recurrent algorithms

• Methods for solving recurrences
– Substitution method
– Recursion tree method
– Master method
– Iteration method
14
 Iteration Method – Example
T(n) = n + 2T(n/2) Assume: n = 2k
T(n) = n + 2T(n/2)
= n + 2(n/2 + 2T(n/4))
= n + n + 4T(n/4)
= n + n + 4(n/4 + 2T(n/8))
= n + n + n + 8T(n/8) n = 2k
… = in + 2iT(n/2i) Taking lg on both sides
lg n = lg (2k )
= kn + 2kT(1) lg n = klg2
lg n = k x 1
= nlgn + nT(1) = Θ(nlgn) lg n = k

15
 Master’s method
• “Cookbook” for solving recurrences of the form:

n
T (n)  aT    f (n)
b
where, a ≥ 1, b > 1, and f(n) > 0

Case 1: if f(n) = O(nlogba -) for some  > 0, then: T(n) = (nlogba)

Case 2: if f(n) = (nlogba), then: T(n) = (nlogba lgn)

Case 3: if f(n) = (nlogba +) for some  > 0, and if

af(n/b) ≤ cf(n) for some c < 1 and all sufficiently large n, then:

T(n) = (f(n))
16
 Examples

T(n) = 2T(n/2) + n

a = 2, b = 2, log22 = 1

Compare nlog22 with f(n) = n

 f(n) = (n)  Case 2

 T(n) = (nlgn)

17