Chapter02 Duc Anany V. Levitin 3e
Chapter02 Duc Anany V. Levitin 3e
Chapter02 Duc Anany V. Levitin 3e
1
Lecture Contents
1. Analysis Framework
2. Asymptotic Notations and Basic Efficiency Classes
3. Mathematical Analysis of Nonrecursive Algorithms
4. Mathematical Analysis of Recursive Algorithms
5. Example: Fibonacci Numbers
6. Empirical Analysis of Algorithms
7. Algorithm Visualization
2
Analysis of Algorithms
• Issues:
- correctness
- time efficiency
- space efficiency
- optimality
• Approaches:
- theoretical analysis
- empirical analysis
3
Theoretical Analysis of Time Efficiency
4
Theoretical Analysis of Time Efficiency
input size
T(n) ≈ copC(n)
running time execution time Number of times
for basic operation basic operation is
executed
5
Input Size and Basic Operation Examples
Visiting a vertex or
Typical graph problem #vertices and/or edges
traversing an edge
6
Empirical Analysis of Time Efficiency
7
Worst-case, Best-case, Average-case Efficiencies
9
Example: Sequential Search
10
Example: Sequential Search
• Worst case
Cworst(n) = n
The last element is equal to a search key or
unsuccessful
• Best case
Cbest(n) = 1
The first element is equal to a search key
11
Example: Sequential Search
• Average case
- We have the following assumptions:
a. probability of successful search is p (0 ≤ p ≤ 1)
b. probability of first match occurring in ith
position is the same for every i.
12
Example: Sequential Search
• Average case
- successful search, probability of first match
occurring in ith position is p/n for every i, and
#comparisons made by the algorithm in such a
situation is obviously i.
- unsuccessful search, #comparisons is n with
probability of such a search being (1 – p).
13
Example: Sequential Search
• Average case
• Exact formula
e.g., C(n) = n(n-1)/2
• Approximate formula with specific multiplicative
constant. e.g., C(n) ≈ 0.5 n2
• Approximate formula with unknown
multiplicative constant. e.g., C(n) ≈ cn2
15
Order of Growth
16
Values of Some Important Functions as n →
17
Lecture Contents
1. Analysis Framework
2. Asymptotic Notations and Basic Efficiency
Classes
3. Mathematical Analysis of Nonrecursive Algorithms
4. Mathematical Analysis of Recursive Algorithms
5. Example: Fibonacci Numbers
6. Empirical Analysis of Algorithms
7. Algorithm Visualization
18
Asymptotic Order of Growth
22
Big-Omega
24
Big-Theta
• f(n) O(f(n))
• f(n) O(g(n)) iff g(n) (f(n))
• If f (n) O(g (n)) and g(n) O(h(n)) , then f(n)
O(h(n))
// Note similarity with a ≤ b
• If t1(n) O(g1(n)) and t2(n) O(g2(n)) , then
t1(n) + t2(n) O(max{g1(n), g2(n)})
(The analogous assertions are true for the Ω and
notations as well.)
26
Using Limits for Comparing Order of Growth
27
L’Hôpital’s Rule and Stirling’s Formula
L’Hôpital’s rule
Stirling’s formula
28
Example
1 constant
log n logarithmic
n linear
n log n n-log-n
n2 quadratic
n3 cubic
2n exponential
n! factorial
31
Lecture Contents
1. Analysis Framework
2. Asymptotic Notations and Basic Efficiency Classes
3. Mathematical Analysis of Nonrecursive
Algorithms
4. Mathematical Analysis of Recursive Algorithms
5. Example: Fibonacci Numbers
6. Empirical Analysis of Algorithms
7. Algorithm Visualization
32
Time Efficiency of Nonrecursive Algorithms
1in i2 = 12+22+…+n2
= n(n+1)(2n+1)/6 n3/3 (n3)
0in ai = 1 + a +…+ an
= (an+1 – 1)/(a – 1) for any a 1
0in 2i = 20 + 21 +…+ 2n
= 2n+1 – 1 (2n )
(ai ± bi ) = ai ± bi, cai = cai,
liuai = limai + m+1iuai
34
Example 1: Maximum Element
35
Example 2: Element Uniqueness Problem
36
Example 3: Matrix Multiplication
37
Example 4: Counting Binary Digits
Algorithm Binary(n)
//Input: A positive decimal integer n
//Output: The number of binary digits in n’s binary
representation
count ← 1
while n > 1 do
count ← count + 1
n ← n/2
return count
1. Analysis Framework
2. Asymptotic Notations and Basic Efficiency Classes
3. Mathematical Analysis of Nonrecursive Algorithms
4. Mathematical Analysis of Recursive
Algorithms
5. Example: Fibonacci Numbers
6. Empirical Analysis of Algorithms
7. Algorithm Visualization
39
Plan for Analysis of Recursive Algorithms
40
Plan for Analysis of Recursive Algorithms
41
Example 1: Recursive Evaluation of n!
Algorithm F(n)
//Computes n! recursively
//Input: A nonnegative integer n
//Output: The value of n!
if n = 0 return 1
else return F(n – 1) × n
42
Solving the Recurrence for M(n)
44
Solving Recurrence for Number of Moves
• Recurrence equation
M(n) = 22[2M(n – 3) + 1] + 2 + 1
= 23M(n – 3) + 22 + 2 + 1.
...
= 2iM(n – i) + 2i-1 + 2i-2 + … + 2 + 1
= 2iM(n – i) + 2i – 1. // Appendix A
...
// M(1) = 1 when i = n – 1
M(n) = 2n-1M(n – (n – 1)) + 2n-1 – 1
M(n) = 2n-1M(1) + 2n-1 – 1 = 2n-1 + 2n-1 – 1 =
M(n) = 2n – 1 (2n). // exponential algorithm
46
Example 3: Count #bits Recursively
Algorithm BinRec(n)
//Input: A positive decimal integer n
//Output: The number of binary digits in n’s binary
representation
if n = 1 return 1
else return BinRec(n/2) + 1
47
Example 3: Count #bits Recursively
• Recurrence relation
48
Example 3: Count #bits Recursively
→ A(2k) = A(1) + k = k
n = 2k → k = log2 n
50
Lecture Contents
1. Analysis Framework
2. Asymptotic Notations and Basic Efficiency Classes
3. Mathematical Analysis of Nonrecursive Algorithms
4. Mathematical Analysis of Recursive Algorithms
5. Example: Fibonacci Numbers
6. Empirical Analysis of Algorithms
7. Algorithm Visualization
51
Fibonacci Numbers
52
Computing the nth Fibonacci Number
n
F(n-1) F(n) 0 1
=
F(n) F(n+1) 1 1
for n ≥ 1
(log n)
53
Explicit Formula for the nth Fibonacci Number
54
Explicit Formula for the nth Fibonacci Number
55
Explicit Formula for the nth Fibonacci Number
56
Explicit Formula for the nth Fibonacci Number
57
Compute the nth Fibonacci Number Recursively
Algorithm F(n)
//Computes the nth Fibonacci number recursively
by using its definition
//Input: A nonnegative integer n
//Output: The nth Fibonacci number
if n ≤ 1 return n
else return F(n – 1) + F(n – 2)
58
Compute the nth Fibonacci Number Recursively
60
Compute the nth Fibonacci Number Iteratively
Algorithm Fib(n)
//Computes the nth Fibonacci number iteratively by using
its definition
//Input: A nonnegative integer n
//Output: The nth Fibonacci number
F[0] ← 0; F[1] ← 1
for i ← 2 to n do
F[i] ← F[i – 1] + F[i – 2]
return F[n]
62
Lecture Contents
1. Analysis Framework
2. Asymptotic Notations and Basic Efficiency Classes
3. Mathematical Analysis of Nonrecursive Algorithms
4. Mathematical Analysis of Recursive Algorithms
5. Example: Fibonacci Numbers
6. Empirical Analysis of Algorithms
7. Algorithm Visualization
63
Lecture Contents
1. Analysis Framework
2. Asymptotic Notations and Basic Efficiency Classes
3. Mathematical Analysis of Nonrecursive Algorithms
4. Mathematical Analysis of Recursive Algorithms
5. Example: Fibonacci Numbers
6. Empirical Analysis of Algorithms
7. Algorithm Visualization
64
Note: This service is not intended for secure transactions such as banking, social media, email, or purchasing. Use at your own risk. We assume no liability whatsoever for broken pages.
Alternative Proxies: