ADA_Unit-1

Analysis and Design of Algorithms – Unit - 1
Definition of Algorithm
An algorithm is a sequence of unambiguous instructions for solving a problem,
i.e., for obtaining a required output for any legitimate input in a finite amount
of time.
Problem
Algorithm
Input “Computer” Output
Figure: Notion of Algorithm.
Properties of an Algorithm
 Definiteness: Each instruction in the algorithm must be clear and

unambiguous.
 Finiteness: Algorithm must terminate after a finite number of steps.
 Effectiveness: Each step must be sufficiently simple and basic. It

should be possible to carry out the instructions of the algorithm.
 Input: Range of inputs for which an algorithm works has to be clearly

specified. Algorithm may accept zero or more inputs.
 Output: Algorithm must produce at least one output.
Smt. Kavitha M, Assistant professor, Dept. of CSE, SIT, Tumkur-3 Page 1

Algorithms with different ideas to compute the GCD of two

non-zero positive integers
Euclid’s algorithm to compute gcd(m,n)

 Euclid’s algorithm is based on repeated application of equality
gcd(m,n) = gcd(n, m mod n)

until the second number becomes 0
 Example:
gcd(60,24) = gcd(24,12) = gcd(12,0) = 12
ALGORITHM Euclid(m, n)
Step 1 If n = 0, return m and stop; otherwise proceed to Step 2

Step 2 Divide m by n and assign the value of the remainder to r
Step 3 Assign the value of n to m and the value of r to n. Go to Step 1.
PSEUDOCODE
ALGORITHM Euclid(m, n)
// computes gcd(m, n) by Euclid’s algorithm

// Input: Two nonnegative, not-both-zero intgers m and n
// Output: Greatest common divisor of m & n
while n ≠ 0 do
r ← m mod n
m← n
n←r
return m
Consecutive Integer Checking Algorithm to compute gcd(m,n)

Step 1 Assign the value of min{m,n} to t
Step 2 Divide m by t. If the remainder of this division is 0, goto Step 3;

otherwise, go to Step 4

Step 3 Divide n by t. If the remainder of this division is 0, return the value of t

as answer and stop;
otherwise, proceed to Step 4
Step 4 Decrease the value of t by 1 and go to Step 2
• Example: gcd(10,6) = 2
t m%t n%t
6 10 % 6 = 4
5 10 % 5 = 0 6%5=1
4 10 % 4 = 2
3 10 % 3 = 1
2 10 % 2 = 0 6%2=0
2 is the GCD, since m % t and n % t are zero.
Note: Unlike Euclid’s algorithm, this algorithm, in the form presented, does not
work correctly when one of its input numbers is zero.
Middle - school procedure to computer gcd(m,n)

Step 1 Find the prime factors of m.
Step 2 Find the prime factors of n.
Step 3 Identify all the common factors in the two prime expansions found in
step1 and step2 (If p is a common factor occurring pm and pn times in
m and n, respectively, it should be repeated min{pm,pn} times.)
Step 4 Compute the product of all the common factors and return it as
gcd(m,n)
• Example:
If m = 60 and n = 24 then
60 = 2 . 2 . 3 . 5
24 = 2 . 2 . 2 . 3
gcd(60,24) = 2 . 2 . 3 =12
This is not a legitimate algorithm since steps 2 and 3 are not defined
clearly.

Generating prime numbers using Sieve of Eratosthenes:
 The algorithm starts by initializing a list of prime candidates with

consecutive integers from 2 to n.
 Then, on the first iteration of the algorithm, it eliminates from the list all
multiples of 2, i.e., 4, 6, and so on.
 Then it moves to the next item on the list, which is 3, and eliminates its
multiples.
 No pass for number 4 is needed since 4 itself and all its multiples are also
multiples of 2 which were already eliminated on a previous pass.
 The next number on list, which is used on the third pass is 5.
 The algorithm continues in this fashion until no more numbers can be
eliminated from the list.
 The remaining integers of the list are the primes needed.
Note: If p is a number whose multiples are being eliminated, then p.p should
not be greater than n, and therefore p cannot exceed √n .
ALGORITHM Sieve(n)
// Implements the Sieve of Eratosthenes
// Input: An integer n ≥ 2
// Output: Array L of all prime numbers less than or equal to n.
for p ← 2 to n do A[p] ← p
for p ← 2 to √n do
if A[p] ≠ 0 // p hasn’t been eliminated on previous passes
j←p*p
while j ≤ n do
A[j] ← 0 // mark element as eliminated
j←j+p
// Copy the remaining elements of A to array L of primes

i←0
for p ← 2 to n do
if A[p] ≠ 0
L[i] ← A[p]
i←i+1
return L

Example: n = 25
2, 3, 5, 7, 11, 13, 17, 19, 23 are the generated prime numbers.
Fundamentals of Algorithmic Problem Solving
 We can consider algorithms to be procedural solutions to problem. These

solutions are not answers but specific instructions for getting answers.
 Sequence of steps one typically goes through in designing and analyzing
an algorithm are as shown in below figure.

 Understanding the problem:
 From a practical perspective, the first thing you need to do before

designing an algorithm is to understand completely the problem
given.
 Read the problem’s description carefully and ask questions if you
have any doubts about the problem, do a few small examples by
hand, think about special cases, and ask questions again if needed.
 An input to an algorithm specifies an instance of the problem the
algorithm solves.
o It is very important to specify exactly the range of instances the
algorithm needs to handle. If you fail to do this, your algorithm
may work correctly for a majority of inputs but crash on some
“boundary” values.
 A correct algorithm is not one that works most of the time, but one
that works correctly for all legitimate inputs.
 If you reject this first step then you will run the risk of unnecessary
rework.
 Activities involved before going for designing the algorithm :
 Ascertaining the capabilities of a Computational Device:
 The central assumption of the Random Access Machine model is

that instructions are executed one after another, one operation at
a time. Accordingly, algorithms designed to be executed on such
machines are called Sequential algorithms.
 Some newer computers may be capable of executing the
operations concurrently, i.e., in parallel. Algorithms that take
advantage of this capability are called Parallel algorithms.

 Make sure about the capabilities of a computational device before

designing so as to choose among sequential algorithms or
parallel algorithms.
 There are some problems that are very complex and have to
process huge volumes of data or deal with applications where
time is critical. In such situations, we must be aware of the speed
and memory available on a particular computer system.
 Choosing between Exact and Approximate problem solving:
 Make decision to choose between solving the problem exactly

(Exact algorithm) or solving it approximately (approximation
algorithm).
 Why do we go for approximation algorithms?
- There are problems that cannot be solved exactly for most of
their instances; examples include extracting square roots,
solving non-linear equations, and evaluating definite integrals.
- Available algorithms for solving a problem exactly can be very
slow because of the problem’s intrinsic complexity.
- Approximation algorithm can be a part of a more sophisticated
algorithm that solves a problem exactly.
 Deciding on Appropriate Data Structure:
 Once we have selected the type of algorithm (Sequential/Parallel,

Exact/Approximation), we have to choose proper data structure
to represent the data and for easy manipulation to achieve
correct result. Algorithms implemented with proper data
structure will result in a good program.
Algorithms + Data Structures = Programs

 Few algorithm design techniques depend intimately on

structuring or restructuring data specifying a problem’s
instance. Data Structure is crucially important for both design
and analysis of algorithms.
 Algorithm Design Techniques:
 An algorithmic design technique is a general approach to solving

problems algorithmically that is applicable to a variety of problems
from different areas of computing.
 These design techniques provides guidance for designing
algorithms for new problems i.e., problems for which there is no
known satisfactory algorithm.
 Every technique will not be applicable to very problem that we
encounter, but they do constitute a powerful collection of tools for
solving a problem.
 Algorithm design techniques make it possible to classify algorithms
according to an underlying design idea; they serve as a natural
way to both categorize and study algorithms.
 Designing an Algorithm:
 Methods of Specifying an Algorithm:
 In the earliest days of computing, flowchart was used for

expressing an algorithm. It used collection of connected
geometric shapes containing descriptions of the algorithm’s
steps. This is suitable for small, simple problems but, not for all
types.
 There are two most widely used options for specifying algorithms:

o Using a Natural Language: Each user may interpret the

statements in different ways. Ambiguity of any natural
language makes the description of algorithms difficult.
o Pseudocode: It is a mixture of a natural language and
programming language-like constructs. A pseudocode is
usually more precise than a natural language.
 Proving an Algorithm’s Correctness:
 Once an algorithm has been specified, you have to prove its

correctness i.e., you have to prove that the algorithm yields a
required result for every legitimate input in a finite amount of time.
Example: In GCD algorithm, gcd(m,n) = gcd(n,m mod n), proof is the
simple observation that the second number gets smaller in every
iteration of the algorithm, and the fact that the algorithm stops
when the second number becomes 0.
 For some algorithms, a proof of correctness is quite easy; for others,
it can be quite complex.
 A common technique for proving correctness is to use
Mathematical Induction because an algorithm’s iterations
provide a natural sequence of steps needed for such proofs.
 In order to show that an algorithm is incorrect, you need just one
instance of its input for which the algorithm fails.
 If the algorithm is found to be incorrect, you need to either redesign
it under the same decisions regarding the data structures, the
design technique, and so on, or, to reconsider one or more of those
decisions.
 The concept of correctness for approximation algorithms is less
straightforward than it is for exact algorithms. For an approximation
algorithm, we usually would like to be able to show that error
produced by the algorithm does not exceed a predefined limit.

 Analyzing an Algorithm:
 There are two kinds of algorithm efficiency:

 Time efficiency indicates how fast the algorithm runs
 Space efficiency indicates how much memory the algorithm
needs
 Other desirable characteristics of an algorithm are

 Simplicity: Simpler algorithms are easier to understand and
easier to program; consequently, the resulting programs usually
contain fewer bugs.
 Generality: We should write generalized algorithms.
 If you are not satisfied with the algorithm’s efficiency, simplicity, or

generality, you must return to the drawing board and redesign the
algorithm.
 Coding an Algorithm:
 Algorithm is coded using suitable data structure in programming

language.
 The program can be tested to know the actual statistics about the
algorithm’s consumption of time and space requirements.
 If less efficient then you can fine tune the code to improve the
speed or you can go for better algorithm.

Fundamentals of the Analysis of Algorithm Efficiency
Analysis Framework
 Analysis of algorithms means investigation of an algorithm’s efficiency
with respect to two resources: running time and memory space.
 Analysis refers to the task of determining how much computing

time and storage an algorithm requires.
o Space complexity of an algorithm is the amount of memory

it needs to run to completion.
o Time complexity of an algorithm is the amount of computer

time it needs to run to completion.
Measuring an Input’s Size

 Time complexity depends on the number of inputs also. i.e., the running
time of an algorithm increases with the input size.
Example: It takes longer time to sort larger arrays, to multiply larger

matrices and so on.
 Therefore, it is logical to investigate an algorithm’s efficiency as a

function of some parameter n indicating the algorithm’s input size.
 In most cases, selecting parameter n is straight forward.
Example:
 For problems of sorting, searching, finding largest element etc.,

the size metric will be the size of the list.

 For problem of evaluating a polynomial P(x) = an xn + . . . + a0 of

degree n, the input size metric will be the polynomial’s degree or
the number of coefficients, which is larger by one than its degree.
 For problem of computing the product of two n x n matrices, the

natural size measures are frequently used matrix order n and the
total number of elements N in the matrices being multiplied.
 The choice of the appropriate size metric can be influenced by

operations of the algorithm.
Example: Consider a spell-checking algorithm:
 If the algorithm examines individual characters

then the size measure will be the number of
characters in the input.
 If the algorithm works by processing words, then

the size measure will be the number of words in
the input.
 The Input size metric for algorithms involving properties of number

(e.g., checking whether a given integer n is prime) will be the
number b of bits in the n’s binary representation:
b= log2 n + 1
Units for Measuring Running Time

 Some standard unit of time measurements – a second, a millisecond, and
so on can be used to measure the running time of a program
implementing the algorithm.

 The obvious drawbacks to above approach are:
 dependence on the speed of a particular computer,
 dependence on the quality of a program implementing the

algorithm
 compiler used in generating the machine code,
 difficulty of clocking the actual running time of the program.
 For measuring the algorithm’s efficiency, we need to have a metric that

does not depend on these hardware and software factors.
To measure an algorithm’s efficiency:
 One possible approach is to count the number of times each of the

algorithm’s operations are executed. This approach is both
excessively difficult and usually unnecessary.
 Another approach is to identify the basic operation(primitive

operation), i.e., the operation contributing the most to the total running
time, and compute the number of times the basic operation is executed
on inputs of size n.
Example:
 Sorting algorithms works by comparing elements of a list being

sorted with each other. For such algorithms, basic operation is a
key comparison.
 Matrix multiplication and polynomial evaluation requires two

arithmetic operations: multiplication and addition. On most

computers, multiplication of two numbers takes longer time than

addition. Hence, the basic operation considered is
multiplication.
 The established framework for the analysis of an algorithm’s time

efficiency suggests measuring time efficiency by counting the number of
times the algorithm’s basic operation is executed on input’s of size n.

 Let, Cop be the execution time of an algorithm’s basic operation on a

particular computer.
C(n) be the number of times this basic operation needs to be executed for
this algorithm.
Then, the running time T(n) can be estimated by
T(n) ≈ Cop . C(n)
 Note: Cop is an approximation & C(n) does not contain any information
about operations that are not basic. Therefore, above formula can give a
reasonable estimate of the algorithm’s running time.
Problem: Assuming C(n) = 1 n (n – 1), how much longer will the algorithm
2
run if we double its input size?
Solution: C(n) = 1 n (n – 1) = 1 n2 - 1 n ≈ 1 n2
2 2 2 2
Therefore, T(2n) ≈ Cop . C(2n) ≈ ½(2n)2 = 4

T(n) Cop . C(n) ½n 2
Note: The Efficiency analysis framework ignores multiplicative constants and

concentrates on the basic operation count’s order of growth for large-size
inputs.
Orders of Growth
 For GCD(m, n) of two small numbers, it is not clear how much more
efficient Euclid’s algorithm is, compared to other two algorithms.
 It is only when we have to find the GCD of two large numbers (eg.,
GCD(31415, 14142)) that the difference in algorithm efficiencies
becomes both clear and important.
 For large values of n, it is the function’s order of growth that counts.

 The function growing the slowest among these is the logarithmic

function.
 A program implementing an algorithm with a logarithmic basic-

operation count runs practically instantaneously on inputs of all
realistic sizes.
 Exponential function 2n and the factorial function n! grows so fast

that their values become large even for small values of n.
 Algorithms that require an exponential number of operations are

practical for solving only problems of very small sizes.
 Another way to appreciate the qualitative difference among the orders of

growth of the functions is to consider how they react to, say, a twofold
increase in the value of their argument n:
 The logarithmic function log2 n increases in value by just 1.

(Because, log2 2n = log2 2 + log2 n = 1 + log2 n)

 The linear function n increases twofold.

 The “n-log-n” function n log2 n increases slightly more than
twofold.
 The quadratic function n2 increases fourfold.
(Because, (2n)2 = 4n2).
 The cubic function n3 increases eightfold.
(Because, (2n)3 = 8n3).
 The value of exponential function 2n gets squared.
(Because, 22n = (2n)2).
 The factorial function n! increases much more than value of 2n.
Worst-Case, Best-Case and Average-Case Efficiencies
 There are many algorithms for which running time depend not only on
an input’s size, but also on the specifics of a particular input.
Sequential Search Algorithm
ALGORITHM SequentialSearch(A[0…n-1], k)
//Searches for a given value in a given array by sequential search
//Input: An array A[0…n-1] and a search key k
//Output: The index of the first element of A that matches k or -1 if
// there are no matching elements.
i ← 0
while i < n and A[i] ≠ k do
i ← i+1
if i < n return i
else return -1
 Running Time of this algorithm can be quite different for the same list of
size n.

Worst – Case Efficiency:
 When there are no matching elements or the first matching element

happens to be the last one in the list, the algorithm makes the largest
number of key comparisons.
Cworst(n) = n
 The Worst – Case Efficiency of an algorithm is its efficiency for the

worst – case input of size n, which is an input for which the algorithm
runs the longest among all possible inputs of that size.
 Analyze the algorithm to see what kind of inputs yields the largest value
of the basic operation’s count C(n) among all possible inputs of size n
and then compute Cworst(n).
Best – Case Efficiency:
 The Best – Case Efficiency of an algorithm is its efficiency for the

best – case input of size n, which is an input for which the algorithm
runs the fastest among all possible inputs of that size.
 Analyze the algorithm to see what kind of inputs yields the smallest
value of the basic operation’s count C(n) among all possible inputs of size
n and then compute Cbest(n).
 For Sequential search, best-case inputs are lists of size n with their first
elements equal to a search key.
Cbest(n) = 1 for successful search.

Cbest(n) = n for unsuccessful search.

Average – Case Efficiency:
 Neither the worst-case analysis nor its best-case counterpart yields the
necessary information about an algorithm’s behavior on a typical or
random input. This is provided by Average-case analysis.
Assumptions for Sequential Search:
(a) The probability of a successful search is equal to P (0 ≤ P ≤ 1).
(b) Probability of first match occurring at ith position is same for every i.
Now, find the average number of key comparisons Cavg(n) as follows:
 In case of successful search, the probability of the first match occurring

in ith position of the list is p/n for every i and number of comparisons is
i.
 In case of unsuccessful search, the number of comparisons is n, with the

probability of such a search being (1-P).

Recapitulation of the Analysis Framework
 Both time and space efficiencies are measured as functions of the

algorithm’s input size.
 Time efficiency is measured by counting the number of times the

algorithm’s basic operation is executed.
 The efficiencies of some algorithms may differ significantly for inputs of

the same size. For such algorithms, we need to distinguish between the
worst-case, average-case, and best-case efficiencies.
 The framework’s primary interest lies in the order of growth of the

algorithm’s running time as its input size goes to infinity.
Asymptotic Notations
 The Efficiency analysis framework concentrates on the order of growth

of an algorithm’s basic operation count as the principal indicator of
the algorithm’s efficiency.
 To Compare and rank the order of growth of algorithm’s basic

operation count, computer scientists use three notations:
O( Big-oh), Ω( Big-omega), and Θ(Big-theta).
 Let, t(n) and g(n) can be any two nonnegative functions defined on the set
of natural numbers.
 t(n) will be an algorithm’s running time (usually indicated by its basic

operation count C(n)), and g(n) will be some simple function to compare
the count with.

Informal Introduction to Asymptotic Notations
 Informally, O(g(n)) is the set of all functions with a smaller or same

order of growth as g(n).
Examples:
 Informally, Ω(g(n)) stands for the set of all functions with a larger or
same order of growth as g(n).
Examples:

 Informally, Θ(g(n)) is the set of all functions that have the same order of
growth as g(n).
Example:
 Every quadratic function ax2 + bx + c with a > 0 is in Θ(n2).
Problem: Using informal definitions of O, Θ and Ω notations, determine

whether the following assertions are true or false:
Note:
• If the order of growth of algorithm1’s basic operation count is higher

than the order of growth of algorithm2’s basic operation count then
algorithm2 can run faster than algorithm1.

• One algorithm is more efficient than another algorithm if its worst-case

running time has a lower order of growth.
Formal Definition of Asymptotic Notations
O – Notation
A function t(n) is said to be in O(g(n)), denoted t(n) Є O(g(n)), if t(n) is

bounded above by some constant multiple of g(n) for all large n, i.e., if
there exist some positive constant c and some nonnegative integer n0
such that
t(n) ≤ c.g(n) for all n ≥ n0
Example Problems:
Prove the following inequalities

Ω – Notation
A function t(n) is said to be in Ω(g(n)), denoted t(n) Є Ω(g(n)), if t(n) is

bounded below by some positive constant multiple of g(n) for all large n,
i.e., if there exist some positive constant c and some nonnegative integer
n0 such that
t(n) ≥ c.g(n) for all n ≥ n0
Example Problem: Prove the following inequalities

Θ-notation
A function t(n) is said to be in Θ(g(n)), denoted t(n) Є Θ(g(n)), if t(n) is

bounded both above and below by some positive constant multiples of
g(n) for all large n, i.e., if there exist some positive constant c1 and c2
and some nonnegative integer n0 such that
c2 g(n) ≤ t(n) ≥ c1g(n) for all n ≥ n0
Example Problem:
Prove the following inequalities

Useful Property Involving the Asymptotic Notations
The following property is useful in analyzing algorithms that comprise

two consecutively executed parts.
Theorem: If t1(n) Є O(g1(n)) and t2(n) Є O(g2(n)) then
t1(n) + t2(n) Є O(max{g1(n), g2(n)}).
Proof: a1, b1, a2, & b2 are four arbitrary real numbers.
If a1 ≤ b1 & a2 ≤ b2 then a1 + a2 ≤ 2 max{b1, b2}.
Since t1(n) Є O(g1(n)), there exists some positive constant c1 and
some nonnegative integer n1 such that
t1(n) ≤ c1g1(n) for all n ≥ n1
Similarly, since t2(n) Є O(g2(n)),
t2(n) ≤ c2g2(n) for all n ≥ n2
Let us denote c3 = max{c1,c2} & consider n ≥ max {n1,n2}
t1(n) + t2(n) ≤ c1g1(n) + c2g2(n)
≤ c3g1(n) + c3g2(n) = c3[g1(n) + g2(n)]
≤ c3 2 max{g1(n), g2(n)}
Hence, t1(n) + t2(n) Є O(max{g1(n), g2(n)}) with c = 2c3=2max{c1, c2} and
n0 = max{n1,n2}
Note: Algorithm’s overall efficiency is determined by the part with a

larger order of growth i.e., its least efficient part.

Basic Efficiency Classes
Class Name Comments

1 constant Short of Best-case efficiencies
log n logarithmic Typically a result of cutting a problem’s size by a constant
factor on each iteration of the algorithm
n linear Algorithms that scan a list of size n belong to this class
n log n n-log-n Many divide-and-conquer algorithms in average case fall
into this category
n2 quadratic Typically, characterizes efficiency of algorithms with two
embedded loops
n3 cubic Typically, characterizes efficiency of algorithms with
three embedded loops
2n exponential Typical for algorithms that generate all subsets of an
n-element set.
n! factorial Typical for algorithms that generate all permutations of
an n-element set
Mathematical Analysis of Nonrecursive Algorithms
General plan for Analyzing Time Efficiency of Nonrecursive

Algorithms:
 Decide on a parameter(s) n indicating input’s size.
 Identify algorithm’s basic operation.
 Check whether the number of times the basic operation is executed

depends only on the input size n. If it also depends on the type of input,
investigate worst, average, and best case efficiency separately.
 Set up summation for C(n) reflecting the number of times the

algorithm’s basic operation is executed.
 Simplify summation using standard formulas and establish the

count’s order of growth.

Example 1: To find the largest element in a list of n numbers.
ALGORITHM MaxElement(A[0…n-1])
//Determines the value of the largest element in a given array
//Input: An array A[0…n-1] of real numbers
//Output: The value of the largest element in A
maxval ← A[0]
for i ← 1 to n-1 do
if A[i] > maxval
maxval ← A[i]
return maxval
Analysis:
Parameter indicating input size: n

Basic operation: Comparison
n-1
C(n) = ∑1= n-1- 1+1 = n – 1 Є Θ (n)
i=1

Example 2: To Check whether all the elements in a given array

are distinct or not.
ALGORITHM UniqueElements(A[0…n-1])
//Determines whether all the elements in a given array are distinct
//Input: An array A[0…n-1]
//Output: Returns “true” if all the elements in A are distinct
// and “false” otherwise
for i ← 0 to n - 2 do
for j ← i+1 to n - 1 do
if A[i] = A[j] return false
return true
 The natural measure of the input’s size is the number of elements in the
array. i.e., n.
 Since the innermost loop contains a single operation (comparison), we
should consider it as the algorithm’s basic operation.
 Here, no. of element comparisons will depend not only on n but also on
whether there are equal elements in the array and, if there are, which
array positions they occupy.
 We shall do only worst case analysis.
 The worst case input is an array for which the number of elements
comparisons Cworst(n) is the largest among all arrays of size n.
 There are two kinds of worst case inputs:
 Arrays with no equal elements and arrays in which the last two
elements are the only pair of equal elements.

 For such inputs, one comparison is made for each repetition of the
innermost loop, i.e., for each value of the loop’s variable j between its
limits i+1 and n-1, and this is repeated for each value of the outer
loop, i.e., for each value of the loop’s variable i between its limits 0
and n-2.
Analysis:
Basic operation: Comparison
n-2 n-1 n-2 n-2
CWorst(n) = ∑ ∑ 1 = ∑ n-1- i-1+1 = ∑n-1-i
i=0 j=i+1 i=0 i=0
n-2 n-2
= (n-1)∑1 - ∑i = (n-1)2 - (n-2)(n-1) = (n-1)n ≈ 1n2 Є Θ(n2)
i=0 i=0 2 2 2
 In the worst case, the algorithm needs to compare all n(n-1)/2 distinct
pairs of its n elements
Example 3: Given two n-by-n matrices A and B, compute their

sum C=A+B.
ALGORITHM MatrixSum (A[0…n-1, 0…n-1], B[0…n-1, 0…n-1])

//Adds two n-by-n matrices
//Input: Two n-by-n matrices A and B
//Output: Matrix C = A + B
for i ← 0 to n - 1 do
for j ← 0 to n - 1 do
C[i,j] ← A[i,j] + B[i,j]
return C

Analysis:

Basic operation: Addition
n-1 n-1 n-1 n-1
C(n) = ∑ ∑1 = ∑ n-1- 0+1 = n ∑1
i=0 j=0 i=0 i=0
= n(n-1-0+1) = n2 Є Θ(n2)
Example 4: Consider the following algorithm.

ALGORITHM Enigma(A[0..n − 1, 0..n − 1])
//Input: A matrix A[0..n − 1, 0..n − 1] of real numbers
for i ←0 to n − 2 do
for j ←i + 1 to n − 1 do
if A[i, j ] ≠ A[j, i]
return false
return true
a) What does this algorithm compute?
b) What is its basic operation?
c) What is the efficiency class of this algorithm?
Soln:
a) This algorithm checks if the input square matrix is symmetric or not.
b) Basic Operation is comparison
c) C(n) is given as follows:
C(n) Є O(n2)
The efficiency class is quadratic.

Recurrence Relations
 An Equation such as
x(n) = x(n-1) + n for n >0 ----- (1)
x(0) = 0 ----- (2)
is called a recurrence equation or recurrence relation (or recurrence)
 Equation(1) defines x(n) not explicitly, i.e., as a function of n, but

implicitly as a function of its value at another point, namely n-1.
 An equation such as (2) is called its initial condition which tells us the
value with which the sequence starts.
 There are several techniques available for solving recurrence relations.

– Backward Substitution

Mathematical Analysis of Recursive Algorithms
General plan for Analyzing Time Efficiency of Recursive

Algorithms:
 Decide on a parameter(s) n indicating input’s size.
 Identify algorithm’s basic operation.
 Check whether the number of times the basic operation is executed can
vary on different inputs of same size; If it can, the worst-case, average-
case, and best-case efficiencies must be investigated separately.
 Set up a recurrence relation, with an appropriate initial condition, for

the number of times the basic operation is executed.
 Solve the recurrence or at least ascertain the order of growth of its

solution.
Example 1: To compute the factorial function F(n) = n!

for an arbitrary nonnegative integer n.
ALGORITHM F(n)
//Computes n! recursively
//Input: A nonnegative integer n
//Output: The value of n!
if n = 0 return 1
else return F(n-1)* n
 For simplicity, we consider n itself as an indicator of this algorithm’s

input size.
 The basic operation of the algorithm is multiplication, whose number

of executions we denote as M(n)

 Since the function F(n) is computed as F(n) = F(n-1).n for n>0, the
number of multiplications M(n) needed to computed it must satisfy the
equality
 Our goal now is to solve the recurrence relation M(n) = M(n-1) + 1, i.e., to
find an explicit formula for M(n) in terms of n only.
 To determine a solution uniquely, we need an initial condition that tells

us the value with which the sequence starts.
 We can obtain this value by inspecting the condition that makes the
algorithm stop its recursive calls:
if n=0 return 1
 When n = 0, the algorithm performs no multiplications. Thus, the initial

condition is
• Thus, we succeed in setting up the recurrence relation and initial

condition for the algorithm’s number of multiplications M(n):
We use Backward substitution method for solving recurrence relation.

Example 2: Tower of Hanoi problem
 In this puzzle, we have n disks of different sizes and three pegs.
 Initially, all the disks are on the first peg in the order of size, the largest
on the bottom and the smallest on top.
 The goal is to move all the disks to the third peg, using the second one as
an auxiliary, if necessary.
 We can move only one disk at a time, and it is forbidden to place a larger
disk on top of a smaller one.
 The problem has a recursive solution that is illustrated below:

 To move n>1 disks from peg1 to peg3, we first move recursively n-1 disks
from peg1 to peg2 (with peg3 as auxiliary), then move the largest disk
directly from peg1 to peg3, and finally, move recursively n-1 disks from
peg2 to peg3 (using peg1 as auxiliary).
 If n=1, we can simply move the single disk directly from the source peg to
the destination peg.
 Let us apply the general plan to this problem.
 The number of disks n is the choice for the input’s size indicator.
 Algorithm’s basic operation is moving one disk.
 Clearly, the number of moves M(n) depends on n only, and we get the
following recurrence equation for it:
M(n) = M(n-1) + 1 + M(n-1) for n>1

i.e., M(n) = 2M(n-1) + 1 for n>1,
M(1) = 1 ( Initial Condition)
We solve this recurrence by backward substitution method

 Thus, we have an exponential algorithm, which will run for an

unimaginably long time even for moderate values of n. This is the most
efficient algorithm possible. But, it is the problem’s intrinsic difficulty
that makes it so computationally hard.
Example: Find the number of binary digits in the binary

representation of a positive decimal integer using recursion.
 Let us set up a recurrence and an initial condition for the number of

additions A(n) made by the algorithm.
 The number of additions made in computing BinRec( n/2 ) is A( n/2 ) ,
plus one more addition is made by the algorithm to increase the returned
value by 1. This leads to the recurrence
A(n) = A( n/2 ) + 1 for n > 1
 Since the recursive calls end when n is equal to 1 and there are no
additions made then, the initial condition is
A(1) = 0
 The presence of n/2 in the function’s argument makes the method of
backward substitutions stumble on values of n that are not powers of 2.
Therefore, the standard approach to solving such a recurrence is to solve
it only for n = 2k and then take advantage of the theorem called the
smoothness rule which claims that under very broad assumptions the
order of growth observed for n = 2k gives a correct answer about the order
of growth for all values of n. (Alternatively, after getting a solution for

powers of 2, we can sometimes fine-tune this solution to get a formula
valid for an arbitrary n.) So let us apply this recipe to our recurrence,
which for n = 2k takes the form
Using Limits for Comparing Orders of Growth

************* END OF UNIT - 1 *************

ADA_Unit-1

Uploaded by

Copyright:

Available Formats

ADA_Unit-1

Uploaded by

Document Information

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

ADA_Unit-1

Uploaded by

Copyright:

Available Formats

Analysis and Design of Algorithms – Unit - 1

Input “Computer” Output

Figure: Notion of Algorithm.

 Definiteness: Each instruction in the algorithm must be clear and

 Finiteness: Algorithm must terminate after a finite number of steps.

 Effectiveness: Each step must be sufficiently simple and basic. It

 Input: Range of inputs for which an algorithm works has to be clearly

 Output: Algorithm must produce at least one output.

Smt. Kavitha M, Assistant professor, Dept. of CSE, SIT, Tumkur-3 Page 1

Algorithms with different ideas to compute the GCD of two

Euclid’s algorithm to compute gcd(m,n)

gcd(m,n) = gcd(n, m mod n)

gcd(60,24) = gcd(24,12) = gcd(12,0) = 12

Step 1 If n = 0, return m and stop; otherwise proceed to Step 2

// computes gcd(m, n) by Euclid’s algorithm

Consecutive Integer Checking Algorithm to compute gcd(m,n)

Step 2 Divide m by t. If the remainder of this division is 0, goto Step 3;

Smt. Kavitha M, Assistant professor, Dept. of CSE, SIT, Tumkur-3 Page 2

Step 3 Divide n by t. If the remainder of this division is 0, return the value of t

Step 4 Decrease the value of t by 1 and go to Step 2

Middle - school procedure to computer gcd(m,n)

Step 2 Find the prime factors of n.

Smt. Kavitha M, Assistant professor, Dept. of CSE, SIT, Tumkur-3 Page 3

Generating prime numbers using Sieve of Eratosthenes:

 The algorithm starts by initializing a list of prime candidates with

// Copy the remaining elements of A to array L of primes

Smt. Kavitha M, Assistant professor, Dept. of CSE, SIT, Tumkur-3 Page 4

2, 3, 5, 7, 11, 13, 17, 19, 23 are the generated prime numbers.

Fundamentals of Algorithmic Problem Solving

 We can consider algorithms to be procedural solutions to problem. These

Smt. Kavitha M, Assistant professor, Dept. of CSE, SIT, Tumkur-3 Page 5

 Understanding the problem:

 From a practical perspective, the first thing you need to do before

 Activities involved before going for designing the algorithm :

 Ascertaining the capabilities of a Computational Device:

 The central assumption of the Random Access Machine model is

Smt. Kavitha M, Assistant professor, Dept. of CSE, SIT, Tumkur-3 Page 6

 Make sure about the capabilities of a computational device before

 Choosing between Exact and Approximate problem solving:

 Make decision to choose between solving the problem exactly

 Deciding on Appropriate Data Structure:

 Once we have selected the type of algorithm (Sequential/Parallel,

Algorithms + Data Structures = Programs

Smt. Kavitha M, Assistant professor, Dept. of CSE, SIT, Tumkur-3 Page 7

 Few algorithm design techniques depend intimately on

 Algorithm Design Techniques:

 An algorithmic design technique is a general approach to solving

 Methods of Specifying an Algorithm:

 In the earliest days of computing, flowchart was used for

Smt. Kavitha M, Assistant professor, Dept. of CSE, SIT, Tumkur-3 Page 8

o Using a Natural Language: Each user may interpret the

 Proving an Algorithm’s Correctness:

 Once an algorithm has been specified, you have to prove its

Smt. Kavitha M, Assistant professor, Dept. of CSE, SIT, Tumkur-3 Page 9

 There are two kinds of algorithm efficiency:

 Other desirable characteristics of an algorithm are

 If you are not satisfied with the algorithm’s efficiency, simplicity, or

 Algorithm is coded using suitable data structure in programming

Smt. Kavitha M, Assistant professor, Dept. of CSE, SIT, Tumkur-3 Page 10

Fundamentals of the Analysis of Algorithm Efficiency