Analysis of Algorithms

Lecture - 1

SE AA-262
 Introduction

 What is Algorithm?
 a clearly specified set of simple instructions to be followed to
solve a problem
 Takes a set of values, as input and
 produces a value, or set of values, as output
 May be specified
 In English , As a computer program , As a pseudo-code
 Data structures is methods of organizing data in memory
 Program = algorithms + data structures
Introduction

 Why need algorithm analysis ?

 writing a working program is not good enough

 The program may be inefficient!

 If the program is run on a large data set, then the running

time becomes an issue
Analyzing and Algo

 To introduce the concept of analyzing algorithms with

respect to the time and space taken to have them
executed
 Purpose: To see if an algorithm is practical
 To compare different algorithms for solving a
problem
 One aspect of software quality is the efficient use of
computer resources :
 CPU time
 Memory usage
Properties of an algorithm

 Finiteness
The process should be terminated after a finite number of steps.
 Effectiveness
Every instruction must be basic enough to be carried out theoretically or by
using paper and pencil.
 Definiteness
Each instruction of the algorithm should be clear and unambiguous.
 Input
An algorithm should have some specified set of inputs n
 Output
At least one output should be returned by the algorithm after the
completion of the specific task based on the given inputs.
 Correctness

An algorithm is said to be correct if :

 For every input, it halts with correct output.

 An incorrect algorithm might not halt at all

OR
It might halt with an answer other than desired
one.


Complexity analysis of an algo

 An essential aspect to data structures is algorithms.

 Data structures are implemented using algorithms.
 An algorithm is a procedure that you can write as a C function or program, or any other
language. An algorithm states explicitly how the data will be manipulated.
 We perform Algorithm Efficiency analysis
Algorithm Efficiency

 Some algorithms are more efficient than others. We would prefer to chose an
efficient algorithm, so it would be nice to have metrics for comparing
algorithm efficiency.
 The complexity of an algorithm is a function describing the efficiency of the
algorithm in terms of the amount of data the algorithm must process.
 There are two main complexity measures of the efficiency of an algorithm:
 Time complexity
 Space complexity
 Time complexity is a function describing the amount of time an algorithm takes in
terms of the amount of input to the algorithm.
For example, to decide which sorting algorithm will take less time to run

 Space complexity is a function describing the amount of memory (space) an

algorithm takes in terms of the amount of input to the algorithm.

Space Complexity of an algorithm is the total space taken by the

algorithm with respect to the input size. Space complexity
includes both Auxiliary space and space used by input.
 Auxiliary space:
Auxiliary Space is the extra space or temporary space used by an
algorithm.
Primitive operations

 Primitive operations are basic computations performed by an algorithm.

 Examples are
 evaluating an expression (expensive)
 assigning a value to a variable(in expensive)
 calling a method (in expensive)
 returning from a method (in expensive) etc.
 Comparing a value( inexpensive)
 Comparing data items(expensive)

 They are easily identifiable in pseudocode and largely independent from the programming language
 In the analytical approach running time is measured by using primitive
operations.
 They contribute much to overall time of algorithm
Example

2 for the write and assign, 4 reads, 3 operators "

 Example

Algorithm arrayMax(A, n):

Input: An array A of n integers Number of operations:
Output: the max element
1. max = A[0] 2
2. 2. for i = 0 to n-1 do 2 n(1 comparision,1 increment , runs n no of
time
3. if (A[i] > max) then max = A[i] 6(n-1)(6 operations , run n-1 times)
4. return max 1
Total: 8n-3 : how??
• 2+2n+6(n-1)+1
• 2+2n+6n-6+1
• 8n-4+1
• 8n-3
The algorithm arrayMax executes about 8n - 3 primitive operations.
 Complexity theory

 Complexity theory - a study of algorithm performance

Time efficiency - a measure of amount of time for an algorithm to execute.
Space efficiency - a measure of the amount of memory needed for an
algorithm to execute.
A good algorithm is correct, but a great algorithm is both correct
and efficient. The most efficient algorithm is one that takes the
least amount of execution time and memory usage possible while
still yielding a correct answer.
 Time complexity

 The thing is that while one algorithm takes seconds to

finish, another will take minutes with even small data
sets. How can we compare different performances and
pick the best algorithm to solve a particular problem?
 we shouldn’t really care about the exact number of operations that are
performed; instead, we should care about how the number of operations
relates to the problem size.
T(P) = t(c)+ tp(instance)
T(p) = time complexity for a problem algo
t(c) = compile time // fix and negligible
 The idea behind time complexity is that it can measure only
the execution time of the algorithm in a way that depends only
on the algorithm itself and its input.

 To express the time complexity of an algorithm, we use

something called the “Big O notation”.
 The Big O notation is a language we use to describe the
time complexity of an algorithm. It’s how we compare the
efficiency of different approaches to a problem, and helps us to
make decisions
 Big O notation expresses the run time of an
algorithm in terms of how quickly it grows relative
to the input (this input is called “n”).
 For example,
 Run time of an algorithm grows “on the order of the size of the input n ”, as
“O(n)”.
 Run time of an algorithm grows “on the order of the square of the size of the input”,
it will be as “O(n²)”.
 Time complexity is understanding the rates at which things can grow. The rate here is
time taken per input size
 Constant Time Complexity

Constant Time Complexity: O(1)

 When time complexity is constant (notated as “O(1)”),

the size of the input (n) doesn’t matter. Algorithms with
Constant Time Complexity take a constant amount of
time to run, independently of the size of n.
 run-time response is not change wrt to the input data,
which makes them the fastest algorithms out there.

Constant Time Complexity

To remain constant, these algorithms shouldn’t

contain loops, recursions or calls to any other
functions
Linear Time
Complexity: O(n)

 time complexity grows in direct

proportion to the size of the input, you
are facing Linear Time Complexity, or
O(n).
 Algorithms with this time complexity
will process the input (n) in “n”
number of operations. This means that
as the input grows, the algorithm takes
proportionally longer to complete
 algorithm visits every element
from the input. More the input Linear Time Complexity
more the time
Logarithmic Time
Complexity: O(log n)

 An algorithm is said to
run in logarithmic
time if its time
execution is
proportional to the
logarithm of the input
size.
 instead of increasing
the time it takes to
perform each
subsequent step, the
time is decreased at a
magnitude that is
inversely proportional
to the input “n”.
 Example !!

 Consider this example: let’s say that you want to look for a word in a dictionary
that has every word sorted alphabetically. There are at least two algorithms to
do that:
 Algorithm A:
• Starts at the beginning of the book and goes in order until it finds the contact you are
looking for.
 Algorithm B:
• Opens the book in the middle and checks the first word on it.
• If the word that you are looking for is alphabetically bigger, then it looks in the right half.
Otherwise, it looks in the left half.
Q: Which one of both is faster? While algorithm A goes word by word O(n),
algorithm B splits the problem in half on each iteration O(log n), achieving the
same result in a much more efficient way.
Quadratic Time
Complexity: O(n²)

 In this type of algorithms, the time it takes to

run grows directly proportional to the square of
the size of the input
 In most scenarios and particularly for large data
sets, algorithms with quadratic time
complexities take a lot of time to execute and
should be avoided
 Nested For Loops run on quadratic time,
because you’re running a linear operation
within another linear operation, or n*n which
equals n².
Exponential Time
Complexity: O(2^n)

 In exponential time algorithms, the growth rate

doubles with each addition to the input (n)
 Algorithms with this time complexity are usually
used in situations where you don’t know that
much about the best solution, and you have to
try every possible combination or permutation
on the data.

Exponential Time Complexity

 Worst case analysis
 Provides an upper bound on running time
 An absolute guarantee
 Average case analysis
 Provides the expected running time
 Very useful, but treat with care: what is “average”?
Random (equally likely) inputs
Real-life inputs
Example 1

Algorithm sum(a[ ] , n) 0
{ 0
S= 0; 1
for i=0 to n do n+1
0
s=s + a[ i ]; n

Return s ; 1
} 0

Time complexity : 2n+3

Example 3

Algo sum(a [][] ,m,n) 0

{ 0
S=0; 1 Time complexity : 2n+2nm+3

for i=1 to n do n+1

for j=1 to m do n(m+1)
S=s+a[ i][j]; nm
Return s; 1
} 0
Example 2

for(int i=0;i<=N;i++)
Let's calculate the time complexity for this particular loop-
assumption-> 1 unit for each comparison, initialization
and arithmetic operation
-- for initialization (i=0) 1
-- for comparison (i<=N) N+1
-- for incremental operator (i++) N

So total instructions which will be executed by machine = 2*N +

2.
Space complexity:

 Space Complexity of an algorithm denotes the total space used or

needed by the algorithm for its working, for various input sizes.
 It also includes the auxiliary space used
 A good algorithm keeps space complexity as low as possible. An
algorithm with lower space complexity is always better than the one
with higher
 Examples

 Algo for adding 2 numbers

Function add(n1,n2)
{
sum= n1+n2;
Return sum;
}
Space complexity:
n1 =4 bytes (assuming int)
n2 = 4 bytes
Sum =4 bytes
Aux space = 4 bytes
total space = 16 bytes = C = O(1)
 Function Sumofnum(arr[],n)
{
1. Sum = 0; sum =4 bytes
2. For(i=0 to n) i= 4 bytes
3. { arr =n* 4 bytes(i.e if arraysize= 5 5*4=20bytes)
sum = sum + arr[i]; Aux space = 4bytes
} Total space = 4N+12bytes
Print (sum); 12 bytes const
} 4n+C
O(N)

}
Memory Usage

 Instruction Space — amount of memory used to save the compiled

version of instructions

 Environmental Stack — when an algorithm is called inside of

another algorithm

 Data Usage — Amount of space used by the variables and