Analysis of Algorithm Introduction

Introduction
 what is a computer program?

a set of instructions written in a specific programming language

that is complete enough for a computer to execute it automatically

 what is an algorithm?
fundamental structure of a computer program
set of instructions written in an abstract form, pseudo-code or flowchart,
independently from any specific computer or programming language
A Valid Algorithm is:
A sequence of unambiguous instructions for solving a problem,
i.e. for obtaining the required output for any legitimate input in
a finite amount of time.

Important Points are:

Non-ambiguity
Range of inputs
The same algorithm can be represented in different ways
Several algorithms for solving the same problem
 Input(s) Algorithm Output(s)

● Example: computing greatest commom divisor, gcd(m,n)

input: Two integer numbers.

output: The greatest integer that divides both m and n evenly.

issues: correctness, efficiency, storage, etc.

Example: The problem of sorting
Issues and Approaches for Algorithm Analysis
● Issues
 Correctness

 Time efficiency

 Space efficiency

 Optimality

● Approaches
 Theoretical analysis

 Empirical analysis
 what are the costs of running a computer program on a computer?
 Computation time and Storage space required.

 cost analysis: we must have an idea of the running time and memory size
required to solve a problem of a certain size

An intractable problem: possible in theory but impossible in practice.

Usually some problem types become intractable when problem size increases.

 we should analyze the algorithm to know in advance the computational costs of

running it
Algorithm Analysis
● Determining performance characteristics.
(Predicting the resource requirements.)
○ Usually running time and memory requirements

○ Computation time (running time) is of primary concern.

● Why analyze algorithms?

○ Choose the most efficient of several possible algorithms for the same problem.

○ Is the best possible running time for a problem reasonably finite for practical purposes?

○ Is the algorithm optimal (best in some sense)? – Is something better possible?

Definition
● Call each simple instruction and access to a word of memory a “primitive operation”
or “step.”

● Running Time is the real clock time the machine takes to execute those operations in
the algorithm, this depends on many factors like machine's configuration, current
load in that machine and many more and thus it is not fixed for a particular
algorithm
● Time complexity is the measurement of an algorithm's time behavior as input size
increases.
● Time complexity can also be calculated from the logic behind the algorithm/code.
Often referred to as the complexity of the algorithm.
● Units of measuring running time: Identify and count the basic operation(s) in
the algorithm.
Complexity and Input
● Complexity of an algorithm generally depends on input size.

Input size is influenced by

○ the data representation, e.g. matrix

○ the operations of the algorithm, e.g. spell-checker

○ the properties of the objects in the problem, e.g. checking if a given integer is a prime number.
Examples
Problem Size of input
Find x in an array The number of the
elements in the array
Multiply two matrices The dimensions of the
matrices
Sort an array The number of elements
in the array
Traverse a binary tree The number of nodes
Solve a system of linear The number of equations,
equations or the number of the
unknowns, or both
Examples of Basic Operations
Problem Basic Operation
Find x in an array Comparison of x with
an entry in the array
Multiplying two Multiplication of two
matrices with real real numbers
entries
Sort an array of Comparison of two
numbers array entries plus
moving elements in
the array
Traverse a tree Traverse an edge
Work done by an algorithm
Let
T(n): running time

cop : execution time for basic operation

C(n) : number of times basic operation is

executed

● Then we have: T(n) ≈ cop C(n)

Worst, Average, and Best-case Complexity
 Worst-case Complexity
 Maximum number steps the algorithm takes for any
possible input.
 Most tractable measure.
 Average-case Complexity
 Average of the running times of all possible inputs.
 Demands a definition of probability of each input, which is
usually difficult to provide and to analyze.
 Best-case Complexity
 Minimum number of steps for any possible input.
 Not always a useful measure. Why?
Pseudo-code representation of an algorithm
to analyze an algorithm easily, we need to represent it in a way that is:

 clear and straightforward (the brain must easily comprehend its structure)
 without any detail of implementation (unnecessary for analysis)
 independent from any programming language

flowchart representation of an algorithm is mostly outdated and not suitable

for analysis
● Pseudo-code conventions:

 based on the principles of structured programming

(sequential structure, conditional structure, iterative structure)

 based on data abstraction:

 no variable declarations, no types
 we do not specify any memory allocation
 no pointers, no pointer arithmetic
● no input - output operations
assignment: variable  value
● if-then-else construct

● while construct

● block of code represented with indentation and vertically stretched square bracket

1D array: A[1..n] with 1..n range of the index  element: A[i]

2D array: A[1..n , 1..m] with 1..n and 1..m ranges of indices  element: A[i , j]

function call: similar to C  arguments are passed by value

except for arrays that are passed by reference
example:

C code:

#include <stdio.h>
#define N 100

void main()
{ pseudo-code:
int i , n = 0;
double v , x[N] , max; max  - ∞
for i  1 to n
while (scanf("%lf" , &v) != EOF) if max < x[i]
{ n++; if (n > N) exit(0); max  x[i]
x[n - 1] = v; }

max = -1e20;
for (i = 0 ; i < n ; i++)
if (max < x[i])
max = x[i];

printf("\n max = %f" , max);

}
Analyzing algorithms
 we must analyze an algorithm in order to predict the costs of executing it on a
computer:

 running time necessary to execute the code  T(n), n: input size

 size of memory necessary for the data structures of the code

why we should know the costs of an algorithm
before coding it:
 better to know in advance if the algorithm can be executed with affordable costs:

 running time of at most a few months...

 memory size inferior to the RAM (or virtual memory) of our computer.

 find the bottlenecks, i.e. the parts of the program that will take the most time

 compare the efficiencies of two (or more) algorithms

● We assume a generic model of computer:

● only one CPU

● instructions are executed one after the other

 no parallelism, no concurrency
Types of formulas for basic operation count

● Exact formula
e.g., T(n) = n(n-1)/2

● Formula indicating order of growth with specific

multiplicative constant
e.g., T(n) ≈ 0.5 n2

● Formula indicating order of growth with unknown

multiplicative constant
e.g., T(n) ≈ c.n2
Example

Let T(n) = 3n(n-1)  3n2

Suppose we double the input size.

How much longer the program will run?

Calculating the running time and memory size of an iterative algorithm

running time:

● an iterative algorithm is made of loops (eventually nested) and of tests over the data

● we must count the total number of iterations and express it as an algebraic expression

loops:
● header line of loop executed one more time than the number of iterations:

○ assignment and test on index executed one more time  exit from the loop
Line
i i<4 printing
1. for i 0 to 3
2. printf("\n i = %d" , i); 0 true i=0
3. printf("\n i = %d" , i); 1 true i=1
2 true i=2
3 true i=3
4 false
Example:

Algorithm LinearSearch(A, key)

1 i1
2 while i ≤ n and A[i] != key
3 do i++
4 if i  n
5 then return true
6 else return false
INPUT: a sequence of n numbers, key to search for.
OUTPUT: true if key occurs in the sequence, false otherwise.

LinearSearch(A, key) cost times

1 i1 c1 1
2 while i ≤ n and A[i] != key c2 x
3 do i++ c3 x-1
4 if i  n c4 1
5 then return true c5 1
6 else return false c6 1
x ranges between 1 and n+1.
So, the running time ranges between
c1+ c2+ c4 + c5 – best case
and
c1+ c2(n+1)+ c3n + c4 + c6 – worst case