UNIT - I

INTRODUCTION TO ALGORITHM
Informal Definition:

An Algorithm is any well-defined computational procedure that

takes some value or set of values as Input and produces a set of
values or some value as output. Thus algorithm is a sequence of
computational steps that transforms the input into the output.

Formal Definition:

An Algorithm is a finite set of instructions that, if followed,

accomplishes a particular task. In addition, all algorithms should
satisfy the following criteria.

 INPUT  Zero or more quantities are externally supplied.

 OUTPUT  At least one quantity is produced.
 DEFINITENESS  Each instruction is clear and unambiguous.
 FINITENESS  If we trace out the instructions of an
algorithm, then for all cases, the algorithm terminates
after a finite number of steps.
 EFFECTIVENESS  Every instruction must very basic
so that it can be carried out, in principle, by a person
using only pencil & paper.

Issues or study of Algorithm:

1. How to device or design an algorithm  creating and algorithm.
2. How to express an algorithm  definiteness.
3. How to analysis an algorithm  time and space complexity.
4. How to validate an algorithm  fitness.
5. Testing the algorithm  checking for error.

Algorithm Specification:
Algorithm can be described in three ways.
1. Natural language like English:
1
 When this way is choused care should be taken, we should
ensure that each & every statement is definite.
2. Graphic representation called flowchart:
This method will work well when the algorithm is small& simple.
3. Pseudo-code Method:
In this method, we should typically describe algorithms as
program, which resembles language like Pascal & algol.

PSEUDO-CODE FOR EXPRESSING AN ALGORITHM:

1. Comments begin with // and continue until the end of line.

2. Blocks are indicated with matching braces {and}.
3. An identifier begins with a letter. The data types of
variables are not explicitly declared.
4. Compound data types can be formed with
records. Here is an example, Node.
Record
{
data type – 1 data-1;
.
.
.
data type – n
data – n; node
* link;
}
Here link is a pointer to the record type node. Individual data
items of a record can be accessed with  and period.

5. Assignment of values to variables is done using the assignment statement.

<Variable>:= <expression>;

6. There are two Boolean values TRUE and FALSE.

 Logical Operators AND, OR, NOT
Relational Operators <, <=,>,>=, =, !=

7. The following looping

statements are
employed. For,
while and repeat-
until
2
While Loop:
While < condition > do
 {
<statement-1>
.
.
.
<statement-n> }

For Loop:
For variable: = value-1 to value-2 step step do
{
<statement-1>
.
.
.
<statement-n>
}
repeat-until:
repeat
<statement-1>
.
.
.
<statement-n>
until<condition>

8. A conditional statement has the following forms.

 If <condition> then <statement>
 If
<condition>
then
<statement-
1> Else
<statement-
1>
Case statement:
Case
{
: <condition-1> : <statement-1>
.
. 3
.
: <condition-n> : <statement-n>
 : else : <statement-n+1>
}
9. Input and output are done using the instructions read & write.
10. There is only one type of procedure: Algorithm,
the heading takes the form, Algorithm
Name (Parameter lists)

 As an example, the following algorithm fields & returns

the maximum of ‘n’ given numbers:

1. algorithm Max(A,n)
2. // A is an array of size n

3. {
4. Result := A[1];
5. for I:= 2 to n do
6. if A[I] > Result then
7. Result :=A[I];
8. return Result;
9. }
In this algorithm (named Max),

A & n are procedure parameters.

Result & I are Local variables.

PERFORMANCE ANALYSIS:

1. Space Complexity:
The space complexity of an algorithm is the amount of
money it needs to run to compilation.

2. Time Complexity:
The time complexity of an algorithm is the amount of computer time
it needs to run to compilation.

Space Complexity:

Space Complexity Example: 4

 Algorithm abc(a,b,c)
{
return a+b++*c+(a+b-c)/(a+b) +4.0;
}

 The Space needed by each of these algorithms is seen to be the

sum of the following component.

1.A fixed part that is independent of the characteristics

(eg:number,size)of the inputs and outputs.

The part typically includes the instruction space (ie. Space for the
code), space for simple variable and fixed-size component variables
(also called aggregate) space for constants, and so on.

variable part that consists of the space needed by component

variables whose size is dependent on the particular problem
instance being solved, the space needed by referenced variables
(to the extent that is depends on instance characteristics), and
the recursion stack space.
a. The space requirement s(p) of any algorithm p may

therefore be written as, S(P) = c+

Sp(Instance characteristics) Where ‘c’ is a

constant.

Example : Algorithm sum(a,n)

{
s=0.0;
f
o
r

I
=
1

t
o
5
n
 d
o

s
=

s
+
a
[
I
]
;
return s;
}

 The problem instances for this algorithm are characterized by

n,the number of elements to be summed. The space needed d by
‘n’ is one word, since it is of type integer.
 The space needed by ‘a’a is the space needed by variables of tyepe
array of floating point numbers.
 This is atleast ‘n’ words, since ‘a’ must be large enough to hold
the ‘n’ elements to be summed.
 So,we obtain Ssum(n)>=(n+s) [ n for a[ ],one each for n,I a& s]

Time Complexity:

The time T(p) taken by a program P is the sum of the compile

time and the run time(execution time)

The compile time does not depend on the instance characteristics.

Also we may assume that a compiled program will be run several times
without recompilation .This rum time is denoted by tp(instance
characteristics).

 The number of steps any problem statemn t is assigned depends on

the kind of statement. For example, comments  0 steps.

Assignment statements  1 steps. [Which does not

involve any calls to other algorithms]

Interactive statement such as for, while

6 & repeat-until Control part of the statement.
1. We introduce a variable, count into the program statement to
increment count with initial value 0.Statement to increment
count by the appropriate amount are introduced into the
program.

This is done so that each time a statement in the original

program is executes count is incremented by the step count of that
statement.

Algorithm:

Algorithm sum(a,n)
{
s= 0.0;
c
o
u
n
t

c
o
u
n
t
+
1
;

f
o
r

I
=
1

t
o

d 7
o
{
c
o
u
n
t

=
c
o
u
n
t
+
1
;

s
=
s
+
a
[
I
]
;

c
o
u
n
t
=
c
o
u
n
t
+
1
;
} 8
c
 o
u
n
t
=
c
o
u
n
t
+
1
;

c
o
u
n
t
=
c
o
u
n
t
+
1
;

r
e
t
u
r
n

s
;
}

 If the count is zero to start with, then it will be 2n+3 on

termination. So each invocation of sum execute a total of 2n+3
steps.

2. The second method to determine

9
the step count of an
algorithm is to build a table in which we list the total number
 of steps contributes by each statement.
First determine the number of steps per execution (s/e) of the
statement and the total number of times (ie., frequency) each
statement is executed.
By combining these two quantities, the total contribution of
all statements, the step count for the entire algorithm is
obtained.
Statement S/e Frequency Total

1. Algorithm Sum(a,n) 0 - 0
2.{ 0 - 0

3. S=0.0; 1 1 1

4. for I=1 to n do 1 n+1 n+1

5. s=s+a[I]; 1 n n

6. return s; 1 1 1

7. } 0 - 0

Total 2n+3

ASYMPTOTIC NOTATIONS

There are different kinds of mathematical notations used to represent time

complexity.
These are called Asymptotic notations. They are as follows:
1. Big oh(O) notation

2. Omega(Ω) notation

3. Theta(ɵ) notation

1. Big oh(O) notation:

 Big oh(O) notation is used to represent upperbound of algorithm runtime.
 Let f(n) and g(n) are two non-negative functions
 The function f(n) = O(g(n)) if and only if there exists
positive constants c and n0 such that f(n)≤c*g(n) for
all n , n ≥ n0. 1
0
Example:
If f(n)=3n+2 then prove that f(n) = O(n)
Let f(n)
=3n+2
, c=4,
g(n)
=n if
n=1
3(1)+2 ≤ 4(1)
3+2 ≤ 4
5 ≤ 4 (F)
if n=2 3n+2≤4n
3(2)+2 ≤ 4(2)
8 ≤ 8 (T)
3n+2 ≤ 4n for all n ≥ 2
This is in the form of f(n) ≤ c*g(n) for all n

≥ n0, where c=4, n0 =2 Therefore, f(n) =

O(n),
2. Omega(Ω) notation:

 Big oh(O) notation is used to represent lowerbound of algorithm runtime.

1
1
 Let f(n) and g(n) are two non-negative functions
  The function f(n) = Ω(g(n)) if and only if there exists positive constants
c and n0
such that f(n) ≥ c*g(n) for all n , n ≥ n0.

Example
f(n)=3n+2 then prove that f(n) = Ω(g(n))
Let
f
(
n
)
=
3
n
+
2
,
c
=
3
,
g
(
1
n 2
 )
=
n

i
f
n
=
1

3(1)+2 ≥ 3(1)
5 ≥ 3 (T)
3n+2 ≥ 4n for all n ≥ 1
This is in the form of f(n) ≥ c*g(n) for all n

≥ n0, where c=3, n0 =1 Therefore, f(n)

= Ω(n).
3. Theta(ɵ) notation:

 Theta(ɵ) notation is used to represent the running time

between upper bound and lower bound.
 Let f(n) and g(n) be two non-negative functions.

The function f(n) = θ(g(n)) if and only if there exists
positive constants c1 , c2 and n0 such that
c1*g(n) ≤ f(n)≤c2* g(n) for all n, n≥n0 .

1
Example: 3
 f(n)=3n+2 then Prove that f(n) = θ(g(n))
Lower bound = 3n+2 ≥ 3n for all n ≥ 1
c1=3,g(n)=n,n0=1 Upper Bound = 3n+2 ≤ 4n
for all n ≥ 2
c2=4, g(n)=n , n0=2 3(n) ≤ 3n+2 ≤ 4(n) for all n, n ≥ 2
This is in the form of c1*g(n) ≤ f(n) ≤ c2* g(n) for all n≥n0
Where c1=3, c2=4, g(n)=n, n0=2

Therefore f(n)= θ(n)

Amortized Analysis is used for algorithms where an occasional operation is very

slow, but most of the other operations are faster. In Amortized Analysis, we analyze a
sequence of operations and guarantee a worst-case average time that is lower than the
worst-case time of a particularly expensive operation.
The example data structures whose operations are analyzed using Amortized Analysis are
Hash Tables, Disjoint Sets, and Splay Trees.
Amortized analysis is a technique used in computer science to analyze the average-case
time complexity of algorithms that perform a sequence of operations, where some
operations may be more expensive than others. The idea is to spread the cost of these
expensive operations over multiple operations, so that the average cost of each operation is
constant or less.
For example, consider the dynamic array data structure that can grow or shrink dynamically
as elements are added or removed. The cost of growing the array is proportional to the size
of the array, which can be expensive. However, if we amortize the cost of growing the array
over several insertions, the average cost of each insertion becomes constant or less.
Amortized analysis provides a useful way to analyze algorithms that perform a sequence of
operations where some operations are more expensive than others, as it provides a
guaranteed upper bound on the average time complexity of each operation, rather than the
worst-case time complexity.
Amortized analysis is a method used in computer science to analyze the average
performance of an algorithm over multiple operations. Instead of analyzing the worst-
case time complexity of an algorithm, which gives an upper bound on the running time of a
single operation, amortized analysis provides an average-case analysis of the algorithm by
considering the cost of several operations performed over time.
The key idea behind amortized analysis is to spread the cost of an expensive operation
over several operations. For example, consider a dynamic array data structure that is resized
when it runs out of space. The cost of resizing the array is expensive, but it can be
amortized over several insertions into the array, so that the average time complexity of an
insertion operation is constant.
Amortized analysis is useful for designing efficient algorithms for data structures such as
dynamic arrays, priority queues, and disjoint-set data structures. It provides a guarantee that
the average-case time complexity of an operation is constant, even if some operations may
1
be expensive. 4
Let us consider an example of simple hash table insertions. How do we decide on table
size? There is a trade-off between space and time, if we make hash-table size big, search
time becomes low, but the space required becomes high.

The solution to this trade-off problem is to use Dynamic Table (or Arrays) . The idea is to
increase the size of the table whenever it becomes full. Following are the steps to follow
when the table becomes full.
1) Allocate memory for larger table size, typically twice the old table.
2) Copy the contents of the old table to a new table.
3) Free the old table.

Advantages of amortized analysis:

1. More accurate predictions: Amortized analysis provides a more accurate

prediction of the average-case complexity of an algorithm over a sequence of
operations, rather than just the worst-case complexity of individual operations.
2. Provides insight into algorithm behavior: By analyzing the amortized cost of an
algorithm, we can gain insight into how it behaves over a longer period of time
and how it handles different types of inputs.
3. Helps in algorithm design: Amortized analysis can be used as a tool for
designing algorithms that are efficient over a sequence of operations.
Useful in dynamic data structures: Amortized analysis is particularly useful in
dynamic data structures like heaps, stacks, and queues, where the cost of an
operation may depend on the current state of the data structure.

Disadvantages of amortized analysis:

1
1. Complexity: Amortized analysis5
can be complex, especially when multiple
operations are involved, making it difficult to implement and understand.
2. Limited applicability: Amortized analysis may not be suitable for all types of
algorithms, especially those with highly unpredictable behavior or those that
depend on external factors like network latency or I/O operations.
3. Lack of precision: Although amortized analysis provides a more accurate
prediction of average-case complexity than worst-case analysis, it may not
always provide a precise estimate of the actual performance of an algorithm,
especially in cases where there is high variance in the cost of operations.
Thus, the total amortized cost of insertion in the Red-Black Tree is O(n).

1
6