Design and analysis of algorithms

(DAA)
R Raja Sekhar
Associate professor ,Dept. of AIML
ADITYA COLLEGE OF ENGINEERING,SURAMPALEM
MOB:9866635319,9381059339
Email:rajasekhar121reddy@gmail.com
rajasekhar_csm@acoe.edu.in
COURSE OBJECTIVES

● 1.Ability to understand and analyze and denote time complexity of

algorithms .
● 2.To introduce the different algorithmic approaches for problem solving
through numerous example problems
● 3.Describe the dynamic programming paradigm and explain when an
algorithmic situation calls for it.recite algorithms that employ this paradigm
synthesize dynamic programming algorithms.
● 4.To provide some theoretical grounding in terms of finding to find the lower
bounds of algorithms and the NP completeness
COURSE OUTCOMES

● Upon the completion of this course students can able to do the following:
● 1.Analyze the performance of a given algorithm, denote its time complexity using
asymptotic notation for recursive and non recursive algorithms.
● 2.List and describe various algorithm approaches and solve problem using divide &
conqer,greedy methods
● 3.Synthesize efficient algorithms dynamic programming approaches to solve common
engineering design issues .
● 4.Organize important algorithmic design paradigms and methods of analysis
;backtracking ,branch&bound Algorithmic approaches
● 5.Demenostarate NP-completeness theory, lower bound theory and string matching.
TEXT BOOKS &REFERENCES

TEXTBOOKS
● 1.Ellis Horwitz ,satraz sahani,sanghudevar rajasekaran ,”fundamentals of computer
algorithms “,2nd edition ,universities press.
● 2.Introduction to algorithms Thomas H.coremen PHI learning.
● 3.Harsh basin-” Algorithm Design & Analysis", Oxford university press.
REFERENCES:
● 1.Horowitz E. Sahani S:”Fundamentals of computer algorithms ,2nd Edition Galgotia
Publications
● 2.S .Sridhar ,”Design and analysis of algorithms “Oxford University Press.
Design and Analysis of algorithms:Syllabus
● UNIT1: Introduction :Algorithm defination,algorithm specification,
performance analysis ,performance measurements ,asymptotic
notation ,randomized algorithms .
Unit.1: Introduction

History of Algorithm

• The word algorithm comes from the name of a Persian author, Abu Ja’far Mohammed ibn
Musa al Khowarizmi (c. 825 A.D.), who wrote a textbook on mathematics.
• He is credited with providing the step-by-step rules for adding, subtracting, multiplying,
and dividing ordinary decimal numbers.
• When written in Latin, the name became Algorismus, from which algorithm is but a small
step
• This word has taken on a special significance in computer science, where “algorithm” has
come to refer to a method that can be used by a computer for the solution of a problem
• Between 400 and 300 B.C., the great Greek mathematician Euclid invented an algorithm
• Finding the greatest common divisor (gcd) of two positive integers.
• The gcd of X and Y is the largest integer that exactly divides both X and Y .
• Eg.,the gcd of 80 and 32 is 16.
Unit.1: Introduction
What is an Algorithm?

Algorithm is a set of steps to complete a task.

For example,
Task: to make a cup of tea.
● Algorithm:
● add water and milk to the kettle,
● boil it, add tea leaves,
● Add sugar, and then serve it in cup.
● ‘’a set of steps to accomplish or complete a task that is described precisely
enough that a computer can run it’’.
● Described precisely: very difficult for a machine to know how much water, milk
to be added etc. in the above tea making algorithm.
● These algorithms run on computers or computational devices..For example,
GPS in our smartphones, Google hangouts.
● GPS uses shortest path algorithm.. Online shopping uses cryptography which
uses RSA algorithm.
What is Algorithm?
● A finite set of instruction that specifies a sequence of operation is to
be carried out in order to solve a specific problem or class of
problems is called an Algorithm
Why study Algorithm?
● As the speed of processor increases, performance is frequently said to
be less central than other software quality characteristics (e.g.
security, extensibility, reusability etc.). However, large problem sizes
are commonplace in the area of computational science, which
makes performance a very important factor.
● This is because longer computation time, to name a few mean
slower results, less through research and higher cost of computation
(if buying CPU Hours from an external party). The study of Algorithm,
therefore, gives us a language to express performance as a function
of problem size.
Algorithm Definition1,Criteria

● An algorithm is a finite set of instructions that, if followed, accomplishes a

particular task. In addition, all algorithms must satisfy the following criteria:

○ Input. Zero or more quantities are externally supplied.

○ Output. At least one quantity is produced.
○ Definiteness. Each instruction is clear and unambiguous.
○ Finiteness. The algorithm terminates after a finite number of steps.
○ Effectiveness. Every instruction must be very basic enough and must be
feasible.
Characteristics of an algorithm
An algorithm is a finite set of instructions that, if followed, accomplishes a
particular task.

● In addition, all algorithms must satisfy the following criteria :

● Input. Zero or more quantities are externally supplied.
● Output. At least one quantity is produced.
● Definiteness. Each instruction is clear and unambiguous
● Finiteness. If we trace out the instructions of an algorithm, then for all cases,
the algorithm terminates
● after a finite number of steps.
● Effectiveness. Every instruction must be very basic so that it can be carried out,
in principle, by a
● person using only pencil and paper. It is not enough that each operation be
definite as in criterion 3; it also must be feasible.
Algorithm Definition2:

An algorithm is a sequence of unambiguous instructions for solving a problem, i.e.,

for obtaining a required output for any legitimate input in a finite amount of time.
• Algorithms that are definite and effective are also called computational
procedures.
• A program is the expression of an algorithm in a programming language
Algorithms for Problem Solving
The main steps for Problem Solving are:
○ 1.Problem definition
○ 2.Algorithm design / Algorithm specification
○ 3.Algorithm analysis
○ 4.Implementation
○ 5.Testing
○ 6.[Maintenance]
● Step1. Problem Definition What is the task to be accomplished?
● Ex: Calculate the average of the grades for a given student

● Step2.Algorithm Design / Specifications: Describe: in natural

language / pseudo-code / diagrams / etc.

● Step3. Algorithm analysis Space complexity - How much space is

required Time complexity - How much time does it take to run the
algorithm Computer Algorithm

● An algorithm is a procedure (a finite set of well-defined instructions)

for accomplishing some tasks which, given an initial state terminate
in a defined end-state The computational complexity and efficient
implementation of the algorithm are important in computing, and
this depends on suitable data structures.
●
● Steps 4,5,6: Implementation, Testing, Maintenance
• Implementation: Decide on the programming language to use C, C++,
Lisp, Java, Perl, Prolog, assembly, etc. ,.
• Write clean, well documented code
• Test, test, test Integrate feedback from users, fix bugs, ensure
compatibility across different versions
• Maintenance. Release Updates, fix bugs
• Keeping illegal inputs separate is the responsibility of the algorithmic
problem, while treating special classes of unusual or undesirable
inputs is the responsibility of the algorithm itself
● Deals with developing and studying efficient algorithms for fixing
computational issues. It entails several steps, which includes problem
formulation, algorithm layout, algorithm analysis, and algorithm
optimization
● The problem formulation process entails identifying the
computational problem to be solved as well as specifying the input
and output criteria.
● The algorithm design process entails creating a set of instructions
that a computer can use to solve the problem. The algorithm analysis
process entails determining the method's efficiency in terms of time
and space complexity.
● Algorithm analysis is often performed by examining the algorithm's
worst-case time and space complexity. The time complexity of an
algorithm refers to the amount of time it takes to clear up a problem
as a characteristic of the input size. The space complexity of an
algorithm refers to the quantity of memory required to solve a
problem as a function of the enter length.
ALGORITHM SPECIFICATIONS

● Name of the algorithm

● Introductory comments
● Steps
● Datatypes
● Assignment
● Expressions
● If-statement
● Case statement
● Looping statements
● Input-output
● Go-To statement
● End statement
ALGORITHM SPECIFICATIONS

● An algorithm specification in Design and Analysis of Algorithms

(DAA) is a precise description of an algorithm that outlines the steps
involved in solving a problem. It typically includes the following
components:
● Algorithm name: A unique identifier for the algorithm.
● Input: A description of the input data that the algorithm takes.
● Output: A description of the output data that the algorithm
produces.
● Steps: A detailed description of the steps that the algorithm
performs, in a clear and unambiguous manner.
● Example: An example of how to use the algorithm to solve a specific
problem.
Here is an example of an algorithm specification for the linear search
algorithm:
● Algorithm name: Linear search

Input: A list of elements and a target element to search for.

Output: The index of the target element in the list, or -1 if the target
element is not found.

Steps:
1. Iterate through the list, comparing each element to the target
element.
2. If the target element is found, return its index.
3. If the end of the list is reached without finding the target element,
return -1.
● list = [1, 5, 3, 2, 4]
target = 3

index = linear_search(list, target)

if index != -1:
print("The target element is at index", index)
else:
print("The target element is not in the list")

● This algorithm specification provides a clear and concise description

of the linear search algorithm, including the input, output, steps, and
an example. It can be used to implement the algorithm in any
programming language.
Process of translating a problem into an algorithm
Sum of n no’s

Abstract solution:

● Sum of n elements by adding up elements one at a time.

Little more detailed solution:

● Sum=0;
● add a[1] to a[n] to sum one element at a time.
● return Sum;

More detailed solution which is a formal algorithm:

● Algorithm Sum (a,n)
● {
● sum:= 0.0;
● for i:= 1 to n do
● sum:=sum+a[i];
● Return sum;
● }
Find largest among list of elements

Abstract solution:

● Find the Largest number among a list of elements by considering one element at a
● time.

Little more detailed solution:

● 1. Treat first element as largest element
● 2. Examine a[2] to a[n] and suppose the largest element is at a [ i ];
● 3. Assign a[i] to largest;
● 4. Return largest;

More detailed solution which is a formal algorithm:

● Algorithm Largest (a,n)
● {
● largest := a[1];
● for i := 2 to n do
● {
● if ( a[i] > largest ) largest := a[i];
● }
● Return largest;
Linear search ● break;
● }
Abstract solution: ● }
● From the given list of elements compare one ● If (flag) then
by one from first element to last ● Write “element found at ‘pos’ position”
● element for the required element ● Else
Little more detailed solution: ● Write “element not found”
● for i := 1 to n do ● End if
● { ● }
● Examine a[i] to a[n] and suppose
● the required element is at a [ j ];
● Write match is found and return position j;
● }
More detailed solution which is a formal algorithm:
● Algorithm Linear search (a, req_ele)
● {
● Flag=0
● for i := 1 to n do
● {
● If (a[i] = req_ele) then
● {
● Flag=1;
● Pos = i;
Selection Sort

Abstract solution:
● From those elements that are currently
unsorted, find the smallest and place it
● next in the sorted list.
Little more detailed solution:
● for i := 1 to n do
● {
● Examine a[i] to a[n] and suppose
● the smallest element is at a [ j ];
● Interchange a[i] and a[j];
● }
More detailed solution which is a formal algorithm:
● Algorithm Selection Sort (a, n)
● // Sort the array a[1 : n] into non decreasing
order.
● {
● for i := 1 to n do
● {
● min:=i;
● for k : i + 1 to n do
● If (a[k]< a[min]) then min:=k;
● t :=a[i]; a[i] := a[min];a[min]:= t;
● }
● }
Bubble Sort ● {
● if ( a[j] > a[j+1] )
Abstract solution: ● {
● Comparing adjacent elements of an array and ● // swap a[j] and a[j+1]
exchange them if they are not in order. In the ● temp := a[j]; a[j] := a[j+1];a[j + 1] := temp;
● process, the largest element bubbles up to the ● }
last position of the array first and then the ● }
● second largest element bubbles up to the ● }
second last position and so on. ● }
Little more detailed solution:
● for i := 1 to n do
● {
● Compare a[1] to a [n-i] pair-wise (each element
with its next)
● and interchange a[j] and a[j+1] whenever a[j] is
greater than a[j+1]
● }
More detailed solution which is a formal algorithm:
● Algorithm BubbleSort (a,n)
● {
● for i := 1 to n do
● {
● for j :=1 to n-i do
Insertion sort insert a[i] at a[k].
● }
Abstract solution: More detailed solution which is a formal algorithm:
● Insertion sort works by sorting the first two ● Algorithm insertionSort(a,n)
elements and then inserting the third element ● {
in its ● for i := 2 to n do
● proper place so that all three are sorted. ● {
Repeating this process, k+1 st element is ● value := a[i]; j := i - 1;
inserted in its ● done := false;
● proper place with respect to the first k ● repeat
elements (which are already sorted) to make ● if a[j] > value
all k+1 ● {
● elements sorted. Finally, 'n' th element is ● a[j + 1] := a[j]; j := j - 1;
inserted in its proper place with respect to the ● if (j < 1) done := true;
first n-1 ● }
● elements so all n elements are sorted. ● else
Little more detailed solution: ● done := true;
● Sort a[1] and a [2] ● until done;
● For i from 3 to n do ● a[j + 1] := value;
● { ● }
● Suppose a[k] (k between 1 and i) is such that ● }
a[k-1] <= a[i] <= a[k].
● Then, move a[k] to a[i-1] by one position and
What is Pseudo Code
A Pseudocode is defined as a step-by-step description of an algorithm.
Pseudocode does not use any programming language in its representation
instead it uses the simple English language text as it is intended for human
understanding rather than machine reading.
Pseudocode is the intermediate state between an idea and its
implementation(code) in a high-level language.
What is the need for Pseudocode
● Pseudocode is an important part of designing an algorithm, it helps the
programmer in planning the solution to the problem as well as the reader in
understanding the approach to the problem. Pseudocode is an intermediate
state between algorithm and program that plays supports the transition of the
algorithm into the program
Pseudo Code

Example:
● IF “1”
print response
“I AM CASE 1”
● IF “2”
print response
“I AM CASE 2”

● Use appropriate naming conventions. The human tendency follows the approach of following what
we see. If a programmer goes through a pseudo code, his approach will be the same as per that, so
the naming must be simple and distinct.
● Reserved commands or keywords must be represented in capital letters.Example: if you are writing
IF…ELSE statements then make sure IF and ELSE be in capital letters.
● Check whether all the sections of a pseudo code are complete, finite, and clear to understand and
comprehend. Also, explain everything that is going to happen in the actual code.
● Don’t write the pseudocode in a programming language. It is necessary that the pseudocode is
simple and easy to understand even for a layman or client, minimizing the use of technical terms.
1. Binary search Pseudocode:

BinarySearch(ARR, X, LOW, HIGH)

repeat till LOW = HIGH
MID = (LOW + HIGH)/2
if (X == ARR[mid])
return MID

else if (x > ARR[MID])

LOW = MID + 1

else
HIGH = MID – 1

2. Quick sort Pseudocode:

QUICKSORT(Arr[], LOW, HIGH) {

if (LOW < HIGH) {
PIVOT = PARTITION(Arr, LOW, HIGH);
QUICKSORT(ARR, LOW, PIVOT – 1);
QUICKSORT(ARR, PIVOT + 1, HIGH);
}
}
 Difference between Algorithm and Pseudocode
Algorithm Pseudocode

An Algorithm is used to
provide a solution to a A Pseudocode is a step-by-step description of an algorithm in code-like structure
particular problem in form of a using plain English text.
well-defined step-based form.

An algorithm only uses simple

Pseudocode also uses reserved keywords like if-else, for, while, etc.
English words

These are a sequence of

These are fake codes as the word pseudo means fake, using code like structure
steps of a solution to a
and plain English text
problem

There are no rules to writing

There are certain rules for writing pseudocode
algorithms

Algorithms can be considered

Pseudocode cannot be considered an algorithm
pseudocode

It is difficult to understand and

It is easy to understand and interpret
interpret
Performance Analysis
● Performance analysis of an algorithm depends upon two factors i.e. amount of
memory used and amount of compute time consumed on any CPU.
● Formally they are notified as complexities in terms of:
Space Complexity.
Time Complexity.

● Space Complexity of an algorithm is the amount of memory it needs to run to

completion i.e. from start of execution to its termination.
● Space need by any algorithm is the sum of following components:
● Fixed Component: This is independent of the characteristics of the inputs and
outputs.
● This part includes: Instruction Space, Space of simple variables, fixed size
component variables, and constants variables.
● Variable Component: This consist of the space needed by component variables
whose size is dependent on the particular problems instances(Inputs/Outputs)
being solved, the space needed by referenced variables and the recursion stack
space is one of the most prominent components. Also this included the data
structure components like Linked list, heap, trees, graphs etc.
 Space Complexity

● Therefore the total space requirement of any algorithm 'A' can be

provided as

Space(A) = Fixed Components(A) + Variable Components(A)

● Among both fixed and variable component the variable part is

important to be determined accurately, so that the actual space
requirement can be identified for an algorithm 'A'.
To identify the space complexity of any algorithm following steps can be
followed:
● Determine the variables which are instantiated by some default
values.
● Determine which instance characteristics should be used to measure
the space requirement and this is will be problem specific.
● Generally the choices are limited to quantities related to the number
and magnitudes of the inputs to and outputs from the algorithms.
● Sometimes more complex measures of the interrelationships among
the data items can used.
 Example: Space Complexity

Algorithm Sum(number,size)\\ procedure will produce sum of all numbers

provided in 'number' list
{
result=0.0;
for count = 1 to size do \\will repeat from 1,2,3,4,....size times
result= result + number[count];
return result;
}
● In above example, when calculating the space complexity we will be looking
for both fixed and variable components. here we have
● Fixed components as 'result','count' and 'size' variable there for total space
required is three(3) words.
● Variable components is characterized as the value stored in 'size' variable
(suppose value store in variable 'size 'is 'n'). because this will decide the size of
'number' list and will also drive the for loop.
● therefore if the space used by size is one word then the total space required by
'number' variable will be 'n'(value stored in variable 'size').
● therefore the space complexity can be written as Space(Sum) = 3 + n;
Time Complexity

● Time Complexity of an algorithm(basically when converted to

program) is the amount of computer time it needs to run to
completion.
● The time taken by a program is the sum of the compile time and the
run/execution time .
● The compile time is independent of the instance(problem specific)
characteristics.
● following factors effect the time complexity:
● Characteristics of compiler used to compile the program.
● Computer Machine on which the program is executed and physically
clocked.
● Multiuser execution system.
● Number of program steps.
Time Complexity

● Therefore the again the time complexity consist of two components

fixed(factor 1 only) and variable/instance(factor 2,3 & 4), so for any
algorithm 'A' it is provided as:
● Time(A) = Fixed Time(A) + Instance Time(A)
● Here the number of steps is the most prominent instance
characteristics and The number of steps any program statement is
assigned depends on the kind of statement like
● comments count as zero steps,
● an assignment statement which does not involve any calls to other
algorithm is counted as one step,
● for iterative statements we consider the steps count only for the
control part of the statement etc.
Time Complexity

● Therefore to calculate total number program of program steps we

use following procedure.
● For this we build a table in which we list the total number of steps
contributed by each statement.
● This is often arrived at by first determining the number of steps per
execution of the statement and the frequency of each statement
executed.
● This procedure is explained using an example.
 Steps per
Frequency Total Steps
execution
Statement

Algorithm Sum(number,size) 0
0 -

{ 0 - 0

result=0.0;
1 1 1

for count = 1 to size do

1 size+1 size + 1

result= result +
number[count]; 1 size size

return result;
1 1 1

} 0 - 0

Total 2size + 3
● In above example if you analyze carefully frequency of "for count = 1
to size do" it is 'size +1' this is because the statement will be executed
one time more die to condition check for false situation of condition
provided in for statement.
● Now once the total steps are calculated they will resemble the
instance characteristics in time complexity of algorithm. Also the
repeated compile time of an algorithm will also be constant every
time we compile the same set of instructions so we can consider this
time as constant 'C'. Therefore the time complexity can be expressed
as: Time(Sum) = C + (2size +3)
● So in this way both the Space complexity and Time complexity can
be calculated.
● Combination of both complexity comprises the Performance analysis
of any algorithm and can not be used independently.
● Both these complexities also helps in defining parameters on basis
of which we optimize algorithm
Worst, Average and Best Case Analysis of Algorithms
Popular Notations in Complexity Analysis of Algorithms
asymptotic notation

1. Big-O Notation
We define an algorithm’s worst-case time complexity by using the Big-O notation, which
determines the set of functions grows slower than or at the same rate as the expression.
Furthermore, it explains the maximum amount of time an algorithm requires to consider all
input values.

2. Omega Notation
It defines the best case of an algorithm’s time complexity, the Omega notation defines whether
the set of functions will grow faster or at the same rate as the expression. Furthermore, it
explains the minimum amount of time an algorithm requires to consider all input values.

3. Theta Notation
It defines the average case of an algorithm’s time complexity, the Theta notation defines when
the set of functions lies in both O(expression) and Omega(expression), then Theta notation is
used. This is how we define a time complexity average case for an algorithm.
Measurement of Complexity of an Algorithm

● Based on the above three notations of Time Complexity there are three cases
to analyze an algorithm:

1. Worst Case Analysis (Mostly used)

● In the worst-case analysis, we calculate the upper bound on the running time
of an algorithm.
● We must know the case that causes a maximum number of operations to be
executed.
● For Linear Search, the worst case happens when the element to be searched
(x) is not present in the array. When x is not present, the search() function
compares it with all the elements of arr[] one by one.
● Therefore, the worst-case time complexity of the linear search would be O(n).
2. Best Case Analysis (Very Rarely used)

● In the best-case analysis, we calculate the lower bound on the running time of
an algorithm.
● We must know the case that causes a minimum number of operations to be
executed.
● In the linear search problem, the best case occurs when x is present at the first
location.
● The number of operations in the best case is constant (not dependent on n).
● So time complexity in the best case would be ?(1)
3. Average Case Analysis (Rarely used)
● In average case analysis, we take all possible inputs and calculate the
computing time for all of the inputs.
● Sum all the calculated values and divide the sum by the total number of inputs.
● We must know (or predict) the distribution of cases.
● For the linear search problem, let us assume that all cases are uniformly
distributed (including the case of x not being present in the array).
● So we sum all the cases and divide the sum by (n+1).
● Following is the value of average-case time complexity.
Average Case Time = \ sum_{i=1}^{n}\ frac{\ theta (i)}{(n+1)} = \ frac{\ theta
(\ frac{(n+1)*(n+2)}{2})}{(n+1)} = \ theta (n)
Which Complexity analysis is generally used?

Below is the ranked mention of complexity analysis notation based on popularity:

1. Worst Case Analysis:
● Most of the time, we do worst-case analyses to analyze algorithms. In the worst
analysis, we guarantee an upper bound on the running time of an algorithm
which is good information.
2. Average Case Analysis
● The average case analysis is not easy to do in most practical cases and it is
rarely done. In the average case analysis, we must know (or predict) the
mathematical distribution of all possible inputs.
3. Best Case Analysis
● The Best Case analysis is bogus. Guaranteeing a lower bound on an algorithm
doesn’t provide any information as in the worst case, an algorithm may take
years to run.
Interesting information about asymptotic notations:

A) For some algorithms, all the cases (worst, best, average) are asymptotically the
same. i.e., there are no worst and best cases.
● Example: Merge Sort does ?(n log(n)) operations in all cases.
● B) Where as most of the other sorting algorithms have worst and best cases.
● Example 1: In the typical implementation of Quick Sort (where pivot is chosen
as a corner element), the worst occurs when the input array is already sorted
and the best occurs when the pivot elements always divide the array into two
halves.
● Example 2: For insertion sort, the worst case occurs when the array is reverse
sorted and the best case occurs when the array is sorted in the same order as
output.
Examples with their complexity analysis: ● int arr[] = { 1, 10, 30, 15 };
● int x = 30;
1. Linear search algorithm: ● int n = sizeof(arr) / sizeof(arr[0]);
●
● // C implementation of the approach ● // Function call
● #include <stdio.h> ● printf("%d is present at index %d", x,
● ● search(arr, n, x));
● // Linearly search x in arr[]. ●
● // If x is present then return the index, ● getchar();
● // otherwise return -1 ● return 0;
● int search(int arr[], int n, int x) ● }
● {
● int i;
● for (i = 0; i < n; i++) { Output
● if (arr[i] == x) 30 is present at index 2
● return i;
● }
● return -1;
● }

● /* Driver's code*/
● int main()
● {
Time Complexity Analysis: (In Big-O notation)

● Best Case: O(1), This will take place if the element to be searched is on the first
index of the given list. So, the number of comparisons, in this case, is 1.
● Average Case: O(n), This will take place if the element to be searched is on the
middle index of the given list.
● Worst Case: O(n), This will take place if:
○ The element to be searched is on the last index
○ The element to be searched is not present on the list
Important points:

● The worst case analysis of an algorithm provides an upper bound on

the running time of the algorithm for any input size.
● The average case analysis of an algorithm provides an estimate of the
running time of the algorithm for a random input.
● The best case analysis of an algorithm provides a lower bound on the
running time of the algorithm for any input size.
● The big O notation is commonly used to express the worst case
running time of an algorithm.
● Different algorithms may have different best, average, and worst case
running times.
Types of Asymptotic Notations in Complexity Analysis of Algorithms

● The main idea of asymptotic analysis is to have a measure of the

efficiency of algorithms
● that don’t depend on machine-specific constants and don’t require
algorithms to be implemented and time taken by programs to be
compared.
● Asymptotic notations are mathematical tools to represent the time
complexity of algorithms for asymptotic analysis
Asymptotic Notations:
● Asymptotic Notations are mathematical tools that allow you to analyze an
algorithm’s running time by identifying its behavior as its input size grows.
● This is also referred to as an algorithm’s growth rate.
● You can’t compare two algorithm’s head to head.
● You compare space and time complexity using asymptotic analysis.
● It compares two algorithms based on changes in their performance as the
input size is increased or decreased.
asymptotic notations:
There are mainly three asymptotic notations:

● Big-O Notation (O-notation)

● Omega Notation (Ω-notation)
● Theta Notation (Θ-notation)
1. Theta Notation (Θ-Notation):

● Theta notation encloses the function from above and below.

● Since it represents the upper and the lower bound of the running time of an algorithm, it is
used for analyzing the average-case complexity of an algorithm.
● Theta (Average Case) You add the running times for each possible input combination and
take the average in the average case.
Let g and f be the function from the set of natural numbers to itself. The function f is said to be
Θ(g), if there are constants c1, c2 > 0 and a natural number n0 such that
c1* g(n) ≤ f(n) ≤ c2 * g(n) for all n ≥ n0
2. Big-O Notation (O-notation):

● Big-O notation represents the upper bound of the running time of an algorithm. Therefore, it gives
the worst-case complexity of an algorithm.
● It is the most widely used notation for Asymptotic analysis.
It specifies the upper bound of a function.
The maximum time required by an algorithm or the worst-case time complexity.
It returns the highest possible output value(big-O) for a given input.
Big-Oh(Worst Case) It is defined as the condition that allows an algorithm to complete statement
execution in the longest amount of time possible.

● If f(n) describes the running time of an algorithm, f(n) is O(g(n)) if there exist a positive constant C
and n0 such that, 0 ≤ f(n) ≤ cg(n) for all n ≥ n0
● It returns the highest possible output value (big-O)for a given input.
● The execution time serves as an upper bound on the algorithm’s time complexity.
3. Omega Notation (Ω-Notation):

Omega notation represents the lower bound of the running time of an algorithm. Thus, it
provides the best case complexity of an algorithm.
● The execution time serves as a lower bound on the algorithm’s time complexity.
● It is defined as the condition that allows an algorithm to complete statement execution in
the shortest amount of time.
● Let g and f be the function from the set of natural numbers to itself. The function f is said to
be Ω(g), if there is a constant c > 0 and a natural number n0 such that
c*g(n) ≤ f(n) for all n ≥ n0
Randomized Algorithms
● Randomized algorithm is a different design approach taken by the standard
algorithms where few random bits are added to a part of their logic.
● They are different from deterministic algorithms; deterministic algorithms
follow a definite procedure to get the same output every time an input is
passed where randomized algorithms produce a different output every time
they’re executed.
● It is important to note that it is not the input that is randomized, but the logic
of the standard algorithm.
Unlike deterministic algorithms, randomized algorithms consider randomized bits
of the logic along with the input that in turn contribute towards obtaining the
output.
However, the probability of randomized algorithms providing incorrect output
cannot be ruled out either.
Hence, the process called amplification is performed to reduce the likelihood of
these erroneous outputs.
Amplification is also an algorithm that is applied to execute some parts of the
randomized algorithms multiple times to increase the probability of correctness.
However, too much amplification can also exceed the time constraints making the
algorithm ineffective.
Classification of Randomized Algorithms

● Randomized algorithms are classified based on whether they have time

constraints as the random variable or deterministic values. They are designed
in their two common forms − Las Vegas and Monte Carlo.
Las Vegas − The Las Vegas method of randomized algorithms never gives incorrect
outputs, making the time constraint as the random variable.
● For example, in string matching algorithms, las vegas algorithms start from the
beginning once they encounter an error. This increases the probability of
correctness. Eg., Randomized Quick Sort Algorithm(Time Complexity=N n(-log
log(n)
.

Monte Carlo − The Monte Carlo method of randomized algorithms focuses on

finishing the execution within the given time constraint.
● Therefore, the running time of this method is deterministic.
● For example, in string matching, if monte carlo encounters an error, it restarts
the algorithm from the same point. Thus, saving time. Eg., Karger’s Minimum
Cut Algorithm (O(mn)).

Time complexity random monte carlo: O(mlogn)