Fundamentals of Analysis of Algorithms

Lecture_5 : Algorithms Design

Strategies(Decrease-and-Conquer )

By: Mutasim Idreis

 Decrease-and-Conquer
The decrease-and-conquer technique
1. Reduce problem instance to smaller instance of
the same problem
2. Solve smaller instance
3. Extend solution of smaller instance to obtain
solution to original instance

Can be implemented either top-down or bottom-up

Also referred to as inductive or incremental approach
12/24/2024
University of gadarif, Presented by: Mutasim Idreis
2
 Decrease-and-Conquer
3 Types of Decrease and Conquer
Decrease by a constant (usually by 1):
• insertion sort
• topological sorting
• algorithms for generating permutations, subsets
Decrease by a constant factor (usually by half)
• binary search and bisection method
• exponentiation by squaring
• multiplication à la russe
Variable-size decrease
• Euclid’s algorithm
• selection by partition
12/24/2024
University of gadarif, Presented by: Mutasim Idreis
3
 Decrease-and-Conquer
Insertion Sort
To sort array A[0..n-1], sort A[0..n-2] recursively and
then insert A[n-1] in its proper place among the sorted
A[0..n-2]

Usually implemented bottom up (nonrecursively)

Example: Sort 6, 4, 1, 8, 5

6|4 1 8 5
4 6|1 8 5
1 4 6|8 5
1 12/24/2024
4 6 8|5
University of gadarif, Presented by: Mutasim Idreis
1 4 5 6 8 4
 Decrease-and-Conquer
Pseudocode of Insertion Sort

12/24/2024
University of gadarif, Presented by: Mutasim Idreis
5
12/24/2024
University of gadarif, Presented by: Mutasim Idreis
6
 Decrease-and-Conquer
Analysis of Insertion Sort
Time efficiency
Cworst(n) = n(n-1)/2  Θ(n2)
Cavg(n) ≈ n2/4  Θ(n2)
Cbest(n) = n - 1  Θ(n) (also fast on almost sorted arrays)

Space efficiency: in-place

Stability: yes
Best elementary sorting algorithm overall
Binary insertion sort
12/24/2024
University of gadarif, Presented by: Mutasim Idreis
7
 Decrease-and-Conquer
Dags and Topological Sorting
A dag: a directed acyclic graph, i.e. a directed graph with no
(directed) cycles
a b a b
a dag not a dag

c d c d

Arise in modeling many problems that involve prerequisite

constraints (construction projects, document version control)
Vertices of a dag can be linearly ordered so that for every edge
its starting vertex is listed before its ending vertex (topological
sorting). Being a dag is also a necessary condition for
12/24/2024
University of gadarif, Presented by: Mutasim Idreis
topological sorting be possible. 8
 Decrease-and-Conquer
Topological Sorting Example
Order the following items in a food chain

tiger

human
fish
sheep
shrimp

plankton
12/24/2024
wheat
University of gadarif, Presented by: Mutasim Idreis
9
 Decrease-and-Conquer
DFS-based Algorithm
DFS-based algorithm for topological sorting
• Perform DFS traversal, noting the order vertices are
popped off the traversal stack
• Reverse order solves topological sorting problem
• Back edges encountered?→ NOT a dag!

Example:
a b c d

e f g h
12/24/2024
Efficiency: Ꝋ(𝒗𝟐 ) 10
12/24/2024
University of gadarif, Presented by: Mutasim Idreis
11
 Decrease-and-Conquer
Source Removal Algorithm
Source removal algorithm
Repeatedly identify and remove a source (a vertex with no
incoming edges) and all the edges incident to it until either
no vertex is left (problem is solved) or there is no source
among remaining vertices (not a dag)
Example:
a b c d

e f g h

12/24/2024
Efficiency: same University
as efficiency of the
of gadarif, Presented DFS-based
by: Mutasim Idreis algorithm
12
12/24/2024
University of gadarif, Presented by: Mutasim Idreis
13
 Decrease-and-Conquer
Generating Permutations
Minimal-change decrease-by-one algorithm
If n = 1 return 1; otherwise, generate recursively the list of all
permutations of 12…n-1 and then insert n into each of
those permutations by starting with inserting n into 12...n-1
by moving right to left and then switching direction for
each new permutation

Example: n=3
start 1
insert 2 into 1 right to left 12 21
insert 3 into 12 right to left 123 132 312
insert 3 into 21 left to right 321 231 213
12/24/2024
University of gadarif, Presented by: Mutasim Idreis
14
 Decrease-and-Conquer
Other permutation generating algorithms
Johnson-Trotter (p. 145)

Lexicographic-order algorithm (p. 146)

Heap’s algorithm ()

12/24/2024
University of gadarif, Presented by: Mutasim Idreis
15
 Decrease-and-Conquer
Generating Subsets
Binary reflected Gray code: minimal-change algorithm for
generating 2n bit strings corresponding to all the subsets of
an n-element set where n > 0
If n=1 make list L of two bit strings 0 and 1
else
generate recursively list L1 of bit strings of length n-1
copy list L1 in reverse order to get list L2
add 0 in front of each bit string in list L1
add 1 in front of each bit string in list L2
append L2 to L1 to get L
return L
12/24/2024
University of gadarif, Presented by: Mutasim Idreis
16
 Decrease-and-Conquer
Decrease-by-Constant-Factor Algorithms
In this variation of decrease-and-conquer, instance size
is reduced by the same factor (typically, 2)

Examples:
• binary search and the method of bisection

• exponentiation by squaring

• multiplication à la russe (Russian peasant method)

• fake-coin puzzle
12/24/2024
• University of gadarif, Presented by: Mutasim Idreis
Josephus problem 17
 Decrease-and-Conquer
Binary Search
Very efficient algorithm for searching in sorted array:
K
vs
A[0] . . . A[m] . . . A[n-1]
If K = A[m], stop (successful search); otherwise, continue
searching by the same method in A[0..m-1] if K < A[m]
and in A[m+1..n-1] if K > A[m]
l  0; r  n-1
while l  r do
m  (l+r)/2
if K = A[m] return m
else if K < A[m] r  m-1
else l  m+1
12/24/2024
return -1 University of gadarif, Presented by: Mutasim Idreis
18
 Decrease-and-Conquer
Analysis of Binary Search
Time efficiency
• worst-case recurrence: Cw (n) = 1 + Cw( n/2 ), Cw (1) = 1
solution: Cw(n) = log2(n+1)

This is VERY fast: e.g., Cw(106) = 20

Optimal for searching a sorted array

Limitations: must be a sorted array (not linked list)

Bad (degenerate) example of divide-and-conquer

Has a continuous counterpart called bisection method for

12/24/2024
solving equations
Universityin onePresented
of gadarif, unknown f(x)
by: Mutasim Idreis = 0
19
 Decrease-and-Conquer
Russian Peasant Multiplication
The problem: Compute the product of two positive integers
Can be solved by a decrease-by-half algorithm based on the
following formulas.
For even values of n:

n * m = n * 2m
2

For odd values of n:

n * m = n – 1 * 2m + m if n > 1 and m if n = 1
2

12/24/2024
University of gadarif, Presented by: Mutasim Idreis
20
 Decrease-and-Conquer
Example of Russian Peasant Multiplication
Compute 20 * 26
n m
20 26
10 52
5 104 104
2 208 +
1 416 416
520
Note: Method reduces to adding m’s values corresponding to
odd n’s.
12/24/2024
University of gadarif, Presented by: Mutasim Idreis
21
 Decrease-and-Conquer
Variable-Size-Decrease Algorithms
In the variable-size-decrease variation of decrease-and-
conquer, instance size reduction varies from one
iteration to another

Examples:
• Euclid’s algorithm for greatest common divisor
• partition-based algorithm for selection problem
• interpolation search
• some algorithms on binary search trees
• Nim and Nim-like games
12/24/2024
University of gadarif, Presented by: Mutasim Idreis
22
 Decrease-and-Conquer
Euclid’s Algorithm
Euclid’s algorithm is based on repeated application of equality
gcd(m, n) = gcd(n, m mod n)

Ex.: gcd(80,44) = gcd(44,36) = ?

One can prove that the size, measured by the second number,
decreases at least by half after two consecutive iterations.
Hence, T(n)  O(log n)

12/24/2024
University of gadarif, Presented by: Mutasim Idreis
23
 Decrease-and-Conquer
Selection Problem
Find the k-th smallest element in a list of n numbers
k = 1 or k = n

median: k = n/2
Example: 4, 1, 10, 9, 7, 12, 8, 2, 15 median = ?

The median is used in statistics as a measure of an average

value of a sample. In fact, it is a better (more robust) indicator
than the mean, which is used for the same purpose.

12/24/2024
University of gadarif, Presented by: Mutasim Idreis
24
 Decrease-and-Conquer
Algorithms for the Selection Problem
The sorting-based algorithm: Sort and return the k-th element
Efficiency (if sorted by mergesort): Θ(nlog n)

A faster algorithm iss based on the array partitioning:

all are ≤ A[s] all are ≥ A[s]

Assuming that the array is indexed from 0 to n-1 and s is a split

position obtained by the array partitioning:
If s = k-1, the problem is solved;
if s > k-1, look for the k-th smallest element in the left part;
if s < k-1, look for the (k-s)-th smallest element in the right part.
12/24/2024
University of gadarif, Presented by: Mutasim Idreis
Note: The algorithm can simply continue until s = k-1. 25
 Decrease-and-Conquer
Two Partitioning Algorithms
There are two principal ways to partition an array:

One-directional scan (Lomuto’s partitioning algorithm)

Two-directional scan (Hoare’s partitioning algorithm)

12/24/2024
University of gadarif, Presented by: Mutasim Idreis
26
 Thank you

12/24/2024
University of gadarif, Presented by: Mutasim Idreis
27