Unit III

Divide and Conquer

Greedy Strategy

Reference for this presentation is

Horowitz and Sahani, "Fundamentals of Computer Algorithms", 2nd edition. Galgotia
publication,, 2008, ISBN: 978 81 7371 6126
Divide and Conquer [D&C]
✔ General Strategy
✔ Control Abstraction
✔ Min/Max Problem
✔ Binary Search
✔ Quick Sort
✔ Merge Sort
 D&C General Strategy (Definition)
• Term ‘Divide and Conquer 'originated in military science

• First used by Julius Caesar. “Enemy forces are fragmented

first and then smaller fragments conquered”

• Given a function to compute on n inputs the divide- and-

conquer strategy suggests splitting the inputs into k
distinct subsets, 1 < k ≤ n yielding k sub problems.

• These sub problems must be solved and then a method

must be found to combine sub solutions into a solution of
the whole.
Divide, Conquer, Combine and Recurrence
• Divide the problem into a number of sub problems
that are smaller instances of the same problem.
• Conquer the sub problems by solving them
recursively. If the sub problem sizes are small
enough, however, just solve the sub problems in a
straight forward manner.
• Combine the solutions to the sub problems into the
solution for the original problem.
• Recurrence go hand in hand with the divide-and-
conquer as the sub problems which are large enough
are solved recursively
Control Abstraction of D&C
Recursive D&C strategy
 Computing time of D&C
• Computing time of D&C is described by the recurrence
relation:

Where:
• T(n) is the time for D&C on any input of size n
• g(n) is the time to compute the answer directly for small
inputs.
• The function f(n) is the time for dividing P and combining
the solutions to sub problems
• For D&C based algorithms that produce sub problems of
the same type as the original problem, it is very natural to
first describe such algorithms using recursion
 D&C: Example
Min/Max Problem
D&C: Example Min/Max Problem Naïve approach
Find maximum
value in an array of
size n
Time complexity
• Naïve algo: O(n)
• D & C algo: O(nlog2 n)
35 Maximum(start_index_of_array, end_index_of_array)
30
25
20 n

15 log(n)

10 nlog(n)

5
0
1 2 3 4 5
6 7 8 9 10
D&C: Example Min/Max Problem
Maximum(x , y)
A[0] =10
A[1] =12
A[2] = 8
A[3] = 2
A[4] = 3
A[5] = 1

x=0, y= 1, return max( a[0], a[1]) ≡ return 12

x=0, y= 2, max1= maximum (0,1) , max2 = maximum (2, 2) , return max( max1, max2)
x=0, y= 5, max1= maximum (0,2), max2 = maximum (3, 5), return max( max1, max2)
ans= maximum ( 0 , 5)

max1= maximum (0,1) (12), max2 = return max( a[2], a[2]) ≡ 8, return max( 12, 8) 12
max1= maximum (0,2) (12), max2 = return max( a[3], a[5]) ≡ 2 , return max( 12, 2) 12
ans= maximum ( 0 , 5)
A[0] =10
D&C: Example Min/Max Problem
A[1] =12
Maximum(x , y)
A[2] = 8
A[3] = 2
A[4] = 3
A[5] = 1

x=3, y= 4, return (max(a(3), a(4)) ≡ 3

x=3, y= 5, max1= maximum (3,4), max2 = maximum (5, 5), return max( max1, max2)
x=0, y= 5, max1= maximum (0,2), max2 = maximum (3, 5), return max( max1, max2)
ans= maximum ( 0 , 5)

max1= maximum (3,4) (3) , max2 = maximum (5, 5) (1) , return max( max1, max2) (3)
max1= maximum (0,2) (12), max2 = (return max( a[3], a[5]) ≡ 3, return max( 12, 3) 12
ans= maximum ( 0 , 5) 12
Recursive Calls Example: Algorithm maximum()
Find Maximum value for given array
Element 5 10 15 2 -1 -50 100 7 3
Index 1 2 3 4 5 6 7 8 9

Tree of Recursive Calls of function maximum()

(1,9)

(1,5) (6,9)

(1,3) (4,5) (6,7) (8,9)

(1,2) (3,3)
Maximum (1,9) Maximum (1,5)
If ( 9-1 <= 1 ) If ( 5-1 <= 1 )
…….. ……..
else else
max1 = maximum(1,5) => 15 max1 = maximum(1,3) => 15
max2 = maximum(6,9) => 100 max2 = maximum(4,5) => 2
return(max1,max2) =>
max(15,100) return(max1,max2) =>
return(100) max(15,100)
return(15)

Maximum (1,3) Maximum (1,2)

If ( 3-1 <= 1 ) If ( 2-1 <= 1 )
…….. return ( max( a[1],a[2] ) =>
else max(5,10)
max1 = maximum(1,2) => 10 return(10)
max2 = maximum(3,3) => 15

return(max1,max2) =>
Recurrence Equation for MinMax Algorithm
D&C: Example
Binary Search
Iterative Binary Search
Examples of Binary Search
Recursive Binary Search
Binary Decision Tree for Binary Search
n=14
 Binary Search: Recurrence
Input: Sorted array A of size n, an element x to be searched

Question: Is x ∈ A

Approach: Check whether A[n/2] = x. If x > A[n/2], then

prune the lower half of the array, A[1, . . . , n/2]. Otherwise,
prune the upper half of the array. Therefore, pruning happens
at every iterations. After each iteration the problem size
(array size under consideration) reduces by half.

Recurrence relation is
T(n) = T(n/2) + 1 ………Otherwise
T(1) = 1 ……..n=1
where T(n) is the time required for binary search in an array of size n.
Binary Search: Recurrence
T(n) = T(n/2) + 1 ………Otherwise
T(1) = 1 ……..n=1
where T(n) is the time required for binary search in an array of size n.

1st step=> T(n)=T(n/2) + 1

2nd step=> T(n/2)=T(n/4) + 1 ……[ T(n/4)= T(n/2^2) ]
3rd step=> T(n/4)=T(n/8) + 1 ……[ T(n/8)= T(n/2^3) ].
kth step=> T(n) = T( n / 2 k ) + 1 + · · · + 1 ….. [1*k times]
Final Equation: T(n) = T( n / 2 k ) + k times 1

So how many times we need to divide by 2 until we have only one element left
Since T(1) = 1,when n = 2k
⇒ log n=k [taken log(base 2) on both sides ] Put k= log n in final eq
T(n) = T(1) + k = 1 + log2 (n).
T(n) = Θ(log2 (n))
Time Complexity of Binary Search
D&C: Example
Merge Sort
Merge Sort
 Examples of Merge Sort

Vertical bar Indicates Boundaries of subarrays

Element a[1] and a[2] are Merged

Merge Sort Example Contd....
 Merge Sort Example Contd....
Second recursive call in merge sort

Element a[6] and a[7] are Merged then a[8] is

merged with a[6:7]
Merge Sort Example Contd....

At this point 2 sorted subarrays and final merge

procedure result
Merging of 2 sorted sub arrays using auxiliary storage
Tree of Calls of Merge Sort(1,10)
Tree of Calls of Merge
 Recurrence of Merge Sort

T(n) = a ……… n=1, a is constant

= 2T(n/2) + cn ……… n>1 C is constant

When n is a power of 2, n = 2k

Home Work: Solve recurrence using substitution method

D&C: Example
Quick Sort
 Quick Sort
• Sorts the array by positioning each element at it's proper
position and partitioning the array into two sub arrays at the
proper position of the moved element.

• Let x be chosen from any position in the array and be

placed in the position i such that
• All elements placed to the left of i are less than or
equal to x
• All elements placed to the right of i are greater than x

Therefore x is ith smallest element of array a[]. Repeat

procedure for a[0] to a[i-1] and a[i+1] to a[n]
 Quick Sort
Given an array of n elements
if array only contains one element
return
else
pick one element to use as pivot
partition elements into two sub-arrays
elements <= pivot
elements > pivot
quicksort() two sub-arrays
return result
 Partition Example
1.While data[Low] <= data[pivot]
Low++

2.While data[High] > data[pivot]

High--

3.If Low < High

swap data[Low] and data[High]

4. While High > Low, go to 1.

5. Swap data[High] and data[pivot_index]

 Partition Example contd...

Pivot

Pivot 40 20 10 80 60 50 7 30 100
index
=0 0 1 2 3 4 5 6 7 8

Low High

Working of Partition solve it on board

1.While data[Low] <= data[pivot]
Low++
20 < 40 low ++

1.While data[Low] <= data[pivot]

Low++
10 < 40 low ++

2.While data[High] > data[pivot]

High - -
100 > 40 High - -

2.While data[High] > data[pivot] No

3.If Low < High
swap data[Low] and data[High]
Swap 80 and 30
1.While data[Low] <= data[pivot]
Low++
30 < 40 low ++

1.While data[Low] <= data[pivot] No

2.While data[High] > data[pivot]
High - -
80 > 40 High - -

2.While data[High] > data[pivot] No

3.If Low < High
swap data[Low] and data[High]
Swap 60 and 7

4. While High > Low, go to 1.

1.While data[Low] <= data[pivot]
Low++
7 < 40 low ++

1.While data[Low] <= data[pivot] No

2.While data[High] > data[pivot]
High—
60 > 40

1.While data[Low] <= data[pivot] No

2.While data[High] > data[pivot]
High—
50 > 40

2.While data[High] > data[pivot] No

3.If Low < High No
4.While High > Low, go to 1. No
5. Swap data[High] and data[pivot_index]
Swap 7 and 40
 Partition Example contd...

Pivot 7 20 10 30 40 50 60 80 100
index
0 1 2 3 4 5 6 7 8
=4

<=data(pivot) Pivot >data(pivot)

Hoare's Method of Partitioning
 Unit III
Greedy Strategy
✔General Strategy
✔ Control Abstraction
✔ Knapsack Problem
✔ Job Sequencing with Deadline
✔ Minimum Spanning Tree
 Definition: Greedy strategy
• Applied for constrained optimization problems
• A greedy algorithm always makes the choice from
feasible solutions that looks best at that moment.
That is, it makes a locally optimal choice in the
hope that this choice will lead to a globally
optimal solution.
• Straightforward design technique
• Can be applied to a wide variety of problems
• E.g.: knapsack, TSP, 8 queens, 8 puzzle, resource
allocation…
 Constrained optimization problem
• Most, though not all, of these problems have n
inputs and requires to obtain a subset that
satisfies some constraints.
• Any subset that satisfies these constraints is
called a feasible solution.
• A feasible solution either maximizes or
minimizes a given objective function.
• A feasible solution that does this is called
an optimal solution.
Control abstraction for Greedy algorithm
GREEDY(A, n)
1. A[1:n] an array contains n inputs
2. solution empty
3. for i 1 to n do
4. x SELECT(A) [as per some optimization
criteria]
5. if FEASIBLE (solution, x) then
6. solution - UNION(solution, x)
7. end
8. end
9. return (solution)
10.end GREEDY
 Select, Feasibility and Union

• SELECT(A) : select an element from array A[]

such that it has a potential for satisfying the
optimality criteria( selection policy)
• FEASIBLE (solution, x): checks if the selected
element x satisfies the feasibility criteria
• UNION(solution, x): integrates the element x in
the solution
Greedy: Example
Knapsack Problem
 Greedy Algorithm: Example 1
Knapsack Problem
• Given N objects and a knapsack or bag.
• Object i has a weight wi and the knapsack has a
capacity M.
• If a fraction xi, 0 <xi < 1,of object i is placed into
the knapsack, then a profit of pixi is earned.
• The objective is to obtain a filling of the
knapsack that maximizes the total profit earned.
• Sincethe knapsack capacityis M, the
total weight of all chosen objects to be at most
 Greedy Algorithm: Example 1
Knapsack Problem

• Knapsack problem: n = 3,m= 20,

• (p1,p2,p3) =(25, 24,15),
• (w1,w2,w3)= (18, 15, 10)
Solution type x1 x2 x3 Σwixi Σpixi
Random selection
Largest profit
Minimum weight
Largest p/w
 Greedy Algorithm: Example 1
Knapsack Problem

• Knapsack problem: n = 3,m= 20,

• (p1,p2,p3) =(25, 24,15),
• (w1,w2,w3)= (18, 15, 10)
Solution type x1 x2 x3 Σwixi Σpixi
Random selection 0.5 0.333 0.25 16.5 24.5
Largest profit 1 0.13 0 20 28.2
Minimum weight 0 0.667 1 20 31.0
Largest p/w 0 1 0.5 20 31.5
 Greedy Algorithm: Example 1
Knapsack Problem
Solution for Largest Profit/Weight
• Knapsack problem: n = 3,m= 20,
(p1,p2,p3) =(25, 24,15) and (w1,w2,w3)= (18, 15, 10)
x1 x2 x3 Σwixi Σpixi
Profit (pi) 25 24 15
Weight (wi) 18 15 10
Pi/wi 25/18 = 24/15 = 1.6 15/10 = 1.5
1.38
Select Largest p/w 0 1 5/10 20 (24 +
5/10*15) =
31.5

Solution is Item’s Selected =( 0, 1, 5/10) and Profit is (24 +5/10*15) = 31.5

Largest P/w will always give optimal solution for knapsack
 Greedy Algorithm: Example 1
Knapsack Problem: Practice
• Knapsack problem: n = 3,m= 20,
• (p1,p2,p3) =(45, 24,16)
• (w1,w2,w3)= (14, 18, 10)
x1 x2 x3 Σwixi Σpixi
Profit (pi)
Weight (wi)
Pi/wi
Select Largest p/w

Solution is Item’s Selected =(-----------) and Profit is = --------

 Greedy Algorithm: Example 1
Knapsack Problem: Practice
• Knapsack problem: n = 7,m= 15,
• (p1,p2,p3, p4,p5,, p6, p7) =(10, 5,15, 7, 6, 18, 3)
• (w1,w2,w3, w4,w5,w6, w7)= (2, 3, 5, 7, 1, 4, 1)
x1 X2 X3 X4 X5 x6 x7 Σwixi Σpixi
Profit (pi) 10 5 15 7 6 18 3
Weight (wi) 2 3 5 7 1 4 1
Pi/wi 5 1.6 3 1 6 4.5 3
Select 1 2/3 1 0 1 1 1
Largest p/w

Solution is Item’s Selected =(1, 2/3, 1, 0,1,1,1) and Profit is = 55.33

Algorithm: greedy strategy: The knapsack problem
1. GREEDY_KNAPSACK(P, W. M, X, n)
2. {// P(0 :n-1) and W(0 :n-1) contain the profits & weights resptly of n objects
3. //objects ordered as P(i)/ W(i) ≥ P(i + l)/W(i + 1)
4. //M is knapsack size and X(0 :n-1) is the solution vector
5. Real cu; integer i, n ;
6. X←0 //initialize solution to zero
7. cu ← M //cu = remaining knapsack capacity
8. For i ← 0 to n-1 do
9. if W(i) > cu then
10. //no space, taken care of outside loop
11. break
12. endif
13. X(i) ← 1
14. cu ← cu - W(i)
15. End
16. if i < n then
17. X(i) ← cu/W(i)
18. //if objects with higher profit could not be accommodated and still space
remains
then fill by lesser profit objects
19. endif
20. } GREEDY _KNAPSACK
 Greedy: Example
Job Sequencing with Deadline
 Greedy Algorithm: Example 1
Job Sequencing Problem
• Job Sequencing without deadline
• Job Sequencing with deadline
• Assumptions
• All jobs are scheduled on single processor
• All jobs are queued at single time
• Total system time for job is
waiting time + processing time
 Greedy algorithm: Example 2: Job sequencing without deadline
For given system schedule 3 jobs with service time as
given n = 3, (t1, t2, t3) = (2, 7, 4). Schedule job for
minimum system time
Schedul Total Time in System or Average Time
In general for e (Total Time / No. of
Jobs)
n jobs, [1, 2, 3] 2 + (2+7) + (2+7 +4) = 24 24/3 = 8
n! possible [1, 3, 2] 2 + (2+4) + (2+4+7) = 21 21/3 = 7
solutions [2, 1, 3] 7 + (7+2) + (7+2+4) = 29 29/3 = 9.6
For this
[2, 3, 1] 7 + (7+4) + (7+4+2) = 31 31/3 = 10.3
problem 3!
[3, 1, 2] 4 + (4+2) + (4+2+7) = 23 23/3 = 7.9
Solutions
[3, 2, 1] 4 + (4+7) + (4+7+2) = 28 28/3 = 9.3
Solution: Minimum Turn around time = 7 with total time 21
Optimal Order = [1, 3, 2]
Service Time = [2, 4, 7]
 Greedy Algorithm: Example 1
Job Sequencing without Deadline
Step 1 : Sort the jobs by service time in a
non-decreasing order ( Shortest Job First )
Step 2 : Schedule the next job of the sorted job list,
include this in solution set
Step 3 : If all instances of a sorted job list are solved,
then return the solution set

Complexity Analysis ≈ nlogn

Sorting of jobs O(nlogn)
Other tasks will take constant time O(n)
 Greedy Algorithm: Example 1
Job Sequencing with Deadline Problem
• Schedule jobs to maximize total profit
• Task completion is associated with profit
• To make profit finish job before deadline, if not finished
no profit at all
• Objective of problem is to construct feasible sequence
which gives maximum profit

Feasible Sequence: Sequence of all jobs end by their deadline

Feasible Set: A set of jobs where at least one sequence is possible
Optimal Sequence: Sequence with maximum profit
Optimal set of jobs: Elements that constitute the optimal sequence
comprises optimal set of jobs
 Greedy Algorithm: Example 1
Job Sequencing with Deadline Problem
• Given a set of n jobs. Associated with job i is an
integer deadline di >=0 and a profit pi > 0. For any
job i the profit pi is earned iff the job is completed by
its deadline.
• Each job on a machine takes one unit of time.
• Only one machine is available for processing jobs.
• A feasible solution for this problem is a subset J, of n
jobs such that each job can be completed within its
deadline.
• The value of a feasible solution is Σpi ∀ i ∈ J
• An optimal solution is a feasible solution with
maximum value.
 Greedy algorithm: example 2: Job sequencing
with deadlines
Algorithm:
1. Sort all jobs in non- increasing order of profit
2. Initialize the resultsequence as first job
from sorted jobs list
3. Do the following for remaining n-1 jobs
If the current job can fit in the current result sequence
without missing the deadline,
then add current job to result sequence
else ignore the current job
 Greedy algorithm: Example 2: Job sequencing with deadlines
Let n = 4, (p1, p2, p3, p4) = (100, 10, 15, 27) and (d1,
d2,d3, d4) = (2, 1, 2, 1). The feasible solutions and their
values are:
Solutions using combinatorial search
Feasible solution Processing value
(i) (1, 2) 2, 1 110
(ii) (1, 3) l,3or3,l 115
N=4
(iii) (1, 4) 4, 1 127
dmax = 2
(iv) (2, 3) 2,3 25
(v) (3, 4) 4,3 42
Selecting ≤ 2
instances (vi) (1) 1 100
out of 4. (vii) (2) 2 10
(viii) (3) 3 15
(ix) (4) 4 27
 Greedy algorithm: example 2: Job sequencing with
deadlines
• Let n = 5, (p1, p2, p3, p4, p5) = (60, 100, 20, 40, 20)
and (d1, d2,d3,d4,d5) = (2, 1, 3, 2, 1). The
feasible solutions and their values are:
Solutions using combinatorial search
Feasible solution Processing value
N=5
dmax = 3 (i) (1, 2,3) 2, 1 , 3 180
(ii) (1,3, 4) 1,4,3 or 4,1,3 120
(iii) (1, 5, 3) 5, 1, 3 100
Selecting ≤ 3
instances (iv) :
out of 5. (v) (2,3,5) Not allowed
(vi) :
 Greedy algorithm: example 2: Job sequencing with deadlines
Practice Problem

• Let n = 5, (J1, J2, J3, J4, J5) = (20, 15,10, 5, 1)

and (d1, d2,d3,d4,d5) = (2, 2, 1, 3, 3).
 Greedy algorithm: Example 2
Job sequencing with deadlines
High level description of job sequencing algorithm
1. Algorithm GreedyJob(d,J,n)
// J is a set of jobs that can be completed by their
deadlines.
1. {
2. J← {1};
3. for i ← 2 to n do 4.{
5. if (all jobs in J ∪{i}can be completed
6.by their deadlines ) then J ← J ∪ {i}; 7. }
8. }
Worst case time complexity θ(n2) :
n for for loop and worst case all jobs searched in if = n
 Greedy: Example
Minimum Spanning Tree
 Greedy algorithm: example 3: Minimum spanning tree
• Let G = ( V, E) be an undirected connected graph. A
subgraph T = ( V, E') of G is a spanning tree of G iff T
is a tree.

• Minimum spanning tree is with minimum

cost
Greedy algorithm: example 3: Minimum spanning tree
• A greedy method to obtain a minimum cost spanning , the
next edge is included according to some optimization
criterion.
• The simplest such criterion would be to choose an edge
with a minimum increase in the sum of the costs of the
edges so far included.
• There are two possible ways to interpret this criterion.
• In the first, the set of edges so far selected form a tree.
Thus, if A is the set of edges selected so far, then A forms
a tree.
• The next edge (u, v) to be included in A is a minimum
cost edge not in A with the property that A U { (u, v)} is
also a tree.
• The corresponding algorithm is known as Prim's
algorithm.
Greedy algorithm: example 3: Minimum spanning tree
 Greedy algorithm: example 3: Minimum spanning
tree Prims Algorithm

1. Begin with a single vertex which will represent

the root of the tree
2. Grow this tree by finding that edge connecting
the root vertex which has the minimum weight
3. Continue to grow the tree until we have n – 1
edges

Time complexity: O(n2)

 Greedy algorithm: example 4: Minimum spanning
tree Kruskal’s Algorithm
• The edges of the graph are considered in non-
decreasing order of cost.
• the set t of edges so far selected for the spanning
tree be such that it is possible to complete into a
tree.
• Thus t may not be a tree at all stages in the
algorithm. It will generally only be a forest since
the set of edges t can be completed into a tree iff
there are no cycles in t.
• This is Kruskal’s minimum spanning tree method
 Greedy algorithm: example 4: Minimum spanning
tree Kruskal’s Algorithm
T = empty spanning tree;
E = set of edges;
N = number of nodes in graph;

while T has fewer than N - 1 edges and E ≠ ∅ do

{
remove lowest cost edge (v, w) from E
if (v, w) does not create a cycle in T
then add (v, w) to
T else ignore (v, w)
}

• Finding an edge of lowest cost can be done just by sorting

the edges
• Efficient testing for a cycle requires a fairly complex
algorithm 22
 Greedy algorithm: example 4: Minimum spanning
tree Kruskal’s Algorithm: lowest cost
• ADS: adjacency list is used to store graph
• Edges are stored in the form of linked list at
each vertex in adjacency list
• Edges are sorted usinginsertion sort on
edge arrival

23
 Greedy algorithm: example 4: Minimum Spanning
Tree(MST) Kruskal’s algorithm: cycle
• A tree is a acyclic data structure
• While buildingMST from graph, algorithm
cycle, is required to check for cycle
• At each step of Kruskal’s algorithm, composite
partial solution (V, A) is a forest
1. If nodes u and v are in the same tree, then adding
edge (u, v) to A creates a cycle
2. If nodes u and v arenotin the same tree,
then adding edge (u, v) does not create cycle

24
 Greedy algorithm: example 4: Minimum Spanning
Tree(MST) Kruskal’s algorithm:
Union/ add
• A tree is a acyclic data structure
• While buildingMST from graph, algorithm
cycle, is required to check for cycle
• At each step of Kruskal’s algorithm,
composite partial solution (V, A) is a forest
1. If nodes u and v are in the same tree, then adding
edge (u, v) to A creates a cycle
2. If nodes u and v arenotin the same tree,
then adding edge (u, v) does not create cycle
25
Thank You