SCS 300 Design and Analysis

of Computer Algorithm

Lecture 6 Greedy Technique

November 8, 2024
 Greedy Method: Definition

⚫ An algorithm which always takes the

best immediate, or local, solution while
finding an answer.
⚫ Greedy algorithms will always find the
overall, or globally, optimal solution for
some optimization problems, but may
find less-than-optimal solutions for
some instances of other problems.

Design and Analysis of Computer Algorithm November 8, 2024 2

 Greedy Algorithm Paradigm

⚫ Characteristics of greedy algorithms:

– make a sequence of choices
– each choice is the one that seems best so far, only
depends on what's been done so far
– choice produces a smaller problem to be solved
⚫ In order for greedy heuristic to solve the problem, it must
be that the optimal solution to the big problem contains
optimal solutions to subproblems

Design and Analysis of Computer Algorithm November 8, 2024 3

 Greedy Algorithm

⚫ Start with a solution to a small subproblem

⚫ Build up to a solution to the whole problem
⚫ Make choices that look good in the short term

⚫ Disadvantage: Greedy algorithms don’t always work (

Short term solutions can be diastrous in the long
term). Hard to prove correct
⚫ Advantage: Greedy algorithm work fast when they
work. Simple algorithm, easy to implement

Design and Analysis of Computer Algorithm November 8, 2024 4

 Example of Greedy Method (1/4)

⚫ Prim's algorithm and Kruskal's algorithm are

greedy algorithms which find the globally
optimal solution, a minimum spanning tree.
⚫ In contrast, any known greedy algorithm to
find an Euler cycle might not find the shortest
path, that is, a solution to the traveling
salesman problem.
⚫ Dijkstra's algorithm for finding shortest paths
is another example of a greedy algorithm
which finds an optimal solution.
Design and Analysis of Computer Algorithm November 8, 2024 5
 Greedy Method (2/4)

⚫ If there is no greedy algorithm which

always finds the optimal solution for a
problem, one may have to search
(exponentially) many possible solutions
to find the optimum.
⚫ Greedy algorithms are usually quicker,
since they don't consider possible
alternatives after the current step.

Design and Analysis of Computer Algorithm November 8, 2024 6

 Greedy Algorithm

Procedure GREEDY(A,n)
// A(1:n) contains the n inputs//
solution   //initialize the solution to empty//
for i  1 to n do
x  SELECT(A)
if FEASIBLE(solution,x)
then solution  UNION(solution,x)
endif
repeat
return(solution)
end GREEDY

Design and Analysis of Computer Algorithm November 8, 2024 7

 Element of the Greedy Strategy

Question?
⚫ How can one tell if a greedy algorithm will solve a
particular optimization problem?
⚫ No general way to tell!!!
⚫ There are 2 ingredients that are exhibited by most
problems that lend themselves to a greedy strategy
– The Greedy Choice Property
– Optimal Substructure

Design and Analysis of Computer Algorithm November 8, 2024 8

 The Greedy Choice Property

⚫ A globally optimal solution can be arrived at by

making a locally optimal (greedy) choice.
⚫ Make whatever choice seems best at the moment.
⚫ May depend on choice so far, but not depend on any
future choices or on the solutions to subproblems

Design and Analysis of Computer Algorithm November 8, 2024 9

 Optimal Substructure

⚫ An optimal solution to the problem contains within it

optimal solutions to subproblems

Design and Analysis of Computer Algorithm November 8, 2024 10

 Designing a Greedy Algorithm

⚫ Cast the problem so that we make a greedy (locally

optimal) choice and are left with one subproblem
⚫ Prove there is always a (globally) optimal solution to
the original problem that makes the greedy choice
⚫ Show that the choice together with an optimal
solution to the subproblem gives an optimal solution
to the original problem

Design and Analysis of Computer Algorithm November 8, 2024 11

 Some Greedy Algorithms

⚫ Change making
⚫ knapsack algorithm
⚫ Huffman codes
⚫ Kruskal's MST algorithm
⚫ Prim's MST algorithm
⚫ Dijkstra's SSSP algorithm
⚫ Travelling salesman
⚫ …

Design and Analysis of Computer Algorithm November 8, 2024 12

 Example of Greedy Method (3/4)

⚫ Consider the problem of making change:

⚫ Coins of values 25c, 10c, 5c and 1c
⚫ Return 63c in change
– Which coins?
⚫ Use greedy strategy:
– Select largest coin whose value was no greater
than 63c
– Subtract value (25c) from 63 getting 38
– Find largest coin … until done

Design and Analysis of Computer Algorithm November 8, 2024 13

 Example of Greedy Method (4/4)

⚫ At any individual stage, select that option which is

“locally optimal” in some particular sense
⚫ Greedy strategy for making change works because of
special property of coins
⚫ If coins were 1c, 5c and 11c and we need to make
change of 15c?
– Greedy strategy would select 11c coin followed by
4 1c coins
– Better: 3 5c coins

Design and Analysis of Computer Algorithm November 8, 2024 14

 Knapsack Problem

⚫ There are n different items in a store

⚫ Item i :
– weighs wi pounds
– worth $vi
⚫ A thief breaks in
⚫ Can carry up to W pounds in his knapsack
⚫ What should he take to maximize the value of his haul?

Design and Analysis of Computer Algorithm November 8, 2024 15

 0-1 vs. Fractional Knapsack

⚫ 0-1 Knapsack Problem:

– the items cannot be divided
– thief must take entire item or leave it behind
⚫ Fractional Knapsack Problem:
– thief can take partial items
– for instance, items are liquids or powders
– solvable with a greedy algorithm…

Design and Analysis of Computer Algorithm November 8, 2024 16

 Greedy Fractional Knapsack Algorithm

⚫ Sort items in decreasing order of value per pound

⚫ While still room in the knapsack (limit of W pounds) do
– consider next item in sorted list
– take as much as possible (all there is or as much as will
fit)
⚫ O(n log n) running time (for the sort)

Design and Analysis of Computer Algorithm November 8, 2024 17

 Knapsack Problem

⚫ We are given n objects and a knapsack. Object i has a weight wi

and the knapsack has a capacity M.
⚫ If a fraction xi, 0  xi  1, of object I is placed into the knapsack
the a profit of pixi is earned.
⚫ The objective is to obtain a filling of the knapsack that
maximizes the total weight of all chosen objects to be at most M

maximize px
1i  n
i i

subject to w x
1i  n
i i M

and 0  xi  1, 1  I  n

Design and Analysis of Computer Algorithm November 8, 2024 18

 Example

Item 3

Item 2 50
Item 1 30
20
10

$60 $100 $120 knapsack

Design and Analysis of Computer Algorithm November 8, 2024 19

 Knapsack 0/1

30 $120 $100 $120

30
20

20 $100 $60 $60

10 10

Total =$220 =$160 =$180

Design and Analysis of Computer Algorithm November 8, 2024 20

 Fractional Knapsack

⚫ Taking the items in order of greatest value per pound

yields an optimal solution

20 $80
30

20 $100

10 $60

Total =$240

Design and Analysis of Computer Algorithm November 8, 2024 21

 Optimal Substructure

⚫ Both fractional knapsack and 0/1 knapsack have an

optimal substructure.

Design and Analysis of Computer Algorithm November 8, 2024 22

 Example Fractional Knapsack
(cont.)

⚫ There are 5 objects that have a price and weight list

below, the knapsack can contain at most 100 Lbs.

Price ($US) 20 30 66 40 60
weight (Lbs.) 10 20 30 40 50

⚫ Method 1 choose the least weight first

– Total Weight = 10 + 20 + 30 + 40 = 100
– Total Price = 20 + 30 + 66 + 40 = 156

Design and Analysis of Computer Algorithm November 8, 2024 23

 Example Fractional Knapsack
(cont.)

Price ($US ) 20 30 66 40 60
weight (Lbs.) 10 20 30 40 50

⚫ Method 2 choose the most expensive first

– Total Weight = 30 + 50 + 20 = 100
– Total Price = 66 + 60 + 20 = 146

half

Design and Analysis of Computer Algorithm November 8, 2024 24

 Example Fractional Knapsack
(cont.)

Price ($US ) 20 30 66 40 60
weight (Lbs.) 10 20 30 40 50
price/weight 2 1 .5 2 .2 1 1 .2
⚫ Method 3 choose the most price/ weight first
– Total weight = 30 + 10 + 20 + 40 = 100
– Total Price = 66 + 20 + 30 + 48 = 164

80%

Design and Analysis of Computer Algorithm November 8, 2024 25

 More Example on fractional
knapsack

⚫ Consider the following instance of the knapsack

problem: n = 3, M = 20, (p1,p2,p3) = 25,24,15 and
(w1,w2,w3) = (18,15,10)

(x1,x2,x3) w x
i i px
i i

1) (1/2,1/3,1/4) 16.5 24.25

2) (1,2/15,0) 20 28.2
3) ( 0,2/3,1) 20 31
4) ( 0,1,1/2) 20 31.5

Design and Analysis of Computer Algorithm November 8, 2024 26

 The Greedy Solution

⚫ Define the density of object Ai to be wi/si. Use as

much of low density objects as possible. That is,
process each in increasing order of density. If the
whole thing ts, use all of it. If not, fill the remaining
space with a fraction of the current object,and discard
the rest.

⚫ First, sort the objects in nondecreasing density, so

that wi/si  w i+1/s i+1 for 1  i < n.

Design and Analysis of Computer Algorithm November 8, 2024 27