Dynamic Programming

Fibonacchi Numbers
𝑓𝑖𝑏 𝑛 = 𝑓𝑖𝑏 𝑛 − 1 + 𝑓𝑖𝑏 𝑛 − 2
• Write a program to find the nth fibonacchi number.
def Fibonacci(n):
if n == 0: return 0
if n == 1: return 1
return Fibonacci(n-1) + Fibonacci(n-2)

• Runtime analysis
𝑇 𝑛 = 𝑇 𝑛 − 1 + 𝑇 𝑛 − 2 + 𝑂(1)
𝑇 𝑛 = Ω(2𝑛/2 )
What’s going on? That’s a lot of
repeated
Consider Fib(8) computation!
8

6 7

4 5 5 6

2 3 3 4 3 4 4 5

0 1 1 2 1 2 2 3 1 2 2 3 2 3 3 4

0 1
1 2 1 2 ……
0 1 0 1 1 2 0 1 0 1 0 1 1 2
0 1 0 1 0 1 0 1
Fibonacchi Numbers
• How to avoid repeated computations?
• Store already computed results

global fib[n] initialized to -1/−∞/NIL

def Fibonacci(n):
if n == 0: return 0
if n == 1: return 1
if fib[n] not computed:
fib[n] = Fibonacci(n-1) + Fibonacci(n-2)
return fib[n]
Fibonacchi Numbers
• How to avoid repeated computations?
• Start from smaller problems and proceed towards bigger ones.

def Fibonacci(n):
local fib[n+1]
fib[0] = 0
fib[1] = 1
for i -> 2 to n:
fib[i] = fib[i-1] + fib[i-2]
return fib[n]
What did we do?

Dynamic Programming!!!
Dynamic Programming
• It is an algorithm design paradigm
• like divide-and-conquer is an algorithm design paradigm.
• Usually, it is for solving optimization problems
• E.g., shortest path, minimum/maximum profit, longest sequences
• (Fibonacci numbers aren’t an optimization problem, but they are a good
example of DP anyway…)
Elements of dynamic programming
1. Optimal Sub-structure Property
• Big problems break up into sub-problems.
• The solution to a problem can be expressed in terms of solutions to
smaller sub-problems.
• Fibonacci:
𝑓𝑖𝑏 𝑛 = 𝑓𝑖𝑏 𝑛 − 1 + 𝑓𝑖𝑏(𝑛 − 2)
Elements of dynamic programming
2. Overlapping Sub-Problem Property
• The sub-problems overlap.
• Fibonacci:
• Both fib[i+1] and fib[i+2] directly use fib[i].
• And lots of different fib[i+x] indirectly use fib[i].
• This means that we can save time by solving a sub-problem just once
and storing the answer.
Elements of dynamic programming
1. Optimal substructure.
• Optimal solutions to sub-problems can be used to find the optimal
solution of the original problem.
2. Overlapping subproblems.
• The subproblems show up again and again

• Using these properties, we can design a dynamic programming algorithm:

• Keep a table of solutions to the smaller problems.
• Use the solutions in the table to solve bigger problems.
• At the end we can use information we collected along the way to find
the solution to the whole thing.
Two ways to think about and/or implement
DP algorithms
• Top-down Approach with • Bottom-up Approach
Memoization
global fib[n] initialized to - def Fibonacci(n):
1/−∞/NIL local fib[n+1]
def Fibonacci(n): fib[0] = 0
if n == 0: return 0 fib[1] = 1
if n == 1: return 1 for i -> 2 to n:
if fib[n] not computed: fib[i] = Fibonacci(i-1) +
fib[n] = Fibonacci(n-1) + Fibonacci(i-2)
Fibonacci(n-2) return fib[n]
return fib[n]
Top-down Approach
• Think of it like a recursive algorithm.
• To solve the big problem:
• Recurse to solve smaller problems
• Those recurse to solve smaller problems
• etc..

• The difference from divide and conquer:

• Keep track of what small problems you’ve already solved to prevent re-
solving the same problem twice.
• Aka, “memo-ization”
Top-down Approach
8

6 7

4 5 5 6

2 3 3 4 3 4 4 5

0 1 1 2 1 2 2 3 1 2 2 3 2 3 3 4

1 2 etc
0 1 1 2
0 1 0 1 1 2 0 1 0 1 0 1 1 2
0 1 0 1 0 1 0 1
13
Top-down Approach
Nodes Pruned
8

6 7

4 5 5 6

2 3 3 4 3 4 4 5

0 1 1 2 1 2 2 3 1 2 2 3 2 3 3 4

1 2 etc
0 1 1 2
0 1 0 1 1 2 0 1 0 1 0 1 1 2
0 1 0 1 0 1 0 1
14
Bottom-up Approach 8

7
• Solve the small problems first
• fill in fib[0],fib[1]
6
• Then bigger problems
• fill in fib[2] 5
•…
4
• Then bigger problems
• fill in fib[n-1] 3

• Then finally solve the real problem. 2

• fill in fib[n] 1

0
Rod-cutting Problem
• Given
• A rod of length n
• A table of prices 𝑝𝑖 for 𝑖 = 1, 2, … , 𝑛,
• Determine the maximum revenue 𝑟𝑛 obtainable by cutting up the rod
and selling all the pieces.
• For example,
Rod-cutting Problem
Rod-cutting Problem
• An optimal solution involving k pieces,
• Each piece has length 𝑖1 , 𝑖2 , 𝑖3 , … , 𝑖𝑘
• 𝑛 = 𝑖1 + 𝑖2 + 𝑖3 + ⋯ + 𝑖𝑘
• The optimal revenue, 𝑟𝑛 = 𝑝𝑖1 + 𝑝𝑖2 + ⋯ + 𝑝𝑖𝑘
Rod-cutting Problem
• Once an initial cut is made,
• The two resulting smaller pieces will be cut independently
• Smaller instance of the rod-cutting problem
• Optimal Sub-structure Property

• Different pieces can be cut into same length pieces (on not)
• Overlapping Sub-structure Property
Rod-cutting Problem
• Assuming an initial cut is made,
• 𝑟𝑛 = max(𝑝𝑛 , 𝑟1 + 𝑟𝑛−1 , 𝑟2 + 𝑟𝑛−2 , … 𝑟𝑛−1 + 𝑟1 )

• Further assuming the initial cut is always the leftmost cut,

• A first piece followed by some decomposition of the remainder
• First piece is not further divided only the remaining is decomposed
• 𝑟𝑛 = max(𝑝𝑖 + 𝑟𝑛−𝑖 : 1 ≤ 𝑖 ≤ 𝑛)
Rod-cutting Problem

Runtime O(2𝑛−1 )
Rod-cutting Problem (Top-down Approach)
Rod-cutting Problem (Bottom-up Approach)

Runtime 𝑂(𝑛2 )
Rod-cutting Problem (Bottom-up Approach)
• Reconstruct the choices that led to the optimal solution

• Store the choices that lead to the optimal solution

• Store the optimal size i of the first piece to cut off when solving a
subproblem of size j
Rod-cutting Problem (Bottom-up Approach)
Rod-cutting Problem (Bottom-up Approach)
Matrix Chain Multiplication
• Given a chain of n matrices, 𝐴1 , 𝐴2 , … , 𝐴𝑛
• Compute the product with standard multiplication algorithm
• Goal: Minimize the number of scalar multiplications

• Example,
• 𝐴1 , 𝐴2 , 𝐴3 with dimensions 10 * 100, 100 * 5, 5 * 50
• ((𝐴1 , 𝐴2 ), 𝐴3 ) perfoms a total of 7500 scalar multiplication
• (𝐴1 , (𝐴2 , 𝐴3 )) perfoms a total of 75000 scalar multiplication
Matrix Chain Multiplication
• Parenthesizing resolves ambiguity in multiplication order
• Fully parenthesized chain of matrices
• Either a single matrix
• Or the product of two fully parenthesized matrix products, surrounded by
parentheses
• Example,
• < 𝐴1 , 𝐴2 , 𝐴3 , 𝐴4 > can be parenthesized in 5 distinct ways.
• (𝐴1 , (𝐴2 , (𝐴3 , 𝐴4 ))), (𝐴1 , ((𝐴2 , 𝐴3 ), 𝐴4 )), ((𝐴1 , 𝐴2 ), (𝐴3 , 𝐴4 )),
((𝐴1 , (𝐴2 , 𝐴3 )), 𝐴4 ), (((𝐴1 , 𝐴2 ), 𝐴3 ), 𝐴4 )
Matrix Chain Multiplication
• Given a chain of n matrices, 𝐴1 , 𝐴2 , … , 𝐴𝑛
• Matrix 𝐴𝑖 has dimensions 𝑝𝑖−1 ∗ 𝑝𝑖
• Goal: Fully parenthesize the product to minimize the number of scalar
multiplications

• Note: determine the order of multiplication not the product itself.

Matrix Chain Multiplication
• Fully parenthesized product
• Split the product into fully parenthesized subproducts

• Let, the first split occurs between kth and (k+1)st matrices

• How many possible ways? Ω(2𝑛 )

Matrix Chain Multiplication
• Let, 𝑚[𝑖, 𝑗] = The minimum number of scalar multiplications needed to
compute the matrix 𝐴𝑖:𝑗

• We need to find 𝑚[1, 𝑛]

Matrix Chain Multiplication
• The optimal parenthesization
• Split the product 𝐴𝑖:𝑗 between 𝐴𝑘 and 𝐴𝑘+1 for some value of 𝑖 ≤ 𝑘 < 𝑗

𝑚 𝑖, 𝑗 = 𝑚 𝑖, 𝑘 + 𝑚 𝑘 + 1, 𝑗 + 𝑝𝑖−1 ∗ 𝑝𝑘 ∗ 𝑝𝑗
• We need to consider all such splits, i.e., all values of k
Matrix Chain Multiplication
• The optimal parenthesization
• Split the product 𝐴𝑖:𝑗 between 𝐴𝑘 and 𝐴𝑘+1 for some value of 𝑖 ≤ 𝑘 < 𝑗

𝑚 𝑖, 𝑗 = 𝑚 𝑖, 𝑘 + 𝑚 𝑘, +1 𝑗 + 𝑝𝑖−1 ∗ 𝑝𝑘 ∗ 𝑝𝑗
• We need to consider all such splits, i.e., all values of k

• How many such 𝐴𝑖:𝑗 subproblems? Θ(𝑛2 )

Matrix Chain Multiplication
Matrix Chain Multiplication
 Matrix Chain Multiplication
𝐴1 𝐴2 𝐴3 𝐴4 𝐴5 𝐴6
30 * 35 35 * 15 15 * 5 5 *10 10 * 20 20 * 25
j\i 1 2 3 4 5 6
1 0

2 15750 0 m[1, 3] = min(

• m[1,1] + m[2, 3] + 30*35*5 = 7875,
3 7875 2625 0 • m[1,2] + m[3, 3] + 30*15*5 = 18000,
)
4 4375 750 0

5 2500 1000 0

6 3500 5000 0
 Matrix Chain Multiplication
𝐴1 𝐴2 𝐴3 𝐴4 𝐴5 𝐴6
30 * 35 35 * 15 15 * 5 5 *10 10 * 20 20 * 25
j\i 1 2 3 4 5 6
1 0

2 15750 0 m[2, 5] = min(

• m[2,2] + m[3, 5] + 35*15*20 = 13000,
3 7875 2625 0 • m[2,3] + m[4, 5] + 35*5*20 = 7125,
• m[2,4] + m[5, 5] + 35*10*20 = 11375
)
4 9375 4375 750 0

5 11875 7125 2500 1000 0

6 15125 10500 5375 3500 5000 0

Elements of dynamic programming (Revisit)
1. Optimal substructure.
• Optimal solutions to sub-problems can be used to find the optimal
solution of the original problem.

2. Overlapping subproblems.
• The subproblems show up again and again
Optimal Substructure Property
• Solution to sub-problems are included in the optimal solution

• Rod-cutting
• Solution to smaller pieces are also part of the solution to the entire rod

• Matrix Chain Multiplication

• Solution to 𝐴𝑖:𝑘 and 𝐴𝑘+1:𝑗 is exactly include in the solution to 𝐴𝑖:𝑗

• How to prove this?

• Cut-and-paste
Longest Common Subsequence
• A strand of DNA consists of a string of molecules called bases
• Adenine, Cytosine, Guanine, and Thymine
• ACGT

• S1 = ACCGGTCGAGTGCGCGGAAGCCGGCCGAA
• S2 = GTCGTTCGGAATGCCGTTGCTCTGTAAA
Longest Common Subsequence
• A strand of DNA consists of a string of molecules called bases
• Adenine, Cytosine, Guanine, and Thymine
• ACGT

• Given a sequence 𝑋 = < 𝑥1 , 𝑥2 , … , 𝑥𝑛 >

• Another sequence 𝑍 = < 𝑧1 , 𝑧2 , … , 𝑧𝑛 > is a subsequence of X
• If there exists a strictly increasing sequence < 𝑖1 , 𝑖2 , … , 𝑖𝑛 > indices of X such
that for all 𝑗 = 1, 2, … , 𝑘, 𝑥𝑖𝑗 = 𝑧𝑗
Longest Common Subsequence
• A strand of DNA consists of a string of molecules called bases
• Adenine, Cytosine, Guanine, and Thymine
• ACGT

• Given two sequences 𝑋 and 𝑌

• A sequence Z is a common sub-sequence if Z is a subsequence of both X and Y

• Goal: find a maximum-length common subsequence of X and Y.

Longest Common Subsequence
• Need to consider all subsequences

• How many subsequences of X?

• 2𝑛
Longest Common Subsequence
• Given a sequence 𝑋 = < 𝑥1 , 𝑥2 , … , 𝑥𝑛 >
• The 𝑖𝑡ℎ prefix of 𝑋 is 𝑋𝑖 = < 𝑥1 , 𝑥2 , … , 𝑥𝑖 >
Longest Common Subsequence
• Let 𝑐[𝑖, 𝑗] = the length of an LCS of the sequences 𝑋𝑖 and 𝑌𝑗
Longest Common Subsequence
Longest Common Subsequence
j 0 1 2 3 4 5 6
i B D C A B A
0 0 0 0 0 0 0 0
1 A 0 0 0 0 1
2 B 0
3 C 0
4 B 0
5 D 0
6 A 0
7 B 0
Longest Common Subsequence
j 0 1 2 3 4 5 6
i B D C A B A
0 0 0 0 0 0 0 0
1 A 0 0 0 0 1 1 1
2 B 0 1 1 1 1 2 2
3 C 0 1 1 2 2 2 2
4 B 0 1 1 2 2 3 3
5 D 0 1 2 2 2 3 3
6 A 0 1 2 2 3 3 4
7 B 0 1 2 2 3 4 4
Sequence Alignment
• How similar are two strings?
• ocurrance vs occurrence
Sequence Alignment
• Edit distance
• Gap penalty and mismatch penalty
• Cost is sum of all penalties
Sequence Alignment
• Given two sequences
• 𝑥1 𝑥2 … 𝑥𝑛 and 𝑦1 𝑦2 … 𝑦𝑛
• An alignment is a set of ordered pairs (𝑥𝑖 , 𝑦𝑗 ) such that each character
appears in at most one pair and there are no crossings.
• Crossing: (𝑥𝑖 , 𝑦𝑗 ) and (𝑥𝑖′ , 𝑦𝑗′ ) cross if 𝑖 < 𝑖′ but 𝑗 > 𝑗′
Sequence Alignment
• Given two sequences
• 𝑥1 𝑥2 … 𝑥𝑛 and 𝑦1 𝑦2 … 𝑦𝑛

• The cost of an alignment M is,

𝑐𝑜𝑠𝑡 𝑀
= ෍ 𝑐𝑜𝑠𝑡𝑚𝑖𝑠𝑚𝑎𝑡𝑐ℎ + ෍ 𝑐𝑜𝑠𝑡𝑔𝑎𝑝 + ෍ 𝑐𝑜𝑠𝑡𝑔𝑎𝑝
𝑥𝑖 ,𝑦𝑗 ∈𝑀 𝑥𝑖 𝑦𝑗
𝑢𝑛𝑚𝑎𝑡𝑐ℎ𝑒𝑑 𝑢𝑛𝑚𝑎𝑡𝑐ℎ𝑒𝑑
Sequence Alignment
• Given two sequences
• 𝑥1 𝑥2 … 𝑥𝑛 and 𝑦1 𝑦2 … 𝑦𝑛

• Goal: Find minimum cost alignment of the two sequences

Sequence Alignment
• Given two sequences
• 𝑥1 𝑥2 … 𝑥𝑛 and 𝑦1 𝑦2 … 𝑦𝑛

• 𝑂𝑃𝑇 𝑖, 𝑗 = minimum cost of aligning 𝑥1 𝑥2 … 𝑥𝑖 and 𝑦1 𝑦2 … 𝑦𝑗

Sequence Alignment
• 𝑂𝑃𝑇 𝑖, 𝑗 = minimum cost of aligning 𝑥1 𝑥2 … 𝑥𝑖 and 𝑦1 𝑦2 … 𝑦𝑗
• Case 1: 𝑂𝑃𝑇 𝑖, 𝑗 matches (𝑥𝑖 , 𝑦𝑗 )
• Pay mismatch for (𝑥𝑖 , 𝑦𝑗 ) + min cost of aligning 𝑥1 𝑥2 … 𝑥𝑖−1 and 𝑦1 𝑦2 … 𝑦𝑗−1
• 𝑐𝑜𝑠𝑡𝑚𝑖𝑠𝑚𝑎𝑡𝑐ℎ + 𝑂𝑃𝑇 𝑖 − 1, 𝑗 − 1
Sequence Alignment
• 𝑂𝑃𝑇 𝑖, 𝑗 = minimum cost of aligning 𝑥1 𝑥2 … 𝑥𝑖 and 𝑦1 𝑦2 … 𝑦𝑗
• Case 2: 𝑂𝑃𝑇 𝑖, 𝑗 leaves 𝑥𝑖 unmatched
• Pay gap for 𝑥𝑖 + min cost of aligning 𝑥1 𝑥2 … 𝑥𝑖−1 and 𝑦1 𝑦2 … 𝑦𝑗
• 𝑐𝑜𝑠𝑡𝑔𝑎𝑝 + 𝑂𝑃𝑇 𝑖 − 1, 𝑗
Sequence Alignment
• 𝑂𝑃𝑇 𝑖, 𝑗 = minimum cost of aligning 𝑥1 𝑥2 … 𝑥𝑖 and 𝑦1 𝑦2 … 𝑦𝑗
• Case 3: 𝑂𝑃𝑇 𝑖, 𝑗 leaves 𝑦𝑗 unmatched
• Pay gap for 𝑦𝑗 + min cost of aligning 𝑥1 𝑥2 … 𝑥𝑖 and 𝑦1 𝑦2 … 𝑦𝑗−1
• 𝑐𝑜𝑠𝑡𝑚𝑖𝑠𝑚𝑎𝑡𝑐ℎ + 𝑂𝑃𝑇 𝑖, 𝑗 − 1
Sequence Alignment
• 𝑂𝑃𝑇 𝑖, 𝑗 = minimum cost of aligning 𝑥1 𝑥2 … 𝑥𝑖 and 𝑦1 𝑦2 … 𝑦𝑗
• Case 1: 𝑂𝑃𝑇 𝑖, 𝑗 matches (𝑥𝑖 , 𝑦𝑗 )
• 𝑐𝑜𝑠𝑡𝑚𝑖𝑠𝑚𝑎𝑡𝑐ℎ + 𝑂𝑃𝑇 𝑖 − 1, 𝑗 − 1

• Case 2: 𝑂𝑃𝑇 𝑖, 𝑗 leaves 𝑥𝑖 unmatched

• 𝑐𝑜𝑠𝑡𝑚𝑖𝑠𝑚𝑎𝑡𝑐ℎ + 𝑂𝑃𝑇 𝑖 − 1, 𝑗

• Case 3: 𝑂𝑃𝑇 𝑖, 𝑗 leaves 𝑦𝑗 unmatched

• 𝑐𝑜𝑠𝑡𝑚𝑖𝑠𝑚𝑎𝑡𝑐ℎ + 𝑂𝑃𝑇 𝑖, 𝑗 − 1
Sequence Alignment
• 𝑂𝑃𝑇 𝑖, 𝑗 = minimum cost of aligning 𝑥1 𝑥2 … 𝑥𝑖 and 𝑦1 𝑦2 … 𝑦𝑗
Sequence Alignment
0-1 Knapsack Problem
• Given,
• n items where item 𝑖 has value
𝑣𝑖 > 0 and weighs 𝑤𝑖 > 0
• Value of a subset of items = sum of
values of individual items.
• Knapsack has weight limit of 𝑊
• Goal. Pack knapsack so as to
maximize total value of items
taken.
• Example, {1, 2, 5} yields 35$ while
{3, 4} yields 40$.
0-1 Knapsack Problem
• 𝑂𝑃𝑇(𝑖, 𝑤) = optimal value of knapsack problem with items 1, … , 𝑖,
subject to weight limit 𝑤
• Goal: Find 𝑂𝑃𝑇(𝑛, 𝑊)
0-1 Knapsack Problem
• 𝑂𝑃𝑇(𝑖, 𝑤) = optimal value of knapsack problem with items 1, … , 𝑖,
subject to weight limit 𝑤

• Case 1: Don’t pick item 𝑖

• 𝑂𝑃𝑇(𝑖, 𝑤) selects best of {1, 2, … , 𝑖 – 1} subject to weight limit 𝑤

• Case 2: Pick item 𝑖

• 𝑂𝑃𝑇(𝑖, 𝑤) selects best of {1, 2, … , 𝑖 – 1} subject to weight limit 𝑤 − 𝑤𝑖
• Collect value 𝑣𝑖
0-1 Knapsack Problem
• 𝑂𝑃𝑇(𝑖, 𝑤) = optimal value of knapsack problem with items 1, … , 𝑖,
subject to weight limit 𝑤

• Case 1: Don’t pick item 𝑖

• 𝑂𝑃𝑇 𝑖, 𝑤 = 𝑂𝑃𝑇(𝑖 − 1, 𝑤)

• Case 2: Pick item 𝑖

• 𝑂𝑃𝑇 𝑖, 𝑤 = 𝑣𝑖 + 𝑂𝑃𝑇(𝑖 − 1, 𝑤)
0-1 Knapsack Problem
• 𝑂𝑃𝑇(𝑖, 𝑤) = optimal value of knapsack problem with items 1, … , 𝑖,
subject to weight limit 𝑤
0-1 Knapsack Problem
0-1 Knapsack Problem
Reference
• Dynamic Programming
• CLRS 4th Ed. Chapter 14 (14.1 – 14.4)
• KT Sections 6.4 (The Knapsack Problem), 6.6