Dynamic programming

All-pairs shortest paths

Let G = (V, E) be a directed graph with n vertices. Let cost be a cost

adjacency matrix for G such that cost (i, i) = 0,1< i < n. Then cost(i,j) is the
length (or cost) of edge{i,j) if (i,j)G E(G) and cost(i, j)= α if i ≠ j and (i,
j)0E(G).The all-pairs shortest-path problem is to determine a matrix A such
that A(i, j)is the length of a shortest path from i to j.

Since each application of this procedure requires 0{n 2) time, the matrix A
can be obtained in 0(n3) time. We obtain an alternate o(n 3) solution to this
problem using the principle of optimality.

A(i, j)= min { min {Ak-1 (i, k) +Ak-1(k, j) },cost(i, j)}

Clearly, A0(i, j) = cost{i, j),1 < i < n, 1 < j < n. We can obtain a recurrence
for Ak(i,j) using an argument similar to that used before. A 2(l,3)≠ min{A1(l,3),
A1 (1,2)+Al(2,3)}= 2

If we want to find the shortest path not just between sand t but between all
pairs of vertices then, one approach would be to execute our general
shortest-path algorithm from |V| times, once for each starting node. The
total running time would then be O(|V|2|E|). We'll now see a better
alternative, the O(|V|3)dynamic programming- based Floyd-War shall
algorithm.

Finding a better algorithm by using dynamic programming

approach, the first question came to our mind is that, whether a better sub-
problem exists for computing distances between all pairs of vertices in a
graph? Simply solving the problem form or e and more pairs or starting
points is un helpful, because it leads right back to the O(|V|2|E|)algorithm.

Suppose we disallow inter mediate nodes altogether. Then we can solve

all- pairs shortest paths at once, the shortest path from u v is simply the
directed ge (u, v), if it exists. Now let us gradually expand the
set of permissible intermediate nodes. We can do this one node at a time,
updating the shortest path lengths at each stage. Eventually this set grows
to all of V, at which point all the vertices are allowed to be on all paths, and
we have found the true shortest path s between vertices of the graph.

More concretely, number the vertices in Vas {1, 2,3 ….,n}, and let
dist(i; j; k) denote the length of the shortest path from I to j in which only
nodes {1,2,…,k} can be used as intermediates. Initially, dist (i;j;0)is the
length of the directed ge between i and j, if it exist, and is α otherwise.

If we expand the inter mediate set to include an extra node k, we

must reexamine all pairs i,j and check whether using k as an inter
mediate point gives us a shorter pa th from I to j. But this is
easy :a shortest path from I to j that use s ka long with possibly other
lower numbered inter mediate nodes goes through k is just once. And we
have already calculated the length of the shortest path from i to k and
from k to j using only lower numbered vertices.

Fig 4.2 Computing Path

Thus, using k gives us a shorter path from i to j if and only if dist(i, k, k- )

+dist(k, j, k-1) <dist(i;j;k 1); In which case dist(i, j, k) should be updated
accordingly. Here is the Floyd-War shall algorithm – andit takes O(|V|3)time.

Algorithm All Path(cost, n)

Step 1: for i := 1 to n Step 2: for j

:=1 to n
Step 2:dist(I,j,0) := 1; Step 3: end for
Step 5: end for
Step 4: for all (i, j) ∈ E
Step 5: dist (i, j, 0) = l (i, j)
Step 6: end for Step 7: for k:=1 to n
Step 8: for i:=1 to n
Step 7: for j :=1 to n
Step 8: dist(i, j,k) = min {dist(i,k,k-1) + dist(k, j, k-1),
dist(i,j,k-1)}
Step 9: end for
Step 11: end for
Step 12: end for

Single Source Shortest Path

Problem: Given a directed graph G(V,E) with weighted edges

w(u,v), define the path weight of a path p as
 The final solution will satisfy certain caveats:
 The graph cannot contain any negative weight cycles (other wise there
would be no minimum path since we could simply continue to follow the
negative weight cycle producing a path weight of -∞).
 The solution cannot have any positive weight cycles
 The solution can be assumed to have no zero weight cycles (since they
would not affect the minimum value).

acyclic (with ≤ |V| distinct vertices) ⇒ ≤ |V| - 1 edges in each path.

There fore given these caveats, we know that the shortest paths must be

We can use this observation on the maximum number of edges on a cycle-

free shortest path to obtain an algorithm to determine a shortest path from
a source vertex to all remaining vertices in the graph.

Let distl[u] be the length of a shortest path from the source vertex v to
vertex u under the constraint that the shortest path contains at most l
edges. Then, dist1[u] = cost[v,u], 1 ≤ u ≤ n. As noted earlier, when there are
no cycles of negative length, we can limit our search for shortest paths to
paths with at most n - 1 edges. Hence, distn-1[u] is the length of an
unrestricted shortest path from v to u.

Our goal then is to compute distn-1[u] for all u. This can be done using the
dynamic programming methodology. First, we make the following
observations:

1. If the shortest path from v to u with at most k, k > 1, edges has not
more than k - 1 edges, then distk[u] - distk-1[u].

2. If the shortest path from v to u with at most k, k > 1, edges has exactly k
edges, then it is made up of a shortest path from v to some vertex j
followed by the edge (j,u). The path from v to j has k- 1 edges, and its
length is distk-1[j]. All vertices j such that the edge (j, u) is in the graph are
candidates for j. Since we are interested in a shortest path, the i that
minimizes distk-1[i] + cost[i, u] is the correct value for j.

These observations result in the following recurrence for dist: distk[u] = min
{distk-1[u], min {distk-1[i] + cost[i,u]}}. This recurrence can be used to
compute distk from distk-1, for k= 2, 3,..., n - 1.
 Algorithm BellmanFord(v, cost, dist, n)
Step 1: for i := 1 to n do Step 2:dist[i] := cost[v, i]; Step 3: end for
Step 4: for k := 2 to n - 1 do
Step 5:for each u such that u≠v and u has at least
one incoming edge
Step 6:for each <i, u> in the graph Step 7:if dist[u] >dist[i] + cost[i,u]
Step 8:dist[u] := dist[i] + cost[i,u]; Step 9:end if
Step 10:end for Step 11:end for Step 12: end for

OPTIMAL BINARY SEARCH TREE

Given a fixed set of identifiers, we wish to create a binary search tree

organization. We may expect different binary search trees for the same
identifier set to have different performance characteristic.
Example, in the case of tree, it takes1, 2, 2, 3, and 4 comparisons,
respectively, to find the identifiers {for, do. while, int, and if}. Thus the
average number of comparisons is 1+2+2+3+4/5=12/5 This calculation
assumes that each identifier is searched for with equal probability and that
no unsuccessful searches are made.
A binary search tree is a tree where the key values are stored in
the internal nodes, the external nodes (leaves) are null nodes, and the keys
are ordered lexicographically, i.e. For each internal node all the keys in the
left sub-tree are less than the keys in the node, and all the keys in the right
sub-tree are greater.

When we know the frequency of searching each one of the keys, it is quite
easy to compute the expected cost of accessing each node in the tree. An
optimal binary search tree is a binary search tree which has minimal
expected cost of locating each node. In our problem, we are not concerned
with the frequency of searching for a missing node. For example:

Node ID 0 1 2 3 4 5

Key A B C D E F
Frequenc 4 1 1 2 8 16
y
Optimal Binary search tree ex1. [ 2*1 + (1+8)*2 + (4+1+16)*3] = 83

The expected cost of successful search is 83, is computed by multiplying

each frequency by its level (starting w ith the root at 1). A different tree will
have a different expected cost:

Optimal Binary search tree ex2 [ 8*1 + (1+16)*2 + (4+2)*3 +

1 *4] = 64

It's clear that the tree in fig 4.3 is not optimal. - It is easy to see that the
nodes having higher frequencies are closer to the root, and then tree
will have a lower expected cost.

In obtaining a cost function for binary search trees, it is useful to add a

external node in place of every empty sub-tree in the search tree. If a binary
search tree represents n identifiers, then there will be exactly n internal
nodes and n+1 external nodes. If a successful search terminates at an
internal node at level l, then the expected cost contribution from the internal
node aiis p(i) * level(ai).

Unsuccessful searches terminates the external nodes, let the

unsuccessful searches terminates at node E i, if the failure node is at level l,
then only l-1 comparisons will be made, so the cost contribution of this node
is q(I) * (level(Ei) -1). The preceding decision leads to the following formula
for the expected cost of a binary search tree. wegan generalize Eq 4.5 to
obtain any c(i,j)
c(i,j) = min {p(k) + c(i, k-1) + c(k,j) + w(i, k-1) + w(k, j)} where i<k<=j

c(i,j) = min {c(i, k-1) + c(k,j)} + w(i,j) where i<k<= j

Equation 4.6 can be solved for c(0,n) by first computing all c(i,j) such
that j-i=1. Next we can compute all c(i,j) such that j-i =2, then all c(i,j) with
j-i=3, etc. if during this computation we record the root r(i,j) of each tree tij,
then an optimal binary search tree can be constructed from these r(i,j).

Example: Let n = 4 and (01,02,03,04) = (do, if, int, while).Let p(l:4) =

(3,3,1,1) and g(0 :4) = (2,3,1,1,1). 16 for convenience have multiplied the p
and g. Initially, we have w(i, i)= q( I ),c( i, I )= 0 and r(i,i)= 0,0 the
observation w(I , j)= P( j )+ q( j ) +w( i, j- 1)

w00 = W11 W22 = w33 = w44=1

2 =3 1 1 c44
coo=0 C11= c22=0 c33=0 =0
roo 0 r22=0 r33=0 r44 = 0
=0 r11 =0
W o1 w12 w23 w34
=8 =7 =3 =3
C 01= c12=7 C23=3 c34= 3
8 r12 =2 r23 =3 r34=4
R 01 =
1
w02 = W13 = w24
14 9 C13= =5
c02=2 16 c24= 8
0 r13=2 r24 = 3
r02 =
1
w03 = W14 =
14 11
c03= c14=
25 r03 19
=1 r14= 2
W04 =
16
c04=
32 ro4
=1

Therefore, c[0, 0] = 0, c[1 , 1] = 0, c[2,2] = 0, c[3,3] = 0, c[4,4] = 0

r[0, 0] = 0, r[1 , 1] = 0, r[2,2] = 0, r[3,3] = 0, r[4,4] = 0

w[0, 0] = g0, w[1 , 1] = g1, w[2,2] = g2, w[3,3] = g3, w[4,4] = g4

w(I , j)= P( j )+ q( j ) +w( i, j- 1)

step1- 1: w(I , j)= P( j )+ q( j ) +w( i, j- 1)

w(0 , 1)= P(1 )+ q( 1 ) +w( 0, 0)

=3+3+2 =8

c(i, j) = min {c(i, k-1) + c(k, j)} + w(i, j)

c (0,1) = min {c(0, 1-1) + c(1,1)} + w(i, j)

=min {c (0, 0) +c(1,1)}+w(0,1)

=min {0+0} +8 =8

R01=1

Step 1-2: w(I , j)= P( j )+ q( j ) +w( i, j- 1)

w(1, 2)= P(2 )+ q(2 ) +w( 1, 1)

=3+1+3 =7

c(i, j) = min {c(i, k-1) + c(k, j)} + w(i, j)

c(1,2) = min {c(1, 2-1) + c(2,2)} + w(1, 2)

=min {c (1, 1) +c(2,2)}+w(1,2)

=min{0+0}+7 =7

R12=2

Step1-3: w(I , j)= P( j )+ q( j ) +w( i, j- 1)

w(2 ,3)= P(3 )+ q( 3 ) +w( 2,2)

=1+1+1 =3

c(i, j) = min {c(i, k-1) + c(k, j)} + w(i, j)

c(2,3) = min {c(2, 3-1) + c(3,3)} + w(2, 3)

=min { c(2,2) +c(3,3)}+w(2,3)

=min{0+0}+3 =3

R23=3

Step1- 4: w(I , j)= P( j )+ q( j ) +w( i, j- 1)

w(3 ,4)= P(4 )+ q( 4 ) +w(3,3)

 =1+1+1 =3

c (i, j) = min {c(i, k-1) + c(k, j)} + w(i, j)

c (3, 4) = min {c(3, 4-1) + c(4,4)} + w( 3,4)

=min { c(3,3) +c(4,4)}+w(3,4)

=min{0+0}+3 =3

R23=4

Step 2-1: w(I , j)= P( j )+ q( j ) +w( i, j- 1)

w(0 ,2)= P(2 )+ q( 2 ) +w(0,1)

=3+1+8 =12

c (i, j) = min {c(i, k-1) + c(k, j)} + w(i, j)

c (0, 2) = min {c(0, 2-1)+ c(2,2)} + w( 0,2)

=min { c(0,1) + c(1,2)}+w(0,2)

=min{8+0}+ 12 =20

R23=1

Step 2-2: w(I , j)= P( j )+ q( j ) +w( i, j- 1)

w(1 ,3)= P(3 )+ q( 3 ) +w(1,2)

=1+1+7 =9

c (i, j) = min {c(i, k-1) + c(k, j)} + w(i, j)

c (1,3) = min {c(1,3-1) + c(3,3)} + w( 1, 3)

=min {c (1, 2)+c (3, 3)} +w (1, 3)

=min {7+ 0} +9 =16

R23=2

Step 2-3: w(I , j)= P( j )+ q( j ) +w( i, j- 1)

w(2 ,4)= P(4 )+ q( 4 ) +w(2,3)

=1+1+3 =5

c (i, j) = min {c(i, k-1) + c(k, j)} + w(i, j)

c (2,4) = min {c(2,3-1) + c(3,4)} + w( 2,4)

 =min {c (2, 2) +c (3, 3)} +w (2,4)

=min {0+ 3} +5 =8

R24=3

Step 3-1: w(I , j)= P( j )+ q( j ) +w( i, j- 1)

w(0 ,3)= P(3 )+ q( 3 ) +w(0,2)

=1+1+12 =14

c (i, j) = min {c(i, k-1) + c(k, j)} + w(i, j)

c (0,3) = min {c(0,3-1) + c(3,3)} + w( 0, 3)

=min {c (0, 1) +c (2, 3)} +w (0, 3)

=min {8+ 3} +14 =25

R03=1

Step 3-2: w(I , j)= P( j )+ q( j ) +w( i, j- 1)

w(1 ,4)= P(4 )+ q(4 ) +w(1,3)

=1+1+9 =11

c (i, j) = min {c(i, k-1) + c(k, j)} + w(i, j)

c (1,4) = min {c(1,2-1) + c(2,4)} + w(1, 4)

=min {c (1, 1) +c (2, 4)} +w (1, 4)

=min {0+ 8} +11 = 19

R14=2

Step 4-1: w(I , j)= P( j )+ q( j ) +w( i, j- 1)

w(0 ,4)= P(4 )+ q(4 ) +w(0,3)

=1+1+14 =16

c (i, j) = min {c(i, k-1) + c(k, j)} + w(i, j)

c (0,4) = min {c(0,2-1) + c(2,4)} + w(0, 4)

=min {c (0, 1) +c (2, 4)} +w (0, 4)

=min {8+ 8} +16 = 32

 R04=1

Algorithm

Algorithm OBST(p, q, n)

for i :=0 to n

w[i,i]:=q[i];r[i,i]:=0;c[i,i]:=0.0;

w[i,i+ i\\:=q[i]+q[i +l]+p[i+ l]i

r[i,i+l]:=i+ l;

c[I, i+ l] :=q[i]+q[i + 1]+p[i +1];

w[n,n] :=q[n];r[n,n\\ :=0;c[n,n]:=0.0;

for m :=2 to n do

for i :=0 to n

j :=i +m;

w[i, j] :=w[i, j - 1]+p[j]+q

k :=Find(c, r, i, j);

Algorithm Find(c, r,, j)

2 ^ 3 min: = oo;
for m :=r[i, j-1] to r[i+ 1, j] do
if (c[i, m-1]+c[m, j])
}

0/1KNAPSACK

A solution to the knapsack problem can be obtained by making a sequence

of decisions on the variables x1, X2, ……….., xn. A decision on variable Xi
involves determining which of the values 0 or 1 is to be assigned to it.

The capacity remaining in the knapsack is m and no profit has accrued or the
capacity remaining is m- wn and a profit of pn has accrued. It is clear that the
remaining decisions Xn,..., X must be optimal with respect to the problem
state resulting from the decision on Xn

Fi(y)= max{fi-1((y), fi-1 (y-wi)+ pi}

Can be solved for fn(m) by beginning with the knowledge fo(y) = 0 for ally and
fi(y) = -α, y<0. Then f1,f2,..., fn can be successively computed using.

We use the ordered set Si={(f(yj), yj)|1<j<k} to represent fi(y). Each member
of Si is a pair (P, W), where P = fi( yj) and W = yj. Notice that S0 = {(0, 0)}.
We can computeSl+1from Sl by first computing.

Si = {(P, W) (P-Pi, W - Wi) € Si}

Consider the knapsack in stance n = 3, (W1, W2, W3) = (2, 3, 4), (P1,P2, P3) =
(1, 2, 5), and m = 6. For these data we have

S0 = {(0,0)}; S1 0={(1,2)}

S1 = {(0,0),(1,2)}; S11={(2,3),(3,5)}

S2 = {(0,0),(1,2),(2,3),(3,5)}; S21 ={(5,4),(6,6),(7,7),(8,9)}

S3 = {(0,0),(1,2),(2,3),(5,4),(6,6),(7,7),(8,9)

Consider the following instance of the knapsack problem: n = 6,

(P1,P2,P3,P4,P5,P6) = {W1,W2,W3,W4,W5,W6} = (100,50,20,10,7, 3), and m
= 165. Attempting to fill the knapsack using objects in the order 1,2,3, 4,
5,and 6,we see that objects1,2, 4, and 6 fit in and yield a profit of 163and a
capacity utilization of 163.We can thus begin with L = 163 as a value with
the property L < i < L,

PLEFT(O)= 190,

PLEFT(l)= 90,

PLEFT(2)= 40,

PLEFT(3)= 20,

PLEFT(4)= 10,

PLEFT(5)= 3,and PLEFT(6)= 0.Eliminatingfrom each Sl any single to nP such

that P+ PLEFT(i)< L, we obtain

S0= {0};S01 = {100}

S1= {100}; S11= {150}

S2= {150}; S21= Ɵ

S3= {150}; S31={160}

S4 = {160};S41 =Ɵ

S5= {160}

If the heuristic were not used, then the computation would have proceeded
as

S0 = {0}

S1 = {0,100}

S2 = {0,50,100,150}

S3 = {0,20,50,70,100,120,150}

S4 = {0,10,20,30,50,60,70,80,100,110,120,130,150,160}

S5 = {0,7,10,17,20,27,30,37,50,57,60,67,70,77,80,87,100,
107,110,117,120,127,130,137,150,157.160}

The value fe(165)can now be determined from S5, using the knowledge
(P6,W6)=(3,3).

TRAVELLING SALESMAN PROBLEM

We have seen how to apply dynamic programming to a subset selection problem

(0/1 knapsack). Now we turn our attention to a permutation problem. Note that
permutation problems usually are much harder to solve than subset problems, as
there are n! Different permutations of n objects where as there are only
2ndifferent subsets of n objects (n!> 2n). Let G = (V,E)be a directed graph with
edge costs cij. The variable cijis defined such that cij> 0 for all i and j

and cij = α if (i,j) ∉ E. Let |V| = n and assume n > 1. A tour of G is a directed simple
cycle that includes every vertex in V. The cost of a tour is the sum of the cost of
the edges on the tour. The traveling salesperson problem is to find a tour of
minimum cost.

Different problems can be viewed as the traveling salesman problem. For

example, suppose we have to define the route a postal van to pick up mail from
mail boxes located at n different sites. If we represent the situation by graphs then
the vertices of the graph will be different cities and the edges of the graph are the
paths between two cities and the weight of a edge can be the distance between
the cities. Our task is to find the route taken by the postal van is a tour with
minimum cost or length.

In the following discussion, without losing the main concept, we take the

consists of an edge (1,k) for some k ∈ V - {1} and a path from vertex k to vertex
tour as a simple path that starts and ends at the starting vertex. Every tour

1. The path from vertex k to

vertex 1 goes through each vertex in V - {1, k} exactly once. It is easy to see that
if the tour is optimal, then the path from k to 1 must be a shortest k to 1 path
going through all vertices in V -

{1,k}. Hence, the principle of optimality holds. Let g(i,S) be the length of a shortest
path starting at vertex i, going through all vertices in S, and terminating at vertex 1.
The function g(1, V - {1}) is the length of an optimal salesman’s tour. From the
principal of optimality it follows that

g(1, V-{1}) = min {c1k + g(k, V - {1, k})}

2kn

In general

min {cij + g( j, S - {j})}

g( i, S ) =
jS

The above equation can be solved for g(1, V - {1}) if we know g(k, V - {1, k}) for
all choices of k. The g values can be obtained by using this equation. Clearly, g(i,
Ø) = cj1, 1≤ i ≤ n. Hence, we can use this equation to obtain g(i, S) for all S of size
1. Then we can obtain g(i,S) for S with |S| = 2, and so on. When |S|< n - 1, the
values of i and S for which g(i, S) is needed are such that i ≠ 1, 1∉ S, and i ∉S.
Consider the directed graph of Fig 4.5(a). The edge lengths are given by matrix
c of

Fig 4.5: Directed graph and Edge matrix c Thus,

g(2,Ø) = c21 = 5 g(3,Ø) = c31 = 6 g(4,Ø) = c41 = 8.

Using the above equation we obtain

g(2,{3}) = c23+ g(3,Ø) = 15 g(2,{4}) = 18

g(3,{2}) = 18 g(3,{4}) = 20

g(4,{2}) = 13 g(4,{3}) = 15

Next, we compute g(i, S) with |S| =2, i ≠ 1, 1∉ S and i ∉ S. g(2,{3,4}) = min

{c23+g(3,{4}),c24+g(4,{3})} = 25

g(3,{2,4}) = min {c32+g(2,{4}),c34+g(4,{2})} = 25

g(4,{2,3}) = min {c42 + g(2,{3}),c43+ g(3,{2})} = 23

Finally, we obtain

g(1, {2,3,4}) = min {c12+g(2, {3,4}), c13+g(3, {2,4}), c14+g(4, {2, 3})}

= min {35,40,43}

= 35

An optimal tour of the graph of Figure has length 35. A tour of this length can be
constructed if we retain with each g(i, S) the value of j that minimizes the right-
hand side of the graph. Let J(i,S) be this value. Then, J(1,{2,3,4}) = 2. Thus the
tour starts from 1 and goes to 2. The remaining tour can be obtained from g(2,
{3, 4}). SoJ(2, {3, 4}) = 4. Thus the next edge is (2,4). The remaining tour is for
g(4, {3}). So J(4, {3}) = 3. The
TRAVELING SALESPERSON PROBLEM

One vertex represents the post office from which the postal van starts and to
which it must return. Edge (i, j) is assigned a cost equal to the distance from
site i to site j. The route taken by the postal van is a tour, and we are
interested in finding a tour of minimum length.

The manufacture proceeds in cycles. In each production cycle, n different

commodities are produced. When the machines are changed from
production of commodity j to commodity j, a changeover cost C I j is incurred

g(1, V-{1})=min {Clk + g(k, V -{1,k})

a b

c d

a->b=10, b->a= 5; b->d=10, d->b=8; b->c=9, c->b=13

a->c=15, c->a=6; a->d=20, d->a=8;

Thus g(2,Ɵ)= C21 = 5,

g(3,Ɵ)= C31 = 6, and

g(4,Ɵ) = c41 = 8. Using we obtain

g(2,{3})= c23+g(3,Ɵ)= 15 g(2,{4})= 18 g(3,{2})= 18 g(3,{4})= 20 g(4,{2})= 13 g(4,

{3})= 15

Next, we compute g(i,S) with |S| =2, i≠1, 1≠ S and i €S.

g(2,{3,4})= min {c23+g(3,{4}),C24+g(4,{3})} = 25

g(3,{2,4})= min {c32+g(2,{4}),C34+g(4,{2})} = 25

g(4,{2,3})= min {c42+g(2,{3}),c43+g(3,{2})} = 23

Finally, from we obtain

g(1,{2, 3, 4})=min {c12+g(2,{3,4}), C13+g(3,{2,4}), C14+g(4,{2, 3})}

= min {35,40, 43}

=min 35