Combinepdf

UNIT-III
All-Pairs Shortest Paths problem: -

Given a graph G, we have to find a shortest path between every pair of vertices. That is, for
every pair of vertices (i, j), we have to find a shortest from i to j.
Let G = (V, E) be a weighted graph with n vertices. Let c be the cost adjacency matrix for G
such that c[i, i]=0, 1≤i≤n and c[i, j]=∞ if i ≠ j and <i, j> ∉ E(G).
When no edge has a negative length, the All-Pairs Shortest Paths problem may be solved by
applying Dijkstra’s greedy Single-Source Shortest Paths algorithm n times, once with each of the n
vertices as the source vertex. This process results in O(n3) solution to the All-Pairs problem.
We will develop a dynamic programming solution called Floyd’s algorithm which also runs in
O(n3) time but works even when the graph has negative length edges (provided there are no negative
length cycles).
We need to determine a matrix A[1:n,1:n] such that A[i, j] is the length of the shortest path
from vertex i to vertex j.
The matrix A is initialized as A0 [i, j] = c[i, j], 1 ≤ i ≤n, 1 ≤ j ≤ n. The algorithm makes n passes over
A. In each pass, A is transformed. Let A1, A2, A3, …, An be the transformations of A on the n passes.
Let Ak[i, j] denote the length of the shortest path from i to j that has no intermediate vertex
larger than k. That means, the path possibly passes through the vertices {1, 2, 3, …, k}, but not
through the vertices {k+1, k+2, …, n}.
So, An[i, j] gives the length of the shortest path from i to j because all intermediate vertices {1, 2, 3,
…, n} are considered.
How to determine Ak[i, j] for any k≥1?
A shortest path from i to j going through no vertex higher than k may or mayn’t go through k.
If the path goes through k, then Ak[i, j]=Ak-1[i, k]+Ak-1[k, j], following the principle of optimality.
If the path doesn’t go through k, then no intermediate vertex has index greater than (k-1).
Hence Ak[i, j]=Ak-1[i, j].
By combining both cases, we get
Ak[i, j]=min{ Ak-1[i, j], Ak-1[i, k]+Ak-1[k, j]}, k≥1.
This recurrence can be solved for An by first computing A1, then A2, then A3, and so on. In the
algorithm, the same matrix A is transformed over n passes and so the superscript on A is not needed.
The kth pass explores whether the vertex k lies on an optimal path from i to j, for all i and j. We
assume that the shortest path (i, j) contains no cycles.
Algorithm Allpaths(c, A, n)
{
//c[1:n,1:n] is the cost adjacency matrix of a graph with n vertices.
//A[i, j] is length of a shortest path from vertex i to vertex j.
//c[i, i]=0, for 1≤i≤n.
for i:=1 to n do
for j:=1 to n do
A[i, j]=c[i, j]; //copy c into A
for k:=1 to n do
for i=1 to n do
for j=1 to n do
A[i, j]:=min(A[i, j], A[i, k]+A[k, j]);
}
Time complexity, T(n)=O(n3).
EXAMPLE: - Find out shortest paths between all pairs of vertices in the following digraph.
Step-1: A0 ← c Step-2: Using, A1[i, j] = min{A0[i, j], A0[i, 1]+A0[1, j]}

A1
0 4 11 0 4 11
6 0 2 6 0 2
3 ∞ 0 3 7 0
Step-3: Using, A2[i, j] = min{A1[i, j], A1[i, 2]+A1[2, j]} Step-4: Using, A3[i, j] = min{A2[i, j], A2[i, 3]+A2[3, j]}
A2 A3
0 4 6 0 4 6
6 0 2 5 0 2
3 7 0
3 7 0
All-Pairs Shortest Paths:

1→2 2→3 3→1
1→2→3 2→3→1 3→1→2
Exercises:
(1)
(2)
-------------------------------------------------------------------------------------------------------------------------
The Traveling Salesperson problem: -
Given a set of cities and distances between every pair of cities, the problem is to find the shortest
possible route that visits every city exactly once and returns to the starting point.
Let the cities be represented by vertices in a graph and the distances between them be represented
by weights on edges in the graph.
Let G(V, E) be the directed graph with n vertices.

Let graph G be represented by cost adjacency matrix C[1:n, 1:n].
For simplicity, C[i, j] is denoted by Cij. If <i, j>∉ E(G), Cij=∞; otherwise Cij≥0.
A tour of G is a directed cycle that includes every vertex in V. The cost of a tour is the sum of the
costs of the edges on the tour.
The travelling salesperson problem aims at finding an optimal tour (i.e., a tour of minimum cost).
Let the given set of vertices be {1, 2, …., n}. In our discussion, we shall consider a tour be a
simple path that starts at vertex 1 and terminates at 1.
Every possible tour can be viewed as consisting of an edge <1, k> for some k∈ V-{1} and a path
from vertex k to vertex 1.
The path from vertex k to vertex 1 goes through each vertex in the set V-{1, k} exactly once.
Suppose the tour consisting of the edge <1, k> (for some k∈ V-{1}) followed by a path from k to 1
is an optimal tour. Then the path from k to 1 should also be optimal. Thus, the principle of optimality
holds.
Let g(i, S) denote the length of the shortest path starting at vertex i, going through all vertices in
set S, and terminating at vertex 1. Thus g(1, V-{1}) gives the length of an optimal tour.
If the optimal tour consists of the edge <1, k>, then from the principle of optimality, it follows
that g(1, V-{1})= C1k+g(k, V-{1,k}).
But we don’t know for which value of k the tour will be optimal. We know that k take can any
value from the set {2, 3, …, n}.
The RHS of the above recurrence can be evaluated for each value of k and the minimum of those
results should be assigned to g(1, V-{1}) resulting in the following recurrence equation:
g(1, V-{1}) = min2≤k≤n {C1k+g(k, V-{1,k})} ------- (1)
Generalizing equation (1) for i not in S, we obtain
g(i, S)=min∀ j∈S {Cij+g(j, S-{j})} ------ (2)
Equation (1) can be solved for g(1, V-{1}) if we know g(k, V-{1,k}) for all choices of k which can
be obtained by using equation (2).
Clearly, g(i, Ø)=Ci1, 1≤i≤n.
The optimal tour can be found by noting the vertex which resulted in minimum cost at each stage.
An algorithm that proceeds to find an optimal tour by making use of (1) and (2) will require
O(n 22n) time.
EXAMPLE: - Consider the directed graph below and its edge lengths given by cost adjacency
matrix.
Find optimal sales person tour from vertex1.
Answer:-
g(1,{2,3,4})=min{C12+g(2,{3,4}),C13+g(3,{2,4}),C14+g(4,{2,3})}
g(2,{3,4})=min{C23+g(3,{4}),C24+g(4,{3})
g(3,{4})=C34+g(4,Ø)=C34+C41
=12+8=20.
g(4,{3})=C43+g(3,Ø)=C43+C 31
=9+6=15.
g(2,{3,4})=min{9+20,10+15}
=25.
g(3,{2,4})=min{C32+g(2,{4}),C34+g(4,{2})}
g(2,{4})=C24+g(4,Ø)=C24+C41
=10+8=18.
g(4,{2})=C42+g(2,Ø)=C42+C21
=8+5=13.
g(3,{2,4})=min{18+13,12+13}
=min{31, 25}
=25.
g(4,{2,3})=min{C42+g(2,{3}),C43+g(3,{2})}
g(2,{3})=C23+g(3,Ø)=C23+C31
=9+6=15.
g(3,{2})=C32+g(2,Ø)=C32+C21
=13+5=28.
g(4,{2,3})=min{8+15,9+28}
=min{23,37}
=23.
g(1,{2,3,4})=min{10+25,15+25,20+23}
=min{35,40,43}
=35.
g(1,{2,3,4})=35.
Construction of the optimal tour step by step:-
g (1,{2,3,4})=C12+g(2,{3,4})  12
g(2,{3,4})=C24+g(4,{3})  124
g(4,{3})=C43+g({3,Ø})  12431
So, the optimal tour is:- 12431.
Reliability Design: -
We need to design a system that functions with maximum reliability. The system consists of n devices connected in
series as shown below: -
…
D1 D2 D3 Dn
.
Let ri be the reliability of device Di (that is, ri is the probability that device i will function properly). Then the
reliability of the entire system is ∏ 𝑟i .
Even if the individual devices are very reliable, the reliability of the entire system may not be very good.
For example, if n=10 and ri=0.99, i≤1≤10, then ∏ 𝑟𝑖= (0.99)10 = 0.904.
Hence it is desirable to have multiple copies of the same device connected in parallel so as to improve the
reliability of the system.
EXAMPLE: -
Let there be mi copies of the device Di at stage i. Then the probability that all copies have a malfunction is, (1-ri)mi.
Hence the reliability of stage i becomes, 1-(1-ri)mi.
Let us assume that the reliability of stage i with mi copies is denoted by a function Φi(mi).
So, Φi(mi)=1-(1-ri)mi.
Then, the reliability of the entire system is, ∏ 𝛷𝑖(𝑚𝑖).
The reliability design problem is to use multiple copies (as many copies as possible) of the devices at each stage so
as to maximize the reliability of the entire system. However, this is to be done under a cost constraint.
Let Ci be the cost of each unit of Di and C be the maximum allowed cost (budget) of the system being designed.
Then the reliability problem is mathematically formulated as: -
maximize ∏ 𝛷𝑖(𝑚𝑖)
subject to
∑ 𝐶𝑖𝑚𝑖 ≤ C
where, mi is an integer and mi ≥ 1, for all i.
Here, mi ≥ 1 means that at each stage i, at least one unit of device Di should be taken.
Assuming that each Ci>0, each mi must be in the range 1≤mi≤ui, where
𝐶− ∑ 𝐶
𝑢 =
𝐶
EXAMPLE: - C=100, n=3, C1=15, C2=20, C3=25.
( )
𝑢 = =3
( )
𝑢 = =3
( )
𝑢 = =2
The dynamic programming approach finds the optimal solution for m1, m2, m3, …, mn.
An optimal sequence of decisions, (i.e., a decision for each mi), can result in an optimal solution.
Solution to the Reliability Design problem using backward recurrence equation:

Let f(i, x) denote the reliability of an optimal solution by considering the stages i, i-1, …, 1 and with remaining
funds (or, budget) x.
Then the reliability of an optimal solution to the original problem is given by f(n, C).
The first decision is made on mn whose value can be chosen from the set {1, 2, 3, …, un}.
Once a value for mn has been chosen, the remaining decisions must be made so as to use the remaining funds
(C-Cnmn) in an optimal way. Using the principle of optimality, we can write:
f(n, C)= 𝑚𝑎𝑥 {𝛷 (𝑚 )𝑓(𝑛 − 1, 𝐶 − 𝐶 𝑚 )}
For any, f(i, x), i≥1, this equation generalizes to
f(i, x)= 𝑚𝑎𝑥 {𝛷 (𝑚 )𝑓(𝑖 − 1, 𝑥 − 𝐶 𝑚 )}
Clearly, f(0, x) =1 for all x, 0≤ x≤C.
Solution using Sets with Ordered pairs: -

We solve the above recurrence equations using the method similar to that used for 0/1 knapsack problem.
Let Si consists of tuples (f, x) that may result from the various decision sequences for m1, m2, m3, …, mi, where f
indicates the reliability of the system by considering stages 1 to i and x indicates the corresponding cost incurred
(or, funds spent so far).
Obtaining Si from Si-1: -
Assume that there is ui total no. of possible values for mi. Then we will obtain ui number of subsets: S1i, S2i, S3i, …,
𝑆 from set Si-1. Then Si can be obtaining by merging all those subsets.
So,
Si = S1i U S2i U S3i U … U 𝑆 .
Sji,1≤ j≤ ui can be obtained from all tuples (f, x) of Si-1 as follows:
Sji= {(f * 𝛷 (j), x+j*𝐶 )}, 1≤ j ≤ ui.
If there is a tuple (f, x) in Sji such that (x+∑ 𝐶 )>C, that tuple can be removed from Sji because such a tuple
will not leave adequate funds to complete the system.
While merging the sets S1i, S2i, S3i, …, 𝑆 to obtain Si, the tuples have to be arranged in the increasing order of
reliabilities and then dominance rule (or, purging rule) has to be applied.
Dominance rule:-
The tuple (f2, x2) dominates (f1, x1) iff f2 ≥ f1 and x2 ≤ x1. The dominated tuple (f1, x1) is discarded from the set Si.
Example:-
Design a 3-stage system in an optimal way with the following data. The costs are $30, $15, $20. The cost of the
system can’t be more than $105. The reliability of each device type is 0.9, 0.8 and 0.5 respectively.
Ans:-
Given C1=30, C2=15, C3=20, C=105
r1=0.9, r2=0.8, r3=0.5.
(𝐶 2 𝐶 3 ) (15 20)
𝑢 = = = =2
𝐶1 30 30
(𝐶 1 𝐶 3 ) (30 20)
𝑢 = = = =3
𝐶2 15 15
(𝐶 1 𝐶 2 ) (30 15)
𝑢 = = = =3
𝐶3 20 20
S0 = {(1,0)}.
Obtaining S1 from S0: -
Since u1=2, the possible choices for m1 are 1 and 2.
For m1=1: -
Φ1(m1)=Φ1(1)=1-(1-r1)m1= 1-(1-0.9)1 = 1- 0.1 = 0.9
The cost incurred is m1c1=1*30=30
S11= {(1*0.9,0+30)} = {(0.9,30)}
For m1=2: -
Φ1(m1) =Φ1(2) =1-(1-r1) m1= 1-(1-0.9)2 = 1- 0.01 = 0.99
S21= {(1*0.99,0+60)} = {(0.99,60)}
Now,
S1 = S11 U S21
S1= {(0.9,30),(0.99,60)}.
Obtaining S2 from S1: -

Since u2=3 the possible choices for m2 are 1, 2 and 3.
For m2=1:-
Φ2(m2)=Φ2(1)=1-(1-r2)m2= 1-(1-0.8)1 = 1-0.2 =0.8
S12={(0.9*0.8,30+15),(0.99*0.8,60+15)}={(0.72,45),(0.792,75)}
For m2=2:-
Φ2(m2)=Φ2(2)=1-(1-r2)m2= 1-(1-0.8)2 =0.96
S22={(0.9*0.96,30+30),(0.99*0.96,60+30)}={(0.864,60),(0.9504,90)}
= {(0.864,60)}
Note: The tuple (0.9504, 90) was removed from S22 because there are no adequate funds to include even one copy
of D3 device in the 3rd stage.
For m2=3:-
Φ2(m2)=Φ2(3)=1-(1-r2)m2= 1-(1-0.8)3 =0.992
S32={(0.9*0.992,30+45),(0.99*0.992,60+45)}={(0.8928,75),(0.98208,105)}
={(0.8928,75)}
Now,
S2 = S12 U S22 U S32 ={(0.72,45),(0.792,75),(0.864,60), (0.8928,75)}
By purging the tuple (0.792,75) from S2,
S2={(0.72,45),(0.864,60),(0.8928,75)}.
Obtaining S3 from S2:-

Since u3=3 the possible choices for m3 are 1, 2 and 3.
For m3=1:-
Φ3(m3)=Φ3(1)=1-(1-r3)m3= 1-(1-0.5)1 =0.5
S13={(0.72*0.5,45+20),(0.864*0.5,60+20),(0.8928*0.5,75+20)}
={(0.36,65),(0.432,80),(0.4464,95)}
For m3=2:-
Φ3(m3)=Φ3(2)=1-(1-r3)m3= 1-(1-0.5)2=0.75
S23 ={(0.72*0.75,45+40),(0.864*0.75,60+40),(0.8928*0.75,75+40)}
={(0.54,85),(0.648,100),(0.6696,115)}
={(0.54,85),(0.648,100)}
For m3=3:-
Φ3(m3)=Φ3(3)=1-(1-r3)m3= 1-(1-0.5)3 =0.875
S33={(0.72*0.875,45+60),(0.864*0.875,60+60),(0.8928*0.875,75+60)}
={(0.63,105),(0.756,120),(0.7812,135)}
={(0.63,105)}
Now,
S3=S13 U S23 U S33
={(0.36,65),(0.432,80),(0.4464,95),(0.54,85),(0.63,105),(0.648,100)}
={(0.36,65),(0.432,80),(0.54,85),(0.648,100)}
The best optimal design has a reliability of 0.648 and cost of $100.
Tracing back the values of mi :

→ The last tuple of S3 is (0.648,100) and it is present in S23 subset. So, m3=2.
→ The tuple (0.648,100) of S23 came from the tuple (0.864,60) of S2.
→ The tuple (0.864,60) of S2 is present in S22 subset. So, m2=2.
→ The tuple (0.864,60) of S22 came from the tuple (0.9,30) of S1.
→ The tuple (0.9,30) of S1 is present in S11 subset. So, m1=1.
So, the optimal solution is: (m1, m2, m3) = (1, 2, 2).
UNIT-III
DYNAMIC PROGRAMMING
→ This technique is used to solve optimization problems.
→ In dynamic programming, we obtain the solution to a problem by performing a
sequence of decisions. We examine the decision sequence to see, if the optimal
decision sequence contains optimal decision subsequences.
→ In dynamic programming, an optimal sequence of decisions is obtained by
using the principle of optimality.
PRINCIPLE OF OPTIMALITY:
The principle of optimality states that, “In an optimal sequence of decisions each
subsequence must also be optimal”.
(OR)
The optimal solution to a problem is composed of optimal solutions to the
subproblems.
0/1 KNAPSACK PROBLEM:

→This is similar to fractional knapsack problem, except that 𝑥i’s are restricted to
have a value either 0 or 1.
→ We are given a set of ‘n’ items (or, objects), such that each item i has a weight
‘wi’ and a profit ‘𝑝i’. We wish to pack a knapsack whose capacity is ‘M’ with a
subset of items such that total profit is maximized.
→ The solution to this problem is expressed as a vector (x1, x2, …, xn), where each
xi is 1 if object i has been placed in knapsack, otherwise xi is 0.
→ The mathematical formulation of the problem is:
maximize ∑𝑛𝑖=1 𝑝𝑖 𝑥𝑖
subject to the constraints
∑𝑛𝑖=1 𝑤𝑖 𝑥𝑖 ≤ 𝑀
and 𝑥i∈{0,1}, 1 ≤ i ≤ n
→ We need to make the decisions on the values of 𝑥1, 𝑥2, 𝑥3, …, 𝑥.
→ When optimal decision sequence contains optimal decision subsequences, we
can establish recurrence equations that enable us to solve the problem in an
efficient way.
→ We can formulate recurrence equations in two ways:
Forward approach
Backward approach
Recurrence equation for 0/1 knapsack problem using Forward approach:
For the 0/1 knapsack problem, optimal decision sequence is composed of optimal
decision subsequences.
Let f(i, y) denote the value (or profit), of an optimal solution to the knapsack
instance with remaining capacity y and remaining objects i, i+1,…, n for which
decisions have to be taken.
It follows that
𝑤 , 𝑖𝑓 𝑤𝑛 ≤ 𝑦
𝑓 (𝑛, 𝑦) = { 𝑛 -------(1)
0, 𝑖𝑓 𝑤𝑛 > 𝑦
and
𝑚𝑎𝑥 (𝑓 (𝑖 + 1, 𝑦), 𝑝𝑖 + 𝑓 (𝑖 + 1, 𝑦 − 𝑤𝑖 )), 𝑖𝑓 𝑤𝑖 ≤ 𝑦
𝑓 (𝑖, 𝑦) = { --(2)
𝑓(𝑖 + 1, 𝑦), 𝑖𝑓 𝑤𝑖 > 𝑦
NOTE: when 𝑤i > y, we cannot place the object i, and there is no choice to make,
but when 𝑤i ≤ 𝑦 we have two choices, viz., if object i can be placed or not. Here
we will take into account that choice which results in maximum profit.
→ f(1, M) is the value (profit) of the optimal solution to the knapsack problem we
started with. Equation (2) may be used recursively to determine f(1, M).
-----------------------------------------------------------------------------------------------
Example 1: Let us determine f(1, M) recursively for the following 0/1 knapsack
instance: n=3 ; (𝑤1, 𝑤2, 𝑤3) = (100,14,10) ; (𝑝1, 𝑝2, 𝑝3) = (20,18,15) and
M=116.
Solution:
f(1,116) =max{f (2, 116), 20 + f(2, 116 − 100)} , since w1<116
=max{f (2, 116), 20 + f(2, 16)}
f(2,116) =max{f (3, 116), 18 + f(3, 116 − 14)} , since w2<116
=max{f (3, 116), 18 + f(3, 102)}
f(3,116) =15 since w3<116
f(3,102) =15 since w3<102
f(2,116) = 𝑚𝑎𝑥{15, (18 + 15)} = 𝑚𝑎𝑥{15, 33} =33

now,
f(2, 16) = max{f (3, 16), 18 + f(3,2)}
f(3,16) =15
f(3,2) =0
So, f(2,16)= max(15, 18+0) =18
f(1,116) = max{33, (20 + 18)} = 38
Tracing back the xi values:
To obtain the values of xi’s, we proceed as follows:
If f(1, M) = f(2,M) then we may set x1=0.
If f(1, M) ≠ f(2,M) then we may set 𝑥1=1.
Next, we need to find the optimal solution that uses the remaining capacity M-𝑤1.
This solution has the value f(2, M-𝑤1). Proceeding in this way, we may determine
the values of all the xi’s.
Determining the xi values for the above example:
f(1, 116) = 38
f(2, 116) = 33
Since f(1, 116) ≠ f(2, 116) → 𝑥1 = 1.
After placing object 1, remaining capacity is 116 -100 =16. This will lead to
f(2, 16).
f(2,16)=18
f(3,16)=15
Since f(2, 16) ≠ f(3, 16) → 𝑥2 = 1.
After placing object 2, remaining capacity is 16-14=2, this will lead to f(3,2).
Since f(3, 2) = 0 → 𝑥3 = 0.
So, optimal solution is: (𝑥1, 𝑥2, 𝑥3) = (1,1,0), which yields the profit of 38.
ALOGORITHM FOR 0/1 KNAPSACK PROBLEM USING FORWARD
APPROCH:
Algorithm f(i, y)
{
if(i=n) then
{
if(w[n]>y) then return 0;
else return p[n];
}
if(w[i]>y) then return f(i+1, y);
else return max [f(i + 1, 𝑦), (𝑝i + f(i + 1, 𝑦 − 𝑤i))];
}
→ INITIALLY THE ABOVE ALGORITHM IS INVOKED AS f(1, M).
TIME COMPLEXITY:
→ let T(n) be the time this code takes to solve an instance with n
objects.
𝑐, 𝑖𝑓 𝑛 = 1
So, 𝑇(𝑛) = {
2𝑇(𝑛 − 1) + 𝑐, 𝑖𝑓 𝑛 > 1
Solving this, we get time complexity equal to O(2𝑛).
→ In general, if there are d choices for each of the n decisions to be made, there
will be 𝑑𝑛 possible decision sequences.
Example: Consider the case n=5; M=10; (𝑤1, 𝑤2, 𝑤3, 𝑤4, 𝑤5) = (2,2,6,5,4); (𝑝1,
𝑝2, 𝑝3, 𝑝4, 𝑝5) = (6,3,5,4,6).
Solution: To determine f(1,10), function f is invoked as f(1,10). The recursive calls
made are shown by the following tree of recursive calls.
→ 27 invocations are done on f. We notice that several invocations redo the work
of previous invocation. For example: f(3,8), f(4,8), f(4,2), f(5,8), f(5,3), f(5,2) are
computed twice.
If we save the result of previous invocations, we can reduce the number of
invocations to 21.
We maintain a table to store the values, as and when the function call computes
there.
By using these table values, we can avoid re-computing the function, when it is
again invoked.
When the recursive program is designed to avoid the re-computation, the
complexity is drastically reduced from exponential to polynomial.
Recurrence equation for 0/1 knapsack problem using backward approach:
Let f(i, y) denote the value (or, profit) of an optimal solution to the knapsack
instance with remaining capacity y and remaining objects i, i-1, … ,1 for which
decisions have to be taken.
It follows that
𝑝 , 𝑖𝑓 𝑤1 ≤ 𝑦
𝑓 (1, 𝑦) = { 1 -------(1)
0, 𝑖𝑓 𝑤1 > 𝑦
and
𝑚𝑎𝑥 (𝑓 (𝑖 − 1, 𝑦), 𝑝𝑖 + 𝑓 (𝑖 − 1, 𝑦 − 𝑤𝑖 )), 𝑖𝑓 𝑤𝑖 ≤ 𝑦
𝑓 (𝑖, 𝑦) = { --(2)
𝑓(𝑖 − 1, 𝑦), 𝑖𝑓 𝑤𝑖 > 𝑦
→ f(n, M) gives the value (or profit) of the optimal solution by including objects n,
n-1, …, 1.
Example: Let us determine f(n, M) recursively for the following 0/1 knapsack
instance: n=3; (𝑤1, 𝑤2, 𝑤3) = (2,3,4); (𝑝1, 𝑝2, 𝑝3) = (1,2,5) and M=6.
Solution:
f(3,6) =max{f (2, 6), 5 + f(2, 2)}
f(2,6) =max{f (1, 6), 2 + f(1, 3)}
f(1,6) = p1 = 1
f(1,3) = p1 = 1
f(2,6) = max{1, 2+1} = 3
f(2,2) = f(1,2) = f(1,2) = 1
Now,
f(3,6) = max{3, 5+1} = 6
Tracing back values of xi:
f(3,6) = 6
f(2,6) = 3
Since f(3,6) ≠ f(2,6), x3=1.
After including object 3, remaining capacity becomes 6-4= 2, this will lead to the
problem f(2,2).
f(2,2) = 1
f(1,2) = 1
Since f(2,2) = f(1,2), x2=0, and this will lead to the problem f(1,2).
Since f(1,2) = 1, x1= 1.
So, optimal solution is: (𝑥1, 𝑥2, 𝑥3) = (1,0,1).
Solving 0/1 knapsack problem using Sets of Ordered Pairs:
Let 𝑠 𝑖 represent the possible state, resulting from the 2𝑖 decision sequences for
𝑥1 , 𝑥2 , 𝑥3 … … 𝑥𝑖 .
A state refers to a tuple (𝑝𝑗 , 𝑤𝑗 ), 𝑤𝑗 being the total weight of objects included in
the knapsack and 𝑝𝑗 being the corresponding profit.
NOTE: 𝑠 0 = {(0,0)}
To obtain 𝑠 𝑖+1 from 𝑠 𝑖 , we note that the possibilities for 𝑥𝑖+1 are 1 and 0.
When 𝑥𝑖+1 = 0, the resulting states are same as for 𝑠 𝑖 .
When 𝑥𝑖+1 =1, the resulting states are obtained by adding (𝑝𝑖+1, 𝑤𝑖+1) to each state
in 𝑠 𝑖 .We call the set of these additional states 𝑠1𝑖.
Now 𝑠 𝑖+1can be computed by merging the states in 𝑠 𝑖 𝑎𝑛𝑑 𝑠1𝑖 together, i.e.,
𝑠 𝑖+1 = 𝑠 𝑖 U 𝑠1𝑖. The states in 𝑠 𝑖+1 set should be arranged in the increasing order of
profits.
NOTE:
1. If 𝑠 𝑖+1 contains two pairs (𝑝𝑗 , 𝑤𝑗 ) and (𝑝𝑘 , 𝑤𝑘 ) with the property that 𝑝𝑗 ≤
𝑝𝑘 and 𝑤𝑗 ≥ 𝑤𝑘 , we say that (𝑝𝑘 , 𝑤𝑘 ) dominates (𝑝𝑗 , 𝑤𝑗 ) and the
dominated tuple (𝑝𝑗 , 𝑤𝑗 ) can be discarded from 𝑠 𝑖+1 . This rule is called
purging rule.
2. By this rule all duplicate tuples will also be purged.
3. We can also purge all pairs (𝑝𝑗 , 𝑤𝑗 ) with 𝑤𝑗 > 𝑀.
Finally profit for the optimal solution is given by p value of the last pair in 𝑠 𝑛 set.
Tracing back values of 𝒙𝒊 ’s:
Suppose that (𝑝𝑘 , 𝑤𝑘 ) is the last tuple in 𝑠 𝑛 , then a set of 0/1 values for the 𝑥𝑖 ’s
can be determined by carrying out a search through the 𝑠 𝑖 sets.
 if (𝑝𝑘 , 𝑤𝑘 ) ∈ 𝑠 𝑛−1, then we will set 𝑥𝑛 = 0.
If (𝑝𝑘 , 𝑤𝑘 )∉ 𝑠 𝑛−1, then (𝑝𝑘 − 𝑝𝑛 , 𝑤𝑘 − 𝑤𝑛 ) ∈ 𝑠 𝑛−1and we will set 𝑥𝑛 = 1. This
process can be done recursively to get remaining 𝑥𝑖 values.
Example: Consider the knapsack instance:
n = 3, (𝑤1 , 𝑤2, 𝑤3) = (2, 3, 4) and (𝑝1 , 𝑝2 , 𝑝3 ) = (1,2,5), and M = 6. Generate the
sets 𝑠 𝑖 and find the optimal solution.
Solution:
𝑠 0 = {(0,0)};
By including object 1,
𝑠10 = {(1,2)};
By merging 𝑠 0 𝑎𝑛𝑑 𝑠10 , we get
𝑠 1 = {(0,0), (1,2)};
𝑠11={(2,3), (3, 5)};
By merging 𝑠 1 𝑎𝑛𝑑 𝑠11, we get
𝑠 2 = {(0, 0), (1, 2), (2, 3), (3, 5)};
𝑠12= {(5, 4), (6, 6), (7, 7), (8, 9)} = {(5, 4), (6, 6)};
By merging 𝑠 2 𝑎𝑛𝑑 𝑠12 , we get
𝑠 3= {(0,0), (1,2), (2, 3), (3, 5), (5, 4), (6, 6)};
By applying the purging rule, the tuple (3, 5) will get discarded.
𝑠 3={(0,0), (1,2), (2, 3), (5, 4), (6, 6)};
Tracing out the value of 𝑥𝑖 :
The last tuple in 𝑠 3 is (6,6) ∉ 𝑠 2. So, 𝑥3 =1.
The last tuple (6,6) of 𝑠 3 came from a tuple (6 − 𝑝3 , 6 − 𝑤3 ) = (6-5, 6-4) = (1, 2)
belonging to 𝑠 2.
The tuple (1, 2) of 𝑠 2, is also present in 𝑠 1. So, 𝑥2 = 0.
The tuple (1, 2) of 𝑠 1, is not present in 𝑠 0. So, 𝑥1 = 1.
So, the optimal solution is: (𝑥1 , 𝑥2 , 𝑥3 ) = (1,0,1)
Example: Consider the knapsack instance:

n = 5, (𝑤1 , 𝑤2, 𝑤3 , 𝑤4 , 𝑤5) = (2, 2, 6, 5, 4) and (𝑝1 , 𝑝2 , 𝑝3 , 𝑝4 , 𝑝5, ) = (6,3,5,4,6),
and M = 10. Generate the sets 𝑠 𝑖 and find the optimal solution.
Tracing out the value of 𝑥𝑖 :
The last tuple in 𝑠 5 is (15,8) ∉ 𝑠 4. So, 𝑥5 = 1.
The last tuple (15,8) of 𝑠 5came from a tuple (15 − 𝑝5 , 8 − 𝑤5 ) = (15-6, 8-4) =
(9, 4) belonging to 𝑠 4.
The tuple (9, 4) of 𝑠 2, is not present in 𝑠 1. So, 𝑥2 = 1.
The tuple (9, 4) of 𝑠 2 came from a tuple (9-𝑝2 , 4-𝑤2 ) = (9-3, 4-2) = (6,2)
belonging to 𝑠 1.
 The tuple (6, 2) of 𝑠 1, is not present in 𝑠 0. So, 𝑥1 = 1.
So, the optimal solution is: (𝑥1 , 𝑥2 , 𝑥3 , 𝑥4 , 𝑥5) = (1,1,0,0,1).
----------------------------------------------------------------------------------------------------
How the dynamic-programming method works? (Not required for theory exam)
The dynamic-programming method works as follows. Having observed that a
naive recursive solution is inefficient because it solves the same subproblems
repeatedly, we arrange for each subproblem to be solved only once, saving its
solution. If we need to refer to this subproblem’s solution again later, we can just
look it up, rather than recompute it. Dynamic programming thus uses additional
memory to save computation time; it serves an example of a time-memory trade-
off. The savings may be dramatic: an exponential-time solution may be
transformed into a polynomial-time solution.
A dynamic-programming approach runs in polynomial time when the number of

distinct subproblems involved is polynomial in the input size and we can solve
each such subproblem in polynomial time.
There are usually two equivalent ways to implement a dynamic-programming

approach.
1. Top-down with memoization:
In this approach, we write the procedure recursively in a natural manner, but
modified to save the result of each subproblem (usually in an array or hash table).
The procedure now first checks to see whether it has previously solved this
subproblem. If so, it returns the saved value, saving further computation at this
level; if not, the procedure computes the value in the usual manner and stores the
result in the table.
2. Bottom-up method:
This approach typically depends on some natural notion of the “size” of a
subproblem, such that solving any particular subproblem depends only on solving
“smaller” subproblems. We sort the subproblems by size and solve them in size
order, smallest first. When solving a particular subproblem, we have already
solved all of the smaller subproblems its solution depends upon, and we have
saved their solutions. We solve each subproblem only once, and when we first see
it, we have already solved all of its prerequisite subproblems.
Solution to 0/1 Knapsack problem using Top-down Dynamic Programming

approach with Memoization:
Let us consider the backward recursive equation.

𝑝 , 𝑦 ≥ 𝑤1
f(1, y) = { 1 }
0 , 𝑦 < 𝑤1
𝑚𝑎𝑥(𝑓 (𝑖 − 1, 𝑦), (𝑝𝑖 + , 𝑓(𝑖 − 1, 𝑦 − 𝑤𝑖 ))), 𝑖𝑓 𝑦 ≥ 𝑤𝑖
f(i, y) ={ }
𝑓 (𝑖 − 1, 𝑦) , 𝑖𝑓 𝑦 < 𝑤𝑖
Let us rewrite the above backward recursive equation as follows:

f(0, y) = 0 for y≥0 and f(i, 0) = 0 for i≥0

f(i, y) ={ }
𝑓 (𝑖 − 1, 𝑦) , 𝑖𝑓 𝑦 < 𝑤𝑖
Initially this function is invoked as f(n, M). This problem’s size is n (i.e., the
number of objects on which decisions have to be taken). Next, it calls the
subproblem f(n-1,y) whose size is n-1, and so on. This recursion is repeated until a
problem of smallest size i.e., f(1, y) is called.
Algorithm f(i, y)
{
// Let T[0:n, 0:M] be a global two dimensional array whose elements are
initialized // with -1 except for row 0 and column 0 which are initialized with 0’s.
if (T[i, y] <0) then // if f(i, y) has not been computed previously
{
if(w[i]>y) then
{
T[i, y] := f(i-1, y);
}
else
{
T[i, y] := max(f(i-1, y), p[i] + f(i-1, y-w[i]));
}
}
return T[i, y];
}
Initially this function is invoked as f(n, M).
What is the time and space complexity of the above solution?
Since our memoization array T[0:n, 0:M] stores the results for all the subproblems,
we can conclude that we will not have more than (n+1)*(M+1) subproblems
(where ‘n’ is the number of items and ‘M’ is the knapsack capacity). This means
that the time complexity will be O(n*M).
The above algorithm will be using O(n*M) space for the memoization array T.
Other than that, we will use O(n) space for the recursion call-stack. So, the total
space complexity will be O(n*M+n), which is asymptotically equivalent
to O(n*M).
Tracing back values of x[i]:
for i:=n to 1 step -1 do

{
if T[i, y]=T[i-1, y] then
x[i]:=0;
else
{
x[i]:=1;
y:=y-w[i];
}
}
Example: consider the case n=5; M=10; (𝑤1 , 𝑤2 , 𝑤3, 𝑤4 , 𝑤5) = (2,2,6,5,4) ;
(𝑝1 , 𝑝2 , 𝑝3 , 𝑝4 , 𝑝5 ) = (6,3,5,4,6)
f(5,10) = max(f(4,10), 6+f(4,6))
f(4,10) = max(f(3,10), 4+f(3,5))
f(3,10) = max(f(2,10), 5+f(2,4))
f(2,10) = max(f(1,10), 3+f(1,8))
f(1,10) = max(f(0,10), 6+f(0,8)) = max(0, 6+0) = 6

f(1,8) = max(f(0,8), 6+f(0,6)) = max(0, 6+0) = 6
Now, f(2,10) = max(6, 3+6) = 9
f(2,4) = max(f(1,4), 3+f(1,2)
f(1,4) = max(f(0,4), 6+f(0,2)) = max(0, 6+0) = 6
f(1,2) = max(f(0,2), 6+f(0,0)) = max(0, 6+0) = 6
Now, f(2,4) = max(6,3+6) = 9
Now, f(3,10) = max(6, 5+6) = 11
f(3,5) = f(2,5)
f(2,5) = max(f(1,5), 3+f(1,3))
f(1,5) = max(f(0,5), 6+f(0,3)) = max(0, 6+0) = 6
f(1,3) = max(f(0,3), 6+f(0,1)) = max(0, 6+0) = 6
Now, f(2,5) = max(6, 3+6) = 9
So, f(3,5) = 9
Now, f(4,10) = max(11, 4+9) = 13
f(4,6) = max(f(3,6), 4+f(3,1))
f(3,6) = max(f(2,6), 5+f(2,0))
f(2,6) = max(f(1,6), 3+f(1,4))
f(1,6) = max(f(0,6), 6+f(0,4)) = max(0, 6+0) = 6
f(1,4) = 6 (Obtained from the table)
Now, f(2,6) = max(6, 3+6) = 9.
f(2,0) = 0
Now, f(3,6) = max(9, 5+0) = 9.
f(3,1) = f(2,1) = f(1,1) = 0
f(4,6) = max(9, 4+0) = 9
Now, f(5,10) = max(13, 6+9) =15.
Column indices (y values)
0 1 2 3 4 5 6 7 8 9 10
0 0 0 0 0 0 0 0 0 0 0 0
Row indices (i values)
1 0 -1(0) -1(6) -1(6) -1(6) -1(6) -1(6) -1 -1(6) -1 -1(6)

2 0 -1(0) -1 -1 -1(9) -1(9) -1(9) -1 -1 -1 -1(9)
3 0 -1(0) -1 -1 -1 -1(9) -1(9) -1 -1 -1 -1(14)
4 0 -1 -1 -1 -1 -1 -1(9) -1 -1 -1 -1(14)
5 0 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1(15)
The optimal solution is: (x1, x2, x3, x4, x5) = (1, 1, 0, 0, 1).
Solution to 0/1 Knapsack problem using Bottom-up Dynamic Programming
approach:
Let us consider backward recursive equation as follows:

f(0, y) = 0 for y≥0 and f(i, 0) = 0 for i≥0

f(i, y) ={ }
𝑓 (𝑖 − 1, 𝑦) , 𝑖𝑓 𝑦 < 𝑤𝑖
Step-01:
 Draw a table say ‘T’ with (n+1) number of rows and (M+1) number of
columns.
 Fill all the boxes of 0th row and 0th column with zeroes as shown below:
0 1 2 3 … M
0 0 0 0 0 … 0
1 0
2 0
3 0
… …
N 0
Step-02:
Start filling the table row wise top to bottom from left to right.
Use the following formula:
T[i , y] = max { T [ i-1 , y ] , pi + T[ i-1 , y – wi ] } , 𝑖𝑓 𝑦 ≥ 𝑤𝑖
= T [ i-1 , y ], 𝑖𝑓 𝑦 < 𝑤𝑖
Here, T[i , y] = maximum profit earned by taking decisions on items 1 to i with
remaining capacity y.
 This step leads to completely filling the table.
 Then, value of the last cell (i.e., intersection of last row and last column)
represents the maximum possible profit that can be earned.
Step-03:
To identify the items that must be put into the knapsack to obtain that maximum
profit (that means to trace back the values of xi),
 Consider the last entry (i.e., cell) of the table.
 Start scanning the entries from bottom to top.
 On encountering an entry whose value is not same as the value stored in the
entry immediately above it, mark the row label of that entry.
 After all the entries are scanned, the marked labels represent the items that
must be put into the knapsack.
Algorithm:
Algorithm KnapSack(n, M)
{
// Let T[0:n, 0:M] be a global two dimensional array whose elements in row 0 and
// column 0 are initialized with 0’s.
for i:=1 to n do
{
for y:=1 to M do
{
if(w[i]>y) then
{
T[i, y] := T[i-1, y];
}
else
{
T[i, y] := max(T[i-1, y], p[i] + T[i-1, y-w[i]]);
}
}
}
return T[n, M];

}
This function is invoked as KnapSack(n, M).

Time and Space Complexity:
Each entry of the table requires constant time θ(1) for its computation.
It takes θ(nM) time to fill (n+1)(M+1) table entries. This means that the time
complexity will be O(n*M). Even though it appears to be polynomial time but
actually it is called pseudo polynomial time because when M≥2n, the time
complexity is actually exponential but not polynomial.
The space complexity is θ(nM).
Tracing back values of x[i]:
for i:=n to 1 step -1 do

{
if T[i, y]=T[i-1, y] then
x[i]:=0;
else
{
x[i]:=1;
y:=y-w[i];
}
}
Example: consider the case n=5; M=10; ; (𝑤1, 𝑤2 , 𝑤3, 𝑤4 , 𝑤5 ) = (2,2,6,5,4) ;

(𝑝1 , 𝑝2 , 𝑝3 , 𝑝4 , 𝑝5 ) = (6,3,5,4,6)
0 1 2 3 4 5 6 7 8 9 10
0 0 0 0 0 0 0 0 0 0 0 0
w1=2, 1 0 0 6 6 6 6 6 6 6 6 6
p1=6
w2=2, 2 0 0 6 6 9 9 9 9 9 9 9
p2=3
w3=6, 3 0 0 6 6 9 9 9 9 11 11 14
p3=5
w4=5, 4 0 0 6 6 6 9 9 10 11 13 14
p4=4
W5=4, 5 0 0 6 6 6 9 12 12 12 15 15
p5=6
The optimal solution is: (x1, x2, x3, x4, x5) = (1, 1, 0, 0, 1).
OPTIMAL BINARY SEARCH TREE:(OBST)
→ We are given a set of sorted identifiers (or, keys) {a1, a2, …, an} such that a1 <
a2 < a3 < ….< an, and probabilities p1, p2, ….., pn with which the keys are searched
for respectively.
→ The problem is to arrange the keys in a Binary Search Tree in a way that
minimizes the expected total search time. Such a BST is called optimal BST
(OBST).
→ We assume that there are n+1 dummy keys named d0, d1, d2, …, dn such that d0
represents all values less than a1 and dn represents all values greater than an and for
i=1, 2, 3, …., n-1 the dummy key di represents all values between ai and ai+1.
→ The dummy keys are leaves (or, external nodes) and the data keys are internal
nodes in the BST.
→ For each dummy key di, we have a search probability qi.
Since every search is either successful or unsuccessful, the probabilities sum to 1.
Therefore: ∑𝑛𝑖=1 𝑝𝑖 + ∑𝑛𝑖=0 𝑞𝑖 = 1.
→ The actual search cost for any key = Number of items examined (or Number
of comparisons made).
→ So, for any key ai, the actual search cost = level(ai).
The expected search cost of ai = level(ai)* pi.
→ For dummy key di, the actual search cost = level(di)-1.
The expected search cost of di = (level(di)-1) * qi.
→ Now, the expected search cost of a Binary Search Tree ‘T’ is,
∑𝑛𝑖=1( level(𝑎𝑖 ) ∗ 𝑝𝑖 ) + ∑𝑛𝑖=0(( 𝑙𝑒𝑣𝑒𝑙 (𝑑𝑖 ) − 1) ∗ 𝑞𝑖 ) → (1)
=Expected search cost of successful searches + Expected search cost of
unsuccessful searches.
Example:
Draw the possible BST’s along with dummy keys for the identifier set (a1, a2,
a3) = (do, if, while), with the probabilities p1= 0.5, p2=0.1, p3= 0.05, q0= 0.15,
q1= 0.1, q2= 0.05 and q3=0.05 and then find out expected search cost of each tree.
EXPECTED SEARCH COST FOR TREE (a):
3 3
∑ ( level (𝑎𝑖 ) ∗ 𝑝𝑖 ) + ∑(( 𝑙𝑒𝑣𝑒𝑙 (𝑑𝑖 ) − 1) ∗ 𝑞𝑖 )

𝑖=1 𝑖=0
= (3*0.5 + 2*0.1 + 1*0.05) + ((4-1)*0.15 + (4-1)*0.1 + (3-1)*0.05 + (2-1)*0.05)
= 2.65
EXPECTED SEARCH COST FOR TREE (b):
= (2*0.5 + 1*0.1 + 2*0.05) + ((3-1)*0.15 + (3-1)*0.1 + (3-1)*0.05 + (3-1)*0.005)

= 1.9
EXPECTED SEARCH COST FOR TREE (c):

= (1*0.5 + 2*0.1 + 3*0.05) + ((2-1)*0.15 + (3-1)*0.1 + (4-1)*0.05 + (4-1)*0.05)
=1.5
EXPECTED SEARCH COST FOR TREE (d):

= (2*0.5 + 3*0.1 + 1*0.05) + ((3-1)*0.15 + (4-1)*0.1 + (4-1)*0.05 +(2-1)*0.05)
= 2.15
EXPECTED SEARCH COST FOR TREE (e):
= (1*0.5 + 3*0.1 + 2*0.05) + ((2-1)*0.15 + (4-1)*0.1 + (4-1)*0.05 + (3-1)*0.05)

= 1.65
The tree (c) has the minimum cost. So, it is OBST.

NOTE:
OBST may not have smallest height.
OBST may not have highest probability key at root.
APPLYING DYNAMIC PROGRAMING TO SOLVE OBST PROBLEM:
→ To apply dynamic programing to the problem of obtaining an OBST, we need

to view the construction of such a tree as the result of a sequence of decisions.
→ The 1st decision to make is, which of the 𝑎i’s should be assigned to the root node
of the BST.
→ If we choose ak as the root node (for 1≤k≤n), then the internal nodes 𝑎1, 𝑎2,
…, 𝑎𝑘−1 and the external nodes 𝑑0, 𝑑1, 𝑑2, 𝑑3, …, 𝑑𝑘−1, will lie in the left
subtree of the root. The remaining nodes, (i.e., 𝑎𝑘+1, …, 𝑎𝑛 and 𝑑𝑘, 𝑑𝑘+1, …,
𝑑𝑛), will lie in the right subtree.
→ Denote BST by ‘T’, its left subtree by ‘L’, and its right subtree by ‘R’.
→ Then for any 𝑎i, i<k, we have:

𝑙𝑒𝑣𝑒lT (𝑎i) = le𝑣𝑒lL (𝑎i) + 1 (i.e., Level of 𝑎i in T = (Level of 𝑎i in L) +1 )
For any 𝑑i, i<k, we have:
𝑙𝑒𝑣𝑒lT (𝑑i) = 𝑙𝑒𝑣𝑒lL(𝑑i) + 1
→ For any 𝑎i, i>k, we have:
𝑙𝑒𝑣𝑒lT (𝑎i) = 𝑙𝑒𝑣𝑒lR (𝑎i) + 1
For any 𝑑i, i≥k, we have:
𝑙𝑒𝑣𝑒lT (𝑑i) = le𝑣𝑒lR (𝑑i) + 1
→ Let Cost(T) denote the estimated cost of BST ‘T’.
Let Cost(L) denote the estimated cost of left subtree ‘L’. Let Cost(R) denote the
estimated cost of right subtree ‘R’.
→ We know that,
𝑛 𝑛
𝐶𝑜𝑠𝑡(𝑇) = ∑( 𝑙𝑒𝑣𝑒𝑙 𝑇 (𝑎𝑖 ) ∗ 𝑝𝑖 ) + ∑(( 𝑙𝑒𝑣𝑒𝑙 𝑇 (𝑑𝑖 ) − 1) ∗ 𝑞𝑖 )

𝑖=1 𝑖=0
→ If ak is the root, then
𝑘−1 𝑘−1
𝐶𝑜𝑠𝑡(𝐿) = ∑ ( 𝑙𝑒𝑣𝑒𝑙𝐿 (𝑎𝑖 ) ∗ 𝑝𝑖 ) + ∑(( 𝑙𝑒𝑣𝑒𝑙𝐿 (𝑑𝑖 ) − 1) ∗ 𝑞𝑖 )

𝑖=1 𝑖=0
𝑛 𝑛
𝐶𝑜𝑠𝑡(𝑅) = ∑ ( 𝑙𝑒𝑣𝑒𝑙𝑅 (𝑎𝑖 ) ∗ 𝑝𝑖 ) + ∑(( 𝑙𝑒𝑣𝑒𝑙𝑅 (𝑑𝑖 ) − 1) ∗ 𝑞𝑖 )

𝑖=𝑘+1 𝑖=𝑘
→ Now,
Cost(T)=levelT(ak)*pk+∑𝑘−1 𝑛
𝑖=1 ( 𝑙𝑒𝑣𝑒𝑙 𝑇 ( 𝑎𝑖 ) ∗ 𝑝𝑖 ) +∑𝑖=𝑘+1( 𝑙𝑒𝑣𝑒𝑙 𝑇 (𝑎𝑖 ) ∗ 𝑝𝑖 ) +
∑𝑘−1 𝑛
𝑖=0 (( 𝑙𝑒𝑣𝑒𝑙 𝑇 (𝑑𝑖 ) − 1) ∗ 𝑞𝑖 )+∑𝑖=𝑘 (( 𝑙𝑒𝑣𝑒𝑙 𝑇 (𝑑𝑖 ) − 1) ∗ 𝑞𝑖 )
=1*pk+∑𝑘−1 𝑛
𝑖=1 ( (𝑙𝑒𝑣𝑒𝑙 𝐿 ( 𝑎𝑖 ) + 1) ∗ 𝑝𝑖 )+∑𝑖=𝑘+1( (𝑙𝑒𝑣𝑒𝑙 𝑅 (𝑎𝑖 ) + 1) ∗
𝑝𝑖 )+∑𝑘−1 𝑛
𝑖=0 ( 𝑙𝑒𝑣𝑒𝑙 𝐿 ( 𝑑𝑖 ) ∗ 𝑞𝑖 ) +∑𝑖=𝑘 (𝑙𝑒𝑣𝑒𝑙𝑅 ( 𝑑𝑖 ) ∗ 𝑞𝑖 )
=pk+∑𝑘−1 𝑛
𝑖=1 ( 𝑙𝑒𝑣𝑒𝑙𝐿 (𝑎𝑖 ) ∗ 𝑝𝑖 )+∑𝑖=𝑘+1( (𝑙𝑒𝑣𝑒𝑙𝑅 (𝑎𝑖 )) ∗
𝑝𝑖 )+∑𝑘−1 𝑛
𝑖=0 ( (𝑙𝑒𝑣𝑒𝑙𝐿 (𝑑𝑖 ) − 1) ∗ 𝑞𝑖 )+∑𝑖=𝑘( (𝑙𝑒𝑣𝑒𝑙𝑅 ( 𝑑𝑖 ) − 1) ∗
𝑞𝑖 )+∑𝑘−1 𝑛 𝑘−1 𝑛
𝑖=1 𝑝𝑖 +∑𝑖=𝑘+1 𝑝𝑖 +∑𝑖=0 𝑞𝑖 +∑𝑖=𝑘 𝑞𝑖
= pk+[ ∑𝑘−1 𝑘−1

𝑖=1 ( 𝑙𝑒𝑣𝑒𝑙𝐿 (𝑎𝑖 ) ∗ 𝑝𝑖 )+∑𝑖=0 ( (𝑙𝑒𝑣𝑒𝑙 𝑇 ( 𝑑𝑖 ) − 1) ∗ 𝑞𝑖 )]+
[∑𝑛𝑖=𝑘+1( (𝑙𝑒𝑣𝑒𝑙𝑅 (𝑎𝑖 ) + 1) ∗ 𝑝𝑖 )+∑𝑛𝑖=𝑘((𝑙𝑒𝑣𝑒𝑙𝑅 (𝑑𝑖 ) − 1) ∗ 𝑞𝑖 )]
+∑𝑘−1 𝑘−1 𝑛 𝑛
𝑖=1 𝑝𝑖 +∑𝑖=0 𝑞𝑖 +∑𝑖=𝑘+1 𝑝𝑖 + ∑𝑖=𝑘 𝑞𝑖
Cost(T)= pk+ Cost(L)+ Cost(R)+ w(0, k-1) + w(k, n)  (2)

𝑗 𝑗
where, w(i, j)= ∑𝑙=𝑖+1 𝑝𝑙 +∑𝑙=𝑖 𝑞𝑙 =(pi+1+…+pj)+(qi+qi+1+…+qj)
→ In equation (2), k can take any value from the set {1, 2, …, n}. If the tree ‘T’ is
optimal, then the Cost(T) in equation (2) must be minimum over all BST’s
containing keys 𝑎1, 𝑎2, …, 𝑎n and dummy keys 𝑑0, 𝑑1, …, 𝑑n. Hence
Cost(L) must be minimum over all BST’s containing keys 𝑎1, 𝑎2, …, 𝑎𝑘−1 and
dummy keys 𝑑0, 𝑑1, …, 𝑑𝑘−1. Similarly, Cost(R) must be minimum over all
BST’s containing keys 𝑎𝑘+1, …, 𝑎𝑛 and dummy keys 𝑑𝑘, 𝑑𝑘+1, …, 𝑑n.
→ Let c(i, j) denote the cost of OBST 𝑇i,j containing keys 𝑎i+1, 𝑎i+2, … , 𝑎j and
dummy keys 𝑑i, 𝑑i+1, …, 𝑑j .
Then for the tree T0, n to be optimal,

We must have
Cost(L) = c(0, k-1) and
Cost(R) = c(k, n)
→ So, cost of OBST T0,n is,
c(0, n) = 𝑚i𝑛1≤𝑘≤𝑛{𝑝𝑘 + c(0, k − 1) + c(k, n) + w(0, k − 1) + w(k, n)} → (3)
→ We can generalize equation (3) to obtain cost for any OBST 𝑇i,j,
c(i, j) = 𝑚i𝑛i+1≤𝑘≤j {𝑝𝑘 + c(i, k − 1) + c(k, j) + w(i, k − 1) + w(k, j) }
→ Since 𝑝𝑘 + w(i, k-1) + w(k, j) = w(i, j),
c(i, j)= w(i, j) + 𝑚i𝑛i+1≤𝑘≤j {c(i, k − 1) + c(k, j)} → (4)
NOTE:
(1) c(i, i) = 0, 0 ≤ i ≤ n, because 𝑇i,i is empty.
(2) w(i, i) = 𝑞i , 0 ≤ i ≤ n, since w(i,j)=(𝑝i+1 + ⋯ +𝑝j )+(𝑞i + 𝑞i+1 + ⋯ + 𝑞j )
(3)
w(i, j) = pj + qj + w(i, j-1)
→ Equation (4), can be solved for c(0, n) by first computing all c(i, j) values such
that j-i = 1. Next we can compute all c(i, j) values such that j-i =2, then all c(i, j)
values with j-i=3, and so on till j-i=n.
→ If during this computation, we record the root r(i, j) of each tree 𝑇i,j, then an
OBST can be constructed from those r(i, j) values.
Note:
r(i, j) is the value of k that minimizes equation (4).
r(i, i) = 0, 0 ≤ i ≤ n.
Example:
Let n = 4 and (a1, a2, a3, a4) = (do, if, int, while). Let p(1 : 4) = (3, 3, 1, 1) and
q(0 : 4) = (2, 3, 1, 1, 1). The p's and q's have been multiplied by 16 for
convenience.
Solution:
Initially, we have w(i, i) = q(i), c(i, i) = 0 and r(i, i) = 0, 0 ≤ i ≤ 4.
So,
w(0,0) = q0 = 2
w(1,1) = q1 = 3
w(2,2) = q2 = 1
w(3,3) = q3 = 1
w(4,4) = q4 = 1
Using equations
c(i, j)= w(i, j) + 𝑚i𝑛i+1≤𝑘≤j {c(i, k − 1) + c(k, j)} and
w(i, j) = p( j) + q( j) + w(i, j-1),
we get,
for j-i =1:

w(0, 1) = p(1) +q(1) +w(0,0) = 3+3+2=8
c(0, 1)=w(0,l) +mink=1{c(0,k-1) +c(k, 1)}=w(0,1) + c(0,0) +c(1, 1)}=8 +0 +0 = 8
r(0, 1) = 1
w(1, 2) = p(2)+q(2)+w(1, 1) = 7
c(1, 2) = w(1, 2) +c(1,1) +c(2, 2) = 7
r(1, 2) = 2
w(2, 3) = p(3)+q(3)+w(2,2) = 3
c(2, 3) = w(2,3)+c(2, 2) +c(3, 3) = 3
r(2, 3) = 3
w(3, 4) = p(4)+q(4)+w(3,3) = 3
c(3, 4) = w(3, 4) +c(3,3) +c(4,4) = 3
r(3, 4) = 4
for j-i =2:

w(0, 2) = p(2) +q(2) +w(0,1) = 3+1+8 =12
c(0, 2) = w(0,2) +min{c(0,0) +c(1, 2), c(0,1)+c(2,2)} = 12+min{0+7,8+0}=
12+7=19
r(0, 2) = 1
w(1, 3) = p(3) +q(3) +w(1,2) = 1+1+7 = 9

c(1,3)=w(1,3)+min{c(1,1)+c(2,3),c(1,2)+c(3,3)}=9+min{0+3,7+0}=9+3=12
r(1, 3) = 2
w(2, 4) = p(4) +q(4) +w(2,3) = 1+1+3 = 5

c(2, 4) = w(2,4) +min{c(2,2) +c(3, 4), c(2,3)+c(4,4)} = 5+min{0+3,3+0}=8
r(2, 4) = 3 (or) 4
for j-i =3:

w(0, 3) = p(3) +q(3) +w(0,2) = 14
c(0, 3) = w(0,3) +min{c(0,0) +c(1,3); c(0,1)+c(2,3); c(0,2)+c(3,3)}
=14+min{0+12; 8+3;19+0}=14+11
=25
r(0, 3) = 2
w(1, 4) = p(4)+q(4)+w(1,3) =1+1+9=11

c(1, 4) = w(1, 4) +min {c(1,1) +c(2, 4); c(1,2)+c(3,4); c(1,3)+c(4,4)}
=11+min{0+ 8;7+3; 12+0}

=11+8=19
r(1, 4) = 2
for j-i =4:

w(0, 4) = p(4) +q(4) +w(0,3) =1+1++14= 16
c(0,4)=w(0,4)+min{c(0,0)+c(1,4);c(0,1)+c(2,4);c(0,2)+c(3,4);c(0,3)+c(4,4)}
=16+min{0+19; 8+8; 19+3; 25+0}

=16+16=32
r(0, 4) = 2
Construction of OBST:
→ From the above table, we see that c(0, 4)=32, is the cost of OBST for the
given identifiers. And, the root of the OBST 𝑇0,4 is 𝑎2.
Hence the left subtree is 𝑇0,1 and the right subtree is 𝑇2,4.
→ Tree 𝑇0,1 has root 𝑎1 and subtrees 𝑇0,0 and 𝑇1,1.
→ Tree 𝑇2,4 has root 𝑎3 and its left and right subtrees are 𝑇2,2 and 𝑇3,4.
→ Thus, with the above data it is possible to construct OBST as shown below:
Example:
Use function OBST to compute w(i,j), r(i,j) and c(i,j), 0 ≤ i<j ≤ 4 for the identifier set
(a1,a2,a3,a4)=(cout, float, if, while) with p(1)= 1/20, p(2)=1/5, p(3)= 1/10, and p(4)=1/20,
q(0)=1/5, q(1)= 1/10, q(2)=1/5, q(3)= 1/20, and q(4)=1/20. Using the r(i,j)’s construct the
optimal binary search tree.
Solution:
Let
p(1 : 4) = (1, 4, 2, 1) and q(0 : 4) = (4, 2, 4, 1, 1). The p's and q's have been multiplied by 20 for
convenience.
Solution:
Initially, we have w(i,i) = q(i), c(i,i) = 0 and r(i,i) = 0, 0 ≤i≤4.
So,
w(0,0) =4
w(1,1)=2
w(2,2)=4
w(3,3)=1
w(4,4)=1
Using Equations
c(i, j) = w(i, j) + 𝑚𝑖𝑛𝑖+1≤𝑘≤𝑗 {c(i, k − 1) + c(k, j)}
w(i,j) = p(j) + q(j) + w(i, j - 1), we get
for j - i =1:
w(0, 1) = p(l) +q(1) +w(0,0) = 1+2+4 = 7
c(0, 1) = w(0,l) +min{c(0,0) +c(l,l)} = 7
r(0,l) = 1
w(l,2) = p(2)+q(2)+w(l,l) = 4+4+2 = 10

c(1,2) = w(l, 2) +min {c(l,1) +c(2, 2)} = 10
r(1,2) = 2
w(2,3) = p(3)+q(3)+w(2,2) = 2+1+4 = 7

c(2, 3) = w(2,3)+min {c(2, 2) +c(3, 3)} = 7
r(2,3) = 3
w(3,4) = p(4)+q(4)+w(3,3) = 1+1+1 = 3

c(3, 4) = w(3, 4) +min {c(3,3) +c(4,4)} = 3
r(3,4) = 4
for j - i =2:
w(0, 2) = p(2) +q(2) +w(0,1) = 4+4+7 =15
c(0, 2) = w(0,2) +min{c(0,0) +c(l, 2), c(0,1)+c(2,2)} = 15+ min{0+10,7+0}= 15+7=22
r(0, 2) = 2
w(1, 3) = p(3) +q(3) +w(1,2) = 2+1+10 = 13

c(1, 3) = w(1,3) +min{c(1,1) +c(2, 3), c(1,2)+c(3,3)}=13+min{0+7,10+0}= 13+7=20
r(1, 3) = 2
w(2, 4) = p(4) +q(4) +w(2,3) = 1+1+7 = 9

c(2, 4) = w(2,4) +min{c(2,2) +c(3, 4), c(2,3)+c(4,4)} =9+ min{0+3,7+0}= 9+3=12
r(2, 4) = 3
for j - i =3:
w(0, 3) = p(3) +q(3) +w(0,2) = 2+1+15=18
c(0, 3) = w(0,3) +min{c(0,0) +c(l,3); c(0,1)+c(2,3); c(0,2)+c(3,3)}
=18+min{0+20, 7+7, 22+0}=18+14=32
r(0,3) = 2
w(l,4) = p(4)+q(4)+w(l,3) = 1+1+13=15

c(1,4) = w(l, 4) +min {c(l,1) +c(2, 4); c(1,2)+c(3,4); c(1,3)+c(4,4)}
=15+min{0+12, 10+3, 20+0}=15+12=27
r(1,4) = 2
for j - i =4:
w(0, 4) = p(4) +q(4) +w(0,3) = 1+1+18=20
c(0, 4) = w(0,4) +min{c(0,0) +c(l,4); c(0,1)+c(2,4); c(0,2)+c(3,4); c(0,3)+c(4,4)}
= 20+min{0+27, 7+12. 22+3, 32+0}=20+19=39
r(0,4) = 2
i→ 0 1 2 3 4
j-i
↓
0 w00=4 w11=2 w22=4 w33=1 w44=1
c00=0 c11=0 c22=0 c33=0 c44=0
r00=0 r11=0 r22=0 r33=0 r44=0
1 w01=7 w12=10 w23=7 w34=3
c01=7 c12=10 c23=7 c34=3
r01=1 r12=2 r23=3 r34=4
2 w02=15 w13=13 w24=9
c02=22 c13=20 c24=12
r02=2 r13=2 r24=3
3 w03=18 w14=15
c03=32 c14=27
r03=2 r14=2
4 w04=20
c04=39
r04=2
Construction for OBST:
From the above table, we see that c(0, 4)=39, is the minimum cost of OBST for the given
identifiers. And the root of the OBST 𝑇0,4 is 𝑎2 .
Hence the left subtree is 𝑇0,1 and the right subtree is 𝑇2,4 .
Tree 𝑇0,1 has root 𝑎1 (since r01=1) and subtrees 𝑇0,0 and 𝑇1,1 .
Tree 𝑇2,4 has root 𝑎3 (since r24=3) and subtrees are 𝑇2,2 and 𝑇3,4 .
Tree 𝑇3,4 has root 𝑎4 (since r34=4) and subtrees 𝑇3,3 and 𝑇4,4 .
Thus, with the above data, it is possible to construct OBST as shown below:
a2 float
a1 cout if
a3
=
a4 while
---------------------------------------------------------------------------------------------------------------------
MATRIX CHAIN MULTIPLICATION:
→ Given a sequence of matrices we need to find the most efficient way to

multiply those matrices together.
→ Since the matrix multiplication is associative, we can multiply a chain of

matrices in several ways.
→ Here, we are actually not interested in performing the multiplication, but in

determining in which order the multiplication has to be performed.
→ Let A be an m×n matrix and B be an n×p matrix.
→ The number of scalar multiplications needed to perform A*B is m*n*p. We will

use this number as a measure of the time (or, cost) needed to multiply two
matrices.
→ Suppose we have to compute the matrix product M 1*M2*…*Mn, where Mi has

dimensions 𝑟i × 𝑟i+1, 1≤i≤n.
M1 : 𝑟1 × 𝑟2
M2 : 𝑟2 × 𝑟3
.
.
.
Mn : 𝑟𝑛 × 𝑟𝑛+1
→ We have to determine the order of multiplication that minimizes the total

number of scalar multiplications required.
→ Consider the case n=4. The matrix product M1*M2*M3*M4 may be computed
in any of the following 5 ways.
1. M1*((M2*M3)*M4)
2. M1*(M2*(M3*M4))
3. (M1*M2)*(M3*M4)
4. ((M1*M2)*M3)*M4
5. (M1*(M2*M3))*M4
Example:
Consider three matrices A2x3, B3x4, C4x5.
The product A*B*C can be computed in two ways: (AB)C and A(BC).
The cost of performing (AB)C is: 2*3*4+2*4*5 = 64.
The cost of performing A(BC) is: 3*4*5+2*3*5 = 90.
So, the optimal (i.e., best) order of multiplication is (AB)C.
(OR)
The best way of parenthesizing the given matrix chain multiplication ABC is,
(AB)C.
→The number of different ways in which the product of n matrices may be

4𝑛
computed, increases exponentially with n, that is Ω( 3 ). As a result, the brute
𝑛2
force method of evaluating all the multiplication schemes and select the best one is
not practical for large n.
DYNAMIC PROGRAMING FORMULATION:

→ We can use dynamic programming to determine an optimal sequence of pair
wise matrix multiplications. The resulting algorithm runs in only O(𝑛3) time.
→ Let 𝑀i, j denote the result of the product chain 𝑀i∗ 𝑀i+1∗ …∗𝑀j, i<j.
Ex: M1*M2*M3 = 𝑀1,3.
Thus 𝑀i, i =𝑀i, 1≤ i ≤ n.

Clearly 𝑀i, j has dimensions: 𝑟i × 𝑟j+1.
→ Let c(i, j) be the cost of computing 𝑀i, j in an optimal way.Thus c(i, i) = 0, 1≤ i
≤ n.
→Now in order to determine how to perform the multiplication 𝑀i, j optimally, we

need to make decisions. What we want to do is to break the problem into sub
problems of similar structure.
In parenthesizing the matrix multiplication, we can consider the highest level

(i.e., last level) of parenthesization. At this level, we simply multiply two matrices
together. That is, for any k, (i≤k<j),
𝑀i, j = 𝑀i, k ∗ 𝑀k+1, j
→ Thus, the problem of determining the optimal sequence of multiplications is

broken up into two questions:
1. How do we decide where to split the chain (i.e., what is k)?
2. How do we parenthesize the sub chains (𝑀i∗𝑀i+1∗…∗𝑀𝑘) and (𝑀𝑘+1∗…∗𝑀 j)?
→ In order to compute 𝑀i, j optimally, 𝑀i, k and 𝑀k+1, j should also be computed
optimally. Hence, the principle of optimality holds.
→ The cost of computing 𝑀i, k ( i . e . , 𝑀i∗𝑀i+1∗…∗𝑀𝑘) optimally is c(i, k). The

cost of computing 𝑀k+1, j (i.e., 𝑀𝑘+1∗…∗ 𝑀 j) optimally is c(k+1, j).
→Since 𝑀i, k has dimensions 𝑟i×𝑟𝑘+1 and 𝑀k+1, j has dimensions 𝑟k+1×𝑟j+1, the
cost of multiplying the two matrices 𝑀i, k and 𝑀k+1, j is, 𝑟i* 𝑟𝑘+1* 𝑟j+1.
→ So, the cost of computing 𝑀i, j optimally is,
c(i, j) = c(i, k) + c(k+1, j)+ 𝑟i* 𝑟𝑘+1* 𝑟j+1.
→ But, in the above equation, k (which is the splitting point of matrix product
chain) can take different values based on the inequality condition i≤k<j.
→ This suggests the following recurrence equation for computing c(i, j):
→ The above recurrence equation for c may be solved recursively. The k value
which results in c(i, j) is denoted by kay(i, j).
→ c(1, n) is the cost of the optimal way to compute the matrix product chain 𝑀1,𝑛.
And kay(1, n) defines the last product to be done or where the splitting is done.
→ The remaining products can be determined by using kay values.
The tree of recursive calls of c( ) function for the matrix product chain
M1*M2*M3*M4:
SOLUTION FOR MATRIX CHAIN MULTIPLICATION:
→ The dynamic programing recurrence equation for c may be solved by

computing each c(i, j) and kay(i, j) values exactly once in the order j-i =1, 2, 3, ..,
n-1.
Example: Apply dynamic programing technique for finding an optimal order of
multiplying the five matrices with r =(10, 5,1,10,2,10).
Solution:
Initially, we have 𝑐ii=0 and 𝑘𝑎𝑦ii=0, 1≤ i ≤ 5.
Using the equation:
For j – i = 1:
c(1,2)=𝑚i𝑛1≤ k <2{𝑐(1,1) + 𝑐(2,2) + 𝑟1 ∗ 𝑟2 ∗ 𝑟3}

=0+0+10*5*1=50
kay(1,2)=1
c(2,3)=𝑚i𝑛2≤ k <3{𝑐(2,2) + 𝑐(3,3) + 𝑟2 ∗ 𝑟3 ∗ 𝑟4}

=0+0+5*1*10=50
kay(2,3)=1
c(3,4)=𝑚i𝑛3≤ k <4{𝑐(3,3) + 𝑐(4,4) + 𝑟3 ∗ 𝑟4 ∗ 𝑟5}

=0+0+1*10*2=20
kay(3,4)=3
c(4,5)=𝑚i𝑛4≤ k <5{𝑐(4,4) + 𝑐(5,5) + 𝑟4 ∗ 𝑟5 ∗ 𝑟6}

=0+0+10*2*10=200
kay(4,5)=4
For j – i = 2:
c(1,3)=𝑚i𝑛1≤ k <3{𝑐(1,1) + 𝑐(2,3) + 𝑟1 ∗ 𝑟2 ∗ 𝑟4; 𝑐(1,2) + 𝑐(3,3) +

𝑟1 ∗ 𝑟3 ∗ 𝑟4}
=min{0+50+10*5*10, 50+0+10*1*10}
=min{550,150}
= 150
kay(1,3)=2
c(2,4)=𝑚i𝑛2≤ k <4{𝑐(2,2) + 𝑐(3,4) + 𝑟2 ∗ 𝑟3 ∗ 𝑟5; 𝑐(2,3) + 𝑐(4,4) +
𝑟2 ∗ 𝑟4 ∗ 𝑟5}
=min{0+20+5*1*2, 50+0+5*10*2}
=min{30, 150}
= 30 kay(2,4)=2
c(3,5)=𝑚i𝑛3≤ k <5{𝑐(3,3) + 𝑐(4,5) + 𝑟3 ∗ 𝑟4 ∗ 𝑟6; 𝑐(3,4) + 𝑐(5,5) +

𝑟3 ∗ 𝑟5 ∗ 𝑟6}
=min{0+200+1*10*10, 20+0+1*2*10}
=min{300, 40}
= 40 kay(3,5)=4
For j – i = 3:
c(1,4)=𝑚i𝑛1≤ k <4{𝑐(1,1) + 𝑐(2,4) + 𝑟1 ∗ 𝑟2 ∗ 𝑟5; 𝑐(1,2) + 𝑐(3,4) +

𝑟1 ∗ 𝑟3 ∗ 𝑟5; 𝑐(1,3) + 𝑐(4,4) + 𝑟1 ∗ 𝑟4 ∗ 𝑟5}
=min{0+30+10*5*2, 50+20+10*1*2, 150+0+10*2*2}
=min{130, 90, 190}
= 90 kay(1,4)=2
c(2,5)=𝑚i𝑛2≤ k <5{𝑐(2,2) + 𝑐(3,5) + 𝑟2 ∗ 𝑟3 ∗ 𝑟6; 𝑐(2,3) + 𝑐(4,5) +

𝑟2 ∗ 𝑟4 ∗ 𝑟6; 𝑐(2,4) + 𝑐(5,5) + 𝑟2 ∗ 𝑟5 ∗ 𝑟6}
=min{0+40+5*1*10, 50+200+5*10*10, 30+0+5*2*10}
=min{90,750,130}
= 90 kay(2,5)=2
c(1,5)=𝑚i𝑛1≤ k <5{𝑐(1,1) + 𝑐(2,5) + 𝑟1 ∗ 𝑟2 ∗ 𝑟6; 𝑐(1,2) + 𝑐(3,5) +

𝑟1 ∗ 𝑟3 ∗ 𝑟6; 𝑐(1,3) + 𝑐(4,5) + 𝑟1 ∗ 𝑟4 ∗ 𝑟6; 𝑐(1,4) + 𝑐(5,5) + 𝑟1 ∗
𝑟5 ∗ 𝑟6}
=min{0+90+10*5*10, 50+40+10*1*10, 150+200+10*10*10,
90+0+10*2*10}
=min{590, 190, 1350, 290}
= 190
kay(1,5)=2
→ If k is the splitting point, 𝑀i,j = 𝑀i,𝑘 ∗ 𝑀𝑘+1,j.
→ From the above table, the optimal multiplication sequence has cost 190. The
sequence can be determined by examining kay(1,5), w h ic h i s equal to 2.
→ So, 𝑀1,5= 𝑀1,2 ∗ 𝑀3,5 = (𝑀1 ∗ 𝑀2) ∗ (𝑀3 ∗ 𝑀4 ∗ 𝑀5)
Since kay(3,5) = 4; 𝑀3,5= 𝑀3,4 ∗ 𝑀5,5 = (𝑀3 ∗ 𝑀4) ∗ 𝑀5
So, the optimal order of matrix multiplication is:
𝑀1,5 = 𝑀1 ∗ 𝑀2 ∗ 𝑀3 ∗ 𝑀4 ∗ 𝑀5 = (𝑀1 ∗ 𝑀2) ∗ ((𝑀3 ∗ 𝑀4) ∗ 𝑀5)
Algorithm:
Algorithm MATRIX-CHAIN-ORDER(r)
{
n := length(r) -1; // n denotes number of matrices
for i:= 1 to n do
c[i, i]:= 0;
for l := 2 to n do // l is the chain length
{
for i := 1 to n-l+1 do // n-l+1 gives number of cells in the current row
{
j := i+l-1;
c[i, j] := ∞;
for k := i to j-1 do
{
q := c[i, k] + c[k+1, j] + ri*rk+1*rj+1;
if q < c[i, j] then
{
c[i, j]:= q;
kay[i, j]:= k;
}
}
}
}
return c and kay;
}
UNIT-IV
BACKTRACKING
 Backtracking: Returning to a previous point.

GENERAL METHOD: -
 It is a technique used to solve many difficult combinatorial problems with a large
search space, by systematically trying and eliminating different possibilities.
 Any problem which deals with searching for a set of solutions or which asks for an
optimal solution satisfying some constraints can be solved using the backtracking
formulation.
 In many applications of the backtrack method, the desired solution is expressed as
an n-tuple or a vector (x1, x2, ..., xn), where the values for xi are chosen from some
finite set Si.
 The solution vector is constructed by considering one component after another.
 After considering the first choice of the next component, the partial solution vector
is evaluated against a given criterion.
(i) If the criterion is satisfied, then the next component is included into the solution
vector.
(ii) If the criterion is not satisfied, we will not include that component and there is
no need of considering the remaining components also. In such a case, we
backtrack by considering the next choice for the previous component of the partial
solution vector.
 In this way, the size of the solution search space of a problem can drastically
reduce by using backtracking method when compared to exhaustive search.
State-Space Trees:
 The process of obtaining a solution to a problem using backtracking can be
illustrated with the help of a state space tree.
 The nodes of the state space tree are generated in the depth first order beginning at
the root node.
 The nodes reflect specific choices made for components of a solution vector. The
root node represents an initial state before the search for a solution begins.
 The root node is at first level. The nodes at second level represent the choices
made for the first component of a solution vector. The nodes at third level
represent the choices made for the second component of a solution vector-, and so
on.
 The root node is considered both a live node and an E-node (i.e., expansion node).
At any point of time, only one node is designated as an E-node. From the E-node,
we try to move to (or, generate) a new node (i.e., child of E-node).
 If it is possible to move to a child node from the current E-node (i.e., if there is any
component yet to be included in the solution vector) then that child node will be
generated by adding the first legitimate choice for the next component of a solution
vector and the new node becomes a live node and also the new E-node. The old E-
node remains as a live node.
 At the newly generated node, we apply the constraint function (or, criterion) to
determine whether this node can possibly lead to a solution. If it cannot lead to a
solution (i.e., if it doesn’t satisfy constraints) then there is no point in moving into
any of its subtrees and so this node is immediately killed and we move back (i.e.,
backtrack) to the most recently seen live node (i.e., its parent) to consider the next
possible choice for the previous component of a solution vector. If there is no such
choice, we backtrack one more level up the tree, and so on. The final live node
becomes new E-node.
 Finally, if the algorithm reaches a complete solution, it either stops (if just one
solution is required) or continues searching for other possible solutions.
N-QUEENS PROBLEM: -
 We are given an nxn chessboard and we need to place n queens on the chess board
such that they are non-attacking each another (i.e., no two queens should lie on the
same row, or same column, or same diagonal).
 Due to the first two restrictions, it is clear that each row and column of the board
will have exactly one queen.
 Let us number the rows and columns of the chessboard 1 through n. The queens
can also be numbered 1 through n.
 Since, each queen must be on a different row, we can assume that queen i will be
placed on row i.
 Each solution to the n-queens problem can therefore be represented as an n- tuple
(x1, x2, …, xn) where xi is the column number on which queen i is placed.
 The xi values should be distinct since no two queens can be placed on the same
column.
HOW TO TEST WHETHER TWO QUEENS ARE ON SAME DIAGONAL?

 We observe that all the elements on the same diagonal that runs from the upper-left
to the lower-right have the same row-column value.
 Also, all the elements on the same diagonal that runs from upper-right to lower-left
have the same row+column value.
 Suppose two queens are placed at positions (i, j) and (k, l). Then they are on the
same diagonal only if
i-j = k-l (or) i+j = k+l
By rearranging the terms, we have
j-l = i-k (or) j-l = k-i
 Therefore, two queens lie on the same diagonal if and only if |j-l| = |i-k|.
Algorithm for n-queens problem using backtracking:

Algorithm NQueens (k, n)
// using backtracking, this procedure prints all possible placements of n queens on
//an n×n chessboard so that they are non-attacking.
{
for i := to n do // for each column i from 1 to n
{
if (place(k, i)) then // check whether kth queen can be placed on column i
{
x[k] :=i;
if(k=n) then write(x[1: n]);
else NQueens(k+1, n);
}
}
}
Algorithm place(k, i)
//returns true if a queen can be placed in kth row and ith column, otherwise it
//returns false.
// x[] is a global array whose first k-1 values have been set.
//Abs(r) returns absolute value of r.
{
for j:=1 to k-1 do
{
if(x[j]=i) or (Abs(x[j]-i)=Abs(j-k)) then
return false;
}
return true;
}
 The NQueens algorithm is invoked as NQueens(1, n).

Portion of the State-space tree generated for solving 4-Queens problem: -
Graph Coloring (or) m-coloring problem: -
Given an undirected graph G and a positive integer m, determine if the graph can
be colored with at most m colors such that no two adjacent vertices of the graph
have the same color. This is called m-coloring problem. This is also known as m-
colorability decision problem.
 Here coloring of a graph means the assignment of colors to all vertices.
 If the solution exists, then display which color is assigned to which vertex.
----------------------------------------------------------------------------------------------------
Side points:
1. m-colorability optimization problem: -
This problem asks for the smallest integer m for which the graph G can be colored.
This integer is referred to as the chromatic number of the graph.
2. Chromatic number of a graph: - Minimum number of colors required to color a
given graph such that no two adjacent vertices have same color.
3. If d is the maximum degree in a given graph, then its chromatic number ≤ (d+1).
Example: Find the chromatic number of the following graphs.
A B A B
D C D C
E
 Let us number the vertices of the graph 1 through n and the colors 1 through m.
 So, a solution to the graph coloring problem can be represented as an n-tuple (x1,
x2, …, xn) where xi is the color of vertex i.
Problem: Find all possible ways of coloring the below graph with m=3.
A B
D C
Solution:
Let A=1, B=2, C=3, and D=4.
Portion of State-space tree generated for solving 3-coloring problem: -
Some of the solutions are: (x1, x2, x3, x4) = (1, 2, 1, 2)

= (1, 2, 1, 3)
= (1, 2, 3, 2)
= (1, 3, 1, 2)
= (1, 3, 1, 3)
= (1, 3, 2, 3)
Algorithm for m-coloring problem using backtracking:

 Suppose we represent a graph by its adjacency matrix G[1:n, 1:n] where G[i, j]=1
if (i, j) is an edge and G[i, j]=0 otherwise.
 Initially, the array x[] is set to zero.
Algorithm mColoring(k)
// This algorithm was formed using the recursive backtracking scheme.
// The graph is represented by its Boolean adjacency matrix G[1:n,1:n].
// All possible assignments of 1, 2, ..., m to the vertices of the graph are printed,
// such that adjacent vertices are assigned distinct integers.
// k is the index of the next vertex to color.
{
while(TRUE)
{
// Generate all legal assignments for x[k].
NextValue(k); // Assign a legal color to x[k].
if (x[k] = 0) then return; //No new color is possible
if (k=n) then //At most m colors have been used to color the n vertices.
write (x[1:n]);
else mColoring(k+1);
}
}
Algorithm NextValue(k)
// x[1], ..., x[k - 1] have been assigned integer values in the range [1, m] such that
// adjacent vertices have distinct integers.
// A value for x[k] is determined in the range [0, m].
// x[k] is assigned the next highest numbered color while maintaining distinctness
// from the adjacent vertices of vertex k. If no such color exists, then x[k] is 0.
{
while(TRUE)
{
x[k] := (x[k]+1) mod (m+1); //Next highest color.
if {x[k] = 0) then return; // All colors have been used.
for j:=1 to n do
{
// Check if this color is distinct from adjacent colors.
if ((G[k,j]≠0) and (x[k] = x[j])) //If (k, j) is an edge and if adjacent
// vertices have the same color.
then break;
}
if (j=n+1) then return; // New color found
} // Otherwise try to find another color.
}
 mColoring algorithm is initially invoked as mColoring(1).

Planar graph:
 A graph is said to be planar iff it can be drawn on a plane in such a way that no two
edges cross each other.
 The chromatic number of a planar graph is not greater than 4.
 Suppose we are given a map, then it can be converted into planar graph as follows:
Consider each region of the map as a node. If two regions are adjacent, then the
corresponding nodes are joined by an edge.
 Consider the following map with five regions:
The corresponding planar graph is:

HAMILTONIAN CYCLES: -
 Let G = (V, E) be a connected graph with n vertices.
 A Hamiltonian cycle is a round-trip path along n edges of G that visits every vertex
once and returns to its starting position.
 In other words, if a Hamiltonian cycle begins at some vertex v1 ϵ G and the vertices
of G are visited in the order v1, v2, …, vn+1, then the edges (vi, vi+1) are in E, 1≤i≤n,
and vi are distinct except for v1 and vn+1 which are equal.
Example:
Consider the following graph:
This graph has the following Hamiltonian cycle:
1—3—4—5—6—7—8—2—1
Note: If a graph has an articulation point, then there will be no Hamiltonian cycles.
Example: The following graph doesn’t contain Hamiltonian cycle.
 We can write a backtracking algorithm that finds all the Hamiltonian cycles in a
graph. We will output only distinct cycles.
 We assume that the vertices of the graph are numbered from 1 to n.
 The backtracking solution vector (x1, x2, …, xn) is defined so that xi represents the
ith visited vertex of the proposed cycle.
 The graph is represented by its adjacency matrix G[1:n, 1:n].
 x[2:n] are initialized to zero. And, x[1] is initialized to 1 because we assume that
cycles start from vertex 1.
 For 2≤k≤n-1, xk can be assigned any vertex v in the range from 1 to n provided it is
distinct from x1, x2, …, xk-1 and there exists an edge between v and xk-1.
 Now, xn can be assigned the remaining vertex, provided there exists an edge to it
from both x1 and xn-1.
Algorithm for Hamiltonian Cycles problem using backtracking:
Algorithm Hamiltonian(k)
// This algorithm uses the recursive formulation of backtracking to find all the
// Hamiltonian cycles of a graph.
// The graph is stored as an adjacency matrix G[1:n, 1:n].
// All cycles begin at node 1.
{
while(TRUE)
{
// Generate values for x[k].
NextValue(k); // Assign a legal next value to x[k].
if (x[k] = 0) then return;
if (k = n) then write (x[1:n]);
else Hamiltonian(k+1);
}
}
Algorithm NextValue(k)
// x[1: k - 1] is a path of k – 1 distinct vertices. If x[k] =0, then no vertex has as yet
// been assigned to x[k].
// After execution, x[k] is assigned to the next highest numbered vertex which
// does not already appear in x[1:k - 1] and is connected by an edge to x[k - 1].
// Otherwise, x[k] =0.
// If k = n, then in addition, x[k] is connected to x[1].
{
while(TRUE)
{
x[k] := (x[k]+1) mod (n+1); // Next vertex.
if (x[k] = 0) then return;
if (G[x[k-1], x[k]] ≠ 0) then // Is there an edge?
{
for j:= 1 to k – 1 do // Check for distinctness.
if (x[j] = x[k]) then break;
if (j = k) then //If true, then the vertex is distinct.
if ((k < n) or ((k = n) and G[x[n], x[1]] ≠ 0)) then
return;
}
}
}
 The algorithm Hamiltonian is initially invoked as Hamiltonian(2).
Example: Find the Hamiltonian cycles for the following graph using backtracking.
Solution:
Portion of State-space tree generated for solving Hamiltonian cycles problem: -
Example: Find the Hamiltonian cycles for the following graph using backtracking.
Solution:
Portion of State-space tree generated for solving Hamiltonian cycles problem: -
SUM OF SUBSETS: -
 Suppose we are given n distinct positive numbers (usually called weights) wi,
1≤i≤n and we need to find all combination (subsets) of these numbers whose sums
are equal to a given integer m.
 Each solution subset is represented by an n-tuple (x1, x2, …, xn) such that xi ∈ {0,1},
1 ≤ i ≤ n.
If wi is not included in subset, then xi =0.
If wi is included in subset, then xi =1.
Example: For n=4, (w1, w2, w3, w4) = (5,3,4,6), and m=9,
(x1, x2, x3, x4) = (1,0,1,0)
= (0,1,0,1)
Size of solution search space is 2×2×2×2×… n times = 2n.
Note: In the above figure, the number in the circle denotes the sum of the weights
considered till now.
EFFICIENT BACKTRACKING SOLUTION FOR SUM OF SUBSETS: -

 We assume that wi’s are initially in non-decreasing order.
 At each stage, we have two choices for xi’s, i.e., 0 and 1.
 In the state space tree, for a node at level i, left child corresponds to xi=1, right
child corresponds to xi=0.
 Suppose we have fixed the values of x1, x2, …, xk-1. Now, if we choose xk=1, then
the following constraints have to be satisfied:
∑ 𝑤𝑥 +∑ 𝑤 ≥m
and ∑ 𝑤 𝑥 + wk+1 ≤ m.
 If the above constraints are not satisfied then we will not proceed further in state
space tree and we will backtrack and make xk=0.
 We will use two variables s and r as follows:
s=∑ 𝑤 𝑥
r=∑ 𝑤
Algorithm for Sum of Subsets problem using backtracking:

 The algorithm is initially invoked as SumOfSub(0, 1, ∑ 𝑤 ).
 The solution vector (x1, x2, …, xn) is initialized to zero.
Algorithm SumOfSub(s, k, r)
// find all subsets of w[1: n] that sum to m.
// The value of x[j], 1≤ j≤ k-1 have already been determined.
// At this point of time, s=∑ 𝑤[𝑗]𝑥[𝑗] and r=∑ 𝑤[𝑗] .
// w[j]’s are in non-descending order.
// It’s assumed that w[1]≤ m and ∑ 𝑤[𝑖] ≥ m
{
// generate left child
x[k]:=1;
if(s+w[k]=m) then // subset found
write(x[1: n]);
else if(s+w[k]+w[k+1] ≤ m) then
SumOfSub(s+w[k], k+1, r-w[k]);
// otherwise generate right child.
if((s+r-w[k] ≥ m) and (s+w[k+1] ≤ m)) then
{
x[k]:=0;
SumOfSub(s, k+1, r-w[k]);
}
}
Example:
Let w= {3,4,5,6} and m=9. Find all possible subsets of w that sum to m. Draw the
portion of the state space tree that is generated.
Solution:
Note: In the state space tree, the rectangular nodes list the values of s, k, and r on
each call to SumOfSub. Initially s=0, k=1, r=18.
Portion of the state space tree that is generated:
Example: Let w = {5, 7, 10, 12, 15, 18, 20} and m=35. Find all possible subsets of w that sum
to m. Draw the portion of the state space tree that is generated.
Solution:
Initially, sum of weights, r=5+7+10+12+15+18+20=87.
One solution is: (x1, x2, x3, x4, x5, x6, x7) = (1,0,1,0,0,0,1).
Example:
Let w= {5,10,12,13,15} and m=30. Find all possible subsets of w that sum to m.
Draw the portion of the state space tree that is generated.
Solution:
Note: In the state space tree, the rectangular nodes list the values of s, k, and r on
each call to SumOfSub. Answer nodes are represented in circles. Initially s=0,
k=1, r=73.
Branch and Bound
 This technique is mostly used to solve optimization problems.

 In branch and bound, a state space tree is constructed in such a way that all
children of an E-node are generated before any other live node becomes an E-node.
 Generated nodes that cannot possibly lead to a feasible solution are discarded. The
remaining nodes are added to the list of live nodes, and then one node from this list
is selected to become the next E-node. This expansion process continues until
either the answer node is found or the list of live nodes becomes empty.
 The next E-node can be selected in one of three ways:
1. FIFO (or) Breadth First Search: This scheme extracts nodes from the list of live
nodes in the same order as they are placed in it. The live nodes list behaves as a
queue.
2. LIFO (or) D Search: The live nodes list behaves as a stack.
3. LC Search (Least Cost Search) (or) Best First Search: The nodes are assigned
ranks based on certain criteria and they are extracted in the order of Best-Rank-
First.
 Brach and Bound involves two iterative steps:

1. Branching:
Splitting the problem into a number of subproblems.
(or)
Generating all the children of an E-node in the state space tree.
2. Bounding:
Finding an optimistic estimate of the best solution to the subproblem.
Optimistic estimate: Upper bound for maximization problems.

Lower bound for minimization problems.
 In case of LC Search Brach and Bound (LCBB) method, after generating the
children of an E-node, the node with the best bound value (i.e., smallest lower
bound in case of minimization problems or largest upper bound in case of
maximization problems) is chosen from the list of all live nodes and is made the
next E-node.
 We terminate the search process at the current node of an LCBB algorithm because
of any one of the following reasons:
1. The node represents an infeasible solution because constraints are not satisfied.
2. The bound value of the node is not better than the value of the best solution seen
so far.
0/1 Knapsack problem:

Given n items with profits (p1, p2, …, pn) and weights (w1, w2, …, wn) and
knapsack capacity M.
maximize ∑𝑛𝑖=1 𝑝𝑖 𝑥𝑖
Subject to the constraints
∑𝑛𝑖=1 𝑤𝑖 𝑥𝑖 ≤ M
and
𝑥𝑖 ∈{0,1}, 1 ≤ i ≤ n.
Solution to the 0/1 Knapsack problem using LCBB:

 0/1 knapsack problem is maximization problem.
How to find the optimistic estimate (i.e., upper bound)?

We relax the integral constraint, i.e., 𝑥𝑖 ∈{0,1}, 1 ≤ i ≤ n. That means, fractions of
the items are allowed.
 Arrange the items in the decreasing order of profit densities (pi/wi values).
 In the state space tree, at every node we record three values, viz.,
W: Sum of the weights of the objects considered till now
P: Sum of the profits of the objects considered till now
UB: Upper bound on the optimal profit
 The upper bound is computed as follows:

Upper Bound = Sum of the profits of the items provided the total weight is less
than or equal to knapsack capacity considering the fractions of items.
Example:
n=4; (𝑤1, 𝑤2, 𝑤3, w4) = (4,7,5,3); (𝑝1, 𝑝2, 𝑝3, p4) = (40,42,25,12) and M=10.
Solution:
i 1 2 3 4
Pi 40 42 25 12
wi 4 7 5 3
pi/wi 10 6 5 4
The optimal solution is: (x1, x2, x3, x4) = (1,0,1,0)
Exercise:
n=3 ; (𝑤1, 𝑤2, 𝑤3) = (2,1,3) ; (𝑝1, 𝑝2, 𝑝3,p4) = (10,4,6) and M=5.
Traveling Salesman Problem using LCBB:
 Let G = (V, E) be a directed graph defining an instance of the traveling salesman

problem.
 We assume that the tour starts at node 1 and ends at node 1. The graph is
represented by its cost adjacency matrix C, where C[i, j] denotes the cost of the
edge <i, j>. If <i, j> ∉ E or if i=j, then C[i, j]= ∞.
 In the state space tree, a lower bound on the optimal tour length is maintained at
each node.
 The lower bound can be obtained by using the reduced cost matrix.
Reduced cost matrix:
A matrix is said to be reduced iff all its rows and columns are reduced.
A row is said to be reduced iff it contains at least one zero and all remaining
elements are non-negative. Similarly, A column is said to be reduced iff it contains
at least one zero and all remaining elements are non-negative.
 We associate a reduced cost matrix with every node in the state space tree using
which we compute the lower bound at that node.
Obtaining the reduced cost matrix and lower bound at root node:
1. In each row of the given cost adjacency matrix, subtract the minimum value of
that row from all the entries of that row.
2. In each column, subtract the minimum value of that column from all the entries
of that column.
3. Lower Bound at root node = Total amount subtracted from all rows and
columns.
Obtaining the reduced cost matrix and lower bound at other nodes in the spate
space tree:
 Let A be the reduced cost matrix for node R in the state space tree. Let S be a child
of R such that the tree edge <R, S> corresponds to including edge <i, j> in the
tour.
 If S is not a leaf node, then the reduced cost matrix for S can be obtained as
follows:
1. Change all entries in row i and column j of A to ∞ and set A[j, 1] to ∞.
2. Reduce all rows and columns in the matrix obtained in step-1 except for those
rows and columns which contain only ∞.
3. If r is the total amount subtracted in step-2, then
LB(S)=LB(R)+ A[i, j]+r.
Example: Solve the following TSP instance using LCBB.
Solution:
The optimal tour is: 1 — 4 — 2 — 5 — 3 — 1
UNIT-V
DETERMINISTIC ALGORITHMS:
The algorithms, in which the result (or, outcome) of every operation is
uniquely defined, are called deterministic algorithms.
NON-DETERMINISTIC ALGORITHMS:
 The algorithms, in which the outcomes of certain operations
may not be uniquely defined but are limited to the specified sets of
possibilities (i.e., possible outcomes), are said to be non-deterministic
algorithms.
 The theoretical (or, hypothetical) machine executing such
operations is allowed to choose any one of these possible outcomes.
 The non-deterministic algorithm is a two-stage algorithm.

1. Non-deterministic stage (or, Guessing stage):
Generate an arbitrary string that can be thought of as a
candidate solution to the problem.
2. Deterministic stage (or, Verification stage):

This stage takes the candidate solution and the problem
instance as input and returns “yes” if the candidate solution
represents actual solution.
 To specify non-deterministic algorithms, we use three functions:

1. Choice(S): arbitrarily chooses one of the elements of set ‘S’.
2. Success( ): signals a successful completion.
3. Failure( ): signals an unsuccessful completion.
 The assignment statement x: = Choice(1, n) could result in x

being assigned with any one of the integers in the range [1, n].
There is no rule specifying how this choice is to be made. That’s

why the name non-deterministic came into picture.
 The Failure( ) and Success( ) signals are used to define a
completion of the algorithm.
 Whenever there is a particular choice (or, set of choices (or)

sequence of choices) that leads to a successful completion of the
algorithm, then that choice (or, set of choices) is always made
and the algorithm terminates successfully.
 A nondeterministic algorithm terminates unsuccessfully, if and

only if there exists no set of choices leading to a success signal.
 The computing times for Choice( ), Failure( ), Success( ) are

taken to be O(1), i.e., constant time.
 A machine capable of executing a non-deterministic algorithm is

called non-deterministic machine.
EXAMLE: 1: NON-DETERMINISTIC SEARCH:
Algorithm Nsearch(A, n, x)
{
//A[1:n] is a set of elements, from which we have to determine
//an index j, such that A[j]:=x, or 0 if x is not present in A.
// Guessing Stage
j := Choice(l, n);
// Verification Stage
if A[j] = x then
{
write(j);
Success( );
}
write(0);
Failure( );
}
 The time complexity is O(1)
EXAMLE 2: NON-DETERMINISTIC SORTING:

Algorithm NSort(A, n)
// A[1:n] is an array that stores n elements, which are positive
//integers.
// B[1:n] is an auxiliary array, in which elements are put at
//appropriate positions. That means, B stores the sorted elements.
{
// guessing stage
for i := 1 to n do
{
j := Choice(l, n); //guessing the position of A[i] in B
B[j] := A[i]; //place A[i] in B[j]
}
// verification stage
for i:= 1 to n -1 do
{
if (B[i] > B[i+ 1]) then // if not in sorted order.
Failure( );
}
Write(B[l : n]); // print sorted list.
Success( );
}
Time complexity of the above algorithm is O(n).

------------------------------------------------------------------------------------
 We mainly focus on nondeterministic decision algorithms.

Such algorithms produce either ‘1’ or ‘0’ (or, Yes/No) as their output.
In these algorithms, a successful completion is made iff the output
is 1. And, a 0 is output, iff there is no choice (or, sequence of choices)
available leading to a successful completion.
The output statement is implicit in the signals Success( ) and
Failure( ). No explicit output statements are permitted in a decision
algorithm.
EXAMPLE: 0/1 KNAPSACK DECISION PROBLEM:

The knapsack decision problem is to determine if there is an
assignment of 0/1 values to xi, 1≤i≤ n such that ∑ ≥ r and
∑ ≤ M. r is a given number. The pi’s and wi’s nonnegative
numbers.
1 Algorithm DKP(p, w, n, M, r, x)
2{
3 W:= 0; P:= 0;
4 for i := 1 to n do
5 {
6 x[i]:= Choice(0, 1);
7 W := W + x[i] * w[i];
P:=P+ x[i] * p[i];
8 }
9 if ((W>M) or (P < r)) then Failure( );
10 else Success( );
11 }
------------------------------------------------------------------------------------
THE CLASSES P, NP, NP-HARD AND NP-COMPLETE:
P is the set of all decision problems solvable by a deterministic
algorithm in polynomial time.
 An algorithm A is said to have polynomial complexity (or,
polynomial time complexity) if there exists a polynomial p( ) such that
the computing time of A is O(p(n)) for every input of size n.
NP (Nondeterministic Polynomial time):
NP is the set of all decision problems solvable by a
nondeterministic algorithm in polynomial time.
A non-deterministic machine can do everything that a deterministic
machine can do and even more. This means that all problems in class
P are also in class NP. So, we conclude that P ⊆NP.
What we do not know, and perhaps what has become the most
famous unsolved problem in computer science is, whether P = NP or
P ≠ NP.
The following figure displays the relationship between P and NP
assuming that P ≠NP.
PN NP
Some example problems in NP:

1. Satisfiability (SAT) Problem:
SAT problem takes a Boolean formula as input, and asks whether
there is an assignment of Boolean values (or, truth values) to the
variables so that the formula evaluates to TRUE.
A Boolean formula is a parenthesized expression that is formed
from Boolean variables and Boolean operators such as OR, AND,
NOT, IMPLIES, IF-AND-ONLY-IF.
A Boolean formula is said to be in CNF (Conjunctive Normal
Form, i.e., Product of Sums form) if it is formed as a collection of sub
expressions called clauses that are combined using AND, with each
clause formed as the OR of Boolean literals. A literal is either a
variable or its negation.
The following Boolean formula is in CNF:
( ∨ ∨ ∨ )∧( ∨ ) ∧( ∨ )
The following formula is in DNF (Sum of Products form):
( ∧ ∧ )∨( ∧ ) ∨( ∧ )
 CNF-SAT is the SAT problem for CNF formulas.
It is easy to show that SAT is in NP, because, given a Boolean
formula E(x1, x2, …, xn), we can construct a polynomial time non-
deterministic algorithm that could proceed by simply choosing
(nondeterministically) one of the 2n possible assignments of truth
values to the variables (x1, x2, …, xn) and verifying that the formula
E(x1, x2, …, xn) is true for that assignment..
Nondeterministic Algorithm for SAT problem:
Algorithm NSAT(E, n)
{
//Determine whether the propositional formula E is satisfiable.
//The variables are x1, x2, …, xn.
// guessing stage.
for i:=1 to n do // Choose a truth value assignment.
xi := Choice(false, true);
// verification stage.
if E(x1, x2, …, xn) = true then Success( );
else Failure( );
}
Time complexity is O(n), which is a polynomial time. So, SAT is
NP problem.
2. CLIQUE PROBLEM:
Clique: A clique of a graph ‘G’ is a complete subgraph of G.

The size of the clique is the number of vertices in it.
Clique problem: Clique problem takes a graph ‘G’ and an integer ‘k’
as input, and asks whether G has a clique of size at least ‘k’.
Nondeterministic Algorithm for Clique Problem:
Algorithm DCK(G, n, k)
{
//The algorithm begins by trying to form a set of k distinct
//vertices. Then it tests to see whether these vertices form a
//complete sub graph.
// guessing stage.
S := Ø; // S is an initially empty set.
for i := 1 to k do
{
t := Choice(l, n);
S := S U {t} // Add t to set S.
}
//At this point, S contains k distinct vertex indices.
//Verification stage
for all pairs (i, j) such that i ∈ S, j ∈ S, and i ≠ j do
if (i, j) is not an edge of G then Failure( );
Success( );
}
A nondeterministic algorithm is said to be nondeterministic
polynomial if the time complexity of its verification stage is
polynomial.
Tractable Problems: Problems that can be solved in polynomial
time are called tractable.
Intractable Problems: Problems that cannot be solved in polynomial
time are called intractable.
Some decision problems cannot be solved at all by any algorithm.

Such problems are called undecidable, as opposed to decidable
problems that can be solved by an algorithm.
A famous example of an undecidable problem was given by Alan
Turing in 1936. It is called the halting problem: given a computer
program and an input to it, determine whether the program will halt
on that input or continue working indefinitely on it.
REDUCIBILITY:
 A decision problem D1 is said to be polynomially reducible to a
decision problem D2 (also written as ∝ ), if there exists a
function t that transforms instances of D1 into instances of D2 such
that:
1. t maps all Yes instances of D1 to Yes instances of D2 and all No
instances of D1 to No instances of D2.
2. t is computable by a polynomial time algorithm.
The definition for ∝ immediately implies that if D2 can be
solved in polynomial time, then D1 can also be solved in polynomial
time. In other words, if D2 has a deterministic polynomial time
algorithm, then D1 can also have a deterministic polynomial time
algorithm.
Based on this, we can also say that, if D2 is easy, then D1 can also be
easy. In other words, is as easy as . Easiness of D2 proves the
easiness of D1.
But, here we mostly focus on showing how hard a problem is
rather than how easy it is, by using the contra positive meaning of the
reduction as follows:
∝ implies that if D1 cannot be solved in polynomial time, then
D2 also cannot be solved in polynomial time. In other words, if D1
does not have a deterministic polynomial time algorithm, then D2 also
can not have a deterministic polynomial time algorithm.
We can also say that, if D1 is hard, then D2 can also be hard. In other
words, D2 is as hard as D1.
To show that problem D1 (i.e., new problem) is at least as hard as
problem D2 (i.e., known problem), we need to reduce D2 to D1 (not D1
to D2).
Reducibility (∝) is a transitive relation, that is, if ∝ and
∝ then ∝ .
NP-HARD CLASS:
A problem ‘L’ is said to be NP-Hard iff every problem in NP
reduces to ‘L’
(or)
 A problem ‘L’ is said to be NP-Hard if it is as hard as any problem
in NP.
(or)
A problem ‘L’ is said to be NP-Hard iff SAT reduces to ‘L’.
Since SAT is a known NP-Hard problem, every problem in NP can be
reduced to SAT. So, if SAT reduces to L, then every problem in NP
can be reduced to ‘L’.
Ex: SAT and Clique problems.
An NP-Hard problem need not be NP problem.

Ex: Halting Problem is NP-Hard but not NP.
NP-COMPLETE CLASS:
A problem ‘L’ is said to be NP-Complete if ‘L’ is NP-Hard and L ∈
NP.
These are the hardest problems in NP set.
Ex: SAT and Clique problems.
Showing that a decision problem is NP-complete:
It can be done in two steps:
Step1:
Show that the problem in question is in NP; i.e., a randomly generated
string can be checked in polynomial time to determine whether or not
it represents a solution to the problem. Typically, this step is easy.
Step2:
Show that the problem in question is NP-Hard also. That means, show
that every problem in NP is reducible to the problem in question, in
polynomial time. Because of the transitivity of polynomial reduction,
this step can be done by showing that a known NP-complete problem
can be transformed into the problem in question, in polynomial time,
as depicted in the figure below.
The definition of NP-completeness immediately implies that if
there exists a polynomial-time algorithm for just one NP-Complete
problem, then every problem in NP can also have a polynomial time
algorithm, and hence P = NP.
Relationship among P, NP, NP-Hard and NP-Complete Classes:
COOK’S THEOREM:
Cook’s theorem can be stated as follows.
(1) SAT is NP-Complete.
(or)
(2) If SAT is in P then P = NP. That means, if there is a
polynomial time algorithm for SAT, then there is a polynomial
time algorithm for every other problem in NP.
(or)
(3) SAT is in P iff P = NP.
Application of Cook’s Theorem:
A new problem ‘L’ can be proved NP-Complete by reducing SAT to
‘L’ in polynomial time, provided ‘L’ is NP problem. Since SAT is
NP-Complete, every problem in NP can be reduced to SAT. So, once
SAT reduces to ‘L’, then every problem in NP can be reduced to ‘L’
proving that ‘L’ is NP-Hard. Since ‘L’ is NP also, we can say that ‘L’
is NP-Complete.
------------------------------------------------------------------------------------
Example Problem: Prove that Clique problem is NP-Complete.
(OR)
Reduce SAT problem to Clique problem.
Solution: See the video at https://www.youtube.com/watch?v=qZs767KQcvE
------------------------------------------------------------------------------------

Combinepdf

Uploaded by

Copyright:

Available Formats

Combinepdf

Uploaded by

Document Information

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Combinepdf

Uploaded by

Copyright:

Available Formats

UNIT-III

All-Pairs Shortest Paths problem: -

Time complexity, T(n)=O(n3).

Step-1: A0 ← c Step-2: Using, A1[i, j] = min{A0[i, j], A0[i, 1]+A0[1, j]}

All-Pairs Shortest Paths:

Let G(V, E) be the directed graph with n vertices.

Find optimal sales person tour from vertex1.

EXAMPLE: - C=100, n=3, C1=15, C2=20, C3=25.

Solution to the Reliability Design problem using backward recurrence equation:

f(n, C)= 𝑚𝑎𝑥 {𝛷 (𝑚 )𝑓(𝑛 − 1, 𝐶 − 𝐶 𝑚 )}

For any, f(i, x), i≥1, this equation generalizes to

f(i, x)= 𝑚𝑎𝑥 {𝛷 (𝑚 )𝑓(𝑖 − 1, 𝑥 − 𝐶 𝑚 )}

Clearly, f(0, x) =1 for all x, 0≤ x≤C.

Solution using Sets with Ordered pairs: -

Sji,1≤ j≤ ui can be obtained from all tuples (f, x) of Si-1 as follows:

Sji= {(f * 𝛷 (j), x+j*𝐶 )}, 1≤ j ≤ ui.

Obtaining S2 from S1: -

Obtaining S3 from S2:-

Tracing back the values of mi :

0/1 KNAPSACK PROBLEM:

f(2,116) = 𝑚𝑎𝑥{15, (18 + 15)} = 𝑚𝑎𝑥{15, 33} =33

Example: Consider the knapsack instance:

A dynamic-programming approach runs in polynomial time when the number of

There are usually two equivalent ways to implement a dynamic-programming

Solution to 0/1 Knapsack problem using Top-down Dynamic Programming

Let us consider the backward recursive equation.

Let us rewrite the above backward recursive equation as follows:

𝑚𝑎𝑥(𝑓 (𝑖 − 1, 𝑦), (𝑝𝑖 + , 𝑓(𝑖 − 1, 𝑦 − 𝑤𝑖 ))), 𝑖𝑓 𝑦 ≥ 𝑤𝑖

Tracing back values of x[i]:

for i:=n to 1 step -1 do

f(1,10) = max(f(0,10), 6+f(0,8)) = max(0, 6+0) = 6

1 0 -1(0) -1(6) -1(6) -1(6) -1(6) -1(6) -1 -1(6) -1 -1(6)

Let us consider backward recursive equation as follows:

𝑚𝑎𝑥(𝑓 (𝑖 − 1, 𝑦), (𝑝𝑖 + , 𝑓(𝑖 − 1, 𝑦 − 𝑤𝑖 ))), 𝑖𝑓 𝑦 ≥ 𝑤𝑖

Column indices (y values)

return T[n, M];

This function is invoked as KnapSack(n, M).

The space complexity is θ(nM).

Tracing back values of x[i]:

for i:=n to 1 step -1 do

Example: consider the case n=5; M=10; ; (𝑤1, 𝑤2 , 𝑤3, 𝑤4 , 𝑤5 ) = (2,2,6,5,4) ;

∑ ( level (𝑎𝑖 ) ∗ 𝑝𝑖 ) + ∑(( 𝑙𝑒𝑣𝑒𝑙 (𝑑𝑖 ) − 1) ∗ 𝑞𝑖 )

EXPECTED SEARCH COST FOR TREE (b):

= (2*0.5 + 1*0.1 + 2*0.05) + ((3-1)*0.15 + (3-1)*0.1 + (3-1)*0.05 + (3-1)*0.005)

EXPECTED SEARCH COST FOR TREE (c):

EXPECTED SEARCH COST FOR TREE (d):

EXPECTED SEARCH COST FOR TREE (e):

= (1*0.5 + 3*0.1 + 2*0.05) + ((2-1)*0.15 + (4-1)*0.1 + (4-1)*0.05 + (3-1)*0.05)

The tree (c) has the minimum cost. So, it is OBST.

APPLYING DYNAMIC PROGRAMING TO SOLVE OBST PROBLEM:

→ To apply dynamic programing to the problem of obtaining an OBST, we need

→ Then for any 𝑎i, i<k, we have:

𝐶𝑜𝑠𝑡(𝑇) = ∑( 𝑙𝑒𝑣𝑒𝑙 𝑇 (𝑎𝑖 ) ∗ 𝑝𝑖 ) + ∑(( 𝑙𝑒𝑣𝑒𝑙 𝑇 (𝑑𝑖 ) − 1) ∗ 𝑞𝑖 )

𝐶𝑜𝑠𝑡(𝐿) = ∑ ( 𝑙𝑒𝑣𝑒𝑙𝐿 (𝑎𝑖 ) ∗ 𝑝𝑖 ) + ∑(( 𝑙𝑒𝑣𝑒𝑙𝐿 (𝑑𝑖 ) − 1) ∗ 𝑞𝑖 )

𝐶𝑜𝑠𝑡(𝑅) = ∑ ( 𝑙𝑒𝑣𝑒𝑙𝑅 (𝑎𝑖 ) ∗ 𝑝𝑖 ) + ∑(( 𝑙𝑒𝑣𝑒𝑙𝑅 (𝑑𝑖 ) − 1) ∗ 𝑞𝑖 )

= pk+[ ∑𝑘−1 𝑘−1

Cost(T)= pk+ Cost(L)+ Cost(R)+ w(0, k-1) + w(k, n)  (2)

Then for the tree T0, n to be optimal,

→ Since 𝑝𝑘 + w(i, k-1) + w(k, j) = w(i, j),

= (20.5 + 10.1 + 20.05) + ((3-1)0.15 + (3-1)0.1 + (3-1)0.05 + (3-1)*0.005)

= (10.5 + 30.1 + 20.05) + ((2-1)0.15 + (4-1)0.1 + (4-1)0.05 + (3-1)*0.05)

→ The number of scalar multiplications needed to perform AB is mn*p. We will

→ Suppose we have to compute the matrix product M 1M2…*Mn, where Mi has