BRANCH AND BOUND

The Branch and Bound Technique is a problem solving strategy, which is most commonly
used in optimization problems, where the goal is to minimize a certain value.

The optimized solution is obtained by means of a state space tree (A state space tree is a tree
where the solution is constructed by adding elements one by one, starting from the root. Note
that the root contains no element).

This method is best when used for combinatorial problems with exponential time complexity,
since it provides a more efficient solution to such problems.

The Algorithm: Branch and Bound Technique

In this technique, the first step is to create a function U (which represents an upper bound to
the value that node and its children shall achieve), that we intend to minimize. We call this
function the objective function. Note that the branch and bound technique can also be used
for maximization problems, since multiplying the objective function by -1 coverts the
problem to a minimization problem.

Let this function have an initial maximum value, according to the conditions of the given
problem. Also, let U0 be the initial value of U. We also calculate a cost function C which will
give us the exact value that the particular node shall achieve.

The next question is the order in which the tree is to be searched. For this, there exist multiple
types of branch and bound, which we shall discuss in detail later. For now, let us assume that
there is a set S consisting of subsets of the given set of values in the order in which they need
to be searched

The algorithm of the branch and bound method for this problem will be as follows:
For each subset s in S, do the following:
1. Calculate the value of U(s).
2. If U(s) U0, then U0 = U(s).

In the end, the the subset s for which the current value of U0 is obtained will be the best
solution to the given problem, and the value of the cost function at that node will give us the
solution. Here, after each level, the value of U0 tells us that there shall be a node with cost
less than that value.

Types of Solutions
For a branch and bound problem, there are two ways in which the solution can be
represented:
1. Variable size solution: This solution provides the subset of the given set that gives the
optimized solution for the given problem. For example, if the we are to select a combination
of elements from {A, B, C, D, E} that optimizes the given problem, and it is found that A, B,
and E together give the best solution, then the solution will be {A, B, E}.

2. Fixed size solution: This solution is a sequence of 0s and 1s, with the digit at the ith position
denoting whether the ith element should be included or not. Hence, for the earlier example,
the solution will be given by {1, 1, 0, 0, 1}.

1
Types of Branch and Bound
There are multiple types of the Branch and Bound method, based on the order in which the
state space tree is to be searched.

FIFO Branch and Bound

The First-In-First-Out approach to the branch and bound problem follows
the queue approach in creating the state-space tree. Here, breadth first search is performed,
i.e., the elements at a particular level are all searched, and then the elements of the next level
are searched, starting from the first child of the first node in the previous level.
For a given set {A, B, C, D}, the state space tree will be constructed as follows :

Here, note that the number assigned to the node signifies the order in which the tree shall be
constructed. The element next to the set denotes the next element to be added to the subset.
Note that if an element is getting added, it is assumed here that all elements in the set
preceeding that element are not added. For example, in node 4, D is getting added. This
implies that elements A, B and C are not added.

LIFO Branch and Bound

The Last-In-First-Out approach to this problem follows the stack approach in creating the
state space tree. Here, when nodes get added to the state space tree, think of them as getting
added to a stack. When all nodes of a level are added, we pop the topmost element from the
stack and then explore it. Hence, the state space tree for the same example {A, B, C, D} will
be as follows:

2
 Here, one can see that the main difference lies in the order in which the nodes have been
explored.

Least Cost-Branch and Bound

This method of exploration uses the cost function in order to explore the state space tree.
Although the previous two methods calculate the cost function at each node, this is not used
as a criterion for further exploration.

In this method, after the children of a particular node have been explored, the next node to be
explored would be that node out of the unexplored nodes which has the least cost. For
example, in the previous example, after reaching node 5, the next node to be explored would
be that which has the least cost among nodes 2, 3, 4, 5.

Problems that can be solved using Branch and Bound

The Branch and Bound method can be used for solving most combinatorial problems. Some
of these problems are given below:
1. Job Sequencing: Suppose there is a set of N jobs and a set of N workers. Each worker takes a
specific time to complete each of the N jobs. The job sequencing problem deals with finding
that order of the workers, which minimizes the time taken to complete the job.

2. 0/1 Knapsack problem: Given a set of weights of N objects and a sack which can carry a
maximum of W units. The problem deals with finding that subset of items such that the
maximum possible weight is carried in the sack. Here, one cannot take part of an object, i.e.,
one can either take an object or not take it. This problem can also be solved using the
backtracking, brute force and the dynamic programming approach.

3. Travelling Salesman Problem: Here, we are given a set of N cities, and the cost of travelling
between all pairs of cities. The problem is to find a path such that one starts from a given
node, visits all cities exactly once, and returns back to the starting city.

3
TRAVELLING SALESMAN PROBLEM
Travelling Salesman Problem (TSP) is defined as “given n cities and distance between each
pair of cities, find out the path which visits each city exactly once and come back to starting
city, with the constraint of minimizing the travelling distance.”

TSP has many practical applications. It is used in network design, and transportation route
design. The objective is to minimize the distance. We can start tour from any random city and
visit other cities in any order. With n cities, n! different permutations are possible. Exploring
all paths using brute force attacks may not be useful in real life applications.

LCBB using state space tree for travelling salesman problem

Branch and bound is an effective way to find better, if not best, solution in quick time by
pruning some of the unnecessary branches of search tree.

It works as follow :
Consider directed weighted graph G = (V, E, W), where node represents cities and
weighted directed edges represents direction and distance between two cities

1. Initially, graph is represented by cost matrix C, where

Cij = cost of edge, if there is a direct path from city i to city j
Cij = ∞, if there is no direct path from city i to city j.

2. Convert cost matrix to reduced matrix by subtracting minimum values from appropriate
rows and columns, such that each row and column contains at least one zero entry.

3. Find cost of reduced matrix. Cost is given by summation of subtracted amount from the
cost matrix to convert it in to reduce matrix.

4. Prepare state space tree for the reduce matrix

5. Find least cost valued node A (i.e. E-node), by computing reduced cost node matrix with
every remaining node.

6. If <i, j> edge is to be included, then do following :

(a) Set all values in row i and all values in column j of A to ∞
(b) Set A[j, 1] = ∞
(c) Reduce A again, except rows and columns having all ∞ entries.

7. Compute the cost of newly created reduced matrix as,

Cost = L + Cost(i, j) + r
Where, L is cost of original reduced cost matrix and r is A[i, j].

8. If all nodes are not visited then go to step 4.

4
Reduction procedure is described below :
Row Reduction:
Matrix M is called reduced matrix if each of its row and column has at least one zero entry or
entire row or entire column has ∞ value. Let M represents the distance matrix of 5 cities. M
can be reduced as follow:
MRowRed = {Mij – min {Mij | 1 ≤ j ≤ n, and Mij < ∞ }}

Consider the following distance matrix:

Find the minimum element from each row and subtract it from each cell of matrix.

Reduced matrix would be:

Row reduction cost is the summation of all the values subtracted from each rows:
Row reduction cost (M) = 10 + 2 + 2 + 3 + 4 = 21

Column reduction:
Matrix MRowRed is row reduced but not the column reduced. Matrix is called column reduced
if each of its column has at least one zero entry or all ∞ entries.
MColRed = {Mji – min {Mji | 1 ≤ j ≤ n, and Mji < ∞ }}

5
To reduced above matrix, we will find the minimum element from each column and subtract
it from each cell of matrix.

Column reduced matrix MColRed would be:

Each row and column of MColRed has at least one zero entry, so this matrix is reduced matrix.
Column reduction cost (M) = 1 + 0 + 3 + 0 + 0 = 4

State space tree for 5 city problem is depicted in Fig. Number within circle indicates the order
in which the node is generated, and number of edge indicates the city being visited.

6
 Example: Find the solution of following travelling salesman problem using branch and
bound method.

Solution:
The procedure for dynamic reduction is as follow:
 Draw state space tree with optimal reduction cost at root node.

 Derive cost of path from node i to j by setting all entries in ith row and jth column as ∞.
Set M[j][i] = ∞

 Cost of corresponding node N for path i to j is summation of optimal cost + reduction cost +
M[j][i]

 After exploring all nodes at level i, set node with minimum cost as E node and repeat the
procedure until all nodes are visited.

 Given matrix is not reduced. In order to find reduced matrix of it, we will first find the row
reduced matrix followed by column reduced matrix if needed. We can find row reduced
matrix by subtracting minimum element of each row from each element of corresponding
row. Procedure is described below:

 Reduce above cost matrix by subtracting minimum value from each row and column.

Reduce it subtracting minimum value from corresponding column. Doing this we get,

7
Cost of M1 = C(1)
= Row reduction cost + Column reduction cost
= (10 + 2 + 2 + 3 + 4) + (1 + 3) = 25
This means all tours in graph has length at least 25. This is the optimal cost of the path.

State space tree

Let us find cost of edge from node 1 to 2, 3, 4, 5.

Select edge 1-2:
Set M1 [1] [ ] = M1 [ ] [2] = ∞
Set M1 [2] [1] = ∞

Reduce the resultant matrix if required.

M2 is already reduced.
Cost of node 2 :
C(2) = C(1) + Reduction cost + M1 [1] [2]
= 25 + 0 + 10 = 35

Select edge 1-3

Set M1 [1][ ] = M1 [ ] [3] = ∞
Set M1 [3][1] = ∞

Reduce the resultant matrix if required.

8
Cost of node 3:
C(3) = C(1) + Reduction cost + M1[1] [3]
= 25 + 11 + 17 = 53

Select edge 1-4:

Set M1 [1][ ] = M1[ ][4] = ∞
Set M1 [4][1] = ∞

Reduce resultant matrix if required.

Matrix M4 is already reduced.

Cost of node 4:
C(4) = C(1) + Reduction cost + M1 [1] [4]
= 25 + 0 + 0 = 25

Select edge 1-5:

Set M1 [1] [ ] = M1 [ ] [5] = ∞
Set M1 [5] [1] = ∞

Reduce the resultant matrix if required.

Cost of node 5:
C(5) = C(1) + reduction cost + M1 [1] [5]
= 25 + 5 + 1 = 31

State space diagram:

9
Node 4 has minimum cost for path 1-4. We can go to vertex 2, 3 or 5. Let’s explore all three
nodes.
Select path 1-4-2 : (Add edge 4-2)
Set M4 [1] [] = M4 [4] [] = M4 [] [2] = ∞
Set M4 [2] [1] = ∞

Reduce resultant matrix if required.

Matrix M6 is already reduced.

Cost of node 6:
C(6) = C(4) + Reduction cost + M4 [4] [2]
= 25 + 0 + 3 = 28

Select edge 4-3 (Path 1-4-3):

Set M4 [1] [ ] = M4 [4] [ ] = M4 [ ] [3] = ∞
Set M [3][1] = ∞

Reduce the resultant matrix if required.

is not reduced. Reduce it by subtracting 11 from column 1.

Cost of node 7:
C(7) = C(4) + Reduction cost + M4 [4] [3]
= 25 + 2 + 11 + 12 = 50

10
Select edge 4-5 (Path 1-4-5):

Matrix M8 is reduced.
Cost of node 8:
C(8) = C(4) + Reduction cost + M4 [4][5]
= 25 + 11 + 0 = 36

State space tree

Path 1-4-2 leads to minimum cost. Let’s find the cost for two possible paths.

Add edge 2-3 (Path 1-4-2-3):

Set M6 [1][ ] = M6 [4][ ] = M6 [2][ ]
= M6 [][3] = ∞
Set M6 [3][1] = ∞

Reduce resultant matrix if required.

11
Cost of node 9:
C(9) = C(6) + Reduction cost + M6 [2][3]
= 28 + 11 + 2 + 11 = 52

Add edge 2-5 (Path 1-4-2-5):

Set M6 [1][ ] = M6 [4][ ] = M6 [2][ ] = M6 [ ][5] = ∞
Set M6 [5][1] = ∞

Reduce resultant matrix if required.

Cost of node 10:

C(10) = C(6) + Reduction cost + M6 [2][5]
= 28 + 0 + 0 = 28

State space tree

12
Add edge 5-3 (Path 1-4-2-5-3):

Cost of node 11:

C(11) = C(10) + Reduction cost + M10 [5][3]
= 28 + 0 + 0 = 28

State space tree:

13
0\1 KNAPSACK PROBLEM
In solving this problem, we shall use the Least Cost- Branch and Bound method, since this
shall help us eliminate exploring certain branches of the tree. We shall also be using the
fixed-size solution here. Another thing to be noted here is that this problem is a maximization
problem, whereas the Branch and Bound method is for minimization problems. Hence, the
values will be multiplied by -1 so that this problem gets converted into a minimization
problem.

Now, consider the 0/1 knapsack problem with n objects and total weight W. We define the
upper bound(U) to be the summation of vixi (where vi denotes the value of that objects, and
xi is a binary value, which indicates whether the object is to be included or not), such that the
total weights of the included objects is less than W. The initial value of U is calculated at the
initial position, where objects are added in order until the initial position is filled.

We define the cost function to be the summation of vifi, such that the total value is the
maximum that can be obtained which is less than or equal to W. Here fi indicates the fraction
of the object that is to be included. Although we use fractions here, it is not included in the
final solution.

Here, the procedure to solve the problem is as follows are:

 Calculate the cost function and the Upper bound for the two children of each node.
Here, the (i + 1)th level indicates whether the ith object is to be included or not.

 If the cost function for a given node is greater than the upper bound, then the node
need not be explored further. Hence, we can kill this node. Otherwise, calculate the
upper bound for this node. If this value is less than U, then replace the value of U with
this value. Then, kill all unexplored nodes which have cost function greater than this
value.

 The next node to be checked after reaching all nodes in a particular level will be the
one with the least cost function value among the unexplored nodes.

 While including an object, one needs to check whether the adding the object crossed
the threshold. If it does, one has reached the terminal point in that branch, and all the
succeeding objects will not be included.

In this manner, we shall find a value of U at the end which eliminates all other possibilites.
The path to this node will determine the solution to this problem.

Time and Space Complexity

Even though this method is more efficient than the other solutions to this problem, its worst
case time complexity is still given by O(2n), in cases where the entire tree has to be explored.
However, in its best case, only one path through the tree will have to explored, and hence its
best case time complexity is given by O(n). Since this method requires the creation of the
state space tree, itsspace complexity will also be exponential.

14
Example

15
16
17
18