Data Structures and Algorithm Unit 14

Unit 14 Backtracking
Structure:
14.1 Introduction
Objectives
14.2 Backtracking Strategy
14.3 8-Queens Problem
14.4 Sum of Subsets
14.5 Knapsack Problem
14.6 Summary
14.7 Terminal Questions
14.8 Answers

14.1 Introduction
In the previous unit, we discussed the concept of dynamic programming and
now in the search for fundamental principles of algorithm design, we will
discuss about backtracking, which represents one of the most general
techniques. Many problems which deal with searching for a set of solutions
or which ask for an operational solution satisfying some constraints can be
solved using the backtracking formulation. The name backtrack was first
coined by D.H Lehmer in the 1950s. Early workers who studied the process
were R.J. Woler, who gave an algorithmic account of it in 1960, and S.
Golomb and L.Barmert who presented a very general description of it in the
form of applications.
Objectives
After studying this unit, you should be able to:
 analyze the 8-queen problem
 solve the problem of the Sum of subsets
 use the applications of the Knapsack problem.

14.2 Backtracking Strategy

In many applications of the backtrack method, the desired solution is
expressible as an ntuple (x1,……..,xn) where the xis are chosen from some
finite set si often the problem to be solved calls for finding one vector that
maximizes a criterion function p(x1,....xn). Sometimes it seeks all vectors that
satisfy P. For example, sorting the array of integers in a [1:n] is a problem

Manipal University Jaipur B1640 Page No. 241

Data Structures and Algorithm Unit 14

whose solution is expressible by an n-tuple, were xi is the index in a of ith

smallest element. The criterion function P is the inequality a[xi]  a[xi+1] for
1  i < n. The set Si is finite and includes the integer 1-through n. Through
sorting is not usually one of the problems solved by backtracking, it is one
example of a familiar problem whose solution can be formulated as an n-
tuple. In this unit we study a collection of problems whose solution are best
done using backtracking.
Suppose mi is the size of set Si. Then there are m=m1 m2 ……. mn n-tuples
that are possible candidates for satisfying the function P. The brute force
approach would be to form all these n-tuples, evaluate each one with P and
save those which yield the optimum. The backtrack algorithm has its virtue,
the ability to yield the same answer, for fewer in trials. Its basic idea is to
build up the solution vector on component at a time and to use modified
criterion functions Pi{x1, ….. xi) (some times called bound functions) to test
whether the vector being formed has any chance of success. The major
advantage of this method is that if it is realized that the partial vector
(x1, x2, …. xi) can in no way lead to an optimal solution, then mi+1…..mn
possible test vectors can be ignored entirely. Many of the problems we solve
using backtracking requires that all the solution satisfy a complex set of
constraints. For any problem, these constraints can be divided into two
categories; explicit and implicit.
Definition
Explicit constraints: Are rules that restrict each xi, to take a value only
from a given set,
common examples of explicit constraints
are xi > 0 or Si = {all non negative real no}
xi = 0 or 1 or Si = {0, 1}
di  xi < ui or Si = {a: li  a  ui}
The explicit constraints depend on the particular instance I of the problem
being solved. All tuples that satisfy the explicit constraints define a possible
solution space for I.
Definition
The implicit constraints are tuples that determine which of the tuples in the
solution space of I satisfy the criterion function. Thus implicit constraints
describe the way in which the xi must relate to each other.

Manipal University Jaipur B1640 Page No. 242

Data Structures and Algorithm Unit 14

Self Assessment Question

1. –––––––––––– are rules that restrict each xi, to take a value only from a
given set.

14.3 8-Queens Problem

A classic combinational problem is to place eight queens on 8x8 chessboard
so that no two “attack” that is, so that no two of them are on the same row,
colours or diagonal. Let us number the rows and columns of the chessboard
1 through 8 (Fig 14.1). The queens can also be numbered 1 through 8.
Since each queen must be on a different row, we can without loss of
generality assume that queen i is to be placed on row i. All solutions to the
8-queens problem can therefore be represented as 8-tuples (x1…… x8),
where xi is the column on which queen i is placed. The explicit constraints
using this formulation are Si = {1, 2, 3, 4, 5, 6, 7, 8}; 1i8. Therefore, the
solution space consists of 88-tuples. The implicit constraints for this problem
are that no two queens can be on the same diagonal. The first of these two
constraints implies that all solutions are permutations of the 8-tuple
(1, 2, 3, 4, 5, 6, 7, 8). This realization reduces the size of the solution space
from 88 tuples to 8! Tuples. We see later how to formulate the second
constraint in terms of the xi. Expressed as an 8-tuple, the solution in Fig.
14.1 is (4, 6, 8, 2, 7, 1, 3, 5).

1 2 3 4 5 6 7 8

1 Q

2 Q

3 Q

4 Q

5 Q
6 Q

7 Q
8 Q

Fig. 14.1: 8-Queens Problem

Manipal University Jaipur B1640 Page No. 243
Data Structures and Algorithm Unit 14

Algorithm 14.1
Algorithm Place (k, i)
// Returns true if a queen can be placed
// in kth row and ith columns otherwise
// it returns false, x[ ] is a global array
// whose first (k – 1) values have been set
// Abs (r) returns the absolute value of r
{
For (i:=1) to k – l) do
if ((x [j] – l // Two in the same column
or (Abs (x [j] = Abs (j – k)))
// or in the same diagonal
Then return false.
Return true:
}
Algorithm NQueens (k, n)
// Using backtracking this procedure prints all
// possible placements of n queens on an
// n x n chessboard so that they are
// non attacking
{
for I : = 1 to n do
{
if Place (k,j) then
{
x [ k] : = i;
if (k=n) then write (x[1:n])
else Nqueen (k+1, n);
}
}
}

At this point, we can see how effective function Nqueen is over the brute
 64 
force approach. For an 8x8 chessboard, there are   possible ways to
8
place 8 pieces or approximately 4.4 billion 8 tuples to examine. However, by

Manipal University Jaipur B1640 Page No. 244

Data Structures and Algorithm Unit 14

allowing only placements of queens on distinct rows and columns, we

require the examination of at most 8! or only 40,3208-tuples.
We can estimate the number of nodes that will be generated by NQueens.
The bounding function is static and no change is made to the function as the
search proceeds. In addition, all nodes on the same level, of the separate
space level, state space tree have the same degree.
As required placement of each queen on the chessboard was chosen
randomly with each choice we kept track of the number of columns a queen
could legitimately be placed on. These numbers are listed beneath each
chessboard. The number following the vector represents the value that
function estimate would produce from these sizes. The coverage of these
five trials is 1625.
So that estimated no. of unbounded nodes is only 2.34% of the total no of
nodes in the 8 queen state space tree.

Self Assessment Questions

2. The total no of nodes in the 8-queen state space is –––––––––––.
3. For an 8x8 chessboard there are ––––––––––possible ways to place 8
pieces.

14.4 Sum of Subsets

Given positive numbers wi, 1 i  n, and m, this problem calls for finding all
subsets of the wi whose sums are m. For example, if n = 4, (w1, w2, w3, w4) =
(11, 13, 24, 7), and m = 31 then the desired subsets are (11, 13, 7) and
(24, 7) rather than represent the solution vector by the wi which sum to m,
we could represent the solution as described by the vector (1,2,4) and (3, 4).
In general, all solutions are k-tuples (x1, x2…….. xk), 1  k  n, and different
solutions may have different sized tuples. The explicit constraints xj  {jlj is
an integer and 1  j  n}. The implicit requires that no two be the same and
that the sum of the corresponding will be m. Since we wish to avoid
generating multiple instance of the same subject (e.g (1, 2, 4) and (1, 4, 2)
represent the same subject), another implicit constraints that is imposed is
xi<xi+1, I  i < k.

Manipal University Jaipur B1640 Page No. 245

Data Structures and Algorithm Unit 14

In another formulation of the sum of subsets problem, each solution subject

is represented by n-tuple (x1, x2, ….. xn) such that xi  {0,1}, 1  i  n. Then,
xi = 0, if wi is not chosen then the above instance are (1, 1, 0, 1) and
(0, 0, 1, 1). The formulation express all solutions using fixed sized tuple.
Thus, we conclude that there may be several ways to formulate a problem
so that all solutions are tuples that satisfy some constraints. One can verify
that for both of the above formulations, the solutions space consists of 2n
distinct tuples.
Backtracking algorithms determine problem solutions by systematically
searching the solution space for the given problem instance. This search is
facilitated by using a tree organization for the solution space. For a given
solution space, many tree organizations may be possible. The next two
examples examine some of the ways to organize a solution into a tree.
We have two possible formulations of the solutions space for the sum of
subsets problem. Fig 14.2 and Fig 14.3 show a possible tree organization
for each of these formulations for the case n=4. The tree of Fig 14.2
corresponds to the variable tuple size formulation. The edges are labeled
such that an edge from a level i node to a level i + 1 node represents a
value for xi. At each node the solution space is partitioned into solution
space. The solution space is defined by all paths from the root node to any
node in the tree, since any such path corresponds to a subset satisfying the
explicit constraints. The possible paths are ( ) (this corresponds to the empty
path from the root to itself), (1), (1,2), (1,2,3), (1,2,3,4), (1,2,4), (1,3,4), (2),
(2,3) and so on. Thus, left most subtree defines all subsets containing wi,
the next subtree defines all subsets containing w2 but not w1, and so on.
The tree of Fig 14.3 corresponds to the fixed tuples size formulation. Edges
from level i nodes to level i + I nodes are labeled with the value of xi, which
is either zero or one. All paths from the root to a leaf node define the
solution space. The left subtree of the root defines all subsets containing wl,
the right subtree defines all subsets not containing wl and so on. Now there
are 24 leaf nodes which represent 16 possible tuples.

Manipal University Jaipur B1640 Page No. 246

Data Structures and Algorithm Unit 14

Fig. 14.2: A possible space organization for the sum of subsets problem.
Nodes are numbered as in breath first search.

At this point, it is useful to develop some terminology regarding tree

organization of solution spaces. Each node in this tree defines a problem
state. All paths from the root to other nodes define the state space of the
problem. Solution states are those problem states s for which the path from
the root to s defines a tuple in the solution space. In the tree Fig 14.2, all
nodes are solutions states whereas in the tree Fig 14.3, only leaf nodes are
solution states. Answer states are those solution states s for which the path
from the root to s defines a tuple that is a member of the set of solutions (i.e.
it satisfies the implicit constraints) of the problem.

Manipal University Jaipur B1640 Page No. 247

Data Structures and Algorithm Unit 14

The tree organization of the solution space is referred to as the state space
tree.

Fig. 14.3: Corresponds to the fixed tuples size formulation.

Suppose we are given distinct positive number and we desire to find all
combinations of these numbers whose sums are m. This is called the sum
of subsets problem. The example in above shows how we could formulate
this problem using either fixed or variable sized tuples. We consider a back
tracking solution using the fixed tuple size strategy. In this case, the element
xi of the solution vector is either one or zero depending on whether the
weight wi, is included or not.
The children of any node in Fig 14.3 are easily generated. For a node at
level i, the left child corresponds to xi = 1 and the right to xi = 0.
A simple choice for the bounding functions Bk (x1, ……. xi) = true if
k n

w
i 1
i xi  w  m
i  k 1
i

Clearly, x1, ……..,xk cannot lead to an answer node if this condition is not
satisfied. The bounding functions can be strengthened if we assure that the
wi’s are initially in non-decreasing order. In this case, x1…… xk cannot lead
to an answer node if

Manipal University Jaipur B1640 Page No. 248

Data Structures and Algorithm Unit 14

w
i 1
i xi  wk 1  m

The bounding functions we use are therefore Bk (x1, …………….,xk) = true iff
k n k

 wi xi 
i 1
 wi  m and
i  k 1
w
i 1
i xi  Wk 1  m

Since our algorithms will not make use of Bn, we need not be concerned by
the appearance of Wn+1 in this function. Although we have now specified all
that is needed is to directly use either of the backtracking scheme, a simpler
algorithms results if we tailor either of these schemes to the problem at
hand. This simplification results from the realization that if xk = 1, then
k n
w i xi  w i m
i 1 i k 1

For simplicity, we redefine the recursive scheme. The resulting algorithms is

SumOfSub (Algorithm 14.2)

Algorithm 14.2
Algorithm SumOfSub (s,k,r)
// Find all subsets of w(k....n) that sum to m
// The values of x[j], 1  j < k, have already
k 1
// been determined S   w[ j]  X[ j]
j 1

n
//and r   w [ j ] . The w[j]’s are in
j k

// non decreasing order It is assumed that

n
// w[l]  m and w[ i ]  m
i 1

{
// Generate left child. Note: S+tw[j]  m
// since Bk–1 is true.

Manipal University Jaipur B1640 Page No. 249

Data Structures and Algorithm Unit 14

x [k] : = 1;
If (s + w[k] = m) then write (x [1: k]);

// sub set found there is no recursive call

// here as w[j] > 0, 1  j  n
else if (s + w[k] + w [k + 1]  m)
then SumOfSub (s + w[k], k + 1, r – w[k]);
// Generate right child and evaluate Bk
if(s + r – w [k]  m) and (s + w [k + 1]  m ) then
{
x[k]:=0
SumOfSub (s, k + 1, r — w [k]):
}
}
k n
Algorithm SumOfSub avoids computing w i ki and  w i each time by
i 1 i  k 1
keeping those values in variables s and r respectively. The algorithm
n  n 
assumes w i  m and  w i  m. The initial call is Sum Of Sub  0 , 1, w i
  .

i 1  i 1 
It is interesting to note that the algorithm does not explicitly use the test k> n
to terminate the recursion. This test is not needed as an entry to the
algorithm sm and s+r  m. Hence, r  0 and so k can be no greater than n.
Also note that in the else if statement, since s + wk < m and s + r  m, it
follows that r  wk and hence k + 1  n, observe also that if s + wk = m, then
xk+1,……………….xn must be zero. These zeros are omitted from the output.
k n
In next statement we do not test for  w i xi  w i  m as we already
i 1 i k 1
know s + r  m and xk + 1.
Self Assessment Questions
4. A simple choice for the bounding functions Bk (x1, ……. xi) is true
if ––––––––––
5. Given positive numbers wi, 1 i  n, and m, the problem calls for finding
all –––––––––––of the wi whose sums are m
Manipal University Jaipur B1640 Page No. 250
Data Structures and Algorithm Unit 14

14.5 Knapsack Problem

Given n positive weights wi, n positive profits Pi, and a positive number m
that is the knapsack capacity, this problem calls for choosing a subset of the
weights such that
w i x i  m and  Pi x i is minimized.
1i n 1i n

The xi’s constitute a zero-one-valued vector.

The solution space for this problem consists of the 2n distinct ways to assign
zero or one values to the xi’s. Thus, the solution source is the same as that
for the sum of subset problem. Two possible tree organizations are possible,
One corresponds to the fixed tuple size formulation and the other to the
variable tuple size formulation. Backtracking algorithms for the knapsack
problem can be arrived at using either of these two state space trees.
Regardless of which is used, bounding functions are needed to help kill
some live nodes without expanding them. A good bounding function for this
problem is obtained by using an upper bound on the value of the best
feasible solution.
Obtainable by expanding the given live node and any of its descendants, if
this upper bound is not higher than the value of the best solution determined
so far, then that live node can be killed.
We continued the discussion using the fixed tuple size formulation, If at
node z, the values of xi, 1  i  k have already been determined, then an
upper bound for z can be obtained by relaxing the requirement xi = 0 on 1 to
0  xi  1 for k + 1  i  n and using the greedy algorithm to solve the
relaxed problem. Function Bound (cp, cw, k) determines an upper bound on
the best solution obtained by expanding any zone z at level of the state
space tree. The object weights and profits are w[i] and p[i]. It is assumed
p( i ) p( i  1 )
that  ;1i n
w [ i ] w( i  1)

Manipal University Jaipur B1640 Page No. 251

Data Structures and Algorithm Unit 14

Algorithm 14.3
Algorithm Bound (cp, cw, k)
// cp is the current profit, cw is the current
// weight total k is the index of the last removed
// item : and m is the knapsack size
{
b:=cp; c:= cw;
for i = k +1 to n do
{
c: = c + w [i]
if (c < m) then b:= b + p [i]
else return b + (cI – (c-m)) / w[i] * p [i];
} return b:
{

From bound it follows that the bound for a feasible left child of a node z is
the same as that for z. Hence, the bounding function need not be used
whenever the backtracking algorithm makes a move to the left child of a
node. The resulting algorithm is Bknap. It was obtained from the recursive
backtracking scheme, Initially set fp: = -1: This algorithm is invoked as
Bknap (1, 0, 0):
n
When fp  —1, x[i], 1  i  n is such that  pi  xi   f p . The path y [i], 1  i
i 1
 k, is the path to the current node. The current weight
k 1 k 1
cw   w i  y i  and c p   pi  y i 
i 1 i 1

Algorithm 14.4
Algorithm Bknap (k, cp, cw)
// m is the size of the knapsack; is the
// no of weights and profits w[ ] and p [ ]
// are the weights and profits
pi 
// p [i] /w [i]  p[i +1] fw  p[ i  1 ]  fw is the final
w i 
// weight of knapsack: fp is the final

Manipal University Jaipur B1640 Page No. 252

Data Structures and Algorithm Unit 14

// maximum profit x[k] = 0 if w[k] is not

// the knapsack else x [k] = 1
{
// Generate left child
If (cw + w [k]  m ) then
{
y [k] = 1:
if (k<n) then Bknap (k+1, cp+p [k], cw + w [k]
if (cp + p[k] > fp) and (k= n) then
{
fp: = cp + p[k]; fw: = cw + w[k]
for j: = 1 to k do x [j]: = y[j]:
}
}
// Generate right child
if (Bound (cp, cw, k)  fp) then
{
y[k]: = 0; if (k < n) then Bknop (k + 1, cp, cw):
If ((cp > fp) and (k = n) then
{
f p : = c p : f w = cw
for j : = 1 to k do
x[j] : = y[j]:
}
}
}

Self Assessment Question

6. If the object weights and profits are w[i] and p[i], It is assumed
that ––––––

14.6 Summary
 In this unit, we studied that any problem can be divided into two
categories explicit and implicit.
 We also studied in this unit a classic combinational problem called as
8-queens problem.

Manipal University Jaipur B1640 Page No. 253

Data Structures and Algorithm Unit 14

 The concept of sum of subsets was discussed with suitable examples.

 The Knapsack problem and its algorithm was also discussed with
suitable explanation

14.7 Terminal Questions

1. Write briefly the concept of 8-Queens Problem
2. Write the Algorithm of sum of Subsets
3. Briefly explain the Knapsack Problem

14.8 Answers
Self Assessment Questions
1. Explicit constraints
7
 j 
2. 1    (8  i)  69,281
j 0  i 0 

 64 
3.  
8
k n
4.  wi xi 
i 1
w  m
i  k 1
i

5. subsets
p( i ) p( i  1 )
6.  ;1i n
w [ i ] w( i  1)

Terminal Questions
1. A classic combinational problem is to place eight queens on 8x8
chessboard so that no two “attack” that is, not two of them are on the
same row, colours or diagonal. Let us number the rows and columns of
the chessboard 1 through 8 (Fig 14.1). The queens can also be
numbered 1 through 8. Since each queen must be on a different row, we
can without loss of generality assume that queen i is to be placed on row
i. All solutions to the 8-queens problem can therefore be represented as
8-tuples (x1…… x8), where xi is the column on which queen i is placed.
The explicit constraints using this formulation are Si = {1, 2, 3, 4, 5, 6, 7,

Manipal University Jaipur B1640 Page No. 254

Data Structures and Algorithm Unit 14

8}; 1i8. Therefore the solution space consists of 88-tuples. The implicit
constraints for this problem are that no two queens can be on the same
diagonal. The first of these two constraints implies that all solutions are
permutations of the 8-tuple (1, 2, 3, 4, 5, 6, 7, 8). This realization
reduces the size of the solution space from 88 tuples to 8! Tuples. (Refer
Section 14.3)
2. Refer Section 14.4
3. Refer Section 14.5

___________________

Manipal University Jaipur B1640 Page No. 255