module5_backtrackingnbranchnbound_2022.pdf

Module 5
Backtracking & Branch and
Bound

The backtracking method
• For problems where no. of choices grow exponentially with problem size
• A given problem has a set of constraints and possibly an objective function
• The solution optimizes an objective function, and/or is feasible.
• We can represent the solution space for the problem using a state space tree
– The root of the tree represents 0 choices,
– Nodes at depth 1 represent first choice
– Nodes at depth 2 represent the second choice, etc.
– In this tree a path from root to a leaf represents a candidate solution
• Definition: We call a node nonpromising if it cannot lead to a feasible (or
optimal) solution, otherwise it is promising
• Main idea: Backtracking consists of doing a DFS of the state space tree,
checking whether each node is promising and if the node is nonpromising
backtracking to the node’s parent

Backtracking
• For some problems, the only way to solve is to check all
possibilities.
• Backtracking is a systematic way to go through all the possible
configurations of a search space.
• Many problems solved by backtracking satisfy a set of
constraints which may divided into two categories: explicit and
implicit.
Explicit constraints are rules that restrict each to take on values only
from a given set
Implicit constraints are rules which relate to each other.
i
x
i
x

Backtracking
• Find whether there is any feasible solution? Decision Problem
• What is best solution? Optimization Problem
• List all feasible solutions? Enumeration Problem

• Problem State is each node in the tree
• State Space is all possible paths from root to other nodes
• Solution State is all possible paths from root to solution
• Answer State satisfies implicit constraints
• A live node is a node which has been generated but whose all children
have not been generated.
• An E-node (i.e., expanding node) is a live node whose children are
currently being generated.
• A dead node is a generated node which is not to be expanded further or
all of whose children have been generated.
• Two ways to generate problem states:
• Breadth First Generation (queue of live nodes)
• Depth First Generation (stack of live nodes)
Generating Problem States

• Depth First Generation (stack of live nodes)
• When a new child C of the current E-node R is generated, this child
becomes the new E-node.
• Then R will become the new E-node again when the subtree C has
been fully explored.
• Corresponds to a depth first search of the problem states.

• Breadth First Generation (queue of live nodes)
• The E-node remains the E-node until it is dead.
• Bounding functions are used in both to kill live nodes without generating all
of their children.
• At the end of the process, an answer node (or all answer nodes) are generated.
• The depth search generation method with bounding function is called
backtracking.
• The breadth first generation method is used in the branch-and-bound
method.

Improving Backtracking: Search Pruning
• Search pruning will help us to reduce the search space and hence get a
solution faster.
• The idea is to avoid those paths that may not lead to a solution as early
as possible by finding contradictions so that we can backtrack
immediately without the need to build a hopeless solution vector.

Greedy vs Backtracking
• Method of obtaining
optimum solution, without
revising previously
generated solutions
• State space tree not created
• Less efficient
• Eg. Job sequencing with
deadlines, Optimal storage
on tapes
• Method of obtaining
optimum solution, may
require revision of previously
generated solutions
• State space tree created
• Efficient to obtain optimum
solution
• Eg. 8 Queen problem, Sum of
subset problem

Applications of Backtracking
• 8 Queen / n Queen Problem
• Sum of Subset problem
• Graph Coloring
• 0/1 Knapsack

Eight Queen Problem
• Attempts to place 8 queens on a chessboard in such a
way that no queen can attack any other.
• A queen can attack another queen if it exists in the same
row, column or diagonal as the queen.
• This problem can be solved by trying to place the first
queen, then the second queen so that it cannot attack the
first, and then the third so that it is not conflicting with
previously placed queens.
• The solution is a vector of length 8 (x1, x2, x3, ... x8)
xi corresponds to the column where we should place the ith queen.
• The solution is to build a partial solution element by element until it is complete.
• We should backtrack in case we reach to a partial solution of length k, that we couldn't
expand any more.

Eight Queen Problem: Algorithm
putQueen(row)
{ for every position col on the same row
if position col is available
place the next queen in position col
if (row<8)
putQueen(row+1);
else success;
}
• Define an 8 by 8 array of 1s and 0s to represent the chessboard
• Note that the search space is very huge:
16,772, 216 (88) possibilities.
• Is there a way to reduce search space?
Yes Search Pruning.

• Since each queen (1-8) must be on a different row, we can assume queen i is
on row i.
• All solutions to the 8-queens problem can be represented as an 8-tuple
where queen i is on column j.
• The explicit constraints are
• The implicit constraints are that no two ’s can be the same (as queens must
be on different columns) and no two queens can be on the same diagonal.
• This implies that all solutions are permutations of the 8-tuple (1,2,…,8),
and reduces the solution space from tuples to 8! tuples.
1
0
,
0 or
x
x i
i 

)
,...,
,
( 8
2
1 x
x
x
.
8
1
},
8
,...,
2
,
1
{ 

 i
Si
8
8
i
x
Eight Queen Problem

• Generalizing earlier discussion, solution space contains all n! permutations of (1,2,…,n).
• The tree below shows possible organization for n=4.
• Tree is called a permutation tree (nodes are numbered as in depth first search).
• Edges labeled by possible values of
• The solution space is all paths from the root node to a leaf node.
• There are 4!=24 leaf nodes in tree.
44
42
39
37
33
31
28
26
23
21
17
15
12
10
7
5
1

1
x

2
x
2 3
1 4
60
58
55
53
49
47 63 65
43
41
38
36
32
30
27
25
22
20
16
14
11
9
6
4 59
57
54
52
48
46 62 64
40
35
29
24
19
13
8
3 56
51
45 61
34
18
2 50
2 3 4
2
3
1
4
1 1
2
3
4
2
3
4 2 3
4
2
3
4
2
3
4 3
4
3 4
1 2 3
4
1 2
3
4
1 2
3
4
1 2
4
1
2 3
2
1
2
3
2
1
3
1 2
1 2
1
1
1
i
x
Four Queen Problem

33
15
1

1
x

2
x
2
1
32
16
14
11
9
29
24
19
13
8
3
18
2
2 3 4
2
1
2
3
4 3
3 4
3
1

3
x

4
x
Backtracking on 4-queens problem
B
B B
B
B
B B
1
2
3
4

Backtracking Algorithm for n-Queens
problem
• Let represent where the ith queen is placed (in row i and
column on an n by n chessboard.
• Observe that two queens on the same diagonal that runs from “upper
left” to “lower right” have the same “row-column” value.
• Also two queens on the same diagonal from “upper right” to “lower
left” have the same “row+column” value
)
,...,
,
( 2
1 n
x
x
x
i
x

• Then two queens at (i,j) and (k,l) are on the same diagonal
• iff i-j=k-l or i+j=k+l
• iff i-k=j-l or j-l = k-i
• iff |j-l|=|i-k| .
• Algorithm PLACE(k,i) returns true if the kth queen can be placed in column i and
runs in O(k) time (see next slide)
• Using PLACE, the recursive version of the general backtracking method can be
used to give a precise solution to the n-queens problem
• Array x[ ] is global in the Algorithm invoked by NQUEENS(1, n).
problem

bool Place(int k, int i)
// Returns true if a queen can be placed in kth row and ith column. Otherwise it returns false.
// x[] is a global array whose first (k-1) values have been set.
// abs(r) returns the absolute value of r.
{
for (int j = 1; j < k; j++)
if ((x[j] == i) // Two in the same column
|| (abs(x[j]-i) == abs(j-k))) // or in the same diagonal
return(false);
return(true);
}
void NQueens(int k, int n)
// Using backtracking, this procedure prints all possible placements of n queens on an n x n
// chessboard so that they are nonattacking
{
for (int i=1; i<=n; i++) {
if (Place(k, i)) {
x[k] = i; //if no conflict place queen
if (k==n) { for (int j=1;j<=n;j++) //dead end
cout << x[j] << ' '; cout << endl; //print board configuration
}
else NQueens(k+1, n); //try next queen next position
}
}
}
problem
Complexity: n!
jth queen at x[j] and kth queen at i

TASK
• Solve problem on slide 16 as per algorithm on slide 19

Sum of Subset Problem
• Given positive numbers and m, find all subsets of ,
whose sum is m.
• If n=4, ={11, 13, 24, 7} and m=31, the desired solution sets
are (11, 13, 7) and (24, 7).
• If the solution vectors are given using the indices of the xi values used, then
the solution vectors are (1,2,4) and (3,4).
• In general, all solutions are k-tuples with and different solutions
may have different values of k.
• The explicit constraints on the solution space are that each
• The implicit constraints are that (so each item will occur
only once) and that the sum of the corresponding ’s be m.
i
w
)
,...,
,
( 2
1 k
x
x
x
n
k 

1
}.
,...,
2
,
1
{ n
xi 
,
1
,
1 n
i
x
x i
i 

 
}
,
,
,
{ 4
3
2
1 w
w
w
w
,
1
, n
i
wi 

1 1
(1)
k n
i i i
i i k
wk w m
  
 
  m
w
k
w k
k
i
i
i 
 

 1
1
)
2
(
hold
)
2
(
and
)
1
(
iff
)
,...,
,
( 2
1 true
x
x
x
B k
k 
The boundary function used uses both of the preceding conditions:
Bounding function

• The next figure gives the tree that corresponds to this variable tuple formation.
• An edge from a level i node to a level i+1 node represents a value for
• The solution space is all paths from the root node to any node in the tree.
• Possible paths include empty path, (1), (1,2), (1,2,3), (1,2,3,4), (1,2,4), (1,3,4), …
• The leftmost subtree gives all subsets containing , the next subtree gives all
subsets containing but not , etc.
.
i
x
1
w
2
w 1
w
16
1

1
x

2
x
2 3
1
4
15
14
13
12
11
10
9
8
7
6
4
3
2 5
2 3 4
4
3
3
4
4
4
4
4

3
x

4
x
Nodes are numbered as in breadth first search
First
Formulation

• Another formulation of this problem represents each solution by an n-tuple
with
• Here if is not chosen and if is chosen
• Given the earlier instance of (11,13,24,7) and m=31, the solutions (11,13,7) and
(24,7) are represented by (1,1,0,1) and (0,0,1,1).
• Here, all solutions have a fixed tuple size. The tree on next slide corresponds to this
formulation (nodes are numbered as in D-search).
• Edge from a level i node to a level i+1 node is labeled with the value of
• All paths from the root to a leaf give solution space.
• The left subtree gives all subsets containing and the right subtree gives all subsets
not containing .
)
,...,
,
( 2
1 n
x
x
x
.
1
},
1
,
0
{ n
i
xi 


0

i
x i
w 1

i
x
i
w
)
1
0
( or
xi
1
w
1
w
Sum of Subset Problem : Second Formulation

9
8
11
10
15
14
17
16
23
22
25
24
29
28
31
30
7
6
13
12
21
20
27
26
5
4
19
18
3
2
1

1
x

2
x

3
x

4
x
0
0
0
0
0
0
0 0
0
0
0
0
0
0
0
1
1
1
1
1
1
1 1
1 1 1
1 1 1 1

Sum of subset Problem
i1
i2
i3
yes no
0
0
0
0
2
2
2
6
6
12 8
4
4
10 6
yes
yes
no
no
no
no
no
no
The sum of the included integers is stored at the node.
yes
yes yes
yes
State SpaceTree for n= 3 items
w1 = 2, w2 = 4, w3 = 6 and S = 6
Solutions:
{2,4} and {6}

A Depth First Search Solution
Pruned State Space Tree
0
0
0
3
3
3
7
7
12 8
4
4
9
5
3
4 4
0
0
0
5 5
0 0 0
0
6
13 7 - backtrack
1
2
3
4 5
6 7
8
10
9
11
12
15
14
13
Nodes numbered in “call” order
w1 = 3, w2 = 4, w3 = 5, w4 = 6;
S = 13

sumOfSubsets ( s,k,r ) //sum, index, remaining sum
//generate left child until s+w[k]≤m
if (s+w[k]=m)
write(x(1:k)) //subset found
else if (s+w[k]+w[k+1]] ≤ m)
sumOfSubsets ( s+w[k],k+1,r-w[k] )
//generate right child
if (s+r-w[k]≥ m) and (s+w[k+1]] ≤ m)
x[k]=0
sumOfSubsets ( s,k+1,r-w[k] )
Initial call sumOfSubsets(0, 0, )


n
i
i
w
1
Sum of subset Algorithm
Complexity: 2n

Given n=6,M=30 and
W(1…6)=(5,10,12,13,15,18).
Ist solution is A -> 1 1 0 0 1 0
IInd solution is B -> 1 0 1 1 0 0
III rd solution is C -> 0 0 1 0 0 1
15,5,33
0,1,73
5,2,68
15,3,58
27,4,46 15,4,46
5,3,58
17,4,46 5,4,4
0,2,68
10,3,587
10,4,46
0,3,58
C
A
B 5,5,33 10,5,33
20,6,18
Xi=1
Xi=0 Xi=1
Xi=0
Xi=0
Xi=1 Xi=0
Xi=1
Xi=0
Xi=0
Xi=0
Xi=1
Xi=1
Xi=1
Xi=1
Xi=1
Xi=0
Xi=0
Xi=0
B 27,5,33
B
27,6,18
B
B

GRAPH / MAP coloring
• Graph Coloring is an assignment of colors
• Proper Coloring is if no two adjacent vertices have the same color
• Optimization Problem
• Chromatic no: smallest no. of colors used to color graph
• The Four Color Theorem states that any map on a plane can be
colored with no more than four colors, so that no two countries with a
common border are the same color

Origin of the problem
m-coloring decision problem: whether the graph can be colored or not
m-coloring optimization problem: min # of colors to color graph
chromatic problem
More than 1 possible solution

Algo
• Number out each vertex (V0, V1, V2, … Vn-1)
• Number out the available colors (C0, C1, C2, … Cm-1)
• Start assigning Ci to each Vi. While assigning the colors note that two
adjacent vertices should not be colored by the same color. And least
no. of colors should be used.
• While assigning the appropriate color, just backtrack and change the
color if two adjacent colors are getting the same.
Complexity: mn

Applications of Graph Theory
• Computer N/W security (Minimum Vertex Cover)
• Timetabling Problem (Vertex Coloring of Bipartite MultiGraph)
• GSM Mobile Phone Networks (Map Coloring)
• Represent Natural Language Parsing Diagrams
• Pattern recognition
• Molecules in chemistry and physics

Branch and Bound
• Branch and bound is used to find optimal solutions to optimization
problems.
• Is applied where Greedy and Dynamic fail.
• Is indeed much slower. Might require exponential time in worst case.
• BnB uses state space tree where in all children of node generated
before expanding any of its child.

Backtracking vs Branch and Bound
• Follows DFS approach
• Solves Decision Problems
• While finding solution, bad
choices can be made
• State space tree is searched until
solution is found
• Space required is O(ht. of tree)
• Applications: N Queens, Sum of
subset
• Follows DFS / BFS / Best First
Search approach
• Solves optimization problems
• Proceeds on a better solution, so
possibility of bad solution is
ruled out
• State space tree is searched
completely since solution can be
found elsewhere
• Space required is O(No. of
leaves)
• Applications: 15 puzzle, TSP

The Branch and Bound Steps
• In BnB, a state space tree is built and all children of E nodes (a live
node whose children are currently been generated) are generated
before any other node can become live node.
• For exploring new nodes either BFS (FIFO queue) or D-search (LIFO
stack) is used or replace the FIFO queue with a priority queue (least-
cost (or max priority)).
– The search for an answer node can often be speeded up using an
“intelligent” ranking function for the live nodes though it requires
additional computational effort.
• In this method, a space tree of possible solutions is generated. Then
partitioning (branching) is done at each node. LB and UB computed
at each node, leading to selection of answer node.
• Bounding functions (when LB>=UB) avoid generation of trees
(fathoming) not containing answer node.

• A branch-and-bound algorithm computes a number (bound) at a node
to determine whether the node is promising.
• The number is a bound on the value of the solution that could be
obtained by expanding beyond the node.
• If that bound is no better than the value of the best solution found so
far, the node is nonpromising. Otherwise, it is promising.
• Besides using the bound to determine whether a node is promising, we
can compare the bounds of promising nodes and visit the children of
the one with the best bound.
• This approach is called best-first search with branch-and-bound
pruning. The implementation of this approach is a modification of the
breadth-first search with branch-and-bound pruning.
The Branch and Bound variants

Branch and Bound Algorithm
Algorithm BnB()
E←new(node) //E is node pointer pointing root node
while(true)
if(E is a final leaf)
write(path from E to root) //E is optimal sol.
return
Expand(E)
if(H is empty) //H is heap for all live nodes
write(“there is no solution”) //E is optimal sol.
return
E← delete_top(H)
Algorithm Expand(E)
Generate all children of E
Compute approximate cost value of each child
Insert each child into heap H

Least Cost search
• In BnB, basic idea is selection of E-node, so that we reach to answer
node quickly.
• Using FIFO and LIFO BnB selection of E-node is complicated (since
expansion of all live nodes required before leading to an answer) and
blind.
• Best First Search and D-search are special cases of LC-search.
• In LC-search an intelligent ranking function (smallest c^(x)) is
formed. c^(x)=f(x)+ ĝ(x) (where f(x) is cost of reaching x from root,
ĝ(x) is an estimate of additional effort needed to reach an answer node
from x).
• LC-search terminates only when either an answer node is found or the
entire state space tree has been generated and searched.
• Note that termination is only guaranteed for finite space trees.

Applications of Branch and Bound
• 8 Puzzle Problem
• Travelling Salesman Problem

8 Puzzle problem using Branch and Bound

TASK
1 3 4 15 1 2 3 4
2 5 12 5 6 7 8
7 6 11 14 9 10 11 12
8 9 10 13 13 14 15
Initial Arrangement Goal Arrangement

module5_backtrackingnbranchnbound_2022.pdf

More Related Content

What's hot

Similar to module5_backtrackingnbranchnbound_2022.pdf

More from Shiwani Gupta

Recently uploaded

module5_backtrackingnbranchnbound_2022.pdf