Solution methods for CSP

Solutions for Constraint Satisfaction Problems

1. Generate-and-Test
2. Backtracking [possibly Dependency Directed]
3. Consistency Driven
Constraint propagation
• Constraint propagation is the process of communicating the domain
reduction of a decision variable to all of the constraints that are
stated over this variable.
• This process can result in more domain reductions. These domain
reductions, in turn, are communicated to the appropriate constraints.
Generate and Test

• We generate one by one all possible complete variable assignments

and for each we test if it satisfies all constraints.
• The corresponding program structure is very simple, just nested
loops, one per variable.
• In the innermost loop we test each constraint.
• In most situation this method is intolerably slow.
Backtracking

• We order the variables in some fashion, trying to place first the

variables that are more highly constrained or with smaller ranges.
• This order has a great impact on the efficiency of solution algorithms
and is examined elsewhere.
• We start assigning values to variables.
• We check constraint satisfaction at the earliest possible time and
extend an assignment if the constraints involving the currently bound
variables are satisfied.
Backtracking search
• Variable assignments are commutative, i.e.,
[ WA = red then NT = green ] same as [ NT = green then WA = red ]

• => Only need to consider assignments to a single variable at

each node

• Depth-first search for CSPs with single-variable assignments is

called backtracking search
Improving Backtracking Search
• General-purpose methods can give huge gains in
speed:
• Which variable should be assigned next?
• Idea 1: Min. remaining values
• Idea 2: Degree heuristic
• In what order should its values be tried?
• Idea 3: Least constraining value
• Can we detect inevitable failure early?
• Idea 4: Forward checking / AC-3 Algorithm
• Can we take advantage of problem structure?
• Idea 5: Tree-structured CSPs
Backtracking example
Backtracking example
Backtracking example
Backtracking example
Example: 4-Queens Problem
X1 X2
1 2 3 4 {1,2,3,4} {1,2,3,4}
1
2
3
4
X3 X4
{1,2,3,4} {1,2,3,4}
Example: 4-Queens Problem
X1 X2
1 2 3 4 {1,2,3,4} {1,2,3,4}
1
2
3
4
X3 X4
{1,2,3,4} {1,2,3,4}
Example: 4-Queens Problem
X1 X2
1 2 3 4 {1,2,3,4} { , ,3,4}
1
2
3
4
X3 X4
{ ,2, ,4} { ,2,3, }
Example: 4-Queens Problem
X1 X2
1 2 3 4 {1, , , } { , ,3,4}
1
2
3
4
X3 X4
{ , , ,} { ,2, , }
Example: 4-Queens Problem
X1 X2
1 2 3 4 {1,2,3,4} { , ,3,4}
1
2
3
4
X3 X4
{ ,2, ,4} { ,2,3, }
Example: 4-Queens Problem
X1 X2
1 2 3 4 {1,2,3,4} { , ,3,4}
1
2
3
4
X3 X4
{ ,2, , } { ,2,3, }
Example: 4-Queens Problem
X1 X2
1 2 3 4 {1,2, , } { , ,3,4}
1
2
3
4
X3 X4
{ ,2, , } { , ,3, }
Example: 4-Queens Problem
X1 X2
1 2 3 4 {1,2, , } { , ,3,4}
1
2
3
4
X3 X4
{ ,2, , } { , ,3, }
Example: 4-Queens Problem
X1 X2
1 2 3 4 {1, , , } { , , ,4}
1
2
3
4
X3 X4
{ ,2, , } { , , , }
Backtrack with Forward checking

Solving CSPs (backtrack) with combination of heuristics plus

forward checking is more efficient than either approach alone

But Forward Checking does not see all inconsistencies

Consider our map coloring search, after we have assigned

WA=red and Q=green
 Forward checking

• WA=Red; Q=green
• Forward checking gives us the third row

• At this point, we can see that this is inconsistent, since NT and SA are
forced to be blue, yet they are adjacent. Forward checking doesn’t see
this, and proceeds onward in the search from this state (as we saw
earlier)
 Constraint propagation
• Forward checking (FC) is in effect eliminating parts of the search
space
• Constraint propagation goes further than FC by repeatedly enforcing
constraints locally
• Needs to be faster than actually searching to be effective
• Arc-consistency (AC) is a systematic procedure for constraint
propagation
Consistency Algorithms

• Although depth-first search over the search space of assignments is

usually a substantial improvement over generate and test, it still has
various inefficiencies that can be overcome.
 Consistency Algorithms
• Suppose you have a CSP with the variables A, B, and C, each
with domain {1, 2, 3, 4}.
• Suppose the constraints are A < B and B < C.
• The assignment A = 4 is inconsistent with each of the possible
assignments to B because dom(B) = {1, 2, 3, 4}.
• In the course of the backtrack search this fact is rediscovered
for different assignments to B and C.
• This inefficiency can be avoided by the simple expedient of
deleting 4 from dom(A), once and for all.
• This idea is the basis for the consistency algorithms.
 Constraint Network
• The consistency algorithms are best thought of as operating over the
network of constraints formed by the CSP:
• There is a node for each variable. These nodes are drawn as ovals.
• There is a node for each constraint. These nodes are drawn as
rectangles.
Constraint Network
 Arc consistency

• An Arc X → Y is consistent if
for every value x of X there is some value y consistent with x
• Consider state of search after WA and Q are assigned:
SA → NSW is consistent: if SA=blue NSW could be =red
 Arc consistency

• X → Y is consistent if for every value x of X there is some value y consistent with x

• We will try to make the arc consistent by deleting x’s for which there is no y (and then check to see if anything else has been affected –
algorithm is in a few slides)

• NSW → SA: if NSW=red SA could be =blue

But, if NSW=blue, there is no color for SA.
So, remove blue from the domain of NSW
Propagate the constraint: need to check Q → NSW SA → NSW V → NSW
If we remove values from any of Q, SA, or V’s domains, we will need to check THEIR neighbors
[continue process on next slide and board]
Arc consistency

• After removing red from domain of V to make V → NSW arc consistent

• SA → V, NSW → V check out; no changes

• Check the remaining arcs: most check out, until we check SA → NT, NT → SA. Whichever is
checked first will result in failure.
Arc consistency

• SA → NT is not consistent
• and cannot be made consistent
• Arc consistency detects failure earlier than FC
• This process was all in one call to the INFERENCE function right after we assigned Q=green. Forward
checking proceeded in the search, assigning a value to V.
Arc consistency checking

• AC must be run until no inconsistency remains

• Trade-off
• Requires some overhead to do, but generally more effective than direct search
• In effect it can eliminate large (inconsistent) parts of the state space more effectively than
search can

• Need a systematic method for arc-checking

• If X loses a value, neighbors of X need to be rechecked:
i.e. incoming arcs can become inconsistent again
(outgoing arcs will stay consistent).
Arc consistency algorithm (AC-2)
function AC-2(csp) returns false if inconsistency found, else true, may reduce csp domains
local variables: queue, a queue of arcs, initially all the arcs in csp
/* initial queue must contain both (Xi, Xj) and (Xj, Xi) */
while queue is not empty do
(Xi, Xj)  REMOVE-FIRST(queue)
if REMOVE-INCONSISTENT-VALUES(Xi, Xj) then
if size of Di = 0 then return false
for each Xk in NEIGHBORS[Xi] − {Xj} do
add (Xk, Xi) to queue if not already there
return true

function REMOVE-INCONSISTENT-VALUES(Xi, Xj) returns true iff we delete a

value from the domain of Xi
removed  false
for each x in DOMAIN[Xi] do
if no value y in DOMAIN[Xj] allows (x,y) to satisfy the constraints
between Xi and Xj
then delete x from DOMAIN[Xi]; removed  true
return removed
 Arc Consistency: Important aspects
• Consistency is a key concept used to reduce the search space by
eliminating values from variable domains that cannot be part of a
consistent solution.
• The difference between 2-arc consistency and 3-arc consistency lies in
how constraints are enforced among variables.
• Algorithm AC-3 is commonly used to achieve this level of consistency.
Arc Consistency or AC-2
2-Arc Consistency (Arc Consistency or AC-2)
• A variable X in a CSP is arc-consistent with another variable Y (for a
constraint C(X,Y) if:
• For every value in the domain of X, there exists at least one value in the
domain of Y that satisfies the constraint C(X,Y).
• If a value in X's domain has no supporting value in Y's domain, it is
removed.
• This process is applied to all directed arcs in the constraint graph.
Arc Consistency or AC-3 or Path Consistency
3-Arc Consistency (Path Consistency)
• A constraint graph is path-consistent at level 3 if, for any pair of
variables X and Y, there exists a third variable Z such that:
• For every valid pair of values in X and Y satisfying C(X,Y), there exists at least
one value in Z that satisfies both C(X,Z) and C(Y,Z).
• Path consistency ensures that binary constraints are compatible with
a third variable.
• It is useful for detecting inconsistencies that may not be visible at the
arc-consistency level.
 Arc consistency algorithm AC-3

• Time complexity: O(#constraints |domain|3)

Checking consistency of an arc is O(|domain|2)
 k-consistency
• A CSP is k-consistent if, for any set of k-1 variables, and for any consistent
assignment to those variables, a consistent value can always be assigned to
any kth variable
• 1-consistency is node consistency
• 2-consistency is arc consistency
• For binary constraint networks, 3-consistency is the same as path consistency
• Getting k-consistency requires time and space exponential in k
• Strong k-consistency means k’-consistency for all k’ from 1 to k
• Once strong k-consistency for k=#variables has been obtained, solution
can be constructed trivially
• Tradeoff between propagation and branching
• Practitioners usually use 2-consistency and less commonly 3-consistency
 Other techniques for CSPs
• Global constraints
• E.g., Alldiff
• E.g., Atmost(10,P1,P2,P3), i.e., sum of the 3 vars ≤ 10
• Special propagation algorithms
• Bounds propagation
• E.g., number of people on two flight D1 = [0, 165] and D2 = [0, 385]
• Constraint that the total number of people has to be at least 420
• Propagating bounds constraints yields D1 = [35, 165] and D2 = [255, 385]
• …
• Symmetry breaking
Trade-offs
• Running stronger consistency checks…
• Takes more time
• But will reduce branching factor and detect more inconsistent partial assignments
• No “free lunch”
• In worst case n-consistency takes exponential time

• Generally helpful to enforce 2-Consistency (Arc Consistency)

• Sometimes helpful to enforce 3-Consistency
• Higher levels may take more time to enforce than they save.
 Local search for CSPs
• Use complete-state representation
• Initial state = all variables assigned values
• Successor states = change a value

• For CSPs
• allow states with unsatisfied constraints (unlike backtracking)
• operators reassign variable values
• hill-climbing with n-queens is an example (we saw in earlier notes)

• Variable selection: randomly select any conflicted variable

• Value selection: min-conflicts heuristic

• Select new value that results in a minimum number of conflicts with the other variables
Local search for CSP
• Local search is a heuristic-based approach to solving Constraint
Satisfaction Problems (CSPs) by iteratively improving a candidate
solution rather than systematically exploring the entire search space.
• Unlike backtracking-based methods, local search does not maintain a
search tree but instead modifies an existing assignment to move
toward a better (more constraint-satisfying) solution.
Local search for CSP
function MIN-CONFLICTS(csp, max_steps) return solution or failure
inputs: csp, a constraint satisfaction problem
max_steps, the number of steps allowed before giving up

current  an initial complete assignment for csp

for i = 1 to max_steps do
if current is a solution for csp then return current
var  a randomly chosen, conflicted variable from VARIABLES[csp]
value  the value v for var that minimize CONFLICTS(var,v,current,csp)
set var = value in current
return failure
Min-conflicts example

• A two-step solution for an 8-queens problem using min-conflicts

heuristic
• At each stage a queen is chosen for reassignment in its column
• The algorithm moves the queen to the min-conflict square, breaking
ties randomly.
Advantages of local search

• Local search can be particularly useful in an online setting

• Airline schedule example
• E.g., mechanical problems require that a plane is taken out of service
• Can locally search for another “close” solution in state-space
• Much better (and faster) in practice than finding an entirely new schedule

• The runtime of min-conflicts is roughly independent of problem size for the 8-queens (observation in the
1990’s that gave rise to much research).
• Can solve the millions-queen problem in roughly 50 steps.
• Why?
• n-queens is easy for local search because of the relatively high density of solutions in state-space
Summary
• CSPs
• special kind of problem: states defined by values of a fixed set of variables, goal test defined by constraints on variable values

• Backtracking=depth-first search with one variable assigned per level

• Heuristics
• Variable ordering and value selection heuristics help significantly

• Constraint propagation does additional work to constrain values and detect inconsistencies
• Works effectively when combined with heuristics

• Iterative min-conflicts is often effective in practice.

Structured CSPs
Tree-structured CSPs
Algorithm for tree-structured CSPs
Nearly tree-structured CSPs

(Finding the minimum cutset is NP-complete.)

 Tree decomposition
• Every variable in original
problem must appear in at least
one subproblem
• If two variables are connected in
the original problem, they must
appear together (along with the
constraint) in at least one
subproblem
• If a variable occurs in two
subproblems in the tree, it must
appear in every subproblem on
the path that connects the two

• Algorithm: solve for all solutions of each subproblem. Then, use the tree-
structured algorithm, treating the subproblem solutions as variables for those
subproblems.
• O(ndw+1) where w is the treewidth (= one less than size of largest subproblem)
• Finding a tree decomposition of smallest treewidth is NP-complete, but good
heuristic methods exists