C ONSTRAINT SATISFACTION

P ROBLEMS

Chapter 5 1
 Outline
♦ CSP examples
♦ Backtracking search for CSPs
♦ Problem structure and problem
decomposition
♦ Local search for CSPs

Chapter 5 2
 C o n st ra i nt sati sfacti on problems ( C S P s )
Standard search problem:
state is a “black box”—any old data
structure that supports goal test,
eval, successor
CSP:
state is defined by variables X i with
values from domain D i

goal test is a set of constraints specifying

allowable combinations of values for
subsets of variables
Simple example of a formal
representati on language
Allows useful general-purpose algorithms with more
power than standard search algorithms
Chapter 5 3
 E xa m p l e : M a p - C o l o r i n g

Northern

Western Territory Queensland

Australia

South
Australia
N
e
w

S
o
u
Tasmania
Variables W A , N T , Q , N S W , V , t
h
S A, T
Domains D i =adjacent
Constraints: { r ed, grregions
een, blue}
must have W
a
different colors e.g., W A =/ N T (if the l
language allows this), or e
s
(W A, N T ) ∈ {(red, green), (red, blue), (green, Chapter 5 4

red), (green, blue), . . .}

V
 E xa m p l e : M a p - C o l o r i n g contd.

Northern

Western Territory Queensland

Australia

South
Australia
N
e
w

S
o
u
t
h

Solutions are assignments satisfying all constraints,

W e.g.,
{ W A = r ed, N T = gr een, Q = r ed, N S W = algr een, V = r ed, S A = blue, T =
gr een} e
s
Chapter 5 5
 C o n st ra i nt g ra p h
Binary CSP: each constraint relates at most two
variables
Constraint graph: nodes are variables, arcs show
constraints

NT
Q
WA SA NS
W

VicVtori
a

General-purpose CSP algorithms use the

graph structure
to speed up search. E.g., Tasmania is an
Chapter 5 6
independent subproblem!
 Varieti es of C S P s
Discrete variables
finite domains; size d ⇒ O(d n ) complete assignments
♦e.g., Boolean CSPs, incl. Boolean satisfiability (NP-
complete) infinite domains (integers, strings, etc.)
♦ e.g., job scheduling, variables are start/end days for
each job
♦ need a constraint language, e.g., StartJob 1 + 5 ≤
StartJob 3
♦ linear constraints solvable, nonlinear undecidable

Continuous variables
♦ e.g., start/end times for Hubble Telescope observations
♦ linear constraints solvable in poly time by LP methods

Chapter 5 7
 Varieti es of constraints
Unary constraints involve a single
variable, e.g., S A / = green
Binary constraints involve pairs of
variables, e.g., S A / = W A
Higher-order constraints involve 3 or more
variables, e.g., cryptarithmetic column
constraints
Preferences (soft constraints), e.g., red is
better than green
often representable by a cost for each
variable assignment
→ constrained optimization problems

Chapter 5 8
 E xa m p l e : C r y p t a r i t h m e ti c

T F T U W R O
W O
+ T W
O F O U
X3 X2 X1
R
Variables: F T U W R O X 1 X 2
X3
Domains: { 0, 1, 2, 3, 4, 5, 6, 7,
8, 9}
Constraints
alldiff(F , T , U, W, R , O )
O + O = R + 10 · X 1 , etc.

Chapter 5 9
 Rea l - wo r l d C S P s
Assignment problems
e.g., who teaches what class
Timetabling problems
e.g., which class is offered when and where?
Hardware configuration
Spreadsheets
Transportation
scheduling Factory
scheduling
Floorplanning

Notice that many

real-world problems
Chapter 5 10
involve real-valued
 S ta n d a rd search formulati on (incremental)
Let’s start with the straightforward, dumb approach,
then fix it
States are defined by the values assigned so far
♦ Initial state: the empty assignment, { }
♦ Successor function: assign a value to an
unassigned variable that does not conflict with
current assignment.
⇒ fail if no legal assignments (not
fixable!)
♦ Goal test: the current assignment is complete

1) This is the same for all CSPs!

2) Every solution appears at depth n with n variables
⇒ use depth-first search
3) Path is irrelevant, so can also use complete-state Chapter 5 11

formulation
 B a c k t ra c k i n g search
Variable assignments are commutative, i.e.,
[W A = r ed then N T = gr een] same as [N T = gr een then
W A = r ed]
Only need to consider assignments to a single variable at each
node
⇒ b = d and there are dn leaves

Depth-first search for CSPs with single-variable

assignments is called backtracking search
Backtracking search is the basic uninformed
algorithm for CSPs Can solve n-queens for n ≈ 25

Chapter 5 12
 B a c k t ra c k i n g search

functi on B ACKTRACKING-SEARCH(csp) returns solution/failure

return R ECURSIVE- B ACKTRACKING({ } , csp)
functi on R ECURSIVE- B ACKTRACKING(assignment, csp) returns
soln/failure
if assignment is complete then return assignment
var ← SELECT-UNASSIGNED-VARIABLE(VARIABLES[csp],
assignment, csp)
for each value in ORDER-DOMAIN-VALUES(var, assignment, csp)
do
if value is consistent with assignment given
CONSTRAINTS[csp] then
add {var = value} to assignment
result ← R ECURSIVE- B ACKTRACKING(assignment, csp)
if result /= failure then return result
remove {var = value} from assignment
return failure

Chapter 5 13
B a c k t ra c k i n g example

Chapter 5 14
B a c k t ra c k i n g example

Chapter 5 15
B a c k t ra c k i n g example

Chapter 5 16
B a c k t ra c k i n g example

Chapter 5 17
 I m p ro v i n g backtracking effi ciency
General-purpose methods can give huge gains in
speed:

1.Which variable should be assigned next?

2.In what order should its values be tried?
3.Can we detect inevitable failure early?
4.Can we take advantage of problem structure?

Chapter 5 18
 M i n i m u m remaining values
Minimum remaining values (MRV):
choose the variable with the fewest
legal values

Chapter 5 19
 Degree heuristi c
Tie-breaker among MRV variables
Degree heuristic:
choose the variable with the most constraints on
remaining variables

Chapter 5 20
 L e a st constraining value
Given a variable, choose the least constraining value:
the one that rules out the fewest values in the remaining
variables

Allows 1 value for SA

Allows 0 values for

SA

Combining these heuristics makes 1000

queens feasible

Chapter 5 21
 For ward checking
Idea: Keep track of remaining legal values for
unassigned variables Terminate search when any
variable has no legal values

WA NT Q NSW V SA T

Chapter 5 22
 For ward checking
Idea: Keep track of remaining legal values for
unassigned variables Terminate search when any
variable has no legal values

WA NT Q NSW V SA T

Chapter 5 23
 For ward checking
Idea: Keep track of remaining legal values for
unassigned variables Terminate search when any
variable has no legal values

WA NT Q NSW V SA T

Chapter 5 24
 For ward checking
Idea: Keep track of remaining legal values for
unassigned variables Terminate search when any
variable has no legal values

WA NT Q NSW V SA T

Chapter 5 25
 C o n st ra i nt propagati on
Forward checking propagates information from assigned to
unassigned vari- ables, but doesn’t provide early detection for
all failures:

WA NT Q NSW V SA T

N T and S A cannot both be blue!

Constraint propagation repeatedly enforces
constraints locally

Chapter 5 26
 A r c consistency
Simplest form of propagation makes each arc
consistent
X → Y is consistent iff
for every value x of X there is some allowed
y

WA NT Q NSW V SA T

Chapter 5 27
 A r c consistency
Simplest form of propagation makes each arc
consistent
X → Y is consistent iff
for every value x of X there is some allowed
y

WA NT Q NSW V SA T

Chapter 5 28
 A r c consistency
Simplest form of propagation makes each arc
consistent
X → Y is consistent iff
for every value x of X there is some allowed
y

WA NT Q NSW V SA T

If X loses a value, neighbors of X need to be

rechecked

Chapter 5 29
 A r c consistency
Simplest form of propagation makes each arc
consistent
X → Y is consistent iff
for every value x of X there is some allowed
y

WA NT Q NSW V SA T

If X loses a value, neighbors of X need to be

rechecked
Arc consistency detects failure earlier than forward
checking Can be run as a preprocessor or after
each assignment Chapter 5 30
 A r c consistency algorithm

functi on AC-3( csp) returns the CSP, possibly with reduced domains
inputs: csp, a binary CSP with variables {X 1 , X 2 , . . . , X n }
local variables: queue, a queue of arcs, initially all the arcs in csp
while queue is not empty do
(X i , X j ) ← R EMOVE- F IRST(queue)
if R EMOVE-INCONSISTENT-VALUES(X i ,
X j ) then for each X k i n NEIGHBORS[X i ]
do
add (X k , X i ) to queue

functi on R EMOVE-INCONSISTENT-VALUES( X i ,
X j ) returns true iff succeeds
removed ← false
for each x in DOMAIN[X i ] do
if no value y in DOMAIN[X j ] allows (x,y) to
satisfy the constraint X i

↔
Chapter 5 31

Xj
 P ro b l e m structure

NT
Q
WA

SA NS
W

VicVtori
a

Tasmania and mainland are independent

subproblems Identifiable as connected
components of constraint graph

Chapter 5 32
 P ro b l e m structure contd.
Suppose each subproblem has c variables out
of n total
Worst-case solution cost is n/c · dc, linear in n
E.g., n = 80, d = 2, c = 20
280 = 4 billion years at 10 million nodes/sec
4 · 220 = 0.4 seconds at 10 million nodes/sec

Chapter 5 33
 Tree-structured C S P s

A E
B D
C F
Theorem: if the constraint graph has no loops, the CSP can be
solved in
O (n d2 ) time
Compare to general CSPs, where worst-case time is O(d n )
This property also applies to logical and probabilistic reasoning:
an important example of the relation between syntactic
restrictions and the complexity of reasoning.

Chapter 5 34
 A l g o r i t h m for tree-structured C S P s
1. Choose a variable as root, order variables from root
to leaves such that every node’s parent precedes it in
the ordering

A E
B D A B C
C D E F F

2. For j from n down to 2, apply

R EMOVEINCONSISTENT(P arent(X j ), X j )

3. For j from 1 to n, assign X j consistently with Pa r e nt ( X j )

Chapter 5 35
 N e a r l y tree-structured C S P s
Conditioning: instantiate a variable, prune its
neighbors’ domains
NT NT
Q Q
WA WA

SA NS NS
W W

VicVtori VicVtori
a a

T T

Cutset conditioning: instantiate (in all ways) a set

of variables such that the remaining constraint
graph is a tree
Cutset size c ⇒ runtime O (dc · (n − c)d2 ), very fast
for small c

Chapter 5 36
 Iterati ve algorithms for C S P s
Hill-climbing, simulated annealing typically
work with “complete” states, i.e., all
variables assigned
To apply to CSPs:
allow states with unsatisfied
constraints operators reassign
variable values
Variable selection: randomly select any
conflicted variable Value selection by min-
conflicts heuristic:
choose value that violates the fewest
constraints
i.e., hillclimb with h(n) = total number of
violated constraints

Chapter 5 37
 E xa m p l e : 4-Queens

States: 4 queens in 4 columns (44 = 256

states)
Operators: move queen in
column Goal test: no
attacks
Evaluation: h(n) = number
of attacks

h=5 h=2 h=0

Chapter 5 38
 Performance of min-confl icts
Given random initial state, can solve n-queens in almost
constant time for arbitrary n with high probability (e.g., n =
10,000,000)
The same appears to be true for any randomly-generated CSP
except in a narrow range of the ratio
number of
R =
constraints
number of
variables
CPU
tim
e

R
critic
al
rati
o
Chapter 5 39
 Summary
CSPs are a special kind of problem:
states defined by values of a fixed set of
variables goal test defined by constraints
on variable values
Backtracking = depth-first search with one variable
assigned per node Variable ordering and value selection
heuristics help significantly Forward checking prevents
assignments that guarantee later failure Constraint
propagation (e.g., arc consistency) does additional work
to constrain values and detect inconsistencies
The CSP representation allows analysis of problem
structure Tree-structured CSPs can be solved in
linear time Chapter 5 40

Iterative min-conflicts is usually effective in