Constraint Satisfaction Problems

Dr. Shahnawaz Qureshi

Standard Search Problems
 Previous search problems:
 Problems can be solved by searching in a space
of states
 state is a “black box” – any data structure that
supports successor function, heuristic function,
and goal test – problem-specific

 Constraint satisfaction problem

 states and goal test conform to a standard,
structured and simple representation
 general-purpose heuristic

2
Constraint Satisfaction Problems (CSP)
 CSP is defined by 3 components (X, D, C):
 state: a set of variables X, each Xi , with values from
domain Di
 goal test: a set of constraints C, each Ci involves some
subset of the variables and specifies the allowable
combinations of values for that subset
 Each constraint Ci consists of a pair <scope, rel>, where
scope is a tuple of variables and rel is the relation, either
represented explicitly or abstractly
 X1 and X2 both have the domain {A, B}
 Constraints:
 <(X1, X2), [(A, B), (B, A)]>, or
 <(X1, X2), X1 ≠ X2>

3
Solution
 Each state in a CSP is defined by an assignment of
values to some or all of the variables
 An assignment that does not violate any constraints
is called a consistent or legal assignment
 A complete assignment is one in which every
variable is assigned
 A solution to a CSP is consistent and complete
assignment
 Allows useful general-purpose algorithms with more
power than standard search algorithms

4
Example: Map Coloring

 Variables: X = {WA, NT, Q, NSW, V, SA, T }

 Domains: Di = {red, green, blue}
 Constraints: adjacent regions must have different colors
 Solution?

5
Solution: Complete and Consistent Assignment

 Variables: X = {WA, NT, Q, NSW, V, SA, T }

 Domains: Di = {red, green, blue}
 Constraints: adjacent regions must have different colors
 Solution? {WA = red, NT = green, Q = red, NSW = green, V = red,
SA = blue, T = red}.

6
Constraint Graph
 Constraint graph: nodes are variables, arcs are
constraints
 Binary CSP: each constraint relates two variables
 CSP conforms to a standard pattern
 a set of variables with assigned values
 generic successor function and goal test
 generic heuristics
 reduce complexity

7
CSP as a Search Problem
 Initial state:
 {} – all variables are unassigned
 Successor function:
 a value is assigned to one of the unassigned variables with
no conflict
 Goal test:
 a complete assignment
 Path cost:
 a constant cost for each step
 Solution appears at depth n if there are n variables
 Depth-first or local search methods work well

8
CSP Solvers Can be Faster
 CSP solver can quickly eliminate large part of
search space
 If {SA = blue}
 Then 35 assignments can be reduced to 25
assignments, a reduction of 87%

 In a CSP, if a partial assignment is not a solution,

we can immediately discard further refinements of it

9
Exercise (1)
 Consider the problem of
crossword puzzle: fitting words
into a rectangular grid. Assume
that a list of words is provided
and that the task is to fill in the
blank squares using any subset of
the list. Formulate this problem
precisely in two ways:
 As a general search problem. Choose
an appropriate search algorithm.
 As a constraint satisfaction problem.
 Word vs. letters

10
Exercise (2)
 Problem formulation as CSP:
 Class scheduling: There is a fixed number of
professors and classrooms, a list of classes to be
offered, and a list of possible time slots for
classes. Each professor has a set of classes that
he or she can teach.

11
Types of Variables
 Discrete variables
 finite domains:
 n variables, domain size d  O (d n) complete assignments
 e.g., Boolean CSPs, such as 3-SAT (NP-complete)
 Worst case, can’t solve finite-domain CSPs in less than
exponential time
 infinite domains:
 integers, strings, etc.
 e.g., job scheduling, variables are start/end days for each job
 need a constraint language, e.g., StartJob1 + 5 ≤ StartJob3
 Continuous variables
 e.g., start/end times for Hubble Space Telescope
observations
 linear constraints solvable in polynomial time by linear
programming
12
Types of Constraints
 Unary constraints involve a single variable,
 e.g., SA ≠ green

 Binary constraints involve pairs of variables,

 e.g., SA ≠ WA

 Higher-order constraints involve 3 or more

variables
 e.g., cryptarithmetic column constraints

13
Real-World CSPs
 Assignment problems
 e.g., who teaches what class

 Timetabling problems
 e.g., which class is offered when and where?

 Transportation scheduling

 Factory scheduling

14
What Search Algorithm to Use?
 Since we can formulate CSP problems as standard
search problems, we can apply search algorithms
from chapter 3 & 4
 If breadth-first search were applied
 branching factor? nd
 tree size? nd * (n-1)d * … * d = n! * dn leaves
 complete assignments? dn

 A crucial property to all CSPs: commutativity

 the order of application of any given set of actions has no
effect on the outcome
 Variable assignments are commutative, i.e., [ WA = red
then NT = green ] same as [ NT = green then WA = red ]

15
Backtracking Search
 Only need to consider assignments to a single
variable at each node  b = d and there are d n
leaves

 Backtracking search is used for a depth-first search

that chooses values for one variable at a time and
backtracks when a variable has no legal values left
to assign

 Backtracking search is the basic uninformed

algorithm for CSPs

16
Backtracking Search Algorithm

17
Backtracking Example

18
Backtracking Example

19
Backtracking Example

20
Backtracking Example

21
Improving Backtracking Efficiency
 We can solve CSPs efficiently without
domain-specific knowledge, addressing the
following questions:

 in what order should variables be assigned,

values be tried?

 what are the implications of the current variable

assignments for the other unassigned variables?

 when a path fails, can the search avoid repeating

this failure?

22
Variable and Value Ordering 1/3
 Minimum remaining values (MRV)
 choose the variable with the fewest “legal”
values
 also called most constrained variable or
fail-first heuristic
 does it help in choosing the first variable?

23
Variable and Value Ordering 2/3
 Most constraining variable:
 selecting the variable that has the largest
number of constraints on other
unassigned variables
 also called degree heuristics
 Tie-breaker among MRV

24
Variable and Value Ordering 3/3
 Given a variable, choose the least
constraining value:
 the one that rules out the fewest values in the
remaining variables

 Combining these heuristics makes 1000

queens feasible

25
Propagating Information through Constraints

 Forward checking:
 Keep track of remaining legal values for unassigned
variables
 Terminate search when any variable has no legal values

26
Propagating Information through Constraints

 Forward checking:
 Keep track of remaining legal values for unassigned
variables
 Terminate search when any variable has no legal values

27
Propagating Information through Constraints

 Forward checking:
 Keep track of remaining legal values for unassigned
variables
 Terminate search when any variable has no legal values

28
Propagating Information through Constraints

 Forward checking:
 Keep track of remaining legal values for unassigned
variables
 Terminate search when any variable has no legal values

29
Propagating Information through Constraints
 Forward checking propagates information from assigned to
unassigned variables, but doesn't provide early detection for
all failures:

 NT and SA cannot both be blue!

 Constraint propagation repeatedly enforces constraints locally
by propagating implications of a constraint of one variable
onto other variables

30
Node Consistency
 A single variable is node-consistent if all the
values in the variable’s domain satisfy the
variable’s unary constraints

 For example, SA dislikes green

 A network is node-consistent if every

variable in the network is node-consistent

31
Arc Consistency
 Simplest form of propagation makes each arc consistent
 X Y is consistent iff
for every value x of X there is some allowed y

32
Arc Consistency
 Simplest form of propagation makes each arc consistent
 X Y is consistent iff
for every value x of X there is some allowed y

33
Arc Consistency
 Simplest form of propagation makes each arc consistent
 X Y is consistent iff
for every value x of X there is some allowed y

 If X loses a value, neighbors of X need to be rechecked

34
Arc Consistency
 Simplest form of propagation makes each arc consistent
 X Y is consistent iff
for every value x of X there is some allowed y

 If X loses a value, neighbors of X need to be rechecked

 Arc consistency detects failure earlier than forward checking
 Can be run as a preprocessor or after each assignment

35
AC-3 Algorithm
 Time complexity O (n 2d 3)

36
Example
 Use the AC-3 algorithm to show that arc consistency is
able to detect the inconsistency of the partial
assignment {WA = red, V = blue} in the following map
coloring problem.

37
One Possible Trace
 One possible trace of the algorithm:
 remove SA-WA, delete R from SA
 remove SA-V, delete B from SA, leaving only G
 remove NT-WA, delete R from NT
 remove NT-SA, delete G from NT, leaving only B
 remove NSW-SA, delete G from NSW
 remove NSW-V, delete B from NSW, leaving only R
 remove Q-NT, delete B from Q
 remove Q-SA, delete G from Q
 remove Q-NSW, delete R from Q, leaving no proper assignment
for Q

38
Path Consistency
 Consider map-coloring with only two colors
 Every arc is consistent initially
 Check the set {WA, SA} path consistenty
with respect to NT

 More generally, k-consistency

 1-consistency = node consistency
 2-consistency = arc consistency
 3-consistency = path consistency

39
Local Search for CSPs
 Hill-climbing, simulated annealing typically work
with "complete" states, i.e., all variables assigned

 To apply to CSPs:
 allow states with unsatisfied constraints
 operators reassign variable values

 Variable selection: randomly select any conflicted

variable

 Value selection by min-conflicts heuristic:

 choose value that violates the fewest constraints
 i.e., hill-climb with h(n) = total number of violated
constraints

40
Example: 4-Queens
 States: 4 queens in 4 columns (44 = 256 states)
 Actions: move queen in column
 Goal test: no attacks
 Evaluation: h(n) = number of attacks

41
8-Queen Example

Min-conflicts is quite effective for many CSPs.

Given a random initial state, can solve n-queens in
almost constant time for arbitrary n with high
probability (e.g., n = 10,000,000)

42
Min-Conflicts Algorithms for CSP

43
Summary
 CSPs are a special kind of problem:
 states defined by values of a fixed set of variables
 goal test defined by constraints on variable values

 Backtracking = depth-first search with one variable assigned

per node

 Variable ordering and value selection heuristics help

significantly

 Forward checking prevents assignments that guarantee later

failure

 Constraint propagation (e.g., arc consistency) does additional

work to constrain values and detect inconsistencies

 Iterative min-conflicts is usually effective in practice

44