18CSC305J– Artificial Intelligence

UNIT – 2
 Unit 2 List of Topics

• Searching techniques – Uninformed search – • AO* research

General search Algorithm • Local search Algorithms-Hill Climbing, Simulated
• Uninformed search Methods – Breadth First Annealing
Search
• Local Beam Search
• Uninformed search Methods – Depth First • Genetic Algorithms
Search
• Uninformed search Methods – Depth limited • Adversarial search Methods-Game playing-
Search
Important concepts
• Game playing and knowledge structure.
• Uniformed search Methods- Iterative Deepening
search
• Game as a search problem-Minimax Approach
• Bi-directional search
• Minimax Algorithm
• Informed search- Generate and test, Best First
search • Alpha beta pruning
• Informed search-A* Algorithm • Game theory problems
 General Search Strategies

•Blind search  traversing the •Heuristic search  search

search space until the goal process takes place by
nodes is found (might be doing traversing search space with
exhaustive search). applied rules (information).
•Techniques: Greedy Best First
•Techniques : Breadth First,
Search, A* Algorithm
Depth first, Depth Limited,
Iterative Deepening search, •There is no guarantee that
Uniform Cost search. solution is found.
•Guarantees solution.
 Important Terms

• Search space  possible conditions and solutions.

• Initial state  state where the searching process started.
• Goal state  the ultimate aim of searching process.
• Problem space  “what to solve”
• Searching strategy strategy for controlling the search.
• Search tree  tree representation of search space, showing
possible solutions from initial state.
Blind Search : Breadth First Search

1 2

3 4
 Blind strategies B F S

Search M e t h o d s

Blind Search : Breadth First Search

BFS characteristics
Completeness: if the branching factor is ‘b’ and the goal
node is at depth d , BFS will eventually fin d it.
Optimality: BFS is optimal if path cost is a non-decreasing
function of depth. a

1 + b + b2 + b3 + . . . + b d + b(b d − 1) =
Time complexity:

O(b d+1
).
Space complexity: O(b d + 1 ) . b
a
Otherwise, the shallowest node may not necessarily be optimal.
b
b branching factor; d depth of the goal node

spring 2011
 Blind Search : Depth First Search (DFS)
Implementation:
fringe = LIFO queue, i.e., put successors at front.
1 2 3

4 5 N+1

…….
 Search M e t h o d s

Blind Search : Depth First Search (DFS)

DFS characteristics
Small space requirements: only the path to the current node
and the siblings of each node in the path are stored.
Backtracking search generates only one successor for
each node.
Completeness: no, if the expanded subtree has an
infinite depth.
Optimality: no, if a solution located deeper, but located in a

Time complexity: O(bm ).

subtree expanded earlier, is found.

Space complexity: O(bm) (linear!).

spring 2011
 Search M e t h o d s

Blind Search : Depth Limited Search (DLS)

In Depth Limited Search, we first set a constraint on how deep (or how far from root)
will we go.

In the above example , If we fix the depth limit to 2, DLS can be

carried out similarly to the DFS until the goal node is found to exist in
the search domain of the tree.

spring 2011
 Unit 2 List of Topics

• Searching techniques – Uninformed search – • AO* research

General search Algorithm • Local search Algorithms-Hill Climbing, Simulated
• Uninformed search Methods – Breadth First Annealing
Search
• Local Beam Search
• Uninformed search Methods – Depth First • Genetic Algorithms
Search
• Uninformed search Methods – Depth limited • Adversarial search Methods-Game playing-
Search
Important concepts
• Game playing and knowledge structure.
• Uniformed search Methods- Iterative Deepening
search
• Game as a search problem-Minimax Approach
• Bi-directional search
• Minimax Algorithm
• Informed search- Generate and test, Best First
search • Alpha beta pruning
• Informed search-A* Algorithm • Game theory problems
Blind Search : Iterative Deepening DFS
(ID-DFS)

DEPTH
LIMITED
SEARCH
Blind Search : Iterative Deepening DFS
(ID-DFS)
 Blind Search : Iterative Deepening DFS Search M e t h o d s

(ID-DFS)
IDS characteristics
Completeness: yes.
Optimality: yes, if step cost = 1.

(d + 1)b0 + db1 + (d − 1)b2 + .

Time complexity:

. . + b d = O(b d ).
Space complexity: O(bd) .
Numerical comparison for b = 10 , d = 5

N(IDS ) = 50 + 400 + 3000 + 20000 + 100000 = 123450

N(BFS) = 10 + 100 + 1000 + 10000 + 100000 + 999990 =
1111100
Conclusion
IDS exhibits better performance, because it does not expand other
nodes at depth d .
 APPLICATION: MAZE GAME

• HOW TO REACH TO THE GOAL?

 APPLICATION: MAZE GAME

• BFS SOLUTION?
– S-1-2-3-5-8-10-12-14-16-19-G
– SEARCH SOLUTION FOUND IN 12
STEPS

 DFS SOLUTION?
 S-1-2-3-6-5-8-9-10-11-13-16-18-G
 SEARCH SOLUTION FOUND IN 14 STEPS
 APPLICATION: MAZE GAME

• BFS SOLUTION?
– S-1-2-3-5-8-10-12-14-16-19-G
– SEARCH SOLUTION FOUND IN 12 STEPS

15
S 5 19

6 7 8 17 18
1 G 21

10
9 20 21
2
12
11 x
3 x 19

14
13
5 4
16
15

19
 APPLICATION: MAZE GAME

• BFS SOLUTION?
– S-1-2-3-5-8-10-12-14-16-19-G
– SEARCH SOLUTION FOUND IN 12 STEPS

15
S 5 19

6 7 8 17 18
1 G 21

10
9 20 21
2
12
3
11 19 x
x
14
13
5 4
16
15

19
 Blind Search : Uniform Cost Search

• This algorithm comes into play when a different cost is

available for each edge.
• The primary goal of the uniform-cost search is to find a
path to the goal node which has the lowest cumulative
cost.
• Uniform-cost search expands nodes according to their path
costs form the root node. It can be used to solve any
graph/tree where the optimal cost is in demand.
• A uniform-cost search algorithm is implemented by the
priority queue.
• It gives maximum priority to the lowest cumulative cost.
• Uniform cost search is equivalent to BFS algorithm if the
path cost of all edges is the same.
Uniform Cost Search - Example
Uniform Cost Search - Example
Uniform Cost Search - Example
Uniform Cost Search - Example
Uniform Cost Search - Example
Uniform Cost Search - Example
Uniform Cost Search - Example
Uniform Cost Search - Example
Uniform Cost Search - Example
 Uniform Cost Search - Example

Minimum is S->D->C->G2
And also G2 is one of the destination nodes thus we found our
path.
In this way we can find the path with the minimum cumulative cost from
a start node to ending node – S->D->C->G2 with cost total cost as
13(marked with green color).
 Uniform Cost Search

Implementation: fringe =
queue ordered by path cost
Equivalent to breadth-first if
all step costs all equal.
Breadth-first is only optimal
if step costs is increasing
with depth.
(e.g. constant). Can we
guarantee optimality for
any step cost?
Uniform-cost Search:
Expand node with
smallest
path cost g(n).
 Blind strategies UCS

Search M e t h o d s

Uniform Cost Search

Uniform-cost search strategy

Expands the node with the lowest cost from the start node
rst. a
Completeness: yes, if cost of each step ≥ s > 0.
Optimality: same as above nodes are expanding in

Time and space complexity: O(b1 +ƒ C ∗/s¶ ) . b

increasing order of cost.

a
If all step costs are equal, UCS=BFS.
C ∗cost of optimal solution
b
 Blind strategies UCS

Search M e t h o d s

Uniform Cost Search

Blind search -Bidirectional Search

32
 Summary of Blind Search Algorithms

Criterion Breadt Dept

h- h-
First First
Time bd bm

Space bd bm
Optimal? Yes No

Complete Yes No
?
b: branching d: solution m: maximum
factor depth depth
33
 Summary of Blind Search Algorithms

Algorithm Space Time Complete Optimal

BFS Theta Theta(b^d) Yes Yes
(b^d)
DFS Theta(d) Theta(b^m No No
)
UCS Theta(b^ Theta(b^d) Yes Yes
(ceil(C*/e))
DLS Theta(l) Theta(b^l) No No
IDS Theta(d) Theta(b^d) Yes Yes

34
Summary of Search Algorithms

35
 Unit 2 List of Topics

• Searching techniques – Uninformed search – • AO* search

General search Algorithm • Local search Algorithms-Hill Climbing, Simulated
• Uninformed search Methods – Breadth First Annealing
Search
• Local Beam Search
• Uninformed search Methods – Depth First • Genetic Algorithms
Search
• Uninformed search Methods – Depth limited • Adversarial search Methods-Game playing-
Search
Important concepts
• Game playing and knowledge structure.
• Uniformed search Methods- Iterative Deepening
search
• Game as a search problem-Minimax Approach
• Bi-directional search
• Minimax Algorithm
• Informed search- Generate and test, Best First
search • Alpha beta pruning
• Informed search-A* Algorithm • Game theory problems
 Informed Search Algorithms

• Generate and Test

• Best-first search

• Greedy best-first search

• A* search
• Heuristics
 Generate-and-test

• Very simple strategy - just keep guessing.

do while goal not accomplished

generate a possible solution
test solution to see if it is a goal

• Heuristics may be used to determine the

specific rules for solution generation.

38
 Generate-and-test
Example - Traveling Salesman Problem
(TSP)
• Traveler needs to visit n cities.
• Know the distance between each pair of cities.
• Want to know the shortest route that visits all
the cities once.
• n=80 will take millions of years to solve
exhaustively!

39
Generate-and-test

TSP Example
A 6 B
1 2
5 3

D 4 C

40
 Generate-and-test Example

• TSP - generation of possible

solutions is done in
A B C D
lexicographical order of
cities:
1. A - B - C-D B C D

2. A - B - D-C
3. A - C - B-D C D B D C B
4. A - C - D-B
...
D C D B B C

41
 Best First Search Algorithms

• Idea: use an evaluation function f(n) for each node

– f(n) provides an estimate for the total cost.
 Expand the node n with smallest f(n).

• Implementation:
Order the nodes in fringe increasing order of cost.

• Special cases:
– greedy best-first search
– A* search
Romania with straight-line dist.
 Greedy best-first search
• f(n) = estimate of cost from n to goal
• e.g., f(n) = straight-line distance from n to
Bucharest
• Greedy best-first search expands the node
that appears to be closest to goal.
Greedy best-first search example
Greedy best-first search example
Greedy best-first search example
Greedy best-first search example
 GBFS is not complete

c
b g

a goal state
start state

f(n) = straightline distance

 Properties of greedy best-first search

• Complete? No – can get stuck in loops.

• Time? O(bm), but a good heuristic can give
dramatic improvement
• Space? O(bm) - keeps all nodes in memory
• Optimal? No
e.g. AradSibiuRimnicu
VireaPitestiBucharest is shorter!
 Unit 2 List of Topics

• Searching techniques – Uninformed search – • AO* search

General search Algorithm • Local search Algorithms-Hill Climbing, Simulated
• Uninformed search Methods – Breadth First Annealing
Search
• Local Beam Search
• Uninformed search Methods – Depth First • Genetic Algorithms
Search
• Uninformed search Methods – Depth limited • Adversarial search Methods-Game playing-
Search
Important concepts
• Game playing and knowledge structure.
• Uniformed search Methods- Iterative Deepening
search
• Game as a search problem-Minimax Approach
• Bi-directional search
• Minimax Algorithm
• Informed search- Generate and test, Best First
search • Alpha beta pruning
• Informed search-A* Algorithm • Game theory problems
 A* search
• Idea: avoid expanding paths that are already
expensive
• Evaluation function f(n) = g(n) + h(n)
• g(n) = cost so far to reach n
• h(n) = estimated cost from n to goal
• f(n) = estimated total cost of path through n to goal
• Best First search has f(n)=h(n)
• Uniform Cost search has f(n)=g(n)
 Applications of
Heuristic Search : A* Algorithm

•Widely known algorithm – (pronounced as “A

star” search).
•Evaluates nodes by combining g(n) “cost to
reach the node” and h(n) “cost to get to the
goal”
•f(n) = g(n) + h(n), f(n)  estimated cost of
the cheapest solution.
•Complete and optimal ~ since evaluates all
paths.

Demystifying AI algorithms
 Heuristic Search : A* Algorithm
Path cost for S-D-G
S :10 f(S) = g(S) + h(S)
2 3 = 0 + 10  10
f(D) = (0+3) + 9  12
A :8 D :9 f(G) = (0+3+3) + 0  6
5
2 3 1 Total path cost = f(S)+f(D)+f(G) 28
Path cost for S-A-G’
E G’ G H
f(S) = 0 + 10  10
:4 :0 :0 :3
f(A) = (0+2) + 8  10
f(G’) = (0+2+2) + 0  4
* Path S-A-G’ is chosen =
Lowest cost Total path cost = f(S)+f(A)+f(G’) 24
Application of Heuristic Search : A* Algorithm –
Snake & Ladder
12 h(n)=0 11 10 9g(n)=8, h(n)=8,
g(n)=1, h(n)=2, f(n)=3 f(n)=16

8 7 6 5
1 2 3 4
g(n)=0, h(n)=11, f(n)=11 g(n)=1, h(n)=2, f(n)=3 g(n)=2, h(n)=9, f(n)=11 g(n)=3, h(n)=8, f(n)=11

1 2 (10) 3 4 5 6 7
2 (10) 3 4 5 6 7 8
3 4 5 6 7 8 9 (4)

4 5 6 7 8 9 (4) 10

5 6 7 8 9 (4) 10 11

6 7 8 9 (4) 10 11 12

7 8 9 (4) 10 11 12 X

8 9 (4) 10 11 12 X
9 (4) 10 11 12 X
10 11 12 X
11 12 X
12 X
Demystifying AI algorithms
A* search example
A* search example
A* search example
A* search example
A* search example
A* search example
 try yourself !

6 1 straight-line distances
3 A D F 1
h(S-G)=10
2 h(A-G)=7
S 4 8 G
B E h(D-G)=1
h(F-G)=1
1 20 h(B-G)=10
C h(E-G)=8
h(C-G)=20
h(G-G)=0

The graph above shows the step-costs for different paths going from the start (S) to
the goal (G). On the right you find the straight-line distances.

1. Draw the search tree for this problem. Avoid repeated states.

2. Give the order in which the tree is searched (e.g. S-C-B...-G) for A* search.
Use the straight-line dist. as a heuristic function, i.e. h=SLD,
and indicate for each node visited what the value for the evaluation function, f, is.
 Admissible heuristics

• A heuristic h(n) is admissible if for every node n,

h(n) ≤ h*(n), where h*(n) is the true cost to reach the
goal state from n.
• An admissible heuristic never overestimates the cost to
reach the goal, i.e., it is optimistic
• Example: hSLD(n) (never overestimates the actual road
distance)
• Theorem: If h(n) is admissible, A* using TREE-SEARCH
is optimal
63
Admissible heuristics – 8 Puzzle Problem

Try it out!
START
1 2 3
7 8 4
6 5

GOAL
1 2 3
8 4
7 6 5

64
 Admissible heuristics – 8 Puzzle Problem

START
1 2 3
START
7 8 4
1 2 3
6 5
7 8 4
6 5
START START START
1 2 3 1 2 3 1 2 3
GOAL 7 4 7 8 4 7 8 4
1 2 3 6 8 5 6 5 6 5
8 4
7 6 5 H=5 H=4 H=6

65
 Admissible heuristics
E.g., for the 8-puzzle:
• h1(n) = number of misplaced tiles
• h2(n) = total Manhattan distance
(i.e., no. of squares from desired location of each tile)

• h1(S) = ?
• h2(S) = ?

66
 Evaluation of Search Algorithms
Completeness
Is the algorithm guaranteed to n d a solution if one exists?

Optimality
When the algorithm n d s a solution, is this the optimal one?

Time complexity
How long does it take to n d a solution? a
a
Often measured as number of nodes generated during search.

Space complexity
How much memory is needed to perform the search? a
a
Often measured as maximum number of nodes stored in memory.
 Unit 2 List of Topics

• Searching techniques – Uninformed search – • AO* search

General search Algorithm • Local search Algorithms-Hill Climbing, Simulated
• Uninformed search Methods – Breadth First Annealing
Search
• Local Beam Search
• Uninformed search Methods – Depth First • Genetic Algorithms
Search
• Uninformed search Methods – Depth limited • Adversarial search Methods-Game playing-
Search
Important concepts
• Game playing and knowledge structure.
• Uniformed search Methods- Iterative Deepening
search
• Game as a search problem-Minimax Approach
• Bi-directional search
• Minimax Algorithm
• Informed search- Generate and test, Best First
search • Alpha beta pruning
• Informed search-A* Algorithm • Game theory problems
 Algorithm AO*

E.g.,
1. forInitialize:
the 8-puzzle: Set G* = {s}, f(s) = h(s)
• h1(n) = number of misplaced
If s  T,tiles
label s as SOLVED
• 2.h2(n)Terminate:
= total Manhattan distance If s is SOLVED, then terminate
(i.e.,
3. no.Select:
of squares from desired
Select alocation of each
non-terminal leaftile)
node n from the
marked sub-tree below s in G*
4. Expand: Make explicit the successors of n
For each successor, m, not already in G*:
Set f(m) = h(m)
If m is a terminal node, label m as
SOLVED
• 5.h1(S) = ?Revision: Call cost-revise(n)
Cost

• 6.h2(S)
Loop: Go To Step 2.
=?

69
 Cost Revision in AO*: cost-revise(n)

E.g.,
1. for theZ 8-puzzle:
Create = {n}
•2. h1(n)
If Z =={ number
} return of misplaced tiles
Select a node m from Z such that m has no descendants in Z
•3. h2(n) = total Manhattan distance
4. If m is an AND node with successors r1, r2, … rk:
(i.e., no. of squares from desired location of each tile)
Set f(m) =  [ f(ri) + c(m, ri) ]
Mark the edge to each successor of m
If each successor is labeled SOLVED, then label m as SOLVED
• If m is an OR node with successors r1, r2, … rk:

Set f(m) = min { f(ri) + c(m, ri) }

Mark the edge to the best successor of m
• h (S) = ? If the marked successor is labeled SOLVED, label m as SOLVED
1If the cost or label of m has changed, then insert those parents of m into Z for which m is a
1.
• h (S) = ?
marked successor
2Go to Step 2.
2.

70
 Searching OR Graphs
E.g.,
• for
Howthe 8-puzzle:
does AO* fare when the graph has only OR nodes?
• • h1(n)
What= number of misplaced
are the roles tiles and upper-bound estimates?
of lower-bound
• h2(n) – =Pruning criteria: LB distance
total Manhattan > UB
(i.e., no. of squares from desired location of each tile)
Searching Game Trees
• Consider an OR tree with two types of OR nodes, namely Min nodes and Max
nodes
• In Min nodes we select the minimum cost successor
• In Max nodes we select the maximum cost successor

• Terminal nodes are winning or loosing states

• h1(S)
– It is often infeasible to search up to terminal nodes
= ?
– We use heuristic costs to compare non-terminal nodes
• h2(S) = ?

71
 Shallow and Deep Pruning
E.g., for the 8-puzzle:
• h1(n) = number of misplaced
ROOT tiles ROOT
• h2(n) = total Manhattan distance
(i.e., no. of squares
A from desired location of each tile)
10
B
10 F

Max node
14 C D
G Min node

• h1(S) = ? 5 E

• h2(S) = ? Cut-off
Shallow Deep Cut-off

72
 Unit 2 List of Topics

• Searching techniques – Uninformed search – • AO* search

General search Algorithm • Local search Algorithms-Hill Climbing, Simulated
• Uninformed search Methods – Breadth First Annealing
Search
• Local Beam Search
• Uninformed search Methods – Depth First • Genetic Algorithms
Search
• Uninformed search Methods – Depth limited • Adversarial search Methods-Game playing-
Search
Important concepts
• Game playing and knowledge structure.
• Uniformed search Methods- Iterative Deepening
search
• Game as a search problem-Minimax Approach
• Bi-directional search
• Minimax Algorithm
• Informed search- Generate and test, Best First
search • Alpha beta pruning
• Informed search-A* Algorithm • Game theory problems
Local search algorithms

74
 Local search algorithms
• In many optimization problems, the path to the goal is
irrelevant; the goal state itself is the solution

• State space = set of "complete" configurations

• Find configuration satisfying constraints, e.g., n-queens
• In such cases, we can use local search algorithms
• keep a single "current" state, try to improve it.
• Very memory efficient (only remember current state)

75
Types of Local Search

• Hill-climbing Search
• Simulation Annealing Search

76
Terminology of Local
Search

77
 Local search algorithms
Hill Climbing

• Searching for a goal state = Climbing to the

top of a hill.
• Generate‐and‐test + direction to move.
• Heuristic function to estimate how close a
given state is to a goal state.

78
 Hill Climbing

Algorithm
1. Evaluate the initial state.
2. Loop until a solution is found or there are no
new operators left to be applied:
 Select and apply a new operator
 Evaluate the new state:
goal  quit
better than current state  new current state

79
 Steepest Ascent Hill Climbing

• Considers all the moves from the current state.

• Selects the best one as the next state.
Steepest-Ascent Hill Climbing
• current  start node
• loop do
– neighbor  a highest-valued successor of current
– if neighbor.Value <= current.Value then return current.State
– current  neighbor
• end loop

80
Steepest Ascent/Descent Hill Climbing

current
6

4 10 3 2 8

What if current had a value of 12?

81
 Hill Climbing Example
A local heuristic function

Count +1 for every block that sits on the correct thing. The goal state has
the value +8.

Count -1 for every block that sits on an incorrect thing. In the initial state
blocks C, D, E, F, G, H count +1 each. Blocks A, B count -1 each , for the total
of +4.

Move 1 gives the value +6 (A is now on the correct support). Moves 2a and
2b both give +4 (B and H are wrongly situated). This means we have a local
maximum of +6.

82
 Hill Climbing Example

A global heuristic function

Count +N for every block that sits on a correct stack of N things. The goal
state has the value +28.
Count -N for every block that sits on an incorrect stack of N things. That
is, there is a large penalty for blocks tied up in a wrong structure.
In the initial state C, D, E, F, G, H count -1, -2,
-3, -4, -5, -6. A counts -7 , for the total of -28.

83
 Hill Climbing Example

Move 1 gives the value -21 (A is now on the correct support).

Move 2a gives -16, because C, D, E, F, G, H
count -1, -2, -3, -4, -5, -1.
Move 2b gives -15, because C, D, E, F, G
count -1, -2, -3, -4, -5.
There is no local maximum!
Moral: sometimes changing the heuristic function is all we need.

84
 Hill-climbing search
• Problem: depending on initial state, can get stuck in local maxima

85
 Hill Climbing: Disadvantages

Local maximum
A state that is better than all of its
neighbours, but not better than
some other states far away.
Plateau
A flat area of the search space in
which all neighbouring states have
the same value.
Ridge
The orientation of the high region, compared to
the set of available moves, makes it impossible
to climb up. However, two moves executed
serially may increase the height. 86
 Hill Climbing: Disadvantages

Ways Out
• Backtrack to some earlier node and try going
in a different direction.
• Make a big jump to try to get into a new
solution.
• Moving in several directions at once.

87
 Unit 2 List of Topics

• Searching techniques – Uninformed search – • AO* search

General search Algorithm • Local search Algorithms-Hill Climbing, Simulated
• Uninformed search Methods – Breadth First Annealing
Search
• Local Beam Search
• Uninformed search Methods – Depth First • Genetic Algorithms
Search
• Uninformed search Methods – Depth limited • Adversarial search Methods-Game playing-
Search
Important concepts
• Game playing and knowledge structure.
• Uniformed search Methods- Iterative Deepening
search
• Game as a search problem-Minimax Approach
• Bi-directional search
• Minimax Algorithm
• Informed search- Generate and test, Best First
search • Alpha beta pruning
• Informed search-A* Algorithm • Game theory problems
Simulated Annealing – Basic Steps

89
 Simulated annealing search
• Idea: escape local maxima by allowing some "bad"
moves but gradually decrease their frequency.

• This is like smoothing the cost landscape.

• One can prove: If T decreases slowly enough, then

simulated annealing search will find a global
optimum with probability approaching 1 (however,
this may take VERY long)

90
 Simulated annealing search
• A variation of hill climbing in which, at the
beginning of the process, some downhill moves
may be made.

• To do enough exploration of the whole space

early on, so that the final solution is relatively
insensitive to the starting state.
• Lowering the chances of getting caught at a local
maximum, or plateau, or a ridge.

91
 Simulated annealing search
Physical Annealing
• Physical substances are melted and then
gradually cooled until some solid state is reached.
• The goal is to produce a minimal‐energy state.
• Annealing schedule: if the temperature is
lowered sufficiently slowly, then the goal will be
attained.
• Nevertheless, there is some probability for a
transition to a higher energy state: eΔE/kT.

92
Simulated annealing algorithm

93
80
Simulated annealing example

94
Simulated annealing example

95
 Simulated annealing example
1. The objective function to minimize is a simple function of two variables:
min f(x) = (4 - 2.1*x1^2 + x1^4/3)*x1^2 + x1*x2 + (-4 + 4*x2^2)*x2^2;
x1, x2 >=0, x1,x2 <=20 5bits Tmin = 50

e(-
Delta DeltaE/T)
Decimal Decimal Binary Decimal Binary Decimal Delta E = f(x')- E<0? < r? =
k=1 Binary x1 x1 Binary x2 x2 f(x) x1' x1' x2' x2' f(x') f(x) Accept e(-DeltaE/T) r Accept
T=300 10 2 1001 925941.73 0 0 1000 8 16128 -9813.733333Yes Accept
T=T/2=150 0 0 1000 8 16128 1 1 1100 1282382.23 66254.23333No 1.494E-192 0.6Accept
T=75 1 1 1100 1282382.23 0 0 1110 14 152880 70497.76667No 0 0.7Accept
T=37.5
16128

6
2 5 1 1 3 2 1 2

96
 Unit 2 List of Topics

• Searching techniques – Uninformed search – • AO* search

General search Algorithm • Local search Algorithms-Hill Climbing, Simulated
• Uninformed search Methods – Breadth First Annealing
Search
• Local Beam Search
• Uninformed search Methods – Depth First • Genetic Algorithms
Search
• Uninformed search Methods – Depth limited • Adversarial search Methods-Game playing-
Search
Important concepts
• Game playing and knowledge structure.
• Uniformed search Methods- Iterative Deepening
search
• Game as a search problem-Minimax Approach
• Bi-directional search
• Minimax Algorithm
• Informed search- Generate and test, Best First
search • Alpha beta pruning
• Informed search-A* Algorithm • Game theory problems
 Local Beam Search

• Keep track of k states rather than just one, as

in hill climbing
• In comparison to beam search we saw earlier,
this algorithm is state-based rather than node-
based.

98
 Local Beam Search

• Begins with k randomly generated states

• At each step, all successors of all k states are
generated
• If any one is a goal, alg halts
• Otherwise, selects best k successors from the
complete list, and repeats

99
 Local Beam Search

• Successors can become concentrated in a small part of state

space
• Stochastic beam search: choose k successors, with probability
of choosing a given successor increasing with value
• Like natural selection: successors (offspring) of a state
(organism) populate the next generation according to its value
(fitness)

100
 Unit 2 List of Topics

• Searching techniques – Uninformed search – • AO* search

General search Algorithm • Local search Algorithms-Hill Climbing, Simulated
• Uninformed search Methods – Breadth First Annealing
Search
• Local Beam Search
• Uninformed search Methods – Depth First • Genetic Algorithms
Search
• Uninformed search Methods – Depth limited • Adversarial search Methods-Game playing-
Search
Important concepts
• Game playing and knowledge structure.
• Uniformed search Methods- Iterative Deepening
search
• Game as a search problem-Minimax Approach
• Bi-directional search
• Minimax Algorithm
• Informed search- Generate and test, Best First
search • Alpha beta pruning
• Informed search-A* Algorithm • Game theory problems
 Genetic Algorithms

• Genetic Algorithms(GAs) are adaptive heuristic search algorithms

that belong to the larger part of evolutionary algorithms. Genetic
algorithms are based on the ideas of natural selection and
genetics.
• They are commonly used to generate high-quality solutions for
optimization problems and search problems.
• n simple words, they simulate “survival of the fittest” among
individual of consecutive generation for solving a problem. Each
generation consist of a population of individuals and each
individual represents a point in search space and possible
solution. Each individual is represented as a string of
character/integer/float/bits. This string is analogous to the
Chromosome.
 Genetic Algorithms

• Search Space

– The population of individuals are maintained within search space. Each

individual represent a solution in search space for given problem. Each
individual is coded as a finite length vector (analogous to chromosome) of
components. These variable components are analogous to Genes. Thus a
chromosome (individual) is composed of several genes (variable components).
– A Fitness Score is given to each individual which shows the ability of an
individual to “compete”. The individual having optimal fitness score (or near
optimal) are sought.
Genetic Algorithms
Genetic Algorithms

• Cross Over

• Mutation
 GA can be summarized as:

• 1) Randomly initialize populations p

• 2) Determine fitness of population
• 3) Until convergence repeat:
– a) Select parents from population
– b) Crossover and generate new population
– c) Perform mutation on new population
– d) Calculate fitness for new population

https://www.geeksforgeeks.org/genetic-algorithms/
Random Number Table for Solving
GA Problem
 GA Calculation

Initial Pop Decimal Fitness Pi Ei Actual New Crossover

Count Population and
Mutation
 GA Example
Initial Decimal F(Z) Pi = Ei = F(z)/ Actual NEW Crossov Mutat Offsprings Decimal
Pop F(z)/Sum Avg(F(z)) Count POP er ion
F(z)
00010 2 4 0.004 0.016 0 1|1010 11110 11110 30

01001 9 81 0.084 0.338 0 11010 - 11110 11110 30

11010 26 676 0.706 2.825 3 11010 11010 26

01110 14 196 0.206 0.81 1 0|1110 01010 01010 10

Sum F(z) 937

Avg F(z) 239.25

F(z) = x^2, 0<=x<=30;

Pc= 0.6, i.e. 0.6*4 = 2.4; i.e. 2 strings
Pm=0.2 , i.e. 0.2*4= 0.8, i.e.1 string
 Unit 2 List of Topics

• Searching techniques – Uninformed search – • AO* search

General search Algorithm • Local search Algorithms-Hill Climbing, Simulated
• Uninformed search Methods – Breadth First Annealing
Search
• Local Beam Search
• Uninformed search Methods – Depth First • Genetic Algorithms
Search
• Uninformed search Methods – Depth limited • Adversarial search Methods-Game playing-
Search
Important concepts
• Game playing and knowledge structure.
• Uniformed search Methods- Iterative Deepening
search
• Game as a search problem-Minimax Approach
• Bi-directional search
• Minimax Algorithm
• Informed search- Generate and test, Best First
search • Alpha beta pruning
• Informed search-A* Algorithm • Game theory problems
 Adversarial search Methods-Game
playing-Important concepts

• Adversarial search: Search based on Game theory-

Agents- Competitive environment
• According to game theory, a game is played between
two players. To complete the game, one has to win the
game and the other looses automatically.’
• Such Conflicting goal- adversarial search
• Game playing technique- Those games- Human
Intelligence and Logic factor- Excluding other factors like
Luck factor
– Tic-Tac-Toe, Checkers, Chess – Only mind works, no luck
works
Adversarial search Methods-Game
playing-Important concepts
• Techniques required to get the
best optimal solution (Choose
Algorithms for best optimal
solution within limited time)
– Pruning: A technique which
allows ignoring the unwanted
portions of a search tree
which make no difference in
its final result.
– Heuristic Evaluation
Function: It allows to
approximate the cost value at
each level of the search tree,
before reaching the goal node.
 Unit 2 List of Topics

• Searching techniques – Uninformed search – • AO* search

General search Algorithm • Local search Algorithms-Hill Climbing, Simulated
• Uninformed search Methods – Breadth First Annealing
Search
• Local Beam Search
• Uninformed search Methods – Depth First • Genetic Algorithms
Search
• Uninformed search Methods – Depth limited • Adversarial search Methods-Game playing-
Search
Important concepts
• Game playing and knowledge structure.
• Uniformed search Methods- Iterative Deepening
search
• Game as a search problem-Minimax Approach
• Bi-directional search
• Minimax Algorithm
• Informed search- Generate and test, Best First
search • Alpha beta pruning
• Informed search-A* Algorithm • Game theory problems
 Game playing and knowledge structure-
Elements of Game Playing search
• To play a game, we use a game tree to know all • For example, in chess, tic-tac-toe, we have two
the possible choices and to pick the best one out. or three possible outcomes. Either to win, to
There are following elements of a game-playing: lose, or to draw the match with values +1,-1 or
• S0: It is the initial state from where a game begins. 0.
• PLAYER (s): It defines which player is having the • Game Tree for Tic-Tac-Toe
current turn to make a move in the state. – Node: Game states, Edges: Moves taken by players
• ACTIONS (s): It defines the set of legal moves to
be used in a state.
• RESULT (s, a): It is a transition model which
defines the result of a move.
• TERMINAL-TEST (s): It defines that the game has
ended and returns true.
• UTILITY (s,p): It defines the final value with which
the game has ended. This function is also known
as Objective function or Payoff function. The
price which the winner will get i.e.
• (-1): If the PLAYER loses.
• (+1): If the PLAYER wins.
• (0): If there is a draw between the PLAYERS.

https://www.tutorialandexample.com/adversarial-search-in-artificial-intelligence/#:~:text=AI%20Adversarial%20search%3A
%20Adversarial%20search,order%20to%20win%20the%20game.
Game playing and knowledge structure-
Elements of Game Playing search
• INITIAL STATE (S0): The top node in the
game-tree represents the initial state in
the tree and shows all the possible choice
to pick out one.
• PLAYER (s): There are two players, MAX
and MIN. MAX begins the game by
picking one best move and place X in the
empty square box.
• ACTIONS (s): Both the players can make
moves in the empty boxes chance by
chance.
• RESULT (s, a): The moves made
by MIN and MAX will decide the outcome
of the game.
• TERMINAL-TEST(s): When all the empty
boxes will be filled, it will be the
terminating state of the game.
• UTILITY: At the end, we will get to know
who wins: MAX or MIN, and accordingly,
the price will be given to them.
 Unit 2 List of Topics

• Searching techniques – Uninformed search – • AO* search

General search Algorithm • Local search Algorithms-Hill Climbing, Simulated
• Uninformed search Methods – Breadth First Annealing
Search
• Local Beam Search
• Uninformed search Methods – Depth First • Genetic Algorithms
Search
• Uninformed search Methods – Depth limited • Adversarial search Methods-Game playing-
Search
Important concepts
• Game playing and knowledge structure.
• Uniformed search Methods- Iterative Deepening
search
• Game as a search problem-Minimax Approach
• Bi-directional search
• Minimax Algorithm
• Informed search- Generate and test, Best First
search • Alpha beta pruning
• Informed search-A* Algorithm • Game theory problems
 Game as a search problem

• Types of algorithms in Adversarial search

– In a normal search, we follow a sequence of actions to
reach the goal or to finish the game optimally. But in
an adversarial search, the result depends on the
players which will decide the result of the game. It is
also obvious that the solution for the goal state will be
an optimal solution because the player will try to win
the game with the shortest path and under limited
time.
• Minmax Algorithm
• Alpha-beta Pruning
 Game Playing vs. Search

• Game vs. search problem

• "Unpredictable" opponent 
specifying a move for every possible
opponent reply

• Time limits  unlikely to find goal,

must approximate
 Game Playing

• Formal definition of a game:

– Initial state
– Successor function: returns list of (move,
state) pairs
– Terminal test: determines when game over
Terminal states: states where game ends
– Utility function (objective function or payoff
function): gives numeric value for terminal
Westates
will consider games with 2 players (Max and Min);
Max moves first.
 Game Tree Example:
Tic-Tac-Toe

Tree from
Max’s
perspective
 Minimax Algorithm

• Minimax algorithm
– Perfect play for deterministic, 2-player game
– Max tries to maximize its score
– Min tries to minimize Max’s score (Min)
– Goal: move to position of highest minimax value
 Identify best achievable payoff against best play
 Minimax Algorithm

Payoff for Max

 Minimax Rule

• Goal of game tree search: to determine one move for Max

player that maximizes the guaranteed payoff for a given game
tree for MAX
Regardless of the moves the MIN will take
• The value of each node (Max and MIN) is determined by (back
up from) the values of its children
• MAX plays the worst case scenario:
Always assume MIN to take moves to maximize his pay-off
(i.e., to minimize the pay-off of MAX)
• For a MAX node, the backed up value is the maximum of the
values associated with its children
• For a MIN node, the backed up value is the minimum of the
values associated with its children
 Minimax procedure

1. Create start node as a MAX node with current board configuration

2. Expand nodes down to some depth (i.e., ply) of lookahead in the game.
3. Apply the evaluation function at each of the leaf nodes
4. Obtain the “back up" values for each of the non-leaf nodes from its
children by Minimax rule until a value is computed for the root node.
5. Pick the operator associated with the child node whose backed up
value determined the value at the root as the move for MAX
 Minimax Search
2

2 1 2 1

2 7 1 8 2 7 1 8 2 7 1 8

This is the move 2

Static evaluator selected by minimax
value
2 1
MAX

MIN
2 7 1 8
 Minimax Algorithm (cont’d)

3 9 0 7 2 6

Payoff for Max

 Minimax Algorithm (cont’d)

3 0 2

3 9 0 7 2 6

Payoff for Max

 Minimax Algorithm (cont’d)

3 0 2

3 9 0 7 2 6

Payoff for Max

 Minimax Algorithm (cont’d)

• Properties of minimax algorithm:

• Complete? Yes (if tree is finite)
• Optimal? Yes (against an optimal opponent)
• Time complexity? O(bm)
• m – maximum depth of tree; b branching factor

• Space complexity? O(bm) (depth-first

exploration, if it generates all successors at
once)
m – maximum depth of the tree; b – legal moves;
•
 Minimax Algorithm

• Limitations
– Not always feasible to traverse entire tree
– Time limitations
• Key Improvement
– Use evaluation function instead of utility
• Evaluation function provides estimate of utility at given
position
 Unit 2 List of Topics

• Searching techniques – Uninformed search – • AO* search

General search Algorithm • Local search Algorithms-Hill Climbing, Simulated
• Uninformed search Methods – Breadth First Annealing
Search
• Local Beam Search
• Uninformed search Methods – Depth First • Genetic Algorithms
Search
• Uninformed search Methods – Depth limited • Adversarial search Methods-Game playing-
Search
Important concepts
• Game playing and knowledge structure.
• Uniformed search Methods- Iterative Deepening
search
• Game as a search problem-Minimax Approach
• Bi-directional search
• Minimax Algorithm
• Informed search- Generate and test, Best First
search • Alpha beta pruning
• Informed search-A* Algorithm • Game theory problems
 α-β Pruning

• Can we improve search by reducing the size of

the game tree to be examined?

 Yes!!! Using alpha-beta pruning

Principle
– If a move is determined worse than another move already
examined, then there is no need for further examination of the
node.
α-β Pruning Example
α-β Pruning Example (cont’d)
α-β Pruning Example (cont’d)
α-β Pruning Example (cont’d)
α-β Pruning Example (cont’d)
 Alpha-Beta Pruning (αβ prune)

• Rules of Thumb

– α is the best ( highest) found so far along the path for

Max
– β is the best (lowest) found so far along the path for Min
– Search below a MIN node may be alpha-pruned if
the its β   of some MAX ancestor
– Search below a MAX node may be beta-pruned if
the its  β of some MIN ancestor.
 Alpha-Beta Pruning Example

1.Search below a MIN

node may be alpha-
pruned if the beta
value is <= to the
alpha value of some
MAX ancestor.
•
2. Search below a MAX
node may be beta-
pruned if the alpha
value is >= to the beta
value of some MIN
ancestor.
 Alpha-Beta Pruning Example

1.Search below a MIN 3

node may be alpha-
pruned if the beta
value is <= to the
alpha value of some
MAX ancestor.
•
3
2. Search below a MAX
node may be beta-
pruned if the alpha
value is >= to the beta
value of some MIN
ancestor.
3
 Alpha-Beta Pruning Example

1.Search below a MIN 3

node may be alpha-
pruned if the beta
value is <= to the
alpha value of some
MAX ancestor.
•
3
2. Search below a MAX
node may be beta-
pruned if the alpha
value is >= to the beta
value of some MIN
ancestor.
3 5

β
 Alpha-Beta Pruning Example

1.Search below a MIN 3

node may be alpha-
pruned if the beta
value is <= to the
alpha value of some
MAX ancestor.
•
3 0
2. Search below a MAX
node may be beta-
pruned if the alpha α
value is >= to the beta
value of some MIN
ancestor.
3 5 0

β
 Alpha-Beta Pruning Example

1.Search below a MIN 3

node may be alpha-
pruned if the beta
value is <= to the
alpha value of some
MAX ancestor.
•
3 0 2
2. Search below a MAX
node may be beta-
pruned if the alpha α α
value is >= to the beta
value of some MIN
ancestor.
3 5 0 2

β
Another Example
1.Search below a MIN
node may be alpha-
pruned if the beta
value is <= to the
alpha value of some
MAX ancestor.

2. Search below a MAX

node may be beta-
pruned if the alpha
value is >= to the beta
value of some MIN
ancestor.
 Example
1.Search below
a MIN node
may be
alpha-pruned
5 if the beta
5 3 value is <= to
the alpha
value of
some MAX
ancestor.
5 3
2. Search
5 6 3 7 below a MAX
α node may be
beta-pruned
if the alpha
value is >= to
the beta
5 6 3 value of
some MIN
5 0 6 1 3 2 4 7 ancestor.
β
 Properties of α-β Prune

• Pruning does not affect final result

• Good move ordering improves effectiveness of

pruning b(e.g., chess, try captures first, then
threats, froward moves, then backward moves…)

• With "perfect ordering," time complexity = O(bm/2)

 doubles depth of search that alpha-beta pruning can
explore
Example of the value of reasoning about which
computations are relevant (a form of metareasoning)
 Unit 2 List of Topics

• Searching techniques – Uninformed search – • AO* search

General search Algorithm • Local search Algorithms-Hill Climbing, Simulated
• Uninformed search Methods – Breadth First Annealing
Search
• Local Beam Search
• Uninformed search Methods – Depth First • Genetic Algorithms
Search
• Uninformed search Methods – Depth limited • Adversarial search Methods-Game playing-
Search
Important concepts
• Game playing and knowledge structure.
• Uniformed search Methods- Iterative Deepening
search
• Game as a search problem-Minimax Approach
• Bi-directional search
• Minimax Algorithm
• Informed search- Generate and test, Best First
search • Alpha beta pruning
• Informed search-A* Algorithm • Game theory problems
 What is Game Theory?

• It deals with Bargaining.

• The whole process can be expressed

Mathematically

• Based on Behavior Theory, has a more casual

approach towards study of Human Behavior.

• It also considers how people Interact in Groups.

 Game Theory Definition

•Theory of rational behavior for interactive decision problems.

• In a game, several agents strive to maximize their (expected) utility

index by choosing particular courses of action, and each agent's final
utility payoffs depend on the profile of courses of action chosen by
all agents.

•The interactive situation, specified by the set of participants, the

possible courses of action of each agent, and the set of all possible
utility payoffs, is called a game;

• the agents 'playing' a game are called the players.

 Definitions

Definition: Zero-Sum Game – A game in

which the payoffs for the players always adds
up to zero is called a zero-sum game.

Definition: Maximin strategy – If we

determine the least possible payoff for each
strategy, and choose the strategy for which this
minimum payoff is largest, we have the
maximin strategy.
 A Further Definition

Definition: Constant-sum and nonconstant-sum game –

If the payoffs to all players add up to the same constant,
regardless which strategies they choose, then we have a
constant-sum game. The constant may be zero or any
other number, so zero-sum games are a class of
constant-sum games. If the payoff does not add up to a
constant, but varies depending on which strategies are
chosen, then we have a non-constant sum game.
 Game theory: assumptions

(1) Each decision maker has available to him two

or more well-specified choices or sequences of
choices.

(2) Every possible combination of plays available

to the players leads to a well-defined end-state
(win, loss, or draw) that terminates the game.

(3) A specified payoff for each player is associated

with each end-state.
Game theory: assumptions (Cont)

(4) Each decision maker has perfect

knowledge of the game and of his opposition.

(5) All decision makers are rational; that is,

each player, given two alternatives, will select
the one that yields him the greater payoff.
Rules, Strategies, Payoffs, and Equilibrium

A game is a contest involving two or more decision

makers, each of whom wants to win
Game theory is the study of how optimal strategies are formulated in
conflict
A player's payoff is the amount that the player wins or
loses in a particular situation in a game.
A players has a dominant strategy if that player's best
strategy does not depend on what other players do.
A two-person game involves two parties (X and Y)
A zero-sum game means that the sum of losses for one player must
equal the sum of gains for the other. Thus, the overall sum is zero
Rules, Strategies, Payoffs, and Equilibrium

Economic situations are treated as games.

The rules of the game state who can do what, and
when they can do it.
A player's strategy is a plan for actions in each
possible situation in the game.
Strategies taken by others can dramatically affect
the outcome of our decisions
In the auto industry, the strategies of competitors to
introduce certain models with particular features can
impact the profitability of other carmakers
 Payoff Matrix - Store X

• Two competitors are planning radio and

newspaper advertisements to increase their
business. This is the payoff matrix for store X.
A negative number means store Y has a
positive payoff

S-159
Game Outcomes

S-160
 Minimax Criterion
Look to the “cake cutting problem” to explain
Cutter – maximize the minimum the Chooser will
leave him
Chooser – minimize the maximum the Cutter will
get
Chooser  Choose bigger piece Choose smaller piece

Cutter

Cut cake as evenly as Half the cake minus a Half the cake plus a
possible crumb crumb

Make one piece bigger Small piece Big piece

than the other
 Minimax Criterion

The game favors competitor X, since all values

are positive except one.
This means X would get a positive payoff in 3 of
the 4 strategies and Y has a positive payoff in
only 1 strategy
Since Y must play the game (do something
about the competition), he will play to minimize
total losses using the minimax criterion.
 Minimax Criterion

For a two-person, zero-sum game, each person chooses

the strategy that minimizes the maximum loss or
maximize one’s minimum gains
Player Y (columns)is looking at a maximum loss of 3
under strategy Y1 and loss of 5 under Y2
Y should choose Y1 which results in a maximum loss of 3
(minimum of 3 and 5) – minimum of the maximums (upper
value of the game)
The minimum payoffs for X (rows) are +3 (strategy X1 )
and -5 (strategy X2)
X should choose strategy X1 – the maximum of the
minumums (lower value of the game)
 Minimax Criterion

If the upper and lower values are the same, the number is called the
value of the game and an equilibrium or saddle point condition exists
The value of a game is the average or expected game outcome if the game is
played an infinite number of times
A saddle point indicates that each player has a pure strategy i.e., the strategy
is followed no matter what the opponent does
 Saddle Point

• Von Neumann likened the solution point to the point in

the middle of a saddle shaped mountain pass
– It is, at the same time, the maximum elevation reached by a
traveler going through the pass to get to the other side and
the minimum elevation encountered by a mountain goat
traveling the crest of the range

S-165
 Pure Strategy - Minimax Criterion

Player Y’s Minimum Row

Strategies Number

Y1 Y2
Player X’s X1 10 6 6
strategies
X2 -12 2 -12

Maximum Column 10 6
Number

S-166
 Mixed Strategy Game

When there is no saddle point, players will play each strategy for a
certain percentage of the time
The most common way to solve a mixed strategy is to use the expected
gain or loss approach
A player plays each strategy a particular percentage of the time so that the
expected value of the game does not depend upon what the opponent does
Y1 Y2 Expected Gain
P 1-P
X1 4 2 4P+2(1-P)
Q
X2 1 10 1P+10(1-p)
1-Q
4Q+1(1-Q) 2Q+10(1-q)
 Mixed Strategy Game
: Solving for P & Q

4P+2(1-P) = 1P+10(1-P)
or: P = 8/11 and 1-p = 3/11
Expected payoff:
1P+10(1-P)
=1(8/11)+10(3/11)
EPX= 3.46

4Q+1(1-Q)=2Q+10(1-q)
or: Q=9/11 and 1-Q = 2/11
Expected payoff:
EPY=3.46
S-168
 Mixed Strategy Game : Example

• Using the solution procedure for a mixed

strategy game, solve the following game

S-169
 Mixed Strategy Game
Example
• This game can be solved by setting up the
mixed strategy table and developing the
appropriate equations:

S-170
Mixed Strategy Game: Example

S-171
Two-Person Zero-Sum and Constant-Sum
Games
Two-person zero-sum and constant-sum games are played according to
the following basic assumption:

Each player chooses a strategy that enables him/her to do the best he/she
can, given that his/her opponent knows the strategy he/she is following.

A two-person zero-sum game has a saddle point if and only if

Max (row minimum) = min (column maximum)
all all
rows columns

(1)
Two-Person Zero-Sum and Constant-Sum
Games (Cont)

If a two-person zero-sum or constant-sum game has a saddle point, the row

player should choose any strategy (row) attaining the maximum on the right
side of (1). The column player should choose any strategy (column) attaining the
minimum on the right side of (1).
In general, we may use the following method to find the optimal strategies and
value of two-person zero-sum or constant-sum game:

Step 1 Check for a saddle point. If the game has none, go on to step 2.
Two-Person Zero-Sum and Constant-Sum
Games (Cont)

Step 2 Eliminate any of the row player’s dominated strategies. Looking at

the reduced matrix (dominated rows crossed out), eliminate any of the
column player’s dominated strategies and then those of the row player.
Continue until no more dominated strategies can be found. Then proceed to
step 3.

Step 3 If the game matrix is now 2 x 2, solve the game graphically.

Otherwise, solve by using a linear programming method.
 Zero Sum Games

• Game theory assumes that the decision maker and the

opponent are rational, and that they subscribe to the
maximin criterion as the decision rule for selecting
their strategy
• This is often reasonable if when the other player is an
opponent out to maximize his/her own gains, e.g.
competitor for the same customers.
• Consider:
Player 1 with three strategies S1, S2, and S3 and Player
2 with four strategies OP1, OP2, OP3, and OP4.
 Zero Sum Games (Cont)

• The value 4 achieved by both players is called

the value of the game
• The intersection of S2 and OP2 is called a
saddle point. A game with a saddle point is
also called a game with an equilibrium
solution.
• At the saddle point, neither player can
improve their payoff by switching strategies
 Zero Sum Games- To do problem!

Let’s take the following example: Two TV channels (1 and 2) are competing
for an audience of 100 viewers. The rule of the game is to simultaneously
announce the type of show the channels will broadcast. Given the payoff
matrix below, what type of show should channel 1 air?
Two-person zero-sum game – Dominance
property

dominance method Steps (Rule)

Step-1:
1. If all the elements of Column-i are greater than or equal to the corresponding elements of
any other Column-j, then the Column-i is dominated by the Column-j and it is removed from the
matrix.
eg. If Column-2 ≥ Column-4, then remove Column-2

Step-2:
1. If all the elements of a Row-i are less than or equal to the corresponding elements of any
other Row-j, then the Row-i is dominated by the Row-j and it is removed from the matrix.
eg. If Row-3 ≤ Row-4, then remove Row-3

Step-3:
Again repeat Step-1 & Step-2, if any Row or Column is dominated, otherwise stop the
procedure.
Two-person zero-sum game – Dominance
property- To do problem!

Player A \Player B B1 B2 B3 B4
A1 3 5 4 2
A2 5 6 2 4
A3 2 1 4 0
A4 3 3 5 2

Solution Player B
B3 B4
A2 2 4 B3 B4
A4 5 2
A2 2 4
Player A
A4 5 2
 The Prisoner’s Dilemma

•The prisoner’s dilemma is a universal concept. Theorists now realize that

prisoner’s dilemmas occur in biology, psychology, sociology, economics, and law.

•The prisoner’s dilemma is apt to turn up anywhere a conflict of interests exists

-- and the conflict need not be among sentient beings.
• Study of the prisoner’s dilemma has great power for explaining why animal
and human societies are organized as they are. It is one of the great ideas of
the twentieth century, simple enough for anyone to grasp and of fundamental
importance (...).
• The prisoner’s dilemma has become one of the premier philosophical and
scientific issues of our time. It is tied to our very survival (W. Poundstone,1992,
p. 9).
 Prisoner’s Dilemma

• Two members of a criminal gang are arrested and

imprisoned.
– They are placed under solitary confinement and have no chance of
communicating with each other

• The district attorney would like to charge them with

a recent major crime but has insufficient evidence
– He has sufficient evidence to convict each of them of a lesser charge
– If he obtains a confession from one or both the criminals, he can convict
either or both on the major charge.
 Prisoner’s Dilemma

• The district attorney offers each the chance to turn

state’s evidence.

– If only one prisoner turns state’s evidence and testifies against his partner he
will go free while the other will receive a 3 year sentence.
– Each prisoner knows the other has the same offer
– The catch is that if both turn state’s evidence, they each receive a 2 year
sentence
– If both refuse, each will be imprisoned for 1 year on the lesser charge
 A game is described by

• The number of players

• Their strategies and their turn
• Their payoffs (profits, utilities etc) at the outcomes of the
game
 Payoff matrix
Game Theory Definition
Normal- or strategic form

Player B

Left Right

Top 3, 0 0, -4
Player A

Bottom 2, 4 -1, 3
 Game Playing
How to solve a situation like this?

• The most simple case is where there is a optimal choice of

strategy no matter what the other players do; dominant
strategies.
• Explanation: For Player A it is always better to choose Top, for
Player B it is always better to choose left.
• A dominant strategy is a strategy that is best no matter what
the other player does.
 Nash equilibrium

• If Player A’s choice is optimal given Player B’s choice, and B’s
choice is optimal given A’s choice, a pair of strategies is a Nash
equilibrium.
• When the other players’ choice is revealed neither player like
to change her behavior.
• If a set of strategies are best responses to each other, the
strategy set is a Nash equilibrium.
 Payoff matrix
Normal- or strategic form

Player B

Left Right

Top 1, 1 2, 3*
Player A

Bottom 2, 3* 1, 2
 Solution

• Here you can find a Nash equilibrium; Top is the best response
to Right and Right is the best response to Top. Hence, (Top,
Right) is a Nash equilibrium.
• But there are two problems with this solution concept.
 Problems

• A game can have several Nash equilibriums. In this case also

(Bottom, Left).
• There may not be a Nash equilibrium (in pure strategies).
 Payoff matrix
Normal- or strategic form

Player B

Left Right

Top 1, -1 -1, 1
Player A

-1, 1
Bottom 1, -1
 Nash equilibrium in mixed strategies

• Here it is not possible to find strategies that are best

responses to each other.
• If players are allowed to randomize their strategies we can
find s solution; a Nash equilibrium in mixed strategies.
• An equilibrium in which each player chooses the optimal
frequency with which to play her strategies given the
frequency choices of the other agents.
 The prisoner’s dilemma

Two persons have committed a crime, they are held in

separate rooms. If they both confess they will serve
two years in jail. If only one confess she will be free
and the other will get the double time in jail. If both
deny they will be hold for one year.
 Prisoner’s dilemma
Normal- or strategic form

Prisoner B

Confess Deny

Confess -2, -2 0, -4
Prisoner A

Deny -4, 0 -1, -1*

Solution
Confess is a dominant strategy for both. If both
Deny they would be better off. This is the
dilemma.
Nash Equilibrium – To do Problems!

HENRY McD (1)

L R L R
JANE KFC
U 8,7 4,6 U 9,9* 1,10
(1)
D 6,5 7,8 D 10,1 2,2

COKE
L R B
PEPSI L R
U 6,8* 4,7
A
D 7,6 3,7 U 7,6* 5,5
D 4,5 6,4
GAME PLAYING & MECHANISM DESIGN

Mechanism Design is the design of games or

reverse engineering of games; could be called
Game Engineering

Involves inducing a game among the players

such that in some equilibrium of the game,
a desired social choice function is implemented
 GAME PLAYING & MECHANISM DESIGN

Mother
Social Planner
Mechanism Designer

Kid 1 Kid 2
Rational and Rational and
Intelligent Intelligent
Example 1: Mechanism Design
Fair Division of a Cake
 GAME PLAYING & MECHANISM DESIGN

Tenali Rama
(Birbal)
Mechanism Designer

Baby
Mother 1 Mother 2
Rational and Rational and
Intelligent Player Intelligent Player

Example 2: Mechanism Design

Truth Elicitation through an Indirect Mechanism
 GAME PLAYING & MECHANISM DESIGN

One Seller, Multiple Buyers, Single Indivisible Item

Example: B1: 40, B2: 45, B3: 60, B4: 80

Winner: whoever bids the highest; in this case B4

Payment: Second Highest Bid: in this case, 60.

Vickrey showed that this mechanism is Dominant Strategy

Incentive Compatible (DSIC) ;Truth Revelation is good for
a player irrespective of what other players report

MECHANISM DESIGN: EXAMPLE 3 : VICKREY AUCTION

 GAME PLAYING & MECHANISM DESIGN

English Auction Dutch Auction

1 1
0, 10, 20, 30, 100, 90, 85, 75,
40, 45, 50, 55, 70, 65, 60,
58, 60, stop. Seller stop. n
n
Buyers Auctioneer or seller Buyers

First Price Auction Vickrey Auction

1 40 40
1
2 50 Winner = 4 2 45 Winner = 4
Price = 60 Price = 60
3 55 60
3

4 60 4 80
Buyers Buyers

Four Basic Types of Auctions

 END
UNIT-2