Artificial Intelligence

Agents
Search

Dr. Junaid Younas

• Modelling Task Environment
• Types of Agents
• Search
Task environments are problems to which agents are solutions

• Agent Type — Player

• Performance Measure
• -1 per step, +10 food, +500 win, -500 die, +200 hit scared ghost
• Environment
• Pacman dynamics (incl ghost behaviour)
• Actuators
• Right, left, up ,down, (stop)
• Sensors
• Entire State is visible
Now model PEAS for an Self-driving car?
 Properties of task environments help in designing appropriate agent
• Fully observable versus partially observable
• if it is possible in principle to determine the complete state of the environment at each me point it is
observable; otherwise it is only par ally observable

• Single agent versus multi-agent

• The environment may contain other agents which may be of the same kind as the agent, or of di erent kinds
• Competitive versus cooperative multi-agents
• Deterministic versus non deterministic
• if the future state of the environment can be predicted in principle given the current state and the set of
ac ons which can be performed it is determinis c; otherwise it is nondeterminis c

• Stochastic versus non deterministic

ti
ti
ti
ti
ti
ff
Properties of task environments help in designing appropriate agent

• Episodic versus sequential

• In an episodic task environment, the agent's experience is divided into atomic episodes. In each
episode the agent receives a percept and then performs a single ac on. The next episode does not
depend on the ac ons taken in previous episodes. In sequen al environments, on the other hand,
the current decision could a ect all future decisions.

• Static versus dynamic

• if the environment only changes as a result of the agent’s ac ons, it is sta c; otherwise it is dynamic
• Discrete versus continuous
• if there are a limited number of dis nct, clearly de ned, states of the environment, the
environment is discrete; otherwise it is con nuous
ti
ff
ti
ti
fi
ti
ti
ti
ti
There are multiple type of Agents

• Selects action based on current percept, ignoring percept history — Re ex

agents

• Only maintain current percept in memory

• Fully observable
• Do not consider the future consequences of their actions
• If car in front is braking then initiate brake — condition-action rule
• Simple but with limited intelligence

fl
 What if environment is partially observable…?

• Agent needs to maintain some internal state

• Requires percept history & some aspects of current state
• Information about the changes in environment — transition model
• Changes in environment re ect in agent’s percept— sensor model
• Combination of these models to maintain the current state— model-based
agent
fl
Maintaining the current state only isn’t enough ….

• Consider a road junction…. What should be the action?

• Requires some destination related information
• Model-based re ex agents + goal based information — Goal-based agents
• Search is used in AI to nd action sequences that helps to achieve agents
goals

• What about rationality of actions to achieve a goal?

• Utility Function — internalisation of performance measure — to maximise
the expected utility — Utility-based agents
fl
fi
AI focuses on building machines that can learn

• Learning allows agent to operate in initially unknown environment and then

build up knowledge about it — 4 components

• Learning elements — responsible for making improvements

• Performance elements — takes in percept & decides action
• Critic elements — feedback for learning elements to modify the
performance elements

• Problem generator — responsible for suggesting actions that will lead to

new and informative experiences
Search Problems: computational process by agent to solve a problem

• A search problem consists of:

• A state space
• For each state, a set Actions(s) of allowable actions {N, E}

• A transition model Result(s,a)

N 1
• A step cost function c(s,a,s’)
• A start state and a goal test E
1

• A solution is a sequence of actions (a plan/path) which transforms the start state to a goal state
 Example: Travelling in Romania
Oradea
• State space: 71
Neamt
• Cities
Zerind 87
151
• Actions: 75
Iasi
• Go to adjacent city Arad
140
92
• Transition model Sibiu
99 Fagaras
118
• Result(A, Go(B)) = B 80
Vaslui

• Action cost Timisoara

Rimnicu Vilcea

• Distance along road link 111 211

142
Lugoj Pitesti
97
• Start state: 70 98
• Arad Mehadia 146 101 85
Urziceni
Hirsova

86
• Goal test: 75
120
138 Bucharest
Drobeta
• Is state == Bucharest? 90
Craiova Eforie
Giurgiu
• Find the optimal solution?
How to reach the destination…. Problem solving process
Oradea
71
Neamt
• Goal formulation Zerind 87
151
75
• Problem formulation Arad
140
Iasi

92
Sibiu
• Abstraction 118
99 Fagaras

Vaslui
80

• Search Timisoara
Rimnicu Vilcea

142
211
•
111
Execution Lugoj 97
Pitesti

70 98
Hirsova
• Evaluation and adjustments Mehadia 146 101 85
Urziceni
75 138 86
Bucharest
Drobeta 120
90
Craiova Eforie
Giurgiu
What’s in a State Space?
The real-world state includes every detail of the environment

A search state abstracts away details not needed to solve the problem

• Problem: Pathing • Problem: Eat-All-Dots

• State representation: (x,y) location • State representation: {(x,y), dot booleans}
• Actions: NSEW • Actions: NSEW
• Transition model: update location • Transition model: update location and
• Goal test: is (x,y)=END possibly a dot boolean
• Goal test: dots all false
State Space Sizes?
• World state:
• Agent positions: 120
• Food count: 30
• Ghost positions: 12
• Agent facing: NSEW

• How many
• World states?
120x(230)x(122)x4
• States for pathing?
120
• States for eat-all-dots?
120x(230)
State Space Graphs and Search Trees
 State Space Graphs

• State space graph: A mathematical

representation of a search problem
• Nodes are (abstracted) world con gurations
• Arcs represent transitions resulting from actions
• The goal test is a set of goal nodes (maybe only one)

• In a state space graph, each state occurs only

once!

• We can rarely build this full graph in memory (it’s

too big), but it’s a useful idea
fi
An Example of search space….
Oradea
71
Neamt

Zerind 87
151
75
Iasi
Arad
140
92
Sibiu Fagaras
99
118
Vaslui
80
Rimnicu Vilcea
Timisoara
142
111 Pitesti 211
Lugoj 97
70 98
85 Hirsova
Mehadia 146 101 Urziceni
75 138 86
Bucharest
Drobeta 120
90
Craiova Eforie
Giurgiu
More Examples
R
L R

L
S S

R R
L R L R

L L
S S
S S
R
L R

S S
State Space Graphs vs. Search Trees
Consider this 4-state graph: How big is its search tree (from S)?
S
a
a b

S G
b G G a

b
G a b G

∞
Important: Lots of repeated structure in the search tree!
Understanding terminologies in Search Trees
• node.STATE
Oradea

• node.PARENT 71
Neamt

Zerind 87
151

• node.ACTION
75
Iasi
Arad
140
92

• node.PATH_COST
Sibiu Fagaras
99
118
Vaslui
80

• Child/successor node
Rimnicu Vilcea
Timisoara
142
111 Pitesti 211
Lugoj 97
• Frontier of search tree 70 98
Hirsova
Mehadia 146 101 85
Urziceni

• Interior region of states

Drobeta
75
120
138 Bucharest
86

90
• Exterior region of states Craiova Giurgiu
Eforie

Frontiers separate interior and exterior regions of states

• What is the relationship between these sets of states after each loop
iteration in GRAPH_SEARCH?
• (Loop invariants!!!)
A B C
Explored Never Seen Explored Never Seen Explored Never Seen

Frontier Frontier Frontier

• What is the relationship between these sets of states after each loop iteration in GRAPH_SEARCH?
• (Loop invariants!!!)
A B C
Explored Never Seen Explored Never Seen Explored Never Seen

Frontier Frontier Frontier

• The frontier states separate the explored states from never seen states
• Frontier is sub-set of Explored, as loop progresses, number of explored states will be
more than the number of states in frontier
• Nodes that are not explored (Never Seen) are distinct from the other two
Understanding terminologies in Search Trees
• Loopy path — repeated state
Oradea
• Redundant path 71
Neamt

87
• Frontiers are organised using
Zerind 151
75
Iasi
queues Arad
140
92
Sibiu Fagaras
99

• Priority queue 118

80
Vaslui

Rimnicu Vilcea
Timisoara

• FIFO queue 111

Lugoj 97
Pitesti 211
142

• LIFO queue 70
Mehadia 146 101 85
Urziceni
98
Hirsova

75 138 86
Bucharest
Drobeta 120
90
Craiova Eforie
Giurgiu

Identify Cyclic search spaces and redundant paths?

Best-first search is a simple form of search algorithm

• Node (n) is selected based on minimum value of evaluation function f(n)

• On each iteration
• A node on frontier is chosen with minimum f(n)
• Checks if it’s goal state, else expand to generate child nodes
• Add each child node to frontier if not already there… What if its already
there?

• Check the COST… if its lower UPDATE

• Return Node with GOAL, or FAILURE
Properties of a Search Algorithm

• Completeness: Guarantees to nd a solution if it exists

fi
Properties of a Search Algorithm

• Completeness: Guarantees to nd a solution if it

exists

• Optimal: Guarantees to nd the path with minimum

costs
fi
fi
Properties of a Search Algorithm

• Completeness: Guarantees to nd a solution if it exists 1

• Optimal: Guarantees to nd the path with minimum costs
• Complexity: b
m
• Time
2
• Space b
• How to nd the complexity
• Branching factor — b
• Maximum depth — m
• Solutions at di erent depths
fi
ff
fi
fi
 Oradea
71
Neamt

Zerind 87
151
75
Iasi
Arad
140
92
Sibiu Fagaras
99
118
Vaslui
80
Rimnicu Vilcea
Timisoara
142
111 Pitesti 211
Lugoj 97
70 98
85 Hirsova
Mehadia 146 101 Urziceni
75 138 86
Bucharest
Drobeta 120
90
Craiova Eforie
Giurgiu
Uninformed search algorithms are also known as blind search algorithms

• Don’t have any knowledge about problem domain

• Perform systematic exploration — may lead to ine ciency
• Breadth- rst search
• Explores all nodes at a given level before moving to
next level

• Evaluation function f(n) is depth of the node

• Always reaches the Solution—Target with optimal
number of actions — uniform cost

• Early goal test versus late goal test

d
• Exponential Complexity — O(b ) — time and memory
fi
ffi
Uniform-cost search is also known as Dijkstra’s algorithm

• Evaluation function is the cost of the path from

root to the current node

• Spreads out in waves of uniform path cost

• Considers all paths systematically in order of
increasing cost

• Complexity in terms of C* and ϵ, as

1+[c*/ϵ]
O(b )
• Best- rst-search with Path-cost
fi
 Depth-first returns first solution regardless of its costs

• Expands the deepest node in the frontier rst

• Search proceed to the deepest level of search tree
• Once Target is found, goal is achieved regardless of
costs

• Doesn’t maintain search graph rather search tree

• E cient for acyclic state spaces, for cyclic state
spaces may stuck in in nite loops
ffi
fi
fi
 Depth-first returns first solution regardless of its costs

• Expands the deepest node in the frontier rst

• Search proceed to the deepest level of search tree
• Once Target is found, goal is achieved regardless
of costs

• Doesn’t maintain search graph rather search tree

• E cient for acyclic state spaces, for cyclic state
spaces may stuck in in nite loops

• Complexity — O(bm)
ffi
fi
fi
Bidirectional search simultaneously searches forward and backward

• Searches forward from Initial state and backward

from goal state at the same time
d/2 d/2
• Reduces complexity — b +b
• Keeps track of two frontiers and search graphs
• Collision provides solutions

Lets work on Assignment #3

 • Task environment
• Types of agent
• Search problems and their formulation
• Best- rst search algorithm
• Uninformed search
• Breadth- rst search algorithm
• Uniform-cost search algorithm
• Depth- rst search algorithm
• Bidirectional search algorithm
• Next Lecture:
• Informed search
fi
fi
fi
Some of the slides are borrowed
from Dr. Seemab’s Lectures