Chapter 3:

Methods of
Inference

Expert Systems: Principles and

Programming, Fourth Edition
 Objectives
• Learn the definitions of trees, lattices, and graphs
• Learn about state and problem spaces
• Learn about AND-OR trees and goals
• Explore different methods and rules of inference
• Learn the characteristics of first-order predicate
logic and logic systems

Expert Systems: Principles and Programming, Fourth Edition 2

 Objectives
• Discuss the resolution rule of inference,
resolution systems, and deduction
• Compare shallow and causal reasoning
• How to apply resolution to first-order predicate
logic
• Learn the meaning of forward and backward
chaining

Expert Systems: Principles and Programming, Fourth Edition 3

 Objectives
• Explore additional methods of inference

• Learn the meaning of Metaknowledge

• Explore the Markov decision process

Expert Systems: Principles and Programming, Fourth Edition 4

 Trees
• A tree is a hierarchical data structure consisting
of:
– Nodes – store information
– Branches – connect the nodes
• The top node is the root, occupying the highest
hierarchy.
• The leaves are at the bottom, occupying the
lowest hierarchy.

Expert Systems: Principles and Programming, Fourth Edition 5

 Trees
• Every node, except the root, has exactly one
parent.
• Every node may give rise to zero or more child
nodes.
• A binary tree restricts the number of children per
node to a maximum of two.
• Degenerate trees have only a single path from
root to its one leaf.

Expert Systems: Principles and Programming, Fourth Edition 6

 Binary Tree

Expert Systems: Principles and Programming, Fourth Edition 7

 Graphs
• Graphs are sometimes called a network or net.
• A graph can have zero or more links between
nodes – there is no distinction between parent
and child (e.g., a map where cities represent
nodes and roads represent links).
• Sometimes links have weights – weighted graph;
or, arrows – directed graph (weights could be
used to represent costs, distances in miles, etc).
• Simple graphs have no loops – links that come
back onto the node itself.

Expert Systems: Principles and Programming, Fourth Edition 8

 Graphs
• A path in a graph is a sequence of nodes [n1, n2, …, nk]
such that there is a link from ni to ni+1.
• A circuit (cycle) is a path through the graph beginning
and ending with the same node (sometimes, defined as a
path crossing the same node at least two times).
• Acyclic graphs have no cycles.
• Connected graphs have links to all the nodes.
• Digraphs are graphs with directed links.
• Lattice is a directed acyclic graph.

Expert Systems: Principles and Programming, Fourth Edition 9

 Simple Graphs

Expert Systems: Principles and Programming, Fourth Edition 10

 Making Decisions
• Trees / lattices are useful for classifying objects in a
hierarchical nature.
• Trees / lattices are useful for making decisions.
• A decision structure = knowledge representation
schema + a method of reasoning about its
knowledge. (classic game of twenty questions)
• We refer to trees / lattices as structures.
• Decision trees are useful for representing and
reasoning about knowledge.

Expert Systems: Principles and Programming, Fourth Edition 11

 Binary Decision Trees
• Every question (e.g., Yes/No question) takes us down one
level in the tree.
• A binary decision tree having N nodes:
– All leaves will be answers.
– All internal nodes are questions.
– There will be a maximum of 2N answers for N questions (That is, N
questions can classify 2N).
• Decision trees can be self learning (If the guess is wrong, a
procedure can be called to query the user for a new , correct
classification question and the answer. A new node,
branches, and leaves are dynamically added. In CLIPS build
keyword could be used to build automatically new rules).
• Decision trees can be translated into production rules.
Expert Systems: Principles and Programming, Fourth Edition 12
 Decision Tree Example

Expert Systems: Principles and Programming, Fourth Edition 13

 Decision structures and Expert
systems
• Decision structures are powerful
classification tools. However, they are
limited because they cannot deal with
variables as an expert system can.

• Expert systems are general purpose tools

rather than simply classifiers.

Expert Systems: Principles and Programming, Fourth Edition 14

 State and Problem Spaces
• A state space can be used to define an object’s
behavior.

• Each state refers to a collection of characteristics

that define the status of the object.

• A state space shows the transitions an object can

make in going from one state to another.

Expert Systems: Principles and Programming, Fourth Edition 15

 Finite State Machine
• A FSM is a diagram describing the finite number of
states of a machine.
• At any time, the machine is in one particular state.
• The machine accepts input and progresses to the
next state.
• FSMs are often used in compilers and validity
checking programs.
• A good design of FSM is to include the possibility
of invalid inputs from every state and provide for
transitions to an error state.
Expert Systems: Principles and Programming, Fourth Edition 16
 State Diagram for a Soft Drink Vending Machine
Accepting Quarters (Q) and Nickels (N)

Expert Systems: Principles and Programming, Fourth Edition 17

 Using FSM to Solve Problems

• Well-formed problems:
– Explicit problem, goal, and operations are
known
– Deterministic – we are sure of the next state
when an operator is applied to a state.
– The problem space is bounded.
– The states are discrete.

Expert Systems: Principles and Programming, Fourth Edition 18

 Using FSM to Solve Problems
• Characterizing ill-structured problems – one having
uncertainties (ch1: travel).
• Goal not explicit: I’am thinking about going somewhere.
• Problem space unbounded: I’am not sure where to go.
• Problem state not discrete: I just like to travel, the destination is
not important.
• Intermediate states difficult to achieve: I don’t have enough money
to go.
• State operators unknown: I don’t know how to get the money.
• Time constraint: I must go soon.

Expert Systems: Principles and Programming, Fourth Edition 19

 AND-OR Trees (Graphs)
and Goals
• 1990s, PROLOG was used for commercial
applications in business and industry.
• PROLOG uses backward chaining to
divide problems into smaller problems and
then solves them.
• AND-OR trees also use backward
chaining.

Expert Systems: Principles and Programming, Fourth Edition 20

 Example 1: AND-OR Trees
(Graphs) and Goals (cont.)
a
b
c
abd
ace
bdf
fg
aeh

Is h is true?

Expert Systems: Principles and Programming, Fourth Edition 21

 AND-OR Trees (Graphs)
and Goals
• AND-OR-NOT lattices use logic gates to
describe problems.

and
Not

Or
Expert Systems: Principles and Programming, Fourth Edition 22
 Types of Logic
• Deduction – reasoning where conclusions must
follow from premises
• Induction – inference from the specific case to
the general
• Intuition – no proven theory
• Heuristics – rules of thumb based on experience
• Generate and test – trial and error

Expert Systems: Principles and Programming, Fourth Edition 23

 Types of Logic
• Abduction – reasoning back from a true condition to
the premises that may have caused the condition
• Default – In the absence of specific knowledge,
assume general or common knowledge by default
• Autoepistemic – self-knowledge
• Nonmonotonic – previous knowledge may be
incorrect when new evidence is obtained
• Analogy – inferring conclusions based on
similarities with other situations
Expert Systems: Principles and Programming, Fourth Edition 24
 Deductive Logic
• Deductive logic can determine the validity of an
argument.
• Argument – group of statements where the last is
justified on the basis of the previous ones

• Syllogism (a type of logical argument)– has two

premises and one conclusion
• Deductive argument – true conclusions must
follow from true premises

Expert Systems: Principles and Programming, Fourth Edition 25

 Syllogisms vs. Rules
• Syllogism:
– All basketball players are tall.
– Jason is a basketball player.
 Jason is tall.

• IF-THEN rule:
IF All basketball players are tall and
Jason is a basketball player
THEN Jason is tall.

Expert Systems: Principles and Programming, Fourth Edition 26

 Categorical Syllogism
• The classical syllogism is a special type
called a categorical syllogism, in which
• Premises and conclusions are defined
using categorical statements of the form:

Expert Systems: Principles and Programming, Fourth Edition 27

 Categorical Syllogisms

The subject is the object being described while the

predicate describes some property of the subject

Expert Systems: Principles and Programming, Fourth Edition 28

 Categorical Syllogisms

Expert Systems: Principles and Programming, Fourth Edition 29

 Proving the Validity of Syllogistic
Arguments Using Venn Diagrams
1. If a class is empty, it is shaded.
2. Universal statements, A and E are always drawn
before particular ones.
3. If a class has at least one member, mark it with an
*.
4. If a statement does not specify in which of two
adjacent classes an object exists, place an * on the
line between the classes.
5. If an area has been shaded, no * can be put in it.
Expert Systems: Principles and Programming, Fourth Edition 30
 Rules of Inference
• Venn diagrams are insufficient for complex
arguments.

• Syllogisms address only a small portion of the

possible logical statements.

• Propositional logic offers another means of

describing arguments.

Expert Systems: Principles and Programming, Fourth Edition 31

 Direct Reasoning
Modus Ponens

Expert Systems: Principles and Programming, Fourth Edition 32

 Truth Table Modus Ponens

Expert Systems: Principles and Programming, Fourth Edition 33

 Some Rules of Inference

Expert Systems: Principles and Programming, Fourth Edition 34

 Rules of Inference

Expert Systems: Principles and Programming, Fourth Edition 35

 The Modus Meanings

Expert Systems: Principles and Programming, Fourth Edition 36

 The Conditional
and Its Variants

Expert Systems: Principles and Programming, Fourth Edition 37

 Limitations of Propositional
Logic
• If an argument is invalid, it should be interpreted as
such – that the conclusion is necessarily incorrect.
P : All basketball players are tall
Q: Ahmad is a basketball player
R: Ahmad is tall
An argument may be invalid because it is poorly
connected.
• The invalidity means that an argument may not be
provable using propositional logic, but may be
provable using predicate logic.

Expert Systems: Principles and Programming, Fourth Edition 38

 First-Order Predicate Logic

• Syllogistic logic can be completely described by

predicate logic.
Type Schema Predicate Representation
A ALL S is P (X)(S(X)P(X))
E No S is P (X)(S(X)P(X))
I Some S is P (X)(S(X)P(X))
I Some S is not P (X)(S(X)P(X))

• First-Order Logic has inference rules: Universal

Instantiation, Modus Ponens, Modus tollens, AND
Introduction and Elimination.
Expert Systems: Principles and Programming, Fourth Edition 39
 Logic Systems
• A logic system is a collection of objects such as
rules, axioms, statements, and so forth in a
consistent manner.
• Each logic system relies on formal definitions of
its axioms (postulates) which make up the formal
definition of the system.
• Axioms cannot be proven from within the system.
• From axioms, it can be determined what can be
proven.

Expert Systems: Principles and Programming, Fourth Edition 40

 Goals of a Logic System
• Be able to specify wffs – well formulated
formulas – that forms the arguments.
ALL S is P
ALL S
• Indicate the rules of inference that are valid.

• Extend itself by discovering new rules of

inference that are valid, and so extending the
range of arguments that can be proven –
theorems.

Expert Systems: Principles and Programming, Fourth Edition 41

 Requirements of a Formal System

1. An alphabet of symbols
2. A set of finite strings of these symbols, the
wffs.
3. Axioms, the definitions of the system.
4. Rules of inference, which enable a wff to be
deduced as the conclusion of a finite set of
other wffs – axioms or other theorems of the
logic system.

Expert Systems: Principles and Programming, Fourth Edition 42

 Requirements of a FS Continued

5. Completeness – every wff can either be proved

or refuted.
6. The system must be sound – every theorem is a
logically valid wff.
If the argument:
A1,…,An; A
Is valid, then A is said to be a theorem of the logical
system.

Expert Systems: Principles and Programming, Fourth Edition 43

 Semantics of a FS
• Interpretation of a wff: assignment of truth values.
• Model: an interpretation in which the wff is true.
• Consistent or satisfiable: if there is an
interpretation that makes it true.
• Valid: always true.

Expert Systems: Principles and Programming, Fourth Edition 44

 Resolution Principle(1)
• Resolution refutation proves a theorem by
negating the statement to be proved and adding
this negated goal to the set of axioms that are
known to be true.

• Use the resolution rule of inference to show that

this leads to a contradiction.

• Once the theorem prover shows that the negated

goal is inconsistent with the given set of axioms,
it follows that the original goal must be consistent.

Expert Systems: Principles and Programming, Fourth Edition 45

 Resolution Principle(2)
• Steps for resolution refutation proofs
– Put the premises or axioms into clause form.
– Add the negation of what is to be proved, in clause
form, to the set of axioms.
– Resolve these clauses together, producing new clauses
that logically follow from them.
– Produce a contradiction by generating the empty
clause.
– The substitutions used to produce the empty clause are
those under which the opposite of the negated goal is
true.
Expert Systems: Principles and Programming, Fourth Edition 46
 Converting First-Order Predicate
wffs to Clausal Form

1. Eliminate the logical connectives  and 

– a  b = (a  b)  (b  a)
– a  b = a  b

2. When possible, eliminate negations or reduce their

scope:
(a) = a
(a  b) = a  b
(a  b) = a  b
(X) a(X) = (X) a(X)
(X) b(X) = (X) b(X)

Expert Systems: Principles and Programming, Fourth Edition 47

 Converting
3. Standardize by renaming all variables so that variables
bound by different quantifiers have unique names.
- (X) a(X)  (X) b(X) = (X) a(X)  (Y) b(Y)
4. Convert wff to prenex form by moving all quantifiers to
the left without changing their order.
5. Eliminate existential quantifiers using Skolem functions.
6. Drop the universal quantifiers as necessary.
7. Convert the matrix to conjunctive normal form.
– (a b) (c d)
= (a (c d)) (b  (c d))
= (a c) (a d) (bc) (b d)

Expert Systems: Principles and Programming, Fourth Edition 48

 Converting

8. Eliminate  signs by writing the wff as a set of clauses.

9. Rename variables in clauses making unique.

Expert Systems: Principles and Programming, Fourth Edition 49

 Resolution Proof Procedure-
Example(1)
1. poor(X) smart(X) happy(X)
2. read(Y)  smart(Y)
3. read(john)
poor(john)
4. happy(Z)  exciting(Z)

Goal: exciting(W)

Expert Systems: Principles and Programming, Fourth Edition 50

 Resolution Proof Procedure-
Example(2)
exciting(W) happy(Z) exciting(Z)

{Z/W}
happy(Z) poor(X) smart(X)  happy(X)

{X/Z}
poor(X) smart(X) read(Y) smart(Y)

{Y/X}
poor(john) poor(Y)  read(Y)

{john/Y}
read(john) read(john)

{}

Expert Systems: Principles and Programming, Fourth Edition 51

 Shallow and Causal Reasoning
• Experiential knowledge is based on experience.
• In shallow reasoning, there is little/no causal
chain of cause and effect from one rule to
another.
• Advantage of shallow reasoning is ease of
programming.
• Frames are used for causal / deep reasoning.
• Causal reasoning can be used to construct a
model that behaves like the real system.
Expert Systems: Principles and Programming, Fourth Edition 52
 Chaining
• Chain – a group of multiple inferences that
connect a problem with its solution
• A chain that is searched / traversed from a
problem to its solution is called a forward chain.
• A chain traversed from a hypothesis back to the
facts that support the hypothesis is a backward
chain.
• Problem with backward chaining is find a chain
linking the evidence to the hypothesis.

Expert Systems: Principles and Programming, Fourth Edition 53

 Causal Forward Chaining

Expert Systems: Principles and Programming, Fourth Edition 54

 Some Characteristics of
Forward and Backward Chaining

Expert Systems: Principles and Programming, Fourth Edition 55

 Other Inference Methods
• Analogy – relating old situations (as a guide) to
new ones.
• Generate-and-Test – generation of a likely
solution then test to see if proposed meets all
requirements.
• Abduction – Fallacy of the Converse
• Nonmonotonic Reasoning – theorems may not
increase as the number of axioms increase.

Expert Systems: Principles and Programming, Fourth Edition 56

 Types of Inference

Expert Systems: Principles and Programming, Fourth Edition 57

 Metaknowledge
• The Markov decision process (MDP) is a good
application to path planning.
• In the real world, there is always uncertainty, and
pure logic is not a good guide when there is
uncertainty.
• A MDP is more realistic in the cases where there
is partial or hidden information about the state
and parameters, and the need for planning.

Expert Systems: Principles and Programming, Fourth Edition 58

 Summary
• We have discussed the commonly used methods
for inference for expert systems.
• Expert systems use inference to solve problems.
• We discussed applications of trees, graphs, and
lattices for representing knowledge.
• Deductive logic, propositional, and first-order
predicate logic were discussed.
• Truth tables were discussed as a means of
proving theorems and statements.
Expert Systems: Principles and Programming, Fourth Edition 59
 Summary
• Characteristics of logic systems were discussed.
• Resolution as a means of proving theorems in
propositional and first-order predicate logic.
• The nine steps to convert a wff to clausal form
were covered.

Expert Systems: Principles and Programming, Fourth Edition 60