Artificial Intelligence

Module-IV:
Reasoning under Uncertainty

Dr. Dwiti Krishna Bebarta

Text Book Referred:

1. Artificial Intelligence, Elaine Rich and Kevin Knight, Tata McGraw -Hill Publications
2. Introduction To Artificial Intelligence & Expert Systems, Patterson, PHI publications
 Reasoning under Uncertainty
Topics to discuss:
• Introduction to Non-Monotonic Reasoning
• Logics for Non monotonic reasoning
• Depth first search with Dependency-directed backtracking
• Justification based Truth Maintenance System
• Statistical Reasoning:
– Bayes Theorem for probabilistic inference
– Certainty Factors and Rule-Based Systems
– Bayesian Belief Networks
– Dempster Shafer Theory
– Fuzzy Logic
Reasoning:
• The reasoning is the mental process of deriving logical
conclusion and making predictions from available
knowledge, facts, and beliefs. Or we can say, "Reasoning
is a way to infer facts from existing data." It is a general
process of thinking rationally, to find valid conclusions.
• In artificial intelligence, the reasoning is essential so that the
machine can also think rationally as a human brain, and can
perform like a human.
Introduction to Non-Monotonic Reasoning
Reasoning
• The reasoning is the act of deriving a conclusion from certain
properties using a given methodology.
• The reasoning is a process of thinking; reasoning is logically
arguing; reasoning is drawing the inference.
• When a system is required to do something, that it has not been
explicitly told how to do, it must reason. It must figure out what it
needs to know from what it already knows. Many types of
Reasoning have been identified and recognized, but many
questions regarding their logical and computational properties still
remain controversial.
• The popular methods of Reasoning include abduction, induction,
model-based, explanation and confirmation. All of them are
intimately related to problems of belief revision and theory
development, knowledge absorption, discovery, and learning.
Logical Reasoning
• Logic is a language for reasoning. It is a collection of rules called Logic
arguments, we use when doing logical reasoning.
• The logic reasoning is the process of drawing conclusions from premises using
rules of inference.
• The study of logic divided into formal and informal logic. The formal logic is
sometimes called symbolic logic.
• Symbolic logic is the study of symbolic abstractions (construct) that capture the
formal features of logical inference by a formal system.
• The formal system consists of two components, a formal language plus a set of
inference rules.
• The formal system has axioms. Axiom is a sentence that is always true within the
system.
• Sentences derived using the system‟s axioms and rules of derivation called
theorems.
• The Logical Reasoning is of our concern in AI.
Approaches to Reasoning
There are three different approaches to reasoning under
uncertainties.
1. Symbolic reasoning
2. Statistical reasoning
3. Fuzzy logic reasoning
Symbolic Reasoning
• The basis for intelligent mathematical software is the
integration of the “power of symbolic mathematical tools” with
the suitable “proof technology”.
• Mathematical reasoning enjoys a property called
monotonicity, that says, “If a conclusion follows from given
premises A, B, C… then it also follows from any larger set of
premises, as long as the original premises A, B, C.. included.”
• Moreover, Human reasoning is not monotonic.
• People arrive at conclusions only tentatively; based on partial
or incomplete information, reserve the right to retract those
conclusions while they learn new facts. Such reasoning non-
monotonic, precisely because the set of accepted
conclusions have become smaller when the set of premises
expanded.
Formal Logic
• Moreover, The Formal logic is the study of inference with
purely formal content, i.e. where content made explicit.
• Examples – Propositional logic and Predicate logic.
• Here the logical arguments are a set of rules for manipulating
symbols. The rules are of two types,
1. Syntax rules: say how to build meaningful expressions.
2. Inference rules: say how to obtain true formulas from
other true formulas.
• Moreover, Logic also needs semantics, which says how to
assign meaning to expressions.
Uncertainty in Reasoning
• The world is an uncertain place; often the Knowledge is imperfect which
causes uncertainty.
• So, Therefore reasoning must be able to operate under uncertainty.
• Also, AI systems must have the ability to reason under conditions of
uncertainty.
Monotonic Reasoning
• A reasoning process that moves in one direction only.
• Moreover, The number of facts in the knowledge base is always
increasing.
• The conclusions derived are valid deductions and they remain so.
• A monotonic logic cannot handle
1. Reasoning by default: because consequences may derive only
because of lack of evidence to the contrary.
2. Abductive reasoning: because consequences only deduced as most
likely explanations. (probable conclusion)
3. Belief revision: because new knowledge may contradict old beliefs.
• Conventional reasoning systems, such as first-order
predicate logic (FOPL), are designed to work with information
has three properties:
– All the facts that are necessary to solve a problem are
present i.e. complete w.r.t. the domain of interest.
– Consistent
– Monotonicity i.e. new facts are consistent with all other
facts that have already been proclaimed.
• If any of these properties is not satisfied then conventional
reasoning systems become inadequate.
• Non-monotonic reasoning systems are designed to solve
problems.
Introduction to Non-Monotonic Reasoning
• Often available knowledge is incomplete.
• However, to model common-sense reasoning, it is necessary
to be able to jump to plausible conclusions from the given
knowledge.
• To draw plausible conclusions it is necessary to make
assumptions.
• Non-monotonic reasoning deals with the problem of deriving
plausible/probable conclusions, but not infallible/reliable, from
a knowledge base (a set of formulas).
• Since the conclusions are not certain, it must be possible to
retract some of them if new information shows that they are
wrong.
Example: let the KB contains:
a. Typically birds fly. ∀ x (bird ( x ) → fly ( x))
b. Penguins do not fly. ∀ x (penguin ( x ) → ~fly ( x))
c. Tweety is a bird. bird(Tweety)
• It is plausible to conclude that
– Tweety flies. fly(tweety)
• However if the following information is added to KB
– Tweety is a penguin. penguin (Tweety )
• the previous conclusion must be retracted and, instead, the
new conclusion that
• Tweety does not fly will hold. ~fly(tweety)
• KB that is inconsistent, then other interpretations are
possible, i.e. different interpretations give rise to different
non-monotonic logics.
Inadequacy of Classical Logic.
• We cannot represent a rule such as "typically birds fly“ as
– ∀ x (bird ( x ) ∧ ¬exception ( x ) → fly ( x))
and then to add
∀ x (exception ( x ) ↔ penguin ( x ) ∨ ostrich ( x ) ∨ canary ( x ) ∨. .)

Logicians distinguish between strong and weak specificity:

according to strong specificity
A↛C, overrides A⇒B→C;
according weak specificity
A↛C, overrides A→B→C.

So, we can read the diagram as follows: Penguins are birds (no
exceptions); Birds usually fly; and Penguins usually don‟t fly (exception).
Some advanced issues for sceptical/doughtful reasoning
• Logics for Non monotonic reasoning
• Default reasoning: it is a form of non-monotonic reasoning
where plausible conclusions are inferred based on general
rules which may have exceptions (defaults). It is non-
monotonic in the sense that additional information may force
us to withdraw earlier conclusions, namely whenever the
additional information shows that the case at hand is
exceptional.
• Two approaches
– Non-monotonic logic
– Default logic
• Non-monotonic logic
– Abduction
– Inheritance
• Abductive reasoning is a type of reasoning that emphasizes
drawing inferences from the existing data. There is no
assurance that the conclusion drawn is accurate, though, as
the information at hand could not be comprehensive.
Conclusions drawn from abductive reasoning are likely to be
true. This type of reasoning determines the most likely
conclusion for a set of incomplete facts by taking it into
account.

Abductive Reasoning in AI - GeeksforGeeks

Example of Abductive Reasoning
• Let‟s take an example: Suppose you wake up one morning
and find that the street outside your house is wet.
• Here are the observations and the process of abductive
reasoning:
• Observation: The street is wet.
• Possible Hypotheses:
– It rained last night.
– A water pipe burst.
– A street cleaning vehicle just passed by.
• Additional Information: You recall that the weather forecast
predicted rain for last night.
• Abductive Reasoning Conclusion: The most plausible
explanation for the wet street, given the forecast and the lack
of any other visible cause, is that it rained last night.
Default logic
• Default logic extends classical logic by non-standard
• inference rules. These rules allows one to express default
properties.
Example:

can be interpreted as:

"„if x is a bird and we can consistently assume that x flies
then we can infer that x flies"‟
Frames in AI: Knowledge Representation and Inheritance
• Frames are data structures used in AI to represent
stereotypical situations or scenarios.
• They encapsulate information about objects, events, and their
interrelationships within a particular context.
• Each frame consists of a set of attributes and values, forming
a template for understanding specific situations.
• For instance, a “restaurant” frame might include attributes
such as “menu,” “waitstaff,” and “tables,” each with its own
set of details.
Here are the main components of frames, along with examples to
illustrate their use:
Slots
Slots are attributes or properties of a frame. They represent the
different aspects or characteristics of the frame‟s concept.
Example: For a “Person” frame, slots might include:
• Name: The individual‟s name
• Age: The individual‟s age
• Occupation: The individual‟s profession
• Address: The individual‟s home address
Facets
Facets provide additional details or constraints for slots, defining
acceptable values or specifying how slots should be used.
Example: For the “Age” slot in the “Person” frame:
• Type: Integer
• Range: 0 to 120
• Default Value: 30
Procedures
Procedures are methods or functions associated with frames
that define how the information within the frame should be
processed or utilized.
Example: In an “Account” frame:
• Procedure: CalculateInterest – A method to compute interest
based on the account balance.
Example of Frame Inheritance
• Let‟s consider an example with a hierarchy of frames in a library system:
• Parent Frame: “LibraryItem”
– Attributes:
• Title
• Author
• Publication Year
• Publisher
• Child Frame 1: “Book” (inherits from “LibraryItem”)
– Inherited Attributes: Title, Author, Publication Year
– Extended Attributes:
• ISBN
• Child Frame 2: “Magazine” (inherits from “LibraryItem”)
– Inherited Attributes: Title, Author, Publication Year
– Extended Attributes:
• Issue Number
Applications of Frames in AI
• Natural Language Processing (NLP)
• Expert Systems
• Robotics
• Cognitive Modelling

Default Value representation

Depth first search with Dependency-directed backtracking
• Chronological backtracking is one way to work on a faulty
plan.
• it begins as soon as a dead end is detected.
• The procedure is to withdraw the most recently made choice,
and its consequences,
• to select an alternative at that choice point, and to move
ahead again.
• If all the alternatives at the last choice point have been
explored already, then go further back until an unexplored
alternative is found.
• Non-chronological Backtracking Another way to work on the faulty
plan.
• The procedure for identifying relevant choices is called
dependency-directed backtracking, or non-chronological
backtracking.
• Dependency-directed backtracking (DDB) is an alternative to
chronological backtracking in artificial intelligence (AI) that's used
to identify and fix faulty plans.
• The procedure is:
– Whenever you reach an impasse
– Trace back through dependencies, identifying all choice points
that may have contributed to the impasse.
– Using depth-first search, find a combination of choices at those
choice points that break the impasse.
– Thus, non-chronological backtracking is an efficient way to find
compatible choices, as long as there is a way of tracing back
over dependencies to find the relevant choice points.
• The problem is how to make a weekly schedule, having at
least 6 hours of study, 2 units of entertainment, and 20 units
of exercise. Expenses must be limited to $30 per week.
• We will solve the problem using dependency-directed
backtracking.
• The first schedule could be any schedule, we have chosen
this one:
Monday Tuesday Wednesday Thursday Friday
ex: walking study 0 study 0 study 0 ex: walking
en: restaurant en: restaurant

This plan is faulty, because, for example, expenses are $40, but they mustn't
be bigger than $30. We must fix this plan, changing the choice which is
connected with the problem. In this plan, that choice could be:
We will change one entertainment choice to reading a book, or to doing
nothing. After this change, the expenses are smaller, and it should check
weather the new plan is the solution.
To use dependency-directed backtracking
• Associate with each node one more justifications. Each
justification corresponds to a derivation process that led to
the node and must contain a list of all the nodes on which its
derivation depend.
• Provide a mechanism that, when given a contradiction node
and its justification, computes the set of assumptions that
trigger the justification. This set of assumptions named as
“no-good” i.e. defined to be minimal set of assumptions such
that if you remove any element from the set, the justification
will no longer valid and the node will be no longer be
believed.
• Provide a mechanism for considering a no-good set and
choosing an assumption to withdraw.
Justification based Truth Maintenance System (TMS)
• TMS is a form of non-monotonic reasoning by permitting the
addition of changing statements to a knowledge base.
• Also called belief-revision or revision management system

Tell
Inference
TMS
Engine (IE)
Ask

TMS
Role of TMS:
• To maintain consistency of knowledge
• Recent information can displace previous conclusions that
are no longer valid.
• TMS should maintain dependency records for all such
conclusions
• Procedure used to perform this process is dependency-
directed backtracking
• Records maintain in the from of dependency-network
Interface functions:
Justifying literals
Justifying constraints
Justifying literals Derived literals Justifying Constraints
{P,W} R (P ∧ W) → R
{P} Q P→Q
{Q,R} S (Q ∧ R) → S
 Derived literals

Justifying Literals
Justifying Constraints

Justifying Constraints

Justifying Literals
Justification Tree: justification functions can produce a tree with
literal S at the root. At each node of the tree, the function
justifying_literals can be used to get children nodes until one
reaches members of the premise set.
• Statistical Reasoning:
– Bayes Theorem for probabilistic inference
– Certainty Factors and Rule-Based Systems
– Bayesian Belief Networks
– Dempster Shafer Theory
– Fuzzy Logic
• Statistical Reasoning:
– Bayes Theorem for probabilistic inference
Uncertainty:
• Till now, we have learned knowledge representation using
first-order logic and propositional logic with certainty, which
means we were sure about the predicates. With this
knowledge representation, we might write A→B, which
means if A is true then B is true
• But consider a situation where we are not sure about
whether A is true or not then we cannot express this
statement, this situation is called uncertainty.
• So to represent uncertain knowledge, where we are not sure
about the predicates, we need uncertain reasoning or
probabilistic reasoning.
Causes of uncertainty:
Following are some leading causes of uncertainty to occur in
the real world.
• Information occurred from unreliable sources.
• Experimental Errors
• Equipment fault
• Temperature variation
• Climate change
• Medical Diagnosis
Bayes' theorem:
• Bayes' theorem is also known as Bayes' rule, Bayes' law,
or Bayesian reasoning, which determines the probability of
an event with uncertain knowledge.
• In probability theory, it relates the conditional probability and
marginal probabilities of two random events.
• Bayes' theorem was named after the British
mathematician Thomas Bayes. The Bayesian inference is
an application of Bayes' theorem, which is fundamental to
Bayesian statistics.
• It is a way to calculate the value of P(B|A) with the knowledge
of P(A|B).
• Bayes' theorem allows updating the probability prediction of
an event by observing new information of the real world.
• Bayes theorem (also known as the Bayes Rule or Bayes
Law) is used to determine the conditional probability of
event A when event B has already occurred.
• The general statement of Bayes‟ theorem is “The
conditional probability of an event A, given the occurrence
of another event B, is equal to the product of the event of
B, given A and the probability of A divided by the probability
of event B.” i.e.
Example: If cancer corresponds to one's age then by using Bayes'
theorem, we can determine the probability of cancer more accurately
with the help of age.

Bayes' theorem can be derived using product rule and conditional

probability of event A with known event B:

• P(A ⋀ B)= P(A|B) P(B)

or
Similarly, the probability of event B with known event A:

• P(A ⋀ B)= P(B|A) P(A)

Equating right hand side of both the equations, we will get:

P(A|B) = [P(B|A) * P(A)] / P(B)

P(A|B) is known as posterior, which we need to calculate, and it will be

read as Probability of hypothesis A when we have occurred an evidence B.
Applying Bayes' rule
• Bayes' rule allows us to compute the single term P(B|A) in
terms of P(A|B), P(B), and P(A). This is very useful in cases
where we have a good probability of these three terms and
want to determine the fourth one. Suppose we want to
perceive the effect of some unknown cause, and want to
compute that cause, then the Bayes' rule becomes:

Theorem of Total Probability

Let E1, E2, . . ., En is mutually exclusive and exhaustive events associated
with a random experiment and lets E be an event that occurs with some Ei.
Then, prove that 𝑛

P(E/Ei) . P(Ei)
𝑖=1
Example: A person has undertaken a job. The probabilities of
completion of the job on time with and without rain are 0.44 and
0.95 respectively. If the probability that it will rain is 0.45, then
determine the probability that the job will be completed on time.
• P(rain) = P(A) = 0.45,
• P(no rain) = P(B) = 1 − P(A) = 1 − 0.45 = 0.55
• Let E1 be the event that the mining job will be completed on
time and E2 be the event that it rains.
• P(E1) = 0.44, and P(E2) = 0.95
• Since, events A and B form partitions of the sample space S,
by total probability theorem, we have
• P(E) = P(A) P(E1) + P(B) P(E2) = 0.7205
Example: There are three pots containing 3 white and 2 black
balls; 2 white and 3 black balls; 1 black and 4 white balls
respectively. There is an equal probability of each pot being
chosen. One ball is equal probability chosen at random. what is
the probability that a white ball is drawn?
• Let E1, E2, and E3 be the events of choosing the first,
second, and third pot respectively. Then,
• P(E1) = P(E2) = P(E3) =1/3
• Let E be the event that a white ball is drawn. Then,
• P(E/E1) = 3/5, P(E/E2) = 2/5, P(E/E3) = 4/5
• P(E) = P(E/E1) . P(E1) + P(E/E2) . P(E2) + P(E/E3) . P(E3)
• Ans: 3/5
Question: what is the probability that a patient has diseases meningitis with a stiff
neck?
Given Data:
• A doctor is aware that disease meningitis causes a patient to have a stiff neck,
and it occurs 80% of the time. He is also aware of some more facts, which are
given as follows:
• The Known probability that a patient has meningitis disease is 1/30,000.
• The Known probability that a patient has a stiff neck is 2%.
• Let „A‟ be the proposition that patient has stiff neck (effect) and
• „B‟ be the proposition that patient has meningitis (Cause).
• so we can calculate the following as:
• P(A|B) = 0.8
• P(B) = 1/30000
• P(A)= 0.02

patient has diseases meningitis with a stiff neck i.e. P(B|A)

P(B|A) = [P(A|B) * P(B)] / P(A) = 0.00133

• Question: From a standard deck of playing cards, a single
card is drawn. The probability that the card is king is 4/52,
then calculate posterior probability P(King|Face), which
means the drawn face card is a king card.

P(king): probability that the card is King= 4/52= 1/13

P(face): probability that a card is a face card= 3/13
P(Face|King): probability of face card when we assume it is
a king = 1
P(King|Face)=?
Certainty Factors and Rule-Based Systems
• Probability-based reasoning adopted Bay‟s theorem for handling
uncertainty
• But to apply Bay‟s theorem, there is a need to estimate a priori and
conditional probabilities which are difficult to be calculated in many
domains.
• For this one practical way of compromising on a pure Bayesian
system is to adopt certainty factors.
• A certainty factor (cf), a number to measure the expert‟s belief with
a range of +1 (definitely true) to -1 (definitely false).
• Example: if the expert says “some evidence is almost certainly
true”, then cf=0.8 would be assigned to this evidence.
• This approach is developed in the MYCIN system.
• MYCIN is an expert system
• MYCIN attempts to recommend appropriate therapies for
patients with bacterial infections
• It interacts with the physician to acquire the clinical data as
required.
• It represents most of its diagnostic knowledge as a set of
rules.
• Associated with each rule is a Certainty Factor i.e. a measure
of the belief or disbelief on a conclusion in the presence of a
set of evidence.
• Uncertain terms and their interpretation in MYCIN

Term Certainty Factor

Definitely not -1.0
Almost certainly not -0.8
Probably not -0.6
May be not -0.4
Unknown -0.2 to +0.2
May be +0.4
Probably +0.6
Almost certainly +0.8
Definitely +1.0
A certainty factor is defined in terms of two components
• MB[h,e] – a measure (between 0 to 1) of belief in hypothesis h
given evidence e. it measures the extend to which the evidence
supports the hypothesis. If the evidence fails to support the
hypothesis then assigned with zero.
• MD[h,e] – a measure (between 0 to 1) of disbelief in hypothesis h
given evidence e. it measures the extend to which the evidence
supports the negation of the hypothesis. If the evidence support
the hypothesis then assigned with zero.
• The values of the CF are determined by domain expert who
creates the knowledge base.

CF[h,e] = MB[h,e] – MD[h,e]

– CF is the certainty factor in the hypothesis H due to evidence E

– MB is the measure of increased belief in H due to E
– MD is the measure of increased disbelief in H due to E
Combining uncertain rules