Wolkite University

College of Computing and Informatics

Department of Computer Science

Chapter Five :Uncertain Knowledge and

Reasoning

Prepared by Adem (MSc.)

August 10, 2022

Prepared by Adem (MSc.) Introduction to Artificial intelligence August 10, 2022 1 / 30
 Outline
1 Introduction
2 Reasoning under Uncertainty

3 Sources of Uncertainty

4 Probability Theory

5 Axioms of Probability

6 Probability distributions

7 Bayes’ Rule

8 Summary

Prepared by Adem (MSc.) Introduction to Artificial intelligence August 10, 2022 2 / 30

Logic and Uncertainty

Uncertainty is Bad for Agents based on Logic

Example: Catching a Flight
Let action At = leave for airport t minutes before flight
Question: Will At get me there on time?
Problems:
Partial observability (road state, other drivers’ plans, etc.)
Noisy sensors (traffic reports)
Uncertainty in action outcomes (flat tire, etc.)
Complexity of modeling and predicting traffic
A purely logical approach leads to conclusions that are too weak
for effective decision making:
A25 will get me there on time if there is no accident on the bridge
and it doesn’t rain and my tires remain intact, etc., etc.
AInf guarantees to get there in time, but who lives forever?
Prepared by Adem (MSc.) Introduction to Artificial intelligence August 10, 2022 3 / 30
Uncertainty

Observed variables or evidence:

agent knows certain things about the state of the world (e.g., sensor
readings).
Unobserved variables:
agent needs to reason about other aspects that are uncertain (e.g.,
where the ghost is).
(Probabilistic) model:
agent knows or believes something about how the known variables
relate to the unknown variables.

Prepared by Adem (MSc.) Introduction to Artificial intelligence August 10, 2022 4 / 30

Reasoning under Uncertainty

A rational agent is one that makes rational decisions (in order to

maximize its performance measure)
A rational decision depends on:
the relative importance of various goals
the likelihood they will be achieved
the degree to which they will be achieved

Prepared by Adem (MSc.) Introduction to Artificial intelligence August 10, 2022 5 / 30

Sources of Uncertainty

Probabilistic assertions summarize effects of

Laziness
facts, observability, etc. Ignorance
Ignorance
lack of explicit theories, relevant facts, observability, etc.
Randomness
Inherently random behavior
Utility theory is used to represent and infer preferences.
Decision theory = probability theory + utility theory

Prepared by Adem (MSc.) Introduction to Artificial intelligence August 10, 2022 6 / 30

Probability Theory

the set of all possible worlds is called the sample space.

The possible worlds are mutually exclusive and exhaustive
two possible worlds cannot both be the case, and
one possible world must be the case.
The Greek letter Ω (uppercase omega) is used to refer to the
sample space, and
ω (lowercase omega) refers to elements of the space, that is,
particular possible worlds.
A fully specified probability model associates a numerical
probability with each possible world.
The basic axioms of probability theory say that every possible
world has a
probability between 0 and 1 and
that the total probability of the set of possible worlds is 1:
Prepared by Adem (MSc.) Introduction to Artificial intelligence August 10, 2022 7 / 30
Cont....

unconditional or prior refer to degrees of belief in propositions in

the absence of any other information.
Probabilities such as and a P(doubles) P(total =11)
conditional or posterior probability is a measure of the probability
of an event given
that (by assumption, presumption, assertion or evidence) another
event has already occurred.
This probability is written where P(doubles |Dies =5 ) where the |
is pronounced "given".
Prepared by Adem (MSc.) Introduction to Artificial intelligence August 10, 2022 8 / 30
Cont...

Frequentism (Empirical)
Probabilities are relative frequencies determined by observation.
For example, if we toss a coin many times, p(heads) is the
proportion of the time the coin will come up heads
But what if we are dealing with events that only happen once?
E.g., what is the probability that a Republican will win the
presidency in 2024?
Reference class problem. E.g., how do we define comparable
elections?
Subjectivism (Bayesian Statistics)
Probabilities are degrees of belief updated by evidence.
How do we assign belief values to statements without evidence?
How do we update our degrees of belief?
What would make sure that agents hold consistent beliefs? E.g.,
The coin will land heads up and tails up at the same time.
Prepared by Adem (MSc.) Introduction to Artificial intelligence August 10, 2022 9 / 30
Axioms of Probability

Probability Theory is governed by the following axioms:

All probabilities are real values between 0 and 1: for all φ, 0 ≤ P(φ
)≤1
Valid propositions have probability 1 P(True) = P( α ∨ ¬ α) = 1
The probability of disjunction is defined as follows:

Prepared by Adem (MSc.) Introduction to Artificial intelligence August 10, 2022 10 / 30

Cont...

A random variable is a function that maps from the domain of

possible worlds Ω (called sample space) to the real numbers
written as X: Ω →
Denoted by capital letters
R: Is it raining?
W: What’s the weather?
Die: What is the outcome of rolling two dice?
V: What is the speed of a car (in MPH)?
Names for values are always lowercase
A Boolean random variable has the range true, false .
the range of weather to be the set sun, cloud, rain,snow
Domain values must be mutually exclusive and exhaustive
R ∈ True, False
W ∈ Sunny, Cloudy, Rainy, Snow
Die ∈(1,1), (1,2), . . . (6,6)
Prepared by Adem (MSc.) Introduction to Artificial intelligence August 10, 2022 11 / 30
Notations

Random variables are written in upper roman letters: X, Y etc.

Realizations of a random variable are written in corresponding
lower case letters. E.g.x1,x2,..xn could be of outcomes of the
random variable X.
The probability value of the realization X is written as P(X = X) .
When clear from context,this will be abbreviated as P(X) .
The probability distribution of the (discrete) random variable X is
denoted as p(X).
This corresponds e.g.to a vector of numbers, one for each of the
probability valuesP(X =Xi) (and not to a single scalar value!).

Prepared by Adem (MSc.) Introduction to Artificial intelligence August 10, 2022 12 / 30

Events and Propositions

Events
Probabilistic statements are defined over events, world states or
sets of states
"It is raining"
"The weather is either cloudy or snowy"
"The sum of the two dice rolls is 11"
Events are described using propositions:
R = True
W = "Cloudy" ∨ W = "Snowy’
D ∈ (5,6),(6,5)

Prepared by Adem (MSc.) Introduction to Artificial intelligence August 10, 2022 13 / 30

Probability distributions
For discrete variables,
the probability distribution can be encoded by a discrete list of the
probabilities of the outcomes, known as the probability mass
function.

Prepared by Adem (MSc.) Introduction to Artificial intelligence August 10, 2022 14 / 30

Joint distributions

A joint probability distribution over a set of random variables

specifies the probability of each outcome

Prepared by Adem (MSc.) Introduction to Artificial intelligence August 10, 2022 15 / 30

Cont..

From a joint distribution,the probability of any event can be

calculated.
Probability that it is hot and sunny?
Probability that it is hot?
Probability that it is hot or sunny?
Interesting events often correspond to partial assignments,
e.g.P(hot).

Prepared by Adem (MSc.) Introduction to Artificial intelligence August 10, 2022 16 / 30

Marginal distributions

The marginal distribution of a subset of a collection of random

variables is the joint probability distribution of the variables
contained in the subset.

Intuitively, marginal distributions are sub-tables which eliminate

variables.

Prepared by Adem (MSc.) Introduction to Artificial intelligence August 10, 2022 17 / 30

Cont...

Prepared by Adem (MSc.) Introduction to Artificial intelligence August 10, 2022 18 / 30

Conditional distributions

P(A) is the unconditional (or prior) probability of fact A

An agent can use the unconditional probability of A to reason
about A in the absence of further information
If further evidence B becomes available, the agent must use the
conditional (or posterior) probability:

Prepared by Adem (MSc.) Introduction to Artificial intelligence August 10, 2022 19 / 30

Cont...

Prepared by Adem (MSc.) Introduction to Artificial intelligence August 10, 2022 20 / 30

Conditional distributions

Prepared by Adem (MSc.) Introduction to Artificial intelligence August 10, 2022 21 / 30

Normalization trick

To get the whole conditional distribution P(X| y) at once, select all

entries in the joint distribution matching Y = y and renormalize
them to sum to one

Prepared by Adem (MSc.) Introduction to Artificial intelligence August 10, 2022 22 / 30

Bayes’ Rule

The product rule gives us two ways to factor a joint distribution:

Prepared by Adem (MSc.) Introduction to Artificial intelligence August 10, 2022 23 / 30

Independence

Two events a and b are independent if and only if

P(a∧ b) = P(a) P(b)
This is equivalent to P(a | b) = P(a) and P(b | a) = P(b)
P(a| b) = P(a)
P(b|a) = P(b)
P(a, b) = P(a)P(b)
Independence is denoted as A ⊥ B .

Prepared by Adem (MSc.) Introduction to Artificial intelligence August 10, 2022 24 / 30

Cont...

Independence is an important simplifying assumption for

modeling,
e.g., Cavity and Weather can be assumed to be independent

Prepared by Adem (MSc.) Introduction to Artificial intelligence August 10, 2022 25 / 30

Cont...

Conditional independence: a and b are conditionally independent

given c (i.e., if we know c) iff
P(a ∧ b| c) = P(a | c) P(b | c)

Conditional independence is denoted as A ⊥ B| C .

Prepared by Adem (MSc.) Introduction to Artificial intelligence August 10, 2022 26 / 30

Bayesian network

A Bayesian network is a directed acyclic graph (DAG) in which:

Each node corresponds to a random variable.
Can be observed or unobserved.
Can be discrete or continuous.
Each edge indicates dependency relationships.
Ifthere is an arrowfrom node to node , is said to be a parent of .
Each node is annotated with a conditional probability distribution

Prepared by Adem (MSc.) Introduction to Artificial intelligence August 10, 2022 27 / 30

Cont...

Prepared by Adem (MSc.) Introduction to Artificial intelligence August 10, 2022 28 / 30

Example

I am at work, neighbor John calls to say my alarm is ringing, but

neighbor Mary does not call. Sometimes it’s set off by minor
earthquakes. Is there a burglar?
Variables: Burglar Earthquake Alarm JohnCalls MaryCalls
Network topology from "causal" knowledge:
A burglar can setthe alarm off
An earthquake can setthe alaram off
The alarm can causeMary to call
The alarm can cause John to call

Prepared by Adem (MSc.) Introduction to Artificial intelligence August 10, 2022 29 / 30

Summary

Uncertainty arises because of laziness and ignorance.

It is inescapable in complex non-deterministic or partially
observable environments.
Probabilistic reasoning provides a framework for managing our
knowledge and beliefs, with the Bayes’ rule acting as the
workhorse for inference.
A Bayesian Network specify full joint distribution.
They are often exponentially smaller than an explicitly
enumerated joint distribution.

Prepared by Adem (MSc.) Introduction to Artificial intelligence August 10, 2022 30 / 30