Cs511 Uncertainty
Cs511 Uncertainty
Cs511 Uncertainty
The type of uncertainty that can occur in knowledgebased systems may be caused by problems with the
data. For example:
1. Data might be missing or unavailable
2. Data might be present but unreliable
ambiguous due to measurement errors.
or
Probabilistic reasoning.
Certainty factors
Dempster-Shafer Theory
1. Classical Probability
The oldest and best defined technique for managing
uncertainty is based on classical probability theory. Let us
start to review it by introducing some terms.
1.
0 P(E) 1.
2.
Compound probabilities
Conditional Probabilities
The probability of an event A, given B occurred, is called
a conditional probability and indicated by
P(A | B)
The conditional probability is defined as
P(A B)
P(A | B) = ------------------, for P(B) 0.
P(B)
An example
As an example of probabilities, Table below shows
hypothetical probabilities of a disk crash using a Brand X
drive within one year.
Brand X
0.6
0.2
0.8
Crash C
No crash C
Total of columns
Brand X
0.1
0.1
0.2
Total of Rows
0.7
0.3
1.0
C
P(C X) P(C X)
C
P(C X) P(C X)
Total of columns P(X)
P(X)
Total of rows
P(C)
P(C)
1
11
Bayes Theorem
Note that conditional probability is defined as
P(H E)
P(H | E) = ------------------, for P(E) 0.
P(E)
i.e., the conditional probability of H given E.
12
Furthermore, we have,
So,
Thus
P(E H)
P(E | H) = --------------P(H)
13
14
No oil
P(O) = 0.4
test
P(-|O)
=0.9
+ test
P(+|O)
= 0.1
P(-O)
=0.36
P(+O)
=0.04
oil
P(O)=0.6
test
P(-|O)
= 0.2
P(-O)
=0.12
Probabilities
Prior
Subjective Opinion
of site: P(Hi)
Conditional
+ test
P(+|O)
= 0.8
P(+O)
=0.48
Joint: P(EH)
=P(E|Hi)P(Hi)
15
test
P(-) = 0.48
No oil
P(O|-)
=3/4
oil
P(O|-)
= 1/4
P(-O)
=0.36
P(-O)
=0.12
+ test
P(+)=0.52
No oil
P(O|+)
= 1/13
P(+O)
=0.04
Probabilities
unconditional
P(E)
Posterior
oil
P(O|+)
= 12/13
Of site:
P(Hi|E) = P(E|
Hi)P(Hi)/P(E)
P(+O)
=0.48
Joint: P(EH)
=P(Hi|E)P(E)
16
P(+)=0.52
D
Quit
+ or -
Act.
Quit or drill
E
Drill
Quit
Drill
B
No oil
P(O|-)
=3/4
Event
Test result
C
oil
P(O|-)
= 1/4
No oil
P(O|+)
= 1/13
-$50,000
-$50,000
-$1,000,000 $1,000,000
-$1,000,000
oil
P(O|+)
= 12/13
Event
Oil or no oil
payoff
$1,000,000
18
P(+)=0.52
-$50,000
D
Quit
+ or -
$846,153
E
Drill
Quit
C
oil
P(O|-)
= 1/4
Act.
Quit or drill
Drill
B
No oil
P(O|-)
=3/4
Event
Test result
No oil
P(O|+)
= 1/13
-$50,000
-$50,000
-$1,000,000 $1,000,000
-$1,000,000
oil
P(O|+)
= 12/13
Event
Oil or no oil
payoff
$1,000,000
19
20
21
(1)
or
(2)
22
= 0.75
Then
= 0.2
(0.75)(0.2)
= --------------(0.31)
= 0.48387
23
24
Propagation of Belief
Note that what we have just examined is very limited
since we have only considered when each piece of
evidence affects only one hypothesis.
25
This equation
assumptions:
is
derived
based
on
several
26
and
P(Hi)
P(E1 | Hi)
P(E2 | Hi)
i=1
(cold)
i=2
(allergy)
i=3
(light sensitive)
0.6
0.3
0.6
0.3
0.8
0.9
0.1
0.3
0.0
27
two
(0.3)(0.6)
P(H1 | E1) = ------------------------------------------ = 0.4
(0.3)(0.6) + (0.8)(0.3) + (0.3)(0.1)
(0.8)(0.3)
P(H2 | E1) = ------------------------------------------ = 0.53
(0.3)(0.6) + (0.8)(0.3) + (0.3)(0.1)
(0.3)(0.1)
P(H3 | E1) = ------------------------------------------ = 0.06
(0.3)(0.6) + (0.8)(0.3) + (0.3)(0.1)
P(H1 | E1 E2)
(0.3)(0.6)(0.6)
= -----------------------------------------------------------(0.3)(0.6)(0.6) + (0.8)(0.9)(0.3) + (0.3)(0.0)(0.1)
= 0.33
28
P(H2 | E1 E2)
(0.8)(0.9)(0.3)
= -----------------------------------------------------------(0.3)(0.6)(0.6) + (0.8)(0.9)(0.3) + (0.3)(0.0)(0.1)
= 0.67
P(H3 | E1 E2)
(0.3)(0.0)(0.1)
= -----------------------------------------------------------(0.3)(0.6)(0.6) + (0.8)(0.9)(0.3) + (0.3)(0.0)(0.1)
= 0.0
29
31
2: Certainty factors
32
P(H) = 1 - P(H)
For the case of a posterior hypothesis that relies on
evidence, E
(1) P(H | E) = 1 - P(H | E)
33
34
35
MD(H,E)
= 1
if P(H) = 1
otherwise
= 1
if P(H) =1
otherwise
37
38
0 means no evidence.
So certainty values greater than 0 favor the hypothesis
40
41
42
43
44
CF
for
the
46
CF1 + CF2
= ------------------------1 - min(|CF1|,|CF2|)
47
Rule 2
CF2(H, e) = CF2(E, e)CF2(H,e)
AND
(min)
OR
(max)
NOT
(-)
AND
(min)
OR
(max)
NOT
(-)
48
49
52
3: Dempster-Shafer Theory
Here we discuss another method for handling uncertainty.
It is called Dempster-Shafer theory. It is evolved during
the 1960s and 1970s through the efforts of Arthur
Dempster and one of his students, Glenn Shafer.
53
Frames of discernment
Given a set of possible elements, called environment,
= {1, 2, ..., n}
that are mutually exclusive and exhaustive.
For example,
= {airline, bomber, fighter}
= {red, green, blue, orange, yellow}
54
An environment
discernment.
is
also
55
called
frame of
56
57
58
If X Y, it is not necessary
that m(X) m(Y)
P(X) P(Y)
No required relationship
between m(X) and m(X)
P(X) + P(X) = 1
59
if x =
60
m0(x) =
0
otherwise
Each proper subset of gets assigned the number 0.
The core of m0 is equal to {}
Now suppose that some evidence has become
available that points to the composite hypothesis
heart-attack or pericarditis with some certainty.
Then the subset {heart-attack, pericarditis} will be
assigned a mass, e.g., 0.4. Due to lack of further
information, the remaining certainty 0.6 is assigned to
.
m1(x) =
0.6
0.4
0
if x =
if x = {heart-attack, pericarditis}
otherwise
if x =
61
m2(x) =
0.7
if x = {heart-attack, pulmonary-embolism,
aortic-dissection }
otherwise
62
Combining evidence
Dempster-Shafer theory provides a function for computing
from two pieces of evidence and their associated masses
describing the combined influence of these pieces of
evidence.
m (X) m (Y)
1
X Y= Z
{B} 0.27
0.03
Bomber
Bomber or fighter
nonbelief
64
Meaning
-----------------------------------------------------------------------[1, 1]
[0, 0]
[0, 1]
[Bel, 1] where 0 < Bel < 1 here
[0, PIs] where 0 < PIs < 1 here
[Bel, PIs] where 0 < Bel PIs < 1 here
65
Completely true
Completely false
Completely ignorant
Tends to support
Tends to refute
Tends to both support and refute
For example,
Bel1({B, F}) = m1({B, F}) + m1({B}) + m1({F})
= 0.7 + 0 + 0 = 0.7
Bel1 Bel2()
= m1 m2() + m1m2({B, F}) + m1m2({B})
= 0.03 + 0.07 + 0.9 = 1
66
Thus,
67
68
m3() = 0.05
{B} 0.045
{B, F} 0.0035
0.0015
Thus
m1m2m3({A}) = 0.0285
m1m2m3({B}) = 0.045
m1m2m3({B, F}) = 0.0035
m1m2m3() = 0.0015
m1m2m3() = 0
Note that for this example, the sum of all the masses is
less than 1
69
m1m2m3(X)
= 0.0285 + 0.045 + 0.0035 + 0.0015 = 0.0785
However a sum of 1 is required because the combined
evidence m1m2m3, is a valid mass and the sum
over all focal elements must be 1.
This is a problem.
The solution to this problem is a normalization of the
focal elements by dividing each focal element by
1-
where is defined for any sets X and Y as
=
m1(X)m2(Y)
X Y =
of
Dempsters
m1(X) m2(Y)
X Y = Z
m1m2(Z) = -------------------------1-
71
Rule
of
72