Probability Theory AI

Download as pdf or txt
Download as pdf or txt
You are on page 1of 16

Artificial Intelligence

Probability Theory
Outline
• Axioms of Probability
• Product and chain rules
• Bayes Theorem
• Random variables
• PDFs and CDFs
• Expected value and variance
Introduction
• Sample space - set of all possible
outcomes of a random experiment
– Dice roll: {1, 2, 3, 4, 5, 6}
– Coin toss: {Tails, Heads}
• Event space - subsets of elements in a
sample space
– Dice roll: {1, 2, 3} or {2, 4, 6}
– Coin toss: {Tails}
• 𝑃𝑟𝑜𝑏𝑎𝑏𝑖𝑙𝑖𝑡𝑦 𝑚𝑒𝑎𝑠𝑢𝑟𝑒 𝑃: → ℝ
• Axioms of Probability
– 0 ≤ 𝑃 𝐴 ≤ 1, for all 𝐴 ∈
– 𝑃(Ω) = 1

𝑁𝑢𝑚𝑏𝑒𝑟 𝑜𝑓 𝑜𝑢𝑡𝑐𝑜𝑚𝑒𝑠 ∈ 𝐴
• 𝑃 𝐴 = lim𝑛→∞
𝑇𝑜𝑡𝑎𝑙 𝑛𝑢𝑚𝑏𝑒𝑟 𝑜𝑓 𝑜𝑢𝑡𝑐𝑜𝑚𝑒𝑠
examples
• Coin flip
– P(H)
– P(T)
– P(H,H,H)
– P(x1=x2=x3=x4)
– P({x1,x2,x3,x4} contains more than 3 heads)
Set operations
• Let 𝐴 = {1, 2, 3} and B = {2, 4, 6}
• 𝐴 ∩ 𝐵 = 2 and 𝐴 ∪ 𝐵 = 1, 2, 3, 4, 6
• 𝐴 − 𝐵 = 1, 3
• Properties:
• 𝑃 𝐴 ∩ 𝐵 ≤ min 𝑃 𝐴 , 𝑃 𝐵
• 𝑃 𝐴∪𝐵 ≤𝑃 𝐴 +𝑃 𝐵
• 𝑃 Ω − 𝐴 = 1 − 𝑃(𝐴)
• If 𝐴 ⊆ 𝐵 then 𝑃 𝐴 ≤ 𝑃(𝐵)
Conditional Probability
𝑃 𝐴∩𝐵 𝑃 𝐴, 𝐵
• 𝑃 𝐴|𝐵 = =
𝑃 𝐵 𝑃 𝐵
• A and B are independent if
• 𝑃 𝐴|𝐵 = 𝑃(𝐴)
• A and B are conditionally independent given C if
• 𝑃 𝐴, 𝐵 𝐶) = 𝑃 𝐴 𝐶)𝑃 𝐵 𝐶)
Conditional Probability
• Joint probability:
𝑃 𝐴1 ,…,𝐴𝑛
• 𝑃 𝐴1 𝐴2 , … , 𝐴𝑛 ) =
𝑃 𝐴2 ,…,𝐴𝑛

• Product rule:
• 𝑃 𝐴1 , … , 𝐴𝑛 = 𝑃 𝐴1 𝐴2 , … , 𝐴𝑛 )𝑃 𝐴2 , … , 𝐴𝑛
• Chain rule of probability:
• 𝑃 𝐴1 , … , 𝐴𝑛 = ς𝑛𝑖=1 𝑃 𝐴𝑖 𝐴1 , 𝐴2 , … 𝐴𝑖−1 )
Conditional Probability
• 𝑃 𝐴|𝐵 − probability of A given B
• Example:
– 𝑃(𝐴) probability that school is closed
– 𝑃 𝐵 probability that it snows
– 𝑃 𝐴|𝐵 probability of school closing if it snows
– 𝑃 𝐵|𝐴 probability of snowing if school is closed
Conditional Probability
• 𝑃 𝐴|𝐵 − probability of A given B
• Example:
– 𝑃(𝐴) probability that school is closed
– 𝑃 𝐵 probability that it snows
P(AnB) 0.005 𝑃 𝐴∩𝐵
𝑃 𝐴|𝐵 =
𝑃 𝐵
P(B) 0.02

P(A|B) 0.25
Conditional Probability
P(C) = 0.02 P(ØC) = 0.98
• We have: P(TP | C) = 0.9 P(ØTP | C) = 0.1
P(TP |ØC) = 0.2 P(ØTP |ØC) = 0.8

• We can now calculate joint probabilities

Has cancer?
Has cancer? Test positive?
Test positive? P(TP, C)
P(TP, C)
yes
yes yes
yes 0.018
yes
yes no 0.002
no yes
yes 0.196
no no 0.784

11
Bayes Theorem
• We can relate P(A|B) and P(B|A) through
Bayes’ rule:
𝑃 𝐵 𝐴)𝑃(𝐴)
𝑃 𝐴 𝐵 =
𝑃(𝐵)
Bayes Theorem
• We can relate P(A|B) and P(B|A) through
Bayes’ rule:
𝑃 𝐵 𝐴)𝑃(𝐴)
𝑃 𝐴 𝐵 =
𝑃(𝐵)

Posterior
Probability
Likelihood Prior
Normalizing Probability
Constant
Bayes Theorem
• We can relate P(A|B) and P(B|A) through
Bayes’ rule:
𝑃 𝐵 𝐴)𝑃(𝐴)
𝑃 𝐴 𝐵 =
𝑃(𝐵)
• 𝑃 𝐵 can be eliminated with law of total
probability:
𝑃 𝐵 = σ𝑗 𝑃 𝐵 𝐴𝑗 𝑃(𝐴𝑗 )
The four attendants of gasoline service station are supposed to wash the windshield of
customers’ car. Jack, who services 20% of all cars, fails to wash the windshield one
time in 20; Tom, who services 60% of all cars, fails to wash the windshield one time in
10; George who services 15% of all cars, fails to wash the windshield one time in 10;
and Peter, who services 5% of all cars, fails to wash the windshield one time in 20. If a
customer complains later that the windshield was not washed, what is the probability
that her car was serviced by Jack?

You might also like

pFad - Phonifier reborn

Pfad - The Proxy pFad of © 2024 Garber Painting. All rights reserved.

Note: This service is not intended for secure transactions such as banking, social media, email, or purchasing. Use at your own risk. We assume no liability whatsoever for broken pages.


Alternative Proxies:

Alternative Proxy

pFad Proxy

pFad v3 Proxy

pFad v4 Proxy