ARTIFICIAL INTELLIGENCE

By
BITS Team AI
Pilani
Pilani | Dubai | Goa | Hyderabad
 BITS
Pilani
Pilani | Dubai | Goa | Hyderabad

Contact Session-6
Probability, Bayesian Networks and Hidden Markov Models
 Facts
 Delhi is the capital of India
 You can reach Bangalore Airport from MG Road within 90 mins if
you go by route A.
 ” Would you call that as a fact?
 Are you sure this would be true always?
 Uncertainty

 You can reach Bangalore Airport from MG Road within 90 mins if you go by
route A.
 There is uncertainty in this information due to partial observability and non
determinism
 Agents should handle such uncertainty

 Previous approaches like Logic represent all possible world states

 Such approaches can’t be used as multiple possible states need to be

enumerated to handle the uncertainty in our information
 Belief

 You can reach Bangalore Airport from MG Road within 90 mins if

you go by route A.

 Such information can only provide a degree of belief, e.g., we are

80% confident that it would be true on any given day

 In order to deal such degree of belief, we need Probability

Theory
 Probability Theory

 Probability provides a way to summarize the uncertainty

 Sample Space: Set of all possible outcomes.
 Ex: After tossing 2 coins, the set of all possible outcomes are
 {HH, HT, TH, TT}
 Event: A subset of a sample space.
 An event of interest might be - {HH}
 Probability Basics – Probability Model

 A fully specified probability model associates a numerical

probability P(ω) with each possible world.
 The basic axioms
 Every possible world has a probability between 0 and 1
 Sum of probabilities of possible worlds is 1
 E.g., P(HH) = 0.25; P(HT) = 0.25; P(TT) = 0.25, P(TH) = 0.25
Probability Basics – Unconditional / Prior

 Unconditional / Prior probabilities: Propositions like

P(sum = 11) or P(two dices rolling equals) are called
Unconditional or Prior probabilities

 They refer to degree of belief in absence of any other

information
Probability Basics - Conditional
 However, most of the time we have some information, we
call it evidence
 E.g., we can be interested in two dice rolling a double (i.e., 1,1 or 2,2,
etc)
 When one die has rolled 5 and the other die is still
spinning
 Here, we not interested in unconditional probability of
rolling a double
 Instead, the conditional or posterior probability for
rolling a double given the first die has rolled a 5
 where | is pronounced “given”
 E.g., if you are going for a dentist for a checkup, P(cavity) = 0.2
 If you have a toothache, then P(cavity | toothache) = 0.6
 Probability Basics - Conditional
 Conditional probabilities can be expressed in unconditional
probabilities
 E.g., for proposition a, b
 Holds when P(b) > 0

 Product rule :
Example
In a factory there are 100 units of a certain product, 5 of which are defective. We
pick three units from the 100 units at random. What is the probability that none of
them are defective?
Let Ai as the event and i=1,2,3
 Probability Distribution

 If there is a random variable Weather with domain {sunny, rain, cloudy,

snow}, we could write
 P(Weather = sunny) = 0.6
 P(Weather = rain) = 0.1
 P(Weather = cloudy) = 0.2
 P(Weather = snow) = 0.1

 P(Weather) = <0.6, 0.1, 0.2, 0.1>

 P defines a probability distribution for the random variable Weather
 P is used for conditional distributions; P(X|Y) gives values for P(X = i | Y = j)
for all combinations of i and j
 Probability Density Function

 For continuous variables (e.g., weight) it is infeasible to write the distribution

as a vector

 Instead, we define the probability as a function of the value the variable can
take
 P(Weight = x) = Normal(x | mean = 60, std = 10)

 It says that the weight of an individual is Normal (Gaussian) distributed with

average weight being 60 kgs and a deviation of 10 kgs

 They are called as Probability Density Function

 Joint Probability Distributions
 Instead of distribution over single variable, we can model distribution over
multiple variables, separated by comma
 E.g., P(A, B) = P(A | B) . P(B)
 P(A, B) is the probability distribution over combination of all values of A and
B
 E.g., if A = Weather and B = Cavity
Independence
 If we have two random variables, TimeToBnlrAirport and
HyderabadWeather
P(TimeToBnlrAirport, HyderabadWeather)
To determine their relation, use the product rule
= P(TimeToBnlrAirport | HyderabadWeather) / P(HyderabadWeather)

 However, we would argue that HyderabadWeather and

TimeToBnlrAirport doesn’t have any relation and hence
P(TimeToBnlrAirport | HyderabadWeather) = P(TimeToBnlrAirport)
This is called Independence or Marginal Independence
Independence between propositions a and b can be written as
 Bayes Rule
 Using the product rule for propositions a and b

 Equating the right hand sides and dividing by P(a)

 This is called the Bayes Rule

Conditional Independence

Can generalize to more than 2 random variables

 E.g., K different symptom variables X1, X2,
… XK, and C = disease
 P(X1, X2,…. XK | C) = Π P(Xi | C)
 Also known as the naïve Bayes assumption
Bayesian Networks

 Full joint probability distributions can be

used for inference however can become
intractable as the number of variables
grow

 Bayesian Networks represent

dependencies among variables
 Can represent full joint probability
distribution
 What is a Bayesian Network

 A Bayesian network is a directed graph in which each node is annotated with

quantitative probability information.
 Each node corresponds to a random variable, which may be discrete or
continuous.
 A set of directed links or arrows connects pairs of nodes. If there is an arrow
from node X to node Y , X is said to be a parent of Y. The graph has no
directed cycles (and hence is a directed acyclic graph, or DAG).
 Each node Xi has a conditional probability distribution P(Xi | Parents(Xi )) that
quantifies the effect of the parents on the node.
Bayesian Networks

 Once the topology of the Bayesian

Network is laid out,
 Will specify the conditional probability
distribution for each variable given its
parents
 Such that the topology and conditional
probabilities suffices to specify the full
joint probability distribution of all
variables
Building a Bayesian Network

BITS Pilani, Deemed to be University under Section 3 of UGC

 BITS Pilani, Pilani Campus

Example Bayesian Net #2

A Burglary Alarm
System
– Fairly reliable on detecting a
burglary
– Also responds to earthquakes
– Two neighbors John and Mary
are asked to call you at work
when Burglary happens and
they hear the Alarm
– John nearly always calls when
he hears the alarm, however
sometimes confuses the
telephone ring with alarm and
calls then too
– Mary like loud music and often
misses the alarm altogether
– Problem: Given the
information that who has /
has not called we need to
estimate the probability of a
burglary
calculate the probability that
alarm has sounded, but neither
burglary nor earthquake
happened, and both John and
Mary called.
Example Bayesian Net #3Traffic Prediction -Travel Estimation

– AI system reminds traveler regarding

• day time
– Travel plan is to reach Delhi and the weather of Delhi may influence
the accommodation plans
– Traveler always take car to reach airport
– Car may be rerouted either due to road block or weekday traffic during
working hours which delays the arrival to airport
– Bars are always observed to be full on
• weekends
– Authorities block roads to safe the processions
– Processions observed during festive season or due to the political rally.
– Problem: Given the information that there is a political rally expected
estimate the probability of late arrival
BITS Pilani, Pilani Campus

Identify the R.Vs

Political
Festival RallY

Dependencies
among RVs

Procession Weather
@ Delhi
Find the
Conditional
Independences

Road
weekend Use ML to get the
best Linearization
among RVs
block

Construct the Bayes

Net
Cars
All bars
route
are
d
full
Encode the Local
dependencies by
CPT

Late
for
BGLR
airport
Example Bayesian Net #3
F ~F
Y ~Y
Identify the R.Vs
T
Political T
F Y P ~P Festival RallY

T T Dependencies
among RVs

T F D ~D
F T Procession Weather
T
@ Delhi Find the
F F Conditional
Independences

P R ~R
W ~W
T Road Use ML to get the
weekend T best Linearization
among RVs
F
block

R W C ~C
W A ~A Construct the Bayes
Net
T T Cars T
All bars
T F route
are F
d
F T full Encode the Local
dependencies by
CPT
F F
Late for
C L ~L BGLR
airport
T

F BITS Pilani, Pilani

Probabilistic Reasoning over Time
Hidden Markov Model

37
 Partial Observability

 Agents in partially observable environment should keep a track of current

state to the extent allowed by sensors
 E.g., Robot moving in a new maze

 Agent maintains a belief state representing the current possible world states
 Transition Model: Using belief state and transition model, the agent can how
the world might evolve in next time step
 Sensor Model: With the observed percepts and sensor model, the agent can
update the belief state
Degree of belief
 Earlier, the states were captured as
 Facts in Logic which can take either True/False

 To capture the degree of belief we will use Probability

Theory
 We model the change in world using a variable for each
aspect of state and at each point in time

 Transition models – describe the probability distribution of

variables at time t, given the state of the world at past times
 Sensor models – describe the probability of each percept at
time t, given the current state of the world
Time and Uncertainty
 Static World: Each random variable would
have a single fixed value
 E.g., Diagnosing a broken car

 Dynamic World: The state information

keeps changing with time
 E.g., treating a diabetic patient, tracking
the location of robot, tracking economic
activity of a nation
Hidden Markov model
Markov model /Markov chain
A Markov process is a process that generates a sequence of
outcomes in such a way that the probability of the next outcome
depends only on the current outcome and not on what happened
earlier.
MARKOV CHAIN: WEATHER EXAMPLE

Design a Markov Chain to

predict the weather of
tomorrow using previous
information of the past days.

states: 𝑆 = 𝑆1, 𝑆2, 𝑆3

 Our model has only 3

𝑆1 = 𝑆𝑢𝑛𝑛𝑦 , 𝑆2 =
𝑅𝑎𝑖𝑛𝑦, 𝑆3 = 𝐶𝑙𝑜𝑢𝑑𝑦.
Contd..
Contd..
state sequence notation: 𝑞1, 𝑞2, 𝑞3, 𝑞4, 𝑞5, … . ., where 𝑞𝑖 𝜖
{𝑆𝑢𝑛𝑛𝑦, 𝑅𝑎𝑖𝑛𝑦, 𝐶𝑙𝑜𝑢𝑑𝑦}.

 Markov Property
Example
Given that today is Sunny, what’s the probability that tomorrow is
Sunny and the next day Rainy?
Example2
Assume that yesterday’s weather was Rainy, and today is
Cloudy, what is the probability that tomorrow will be Sunny?
WHAT IS A HIDDEN MARKOV MODEL (HMM)?

A Hidden Markov Model, is a stochastic model where the states

of the model are hidden. Each state can emit an output which is
observed.
Imagine: You were locked in a room for several days and you
were asked about the weather outside. The only piece of
evidence you have is whether the person who comes into the
room bringing your daily meal is carrying an umbrella or not.
 What is hidden? Sunny, Rainy, Cloudy
 What can you observe? Umbrella or Not
Markov chain Vs HMM
Markov chain HMM
Hidden Markov Models (Formal)

• States Q = q1, q2…qN;

• Observations O= o1, o2…oN;
• Transition probabilities
• Transition probability matrix A = {aij}

• Emission Probability /Output probability

• Output probability matrix B={bi(k)}

• Special initial probability vector 

First-Order HMM Assumptions
• Markov assumption: probability of a state depends
only on the state that precedes it

P(qi | q1...qi 1) P(qi | qi 1)


How to build a second-order HMM?
• Second-order HMM
• Current state only depends on previous 2 states
• Example
• Trigram model over POS tags
Markov Chain for Weather
What is the probability of 4 consecutive warm
days?

Sequence is
warm-warm-warm-warm
And state sequence is
3-3-3-3
P(3,3,3,3) =
3a33a33a33a33 = 0.2 x (0.6)3 = 0.0432
 Hidden Markov Models
 It is a sequence model.
 Assigns a label or class to each unit in a sequence, thus
mapping a sequence of observations to a sequence
of labels.
 Probabilistic sequence model: given a sequence of units
(e.g. words, letters, morphemes, sentences), compute a
probability distribution over possible sequences of labels
and choose the best label sequence.
 This is a kind of generative model.

BITS Pilani, Pilani Campus

 Hidden Markov Model (HMM)
Oftentimes we want to know what produced the sequence – the
hidden sequence for the observed sequence. For example,
– Inferring the words (hidden) from acoustic signal (observed) in
speech recognition
– Assigning part-of-speech tags (hidden) to a sentence
(sequence of words) – POS tagging.
– Assigning named entity categories (hidden) to a sentence
(sequence of words) – Named Entity Recognition.

BITS Pilani, Pilani Campus

 Problem 1:
Observation Likelihood
• The probability of a observation sequence given a model
and state sequence
• Evaluation problem

BITS Pilani, Pilani Campus

 Problem 2:

• Most probable state sequence given a model and an

observation sequence
• Decoding problem

BITS Pilani, Pilani Campus

 Problem 3:

• Infer the best model parameters, given a partial model and an

observation sequence...
– That is, fill in the A and B tables with the right numbers --
• the numbers that make the observation sequence most likely
• This is to learn the probabilities!

BITS Pilani, Pilani Campus

 Solutions
Problem 1: Forward (learn observation sequence)
Problem 2: Viterbi (learn state sequence)
Problem 3: Forward-Backward (learn probabilities)
– An instance of EM (Expectation Maximization)

BITS Pilani, Pilani Campus

 Example :HMMs for Ice Cream

You are a climatologist in the year 2799 studying global warming

You can’t find any records of the weather in Baltimore for summer of
2007
But you find Jason Eisner’s diary which lists how many ice-creams
Jason ate every day that summer
Your job: figure out how hot it was each day

BITS Pilani, Pilani Campus

Hidden Markov Model
Example -1 – Viterbi Algorithm - Initialization

P(C)*P(3|C) = 0.2*0.1 = 0.02

C

*
Hot Cold
<S> 0.8 0.2
A Hot Cold

H Hot 0.6 0.4

Cold 0.5 0.5
P(H)*P(3|H) = 0.8*0.4 = 0.32
B 1 2 3
Hot .2 .4 .4
Cold .5 .4 .1

BITS Pilani, Pilani Campus

 Hidden Markov Model
Example -1 – Viterbi Algorithm - Recursion

P(C)*P(C|C)*P(1|C) = 0.02*0.5*0.5 = 0.005

P(H)*P(C|H)*P(1|C) = 0.32*0.4*0.5 = 0.064
C C
0.02

* Hot Cold
<S> 0.8 0.2
A Hot Cold

Hot 0.6 0.4

H H Cold 0.5 0.5
0.32
P(C)*P(H|C)*P(1|H) = 0.02*0.5*0.2 = 0.002 B 1 2 3
P(H)*P(H|H)*P(1|H) = 0.32*0.6*0.2 = 0.0384 Hot .2 .4 .4
Cold .5 .4 .1

BITS Pilani, Pilani Campus

 Hidden Markov Model
Example -1 – Viterbi Algorithm – Termination through Back Trace
P(C)*P(C|C)*P(3|C) = 0.064*0.5*0.1 = 0.032
P(H)*P(C|H)*P(3|C) = 0.0384*0.4*0.1 = 0.0015
C C C
0.02 0.064

* Best
Sequence:
Hot Cold
Hot🡪Cold🡪Cold
<S> 0.8 0.2
A Hot Cold

H H H Hot 0.6 0.4

Cold 0.5 0.5
0.32 0.0384
P(C)*P(H|C)*P(3|H) = 0.064 *0.5*0.2 = 0.0064 B 1 2 3
P(H)*P(H|H)*P(3|H) = 0.0384 *0.6*0.2 = 0.0046 Hot .2 .4 .4
Cold .5 .4 .1

BITS Pilani, Pilani Campus

Hidden Markov Model
Example -1 – Viterbi Algorithm

Source Credit : Speech and Language Processing - Jurafsky and Martin

BITS Pilani, Pilani Campus

Hidden Markov Model
Example -1 – Viterbi Algorithm

Source Credit : Speech and Language Processing - Jurafsky and Martin

BITS Pilani, Pilani Campus

Hidden Markov Model
Example -4 – Naïve Search

Hot Cold
<S> 0.8 0.2

HHH P(H)*P(1|H)*P(H|H)*P(3|H)*P(H|H)*P(1|H)
A Hot Cold
HHC
HCC Hot 0.7 0.3
CCC Cold 0.4 0.6

CHC P(C)*P(1|C)*P(H|C)*P(3|H)*P(C|H)*P(1|C) 0.0024

=0.2*0.5*0.4*0.4*0.3*0.5 B 1 2 3
CCH Hot .2 .4 .4
CHH Cold .5 .4 .1

HCH
BITS Pilani, Pilani Campus