Scribd
Upload
10 views

NLP Lecture 01-10-Hmm

Hidden Markov Models (HMMs) are statistical models that represent sequences of observable events dependent on hidden states, characterized by states, observations, and transition probabilities. Key algorithms for HMMs include the Forward Algorithm for likelihood estimation, the Viterbi Algorithm for decoding, and the Baum-Welch Algorithm for parameter learning. HMMs are widely applied in fields such as Natural Language Processing, speech recognition, and bioinformatics.

Uploaded by

Al Mahmud Zayeef
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
10 views

NLP Lecture 01-10-Hmm

Hidden Markov Models (HMMs) are statistical models that represent sequences of observable events dependent on hidden states, characterized by states, observations, and transition probabilities. Key algorithms for HMMs include the Forward Algorithm for likelihood estimation, the Viterbi Algorithm for decoding, and the Baum-Welch Algorithm for parameter learning. HMMs are widely applied in fields such as Natural Language Processing, speech recognition, and bioinformatics.

Uploaded by

Al Mahmud Zayeef
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
You are on page 1/ 9

Hidden Markov Models (HMMs)

Md Mynoddin

Assistant Professor (CSE), RMSTU

26 Nov 2024

Md Mynoddin (Assistant Professor (CSE), RMSTU)Hidden Markov Models (HMMs) 26 Nov 2024 1/9
Introduction to Hidden Markov Models (HMMs)

Definition:
A Hidden Markov Model is a statistical model that represents a
sequence of observable events dependent on a sequence of hidden
states.
Key Characteristics:
Hidden States: States are not directly observable.
Markov Property: The next state depends only on the current state.
Observations: Each state generates an observable output.
Widely used in NLP, Speech Recognition, and Bioinformatics.

Md Mynoddin (Assistant Professor (CSE), RMSTU)Hidden Markov Models (HMMs) 26 Nov 2024 2/9
Components of an HMM

A Hidden Markov Model is defined by:


States (S):
A finite set of hidden states, S = {s1 , s2 , . . . , sn }.
Observations (O):
A finite set of observable events, O = {o1 , o2 , . . . , om }.
Transition Probabilities (A):
A = {aij }, where aij = P(sj |si ).
Emission Probabilities (B):
B = {bj (ok )}, where bj (ok ) = P(ok |sj ).
Initial Probabilities (π):
πi = P(si ), the probability of starting in state si .

Md Mynoddin (Assistant Professor (CSE), RMSTU)Hidden Markov Models (HMMs) 26 Nov 2024 3/9
Example of an HMM
Consider a weather prediction problem:
States: Hidden states are Rainy and Sunny.
Observations: Observables are Walk, Shop, and Clean.
Transition Probabilities (A):
 
0.7 0.3
A=
0.4 0.6

Emission Probabilities (B):


 
0.1 0.4 0.5
B=
0.6 0.3 0.1

Initial Probabilities (π):


 
π = 0.6 0.4

Md Mynoddin (Assistant Professor (CSE), RMSTU)Hidden Markov Models (HMMs) 26 Nov 2024 4/9
Three Fundamental Problems of HMMs

Problem 1 (Likelihood Estimation):


Given an observation sequence O and model λ = (A, B, π), compute
P(O|λ).
Solved using the Forward Algorithm.
Problem 2 (Decoding):
Find the most probable sequence of states S = {s1 , s2 , . . . } given O
and λ.
Solved using the Viterbi Algorithm.
Problem 3 (Learning):
Adjust parameters A, B, and π to maximize P(O|λ).
Solved using the Baum-Welch Algorithm.

Md Mynoddin (Assistant Professor (CSE), RMSTU)Hidden Markov Models (HMMs) 26 Nov 2024 5/9
Forward Algorithm

Computes P(O|λ), the probability of an observation sequence.


Uses dynamic programming to compute probabilities recursively.
Recurrence relation:
N
!
X
αt (j) = αt−1 (i)aij bj (ot )
i=1

Complexity: O(N 2 T ), where N is the number of states and T is the


length of the observation sequence.

Md Mynoddin (Assistant Professor (CSE), RMSTU)Hidden Markov Models (HMMs) 26 Nov 2024 6/9
Viterbi Algorithm

Finds the most probable sequence of hidden states.


Recurrence relation:
N
δt (j) = max [δt−1 (i)aij ] bj (ot )
i=1

Backtracking to recover the best path:

st = arg max δt (i)


i

Complexity: O(N 2 T ).

Md Mynoddin (Assistant Professor (CSE), RMSTU)Hidden Markov Models (HMMs) 26 Nov 2024 7/9
Applications of HMMs

Natural Language Processing (NLP):


Part-of-Speech Tagging
Named Entity Recognition
Speech Recognition:
Mapping audio signals to phonemes and words.
Bioinformatics:
Gene prediction and protein modeling.
Finance:
Modeling stock market trends.

Md Mynoddin (Assistant Professor (CSE), RMSTU)Hidden Markov Models (HMMs) 26 Nov 2024 8/9
Summary

HMMs are powerful tools for modeling sequences with hidden states.
Key components: states, observations, transitions, emissions, and
initial probabilities.
Solving HMM problems:
Likelihood estimation: Forward Algorithm.
Decoding: Viterbi Algorithm.
Parameter learning: Baum-Welch Algorithm.
Widely used in NLP, speech recognition, and bioinformatics.

Md Mynoddin (Assistant Professor (CSE), RMSTU)Hidden Markov Models (HMMs) 26 Nov 2024 9/9

You might also like

pFad - Phonifier reborn

Pfad - The Proxy pFad of © 2024 Garber Painting. All rights reserved.

Note: This service is not intended for secure transactions such as banking, social media, email, or purchasing. Use at your own risk. We assume no liability whatsoever for broken pages.


Alternative Proxies:

Alternative Proxy

pFad Proxy

pFad v3 Proxy

pFad v4 Proxy