Scribd
Upload
23 views19 pages

Stat NLP

Statistical natural language processing (NLP) aims to perform statistical inference for the field of NLP. It views language processing as a noisy channel of information transmission and requires models that characterize the transmission probabilities. Statistical NLP uses techniques like maximum likelihood classification, hidden Markov models, and conditional class density functions to model language. It helps NLP handle challenges like ambiguity, provide degrees of grammaticality, structural preference, and be more error tolerant compared to traditional rule-based approaches.

Uploaded by

Noun Nouna
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PPT, PDF, TXT or read online on Scribd
23 views19 pages

Stat NLP

Statistical natural language processing (NLP) aims to perform statistical inference for the field of NLP. It views language processing as a noisy channel of information transmission and requires models that characterize the transmission probabilities. Statistical NLP uses techniques like maximum likelihood classification, hidden Markov models, and conditional class density functions to model language. It helps NLP handle challenges like ambiguity, provide degrees of grammaticality, structural preference, and be more error tolerant compared to traditional rule-based approaches.

Uploaded by

Noun Nouna
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PPT, PDF, TXT or read online on Scribd
You are on page 1/ 19

Statistical Natural Language

Processing
What is NLP?
 Natural Language Processing (NLP), or
Computational Linguistics, is concerned with
theoretical and practical issues in the design
and implementation of computer systems for
processing human languages

 It is an interdisciplinary field which draws on


other areas of study such as computer science,
artificial intelligence, linguistics and logic
Applications of NLP
 naturallanguage interfaces to
databases
 programs for classifying and retrieving
documents by content
 explanation generation for expert
systems
 machine translation
 advanced word-processing tools
What makes NLP a
computational challenge?
 Ambiguous nature of Natural Language.
 There are varied applications for
language technology
 Knowledge representation is a difficult
task.
 There are different levels of information
encoded in our language
What is statistical NLP?
 Statistical NLP aims to perform
statistical inference for the field of NLP
 Statistical inference consists of taking
some data generated in accordance
with some unknown probability
distribution and making inferences.
Motivations for Statistical NLP
 Cognitive modeling of the human language
processing has not reached a stage where
we can have a complete mapping between
the language signal and the information
contents.
 Complete mapping is not always required.
 Statistical approach provides the flexibility
required for making the modeling of a
language more accurate.
Idea behind Statistical NLP
 View language processing as a noisy
channel information transmission.
 The approach requires a model that
characterizes the transmission by giving
for every message the probability of the
observed output
Statistical Modeling and
Classification
 Primitive
acoustic features
 Quantization
 Maximum likelihood and related rules
 Class conditional density function
 Hidden Markov Model Methodology
Details….
Primitive acoustic features are used to
estimate the speech spectrum on the basis of
its statistical properties.
By means of quantization a typical speech
signal can be represented as a sequence of
symbols and can be mapped using statistical
decision rules into a multidimensional
acoustic feature space, thus classifying the
signal.
Maximum Likelihood
Although there is no direct method for computing the
probability of a phonetic unit given its acoustic
features,we can use Bayes rule to estimate the
probability of a phonetic class given its features
from the likelihood of the features given the
class. This method leads to the maximum likelihood
classifier which assigns an unknown vector to that
class whose probability density function conditioned
on the class has the maximum value.

Another variant of the maximum likelihood methodology


is clustering.
Hidden Markov Models
A Hidden Markov Model, is a set of states (lexical
categories in our case) with directed edges
labeled with transition probabilities that
indicate the probability of moving to the state at
the end of the directed edge, given that one is
now in the state at the start of the edge. The
states are also labeled with a function which
indicates the probabilities of outputting different
symbols if in that state (while in a state, one
outputs a single symbol before moving to the
next state). In our case, the symbol output from a
state/lexical category is a word belonging to that
lexical category.
Hidden Markov Models (cont.)
Conditional Class Density
Function
All statistical methods of speech
recognition depend on the class
conditional density function.
These, in turn, depend on the existence of
a sufficiently large, correctly labeled
training set and well understood
statistical estimation techniques
How does statistics help
 Disambiguation may be achieved by
using stochastic context free grammars
 It helps in providing degrees of
grammaticality
 Naturalness
 Structural preference
 Error Tolerance
Example using stochastic
CFG
for example consider the sentence
“ John Walks “
The grammar is as follows :
1 S -> NP V 0.7
2 S -> NP 0.3
3 NP -> N 0.8
4 NP -> N N 0.2
5 N -> John 0.6
6 N -> Walks 0.4
7 V -> Walks 1.0
The numbers on the right represent the weights for each rule.The weight of the
analysis is the product of the weights of the rules used in the derivation.
Predicting the right sentence that is perceived is
based on these weights.
Degrees of grammaticality
 Traditional approaches to NLP do not
accommodate gradations of
grammaticality. A sentence is either
correct or not.
 In some cases acceptability may vary
with the structure and context of the
sentence.
Structural Preference
Consider the sentence
“ The emergency crews hate most is domestic
violence.”
The correct interpretation is:
“The emergency [that the crews hate most] is domestic
violence.”
These preferences can be seen more as structural
preferences rather than parsing preferences.
Statistical approaches can easily handle such structural
preferences.
Error Tolerance
A remarkable property of human
language comprehension is error
tolerance.
 Many sentences that the traditional
approach classifies as ungrammatical
can actually be interpreted by statistical
NLP techniques.
Conclusions
 Free and commercial software is now
available that provides a lot of NLP features.
(e.g. Microsoft XP has a speech recognition
software by which users can control menus
and execute commands)
 A lot of research is going into developing new
applications and investigating new techniques
and approaches that will make Statistical NLP
more feasible in the near future.

You might also like

pFad - Phonifier reborn

Pfad - The Proxy pFad of © 2024 Garber Painting. All rights reserved.

Note: This service is not intended for secure transactions such as banking, social media, email, or purchasing. Use at your own risk. We assume no liability whatsoever for broken pages.


Alternative Proxies:

Alternative Proxy

pFad Proxy

pFad v3 Proxy

pFad v4 Proxy