Word senses and Wordnet

 Introduction
 Word senses
 Wordnet
 Word sense disambiguation
Introduction
A word sense is the locus of word meaning; definitions and
meaning relations are defined at the level of the word sense
rather than wordforms.
Words are ambiguous: the same word can be used to mean
different things.
Word sense disambiguation (WSD), has a very long history
in computational linguistics and many applications, e.g.
question answering.
Knowing the relation between multiple senses of a word
can play an important role in language understanding.
WordNet is a large database of lexical relations for English,
and a variety of languages.
CS3TM20 © XH 2
Word Senses
 How can we define the meaning of a word sense?
 We introduced the standard computational approach of
representing a word as an embedding, a point in semantic
space.
 Dictionaries or thesauruses give textual definitions for each
sense called glosses.
bank:
1. financial institution that accepts deposits and channels the money into lending
activities
2. sloping land (especially the slope beside a body of water
 Dictionaries often give example sentences along with
glosses, which are used to help build a sense
representation. CS3TM20 © XH 3
Relations Between Senses
• Synonymy
Synonyms include such pairs as
couch/sofa vomit/throw up filbert/hazelnut
car/automobile
• Antonymy
long/short big/little fast/slow cold/hot
dark/light rise/ fall up/ down
in/out
another group of antonyms, reversives, describe
change or movement in opposite directions, such as
rise/fall or
up/down CS3TM20 © XH 4
Hyponym/hypernym

Car is a hyponym of vehicle,

dog is a hyponym of animal
Conversely, we say that vehicle is a hypernym of car, and
animal is a hypernym of dog

 Hypernymy is useful for tasks like textual entailment

or question answering; knowing that leukemia is a type of
cancer, for example, would certainly be useful in
answering questions about leukemia.
 Another name for the hypernym/hyponym structure is the
IS-A hierarchy CS3TM20 © XH 5
Meronymy
Another common relation is the part-whole relation.
A leg is part of a chair;
a wheel is part of a car.
 We say that wheel is a meronym of car, and car is a
holonym of wheel.

CS3TM20 © XH 6
Metonymy
 is the use of one aspect of a concept or entity to refer to other
aspects of the entity or to the entity itself.
 We are performing metonymy when we use the phrase the
White House to refer to the administration whose office is
in the White House.

AUTHOR ↔ WORKS OF AUTHOR

(Jane Austen wrote Emma) (I really love Jane Austen)

FRUITTREE ↔ FRUIT
(Plums have beautiful blossoms)(I ate a preserved plum
yesterday)
CS3TM20 © XH 7
Structured Polysemy
 The senses of a word can also be related semantically,
 We call the relationship between them structured
polysemy.
The bank is on the corner of Nassau and Witherspoon.
 This sense, perhaps bank means something like “the
building belonging to a financial institution”.
 These two kinds of senses occur together for many other
words as well (school, university, hospital, etc.).
 Thus, there is a systematic relationship between senses
that we might represent as
BUILDING↔ORGANIZATION.
CS3TM20 © XH 8
Wordnet

 Words are ambiguous: the same word can be used to

mean different things.
 WordNet, a large online database that represents word
senses—with versions in many languages.
 WordNet also represents relations between senses
 Knowing the relation between two senses can play an
important role in language understanding.
 Practical this week on wordnet

CS3TM20 © XH 9
Word sense disambiguation :

The task of selecting the correct sense for a word is called

word sense disambiguation
 knowledge-based algorithms, rely solely on WordNet or
other such resources and don’t require labeled data.
 While supervised algorithms generally work better,
knowledge-based methods can be used in
languages or domains where thesauruses or
dictionaries but not sense labeled corpora are available.

CS3TM20 © XH 10
The Lesk algorithm

CS3TM20 © XH 11
The Lesk algorithm

This is the oldest and most powerful knowledge-based WSD

method.
 Lesk is a family of algorithms that choose the sense whose
dictionary gloss or definition shares the most words with the
target word’s neighborhood.
 The Simplified Lesk algorithm. The COMPUTEOVERLAP
function returns the number of words in common between two
sets, ignoring function words or other words on a stop list.
 The original Lesk algorithm defines the context in a more
complex way.
CS3TM20 © XH 12
Example: (exam style question)
Disambiguating the word bank in the following context:
“The bank can guarantee deposits will eventually cover future
tuition costs because it invests in adjustable rate mortgage
securities”, given the following two WordNet senses:
Bank1 Gloss: a financial institution that accepts deposits and channels
the money into lending activities
Examples: “he cashed a check at the bank”,
“that bank holds the mortgage on my home”
Bank2 Gloss: sloping land (especially the slope beside a body of water)
Examples: “they pulled the canoe up on the bank”,
“he sat on the bank of the river and watched the currents”

CS3TM20 © XH 13
 “The bank can guarantee deposits will eventually cover future
tuition costs because it invests in adjustable-rate mortgage
securities”
Bank1 Gloss: a financial institution that accepts deposits and channels
the money into lending activities
Examples: “he cashed a check at the bank”,
“that bank holds the mortgage on my home”
Bank2 Gloss: sloping land (especially the slope beside a body of water)
Examples: “they pulled the canoe up on the bank”,
“he sat on the bank of the river and watched the currents”
Answer: Sense Bank1 has two non-stop words overlapping
with the context, deposits and mortgage, while sense Bank2
has zero words, so sense Bank1 is chosen.
CS3TM20 © XH 14
Information extraction
 Introduction
 Named entity recognition
 Information extraction
 Chunking
 Introduction
 The process of information extraction (IE) turns the
unstructured information embedded in texts into structured
data, e.g for populating a relational database.
 The task of relation extraction: finding and classifying
semantic relations among the text entities.
 These are often binary relations like child-of, employment,
part-whole, and geospatial relations.
 Information extraction systems generally simply identify and
classify the segments in a text that are likely to contain
valuable information.
 One kind of partial parsing is known as chunking.
CS3TM20 © XH 16
Named Entities (NE) recognition
Named entity, in its core usage, means anything that can be referred to with
a proper name. Most common 4 tags:
• PER (Person): “Marie Curie”
• LOC (Location): “New York City”
• ORG (Organization): “Stanford University”
• GPE (Geo-Political Entity): "Boulder, Colorado"
Often multi-word phrases
But the term is also extended to things that aren't entities:
dates,
times,
prices
 Segmentation issues
 In POS tagging, no segmentation problem since each word
gets one tag.
 In NER we must find and segment the entities!
 Type ambiguity
 Applications:
 Sentiment analysis: consumer’s sentiment toward a particular
company or person?
 Question Answering: answer questions about an entity?
 Information Extraction: Extracting facts about entities from
text.
Information Extraction
Examples
Job market surveillance software
aspect level sentiment analysis

 Names of People, Places,

Organizations
 Healthcare area
Names of symptoms
Relation extraction
In most standard information extraction applications, the
domain elements correspond
to the named entities that occur in the text,
to the underlying entities that result from coreference resolution, or
to entities selected from a domain ontology.

ACE project
Chunking (Partial Parsing)

Chunk
token

Partial/shallow parse: Chunk

Many NLP tasks do not require complex, complete parse

trees for all input
Chunking: means grouping of words/tokens into chunks
Partial/Shallow parse
Works on top of POS tagging.
Uses POS-tags as input and provides chunks as
output.
21
Chunking (Partial Parsing)

Chunk
token

Chunk
Why chunking?
Chunking can break sentences into phrases that are more
useful than individual words and yield meaningful results.
Chunking is very important when you want to extract
information from text such as locations, person names.
(entity extraction)
Easy to implement
22
Regex based Chunking

• Grammar
• <DT>? → one or zero determiner
<JJ>* → zero or more adjectives
<NN> → Noun
• Practical this week
>>> sentence = [("the", "DT"), ("little", "JJ"), ("yellow", "JJ"), ... ("dog", "NN"), ("barked", "VBD"), ("at",
"IN"), ("the", "DT"), ("cat", "NN")]
>>> grammar = "NP: {<DT>?<JJ>*<NN>}"
>>> cp = nltk.RegexpParser(grammar)
>>> result = cp.parse(sentence)
>>> print(result)
(S (NP the/DT little/JJ yellow/JJ dog/NN) barked/VBD at/IN (NP the/DT cat/NN))
>>> result.draw()

23
Machine learning based Chunking

• Annotation
• Penn Treebank dataset
• The chunk tags use the IOB format.
• IOB : Inside, Outside, Beginning
• B- prefix before a tag indicates, it’s the beginning of a chunk
• I- prefix indicates that it’s inside a chunk
• O- tag indicates the token doesn’t belong to any chunk

• Training a classifier
• Evaluation

24