Pattern and Face Recognition

Spandana Rao
● Distal Stimulus: The actual object in the real world (e.g., a tree or car).
● Proximal Stimulus: The sensory data received by sense organs (e.g., the
retinal image formed in the eyes from reflected light).
● Percept: The meaningful interpretation of the proximal stimulus, such as
recognizing trees or cars from upside-down, two-dimensional retinal images.
Does our knowledge, and other rules we have learned
throughout our life, maybe influence our perception of
the world?
Bottom-up theories

Bottom-up theories describe approaches where perception starts with the stimuli whose appearance you take
in through your eye. You look out onto the cityscape, and perception happens when the light information is
transported to your brain. Therefore, they are datadriven (i.e., stimulus-driven) theories.
Top-down Theories

suggest that perception is driven by high-level cognitive processes, prior knowledge, and expectations.

These high-level processes then guide and shape how sensory data, such as perceptual stimuli, are
interpreted.

Con of top-down theory

● Expectations play a crucial role in perception; they can cause us to "see" things that aren't there based
on what we expect to see.
● Example: If people expect to see someone in a specific place, they might mistakenly perceive another
person who looks vaguely similar as that person.
Reality: A mix of both

Top-down and bottom-up approaches: While these are often presented as

opposites, they actually address different aspects of the same cognitive process.

A comprehensive theory of perception will need to integrate both top-down

(knowledge-driven) and bottom-up (sensory-driven) processes.
Bottom-Up Theories

Direct Perception

Acitivity

What letters are these?

A H
What letter is this?
Can you read this?
Gibson’s Theory of Direct Perception

● also called ecological perception - opposed associationism and believed that contextual information
in the real world is sufficient for perception, without needing higher-level cognitive processes.
● Hoffding function: questions how perception connects to stored knowledge
● Ecological model: Gibson emphasized perception in real-world settings, where rich contextual
information is present, as opposed to controlled laboratory conditions
● Neuroscience and direct perception: Mirror neurons, which activate both when a person acts and
when they observe others acting, support direct perception. This suggests that we can understand
others' emotions, expressions, and movements before consciously forming hypotheses.
● Separate neural pathways: Studies show that form, color, and texture are processed independently
in the lateral occipital area, further supporting direct perception without needing complex thought
processes.
Template Matching Theory

Activity
● Template Theories: These propose that we have numerous detailed templates stored in our
minds, which we use to recognize patterns by matching what we observe with the
templates.
● Examples: Fingerprint recognition, UPC barcodes, numeral recognition on checks, and
chess players using previous games for fast recognition are real-world examples.
● Challenges of template matching: Template matching works well for exact tasks like
identifying bank numerals but is less effective for everyday perception, where stimuli vary
in size, shape, or orientation (e.g., different views of the same face).
● Criticism of template matching: The theory struggles to explain our ability to recognize
letters and objects across different forms, sizes, and orientations. It’s impractical to store
countless templates for each possible variation of an object.
● Neuroscience and template theories: The left fusiform gyrus activates more when
presented with letters than digits, suggesting specialization for different stimuli in the
visual cortex.
● Handwriting recognition: While humans can easily perceive handwritten text despite
variations, computers struggle with handwriting due to its variability. Machines typically
rely on algorithms to interpret context and angular positions of letters.
● Human vs. robot perception: Humans have a perceptual advantage over robots because
of our vast knowledge of the environment, which helps us recognize regularities that
robots can’t easily detect.
Feature-Matching Theories:
● These suggest that instead of matching whole patterns to templates, we
compare individual features of a stimulus to those stored in memory.
1. Pandemonium Model:
● A feature-matching theory developed by Oliver Selfridge, where four types
of "demons" (image, feature, cognitive, and decision) metaphorically analyze
and process the features of a stimulus.
In Oliver Selfridge’s Pandemonium Model, there are four kinds of demons:
1. image demons
2. feature demons
3. cognitive demons
4. decision demons.
According to Oliver Selfridge’s
feature-matching model, we recognize
patterns by matching observed features
to features already stored in memory. We
recognize the patterns for which we have
found the greatest number of matches
Activity
2. Global Precedence Effect (overall shape): Studies (Navon, 1977; Kimchi, 1992) show
that people can recognize global patterns (like a large letter) more quickly than local details,
especially when local elements are tightly spaced.

3. Local Precedence Effect (small scle details): When local features are spaced further
apart, people more easily recognize individual local features over the global pattern.

The relationship between local and global features can either facilitate or interfere with
perception, depending on the arrangement of stimuli.
Recognition-by-Components (RBC) Theory

● Developed by Irving Biederman (1987), RBC theory explains how we

recognize 3-D objects by decomposing them into simple geometric shapes
called geons (geometrical ions).
● Geons: These include basic shapes like bricks, cylinders, wedges, cones, and
curved-axis versions. They are simple, easily recognized from various
viewpoints, and can be recomposed to form a variety of objects.
Activity

Try creating your own image of a

dog using only geons!
● Object Recognition Process: According to RBC theory, we recognize objects by:
○ Observing edges of objects.
○ Breaking them down into geons.
○ Reconstructing geons into various forms to identify objects.
● Viewpoint-Invariant Recognition: Geons are distinct from various perspectives, making object recognition consistent
even with partial views or visual noise.
● Neuroscience Support: Cells in the inferior temporal cortex are sensitive to changes in geons, reacting strongly to
viewpoint-invariant features (Vogels et al., 2001).
● Limitations of RBC Theory:
○ RBC explains general object classification but struggles to explain recognition of specific objects (e.g.,
distinguishing between different faces).
○ The theory requires refinement, particularly in describing the relationships between parts of objects.
● Criticism from Neuroscience: Some neurons respond primarily to one specific view of an object and decrease their
response as the object rotates, challenging the idea of complete viewpoint invariance (Logothetis, Pauls, & Poggio, 1995).
Activity
Think of the most ‘doggiest’ dog you
can think of!
Prototype Matching
● Prototype matching is a perceptual model that addresses limitations in template-matching
and feature analysis models by using idealized representations (prototypes).
● Instead of requiring an exact match, inputs are compared to these prototypes, allowing
flexibility in recognition. Prototypes represent idealized versions of objects (e.g., a typical
letter, dog, or cup).
● Unlike template models, prototype models do not rely on specific features; the more features
shared, the higher the probability of recognition.
● Prototypes account for both an object’s features and their relationships, enhancing flexibility.
● Posner and Keele (1968) showed that prototypes are quickly formed and used for
recognizing distorted dot patterns.
● Research shows prototypes also influence recognition of real-world images, such as altered
faces.
● Cabeza et al. (1999) demonstrated a "prototype effect" with altered facial photographs.
Top-Down Theories of Perception

focus on how higher-level cognitive processes (e.g., knowledge, expectations, and

prior experiences) influence perception. This contrasts with bottom-up
approaches, where perception is driven by raw sensory data. Here's an overview:
Constructive Perception:

● Perception as a Cognitive Construction: The theory posits that perceivers actively build their perception of a stimulus. Sensory
data serve as the foundation, but perception is influenced by cognitive factors like expectations, prior knowledge, and inferences.
● Intelligent Perception: Higher-order thinking and learning play a crucial role in perception, making it an intelligent, interactive
process. The mind does not merely passively receive data but constructs meaning based on past experiences and environmental
context.

Key Aspects of Constructive Perception:

1. Sensory Data: The information received from the environment (e.g., light hitting the eyes).
2. Stored Knowledge: What the perceiver already knows and has experienced in the past.
3. Inference: The perceiver uses high-level reasoning to fill in gaps or make judgments about what they perceive.
Perceptual Constancies:

● Size Constancy: Despite changes in the retinal image (e.g., a car getting bigger as it approaches), we perceive the object as remaining the same size.

● Color Constancy: We perceive familiar objects (e.g., bananas) as retaining their color despite changes in lighting.

Unconscious Inference:

● We often make unconscious judgments based on prior learning. For example, even when part of a stop sign is obscured by vines, we can infer from its shape and
color that it is a stop sign.

Context Effects:

● Contextual Influence: The context in which an object is seen affects how quickly or accurately it is recognized. For example, a loaf of bread is recognized faster
in a kitchen than in an unrelated setting like a garage.

● Configural-Superiority Effect: More complex configurations of objects are easier to recognize than individual elements in isolation. For instance, spotting a
unique figure among triangles is easier than identifying a unique line among slanted lines.

Interaction with Intelligence:

● Perception and intelligence are intertwined. What we perceive is influenced by what we expect or think based on previous experience.
Word-superiority effect - When people are presented with strings of letters, it is easier for them to identify a single
letter if the string makes sense and forms a word instead of being just a nonsense sequel of letters. For example, it is
easier to recognize the letter “o” in the word “house” than in the word “huseo”

Limitations of Extreme Views:

● An extreme top-down view would underestimate the importance of sensory data and could lead to perceptual
errors (e.g., mistaking a stranger for a friend based on expectation).
● An extreme bottom-up view would ignore the role of past knowledge and experience, failing to explain how
we efficiently process complex stimuli.

Balanced View:

● A more nuanced approach involves interaction between bottom-up and top-down processes, suggesting
that both sensory input and cognitive factors shape perception, leading to accurate and efficient understanding
of the world.
● Visual Control of Action vs. Perception: The brain uses different pathways for perceiving
objects and acting upon them. When perceiving for action, like reaching for an object, we
process it more analytically. When simply observing, we take a more holistic approach.
● Configural-Superiority Effect: This phenomenon, where more complex configurations
are easier to recognize than individual elements, highlights the interplay of bottom-up
sensory input and top-down cognitive processing. The brain uses both the immediate
sensory data and contextual understanding to interpret patterns.
How Do Bottom-Up Theories and Top-Down Theories Go Together?

● Complementary Processes: Instead of viewing these approaches as mutually exclusive, they can be integrated. Sensory
information may often be sufficient, but top-down processes can enhance perception when sensory data are degraded or
incomplete (e.g., poor lighting, partially obscured objects, or noisy environments).
● Balance Between Sensory and Cognitive Information: The bottom-up theory is valuable when sensory data are rich and
unambiguous, while the top-down approach plays a more prominent role in situations requiring higher-level processing,
such as interpreting degraded stimuli or making quick decisions based on prior experience.

Experimental Support for Integration:

● When sensory information is compromised (e.g., due to poor lighting, visual noise, or brief exposure), top-down processing
helps by filling in gaps through prior knowledge or unconscious inference.
● Attentional Focus and Perception: Research suggests that early stages of visual processing focus on raw sensory input,
but later stages are influenced by attention and cognitive goals. For example, recognizing an object visually can differ based
on whether one intends to interact with it (e.g., seeing a phone vs. planning to pick it up).
Perception of Objects and Forms

1. Viewer-centered Representation
2. object-centered representation
Viewer-Centered Representation:

● In this approach, perception is subjective and dependent on how the object appears to the viewer. The object
is perceived in relation to the observer’s perspective, which means its shape and orientation may vary based on
the viewer's angle.
● Multiple views of the object from different angles are stored in memory. When recognizing an object, the
mind rotates or adjusts these stored images until one fits the current perception.
● For example, if you're looking at a chair from the side, you store that particular view, and when you encounter
a chair again from a different angle, you match it with the stored side view by mentally rotating the new
perception.
Object-Centered Representation:

● In contrast, this approach posits that we perceive objects independently of our viewpoint. The
shape of the object remains constant regardless of the angle from which we see it.
● The brain establishes the major and minor axes of the object to form a stable representation that
doesn't change based on the viewer's position. This allows for consistent recognition of objects even
when their appearance from a certain viewpoint is unfamiliar.
● For example, a cup is perceived with its handle on the right or left depending on the angle, but its
overall structure (as a cylindrical object with a handle) remains constant.
Integrating Object-centered and viewer-centered
● Instead of considering these two approaches as mutually exclusive, researchers suggest
that both forms of representation may coexist on a continuum. Depending on the
situation, we may switch between viewer-centered and object-centered mechanisms.
○ Viewer-Centered Mechanisms: Helpful when recognizing objects from varying
perspectives or when angle and orientation are key to perception (e.g., identifying an
object that is upside-down).
○ Object-Centered Mechanisms: Useful for recognizing stable features of objects that
remain constant across different views.
Landmark-Centered Representation:

● A third approach involves perceiving objects relative to landmarks or

familiar points of reference. This is common when navigating unfamiliar
spaces. For instance, when exploring a new city, you might perceive the
environment in relation to a prominent landmark like your hotel.
● This kind of representation highlights how perception can also be influenced
by spatial relations between objects and a fixed, known point.
Practical Applications:

● These perceptual strategies are often used in fields like photography and
fashion to create desired effects. For example:
○ Models might use angles to manipulate the viewer's perception of their
features, such as positioning the face to shorten the appearance of a long
nose.
○ Designers use optical illusions in clothing to enhance or de-emphasize
certain body features.
Pattern and Face recognition Approaches
Activity
Two Systems for Pattern and Face recognition - Martha Farah et al

1. Feature Analysis System:

● This system specializes in recognizing parts of objects and assembling them into a coherent whole.
● It is particularly useful for object recognition in situations where individual components (such as
parts of a flower) need to be analyzed. For example, when examining the features of a tulip in
biology class (e.g., its petals, stamen, pistil), you're using this system.
● Feature analysis is also occasionally used in face recognition, especially when we encounter
someone familiar but struggle to recognize them at first. We might focus on specific features like
their eyes or nose to make the connection.
2. Configurational System:

● This system is more adept at recognizing larger configurations or holistic patterns, especially with faces.
● It is not as effective at breaking down parts of objects but is highly sensitive to recognizing familiar patterns at
once.
● For instance, when you spot a friend in a crowd, you recognize them instantly, not by analyzing individual
features, but by recognizing the whole configuration of their face.
● Face recognition relies more on this configurational system, which is why we may fail to notice subtle
changes in someone's appearance, like a new pair of glasses, because we recognize the entire face holistically.
Face Recognition and the Brain:

● Research suggests that the fusiform gyrus in the temporal lobe is crucial for
recognizing faces. This region responds intensely when we look at faces, but
less so for other objects.
● There is evidence to suggest that face recognition is special even from
infancy. For example, infants are more likely to track a photograph of a
human face than an equally complex non-face stimulus.
Activity - Stare at these faces
Research on Face and Object Recognition:

● A study by Farah and colleagues compared people's ability to recognize parts of faces versus parts of houses. Participants had
more difficulty recognizing individual parts of faces (e.g., a nose or an ear) compared to whole faces. In contrast, they
recognized parts of houses (e.g., a window or door) just as well as they recognized whole houses.
● This supports the idea that face recognition is configurational and involves holistic processing, while recognizing non-face
objects, like houses, can rely more on feature analysis.

● Research backs the idea that facial recognition is special, where even infants have the ability to track movements of a photograph
of a human face more rapidly than they track movements of other objects of similar complexity (Farah, 2000a).

● Staring at distorted faces - If you stare at a distorted face for a while and then stare at a normal face, the normal face will look
distorted in the opposite direction.
Face and Emotion Interaction:

● Cognitive processing of faces can also interact with the emotions expressed by the face.
Research has shown an age-related “face positivity” effect where older adults prefer
looking at happy faces and tend to avoid sad or angry faces, unlike younger adults.
● Additionally, happy faces are often rated as more familiar than neutral or negative faces.
● The amygdala plays a role in processing emotional expressions, particularly negative ones
like fear.
● there may be a difference in how highly anxious individuals versus less anxious
individuals process fearful expressions. Highly anxious people seem to process fear
automatically, even when not paying attention to it, while less anxious people may not.
Attention and Perception

● Inattentional Blindness: When our attention is directed elsewhere, significant events or changes can go unnoticed, such as when people
miss a gorilla walking through a scene or fail to see traffic signals
● Attentional Blink Lab Experiment: In a study where participants searched for letters among rapidly presented digits (10 items per second),
the first letter was detected 90% of the time, but the second letter, if shown shortly after the first, was missed up to 70% of the time. This
phenomenon, termed the attentional blink, illustrates the limitations of our attentional capacity (Raymond et al, 1992).
○ Unattended Targets: The study proposed that although missed targets are processed at some level, they can be lost or forgotten
while waiting for the first target to be encoded.

Attention and Memory Formation

● Importance of Attention: Simply seeing or reading information is insufficient for encoding it into memory; focused attention is crucial. If
you do not actively attend to information, it is unlikely to form reliable memory traces in the brain.
● Memory Encoding and Retrieval: fMRI research confirmed that attention plays a vital role both during the initial encoding of information
and during retrieval, underscoring the challenges students face in learning without focused attention.
Neuroscience of face recognition
Functional Magnetic Resonance Imaging (fMRI) studies

● Parahippocampal Gyrus: This brain region is specifically associated with scene recognition. In the study, scenes presented as potential
second targets showed significant brain activity in this area, even when participants reported not seeing them, indicating that subliminal
perception occurs (Marois et al., 2004).

1. Fusiform Gyrus and Emotion

● Activation by Emotion: The fusiform gyrus shows increased activation when participants are asked to name the expression of a face rather
than the person. This suggests that emotional processing is closely linked to face recognition (Ganel et al., 2005).
● Autism and Emotional Recognition: Individuals with autism exhibit reduced activation in the fusiform gyrus and face challenges with
emotional recognition. Although they can learn to identify emotions, this process is effortful and does not lead to automatic recognition or
increased activation in the fusiform gyrus (Bolte et al., 2006; Hall, Szechtman, & Nahmias, 2003).
Nof face and pattern recognition

2. Debate on Fusiform Gyrus Specialization

● Specialization for Face Perception: There is ongoing debate about whether the fusiform gyrus is solely specialized for face recognition or
if it is involved in other forms of perception. While it shows the greatest activation during face processing, other brain areas may also be
engaged, indicating that the perception process is not binary but exists on a continuum (Haxby et al., 2001; Haxby, Gobbini, & Montgomery,
2004; O’Toole et al., 2005).

3. Expert-Individuation Hypothesis

● Visual Expertise Activation: This hypothesis suggests that the fusiform gyrus activates in response to items that an individual has expertise
in. For example, a bird expert can distinguish between similar-looking birds, resulting in fusiform gyrus activation when shown images of
those birds. This activation is also observed in experts of other categories, such as cars.
● Learning and Activation: Training individuals to differentiate between similar objects, including abstract figures, leads to observable
activation in the fusiform gyrus, further supporting the idea that this region plays a critical role in recognizing patterns and faces (Gauthier et
al., 1999, 2000; Rhodes et al., 2004; Xu, 2005)
Prosopagnosia

● Prosopagnosia is the inability to recognize faces, which suggests damage to the configurational system responsible for face
recognition (Damasio, Tranel, & Damasio, 1990; De Renzi Faglioni, Grossi, & Nichelli, 1991; Farah, 2004).
● Emotion Recognition vs. Identity Recognition: Individuals with prosopagnosia can still perceive facial expressions (e.g.,
happiness, sadness, anger) but cannot identify whether the person is a stranger, friend, or family member, including their
own mother.

Brain Areas Involved

● Right Fusiform Gyrus: Damage to the right fusiform gyrus, whether unilateral or bilateral, is especially influential in
prosopagnosia. This region plays a crucial role in facial recognition.
● Bilateral Lesions: If bilateral lesions include the right anterior temporal lobe, they particularly affect facial memories,
exacerbating recognition difficulties (Barton, 2008).
Reading Disabilities and System Interaction

● Element-Based System: Early reading difficulties may arise from damage to the first,
element-based system, which recognizes individual features of letters to form words.
● Transition Between Systems: A typical reader often transitions from recognizing letters
individually to understanding words as complete units (the second system). Some reading
disabilities may occur when this transition does not successfully take place, resulting in
difficulty moving from element-based recognition to whole-word recognition.
Research on culture and face recognition

1. Easier Recognition Within Ethnic Groups

● Own Ethnic Group Advantage: People tend to find it easier to recognize

faces of individuals who belong to their own ethnic group. For instance, an
African-American individual might find it easier to distinguish between
black faces than white or Asian faces.
● Familiarity Factor: This phenomenon is often attributed to familiarity with
faces within one’s own ethnic group, suggesting that exposure plays a
significant role in facial recognition ability.
2. Social Categorization Impact

● Recognition Difficulty Across Groups: Research indicates that social categorization affects face recognition. When a
person is perceived as an "out-group" member—someone who does not belong to the viewer's ethnic group—it becomes
more challenging to recognize that person’s face.
● Influence of Imaginary Differences: The effect of social categorization is so robust that it can be triggered by even
imaginary differences, such as personality labels like “red” or “green.” For example, a person may find it easier to
recognize individuals identified with a “red” personality over those labeled as “green,” regardless of their racial
background.

3. Experimental Evidence

● Altered Hairdos: Studies have shown that even superficial changes, such as adding an African-American or Latino
hairstyle to a white face, can impact recognition ability, further illustrating how social categories influence perception
(Bernstein et al., 2007; MacLin & Malpass, 2001, 2003; Ge et al., 2009).
References

Textbooks
● Sternberg
● Galotti