3.2 Core study 2: Baron-Cohen et al.

(Eyes test)

Baron-Cohen, S., Wheelwright, S., Hill, J., Raste, Y., & Plumb, I. (2001).
The ‘Reading the Mind in the Eyes’ test revised version: a study with
normal adults, and adults with Asperger syndrome or high-functioning
autism. Journal of child psychology and psychiatry, 42(2), 241–251.

The psychology being investigated

A person with an autism spectrum disorder (ASD) does not fully develop
cognitive processes linked to social interaction. ASD occurs in
approximately 1% of the population. Individuals with ASD share difficulties
in social functioning, such as limited social sensitivity, communication and
ability to cope with change, and may have narrow interests. The term ASD
is now used to refer to both high-functioning autism (HFA) and Asperger
syndrome (AS), however the terms HFA and AS were used at the time that
the Baron-Cohen et al. study was published.

Baron-Cohen et al. suggest that people with ASD lack or have an

underdeveloped cognitive process called a theory of mind. A theory of mind
is a cognitive ability enabling people to realise that others have different
feelings, beliefs, knowledge, and desires from their own. Individuals with
ASD find it difficult to understand that other people have their own plans,
thoughts, or points of view. A theory of mind is often linked to empathy.
Empathy is the ability to understand the world as another person does; to
appreciate their feelings or emotional state separate from our own.

There are several different tests available to measure a person’s theory of

mind but most of these are designed for children. Baron-Cohen et al.
(1997) developed a test called the ‘Reading the Mind in the Eyes’ task to
use with adults to test their ability to identify the emotional state that other
people are in. They suggest that this task tests the first stage of theory of
mind: assigning an appropriate mental state to another person, and argues
that this task is a good measure of social cognition. If this ability is limited,
the individual’s social sensitivity is reduced.
Background

Using their first version of the ‘Reading the Mind in the Eyes’ task, Baron-
Cohen et al. (1997) investigated whether adults with ASD had problems
with theory of mind. They compared a group of individuals with ASD to
individuals who did not have a diagnosis of ASD. The participants were
shown photographs of eyes and asked to identify the emotion that was
being shown from two options (see Figure 3.7).

The ‘Reading the Mind in the Eyes’ task (Eyes test) was believed to use the
cognitive processes associated with a theory of mind as participants
needed to empathise with the person in the photograph to work out their
emotional state.

The results of this study (using the first version of the task) by Baron-Cohen
et al. (1997) showed that ASD participants correctly identified fewer
emotions than participants in the non-ASD group. However, the
researchers identified several practical issues with the original task, which
were solved in the revised version (see Table 3.3).

KEY WORDS

• autism spectrum disorder (ASD): a diagnostic category

(previously including autism and Asperger syndrome). Symptoms,
appearing in childhood, present a range of difficulties with social interaction
and communication and restricted, repetitive, or inflexible behaviours or
interests.
• theory of mind: a cognitive ability that enables one person to
comprehend that other people have separate feelings, beliefs, knowledge,
and desires that can be different from their own. It enables one person to
detect the emotional state of another person.

Problems with the original Eyes Solutions in the revised Eyes

test test
The questions were forced choice, The number of options choices for
with only two options which were responding was increased to four,
always opposite in meaning and they were not opposites.
(semantic opposites), e.g.
Sympathetic or Unsympathetic, so
the task was too easy.
Only 25 sets of eyes were used in 36 sets of eyes were used in the
the test (so many in the ASD group final analysis of the revised Eyes
scored 24 or 25, causing a ceiling test results.
effect).
The eyes illustrated both basic Only eyes expressing complex
emotions and complex emotions emotions were used.
and the former were too easy.
The emotion in some photos could These sets of eyes were deleted
be solved by checking the direction from the test.
in which the eyes were gazing, e.g.
ignoring.
Imbalance of male and female Equal number of male and female
faces faces
Participants might not have Participants were given a glossary
understood the words in the task

Table 3.3: Differences between the original ‘Reading the Mind in the Eyes’
task and the revised version of the test

KEY WORD

• ceiling effect: this occurs when a test is too easy and all
participants in a condition achieve a very high score. This is problematic as
it does not allow the researcher to differentiate between results.
KEY WORDS

• basic emotions: feelings such as happy, sad, angry, afraid,

and disgust. They are understood worldwide, and by very young children,
and can be recognised without the need to attribute a belief to the person.
 • complex emotions: require an understanding of someone
else’s cognitive state, that is, the attribution of a belief or intention to the
person. They are therefore harder to identify.

Aim

The main aim of this research was to investigate whether an improved

‘revised’ version of the Eyes test would show a clear impairment in a group
of adults with ASD to assess its effectiveness.

The researchers also wanted to test whether there was an association

between performance on the revised Eyes test and measures of traits of
ASD, and to investigate whether there were sex differences in those
without ASD on this task.

They were testing five hypotheses:

• Participants with ASD will score significantly lower scores on

the revised Eyes test than the control group.
• Participants with ASD will score significantly higher on the
Autism Spectrum Quotient Test (AQ) measure than the control group.
• Females in the ‘normal’ groups (Groups 2 and 3) will score
higher on the Eyes test than males in those groups.
• Males in the ‘normal’ group (Group 3) would score higher on the
AQ measure than females.
• Scores on the AQ and the Eyes test would be negatively
correlated.

KEY WORD

• Autism Spectrum Quotient Test (AQ): a self-report

questionnaire with scores ranging from 0 to 50. A higher score suggests
that the person completing it has more autistic traits.

Method
Research method and design

This was a laboratory experiment as the environment in which the

participants were tested was not comparable to everyday situations in
which we detect the emotions of others.

This is an example of a quasi-experiment. In a true experiment, participants

can be randomly assigned to any of the experimental conditions. This is not
the case in a quasi-experiment, where participants are usually assigned to
a condition based on a given characteristic (e.g. whether they have ASD).

The independent variable was the type of participant in each condition.

There were three control or comparison groups in this study (Groups 2–4)
and the experimental group containing participants with AS or HFA (Group
1). The experimental design was an independent groups design because
comparisons were made between different groups of participants; Group 1,
consisting of participants with ASD, were compared to the control groups 2,
3, and 4 (see Table 3.4).

There were two key measures of the dependent variable. The first was a
score on the revised Eyes test. For those participants in the AS/HFA and
the IQ (Intelligence Quotient)-matched control condition, IQ was also
measured. IQ was assessed using the short WAIS-R, which measures four
aspects of intelligence using tests of: block design, vocabulary, similarities,
and picture completion.

Sample

The study used four groups of participants. These differed in several ways
(see Table 3.4). Group 1 consisted of all males, and groups 2 and 3
included both males and females.

KEY WORDS

• quasi-experiment: quasi means ‘almost’, and refers to the fact

that these experiments often have lots of control over the procedure, but
not over how participants are allocated to conditions.
• IQ (Intelligence Quotient): a measure of intelligence that
produces a score representing a person’s mental ability. The average
range of IQ is between 85 and 115.

Group 1: AS/HFA

The group comprised 15 adult males with AS or HFA with a mean IQ score
of 115 and mean age of 29.7 years. The sample was self-selecting through
adverts in the Autistic Society magazine and support groups and all had
been diagnosed in specialist centres using the Diagnostic and Statistical
Manual (DSM) or International Classification of Disorders (ICD) criteria.

Group 2: Adult comparison group

The group comprised ‘normal’ adults, who did not have a diagnosis of
AS/HFA. They were selected from adult community and education classes
in Exeter (UK) and public library users in Cambridge (UK) with a mean age
of 46.5 years.

Group 3: Student comparison group

The group comprised ‘normal’ students, who did not have a diagnosis of
AS/HFA, from the University of Cambridge with a mean age of 20.8 years.
Cambridge is a highly selective university so these students are not
representative of the general population.

Group 4: IQ Matched group

The group comprised 14 IQ-matched participants with those in the AS/HFA

group with a mean age of 28 years and mean IQ score of 116. These
participants were randomly selected from the general population.

Procedure

Following the changes that Baron-Cohen et al. made to the original Eyes
test, the revised Eyes test was used in this experiment as a measure of
theory of mind. They started with 40 sets of eyes, target words and foil
words chosen by the first two authors, Simon Baron-Cohen and Sally
Wheelwright. The one target word and three foil (alternative) words for
each set of eyes were developed using groups of eight judges. At least five
of the judges had to agree that the target word was the most appropriate
for the eyes. If more than two of the judges selected a foil word instead of
the target word, a new target word, new foils, or both, were generated and
the item was retested until it met the criterion. In addition, when the results
of Groups 2 and 3 were combined after testing, four of the items produced
inconsistent results, that is, these control participants often chose foil
words. These four items were removed from the analysis so the final
results were based on 36 sets of eyes (18 male, 18 female), each with four
choices of emotion for the face of the target (e.g. reflective, aghast,
irritated, impatient).
To begin the Eyes test, each participant read through a glossary of words
used in the experiment to describe the emotions to ensure they knew the
meaning of each word (Figure 3.8) and were told they could refer back to
the glossary whenever they needed to. Each participant was then shown a
practice item followed by the sets of eyes. For each set of eyes, they had to
pick the correct word for the emotion in the photograph from a set of four
possibilities.

The Eyes test for all participants, and a gender recognition test for Group 1
given as a control task, were completed with a researcher in a quiet room
in Exeter or Cambridge. A pilot study had shown that ‘normal’ adults
usually achieved 100% on gender recognition, so the control groups were
not tested on this. The purpose of this control test was to show that
participants in Group 1 were able to identify characteristics of the eyes
used in the test that were not dependent on having a theory of mind.
Participants in all conditions except the ‘normal’ adult comparison (Group
2) were also asked to complete the AQ test (a questionnaire) at home and
returned it by post.

All participants consented to take part in the study and were aware of the
nature of the research. The data was also made anonymous so that it was
not possible to identify any individual from their scores.

FLUSTERED

• confused, nervous and upset

• Sarah felt a bit flustered when she realised how late she was
for the meeting and that she had forgotten an important document.

RELAXED

• taking it easy, calm, carefree

• On holiday, Pam felt happy and relaxed.
Figure 3.8: Two items from the glossary used in the study by Baron-Cohen
et al.

RESEARCH METHODS

A pilot study is used to check the validity and reliability of the procedures. It
is not a check of whether the study is going to ‘work’ or whether it is ethical,
although they can be conducted to find out whether a question is worth
investigating. When you have finished reading the procedure for this study,
decide why the pilot study was important.

Experimental comparisons were used to test each of the first four

hypotheses. In addition, the results were used in two tests of correlational
relationships. One was between AQ and Eyes test scores and the other
between the IQ and Eyes test scores. These scores were the ‘measured
variables’ or ‘co-variables’.

KEY WORDS

• foil: a ‘foil’ is something that is used as a contrast to something

else. In Baron-Cohen et al.’s study the ‘foil words’ were the (incorrect)
alternative words participants could choose to describe the eyes.

KEY WORDS

• Diagnostic and Statistical Manual (DSM): published by the

American Psychiatric Association, it is used as a classification and
diagnostic tool by doctors, psychiatrists, and psychologists across the
globe.
• International Classification of Disorders (ICD): published by
the World Health Organization (WHO) and although similar to the DSM, it
has a wider scope and covers all health-related conditions, not only mental
health and psychological conditions.
ACTIVITY 3.3

Find two photographs of unknown people, i.e., not people who are friends,
live locally, or are famous. Choose two that have different facial
expressions and cut the image down so that it just shows the eyes. For
each of your two photographs, choose four different words that could be
used, one for the correct description of the expression and three foil words.

Why did you choose the words you did? Are you certain they are not
semantic opposites? Do you think any of the words you have chosen as
foils are too close to the real expression, meaning they could also be a
plausible answer?

If you can discuss these questions and your answers with someone else,
do so. However, remember that these are just your images of eyes and
sets of words, they are not from the Eyes test and do not indicate any
ability or otherwise in anybody that responds to them. This is a very
important ethical consideration.

Results

On the Eyes test, participants with AS/HFA (Group 1) correctly identified

significantly fewer target words than participants in the three comparison
groups (Table 3.5, Figure 3.9). No participant in any of the conditions
checked the glossary for more than two definitions during the experiment.
In the Adult comparison and Student comparison groups (Groups 2 and 3),
sex differences were apparent between males and females on the Eyes
test but this was not significant. All participants in the AS/HFA condition
scored 33 or above out of 36 on the gender recognition task.

On the AQ task, participants with AS/HFA scored significantly higher than

the Student comparison and IQ-matched comparison groups (see Figure
3.10). There was also a smaller, but significant difference between male
and female AQ scores in the Student comparison group.
Figure 3.9: Average number of words correctly identified on the Eyes test

Conclusion

The results indicate that the participants in the AS/HFA group had a
specific deficit in a cognitive process that should help them to identify
emotions in other individuals, i.e. that contribute to a theory of mind. This
conclusion is validated by the AS/HFA group being able to identify the
gender of the eyes accurately. This shows that the Eyes test was able to
detect a subtle and specific impairment in the otherwise intelligent
participants in the HFA/AS group.

There was evidence of a sex difference between males and females in the
comparison groups; although less marked, the pattern of results for males
was similar to the participants with ASD. Males had lower scores on the
Eyes test and showed greater levels of autistic traits on the AQ test than
females. However, some of these differences were not significant so further
research is needed to clarify whether there are differences between the
sexes in terms of their AQ or ability to attribute emotions to others.

This study showed that the revised Eyes test is a more sensitive measure
of adult social intelligence than that used in previous studies, so will allow
future research to discriminate individual differences in a more meaningful
way.

Strengths and weaknesses

The main method was a laboratory experiment as all participants

completed the task in a standardized way, through administration of the
Eyes test by a researcher. This allowed confounding variables to be
controlled, for example reducing the risk of distractions by being in a quiet
room and ensuring all participants had read the glossary before starting.
This improves the internal validity of the experiment and allows the
research to be repeated to check the reliability of the results. The validity of
the Eyes test has improved compared to previous versions due to the
changes made which have led to normal performance being below the
ceiling of the test.
As this is a quasi-experiment, it was not possible for Baron-Cohen et al. to
randomly allocate participants to the conditions. This could introduce a
confounding variable, as there could be a factor other than ASD, which
happens to be different between the ASD and control groups, which is
causing the difference in the scores between the conditions. This study
partly resolved this issue by having two different control groups so that
some participants were matched on IQ to make the groups as similar as
possible.

Although the Eyes test was vastly improved for this research, there are still
several issues that affect the ecological validity of the task. In an everyday
situation, a person’s eyes would not be static or be shown in isolation from
the rest of their face. Consequently, any attempt to apply the results from
this research to an everyday situation will be flawed. Future research might
choose to use videos of eyes rather than images, and whole faces rather
than just eyes, to improve the validity of any conclusions.

The experimental sample in this research (the AS/HFA participants) was

only 15 participants so they may not be representative of all individuals with
a diagnosis of AS/HFA. Caution is therefore needed if generalisations are
made from the findings.

Ethical issues

As all participants were able to give informed consent and their data was
kept confidential, few ethical issues were raised within the study itself.
However, ethical issues could arise from the findings. Evidence such as
this, which identifies ‘normal’ performance of control groups and ‘impaired’
performance of the AS/HFA group could be seen as representing
neurodiverse groups in a negative way. Alternatively, by identifying such
differences, this research has the capacity to provide both an
understanding of the nature of the experience of people with ASD and,
therefore, the potential for greater understanding in society and potentially
in situations such as school or the workplace. This would be an ethical
strength.

ACTIVITY 3.4
Can we be sure it is really all in the eyes? What else could be important
when we are interpreting emotions? To answer this question, think about
whether there are any issues with using photographs of eyes rather than a
real person in this study. How could this experiment be conducted in a
more ecologically valid way? What impact would it have?

Summary

Baron-Cohen et al.’s study investigated how a lack of a ‘theory of mind’ in

adults with autism spectrum disorder (ASD) relates to problems with
recognising emotions. The ASD group’s scores on the Eyes test were lower
than those of the control group even though their IQ was no different. There
was also a negative correlation between the Eyes test score and Autistic
Spectrum Quotient. The test itself was valid because the still images of the
eyes were standardised in terms of size and colour (all black and white)
although this could also be a weakness as emotions are usually detected
on live faces, which move. The findings suggest that the revised Eyes test
is better at detecting individual differences in social sensitivity than the
previous version, i.e. it is more valid.

Questions

5. Choose one possible emotion that Baron-Cohen et al. could

have used.
• a. State the emotion you have chosen.
• b. Write a glossary entry for this emotion.
• c. List some possible foils that could be used with this emotion
and justify why they are suitable.
6. If Baron-Cohen et al. had found that there was still a ceiling
effect, what could have been done to solve this problem?

Individuals who had a diagnosis of AS/HFA performed significantly

worse on the Eyes test than ‘normal’ individuals, suggesting that the ability
to read emotions in the eyes is an individual skill that is developed, rather
than being the result of the external environment. The environment was
standardised across those participants in both the AS/HFA and ‘normal’
conditions, providing further support for the individual explanation.

Children as participants

Baron-Cohen et al.’s research does not focus on children as participants,

so it might be useful to consider how similar research could be conducted
on younger individuals. Baron-Cohen et al.’s research was conducted on
adult male participants using pictures of adult eyes and relatively
sophisticated words to describe the emotion displayed. This would not be
appropriate for children and alternative methods should be considered to
investigate similar aims in children. Baron-Cohen has devised a ‘Theory of
Mind’ test for children called the ‘Sally and Anne’ test that involves asking
children questions following a short scene that is acted out with dolls.