Developing and Evaluating A BCI Video Game For Neurofeedback Training: The Case of Autism

Multimedia Tools and Applications (2019) 78:13675–13712
https://doi.org/10.1007/s11042-018-6916-2
Developing and evaluating a BCI video game

for neurofeedback training: the case of autism
Jose Mercado 1 & Ismael Espinosa-Curiel 2 & Lizbeth Escobedo 3 & Monica Tentori 1
Received: 10 May 2018 / Revised: 3 October 2018 / Accepted: 19 November 2018 /

Published online: 29 November 2018
# Springer Science+Business Media, LLC, part of Springer Nature 2018
Abstract
BCI video games are making brain training increasingly popular and available; yet scientific
evidence to support its efficacy is lacking. Real-life descriptions of BCI video games deploy-
ments in concrete scenarios are urgently needed. In this paper, we report a use case of the
development and pilot-testing of a BCI video game designed to support children with autism
when attending to Neurofeedback training sessions, called FarmerKeeper. Caring for children
with autism may impose new cognitive, motor, behavioral, and attention challenges that
current solutions targeted for other populations may not address. The goal of the game is to
maintain children’s attention above a threshold to control a runner who is seeking for lost farm
animals. FarmerKeeper uses a consumer-grade BCI headset to read user’s attention. We
evaluated FarmerKeeper’s usability and user experience through a 4-weeks deployment study
with 12 children with autism. Our quantitative results show FarmerKeeper outperforms a
commercial BCI video game used for neurofeedback training, and qualitatively, FarmerKeeper
could successfully support children with autism when attending to neurofeedback training
sessions by possibly improving their attention and reducing their anxiety. We close reflecting
on our design aspects and discussing directions for future work.
Keywords Autism . Brain-computer Interface . Neurofeedback . Video game . BCI . Attention
1 Introduction
BBrain training^ is paramount for age-appropriate development. Brain training involves the
exercising of cognitive function through regular completion of activities and tests that chal-
lenge cognition [76] –like playing Sudoku. Computer programs to improve brain performance
* Jose Mercado
jmercado@cicese.edu.mx
1
Department of Computer Science, CICESE, Ensenada, B.C., Mexico
2
CICESE-UT3, Tepic, Nayarit, Mexico
3
School of Engineering, CETYS Universidad, Tijuana, B.C., Mexico
13676 Multimedia Tools and Applications (2019) 78:13675–13712
are a booming multimillion-industry making brain training increasingly popular and available
[1]. For example Lumosity™ [29] is a web-platform consisting of a suite of games with
challenges related to memory, attention, flexibility, speed of processing and problem solving
[87]. Indeed, computerized brain training approaches have shown promising results in im-
proving attention [26] and are appropriate for cognitive empowerment [52].
A popular type of brain training is Neurofeedback [68]. Neurofeedback demands from a
user to train their brain activity by maintaining their brainwave patterns near to a Bthreshold^
defined by a trainer [18]. Brain activity generates minimal electric signals that can be measured
and recorded through an EEG (Electroencephalography) using electrodes placed on the users’
scalp or on another area on the head like the earlobe(s), the mastoid, the chin or the nose tip
[66]. These electrodes are connected to an EEG amplifier. The EEG data can be read in real-
time and it is presented to users through visual and/or auditory stimuli [28]. For example, users
may consult a bar graph that goes either up or down depending if their brainwave patterns are
either above or below a threshold. The goal of the training is to ask users to keep the bars
within a threshold set by the trainer. Users must learn to control and regulate their brain activity
to keep the bars as close as possible to the defined threshold. A neurofeedback training session
usually lasts between 20 and 30 min, after the electrodes are being placed [28]. If the session
lasts more than 30 min, symptoms of tiredness or a headache may occur. To avoid these
symptoms, especially for new users, it is advisable to subdivide the session in activity blocks
of 3 to10 min each, and between each block of activity, enable for small breaks of 1 to 3 min
[28]. Research has shown that within the first five to ten neurofeedback training sessions, and
depending on the severity of the problem, users start noticing initial improvements mostly
related to cognitive flexibility and control [28]. These cognitive improvements persist in long-
term even after the training is suspended; in contrast, to pharmacological treatments that
symptoms may reappear when suspended [45].
Despite the advantages offered by neurofeedback training deploying such a solution in real-
life is not an easy task. Potential trainees face numerous challenges that limit their ability to
deploy such novel training methods. First, neurofeedback training is very expensive and
typically involves high cost and complex systems using Brain-Computer Interfaces (BCI) –a
communication system that monitors brain activity and translates certain characteristics into
commands that operate a device or a visual interface. Second, it requires for 24/7 support by a
specialist with both clinical and technical knowledge [28]. Third, users frequently found the
visualizations of the brainwave patterns too abstract and confusing [48].
The evolution of the technology to a more accessible and manageable consumer-grade BCI
headsets (i.e., devices with electrodes, an embedded analog to digital converter, and a
communication protocol able to send information from the headset to a computer) like Epoc,
Mindwave, and Muse [51] has reduced the gap between cost and accessibility [74]. As a
consequence research in HCI has explored the use of BCI video games1 (i.e., video games
controlled by a BCI), to uncover a more comprehensible feedback about the brain activity that
aim to keep users focused during a neurofeedback training session; yet scientific evidence to
support its efficacy is lacking. Modest efforts have been reported in some studies with
neurotypical children or children with Attention Deficit and Hyperactivity Disorder (ADHD)
1
Or BCI Serious game. The domain boundaries of the definition of Serious Games is still subject to debate [64,
83]; and although, the concept was originally proposed to support education [2], research has been exploring the
use of serious games in other contexts and fields [64] including health-care [23, 62], training [57] and military
[43].
Multimedia Tools and Applications (2019) 78:13675–13712 13677
[39, 48]; however, the widely held belief that consumer-grade headsets and BCI video games
improve attention in the wider population lacks empirical support [54]. Such real-life descrip-
tions of BCI video games deployments are urgently needed.
Beyond the studied populations, an interesting scenario to study the challenges associated
to the use of BCI video games for Neurofeedback training is the care of children with autism.
Autism is a neurodevelopmental disorder affecting 1 in 59 children [6]. It is associated with
persistent deficits in social communication and social interaction, language, and restricted
and repetitive behaviors patterns, interests, or activities [4]. The severity level of autism is
based on the amount of support individuals with autism need when communicating with
others and when conducting their activities. An individual with severe autism would require
very substantial or substantial support [53]. For example, a child with severe autism is not
verbal, has no social interaction, exhibits stereotyped movements, and may be hypersensitive
to light, sound and touch. In contrast, high-functioning children with autism, are usually,
highly verbal, may move independently; but, their behaviors and social interactions may
appear Batypical^ –for example, they would use an odd tone of voice and may appear to be
rude and invading the personal space when talking to others. Attention Deficit/Hyperactivity
Disorder (ADHD) is a common neurodevelopmental disorder associated with autism, as
studies indicate that 50–70% of youth with autism have comorbid ADHD [5]. Due such
common comorbid of ADHD and autism, individuals with autism, especially children,
frequently exhibit poor and lack of attention, lack of engagement, and impatience when
taking turns [25, 37]. Attention problems could be of paramount importance for children with
autism. As attention abilities are critical skills for learning and age-appropriate development
[50], attention problems may interfere with their cognitive efficacy, socialization, health,
safety, and quality of life [33].
Current research had made the compelling argument that BCI video games for
neurofeedback training improve the attention problems of neurotypical children and children
with ADHD [12]. But, the needs of children with autism may differ and include more
behavioral and cognitive challenges that could make more challenging to explore the feasi-
bility and applicability of BCI video games for neurofeedback training in this concrete
scenario. First, the stereotyped movements of children with autism may increase the noise
being read with the BCI. Second, due to their hypersensitivity issues, they may not tolerate to
wear a headset and the electrodes. Finally, due their attention problems, they need constant
guidance from trainers to redirect their attention before they disengage from the training. These
cognitive, motor, behavioral, and attention challenges of children with autism may reduce their
opportunities to use some of the commercial solutions for neurofeedback training. As a
consequence, there is limited work exploring if BCI video games are appropriate for children
with autism. Open questions about what are the appropriate visualizations, how much guid-
ance children with autism need, and what is the appropriate equipment for this population; still
remain.
In this work, we present a first attempt in the direction to increase our understanding of the
usability and user experience of BCI video games to be used as a neurofeedback training
solution for children with autism. Taking into account our previous understanding of the
design space of BCI video games we focus on exploring their impact on attention. Although,
by doing so we are not addressing the core symptoms of autism; but a potential consequence of
their condition, including their impairments in attention; it could serve as a vehicle to explore
remaining open questions in this domain and deepen our understanding of the design space of
BCI video games in a concrete scenario.
The main contributions of this work are
& A prototype of a BCI video game, called FarmerKeeper, for neurofeedback training.
& Qualitative and quantitative evidence showing in which situations FarmerKeeper outper-
forms a commercial solution to help children with autism practice neurofeedback training.
& Reflections from our co-design process and a set of design aspects that could guide the
design of BCI video games for neurofeedback training.
2 Related work
In this section, we first describe research related to demonstrate how traditional brain training
has positive impact in cognitive functioning. Then, we describe how traditional neurofeedback
training has been gamified and its consequences. Finally, we describe research using BCI
video games for neurofeedback training, especially those using consumer-grade BCI headsets.
2.1 Traditional brain training: Neurofeedback
In 1976, Lubar et al. [46] showed that for humans it is possible to recondition and retrain the
brain activity with neurofeedback training. Based on their findings, neurofeedback training
began to be explored to improve cognitive functions on healthy individuals [38, 79, 88] and
individuals with some neurodevelopmental condition (e.g., Attention Deficit Disorder, ADHD,
learning disabilities, stroke, epilepsy, cognitive dysfunction, depression, anxiety, and others)
[28, 40, 56].
For example, on research to improve cognitive functions on healthy individuals, Vernon,
et.al. conducted a study with 30 undergraduate medical students (20–28 years old) to evaluated
the effect of distinct neurofeedback trianing protocols on aspects of cognitive performance.
Participants completed eight neurofeedback training sessions over a 4-week period. During the
neurofeedback training session, the participant’s task was to increase the size of a bar
(representing a brainwave pattern) and simultaneously decrease the size of others bars
(representing inhibitory brainwave pattern), when they met this goal a tone was played and
a symbol appeared to indicate a point scores, with the aim to score as many points as possible.
The results of this study shows a positive improvement on cognitive processing in the
participants. On the other hand, on research to improve cognitive functions on individuals
with some neurodevelopmental condition, in the study conducted by Pineda et al. [56],
children with autism control a muted video of a landscape when their brainwave patterns are
near to a predefined threshold. Similarly, Kouijzer et al. [40] studied the effects of
neurofeedback training in reducing the symptoms of children with autism. To do so, children
with autism attended a traditional neurofeedback training. The amplitude of a particular
brainwave pattern is used to control a bar graph. While observing the amplitude of their brain
waves, the children were instructed to Btry to move down (or up) their brain activity using the
feedback to guide you^. A criterion line was shown together with the bar graph reflecting the
goal during training. Visual and audio rewards fitting the participant’s age and interest were
provided when the participant met the criteria set by the criterion line.
This body of work demonstrates that neurofeedback training improves cognitive functions
on healthy individuals and individuals with some neurodevelopmental condition. However,
neurofeedback training is very expensive, since these training typically involved high cost and
complex BCI systems (not easy to use for most children with autism), additionally, they
required the neurofeedback trainer has technical savvy [28].
2.2 Gamifying the neurofeedback training
Due to the repetitive aspects and abstract visualizations of neurofeedback training [32],
researchers began to incorporate some playful applications (i.e., mini-games) that provide
more fun and engagement than standard neurofeedback training approaches. For example,
instead of control the sizes of bars or waiting for an auditory signal, the children had to
complete tasks such as make a diver sink to reach a treasure [17], make an emoticon smile [8],
make a monkey climbs a tree for food [8], make a ball blink [8] or make a dolphin dive to the
ocean ground to collect coins [71]. The main characteristic of the applications or mini-games
using neurofeedback training is that players must discover how to self-regulate their brain
activity and maintain it in a particular cognitive state (e.g., attention, relax, etc.).
Since these new playful applications began to be successful to provide a better engagement
for children than standard neurofeedback training approaches, and the fact children often do
not like to sit still and focus on training activities, researchers started to explorer and introduce
more engaging games, combine neurofeedback training with BCI video games [84]. For
example, Pope and Palsson [58] used a neurofeedback training system developed by the
NASA to modulate the controller input of Sony Playstation™ video games (i.e., correct brain-
wave patterns were rewarded with a more responsive controller). To evaluate their proposal,
they conducted a study with children with ADHD. The children were divided into two groups,
the first group used traditional neurofeedback training visualizations, and the second group
played Sony Playstation™ video games with their proposed controller. In this study, the
second group showed a 65% of improvement in ADHD symptoms while the first group only
improved 48%. As a result, this work shows that video games positively influences a child’s
motivation to conduct a neurofeedback training.
Although this kind of gamification address a more fun and engagement visualizations than
standard neurofeedback training, questions about the design aspects and guidance to promote
the self-regulation of the brainwave patterns are remaining; mainly for children with
neurodevelopmental disorders, like autism.
2.3 BCI video games for neurofeedback training
Recent work in HCI has explored the use of BCI video games to give the guidance needed to
promote the self-regulation of their brainwave patterns, for both neurotypical people and with
neurodevelopmental disabilities. These efforts work to uncover more comprehensible feedback
[47, 48] and to support the user to be focused during neurofeedback training [44, 58, 80].
Some research projects have proposed to modify commercial video games to turn them into
BCI video games for neurofeedback training. For example, Mandryk et al. [47] developed an
add-on turning commercial video games into BCI video games for neurofeedback training.
Their approach uses visual feedback via texture-based graphical varying in their obfuscation,
and that is overlaid on an underlying game. The intensity of the obfuscation increases as the
users’ attention moves away from a threshold defined by the neurofeedback trainer. This add-
on uses the MindWave headset from NeuroSky. An evaluation of the effectiveness of the add-
on with 16 children with Fetal Alcohol Spectrum Disorder (FASD) shows it could reduce the
symptoms of ADHD [48]. However, the use of texture-based graphical overlays could be an
abstract representation of brain activity for children with other neurodevelopmental disorders
like autism.
Others have been focused in developing BCI video games from scratch and evaluate their
impact on the neurofeedback training, Wang et al. [80] proposed to design and develop aspects
to create BCI video games for medical applications. They developed 2 BCI video games,
Dancing Robot and Brain Chi [80]. Both BCI video games use the Epoc headset from Emotiv
to obtain the brain activity and calculate the user’s attention to control the BCI video game.
However, on their work only prove the viability of using the consumer-grade BCI headset to
control the BCI video game and compared some features extraction and classification methods
to calculate the user’s attention. Schoneveld et al. [65] evaluated MindLight (a previously BCI
video games development on their Lab at Playnice Institute) on reduction in child anxiety. In
MindLight children use a one-channel, dry-sensor EEG headset to control a light that glows
from the avatar’s head in the game. The more relaxed players become, the brighter the
Bmindlight^ shines; the light is the only way that players can see in a dark haunted house.
When players become anxious, the light dims and they are forced to regain their calm to see
again; thus through neurofeedback reinforcement mechanics, the child is trained to identify
and shift his mind states. On this study, 136 children in elementary schools (8–13 years old),
randomly assigned to play Mindlight or a control game, participated on 5 sessions of 1 h each,
schedule twice a week. Results show an overall significant reduction in child anxiety for booth
groups. Lim, C.G. et al. [15] describes a 10-weeks pilot study to evaluate if the use of a cars’
race BCI video game could improve inattentive symptoms of ADHD. Parents and teachers
evaluated rating inattentive score, improved more in the intervention group. Later, Lim, C.G.
et al. developed CogoLand [44], a more intensive BCI video game where the child needs to
concentrate to move an avatar and run around an island in the shortest time possible.
CogoLand monitored attention through the Mindwave headset with dry EEG sensors. Authors
found significant improvements on inattentive and hyperactive-impulsive symptoms on the
ADHD rating scale. But questions remain as to study the design considerations, such as the
appropriate visual and auditory stimulus and how to keep engagement of the children with
autism throughout several the neurofeedback training sessions.
Although this body of work shows neurofeedback training has positive impact for brain
training, and that BCI video games can keep children focused during neurofeedback training
and give them more appropriate guidance to help them with the self-regulation of their
brainwave patterns; to our knowledge, no research projects have studied needs and the design
space of BCI video games to support neurofeedback training of children with autism.
3 Designing FarmerKeeper
In this section, we describe the user-centered research process we followed to design

FarmerKeeper (Table 1). FarmerKeeper is a BCI video game to support neurofeedback
training that is controlled by a consumer-grade BCI headset –it was designed taking into
consideration the needs of children with autism.
3.1 Research process
Our research process involved a qualitative study including interviews and observation to
uncover the needs of our potential users. We supplemented our understanding from our
Table 1 Data collected and activities conducted to design FarmerKeeper
# Data Purpose Participants
1 Neurofeedback training To understand how neurofeedback 1 HCI expert

training works in practice, the first 1 neurofeedback expert
author of this paper took a certification
with 36 h of theoretical background
and 6 h of practice with patients
4 Semi-structured interviews We conducted interviews with different 1 neurofeedback expert
(t = 170 min, m = 42.5 min, stakeholders of the process to 1 child psychologist
sd = 12.58 min) understand current challenges faced 1 serious games expert
and strategies used when treating 1 graphic designer
attention impairments with children
with autism
3 Sessions of Passive Observation We observed several neurofeedback 1 neurofeedback expert
(t = 180 min, m = 30 min training sessions to understand 2 trainees
sd = 0 min) challenges faced and strategies used. 1 child psychologist
We used the think-aloud method in one 2 HCI experts
observed session to better uncover the
process
5 Participatory Design Sessions We conducted several design sessions 4 HCI experts
(t = 11 h, m = 2 h 12 min, with a interdisciplinary team of 1 child psychologist
sd = 1 h 6 min) specialists in autism, design, 1 graphic designer
neurofeedback and HCI to propose 4 1 neurofeedback trainer
low-fidelity prototypes of BCI video
games
2 Co-design Sessions (t = 2 h 29 min, We conducted several co-design sessions 2 HCI experts
m = 1 h 14 min, sd = 11 min 10s) involving specialists and potential users 16/5 teachers (child
to select from the alternatives proto- psychologist)
types proposed and specify the selected
prototype
qualitative study and the literature with several design sessions to propose different prototypes
of BCI video games. We audio-recorded and transcribed all the interviews. We took notes of
interesting aspects that took place during our observations.
We used qualitative techniques including open and axial coding [72] to score potential
emergent themes from the collected data (including our interviews and observations). We then
group the uncovered emergent themes in an affinity diagram [10]. Three HCI experts made the
qualitative analysis, comparing and discussing the emergent themes. To analyze these data we
used atlas.ti [22] –a software to support qualitative research. We materialized our data
collected from our participatory design sessions and co-design sessions in sketches, story-
boards, and new ideas for potential features for the system (e.g., activities, stimuli). We
analyzed the data collected during our design sessions using techniques from rapid contextual
design [34].
During the design sessions and taking into consideration our common understand-
ing of the process and strategies used when conducting a neurofeedback training
session, our design team proposed four low-fidelity prototypes as potential alternatives
of BCI video games. From the four low-fidelity prototypes, our design team selected
the prototype of a BCI video game using a runner being controlled based on the
user’s attention being read with a headset. This prototype, that it was called
FarmerKeeper, uses a Farm as its main topic. The goal of the game is to help the
runner to collect animals scattered throught the road and redirect them to their pens.
Our design team chose the topic of a farm based on their experience working with
children with autism –but, specialists agreed that for future versions this topic could
be personalized to each child’s interests and needs.
To discuss possible design improvements and evaluate the design of the proposed
prototype, we conducted two design sessions with the potential users of the system.
Several teachers from Pasitos –a school-clinic located in Tijuana, Mexico where 18
psychologists-teachers care for close to 60 children with autism, attended to this
design session. This design session was conducted at Pasitos and teachers, in collab-
oration with specialists, had the chance to co-design the levels of the game, prompts
and rewards to be included in the game. After several improvements to the prototype,
the teachers answered the SUS questionnaire [14] to fill their perceptions about the
perceived usability of the last version of the designed prototype. As a result of the
SUS questionnaire (avg. score 78.13 on a scale of 0–100), and according to the
acceptability ranges and adjective ratings of SUS, participants perceived the design of
FarmerKeeper as good and acceptable.
3.2 Neurofeedback training in autism – A prototypical lesson
As a result from our qualitative study, we uncover the phases of a neurofeedback session (Fig. 1).
Such phases were used as a basis to design the levels of the game and were discussed on
each design session. On a traditional neurofeedback session the trainer defines a Bthresh-
old^ of attention based on trainees’ capabilities and needs, the number of activity blocks,
breaks and duration. The trainer usually uses a computer-based application to set-up this
information. This application is available in the equipment kit clinics buy to conduct
neurofeedback training. The trainee must use a cap or a hat of electrodes or a BCI headset
to read their brainwaves signal.
First, the trainer must set-up the training session by specifying the threshold and placing the
electrodes on the head of the trainee. After calibration, the trainer usually plays some music to
help trainees relax and slowly start to figure out how to control their brainwave signals. Then,
the trainer must actively complete several blocks of activities, with breaks in between. During
an activity block the trainee must maintain their brainwave signals as close as possible to the
defined threshold by adjusting the visual and audible stimuli available in the visualization app
from the neurofeedback equipment kit. During breaks, trainers can remove the electrodes to
reduce trainees’ anxiety. Visualizations are usually in form of graphs or games.
To exemplify how a neurofeedback training session is conducted with children with autism,
the problems faced and strategies used, here we present an example scenario:
Mate is a 6-year-old child with severe autism who exhibits attention problems and who
is completing a neurofeedback training course –twice a week for two months. Ms. Patty,
(Wearing the electrodes or a BCI headset)
Activity
Setting up Relax phase Break phase End
phase
Fig. 1 Diagram of phases on a neurofeedback training session

the neurofeedback trainer, configures on the neurofeedback training application three

blocks of activity of 3 minutes with one 1-minute break on each block. Also, Ms. Patty
selects the threshold of 40 attention units; as it is the low boundary of the normal
attention range (40-60) from the attention scale of 0 to 100 on the application. Next, Ms.
Patty tells Mate she is going to place the electrodes in Mate’s head, but Mate is a little
anxious, and he’s moving a lot. Ms. Patty asks Mate to settle down. Ms. Patty plays on
the computer a video of a landscape with relaxing music. Mate watches the video and
settles down. While Mate is watching the video, Ms. Patty places the electrodes in
Mate’s head and starts the first activity (Fig. 1, Setting up).
The activity phase consists on reaching a target line by moving up a bar graph –the
height of the bar changes based on child’s read attention (more attention will result in a
taller bar graph). Also, an instrumental music is being played in the background –
music’s volume is also controlled with child’s attention. During the first minute, Mate’s
attention is above the threshold and the bar graph is above the target line. Mate can also
hear the instrumental music. When Mate’s attention decreases, the bar graph decreases
below the target line and Mate can’t no longer hear the sounds. Mate gets anxious and
starts flapping his arms and rocking his body touching the electrodes; like if he would
like to take them off (Fig. 1, Setting up). Ms. Patty starts a break, as too much noise
makes impossible to read Mate’s attention. When Mate settles down (Fig. 1, Break), Ms.
Patty starts the activity again. But Mate can’t complete the activity as he is too anxious
and unable to control his attention. Ms. Patty takes off the electrodes and congratulates
Mate for his hard work. Ms. Patty invites Mate to come back for another session (Fig. 1,
End).
This scenario shows how children loose motivation and interest during a neurofeedback
training session, resulting in a poor performance without completing the training. We
also highlight the workload of neurofeedback trainers demanding from them to both
redirect trainees’ attention while adjusting the threshold and monitoring the brainwave
signals.
3.3 Design implications for a BCI video game to support neurofeedback training
for children with autism
Based on our literature review and research process, we found three design implications and
eight design aspects for developing a BCI video game for neurofeedback training for children
with autism.
3.3.1 Catalyze engagement
During neurofeedback training, children must discover how to self-regulate their brain activity
and maintain it in a particular cognitive state (e.g., attention, relax, etc.) [74]. Self-regulation is
a difficult and repetitive process; as a consequence, children easily lose motivation and interest.
We found out that using a story, providing rewards based on achievements and enabling an
emotional connection with the character of the game could contribute to engagement –this is
consistent with what has been reported in the literature on video games related to the creation
of a storyline, and the use of goals and rewards [81].
& Maintaining the story simple. Our results show that a video game for children with
autism should have a simple and comprehensible story with a clearly defined goal. A
physiologist explained:
B…The story of the game should be something easy to understand, if it is something
simple and comprehensible, the children may stay focused…^ [teacher (child psychol-
ogist) 3]
During our design process, specialists discussed different stories related to superheroes and
sports, with multiple challenges on each level; however, specialists agreed that having
something too complex may be very difficult for children with autism to understand, specially
for youngers. So, specialists decided to define a more general and simple story matching the
concepts children are learning in their classroom. They came up with the story of a farm, as
children with autism love animals and their classroom activities are filled of examples using
animals. After several iterations, specialists sketeched a story where an old farmer asks two
little farmers for help to find the farm animals, like pigs and cows, that had escaped from their
pens due to a storm. To avoid repetition specialists suggested to include different animals and
vary their pen and scenarios –like piggies go into a pig pen and cows play in a pasture. By
doing so, specialists agreed that most children with autism could easily understand the
proposed story and would enjoy the topic.
B…We commonly use images of animals to support other therapies [in autism]. For
example, we use images of farm animals to teach them how to match animals to sounds
or colors [like a cow Bmoos^ and it is white and black]. Most children like them, so a
topic with animals could be easily understood by them. It could also be a general and
enjoyable topic for most of them…^ [teacher (child psychologist)5].
& Providing rewards for short and long-term achievements. Rewards also help to keep
children focused.
B… [you could keep children engaged] by giving away rewards, and constantly
providing them with visual stimuli, like coins, stars or awards, that kind of things …^
[designer]
Specialists then discussed different type of rewards that might be appealing for children. Based
on the literature [27] and to be consistent with our common understanding of rewards for video
games, we decided to give away coins as rewards. So the coins will be scattered throughout the
road and the runner must collect them –like in Mario Bros. However, after seeing the sketch of
the rewards, specialists thought that coins are not appropriate for children with autism, as they
still do not understand the concept of coins. So they proposed to take advantage of the story and
use animals instead. So children must collect animals on the road (Fig. 2). Specialists
also discussed the importance of differentiating between achieving a short versus a long-term
goal. So they proposed to have different kind of animals in the road, like having a muddy pig
after collecting five clean pigs. Subtle changes in the appearance of animals will encourage
children to stay focused for a more sustained period of time. Children could interpret this
change as a reward for little achievements and could serve as a. progress indicator to keep them
focused in reaching a major goal. In long term, after completing the training of the day, children
will get rewards in the form of balloons and fireworks. Also the number of collected animals
could be used as Bmoney^ to buy accessories to personalize their avatar in a store.
Fig. 2 Main changes made to the main goal of the activity phase on the initial prototype of FarmerKeeper.
Collecting coins during the activity phase (left), seeking for animals during the activity phase (center), and
different kind of pigs on the road as a reward for little achievements (right)
& Enabling users to relate with the character of the game. Also, we found out that the
development of characters is important for creating an immersive environment as users can
develop emotional connections with characters (e.g., identifying with the characters and
having empathy for them) [19]. According to the literature [7, 31] and based on the input
from specialists during our design sessions, we found out that the BCI video game should
enable users to customize the sex [6] and the characteristics (like clothes, skin color) of the
character of the game [4]. By doing so, the BCI video game could help users to more easily
relate to the character.
B… [for children with autism] the videogame is more attractive when it has a character.
Having a character will help children with autism to identify with [the game], they may
feel as they are represented in the videogame. [This character empathy] will motivate
them to have a better performance in the videogame…^ [serious games expert].
For children with autism, all teachers agreed the character should be easy to understand, and
that using morphological characteristics [11], like making the animals as much human as
possible. For sex customization, our prototype enables children to select either a male or a
female little farmer. In addition, users can use their earned coins to buy accessories in the store
like hats, t-shirts, pants, skin hair and color, and so on. Although, other researchers have
reported other forms of customization our specialists wanted to keep the game as simple as
possible to avoid confusion.
3.3.2 Mimic current strategies on prompting
Our results also show that specialists use a variety of strategies to help children with
autism stay on task. Strategies involve giving away verbal, visual, and physical
prompts when needed [30]. These prompts could be implemented in the game by
designing appropriate visual and audible stimuli [21]. Specialists agreed that prompts
must be short, specific and mimic common language currently being used with
children with autism.
& Embodying authority figures. We found out that children perceived caregivers as a
person who represents authority and as a consequence their prompts are highly
effective [9]. So specialists agreed that the video game must mimic this authority
figures to give away prompts and instructions. Our specialists decided to design an
old farmer that will represent this authority figure and will mimic the role of
caregivers in real life. It will mimic the language and the prompts being used in
Pasitos.
B… The instructions must be specific, must be short, they can’t be long instructions. For
example: ‘put attention’, ‘don’t move’.^ … [teacher (child psychologist) 4]
The old farmer explains the story to children and appears every time the child loses attention or
is moving his body. For example, when the child loses attention, the old farmer prompts the
child with verbal instructions like BYou can do it^ or BStay focused. Concentrate^; and when
the child is moving, the old farmer appears and highlights children’s movements by saying
BSettle down^ or BOh, oh!, no moving^.
& Combining cross modal correspondences. When designing the stimuli of the game,
specialists also explained that some children attend more to visual stimuli and others to
auditory stimuli; so it will be better if both kinds of stimuli are represented in the game.
B…you have to have something visual to attract children's attention and something
else… not just the visual part, you have to supplement it with sounds…^ [neurofeedback
expert].
However, to ease the sensory integration, the sounds, visuals and activities must have
a correspondence. In many everyday situations, our senses are bombarded by many
different unisensory signals at any given time. BTo gain the most veridical, and least
variable, estimate of environmental stimuli/properties, we need to combine the indi-
vidual noisy unisensory perceptual estimates that refer to the same object, while
keeping those estimates belonging to different objects or events separate^ [70].
Crossmodal correspondence should be personalized for each prompt and their qualities
including the voice being used, the volume, the periodicity and so on.
3.3.3 Promote a good posture
During neurofeedback training, trainers ask children to move as little as possible and
avoid abrupt movements, as moving increases the noise being read by the BCI
headset. However, our specialists explained that children attended this instruction only
at the begging of the session, and as the session progresses children begin to get
bored, tired, or distracted. Distractions heavily reflect on children changing their
posture and increasing their movements.
B… [some of the problems we faced when conducting a neurofeedback session with
children with autism] include having to deal with their movements and their poor
posture, especially with those children having attention deficit disorders. Children need
to be seated, straight in a chair… When [children] begin to be distracted or bored, they
start to move, and do not maintain the posture we need for them to have during the
training …^ [neurofeedback expert]
We found out neurofeedback trainers spend a considerable amount of time redirecting
children’s attention through verbal instructions trying to reduce children’s movements
[51]; instructing children to keep a Bgood^ posture by asking them to settle down,
seat straight and try to move as little as possible. Indeed, in previous studies we have
conducted with children with autism using headsets [16], we have found out that by
correcting their posture they tend to move less. So, to avoid children’s movements
during a neurofeedback training session a BCI video game must increase the aware-
ness of children towards their movements, avoid modeling movements, and enable for
a stress-free environment.
& Increasing awareness of atypical movements. During our design discussions, specialists
agree that children will move. So beyond asking children to stop moving the BCI video
game should actively let children know when they are moving as most children with
autism will not realize they are moving –literature has shown individuals autism lack or
have poor body awareness [86]. The BCI video game should include visual and/or
auditory stimuli as indicators to let the children know when they are moving and prompt
them to face the screen and keep a good posture [3]. During the design process, after
specialists discussed differents solutions to increase awareness of atypical movements,
they agreed that the best solution will be to avoid children to continue playing the game.
So, they suggested to blur the screen of the game to let children completeley focus on the
instructions provided by the old farmer.
& Avoiding modeling movements. In addition, specialists discussed that it is important to
avoid having stimuli that may subtly catalyze movements. Going back to discussing the
story of the game, they agreed that having a game of sports or superheroes will contribute
to children’s movements –as the appeareance of the characters and their behaviour may
model movements and children could try to imitate their movements. Although, it will be
impossible to get rid of all the behaviours that might hint movement, specially for a runner
game; specialists agreed to reduce Bmovements^ behaviours as much as possible.
& Encouraging a stress-free environment to reduce anxiety. During the design process,
our specialists explained children need to be comfortable and relaxed to increase
their chances to have a better performance and reduce their anxiety that could
reflect on more movements. As anxiety is a prevalent symptom in most children
with autism [78]; it ideally, should be managed before the sessions start and
between activity phases.
B…anxiety is a cardinal symptom in children with autism… [if children are anxious]
first, you need to control their anxiety, and then you can start the session or figure it out
how to deal with other behavior problems…^ [neurofeedback expert]
Strategies reported in the literature to help children stay calm involve using relaxing sounds
(e.g., naturalistic sounds) and intermittent pauses to enable children to take a break from the
activity [36]. These breaks should mimic the relax phase currently available in a
neurofeedback training session.
B… [the relax phase] should be quiet and avoid having unexpected objects moving
abruptly to help children to be calm.^ … [teacher (child psychologist) 3].
Our specialists propose to use two different type of Brelaxing^ phases. The first one, should be
at the beginning of the session (Relax Phase) to help children get prepared for the activity.
Specialists envisioned having a short-movie of a landscape with few representative objects of
the farm (e.g., a tractor, a barn, some trees). Such a movie should not have abrupt movements
and the behavior of the animals appearing in the movie should be calmed and avoiding
surprises. The second one, should be used between activity phases as a short break. Specialists
propose to have a short-animation of farm animals grabbing a bite, drinking water or playing
after getting back to their pens. This clip should be related to the activity to ease transitions.
The visual stimuli of the movies should be supplemented with relaxing nature sounds like the
sound of the wind.
4 Developing FarmerKeeper
FarmerKeeper is a BCI video game designed to support neurofeedback training for children
with autism. It uses a consumer-grade BCI headset to read user’s attention and control the
visual stimuli in the game.
4.1 Game mechanics and features
The game story unfolds on a farm. The goal of the game is to maintain children’s attention
above a threshold to control a runner who is seeking for lost farm animals to take them back to
their pens.
At the beginning of the game, an old farmer tells the child that there was a storm that scared
the animals who escaped from their pens and got lost. This old farmer plays the role of the
neurofeedback trainer within the game, giving away visual and verbal instructions to keep the
child motivated and focused, and helping the child to maintain the right posture throughout the
game. The little farmer plays the role of the child in the game. The little farmer drives a truck to
seek for the lost animals and returns them to their pens. The user’s attention read with the BCI
headset, controls the speed of the truck, according to a threshold defined by the neurofeedback
trainer at the beginning of the activity. This threshold can be used to increase the difficulty of
the game according to each child progress through the sessions and overall capabilities. Also,
FarmerKeeper has some scenarios, where the child could seek for different farm animals (i.e.,
pigs, hens, and cows).
FarmerKeeper consists of two main sections: the configuration/customization section and the
neurofeedback training section. In the configuration/customization section, the neurofeedback trainer
may create the child’s profile setting up aspects such as the duration of the activities and the breaks.
The neurofeedback trainer can play the story of the game, and when having money, helps the child to
buy items in the store to personalize the appearance of the truck and the little farmer (Fig. 3).
Fig. 3 FarmerKeeper’s configuration/customization section. Users menu (top-left), User’s configuration (top-
center) and Main Menu (top-right). Story (bottom-left), player selection and customization (bottom-center), and
scenarios’ map (bottom-right)
Also, the neurofeedback trainer selects the scenario from the map to choose the animal the child
will seek.
The neurofeedback training section consists of three phases (Fig. 4):
& The Relax phase (Fig. 4, top-left). This phase shows a video tour of the places of the farm
the child must visit to find the lost animals. It uses an environmental background sound of
the wind to reduce the child’s anxiety. The neurofeedback trainer may use this time to
place the BCI headset on the child’s head.
& The Activity phase (Fig. 4, top-center). In this phase, the little farmer drives a tractor to
search for the lost animals. The player’s attention controls the tractor’s speed. When the
child has good attention (i.e., attention above a threshold set by the neurofeedback trainer),
the little farmer will start to run and collects the lost animals appearing in the center of the
road (Fig. 4, top-center). When the child loses attention (i.e., attention below the threshold
set by the neurofeedback trainer), the little farmer will slow down. The volume of the
background music will also gradually increase or decrease according to the player’s
attention. If the child’s attention is very low the music of the game and the little farmer
will stop completely. When the little farmer stops for a few seconds, he will begin to make
gestures with the goal of regaining the child’s attention. If the child doesn’t recover the
attention, the old farmer will appear giving a short instruction to try to recover the child’s
attention and motivate him to continue playing the game (Fig. 4, bottom-left). Also, during
this phase, if the signal received by the BCI headset is too noisy, due to movements, the
screen becomes blurry. The old farmer will give a short instruction to guide the child to
keep a good posture and to be able to continue in the game (Fig. 4, bottom-right). If the
child stops moving, the screen will return to normal, and he could continue with the
activity. The Activity phase ends when the child reaches a predefined time for the activity
specified by the neurofeedback trainer.
& The Break phase (Fig. 4, top-right). The little farmer arrives at one of the animal pens, and
FarmerKeeper shows an animation of the lost animals returning to their pens. Once back at
their pens, the newly recovered animals are playing, grabbing a bite and drinking water.
When the animals return to their pens, the old farmer gives away to the user coins as a
reward for his help. Later, the child could use these coins in the store. The Break phase
ends when the child reaches a predefined time for the break specified by the neurofeedback
Fig. 4 FarmerKeeper’s neurofeedback training section. Relax (top-left), Activity (top-center) and Break (top-
right) phases. The activity phase during no-attention (bottom-left) and movement (bottom-right)
trainer. At the end of this phase, the neurofeedback trainer could choose between finishing
the session or continue with other block of Activity. When deciding to continue, the trainer
can select another animal to seek.
4.2 Scenario of use
To exemplify how FarmerKeeper is being used in practice, here we revisit our presented
scenario (see section 3.2):
Today, Mate is going to play with FarmerKeeper and Ms. Patty selects his profile from
the Users menu (Fig. 3, top-left). She specifies that the activity should last 3 minutes
with 1-minute breaks in between. Ms. Patty establishes an attention threshold of 40
attention units (Fig. 3, top-center). Ms. Patty selects the green truck (Fig. 3, bottom-
center). Mate asks Ms. Patty for the blue truck; but, she explains to Mate that the blue
truck is blocked and he needs to seek for some animals to win coins and unblock it.
Since Mate is a little anxious and moving a lot, Ms. Patty enters to the Relaxation phase.
Mate and Ms. Patty watch a short-movie of the farm places Mate must visit to seek for
the lost animals (Fig. 4, top-left). Mate settles down. Then Ms. Patty places the
BrainLink headset on Mate’s head and explains to him that he needs the headset to
seek for the animals.
Moving on, when Ms. Patty enters the FarmerKeeper’s map (Fig. 3, bottom-right), Mate
says he would like to seek for pigs. Ms. Patty selects the pig’s scenario and starts the
activity. During the first minute Mate’s attention is above the threshold and the little
farmer starts running with good speed (Fig. 4, top-center). When Mate gets distracted and
his attention decreases below the threshold, the little farmer stops running. The old farmer
shows up and says Bcome on, you can do it^ (Fig. 4, bottom-left). Mate returns his
attention to FarmerKeeper and the little farmer’s running speed increases. When Mate
touches the BrainkLink device the screen of FarmerKeeper gets blurry. The old farmer
shows up again and says Bsettle down^ (Fig. 4, bottom-right). Mate regains concentra-
tion. After 3 minutes of catching pigs while playing with FarmerKeeper Mate takes a 1-
minute break. During this time the old farmer congratulates Mate for all the animals he
has found by thanking him for his help and give him some coins. Mate watches the
animals he found moving, eating and drinking water in their pen (Fig. 4, top-right). Mate
laughs. Mate asks Ms. Patty to seek for cows and successfully finishes the training.
Ms. Patty takes off the BrainLink headset and congratulates Mate for his hard work. Ms.
Patty invites Mate to come back again and reminds him that the next time he will be able
to unblock the blue truck and use it to seek for more animals.
4.3 FarmerKeeper’s architecture
We used the BrainLink headset as it is easy to set up and its internal ThinkGear Chip measures
attention in the scale of 0 to 100 at a rate of 1 Hz, through its proprietary algorithm eSense.
Also the Think-Gear Chip measures the signal quality in the scale of 0 to 255 (the higher the
number, the more noise is detected) at a rate of 1 Hz. In addition, compared to other consumer-
grade headsets (i.e., Epoc, Mindwave, Muse [51]), we found BrainLink to be the more
appropriate to use with children with autism as its simple and its minimalist design could
reduce potential anxiety from wearing bulky head accessories. However, we developed
FarmerKeeper to be flexible enough to work with others BCI headsets.
We used the values from the eSense algorithm as an input of users’ attention [24, 42, 55, 61,
85] to control the BCI video game. When the read value is greater than a threshold defined by
the neurofeedback trainer it means the user is paying attention; otherwise, the user is distracted.
The neurofeedback trainer uses a configuration screen in FarmerKeeper to set up and change
this value as needed.
We do not infer users’ posture; instead, we take advantage of the effect posture has on
movements and whenever the user is moving and adding noise into the signal, the system
prompts them to maintain a good posture. To infer users’ movements, we empirically defined a
threshold of the signal quality using the Think-Gear Chip of the BCI headset. The threshold
was specified based on observations and data from a previous study we conducted with
children with autism using the Neurosky headset [16]. So, if the signal quality value is greater
(>) than 25 the BCI video game infers noise and stops the game; if the signal quality value is
less (<=) than 25 the noise is minimal and the BCI video game continues to run.
FarmerKeeper’s architecture has two nodes: a < <BCI headset> > node representing the
BCI headset used to measure the user’s brain activity and a < <PC Host> > node responsible
for running the game mechanics of the BCI video game (Fig. 5). FarmerKeeper was imple-
mented as a 2d video game in C# using the Unity® Software [20].
The <<PC Host> > node includes the following components:
– OpenVibe < <executable>>: this component is an open-source software platform dedicat-

ed to design, test, and use of BCI [63]. It receives data from a BCI headset through a
Bluetooth connection.
– Game Manager: this is the engine of the BCI video game. It coordinates the execution of the
rest of the components. This component interacts through a VRPN (Virtual Reality Peripheral
Network) with the OpenVibe component to obtain the data from the BCI headset.
– The relax phase manager, the activity phase manager, and the break phase
manager: these components run as the engines for each neurofeedback training
<<PC Host>> << BCI headset>>

BrainLink
Bluetooth
Game VRPN <<Firmware>>
OpenVibe
Manager ThinkGear
Acvity Break
Relax Phase Shop Database
Phase Phase
Manager Manager Handler
Manager Manager
Database
Sprites
Fig. 5 FarmerKeeper’s architecture

phase (Fig. 1) loading the graphics and audio resources, and the rules governing
each state.
– Shop Manager: this component is the engine handling the shop. It loads the items
available in the shop, keeps track of the items each player has already bought and allows
the player to customize the avatar selected to play.
– Database Handler: this component connects to a database, and it is used to gather specific
information about the players from the database (e.g., player preferences such name,
avatar, score). Also, this component is responsible for saving in the database potentially
useful data to keep track of the records of each player’s performance.
5 Evaluation methods
To evaluate FarmerKeeper, we conducted a within-subjects study at Pasitos. The study aims to

answer the following research questions:
– Does FarmerKeeper outperforms in terms of usability and user experience a commercial

BCI video game commonly used for neurofeedback training?
– Which are the potential benefits that FarmerKeeper offers to children with autism?
From all commercially available BCI video games used for neurofeedback training, we
selected BrainCats as it is one of the most popular game being used in several specialized
clinics using BrainMaster Technologies [49].
BrainCats is a race of 5 wild cats. The user selects one cat that will run according to the
user’s attention obtained by a BCI headset. The remaining cats run according to an automated
function of the system independent of the user’s attention. The speed of all the cats depends on
the race’s length. The neurofeedback trainer manually establishes the race’s length before
starting the game. During the race, if the user’s attention is above a pre-defined threshold the
borders of the race track will light up around the user’s selected cat as feedback of good
performance, together with the change of speed of the selected cat. At the end of a race, the
user wins some random trophies (e.g., ladybugs, hummingbirds, and monkeys) according to
the user’s final place in the race. If the user gains a monkey’s trophy, he will hear a sound of a
monkey at the beginning of the next race. During the race, when the user’s attention is above
the pre-defined threshold, all the earned trophies are animated as feedback for good attention
(See section 5.3.2. for an example).
5.1 Participants
Twelve children with autism from Pasitos, diagnosed with severe autism [4], were
voluntarily enrolled in the study (for simplicity of reading, we will now refer to the
children with autism participating in our evaluation study as participants). Participants
aged between 4 and 11 years old (avg. age = 6 years; sd = 2 years). Most of the participants
were not verbal, had comorbid ADHD with inattentive presentation, were hypersensitive,
and did not follow a pharmacological treatment. Also, none of the participants had
attended to neurofeedback training sessions before this work. However, the first author
of this paper is trained in neurofeedback techniques and gave a short training to the
physiologists involved in the study. A trainer trained in using neurofeedbak
(neurofeedback trainer) conducted the sessions for all the participants; but, the teacher of
each participant attended to all the sessions. Eight different teachers participated in the
study. To avoid bias when answering the questionnaires during the evaluation, none of
these teachers were previously involved in our design process. The neurofeedback trainer,
all the teachers, and all the parents on behalf of their children, consented to the study.
5.2 Setting up and installation
We equipped one therapy room at Pasitos with a desktop computer used by the neurofeedback
trainer to control the BCI video games and the EEG data being recorded during the sessions.
The desktop computer was instrumented with a second monitor to display the BCI video game
to the participant; a video camera (side view) to monitor participants’ interactions and
reactions, and a GP3 eye-tracker to record the eye-gaze of participants during the session.
Participants used the BrainLink headset as the BCI headset during the neurofeedback training
sessions. We removed all the potential available stimuli inside the room, like furniture, frames,
and visual supports (Fig. 6).
5.3 Procedure
Overall the procedure consisted of two steps: a sensitization process and the neurofeedback
training session.
5.3.1 Sensitization process
First, as participants don’t easily accept to wear accessories on their heads, we conducted a
sensitization process. During this process, we asked each participant to wear a dummy-headset
consisting on a velcro strip and a clip on the earlobe to help them get used to wearing
something similar to the BrainLink headset. Participants used the dummy-headset before
attending to the neurofeedback training sessions on a daily basis during their everyday
Fig. 6 A screenshot from the video camera (side view) showing the hardware installation in the room for
neurofeedback training
activities in Pasitos. The sensitization process for each participant lasted around 1 week, and
the time they used the dummy-headset varied according to their tolerance to it.
5.3.2 Neurofeedback training sessions
Before each neurofeedback training session, we calibrated the eye-tracking (4 min approx.).
This phase is not needed for the session; but, mainly used to obtain data for later analysis.
Our study was within-subjects –meaning, all participants played with both FarmerKeeper
and BrainCats (Fig. 7). Participants completed two neurofeedback training sessions of 15 min
each approx., on different days. In the first day, participants played with either FarmerKeeper
or BrainCats. In the second day, participants played the other BCI video game. To counter-
balance and avoid learning effects, half of the participants first played FarmerKeeper and the
other half first played Braincats. Participants were randomly assigned to each group. To reduce
potential noise when recording the EEG data during neurofeedback training, participants were
instructed to remain as still as possible and avoid abrupt movements. Although this could be
challenging for some participants, the neurofeedback trainer gave away verbal or physical
prompts asking them to remain as still as possible when needed.
For each session, the neurofeedback trainer first introduced the goal of the BCI video game
to the participant. When playing with FarmerKeeper participants watched the story of the game;
in contrast, when playing with BrainCats the neurofeedback trainer verbally explained to the
participant the story of the game. Then, for around 2 min (approx.), participants completed a
relaxation task while the neurofeedback trainer placed the headset (e.g., relax phase). Next,
participants selected their character and completed three blocks of activity (i.e., one of 30 s,
2 min, and 5 min), with breaks between activities lasting for half of each activity. For both
games the neurofeedback trainer specified an attention’s threshold of 40 attention units.
After each session, teachers answered the core-game module of the GEQ questionnaire [35]
to evaluate the game experience according to each child’s behavior during the session.
Although teachers were not the final users, they are the primary caregivers of the participants,
and we used them as Bproxies^ to gather their perceptions –as other studies have used proxies
with non-verbal populations [75].
After participants completed the two sessions, using both FarmerKeeper and Braincats,
participants answered the FunSorter Questionnaire of the FunToolKit [60] to evaluate their
Fig. 7 Neurofeedback training sessions. A screenshot from the video camera (side view) during a neurofeedback
training session using FarmerKeeper (left) and BrainCats (right)
preference for one game. Since, the FunSorter Questionnaire is an easy, short, and compre-
hensible questionnaire, participants answered it themselves selecting one of two cards, each
one with an image of each BCI video game (e.g., FarmerKeeper and BrainCats).
At the end of the evaluation, teachers answered the System Usability Scale (SUS) survey
[14] and the User Experience Questionnaire (UEQ) [59] per condition. Also, we interviewed
teachers to compare both conditions and gather their opinions about the BCI video games and
collect their feedback.
5.3.3 Data collection2
We used the Game Experience Questionnaire [35] (n = 24, one per session per game), the
System Usability Scale [14] (n = 16, one per teacher per game), the User Experience Ques-
tionnaire [59] (n = 16, one per teacher per game) and the FunSorter [60] (n = 12, one per
participant) to evaluate FarmerKeeper.
The Game Experience Questionnaire (GEQ) [35] consists of three different modules: the
core module (evaluates actual user experiences related to game play), the social presence
module (concerns gaming with others) and the post game module (concerns experiences once
a player has stopped gaming). To avoid overload, we selected only the core module of the
GEQ as we wanted to measure participants’ experiences during gameplay. The GEQ core
module has a total of 33 items and assesses game experience as scores on seven components:
Immersion, Flow, Competence, Positive and Negative Affect, Tension, and Challenge. The
components’ scores are computed as the average value of its items (mentioned on the manual
of the GEQ). Each item uses a 5-point Likert scale, from 0 (not at all) to 4 (extremely).
We selected the System Usability Scale (SUS) [14] as it is a generic, quick, easy to
administer, and provides a high-level usability perceptions form users. The SUS has a total
of 10 items which cover a variety of aspects of system usability, such as the need for support,
training, and complexity of the evaluated system. Each item uses a 5-point Likert scale, from 1
(strongly disagree) to 5 (strongly agree) to measure user satisfaction. The SUS yields a single
number representing a composite measure of the overall usability of the system being studied.
SUS scores have a range of 0 (worst) to 100 (best).
We selected the User Experience Questionnaire (UEQ) [59] as it enables a quick assessment
of the user experience and has been released in Spanish. The UEQ contains 6 scales/
components: attractiveness, perspicuity, efficiency, dependability, stimulation and novelty.
The attractiveness scale/component has 6 items, all other have 4 items, with a total of 26
items. The UEQ uses bipolar terms scored with a 7-point Likert scale, from −3 to +3. Thus −3
represents the most negative answer, 0 a neutral answer, and + 3 the most positive answer. The
score for each scale/component goes from −1 to 2.5, and the authors provide an excel-tool to
compute the scores automatically.
We also decided to use the SUS and the UEQ questionnaires as both provide a data
benchmark that enable us to compare our results with those available in the literature.
Finally, we used the Fun Sorter tool from the Fun Toolkit [60]. The Fun Toolkit is a
questionnaire designed to be used for children. It is composed of three surveys: the
Smileyometer, the Again-again, and the FunSorter. The Smileyometer uses a 5-point Likert
Scale, from 1/very sad face (awful) to 5/very happy face (brilliant) and can be used before the
2
The EEG data and a transcript of our videos is available in our websites or could be directly requested by email
to the first author.
technology experience to measure the expectations of the child and after the technology
experience to measure the experienced feelings or experience fun of the child. The Again-
again survey uses a table designed to capture children’s engagement by asking them if whether
or not they would like to do the activity again. And the Fun Sorter allows children to rank
items against one or more constructs according to five concepts: like more, more fun, easy
nice, I would like it as a gift. This was intended to record the children’s opinions towards the
technology to obtain their engagement. As a control for the FunSorter tool, we counterbalance
the order of the answers. For our particular case, children answered the three surveys (The
Simileyometer, the Again-again, and the FunSorter). However, when answering the
Smileyometer and the Again-again survey most participants selected the happy face, most of
the time. Teachers explained to us that most participants did not understand how these tools
worked and that their selection of the happy face may not be representative of their prefer-
ences. –as this population relate Bhappy faces^ to positive things and identified them as
rewards. Therefore, we only report data from the Fun Sorter tool, to avoid bias when
presenting our results.
We audio recorded and transcribed all the interviews (n = 8) for later analysis. All interviews
lasted about 2:48:59 h (avg = 21:07 min, sd = 04:15 min). The semi-structured interview
consisted of questions about use and adoption, potential benefits, and use of the BCI headset
using both FarmerKeeper and BrainCats. All sessions (n = 24) were video-recorded with a
video camera (side view) to analyze participant’s attention and emotions. All videos lasted
about 05:13:55 h (avg = 13:05 min, sd = 02:21 min). We stored logs from the use of
FamerKeeper and BrainCats to further analyze participants’ activity during the sessions. We
recorded all the data received by the Brainlink headset to analyze the attention of each
participant during sessions. We also collected and recorded data from the GP3 eye-tracker to
analyze where was the participant’s attention based on where their eyes gaze during the session.
5.3.4 Data analysis
We analyzed the results from the GEQ, SUS, UEQ, and FunSorter separately, using the
formulas specified by each survey [14, 35, 59, 60]. For the four surveys (e.g., GEQ, SUS,
UEQ, and FunSorter), we compared the results of FarmerKeeper against BrainCats. Also, we
compared the results of both BCI video games against a benchmark of data available for the
SUS [14] and the UEQ [59] questionnaires.
To analyze qualitative data from interviews, we used deductive analytical approaches,
based on our initial research questions focused on understanding the overall interaction
experience of participants using FarmerKeeper. We used deductive coding to examine how
observed behaviors and reported perceptions supported or contradicted our research questions.
We additionally used inductive approaches to theme analysis from our data. To support our
inductive analysis, we followed a qualitative approach including the use of techniques to
derive grounded theory [72] and affinity diagramming [10] (e.g., open and axial coding [72]).
Using these techniques, quotes and/or events obtained from interviews were grouped to
uncover emergent themes related to system usability, user experience, use and adoption, and
the impacts on attention by the use of FarmerKeeper. We used atlas.ti [22] for data analysis.
To analyze video recordings, we used sequential analysis using a coding scheme
(Table 2) for the systematic coding of the data. We estimated the total time partici-
pants stayed focused and the frequency of participants’ emotions. We calculated these
data per participant, per condition.
Table 2 Definition of the codification scheme used to code the target behaviors
Category Target behavior Definition Type of

measurement
User’s behaviors Attention On task The child is engaged and focused on the activity Time
Off task The child is distracted Time
Emotions Positive The child is smiling (joy), laughing (amused), Frequency
excited to continue (happy)
Negative The child is yawning (bored), stretching or Frequency
looking for other position in the chair (bored),
refusing to continue (annoying)
For the eye-tracking analysis, we selected only the data from participants who completed
the calibration process and with more than 80% of the eye-tracking data recorded without
noise (e.g., when participants get outside the vision’s field of the eye-tracking due to their
movement). We selected 10 of 12 participants to evaluate FarmerKeeper and 7 of 12
participants for BrainCats. First, we calculated a general heat map of all the sessions for each
participant, and then we calculated an average heat map of all the selected participants using
each BCI serious game. Then to avoid outliner data and highlight the most viewed areas, we
applied an outlier filter of 20%, as the average of time the eye-tracking sensor was capturing
data without noise was about 80%.
From the data recorded by the BCI headset, we extracted the attention and the signal quality
data for each activity phase (i.e., 30seg, 2 min and 5 min) of each session per participant, per
condition. First, we processed the attention data of each session to reduce noise and outlier
values applying an average filter where the value of the signal quality indicates noise data.
Then, to measure sustained attention during the activities, we computed how many blocks of
attention span (e.g., the amount of concentrated time a person can spend on task without
becoming distracted) each participant completed. We defined sustained attention as 8 s of
uninterrupted attention [13]. This time span has been demonstrated as the average sustained
attention for a neurotypical individual.
Although our sample is small, we conducted a statistical analysis as a first step to test the
significance of our results. We used the Shapiro-Wilk Normality Test [67] to discriminate from
parametric and non-parametric data. For our data with a normal distribution we used a
dependent t-test [73] –except, for the data from the FunSorter questionnaire for which we
used a Fisher’s Exact Test [46] due its classification nature. For our non-parametric data we
used the Wilcoxon signed-rank test [82]. We defined a confidence interval of α = 0.05 (i.e., the
results are significate at p < =0.05) to all our statistical tests.
6 Results
Overall, our results suggest our participants found FarmerKeeper to be more suitable to
support neurofeedback training than BrainCats. Also, our results suggest participants and
teachers found FarmerKeeper more enjoyable, easy to use, and attractive than BrainCats.
Teachers perceived a positive and enjoyable game experience on both BCI video games.
However, participants exhibited more positive emotions and less negative emotions using
FarmerKeeper than when using BrainCats. Teachers agreed both BrainCats and FarmerKeeper
could have potential benefits in improving attention problems. Particularly, when using
FarmerKeeper, teachers also identified children with autism were less anxious, learning to
follow rules, and improving their behavior when transitioning between activities.
6.1 Usability
The results from the SUS questionnaire [14] (Fig. 8) show teachers (on average) agreed
FarmerKeeper was above the range of Bacceptable^ (avg. = 80.63, Fig. 8) whereas they
found BrainCats just a little above the range of Bacceptable^ (avg. = 70.63). Also, all of
the teachers (8 out of 8) rated FarmerKeeper as above the average while only half (4 out
of 8) found BrainCats to be above average. These results show that teachers’ perception
of usability of FarmerKeeper is above the one perceived of BrainCats and to a bench-
mark of data from other studies using the SUS questionnaire [14]. Overall, these results
show that FarmerKeeper is Bgood^ and that FarmerKeeper has better usability than
BrainCats (p = 0.012307).
6.2 User experience
Our results suggest both teachers and participants perceived FarmerKeeper to have a better
user experience than BrainCats. Also, suggest participants and teachers found FarmerKeeper
more enjoyable, easy to use, and attractive. Teachers perceived a positive and enjoyable game
experience on both games; however, participants exhibited more positive emotions and less
negative emotions using FarmerKeeper than when using BrainCats.
6.2.1 User experience according to the UEQ survey
According to the UEQ [59], the results suggest teachers perceived FarmerKeeper has a better
user experience than BrainCats (Fig. 9). Analyzing our UEQ results by categories (Table 3),
teachers scored FarmerKeeper high (avg. = 1.89) on all the categories, against BrainCats (avg.
Fig. 8 SUS results. The Score is a value between 0 and 100, where 0 signifies that a user founds a system
absolutely useless and 100 that a user found a system optimally useful. Results above 68 signify Bacceptable^
Fig. 9 Results from UEQ compared to the Benchmarks
= 0.97), that was scored only as dependability high, with low stimulation and with other above
the average (Fig. 9).
Comparing our results with a benchmark of data from other studies using the UEQ [59] to
evaluate interactive systems [41]; the results suggest teachers perceived FarmerKeeper as
Bexcellent^ (Fig. 9) and BrainCats to be Bon average^. Analyzing the results per categories
FarmerKeeper was perceived as Bexcellent^ in all categories and BrainCats was perceived a
little Babove average^ in attractiveness, learnability, dependability, and novelty, and below
average in the categories related to efficiency and stimulation. Our results suggest teachers
found FarmerKeeper to have a better user experience than BrainCats and teachers are more
motivated to use FarmerKeeper. These results could be partially explained as teachers per-
ceived FarmerKeeper to be less tedious than BrainCats; since FarmerKeeper has the possibility
to change the stimuli by selecting different scenarios and animals increasing attractiveness and
stimulation..
B… [participants liked more] the farm [referring to the FarmerKeeper game] because it
was changing the animals and kept [participants] focused. In the leopards [referring to
the BrainCats game] they were like ‘I already won once’, and maybe the next race was
Table 3 UEQ results
Categories BrainCats FarmerKeeper
Mean STD N Confidence Confidence Mean STD N Confidence Confidence

interval interval
Attractiveness 1.56 0.88 8 0.61 0.95 2.17 2.25 0.60 8 0.41 1.84 2.66
Perspicuity 1.19 1.31 8 0.91 0.28 2.09 1.97 0.92 8 0.64 1.33 2.61
Efficiency 0.66 1.43 8 0.99 −0.34 1.65 1.88 0.67 8 0.46 1.41 2.34
Dependability 1.66 0.78 8 0.54 1.12 2.20 1.88 0.55 8 0.38 1.49 2.26
Stimulation 0.41 1.89 8 1.31 −0.91 1.72 1.66 0.76 8 0.52 1.13 2.18
Novelty 0.94 1.11 8 0.77 0.17 1.71 1.72 0.87 8 0.60 1.12 2.32
8 teachers responded to the UEQ questionnaire (N = 8), to evaluate BrainCats (left) and FarmerKeeper (right).
Score − 1 to 2.5. Confidence intervals (p = 0.05) per category. The confidence value is a measure for the precision
of the estimation of the mean, the smaller the confidence value, the higher is the precision of the estimation
not so stimulating. Instead, when playing with [FarmerKeeper] they first looked for
piggies, then cows, and the stimuli were changing…^ [teacher 2].
6.2.2 Game experience according to the core-game module of the GEQ questionnaire
Our results from the core-game module of the GEQ questionnaire [35] show that both, Farmer
Keeper and BrainCats, have a positive and enjoyable game experience. We found the score for
FarmerKeeper to be just a little above the score of BrainCats (avg. = 0.38, Fig. 10); with a
significant difference in the items related to flow (the feelings of enjoyment that occur when
balancing skill and challenge, p = 0.0314) and tension/annoyance (p = 0.0420). The other
items did not had a significant difference (Competence p = 0.3184, Sensory and imaginative
immersion p = 0.2983, Challenge p = 1.00, Negative affect p = 0.1123, and Positive affect p =
0.1918). This lack of significant difference is somehow expected as for example FarmerKeeper
does not provide an experience of competence, immersion, or challenge.
These results could be partially explained using our analysis from our interviews. Teachers
agreed that both, FarmerKeeper and BrainCats, have positive effects on participants, and each
condition motivated and engaged participants due to different reasons (e.g., FarmerKeeper by
the graphics and topic, and BrainCats by being the first to reach the goal).
B…In [BrainCats, participants] were very motivated to be the first one to win the race,
and in FarmerKeeper, I think [they were motivated by] the graphics and the topic…^
[teacher 1]
We suggest that the little negative effects (i.e., tension/annoyance and negative affect) could be
attributed to the graphical user interface, the input modality, and to the length of the last
activity phase. Since the BCI headset was the controller used as input and participants were
used to grabbing something with their hand to interact with video games, sometimes they felt
as they were not doing anything and began to bore and despair; especially when the activity
Fig. 10 Results from the core-game module of the GEQ Questionnaire. Score from 0 to 4, (for * low score is
better)
lasted a long time (e.g., 5 min). This behavior was more common in BrainCats than in
FarmerKeeper (Fig. 10).
B… at first, as [participants] didn’t like the input device, some of them were used to use
an iPad or videogames; they were more comfortable when grabbing something with
their hand to interact…^ [teacher 2]
6.2.3 Use and adoption
According to teachers, when playing FarmerKeeper, none of the participants showed frustra-
tion problems. In contrast, when using BrainCats, 3 out of the 12 participants got frustrated,
bored or they didn’t want to complete the the training.
B… participants liked more [FarmerKeeper]. In [BrainCats] they liked playing the first
time, but then they got bored or didn’t want to continue playing with longer activities.
And in [FarmerKeeper], they like playing more than once, they could play it for longer
…^ [teacher]
Using the data we scored from videos (see how we analyze video data in section 5.3.4) our
results show participants exhibited more positive emotions (FarmerKeeper avg. = 6.33,
BrainCats avg. = 3.17; avg. diff = 3.17, p = 0.18352) and less negative emotions
(FarmerKeeper avg. = 1.67, BrainCats avg. = 4.5; avg. diff = 2.8333, p = 0.00932) during
the full session using FarmerKeeper than with BrainCats (Fig. 11). Of course our sample is
small to have a statistical significance but we attribute these results to the fact that overall
games for children with autism must encourage positive emotions and not strengthen the
negative ones [69], in this way due to our p values we can guarantee that FarmerKeeper will
incite more positive emotions as BrainCats does, and absolutely not negative emotions.
Using the data we scored from videos we analyzed the emotions participants exhibited only
during the activity phases (e.g., 30 s, 2 min, and 5 min), and we found a similar behavior with
no significant difference for the first two activity phases (30 s and 2 min) either when using
BrainCats and FarmerKeeper. For the last activity phase (5 min.), participants exhibited more
positive emotions (avg. diff = 0.75, p = 0.34495) and less negative emotions (avg. diff =
2.416667, p = 0.00672) with FarmerKeeper than with BrainCats (Fig. 12). This result agrees
Fig. 11 Positive (left) and negative (right) emotions during the full neurofeedback sessions per participant
Fig. 12 Positive (left) and negative (right) emotions during the last activity phase (5 min) per participant
with the analysis from the entire neurofeedback training, highlighting that FarmerKeeper will
not provoke negative emotions with a p value of 0.00672. The negative emotions participants
exhibited when using BrainCats were related to being bored and frustrated, some of them
occurred when the activity phase was getting longer (e.g., 2 min and 5 min) (81.5%) and others
occurred when transitioning between phases (18.5%).
The results from the FunSorter [60], show that 9 out of 11 participants liked FarmerKeeper
more than BrainCats (2 out of 11), and also, they found FarmerKeeper more enjoyable, easy to
use, and attractive than BrainCats (Fig. 13, I like more p = 0.0654, Most fun p = 0.2265, Easier
p = 0.0654, Prettier p = 0.2265, and Prefer as gift p = 0.0654). A more longitudinal study and
bigger sample is needed to establish significant result around enjoyment and easy to use.
According to teachers, participants who liked BrainCats more than FarmerKeeper enjoyed
the competitive challenge available in BrainCats and not available in FarmerKeeper.
B… he [a participant] liked BrainCats more, he likes to play videogames, so I think he
better relates the concept of winning, reaching a goal, and the competition part of
[BrainCats]…^ [teacher 1]
Fig. 13 Results from Fun Sorter

Although FarmerKeeper was more enjoyable for most of the participants (9 out of 11), all
teachers perceived the last activity phase (5 min) to be somewhat longer to be completed by
participants without getting bored. As a consequence, teachers suggested incorporating some
changes to avoid participants’ boredom and maintain them motivated to finish the activities of
the game.
6.3 Potential benefits
Our qualitative results suggest potential benefits mostly related to the improvement in
attention. Also, as FarmerKeeper is designed implementing the phases of a neurofeedback
training [Section 3.2], children could learn to be calm before and during the activity, as
well as learning to follow rules, and improve their behavior when transitioning between
activities.
6.3.1 Attention during the session
Using the data we scored from videos our quantitative results show on average participants
were 8% (2:00 min approx., p = 0.00288) of the time paying more attention during the full
session (avg. full session = 13:05 min) when using FarmerKeeper (97.15% of full session) than
with BrainCats (89.15% of full session). 11 out of 12 participants were more focused in the
session when using FarmerKeeper (Fig. 14).
Teachers attribute children are more focused on FarmerKeeper than on BrainCats, since
FarmerKeeper is perceived as more interactive and with more changes, against BrainCats,
which is perceived as repetitive and monotonous, offering FarmerKeeper a more enjoyable
experience.
Fig. 14 Percentage of time paying attention during each session, per participant. (From video analysis)
B… I think children like more FarmerKeeper, as it is more interactive, there are more
changes, more appropriate stimuli, and the option of choice between different animals to
seek, against BrainCats that is more repetitive and monotonous^ ... [teacher 8]
Results from our analysis of the eye-tracking data show participants were mainly focused on
the center of the screen (i.e., where the character is moving, and the animals appear) when
using FarmerKeeper (Fig. 15, bottom-left); while when using BrainCats, participants’ attention
was more dispersed across the screen (Fig. 15, bottom-right). The average heat-map from
BrainCats shows participants’ attention on areas outside of the trajectory of the leopard, getting
distracted by another stimulus (e.g., animal’s trophies) competing for the attention of the user.
According to some teachers, when the stimulus on the screen makes the attention of the
participant more dispersed, in some cases, the participant could lose the objective of the game
and therefore the interest on continuing with the session.
6.3.2 Attention during activity phases
Analyzing our data per activity phases (e.g., 30 s, 2 min, and 5 min), using the analysis from
videos and the EEG data we found out a similar behavior on participant’s attention during the
first two activity phases (30s. and 2 min) while using both, FarmerKeeper and BrainCats.
Nevertheless, during the last activity phase (5 min), on average there is a difference of 15.7%
of attention’s percentage of the total time between the two conditions (47.1 s out of 300 s, p =
0.00222). And during this last activity phase, all participants (12 out of 12) paid more attention
using FarmerKeeper (Fig. 16) than when using BrainCats.
Our results of the analysis of the EEG data show participants were paying more attention on
the 5 min. Activity phase using FarmerKeeper than when using BrainCats (on avg. 10 blocks
more of the attention span of 8 s each, p = 0.012917). The statistical significance is meaningful
due the small sample we have but encourage us to re-evaluate this impact of attention in a
bigger study. According to these results, 9 out of 12 participants were more focused using
FarmerKeeper than when using BrainCats, 2 out of 12 were more focused when using
BrainCats than using FarmerKeeper, and 1 out of 12 was equally focused with both video
games (Fig. 17).
Overall, all these results from participants’ attention (i.e., videos, EEG and eye-tracking
analysis) suggest FarmerKeeper is more suitable for neurofeedback training than BrainCats.
Especially, because participants were more time focused when using FarmerKeeper, and this
attention was sustained in a longer period of time –longer sustained attention is crucial to to
achieve better results during neurofeedback training.
Fig. 15 Eye-tracking results. Image of the average heat-map for of FarmerKeeper (left) and BrainCats (right)
with an outliner filter of 20%
Fig. 16 Percentage of time paying attention during the 5 min activity phase, per participant. (From video
analysis)
7 Conclusions and future work
This paper describes the design process followed for developing FarmerKeeper, a BCI video
game to support neurofeedback training for children with autism, controlled by a consumer-
grade BCI headset. We found three design implications and eight design considerations a BCI
Fig. 17 Blocks of attention (Attention Span of 8 s) using the EEG data, per participant
video game should incorporate to appropriately support neurofeedback training for children
with autism. Moreover, we presented the results of a 4-weeks usability and user experience
study of FarmerKeeper with children with autism. The results of the evaluation suggest that
FarmerKeeper is easy to use, useful and may have numerous benefits for attention on children
with autism.
Although the research reached our goals, we observed some unavoidable limitations. This
research was conducted in only one school-clinic of children with autism. Moreover, we
conducted the evaluation with a small sample size of participants; therefore, to generalize the
results for other populations, the study should be replicated with a much larger population to
gather data to get a more realible statistical significance. Additionally, children with autism
only played once with the BCI video games. Then, the study also should be replicated in the
long term to avoid the bias due to the Bnovelty effect^ and to get a more realible knowledge
about the acceptance and adoption of the BCI video games –researchers must particularly
measure if participants maintain a positive attitude and their engagement over time. However,
given that the objective of this study was not to generalize our findings but to get an overall
impression of the potential of FarmerKeeper in a particular use case, we feel that our results are
valuable for researchers exploring the design space of BCI video games to support
neurofeedback training of children with autism. As future work, we plan to conduct a long-
term study of the use of FarmerKeeper. We will reflect on the results from this evaluation to re-
design our prototype and include collaborative challenges, less monotonous activities, and
short objectives into the main objective of the activity. Using this new version of
FarmerKeeper we will test the efficacy of such a system in improving the attention impair-
ments of children with autism and investigate its potential clinical benefits. In terms of
adoption we plan to investigate the acceptance of FarmerKeeper after the Bnovelty effect^
wears off, game engagement, and general usability aspects related to its usage.
Another limitation of this study is that we are only focusing in using neurofeedback training
to help children with autism cope with attention issues. However, to truly treat autism,
innovative solutions must be focused in targeting the core symptoms of autism by handling
issues related to language impairments, social interactions, and behavior. Although, this is a
first effort to provide evidence of the potential of BCI video games to support individuals with
autism; researchers must explore how such solutions could also be instrumental to be used in a
therapeutic and clinical context for autism.
Also, there are some technical limitations on the development of FarmerKeeper. We
assumed that the eSense algorithm of the ThinkGear chip on the Brainlink headset provides
reliable measures when inferring the attention of its users. But, this inference is really just an
Bapproximation^ and as eSense is a proprietary algorithm, its documentation does not provide
a detailed explanation of its design. For our particular case, we used the openvibe component
instead of the headband API to return the control to the users and enable them to manually
change the way to obtain the attention values as input to the BCI video game when needed. So,
as future work, research must explore more reliable and robust algorithms to measure attention
that could be integrated into FarmerKeeper thought the openvibe component. In addition,
during the development of FarmerKeeper, we found out that by controlling the posture of the
users might reduce the noise associated with users’ movements. However, we did not infer
users’ posture, and more work is required to propose a robust detection method to avoid noise
due to user’s movements and bad posture. Researchers must explore both invasive and non-
invasive solutions, like using vision-based algorithms to track the users’ posture when playing
BCI video games. Finally, when designing FarmerKeeper we found out that enable users to
easily relate to the character of the game is important. And although FarmerKeeper enables
some sort of customization, open questions remain to truly investigate how to better achieve
this emotional connection between users and the character of the game [77]. As future work,
we propose to investigate how the character of the game could mimic users’ behaviors and
emotions to achieve this desired level of empathy.
A BCI video game implementing the design considerations proposed could close the gap
between accessibility and reliability of neurofeedback training for children with autism. In
addition, this project has the potential to transform neurofeedback training towards the use of a
more accessible solution that could potentially bring the latest advances in neuroscience to
homes and clinics of children with autism. This proposal could potentially reduce costs and
barriers associated with the non-pharmacological treatment of autism. And, could help to
increase the applications of ubiquitous computation and HCI in the area of neurosciences,
allowing the growth in the development of ubiquitous innovative systems to assist
neurofeedback training mainly for children with autism.
Acknowledgments We thank all the participants enrolled in this study and the researchers and reviewers who
provide helpful comments on previous versions of this document. We also thank CONACYT for the first author
fellowship and we thank to the CONACYT project #2209 of the fourth author for their financial support.
Compliance with ethical standards
Conflict of interest Authors declare that they have no conflict of interest.
Publisher’s Note Springer Nature remains neutral with regard to jurisdictional claims in published maps and
institutional affiliations.
References
1. Aamodt S, Wang S (2007) Exercise on the brain. New York Times

2. Abt CC (1970) Serious games: the art and science of games that simulate life. New Yorks Viking, New
York, p 6
3. Alves S, Marques A, Queirós C, Orvalho V (2013) Lifeisgame: a serious game about emotions for children
with autism spectrum disorders. PsychNology J 11:191–211
4. American Psychiatric Association (2013) Diagnostic and statistical manual of mental disorders, (DSM-5®).
American Psychiatric Pub
5. Antshel KM, Zhang-James Y, Wagner KE et al (2016) An update on the comorbidity of ADHD and ASD: a
focus on clinical management. Expert Rev Neurother 16:279–293
6. Autism, Investigators DDMNSY 2010 P (2014) Prevalence of autism spectrum disorder among children
aged 8 years—autism and developmental disabilities monitoring network, 11 sites, United States, 2010.
Morb Mortal Wkly Rep Surveill Summ 63:1–21
7. Bailey R, Wise K, Bolls P (2009) How avatar customizability affects children ’ s arousal and subjective
presence during junk food – sponsored online video games. Cyberpsychology Behav 12:277–283.
https://doi.org/10.1089/cpb.2008.0292
8. Bakhshayesh AR, Hänsch S, Wyschkon A et al (2011) Neurofeedback in ADHD: a single-blind randomized
controlled trial. Eur Child Adolesc Psychiatry 20:481–491. https://doi.org/10.1007/s00787-011-0208-y
9. Bernardini S, Porayska-Pomsta K (2013) Planning-based social partners for children with autism. Twenty-
Third Int Conf …, pp 362–370
10. Beyer H, Holtzblatt K (1999) Contextual design. Interactions 6:32–42
11. Birk MV, Atkins C, Bowey JT, Mandryk RL (2016) Fostering intrinsic motivation through avatar identi-
fication in digital games. Proc 2016 CHI Conf Hum Factors Comput Syst - CHI ’16, pp 2982–2995.
https://doi.org/10.1145/2858036.2858062
12. Blandón DZ, Munoz JE, Lopez DS, Gallo OH (2016) Influence of a BCI neurofeedback videogame in
children with ADHD. Quantifying the brain activity through an EEG signal processing dedicated toolbox.
In: Computing Conference (CCC), 2016 IEEE 11th Colombian, pp 1–8
13. Brain S (2016) Attention span statistics – statistic brain. http://www.statisticbrain.com/attention-span-
statistics/. Accessed 16 Feb 2018
14. Brooke J (2013) SUS : a retrospective. J Usability Stud 8:29–40
15. Choon Guan Lim B, Lee T-S, Guan C et al (2010) Effectiveness of a brain-computer Interface based
Programme for the treatment of ADHD: a pilot study. Psychopharmacol Bull 4343:73–82
16. Cibrian FL, Mercado J, Escobedo L, Tentori M (2018) A step towards identifying the sound preferences of
children with autism. In: Proceedings of the 12th EAI international conference on pervasive computing
technologies for healthcare, pp 158–167
17. Cmiel V, Janousek O, Kolarova J (2011) EEG biofeedback. Proc 4th Int Symp Appl Sci Biomed Commun
Technol - ISABEL ’11, pp 1–5. https://doi.org/10.1145/2093698.2093752
18. Coben R, Linden M, Myers TE (2010) Neurofeedback for autistic spectrum disorder: a review of the
literature. Appl Psychophysiol Biofeedback 35:83–105. https://doi.org/10.1007/s10484-009-9117-y
19. Dickey MD (2006) Game design narrative for learning: appropriating adventure game design narrative
devices and techniques for the design of interactive learning environments. Educ Technol Res Dev 54:245–
263. https://doi.org/10.1007/s11423-006-8806-y
20. Engine U-G (2015) Unity technologies. Available Unity Technol
21. Escobedo L, Tentori M, Quintana E et al (2014) Using augmented reality to help children with autism stay
focused. IEEE Pervasive Comput 13:38–46. https://doi.org/10.1109/MPRV.2014.19
22. Friese S (2014) Qualitative data analysis with ATLAS. ti. Sage, London
23. Fuchslocher A, Niesenhaus J, Krämer N (2011) Serious games for health: an empirical study of the game
Bbalance^ for teenagers with diabetes mellitus. Entertain Comput 2:97–101
24. Garzotto F, Gelsomini M, Pappalardo A et al (2016) Using brain signals in adaptive smart spaces for
disabled children. Proc 2016 CHI Conf Ext Abstr Hum Factors Comput Syst - CHI EA ’16, pp 1684–1690.
https://doi.org/10.1145/2851581.2892533
25. Goldstein G, Johnson CR, Minshew NJ (2001) Attentional processes in autism. J Autism Dev Disord 31:
433–440. https://doi.org/10.1023/A:1010620820786
26. Green CS, Bavelier D (2008) Exercising your brain: a review of human brain plasticity and training-induced
learning. Psychol Aging 23:692
27. Hallford N, Hallford J (2001) Swords and circuitry: a designer’s guide to computer role-playing games.
Premier Press, Incorporated, Boston
28. Hammond DC (2011) What is neurofeedback: an update. J Neurother 15:305–336. https://doi.org/10.1080
/10874208.2011.623090
29. Hardy J, Scanlon M (2009) The science behind lumosity. San Fr CA Lumos Labs, San Francisco
30. Hayes GR, Hirano S, Marcu G et al (2010) Interactive visual supports for children with autism. Pers Ubiquit
Comput 14:663–680. https://doi.org/10.1007/s00779-010-0294-8
31. Hefner D, Klimmt C, Vorderer P (2007) Identification with the player character as determinant of video
game enjoyment. In: Entertainment computing–ICEC 2007. Springer, pp 39–48
32. Heinrich H, Gevensleben H, Strehl U (2007) Annotation: neurofeedback - train your brain to train
behaviour. J Child Psychol Psychiatry Allied Discip 48:3–16. https://doi.org/10.1111/j.1469-
7610.2006.01665.x
33. Heyvaert M, Saenen L, Campbell JM et al (2014) Efficacy of behavioral interventions for reducing problem
behavior in persons with autism: an updated quantitative synthesis of single-subject research. Res Dev
Disabil 35:2463–2476
34. Holtzblatt K, Wendell JB, Wood S (2004) Rapid contextual design: a how-to guide to key techniques for
user-centered design. Elsevier, San Francisco
35. IJsselsteijn W, Van Den Hoogen W, Klimmt C et al (2008) Measuring the experience of digital game
enjoyment. Proceedings of measuring behavior, pp 88–89
36. Keay-Bright W (2007) Can computers create relaxation? Designing ReacTickles© software with children
on the autistic spectrum. CoDesign 3:97–110. https://doi.org/10.1080/15710880601143443
37. Keehn B, Müller RA, Townsend J (2013) Atypical attentional networks and the emergence of autism.
Neurosci Biobehav Rev 37:164–183. https://doi.org/10.1016/j.neubiorev.2012.11.014
38. Keizer AW, Verment RS, Hommel B (2010) Enhancing cognitive control through neurofeedback: a role of
gamma-band activity in managing episodic retrieval. Neuroimage 49:3404–3413. https://doi.org/10.1016/j.
neuroimage.2009.11.023
39. Kim SK, Yoo EY, Lee JS et al (2014) The effects of neurofeedback training on concentration in children
with attention deficit / hyperactivity disorder. Int J Bio-Science Bio-Technology 6:13–23
40. Kouijzer MEJ, van Schie HT, de Moor JMH et al (2010) Neurofeedback treatment in autism. Preliminary
findings in behavioral, cognitive, and neurophysiological functioning. Res Autism Spectr Disord 4:386–
399. https://doi.org/10.1016/j.rasd.2009.10.007
41. Laugwitz B, Held T, Schrepp M (2008) Construction and evaluation of a user experience questionnaire. HCI
Usability Educ Work, pp 63–76. https://doi.org/10.1007/978-3-540-89350-9_6
42. Lee K (2009) Evaluation of attention and relaxation levels of archers in shooting process using brain wave
signal analysis algorithms. Sci Sensit 12:341–350
43. Lim C-W, Jung H-W (2013) A study on the military serious game. Adv Sci Technol Lett 39:73–77
44. Lim CG, Lee TS, Guan CT et al (2012) A brain-computer Interface based attention training program for
treating attention deficit hyperactivity disorder. PLoS One 7:8. https://doi.org/10.1371/journal.
pone.0046692
45. Lubar JF (1995) Neurofeedback for the management of attention-deficit/hyperactivity disorders. In:
Chapter in Schwartz MS (ed) Biofeedback: a practitioner’s guide. Guilford Press, New York, pp 493–522
46. Lubar JF, Bahler WW (1976) Behavioral management of epileptic seizures following EEG biofeedback
training of the sensorimotor rhythm. Biofeedback Self Regul 1:77–104. https://doi.org/10.1007
/BF00998692
47. Mandryk RL, Kalyn M, Dang Y et al (2012) Turning off-the-shelf games into biofeedback games. Proc 14th
Int ACM SIGACCESS Conf Comput Access - ASSETS ’12, pp 199. https://doi.org/10.1145
/2384916.2384952
48. Mandryk RL, Dielschneider S, Kalyn MR et al (2013) Games as neurofeedback training for children with
FASD. Proc 12th Int Conf Interact Des Child - IDC ’13, pp 165–172. https://doi.org/10.1145
/2485760.2485762
49. Mandryk RL, Nacke LE, Mandryk RL (2016) Biometrics in gaming and entertainment technologies. In:
Biometrics a data driven world trends, Technol challenges, pp 191–224
50. May T, Rinehart N, Wilding J, Cornish K (2013) The role of attention in the academic attainment of children
with autism spectrum disorder. J Autism Dev Disord 43:2147–2158. https://doi.org/10.1007/s10803-013-
1766-2
51. Mihajlovic V, Grundlehner B, Vullers R, Penders J (2015) Wearable, wireless EEG solutions in daily life
applications: what are we missing? IEEE J Biomed Heal Informatics 19:6–21. https://doi.org/10.1109
/JBHI.2014.2328317
52. Nouchi R, Taki Y, Takeuchi H et al (2012) Brain training game improves executive functions and processing
speed in the elderly: a randomized controlled trial. PLoS One 7:e29676. https://doi.org/10.1371/journal.
pone.0029676
53. Ochs E, Kremer-Sadlik T, Sirota KG, Solomon O (2004) Autism and the social world: an anthropological
perspective. Discourse Stud 6:147–183
54. Owen AM, Hampshire A, Grahn JA et al (2010) Putting brain training to the test. Nature 465:775–778.
https://doi.org/10.1038/nature09042
55. Patsis G, Sahli H, Verhelst W, De Troyer O (2013) Evaluation of attention levels in a Tetris game using a
brain computer interface. In: Carberry S, Weibelzahl S, Micarelli A, Semeraro G (eds) User modeling,
adaptation, and personalization. Springer Berlin Heidelberg, Berlin, pp 127–138
56. Pineda JA, Brang D, Hecht E et al (2008) Positive behavioral and electrophysiological changes following
neurofeedback training in children with autism. Res Autism Spectr Disord 2:557–581. https://doi.
org/10.1016/j.rasd.2007.12.003
57. Planchon J, Vacher A, Comblet J et al (2018) Serious game training improves performance in combat life-
saving interventions. Injury 49:86–92
58. Pope AT, Palsson OS (2001) Helping video games Brewire our minds^. Int J Soc Psychiatry 54:370–382
59. Rauschenberger M, Schrepp M, Perez-Cota M et al (2013) Efficient measurement of the user experience of
interactive products. How to use the user experience questionnaire (UEQ). Example: Spanish language
version. Int J Interact Multimed Artif Intell 2:39–45. https://doi.org/10.9781/ijimai.2013.215
60. Read JC, MacFarlane S (2006) Using the fun toolkit and other survey methods to gather opinions in child
computer interaction. In: Proceedings of the 2006 conference on Interaction design and children, pp 81–88
61. Rebolledo-Mendez G, Dunwell I, Martínez-Mirón EA et al (2009) Assessing NeuroSky’s usability to detect
attention levels in an assessment exercise. In: Jacko JA (ed) Human-computer interaction. New trends.
Springer Berlin Heidelberg, Berlin, pp 149–158
62. Rego P, Moreira PM, Reis LP (2010) Serious games for rehabilitation: a survey and a classification towards
a taxonomy. In: Information Systems and Technologies (CISTI), 2010 5th Iberian Conference on, pp 1–6
63. Renard Y, Lotte F, Gibert G et al (2010) OpenViBE : an open-source software platform to design , test , and
use brain – computer interfaces in real and virtual environments. Presence 19:35–53. https://doi.org/10.1162
/pres.19.1.35
64. Sawyer B, Smith P (2008) Serious games taxonomy. In: Slides from the serious games summit at the game
developers conference
65. Schoneveld EA, Malmberg M, Lichtwarck-Aschoff A et al (2016) A neurofeedback video game
(MindLight) to prevent anxiety in children: a randomized controlled trial. Comput Hum Behav 63:321–
333. https://doi.org/10.1016/j.chb.2016.05.005
66. Schwartz MS, Andrasik F (2017) Biofeedback: a practitioner’s guide. Guilford Publications, New York
67. Shaphiro SS, Wilk MB (1965) An analysis of variance test for normality. Biometrika 52:591–611
68. Sitaram R, Ros T, Stoeckel L et al (2017) Closed-loop brain training: the science of neurofeedback. Nat Rev
Neurosci 18:86–100. https://doi.org/10.1038/nrn.2016.164
69. Southam-Gerow MA, Kendall PC (2002) Emotion regulation and understanding: implications for child
psychopathology and therapy. Clin Psychol Rev 22:189–222. https://doi.org/10.1016/S0272-7358(01
)00087-3
70. Spence C (2011) Crossmodal correspondences: a tutorial review. Atten Percept Psychophysiol 73:971–995.
https://doi.org/10.3758/s13414-010-0073-7
71. Steiner NJ, Frenette EC, Rene KM et al (2014) Neurofeedback and cognitive attention training for children
with attention-deficit hyperactivity disorder in schools. J Dev Behav Pediatr 35:18–27. https://doi.
org/10.1097/DBP.0000000000000009
72. Strauss A, Corbin J (1998) Basics of qualitative research techniques. Sage publications, Thousand Oaks
73. Student (1908) The probable error of a mean. Biometrika 6:1–25
74. Tan D, Nijholt A (2010) Brain-computer interfaces and human-computer interaction. In: Brain-computer
interfaces. Springer, pp 3–19
75. Tang ST, McCorkle R (2002) Use of family proxies in quality of life research for cancer patients at the end
of life: a literature review. Cancer Investig 20:1086–1104
76. Tang YY, Posner MI (2014) Training brain networks and states. Trends Cogn Sci 18:345–350. https://doi.
org/10.1016/j.tics.2014.04.002
77. Turkay S, Kinzer CK (2014) The effects of avatar-based customization on player identification. Int J
Gaming Comput Simulations 6:1–25. https://doi.org/10.4018/ijgcms.2014010101
78. van Steensel FJ, Bögels SM, Perrin S (2011) Anxiety disorders in children and adolescents with autistic
spectrum disorders: a meta-analysis. Clin Child Fam Psychol Rev 14:302–317. https://doi.org/10.1007
/s10567-011-0097-0
79. Vernon D, Egner T, Cooper N et al (2003) The effect of training distinct neurofeedback protocols on aspects
of cognitive performance. Int J Psychophysiol 47:75–85. https://doi.org/10.1016/S0167-8760(02)00091-0
80. Wang Q, Sourina O, Nguyen MK (2010) EEG-based Bserious^ games design for medical applications. Proc
- 2010 Int Conf Cyberworlds, CW 2010, pp 270–276. https://doi.org/10.1109/CW.2010.56
81. Whyte EM, Smyth JM, Scherf KS (2015) Designing serious game interventions for individuals with autism.
J Autism Dev Disord 45:3820–3831. https://doi.org/10.1007/s10803-014-2333-1
82. Wilcoxon F (1946) Individual comparisons of grouped data by ranking methods. J Econ Entomol 39:269.
https://doi.org/10.1093/jee/39.2.269
83. Wilkinson P (2016) A brief history of serious games Phil. Entertain Comput Serious Games 9970, pp 17–
41. https://doi.org/10.1007/978-3-319-46152-6
84. Wilkinson N, Ang RP, Goh DH (2008) Online video game therapy for mental health concerns: a review. Int
J Soc Psychiatry 54:370–382
85. Yoon H, Park S-W, Lee Y-K, Jang J-H (2013) Emotion recognition of serious game players using a simple
brain computer interface. 2013 Int Conf Inf Commun Technol Converg ICTC 2013, Oct 14, 2013 - Oct 16,
2013, pp 783–786. https://doi.org/10.1109/ICTC.2013.6675478
86. Zalapa R, Tentori M (2013) Movement-based and tangible interactions to offer body awareness to children
with autism. In: Urzaiz G, Ochoa SF, Bravo J et al (eds) Ubiquitous computing and ambient intelligence.
Context-awareness and context-driven interaction. Springer International Publishing, Cham, pp 127–134
87. Zickefoose S, Hux K, Brown J, Wulf K (2013) Let the games begin: a preliminary study using attention
process training-3 and lumosity™ brain games to remediate attention deficits following traumatic brain
injury. Brain Inj 27:707–716. https://doi.org/10.3109/02699052.2013.775484
88. Zoefel B, Huster RJ, Herrmann CS (2011) Neurofeedback training of the upper alpha frequency band in
EEG improves cognitive performance. Neuroimage 54:1427–1431. https://doi.org/10.1016/j.
neuroimage.2010.08.078
Jose Mercado , is a Ph. D. student in the Computer Science Department at the Center for Scientific Research and
Higher Education (CICESE), with a background in Computer System Engineering, his research interest are
Human-Computer Interaction, Brain-Computer Interfaces, Ubiquitous Computing, Vision-based Techniques and
Pattern Recognition and Classification. Contact him at jmercado@cicese.edu.mx
Ismael Espinosa-Curiel , is an associate professor in CICESE-UT3, Tepic, Nayarit. His research interest are
Human- Computer Interaction, Software Engineering, Agile Software Development and IT innovation. Contact
him at ecuriel@cicese.edu.mx
Lizbeth Escobedo , is an assistant professor in Cetys Universidad, Tijuana, in the School of Engineering where
she teaches, and researches how humans interact with their environment to design, implement and evaluate the
impact of technology that mainly helps people with cognitive disabilities, their families and caregivers, thus
raising their quality of life. Her research interests are Human-Computer Interaction (HCI), Ubiquitous Comput-
ing, Cognitive Assistive Technology (CAT), and Mobile Technologies. Contact her at
lizbeth.escobedo@cetys.mx
Monica Tentori , is an associate professor in CICESE, where she investigates the human experience of
ubiquitous computing to inform the design of ubiquitous environments that effectively enhance humans’
interactions with their world. Tentori was the first latin American woman in receiving the Microsoft Faculty
Fellowship. Contact her at mtentori@cicese.mx

Developing and Evaluating A BCI Video Game For Neurofeedback Training: The Case of Autism

Uploaded by

Copyright:

Available Formats

Developing and Evaluating A BCI Video Game For Neurofeedback Training: The Case of Autism

Uploaded by

Document Information

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Developing and Evaluating A BCI Video Game For Neurofeedback Training: The Case of Autism

Uploaded by

Copyright:

Available Formats

Multimedia Tools and Applications (2019) 78:13675–13712

Developing and evaluating a BCI video game

Received: 10 May 2018 / Revised: 3 October 2018 / Accepted: 19 November 2018 /

Keywords Autism . Brain-computer Interface . Neurofeedback . Video game . BCI . Attention

The main contributions of this work are

2.1 Traditional brain training: Neurofeedback

2.2 Gamifying the neurofeedback training

2.3 BCI video games for neurofeedback training

In this section, we describe the user-centered research process we followed to design

3.1 Research process

Table 1 Data collected and activities conducted to design FarmerKeeper

# Data Purpose Participants

1 Neurofeedback training To understand how neurofeedback 1 HCI expert

3.2 Neurofeedback training in autism – A prototypical lesson

(Wearing the electrodes or a BCI headset)

Fig. 1 Diagram of phases on a neurofeedback training session

the neurofeedback trainer, configures on the neurofeedback training application three

3.3.1 Catalyze engagement

3.3.2 Mimic current strategies on prompting

3.3.3 Promote a good posture

4.1 Game mechanics and features

4.2 Scenario of use

4.3 FarmerKeeper’s architecture

– OpenVibe < <executable>>: this component is an open-source software platform dedicat-

<<PC Host>> << BCI headset>>

Fig. 5 FarmerKeeper’s architecture

To evaluate FarmerKeeper, we conducted a within-subjects study at Pasitos. The study aims to

– Does FarmerKeeper outperforms in terms of usability and user experience a commercial

5.2 Setting up and installation

5.3.1 Sensitization process

5.3.2 Neurofeedback training sessions

5.3.3 Data collection2

5.3.4 Data analysis

Category Target behavior Definition Type of

6.2 User experience

6.2.1 User experience according to the UEQ survey

Fig. 9 Results from UEQ compared to the Benchmarks

Table 3 UEQ results

Categories BrainCats FarmerKeeper

Mean STD N Confidence Confidence Mean STD N Confidence Confidence

6.2.3 Use and adoption

Fig. 13 Results from Fun Sorter

6.3 Potential benefits

6.3.1 Attention during the session

6.3.2 Attention during activity phases

7 Conclusions and future work

Compliance with ethical standards

Conflict of interest Authors declare that they have no conflict of interest.

1. Aamodt S, Wang S (2007) Exercise on the brain. New York Times

You might also like

Pfad - The Proxy pFad of © 2024 Garber Painting. All rights reserved.