Cognition For Human Robot Interaction - Spectra 2014

NRL FEATURES
for
HUMAN-ROBOT INTERACTION
Scientists from the Naval Research Laboratorys Navy Center for Applied Research in Artificial Intelligence demonstrate advances in
cognitive science, cognitive robotics, and humanrobot interaction with the help of teammate Octavia (center). Left to right: Ed Lawson,
Greg Trafton, Laura Hiatt, Sunny Khemlani, Bill Adams, Priya Narayanan, Frank Tamborello, Tony Harrison, Magda Bugajska.
Our overarching goal is to give

robots a deep understanding
of how people think. There are
three benefits of this to the scientific community. First, by improving

understanding of how people think
and interact with the world, we are
pushing the boundaries of the field
of cognitive science. Second, we can
leverage this knowledge of people to
help robots be better at tasks they
typically are unable to perform well,
such as multimodal communication
and computer vision. Third, this deep
understanding of humans allows
robots to better predict and understand a human partners behavior
and, ultimately, be a better and more
helpful teammate.
To accomplish this, we build process models of fundamental human
cognitive skills perception, memory,
attention, spatial abilities, and theory
of mind and then use those models
as reasoning mechanisms on robots
and autonomous systems. Most of
our work is done using the computa-
10
tional cognitive architecture ACT-R/E.

A cognitive architecture is a processlevel theory about human cognition.
It provides a rich environment for
modeling and validating different
hypotheses about how people think.
ACT-R/E provides a set of computational modules, correlated with different functional regions of the brain,
that work together to explain both
the limitations of human cognition
(i.e., we dont have unlimited working memory) and the strengths (i.e.,
we are good at inferring connections
between concepts and interacting
with the physical world):
Declarative Module: Manages
the creation and storage of factual
knowledge; selects what chunks
(facts or memories) will be thought
about at any given time.
Procedural Module: Manages the
creation and storage of procedural
knowledge, or chunks; selects
what production (if-then rule) will
fire at any given time.
Intentional and Imaginal

Modules: Provide support for goaloriented cognition and intermediate
problem state representations.
Visual and Aural Modules: Enable
the architecture to see and hear
perceptual elements in the models
world.
Configural and Manipulative
Modules: Enable the architecture
to spatially represent perceptual
elements in the models world.
Temporal Module: Allows the
architecture to keep track of time;
acts as a noisy metronome.
Motor and Vocal Modules: Provide functionality for the model to
move and speak with an appropriate time course.
Each module, save the procedural
module, is associated with a limitedcapacity buffer, representing what the
model is thinking about / planning to
SPECTRA
NRL FEATURES
say / looking at / etc. Together, the
contents of these buffers make up
working memory in ACT-R/E. While
each of the modules and buffers is
theoretically motivated and validated
on its own, ACT-R/Es strength lies
in the complex interaction of these
components, shown below.
would allow the memory to become

familiar enough to be remembered.
ACT-R/Es fidelity to the way the
human mind integrates, stores, and
retrieves all this information is what
provides its true predictive and explanatory power. In this case, model-
The ACT-R/E architecture. The complex interaction of ACT-R/Es many

theoretically motivated and validated components provides its true predictive and
explanatory power.
To illustrate, consider an example

of a conference attendee meeting
someone at the conference and then
attempting to remember his or her
name later in the evening. The attendee would need to not only attend
to the new persons face, voice, and
clothing, but also bind those individual concepts to the name of the
individual. The attendee would then
need to rehearse the name several
times so it would not be forgotten.
When the attendee saw the new person a bit later, he or she would need
to take available cues (perhaps only
visual cues like face and clothing)
and attempt to retrieve from memory
the name associated with those
cues. Priming from contextual cues,
like the face, clothing, and the party
itself, would boost the activation of
the memory, and the earlier rehearsal
WINTER 2014
ing the humans experiences at the

conference provides both descriptive information about how a human
could err in this situation (e.g., people
that look similar, or that were met at
similar times or similar parties, may
be easily confused) and predictive
information (e.g., someones name
is unlikely to be remembered after a
long break without further rehearsal
or strong contextual cues).
In this article, we present two different systems in which ACT-R/Es fidelity to human cognition is leveraged
to develop intelligent systems that
can more functionally assist a human
teammate. In the first example, ACTR/Es familiarity and context mechanisms help an autonomous system
better perceive ambiguous or obfuscated objects that a robot might
see in the real-world environments

our warfighters encounter (such as
jungles, deserts). In the second, ACTR/Es goal structure, together with its
familiarity and context mechanisms,
allow an intelligent system to help
humans avoid certain classes of errors, such as those commonly made
in Navy vehicle maintenance procedures.
Computer Vision
Recent work in computer vision,
performed by Hoiem, Efros, and Herbert in 2006, and Oliva and Torralba
in 2007, among others, has shown
that including contextual information
can greatly affect the efficiency of
object recognition in terms of both
the speed and the accuracy of the
processing process. These existing
computer vision approaches, however, typically rely on static, aggregated
statistics of various features in the
scene to provide their context. While
such approaches are promising, this
limitation renders them unable to
perform competitively with human
perception; a richer, more human-like
representation of contextual inference, learned over time, like that in
ACT-R/E, may be the key to their
success.
Context in ACT-R/E takes the form
of associations between related
concepts that are learned over time.
Concepts become associated when
they are thought about at roughly the
same time; the more they are thought
about in proximity to each other, the
stronger their association becomes.
This type of context is rich, in that it
can capture nuanced relationships
between concepts and facts that
are not explicitly linked; this context
is also dynamic, in that it is learned
online and adjusted over time based
on the models experience. Object
recognition algorithms, like LVis, a
biologically plausible object recognition system developed by Randall
OReilly at the University of Colorado
Boulder, can utilize this information to
improve recognition in cases where
using visual features alone is difficult.
11
NRL FEATURES
For example, when looking in the
kitchen, context may suggest related
concepts such as oranges or lemons.
Any ambiguities that might arise from
other similar objects (such as an orange ball) can be quickly resolved by
incorporating contextual information,
resulting in the correct identification.
Our initial experiments using a large
database of objects have shown that
a system that combines context and
LViss object recognition algorithms is
able to increase recognition accuracy
as compared to using LVis without
context (see below). While these
results are simple, they shed light on
the powerful tool that context can be,
and demonstrate how it can be used
to build to autonomous systems that
are better able to support and extend
the Navys capabilities.
Sample objects from the RGB-D

dataset developed by Lai et al. at
the University of Washington. The
objects in the dataset are relatively
low resolution, similar to what a
robot would see as it moves around
the world. Here, a battery (left) and
dry eraser (right) can potentially
have very similar visual outlines and
contours, but are typically found in
different situations; context can help
differentiate between these two object
classes.
Error Prediction
With the rapid rise of communication technologies that keep people
accessible at all times, issues of
interruptions and multitasking have
become mainstream concerns. For
example, the New York Times in 2005
and Time magazine in 2006 both
reported stories about interruptions
and multitasking, and how they affect
performance by increasing human error. In 2005, the information technology research firm Basex estimated
the economic impact of interruptions
to be around $588 billion a year
due to losses from increased task
12
One of our robots, Octavia, correctly recognizes an orange ball when it is in context.
Octavia is a humanoid robot we use to investigate how to build intelligent systems
that can serve as better teammates to their human partners.
completion time or errors. Given the

prevalence of interruptions, building systems that can help remind an
individual what they were doing or
where they were in a task can have a
large impact on individual and group
productivity.
We built an ACT-R/E model that
emulates the process people go
through as they get interrupted
during a procedural task and then
have to resume it. As ACT-R/E progresses through a task, it maintains a

representation of where it is currently
engaged in the task. For familiar
tasks, this representation always associates, via context, an action to the
next action to be performed. When a
task is interrupted, the model loses
track of its goal and, upon resumption of the task, must remember what
it was working on. Sometimes, contextual cues associate to future steps
SPECTRA
NRL FEATURES
beyond the next correct one, and
the system skips a step. Alternately,
sometimes the familiarity of past actions surpasses the familiarity of the
current action, and ACT-R/E remembers an action prior to the last one it
completed, and repeats a step.
Using ACT-R/E in this way, we can
explain (and thus can better predict)
how, when interrupted, people tend
to skip or repeat steps, even in a
familiar task, based on the tasks
context and strengthening. This accomplishment will help us develop
intelligent systems that can mitigate
human error risks in dangerous
procedures, with both monetary and
functional benefits.
Conclusion
Our approach to intelligent systems
is multidimensional. First, in the
cognitive science tradition, we attain
a deep understanding of how people
think: our ACT-R/E models faithfully
capture peoples behavior as they
perceive, think about, and act on the
world around them. Second, we use
this understanding to make intelligent
systems better by taking advantage
of peoples strengths. Finally, the
models help to reveal limitations and
potential failings of human cognition, which our intelligent systems
can then take steps to correct.
Overall, our efforts to attain a deep
understanding of human cognitive
strengths and limitations allow us
to build more functional intelligent
systems that are better able to serve
their human teammates.
By Laura M. Hiatt, Frank P. Tamborello, II,
Wallace E. Lawson, and J. Gregory Trafton
Navy Center for Applied Research in
Artificial Intelligence
N av y Ce n t e r fo r
A p p l i e d R es e a r c h i n
A rt i f i c i a l I n t e l l i g e n c e
Navy Center for Applied Research

in Artificial Intelligence (NCARAI) has
The
been involved in basic and applied research in

artificial intelligence, human factors, and humancentered computing since its inception in 1981. It
is part of the Information Technology Division of
the Naval Research Laboratory and is directed by
Alan Schultz, who is also director of the Laboratory for Autonomous Systems Research.
The Adaptive Systems group conducts research
focused on techniques that allow systems to change
their level of autonomy on the fly, adapt to changes in
their environment and in their own capabilities, learn new
behaviors through interaction with the world, and interact
more naturally with humans.
The Intelligent Systems group performs research in
cognitive science, cognitive robotics and human-robot
interaction, predicting and preventing procedural errors,
the cognition of complex visualizations, interruptions and
resumptions, and spatial cognition, to enable intelligent
systems that can work more effectively with people.
The Interactive Systems group develops and enhances
computer interfaces for autonomous and intelligent
systems, spanning human-computer and human-robot
interactions. Research includes linguistic, auditory, and
phonological analyses to link natural language to modes
of human-machine interaction.
The Perceptual Systems group examines issues related
to sensors required to support autonomous platform navigation, scene interpretation, and teamwork. Techniques
include passive monocular vision, and passive and active
stereo and triocular ranging methods, along with algorithms to rapidly interpret the scene.
http://www.nrl.navy.mil/itd/aic/
WINTER 2014
13

Cognition For Human Robot Interaction - Spectra 2014

Uploaded by

Copyright:

Available Formats

Cognition For Human Robot Interaction - Spectra 2014

Uploaded by

Document Information

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Cognition For Human Robot Interaction - Spectra 2014

Uploaded by

Copyright:

Available Formats

NRL FEATURES

Our overarching goal is to give

three benefits of this to the scientific community. First, by improving

tional cognitive architecture ACT-R/E.

Intentional and Imaginal

would allow the memory to become

The ACT-R/E architecture. The complex interaction of ACT-R/Es many

To illustrate, consider an example

ing the humans experiences at the

see in the real-world environments

Sample objects from the RGB-D

completion time or errors. Given the

have to resume it. As ACT-R/E progresses through a task, it maintains a

Navy Center for Applied Research

been involved in basic and applied research in

You might also like

Pfad - The Proxy pFad of © 2024 Garber Painting. All rights reserved.