Restoration

of Whole Body
Movement
Toward a Noninvasive
Brain–Machine
Interface System

© IMAGESTATE

Digital Object Identifier 10.1109/MPUL.2011.2175635

Date of publication: 6 February 2012

34 IEEE PULSE ▼ JANUARY/FEBRUARY 2012 2154-2287/12/$31.00©2012 IEEE

 By José L. Contreras-Vidal, Alessandro Presacco,
Harshavardhan Agashe, and Andrew Paek

T
his article highlights recent advances in the design of National Limb Loss Information Center. At least 1,600 of them
noninvasive neural interfaces based on the scalp elec- are U.S. veterans wounded in Iraq and Afghanistan, and their
troencephalogram (EEG). The simplest of physical tasks, rehabilitation is a priority of the Defense Advanced Research
such as turning the page to read this article, requires an Projects Agency (DARPA), the military’s research and develop-
intense burst of brain activity. It happens in milliseconds ment entity.
and requires little conscious thought. But for amputees Our team is in the preliminary stages of pairing EEG-based
and stroke victims with diminished motor-sensory skills, this findings with the ongoing research at APL in Laurel, Mary-
process can be difficult or impossible. land. Here, a team of engineers and medical experts are work-
Our team at the University of Maryland, in conjunction ing on the DARPA-funded modular prosthetics limb (MPL),
with the Johns Hopkins Applied Physics Laboratory (APL) and the next-generation robotic arm that functions like a normal
the University of Maryland School of Medicine, hopes to offer limb. The APL team is building the arm, while our group is
these people newfound mobility and dexterity. In separate re- working on one of the options for the control system using
search thrusts, we’re using data gleaned from scalp EEG to de- our EEG-based decoders. Our team members come from back-
velop reliable brain–machine interface (BMI) systems that could grounds in bioengineering, electrical engineering, neurosci-
soon control modern devices such as prosthetic limbs or powered ence, kinesiology, as well as the university’s program in neu-
robotic exoskeletons. roscience and cognitive science.
EEG externally measures electrical ac- To design the neural interfaces, we
tivity generated by large neural networks use EEG to acquire brain signals that will
in the brain, and research in our labo- be fed to neural decoders (filters) that
ratory was the first to demonstrate the translate them into control signals for
feasibility of inferring voluntary natural driving the dexterous finger movements
movement from EEG signals, essentially of the robotic arm. The filters are de-
decoding human brain activity used for signed based on the data collected during
physical movement (Figure 1). While a short calibration period that involves
similar but invasive neural interface tech- imagined and observed movement.
nology under development allows users to Though EEG monitoring is safer
think commands that are sent to sophisti- than other approaches, many in the
cated upper- or lower-limb prosthetics or scientific community had deemed it
used to control computer cursors, we re- unreliable for a brain–computer in-
cently reported the first EEG-based neu- terface, mainly because they believed
ral interface (needing only a single train- that the human skull blocks much of
ing session before subjects can operate the detailed brain activity needed for
it) that employs continuous decoding of precision-controlled prosthetics. An
imagined, continuous hand movements. article we published in Journal of Neu-
roscience [1] questioned that premise,
FIGURE 1 A noninvasive EEG-based neu-
Noninvasive Neural Decoding ral interface is easier to repair or replace, showing that we could capture and de-
of Movement if needed, and the technology is very code three-dimensional hand motions
user friendly requiring only a fabric cap from the amplitude modulations of the
Decoding of Upper-Limb Movements and the slight inconvenience of some smoothed EEG signals in the lower fre-
goo on a person’s head where the sen-
There are more than 1.8 million ampu- sors are attached. (Photograph by John quency delta band (<4 Hz) emanating
tees in the United States according to the T. Consoli, University of Maryland.) from the scalp. Our method produces

JANUARY/FEBRUARY 2012 ▼ IEEE PULSE 35

 Decoding of Human Bipedal Locomotion
r2 In the work presented at the 2010 Neural Engineering, Science,
0.21 and Technology Forum sponsored by DARPA, we showed that
the linear and angular movements of the ankle, knee, and hip
joints during human treadmill walking can be inferred from
high-density scalp EEG with decoding accuracies comparable to
that from similar studies of nonhuman primates with electrodes
implanted in their brains [3], [4] (Figure 2).
We like to think that our interface will come in two flavors.
One is for the restoration of function and includes robotics work
0
with the Johns Hopkins laboratory. The other is for rehabilita-
tion, specifically for stroke victims with brain injuries that affect
FIGURE 2 A representative scalp map of the spatial distribution
(r 2) of decoding accuracies across scalp electrodes for the joint motor-sensory control or patients with spinal cord injury (SCI).
angle of the right ankle joint in a healthy individual. Note the There is a big push in brain science to understand what exercise
sparse network, in the EEG sensor space, that contains informa- does in terms of motor learning or motor retraining of the hu-
tion about the ankle position.
man brain.
In the past year, we have partnered with the colleagues in
data comparable to almost any invasive method—data that can the Department of Physical Therapy and Rehabilitation at the
be used to drive complex robotic devices like the MPL. University of Maryland School of Medicine in Baltimore. The re-
To get these data, we instructed subjects to randomly select search, funded in part by the National Institutes of Health and a
and reach eight different targets on a computer screen. Hori- seed grant program between the University of Maryland and the
zontal, vertical, and depth velocities were then recorded, and University of Maryland, Baltimore, has tracked the neural activ-
EEGs were used to predict their trajectories. Also, standardized ity of people in motion.
low-resolution brain electromagnetic tomography (sLORETA) Subjects were instructed to walk for five minutes on a tread-
showed that distributed current density sources related to hand mill, while EEG and kinematic parameters, such as Cartesian
velocity are concentrated in the contralateral precentral gyrus, positions, joint angle, and angular velocity, were recorded. The
postcentral gyrus, and inferior parietal lobule regions of the EEG and kinematics signals were filtered offline (in the delta
brain. These brain areas are typically associated with sensorimo- lower frequency band), and a linear decoder (a Wiener filter
tor performance. with memory—signals up to 100 ms in the past were used as
In subsequent articles and presentations, we provided new inputs to the decoder) trained offline to find the optimal weights
validation of the effectiveness of EEG signals. An article in Jour- to decode the kinematic parameters [4] (Figure 3).
nal of Neural Engineering [2] demonstrated that people using a Once again, we are intent on matching specific brain activ-
noninvasive EEG-based brain–computer interface, with mini- ity recorded in real time with exact intended lower-limb move-
mal training, were able to control a computer cursor with perfor- ments. We believe that these data could help stroke victims in
mance comparable to that of invasive brain–computer interfaces several ways. One is a prosthetic device called an anklebot or
using implanted electrodes. ankle robot that currently stores data from a normal human gait
and assists partially paralyzed people. We are working to aug-
ment the anklebot with our BMI to actively include the patient
in the control loop, thereby enhancing cortical plasticity. Provid-
15 ing movement feedback to the patient is also important for reha-
10 bilitation, and we are investigating ways to reflect motion and
contact signals from the anklebot back to the patient. People who
5 are less mobile commonly suffer from other health issues such
Joint Angle (°)

as obesity, diabetes, or cardiovascular problems, so rehabilitation

0
experts normally want to get stroke survivors up and moving by
–5 whatever means possible.
The second use of the EEG data in stroke victims is more com-
–10
plex but still offers exciting possibilities. By decoding the motion
–15 Actual of a normal gait, rehabilitation experts can try and teach stroke
Predicted
victims to think certain ways and match their own EEG signals
–20
1 2 3 4 5 6 7 8 with the normal signals. This could retrain healthy areas of the
Time (s) brain through neuroplasticity.
In addition to improving the rehabilitation of stroke pa-
FIGURE 3 Decoded ankle joint kinematics of a healthy able- tients, our BMIs could be used to control lower-limb powered
bodied participant using a linear decoder. Blue is the joint angle
exoskeletons to restore walking after SCI. In this scenario,
recorded during treadmill walking, and red is the joint angle
of the ankle that we predicted using our EEG-based neural EEG decoders are calibrated using visual perception of un-
interface. steady locomotion such as an observation of an avatar turning

36 IEEE PULSE ▼ JANUARY/FEBRUARY 2012

FIGURE 4 Noninvasive BMI systems for the control of walking av-
atars that could soon control sophisticated prosthetic devices.
Shown are bioengineering student Steve Graff (foreground,
with cap), kinesiology doctoral student Alessandro Presacco
FIGURE 5 Demonstration of the Rex-powered robotic exoskel-
(center), and lead researcher and electrical engineer José L.
eton (Rex Bionics) by a wheelchair user wearing a wireless
Contreras-Vidal, Ph.D (background). (Photograph by John T.
64-channel active EEG cap used in our neural interface devel-
Consoli, University of Maryland.)
opment. Shown are the time series of the user’s brain waves
as recorded by the EEG cap. From left: Faisal Almesfer, Steve
during walking, stepping up or down stairs, and so on (Fig- Holbert, and Jedy Shishbaradaran. (Photograph by Joy Wilson,
ure 4). These decoders could then be used to control smart Department of Health and Human Performance, University
of Houston.)
powered lower-limb exoskeletons fitted to the SCI patients to
regain walking capabilities. It is likely that a shared control ap-
proach that allows human–machine confluence will be critical supported in part by National Institute of Neurological Disorders
to minimize cognitive effort and achieve multitasking capabili- and Stroke under award number R01NS075889, National Sci-
ties (Figure 5). ence Foundation under award no IIS-1064703, the University
of Maryland at College Park–University of Maryland at Balti-
Conclusions more Seed Grant Program, and the National Academies Keck
We strongly believe that our laboratory is on track to develop, Future Initiative on Smart Prosthetics.
test, and make available to the public (within the next few years)
safe, reliable, and noninvasive BMI to robotic systems that can José L. Contreras-Vidal (pepe.contreras@ee.uh.edu), Andrew
bring life-changing technology to millions of people who have Paek (apaek@umd.edu), and Harshavardhan Agashe (agashe@
difficulty generating uninhibited movement. umd.edu) are with the Department of Electrical and Computer Engi-
To accomplish this vision, our team is now pursuing new neering, University of Houston, Texas. Alessandro Presacco (apre-
meaningful partnerships with The Methodist Hospital Re- sacc@umd.edu) is with the Department of Kinesiology, University of
search Institute (Houston, Texas) and the Department of Elec- Maryland, College Park, Maryland.
trical and Computer Engineering at the University of Houston
(UH)—the new home for our Laboratory for Noninvasive BMI References
Systems. We are collaborating with Rex Bionics, innovators in [1] T. J. Bradberry, R. J. Gentili, and J. L. Contreras-Vidal, “Recon-
advanced technology robotics, to provide a noninvasive neu- structing three-dimensional hand movements from noninvasive
rological interface to their independent robotic exoskeleton electroencephalographic signals,” J. Neurosci., vol. 30, no. 9, pp.
[Rex (www.rexbionics.com), see Figure 5]. This partnership 3432–3437, 2010.
will demonstrate the capability to achieve thought-controlled [2] T. J. Bradberry, R. J. Gentili, and J. L. Contreras-Vidal, “Fast at-
(hands free) exoskeleton use and prove medical benefits and tainment of computer cursor control with noninvasively acquired
quality of life enhancements realized by exoskeleton users. brain signals,” J. Neural Eng., vol. 8, no. 5, pp. 1–9, 2011.
We will be teaming with our clinical partners at Methodist
and engineers at UH to join efforts to conduct clinical trials [3] N. A. Fitzsimmon, M. A. Lebedev, I. D. Peikon, and M. A. Nicolel-
of BMI to robotic systems to assess safety and efficacy, and is, “Extracting kinematic parameters for monkey bipedal walking
use our noninvasive neural interfaces as tools for reverse- from cortical neuronal ensemble activity,” Front. Integr. Neurosci.,
translational studies of brain plasticity and human–machine vol. 3, pp. 1–19, 2009.
interaction/confluence. [4] A. Presacco, R. Goodman, L. Forrester, and J. L. Contreras-Vidal,
“Neural decoding of treadmill walking from noninvasive elec-
Acknowledgments troencephalographic signals,” J. Neurophysiol., vol. 106, no. 4, pp.
The University of Maryland Communications Specialists Lee 1875–1887, 2011.
Tune and Tom Ventsias contributed to this article. This work was

JANUARY/FEBRUARY 2012 ▼ IEEE PULSE 37