Neurol Ther (2019) 8:351–365

https://doi.org/10.1007/s40120-019-00153-8

REVIEW

Deep Learning and Neurology: A Systematic Review

Aly Al-Amyn Valliani . Daniel Ranti . Eric Karl Oermann

Received: June 27, 2019 / Published online: August 21, 2019

Ó The Author(s) 2019

ABSTRACT connectome mapping for the diagnosis of Alz-

heimer’s, autism spectrum disorder, and atten-
Deciphering the massive volume of complex tion deficit hyperactivity disorder; and mining
electronic data that has been compiled by hos- of microscopic electroencephalogram signals
pital systems over the past decades has the and granular genetic signatures. We addition-
potential to revolutionize modern medicine, as ally note important challenges in the integra-
well as present significant challenges. Deep tion of deep learning tools in the clinical setting
learning is uniquely suited to address these and discuss the barriers to tackling the chal-
challenges, and recent advances in techniques lenges that currently exist.
and hardware have poised the field of medical
machine learning for transformational growth.
The clinical neurosciences are particularly well Keywords: Artificial intelligence; Biomedical
positioned to benefit from these advances given informatics; Computer vision; Connectome
the subtle presentation of symptoms typical of mapping; Deep learning; Genomics; Machine
neurologic disease. Here we review the various learning; Neurology; Neuroscience
domains in which deep learning algorithms
have already provided impetus for change—ar-
eas such as medical image analysis for the INTRODUCTION
improved diagnosis of Alzheimer’s disease and
the early detection of acute neurologic events; Twenty-first century healthcare is marked by an
medical image segmentation for quantitative abundance of biomedical data and the devel-
evaluation of neuroanatomy and vasculature; opment of high-performance computing tools
capable of analyzing these data. The availability
Enhanced digital features To view enhanced digital of data and increased speed and power of
features for this article go to https://doi.org/10.6084/ computer systems together present both
m9.figshare.9272951.
opportunities and challenges to researchers and
Aly Al-Amyn Valliani and Daniel Ranti contributed healthcare professionals. Most significantly,
equally to this article. they provide the potential to discover new dis-
ease correlates and translate these insights into
A. A.-A. Valliani
D. Ranti
E. K. Oermann (&) new data-driven medical tools that can improve
Department of Neurological Surgery, Mount Sinai the quality and delivery of care. However, such
Health System, 1 Gustave Levy Pl, New York, NY
10029, USA
advancements require the navigation of high-
e-mail: eric.oermann@mountsinai.org dimensional, unstructured, sparse, and often
352 Neurol Ther (2019) 8:351–365

incomplete data sources, with the outcomes towards the development of novel diagnostic
being cumbersome to track. Identifying novel and prognostic tools. Deep learning techniques
clinical patterns amidst this complexity is defi- are particularly promising in the neurosciences
nitely not a trivial task [1–3]. where clinical diagnoses often rely on subtle
Modern representation learning methods symptoms and complicated neuroimaging
enable the automatic discovery of representa- modalities with granular and high-dimensional
tions needed to generate insights from raw data signals. In this article, we discuss the applications
[4]. Deep learning algorithms are an example of of deep learning in neurology and the ongoing
such representation learning approaches that challenges, with an emphasis on aspects relevant
hierarchically compose nonlinear functions to to the diagnosis of common neurologic disor-
transform raw input data into more sophisti- ders. However, our aim is not to provide com-
cated features that enable the identification of prehensive technical details of deep learning or
novel patterns [5]. Such approaches have its broader applications. We begin with a brief
proved to be essential in modern engineering overview of deep learning techniques followed
breakthroughs—from face recognition and self- by a review of applications in the clinical neuro-
driving cars to chat-bots and language transla- science field. We conclude the review with a
tion [6–12]. In medicine, the successful appli- short discussion on existing challenges and a
cation of deep learning algorithms to routine look to the future. This article is based on previ-
tasks has enabled a flood of academic and ously conducted studies and does not contain
commercial research, with publications on var- any studies with human participants or animals
ious applications growing from 125 published performed by any of the authors.
papers identified as machine learning publica-
tions in arXiv, the electronic scientific and
engineering paper archive, in 2000, to more FUNDAMENTALS OF DEEP
than 3600 by November of 2018 (see Fig. 1). LEARNING
The multidiscipline of clinical neurosciences
has similarly experienced the beginnings of an Machine learning is a subset of artificial intelli-
impact from deep learning, with movement gence that learns complex relationships among

Fig. 1 Machine learning publications in PubMed by year through 2018 showing the exponential growth of interest in the
field, as reported by the US National Library of Medicine of the National Institutes of Health [13]
Neurol Ther (2019) 8:351–365 353

variables in data [14]. The power of machine techniques often require manual feature engi-
learning comes from its ability to derive pre- neering and suffer from overhead that limits
dictive models from large amounts of data with their utility in scenarios that require near real-
minimal or, in some cases, entirely without the time decision-making.
need for prior knowledge of the data or any Deep learning differs from traditional
assumptions about the data. One of the most machine learning in how representations are
widely discussed modern machine learning automatically discovered from raw data. In
algorithms, the artificial neural network (ANN), contrast to ANNs, which are shallow feature
draws inspiration from biological neural net- learning techniques, deep learning algorithms
works that constitute mammalian brains. The employ multiple, deep layers of perceptrons
functional unit of the ANN is the perceptron, that capture both low- and high-level repre-
which partitions input data into separable cat- sentations of data, enabling them to learn richer
egories or classes [15]. When hierarchically abstractions of inputs [5]. This obviates the need
composed into a network, the perceptron for manual engineering of features and allows
becomes an essential building block for modern deep learning models to naturally uncover pre-
deep neural networks (DNNs), such as multi- viously unknown patterns and generalize better
layer perceptron classifiers. Similar examples of to novel data. Variants of these algorithms have
commonly used traditional machine learning been employed across numerous domains in
algorithms include linear regression (LR), logis- engineering and medicine.
tic regression, support vector machines (SVMs), Convolutional neural networks (CNNs) have
and the Naı̈ve Bayes classifier (Fig. 2). garnered particular attention within computer
These traditional machine learning methods vision and imaging-based medical research
have been important in furthering advance- [17, 18]. CNNs gather representations across
ments in medicine and genomics. As an exam- multiple layers, each of which learns specific
ple, LR has proven useful in the search for features of the image, much like the human
complex, multigene signatures that can be visual cortex is arranged into hierarchical layers,
indicative of disease onset and prognosis, tasks including the primary visual cortex (edge
which are otherwise too intricate and cumber- detection), secondary visual cortex (shape
some even for researchers with professional detection), and so forth [19]. CNNs consist of
training [16]. Although such tools have been convolutional layers in which data features are
very effective in parsing massive datasets and learned: pooling layers, which reduce the
identifying relationships between variables of number of features, and therefore computa-
interest, traditional machine learning tional demand, by aggregating similar or

Fig. 2 Breakdown of algorithm types in the machine learning family that are commonly used in medical subdomain
research and analyses
354 Neurol Ther (2019) 8:351–365

redundant features; dropout layers, which of data. As the two networks engage in this
selectively turn off perceptrons to avoid over- adversarial process, the fidelity of the generated
reliance on a single component of the network; data gradually improves. In some contexts, the
and a final output layer, which collates the resulting data have been utilized to augment
learned features into a score or class decision, existing datasets [29].
i.e., whether or not a given radiograph shows These strides in deep learning are largely due
signs of ischemia. These algorithms have to breakthroughs in computing capabilities and
achieved rapid profound success in image clas- the open-source nature of research in the field.
sification tasks and, in some cases, have mat- The application of graphics processing units to
ched board-certified human performance deep learning research has dramatically accel-
[20–24]. erated the size and complexity of algorithm
Recurrent neural networks and variants, such architectures and simultaneously reduced the
as long short-term memory (LSTM) and gated time to train such algorithms from months to
recurrent units, have revolutionized the analy- the order of days. The consequence has been
sis of time-series data that can be found in high-throughput research characterized by
videos, speech, and texts [25]. These algorithms rapid experimentation, ultimately enabling
sequentially analyze each element of input data more efficacious algorithms. In addition, the
and employ a gating mechanism to determine rise of open-source deep learning frameworks,
whether to maintain or discard information such as TensorFlow, Keras, PyTorch, Caffe, and
from prior elements when generating outputs. others, has increased accessibility to technical
In this manner, they efficiently capture long- advances and facilitated the sharing of ideas
term dependencies and have revolutionized and their rapid application across various
machine translation, speech processing, and domains [30, 31]. The truly collaborative nature
text analysis. of deep learning research has led to surprising
Autoencoders (AEs) are a class of unsuper- innovations and changed the landscape of
vised learning algorithms that discover mean- medical research and care.
ingful representations of data by learning a
lower-dimensional mapping from inputs to
outputs [26, 27]. They are composed of an LITERATURE REVIEW
encoder, which learns a latent representation of
the input, and a decoder, which reconstructs In this article, we review and summarize pub-
the input from the latent representation. By lished literature on the application of deep
constraining the latent representation to a learning to the clinical neurosciences. We used
lower dimensionality than the input, AEs are search engines and repositories such as Google
able to learn a compressed representation of Scholar, PubMed, ScienceDirect, and arXiv to
data that contains only the features necessary to identify and review existing literature and per-
reconstruct the input. Such algorithms are often formed keyword searches of these databases
employed to learn features that can be subse- using the following terms: ‘‘deep learning,’’
quently utilized in conjunction with the deep ‘‘machine learning,’’ ‘‘neurology,’’ ‘‘brain,’’ and
learning techniques previously discussed. ‘‘MRI.’’ Following a comprehensive review of
Generative adversarial networks are a newer the literature initially retrieved, we identified
class of algorithms aimed at generating novel 312 articles as containing one or more keywords
data that statistically mimic input data by associated with our queries. Of these articles,
approximating a latent distribution for the data 134 were subsequently identified as being rele-
[28]. Such algorithms are composed of two vant to the subject of this review. Following
competing (‘‘adversarial’’) networks: a genera- collation of the relevant articles, we grouped
tor, which produces synthetic data from noise articles first into broad modalities, namely
by sampling from an approximated distribu- image classification, image segmentation,
tion, and a discriminator, which aims to dif- functional connectivity and classification of
ferentiate between real and synthetic instances brain disorders, and risk prognostication.
Neurol Ther (2019) 8:351–365 355

Within these areas, we then grouped publica- Among the first such applications, Suk and Shen
tions into disease applications. We focused our utilized a stacked AE to learn multimodal brain
discussion on the clinical implications of the representations from structural MRI and posi-
developments in the field. tron emission tomography (PET), and incorpo-
rated those features with cerebrospinal fluid
biomarker data and clinical scores from the
DEEP LEARNING IN NEUROLOGY Mini-Mental State Examination (MMSE) and
Alzheimer’s Disease Assessment Scale-Cognitive
The deep learning techniques described above subscale (ADAS-Cog) to train an SVM classifier
are playing an increasingly crucial role in neu- that improved diagnostic accuracy [34]. Other
rological research, tackling problems within approaches pre-trained a stacked AE using nat-
several subdomains. First, radiological image ural images (everyday images) prior to training
classification and segmentation has been a tra- on brain MR images in order to learn more
ditional locus of deep learning development high-fidelity anatomical features, such as gray
efforts. Image classification and segmentation matter and structural deformities, for incorpo-
tasks are uniquely suited to deep learning due to ration into a CNN [35]. Variations on these
the high-dimensional nature of neuroimaging approaches have been used to incrementally
data which is unfavorable to manual analysis, improve diagnostic performance [36–42].
combined with the naturally digital nature of Whereas older approaches were limited to
most modern imaging. Secondly, deep learning two-dimensional (2D) slices of medical images
has been applied to functional brain mapping due to computational constraints, newer appli-
and correlational studies using functional cations have been able to incorporate the full
magnetic resonance imaging (fMRI) data for 3D volume of an imaging modality for AD
tasks such as prediction of postoperative sei- detection. Among the first such examples was
zure. Lastly, diagnostic prognostication with work by Payan and Montana in which they
deep learning using multiple data types, trained a sparse AE on 3D patches of MRI scans
including lab values, images, notes, among to learn a volumetric brain representation that
others, has been used to assign disease risk. In was used to pre-train a 3D CNN for AD diagnosis
the following sections, we discuss the successes [43]. More recently, Hosseini-Asl et al. used an
and challenges inherent in the deep learning adaptable training regime with a 3D CNN pre-
approaches adopted towards these tasks, as well trained by a convolutional AE to learn general-
as the limitations and difficulties that such izable AD biomarkers [44, 45]. This approach
methods face within the field of neurology and was notable because it allowed the transfer of
within medicine as a whole. learned features from the source CADDementia
dataset to the target ADNI dataset, resulting in
Medical Image Classification state-of-the-art AD diagnosis accuracy on an
external dataset. Analogous work with volu-
The first application of deep learning in medi- metric data has been conducted in the com-
cine involved the analysis of imaging modali- puted tomography (CT) domain to differentiate
ties, especially those for the detection of AD from brain lesions and the processes of
Alzheimer’s disease (AD) and other cognitive normal aging [46].
impairments. A variety of publicly available The most recent work has built on existing
databases, such as the Alzheimer’s Disease work in AD diagnosis and focused on predicting
Neuroimaging Initiative (ADNI) and Brain the onset of AD in at-risk patients in order to
Tumor Segmentation Benchmark (BraTS), have stem progression of the disease. Ding et al. used
become available to spur advancements in fluorine-18-fluorodeoxyglucose PET scans of the
neuroimaging analysis [32, 33]. brain derived from the ADNI database to train a
Early approaches used AEs in conjunction CNN to diagnose AD [47]. Unlike many inves-
with a classifier to distinguish AD, mild cogni- tigators before them, however, the authors
tive impairments (MCI) and healthy controls. evaluated the efficacy of their algorithm against
356 Neurol Ther (2019) 8:351–365

data from the long-term follow-up of patients segmentations by expert raters show consider-
who did not have AD at the time. Interestingly, able variation in images obscured by field arti-
they found that the algorithm predicted onset facts or where intensity gradients are minimal,
of AD on average 75.8 months prior to the final and rudimentary algorithms struggle to achieve
diagnosis on an independent dataset, which consistency in an anatomy that can vary con-
surpassed the diagnostic performance of three siderably from patient to patient [33]. In light of
expert radiologists. these factors, deep learning segmentation of
Deep learning-based image classification has neuroanatomy has become a prime target for
also been applied in the diagnosis of acute efforts in deep learning research.
neurologic events, such as intracranial hemor- Measurement of the performance of neu-
rhage (ICH) and cranial fractures, with the aim roanatomical segmentation algorithms has
of reducing time to diagnosis by optimizing been standardized by the BraTS, which was
neuroradiology workflows. Titano et al. trained established at the 2012 and 2013 Medical Image
a 3D CNN in a weakly supervised manner on Computing and Computer Assisted Interven-
37,236 CT scans to identify ICH for the pur- tions (MICCAI) conference [33]. Prior to the
poses of triaging patient cases [48]. They lever- establishment of this challenge, segmentation
aged a natural language processing algorithm algorithms were often evaluated on private
trained on 96,303 radiology reports to generate imaging collections only, with variations in the
silver-standard labels for each imaging study imaging modalities incorporated and the met-
and validated the efficacy of their CNN on a rics used to evaluate effectiveness. The estab-
subset of studies with gold standard labels gen- lishment of BraTS has been critical in
erated by manual chart review [49]. The inves- standardizing the evaluation of various models
tigators conducted a double-blind randomized for the determination of which to pursue in
control trial to compare whether the algorithm clinical practice. At the time of BraTS estab-
or expert radiologists could more effectively lishment, the models being evaluated were lar-
triage studies in a simulated clinical environ- gely simple machine learning models, including
ment and found that the CNN was 150-fold four random forest-based segmentation models
faster in evaluating a study and significantly [33]. Since then, there has been considerable
outperformed humans in prioritizing the most advancement in performance, largely based on
urgent cases. Subsequent studies have similarly the adoption of CNNs for anatomical
demonstrated the potential for deep learning to segmentation.
optimize radiology workflows in the diagnosis The traditional computational approach to
of ICH and detect as many as nine critical segmentation is to employ an atlas-based seg-
findings on head CT scans with sensitivity mentation, namely the FreeSurfer software,
comparable to that of expert radiologists which assigns one of 37 labels to each voxel in a
[50–52]. 3D MRI scan based on probabilistic estimates
[56]. In a recent comparative study, Wachinger
Medical Image Segmentation et al. designed and applied a deep CNN, called
DeepNAT, for the purposes of segmenting neu-
Segmentation of radiological brain images is roanatomy visualized in T1-weighted MRI scans
critical for the measurement of brain regions, into 25 different brain regions. The authors
including shape, thickness, and volume, that used the MICCAI Multi-Atlas Labeling chal-
are important for the quantification of struc- lenge, consisting of 30 T1-weighted images, in
tural changes within the brain that occur either addition to manually labeled segmentations
naturally or due to various disease processes [53, 57]. When the authors compared the cur-
[53]. Accurate structural classification is partic- rent clinical standard, FreeSurfer, which uses its
ularly important in patients with gliomas, the own anatomical atlas to assign anatomic labels,
most common brain tumor type, with less than to DeepNAT, they found that DeepNAT showed
a 2-year survival time [54, 55]. Manual statistically significant performance improve-
ments. Performance in segmentation was
Neurol Ther (2019) 8:351–365 357

measured using a Dice volume overlap score, as high-intensity regions in T2-fluid-attenuated

with DeepNAT achieving a Dice score of 0.906, inversion recovery (FLAIR) MRI scans [63]. In
in comparison to FreeSurfer’s 0.817 [53]. addition to quantifying PVSs, U-nets have been
In addition to tissue-based segmentation used in segmentation efforts to identify regions
efforts, vascular segmentation has been an area of abnormally intense white matter signals. In
of deep learning research aimed at quantifying 2019, Jeong et al. proposed a sailiency U-net, a
brain vessel status. Traditional vessel segmen- U-net combined with simple regional maps,
tation relies on either manual identification or with the aim to lower the computational
rule-based algorithms since there is no equiva- demand of the architecture while maintaining
lent atlas-based method for brain vessels as performance in order to identify areas of signal
there is for neuroanatomy. In their recent study intensity in T2-FLAIR MRI scans of patients with
on blood vessel segmentation, Livne et al. AD [62, 64]. Their model achieved a Dice coef-
applied a U-net model to labeled data from 66 ficient score of 0.544 and a sensitivity of 0.459,
patients with cerebrovascular disease and then indicating the utility of such a model to aug-
compared the method to the traditional vascu- ment clinical image analysis [62]. The efforts
lar segmentation method of graph-cuts. The described above in neuroanatomical segmenta-
U-net model outperformed graph-cuts, achiev- tion and anomaly detection highlight the ver-
ing a Dice score of 0.891 compared to 0.760 for satility of deep learning in analyzing an
graph-cuts [58]. Of note, the model, which was inherently complex organ system.
trained on 3T MRI time-of-flight images, failed
to generalize well to 7T images [58]. Functional Connectivity
Quantification of changes in white matter as and Classification of Brain Disorders
biomarkers for disease processes has been a
third area of deep learning segmentation efforts Research in diagnostic support using multiple
in neurology. Perivascular spaces (PVSs) are modalities has been a key area of focus in deep
small spaces surrounding blood vessels that can learning research, particularly in disease spaces
be caused by the stress-induced breakdown of such as AD, autism spectrum disorder (ASD),
the blood–brain barrier by various inflamma- and attention deficit hyperactivity disorder
tory processes [59, 60]. While PVSs have been (ADHD). For all of these diseases, the onset can
implicated in a wide range of disease processes, be insidious, and diagnosis is reliant on non-
the quantification of these spaces is difficult due specific symptoms, such as distractibility and
to their tubular and low-contrast appearance hyperactivity in the case of ADHD, which
even on those clinical MRI machines with the results in poor sensitivity and specificity for
highest-approved resolution [61]. In one 2018 clinical diagnostic testing; in fact, the sensitiv-
study, Lian et al. used a deep CNN to evaluate ity of the American Psychiatric Association’s
PVSs in 20 patients scanned on a 7T MRI Diagnostic and Statistical Manual testing for
machine, comparing these to gold-standard ADHD is between 70 and 90% [65]. Further-
manual labels. Their deep CNN outperformed more, delays in diagnosis inevitably delay
unsupervised algorithmic methods, such as a treatment, resulting in the treatment being less
Frangi filter, as well as a U-net deep learning effective or entirely ineffective [65]. Using fMRI
model, achieving a positive predictive value and connectome mapping alongside clinical
(PPV) of 0.83 ± 0.05, compared to a PPV of and demographic data points, multidisciplinary
0.62 ± 0.08 for the Frangi filter and 0.70 ± 0.10 teams have sought to improve upon the accu-
for the U-net. racy of currently utilized neurological tests.
U-net models have also been leveraged in Within the realm of AD and disorders
quantifying white matter hyperintensities as implicated in MCIs, deep learning has been
biomarkers for age-related neurologic disorders increasingly adopted as a method to analyze
[62]. White matter changes have been shown to neural connectivity information. Although
be involved in various forms of cortical much of the work in connectome mapping has
dementia, such as AD, and manifest themselves
358 Neurol Ther (2019) 8:351–365

relied on less complex classifiers, recent publi- Risk Prognostication

cations have explored the benefits of deep
learning [66, 67]. When applied to fMRI data, In addition to widespread research on deep
deep learning has several advantages over sim- learning applications for image classification
pler SVMs and Lasso models, and exhibits an and segmentation, researchers have applied
exponential gain in accuracy over simpler deep learning to a variety of other neurology-
models with increasing volumes of training data specific and general medicine data for the pur-
[5, 68]. Meszlenyi et al. utilized a variant of a poses of risk prognostication. These efforts have
convolutional neural network called a connec- been applied to electroencephalogram (EEG)
tome convolutional neural network (CCNN) to signals and genetic biomarkers in the hope of
classify MCI in a relatively small dataset of predicting clinically meaningful events. Neu-
functional connectivity data from 49 patients rologists frequently rely on EEG data for the
[67]. Although accuracies were comparable management and diagnosis of neurological
between the deep learning and less complex dysfunction, in particular epilepsy and epileptic
classifiers (53.4% accuracy for the CCNN com- events. Several studies using deep learning
pared to 54.1% for the SVM), the authors pos- methods have investigated its utility when
tulate that the accuracy benefits of the CCNN applied to preictal scalp EEGs as a predictive
architecture are well suited to fMRI tasks as tool for seizures [72–74]. The most successful of
dataset sizes expand [67]. these efforts included a LSTM network, which is
Deep learning classifiers have been applied particularly useful for interpreting time-series
numerous times toward the accurate diagnosis data, allowing a model to allocate importance
of ASD using fMRI data. In one study published to previously seen data in a sequence when
in 2015, Iidaka et al. selected 312 patients with interpreting a given datapoint. These algo-
ASD and 328 control patients from the Autism rithms are uniquely suited to large sequences of
Brain Imaging Data Exchange (ABIDE), together data and have proved their efficacy in predict-
with 90 regions of interest, and used a proba- ing epileptic events [73].
bilistic neural network to classify individuals In their 2018 study, Tsiouris et al. used a two-
with ASD. Their method achieved a classifica- layer LSTM-based algorithm to predict epileptic
tion accuracy of 90% [69]. Additionally, Chen seizures using the publicly available CHB-MIT
et al. published a classifier based on a con- scalp EEG database. While previous efforts had
structed functional network and additional data been made using CNNs and scalp EEGs to pre-
from the ABIDE dataset in a clustering analysis dict epileptic events, the novel use of an LSTM
aimed at grouping discriminative features and set a new state-of-the-art over traditional
found that many discriminative features clus- machine learning algorithms and other deep
tered into the Slow-4 band [70]. learning algorithms. Following feature extrac-
In the realm of ADHD, several efforts have tion, the LSTM was provided several meaningful
been made to use publicly available imaging features, including statistical moments, zero
data and deep learning algorithms for diagnosis. crossings, Wavelet Transform coefficients,
In a study published in 2014, Kuang et al. power spectral density, cross-correlation, and
attempted to classify ADHD using a deep belief graph theory, to use in the prediction of sei-
network, comprised of stacked Boltzmann’s zures. Notably, the authors compared the pre-
machines trained on the public ADHD-200 dictive ability of the raw EEG data to the
dataset [71]. Using time-series fMRI data, the extracted features and determined that feature
deep belief network achieved an accuracy of extraction improved model performance [73].
35.1%. While each of the above classifiers have This model configuration achieved a minimum
achieved results that are either on-par or less of 99.28% sensitivity and 99.28% specificity
accurate than clinical diagnoses using fMRI across the 15-, 30-, 60-, and 120-min preictal
data, methods are expected to improve dra- periods, as well as a maximum false positive rate
matically as the quantity of labeled data con- of 0.11/h. Similar experiments on the CHB-MIT
tinues to grow [71]. scalp EEG database using CNNs, as opposed to
Neurol Ther (2019) 8:351–365 359

LSTMs, achieved worse results, namely poorer potential to improve current standards for pre-
sensitivity and a higher hourly rate of false dicting clinical events.
positives [75, 76].
Genetic data has been another important
area of research and development for precision CHALLENGES
medicine. Predictive tasks in large-scale geno-
mic profiles face high-dimensional datasets that Despite the profound biomedical advances due
are often pared down by experts who hand-se- to deep learning algorithms, there remain sig-
lect a small number of features for training nificant challenges that must be addressed
predictive models [77]. In ASD, deep learning before such applications gain widespread use.
has played a particularly important role in We discuss some of the most critical hurdles in
determining the impact of de-novo mutations, the following sections.
including copy number variants and point
mutations, on ASD severity [78]. Using a deep Data Volume
CNN, Zhou et al. modeled the biochemical
impact of observed point mutations in 1790 Deep neural networks are computationally
whole-genome sequenced families with ASD, on intensive, multilayered algorithms with param-
both the RNA and DNA levels [78]. This eters on the order of millions. Convergence of
approach revealed that both transcriptional and such algorithms requires data commensurate
post-transcriptional mechanisms play a major with the number of parameters. Although there
role in ASD, suggesting biological convergence are no strict rules governing the amount of data
of genetic dysregulation in ASD. required to optimally train DNNs, empirical
Genomic data, either alone or in conjunc- studies suggest that tenfold more training data
tion with neuroimaging and histopathology, relative to the number of parameters is required
has provided cancer researchers a wealth of data to produce an effective model. It is no surprise
on which to perform cancer-related predictive then that domains, such as computer vision and
tasks [77, 79, 80]. Deep learning offers several natural language processing, have seen the most
advantages when working simultaneously with rapid progress due to deep learning given the
multiple data modalities, removing subjective wide availability of images, videos, and free-
interpretations of histological images, accu- form text on the Internet.
rately predicting time-to-event outcomes, and Biomedical data on the other hand is mostly
even surpassing gold standard clinical para- decentralized—stored locally within hospital
digms for glioma patient survival [80]. Using systems—and subject to privacy constraints that
high-powered histological slices and genetic make such data less readily accessible for
data, namely IDH mutation status and 1p/19q research. Furthermore, given the complexity of
codeletion, on 769 patients from The Cancer patient presentations and disease processes,
Genome Atlas (TCGA), Mobadersaney et al. reliable ground truth labels for biomedical
used a survival CNN (SCNN) to predict time-to- applications are extremely expensive to obtain,
event outcomes. The histological and genetic often requiring the efforts of multiple highly
model performed on par with manual histologic specialized domain experts. This paucity of
grading or molecular subtyping [80]. In a sec- labeled data remains an important bottleneck in
ond paper by this group, SCNNs were shown to the development of deep learning applications
outperform other machine learning algorithms, in medicine.
including random forest, in classification tasks
using genetic data from multiple tumor types, Data Quality
including kidney, breast, and pan-glioma can-
cers [77]. The ability of deep learning algo-
Healthcare data are fundamentally ill-suited for
rithms to reduce subjectivity in histologic
deep learning applications. Electronic medical
grading and disentangle complex relationships
records are highly heterogeneous, being
between noisy EEG or genetic data, has the
360 Neurol Ther (2019) 8:351–365

composed of clinical notes, a miscellany of made by algorithms and enable their wide-
various codes, and other patient details that spread adoption. As such, major efforts within
may often be missing or incomplete. Clinical the deep learning community to tackle prob-
notes consist of nuanced language and acro- lems of interpretability and explainability have
nyms that often vary by specialty and contain the potential to be particularly beneficial for
redundant information that provides an inac- facilitating the use of deep learning methods in
curate temporal representation of disease onset healthcare.
or progression. Diagnosis codes suffer from a
similar fate as they track billing for insurance Legal
purposes instead of health outcomes. This
inherent complexity makes it impossible for Medical malpractice rules govern standards of
deep learning algorithms to parse signal from clinical practice in order to ensure the appro-
noise. priate care of patients. However, to date, no
standards have been established to assign cul-
Generalizability pability in contexts where algorithms provide
poor predictions or substandard treatment rec-
Although existing deep learning applications ommendations. The establishment of such reg-
have garnered success in silico, their widespread ulations is a necessary prerequisite for the
adoption in real-world clinical settings remains widespread adoption of deep learning algo-
limited due to concerns over their efficacy rithms in clinical contexts.
across clinical contexts. Much of the concern is
based on the tendency of deep learning algo- Ethical
rithms to overfit to the statistical characteristics
of the training data, rendering them hyper- Incidental introduction of bias must be care-
specialized for a hospital or certain patient fully evaluated in the application of deep
demographic and less effective on the popula- learning in medicine. As discussed previously,
tion at-large [81, 82]. The siloed existence of deep learning algorithms are uniquely adept at
healthcare data in hospitals and the hetero- fitting to the characteristics of the data on
geneity of data across healthcare systems make which they are trained. Such algorithms have
the task of developing generalizable models the capability to perpetuate inequities against
even more difficult. And even when multi-in- populations underrepresented in medicine and,
stitutional data are acquired, the data are often by extension, in the very healthcare data used
retrospective in nature, which prevents practi- to train the algorithms. Furthermore, recent
cal assessment of algorithm performance. research evaluating algorithmic bias in a com-
mercial healthcare algorithm provides a cau-
Interpretability tionary tale on the importance of critically
evaluating the very outcomes algorithms are
The power of deep learning algorithms to map trained to predict [83].
complex, nonlinear functions can render them
difficult to interpret. This becomes an impor-
tant consideration in healthcare applications CONCLUSION
where the ability to identify drivers of outcomes
becomes just as important as the ability to Deep learning has the potential to fundamen-
accurately predict the outcome itself. In the tally alter the practice of medicine. The clinical
clinical setting, where clinical decision support neurosciences in particular are uniquely situ-
systems are designed to augment the decision- ated to benefit given the subtle presentation of
making capacity of healthcare professionals, symptoms typical of neurologic disease. Here,
interpretability is critical to convince healthcare we reviewed the various domains in which deep
professionals to rely on the recommendations learning algorithms have already provided
Neurol Ther (2019) 8:351–365 361

impetus for change—areas such as medical Disclosures. Aly Al-Amyn Valliani, Daniel
image analysis for improved diagnosis of AD Ranti and Eric Karl Oermann have nothing to
and the early detection of acute neurologic disclose.
events; medical image segmentation for quan-
titative evaluation of neuroanatomy and vas- Compliance with Ethics Guidelines. This
culature; connectome mapping for the article is based on previously conducted studies
diagnosis of AD, ASD, and ADHD; and mining and does not contain any studies with human
of microscopic EEG signals and granular genetic participants or animals performed by any of the
signatures. Amidst these advances, however, authors.
important challenges remain a barrier to inte-
gration of deep learning tools in the clinical Data availability. Data sharing is not
setting. While technical challenges surrounding applicable to this article as no datasets were
the generalizability and interpretability of generated or analyzed during the current study.
models are active areas of research and progress,
more difficult challenges surrounding data pri- Open Access. This article is distributed
vacy, accessibility, and ownership will necessi- under the terms of the Creative Commons
tate conversations in the healthcare Attribution-NonCommercial 4.0 International
environment and society in general to arrive at License (http://creativecommons.org/licenses/
solutions that benefit all relevant stakeholders. by-nc/4.0/), which permits any noncommer-
The challenge of data quality, in particular, may cial use, distribution, and reproduction in any
prove to be a uniquely suitable target for medium, provided you give appropriate credit
addressing using deep learning techniques that to the original author(s) and the source, provide
have already demonstrated efficacy in image a link to the Creative Commons license, and
analysis and natural language processing. indicate if changes were made.
Overcoming these hurdles will require the
efforts of interdisciplinary teams of physicians,
computer scientists, engineers, legal experts,
and ethicists working in concert. It is only in
REFERENCES
this manner that we will truly realize the
1. Jensen PB, Jensen LJ, Brunak S. Mining electronic
potential of deep learning in medicine to aug-
health records: towards better research applications
ment the capability of physicians and enhance and clinical care. Nat Rev Genet. 2012;13(6):395–405.
the delivery of care to patients.
2. Luo J, Wu M, Gopukumar D, Zhao Y. Big data
application in biomedical research and health care:
a literature review. Biomed Inform Insights.
ACKNOWLEDGEMENTS 2016;19(8):1–10.

3. Kohli MD, Summers RM, Geis JR. Medical image

data and datasets in the era of machine learning-
Funding. No funding or sponsorship was whitepaper from the 2016 C-MIMI meeting dataset
received for this study or publication of this session. J Digit Imaging. 2017;30(4):392–9.
article.
4. Bengio Y, Courville A, Vincent P. Representation
learning: a review and new perspectives. IEEE Trans
Authorship. All named authors meet the Pattern Anal Mach Intell. 2013;35(8):1798–828.
International Committee of Medical Journal
Editors (ICMJE) criteria for authorship for this 5. LeCun Y, Bengio Y, Hinton G. Deep learning. Nat-
article, take responsibility for the integrity of ure. 2015;521(7553):436–44.
the work as a whole, and have given their 6. Li H, Lin Z, Shen X, Brandt J, Hua G. A convolu-
approval for this version to be published. tional neural network cascade for face detection. In:
Proceedings of IEEE conference on computer vision
and pattern recognition. Boston, MA. 2015.
pp. 5325–34.
362 Neurol Ther (2019) 8:351–365

7. Gilani SZ, Mian A. Learning from millions of 3D 20. Gulshan V, Peng L, Coram M, Stumpe MC, Wu D,
scans for large-scale 3D face recognition. 2017. Narayanaswamy A, et al. Development and valida-
http://arxiv.org/abs/1711.05942. tion of a deep learning algorithm for detection of
diabetic retinopathy in retinal fundus photographs.
8. Ramanishka V, Chen Y-T, Misu T, Saenko K. Toward JAMA. 2016;316(22):2402–10.
driving scene understanding: a dataset for learning
driver behavior and causal reasoning. In: Proceed- 21. Esteva A, Kuprel B, Novoa RA, et al. Dermatologist-
ings of IEEE conference on computer vision and level classification of skin cancer with deep neural
pattern recognition. Salt Lake City, UT. 2018. networks. Nature. 2017;542(7639):115–8.
pp. 7699–707.
22. Haenssle HA, Fink C, Schneiderbauer R, et al. Man
9. Maqueda AI, Loquercio A, Gallego G, Garcia N, against machine: diagnostic performance of a deep
Scaramuzza D. Event-based vision meets deep learning convolutional neural network for dermo-
learning on steering prediction for self-driving cars. scopic melanoma recognition in comparison to 58
2018. http://arxiv.org/abs/1804.01310. dermatologists. Ann Oncol. 2018;29(8):1836–42.

10. Mazaré P-E, Humeau S, Raison M, Bordes A. Train- 23. De Fauw J, Ledsam JR, Romera-Paredes B, et al.
ing millions of personalized dialogue agents. 2018. Clinically applicable deep learning for diagnosis
http://arxiv.org/abs/1809.01984. and referral in retinal disease. Nat Med.
2018;24(9):1342–50.
11. Zhang S, Dinan E, Urbanek J, Szlam A, Kiela D,
Weston J. Personalizing dialogue agents: I have a 24. Poplin R, Varadarajan AV, Blumer K, et al. Predic-
dog, do you have pets too? 2018. http://arxiv.org/ tion of cardiovascular risk factors from retinal fun-
abs/1801.07243. dus photographs via deep learning. Nat Biomed
Eng. 2018;2(3):158–64.
12. Wu Y, Schuster M, Chen Z, et al. Google’s neural
machine translation system: bridging the gap 25. Lipton ZC, Berkowitz J, Elkan C. A critical review of
between human and machine translation. 2016. recurrent neural networks for sequence learning.
http://arxiv.org/abs/1609.08144. 2015. http://arxiv.org/abs/1506.00019.

13. US National Library of Medicine National Institutes 26. Rumelhart DE, McClelland JL. Learning internal
of Health. PubMed. 2019. https://www.ncbi.nlm. representations by error propagation. In: Parallel
nih.gov/pubmed/?term=Machine?Learning. distributed processing: explorations in the
microstructure of cognition: foundations. Wacht-
14. Mitchell TM. The discipline of machine learning, endonk: MITP Verlags-GmbH & Co. KG; 1987.
vol. 9. Pittsburgh: School of Computer Science, pp. 318–62.
Carnegie Mellon University; 2006.
27. Hinton GE, Salakhutdinov RR. Reducing the
15. Rosenblatt F. The perceptron: A probabilistic model dimensionality of data with neural networks. Sci-
for information storage and organization in the ence. 2006;313(5786):504–7.
brain. Psychol Rev. 1958;65:386–408. http://dx.doi.
org/10.1037/h0042519. 28. Goodfellow IJ, Pouget-Abadie J, Mirza M, et al.
Generative adversarial networks. 2014. http://arxiv.
16. Ogutu JO, Schulz-Streeck T, Piepho H-P. Genomic org/abs/1406.2661.
selection using regularized linear regression models:
ridge regression, lasso, elastic net and their exten- 29. Shin H-C, Tenenholtz NA, Rogers JK, et al. Medical
sions. BMC Proc. 2012;6[Suppl 2]:S10. image synthesis for data augmentation and
anonymization using generative adversarial net-
17. Krizhevsky A, Sutskever I, Hinton GE. ImageNet works. In:Proc Third International Workshop,
classification with deep convolutional neural net- SASHIMI 2018, held in conjunction with MICCAI
works. In: Pereira F, Burges CJC, Bottou L, Wein- 2018, Granada, Spain, September 16, 2018. In:
berger KQ, editors. Advances in neural information Gooya A, Goksel O, Oguz I, Burgos N, editors.
processing systems, vol. 25. New York: Curran Simulation and synthesis in medical imaging.
Associates, Inc.; 2012; 1097–105. Cham: Springer International Publishing;
2018:1–11.
18. Szegedy C, Liu W, Jia Y, et al. Going deeper with
convolutions. 2014. http://arxiv.org/abs/1409.4842. 30. Shi S, Wang Q, Xu P, Chu X. Benchmarking state-
of-the-art deep learning software tools. 2016.
19. Saba L, Biswas M, Kuppili V, et al. The present and http://arxiv.org/abs/1608.07249.
future of deep learning in radiology. Eur J Radiol.
2019;114:14–24. 31. Liu J, Dutta J, Li N, Kurup U, Shah M. Usability
study of distributed deep learning frameworks for
Neurol Ther (2019) 8:351–365 363

convolutional neural networks. 2018. https://www. neural networks. 2015. http://arxiv.org/abs/1502.

kdd.org/kdd2018/files/deep-learning-day/DLDay18_ 02506.
paper_29.pdf.
44. Hosseini-Asl E, Gimel’farb G, El-Baz A. Alzheimer’s
32. Petersen RC, Aisen PS, Beckett LA, et al. Alzheimer’s disease diagnostics by a deeply supervised adapt-
Disease Neuroimaging Initiative (ADNI): clinical able 3D convolutional network. 2016. http://arxiv.
characterization. Neurology. 2010;74(3):201–9. org/abs/1607.00556.

33. Menze BH, Jakab A, Bauer S, et al. The Multimodal 45. Hosseini-Asl E, Ghazal M, Mahmoud A, et al. Alz-
Brain Tumor Image Segmentation Benchmark heimer’s disease diagnostics by a 3D deeply super-
(BRATS). IEEE Trans Med Imaging. vised adaptable convolutional network. Front
2015;34(10):1993–2024. Biosci. 2018;1(23):584–96.

34. Suk H-I, Shen D. Deep learning-based feature rep- 46. Gao XW, Hui R. A deep learning based approach to
resentation for AD/MCI classification. Med Image classification of CT brain images. In: 2016 SAI
Comput Comput Assist Interv. 2013;16(Pt computing conference (SAI). London, UK. 2016.
2):583–90. pp. 28–31. http://ieeexplore.ieee.org.

35. Gupta A, Ayhan M, Maida A. Natural image bases to 47. Ding Y, Sohn JH, Kawczynski MG, et al. A deep
represent neuroimaging data. In: Proceedings of learning model to predict a diagnosis of Alzheimer
30th international conference on machine learn- disease by using 18F-FDG PET of the brain. Radiol-
ing. vol. 28. Atlanta, GA. 2013. pp. 987–94. ogy. 2019;290(2):456–64.

36. Li F, Tran L, Thung K-H, Ji S, Shen D, Li J. Robust 48. Titano JJ, Badgeley M, Schefflein J, et al. Automated
deep learning for improved classification of AD/ deep-neural-network surveillance of cranial images
MCI Patients. Machine learning in medical imag- for acute neurologic events. Nat Med.
ing. New York: Springer International Publishing; 2018;24(9):1337–41.
2014:240–7.
49. Zech J, Pain M, Titano J, et al. Natural language-
37. Liu S, Liu S, Cai W, Pujol S, Kikinis R, Feng D. Early based machine learning models for the annotation
diagnosis of Alzheimer’s disease with deep learning. of clinical radiology reports. Radiology.
In: 2014 IEEE 11th international symposium on 2018;30:171093.
biomedical imaging (ISBI). Beijing, China. 2014.
pp. 1015–8. http://ieeexplore.ieee.org. 50. Arbabshirani MR, Fornwalt BK, Mongelluzzo
GJ,et al. Advanced machine learning in action:
38. Liu S, Liu S, Cai W, et al. Multimodal neuroimaging identification of intracranial hemorrhage on com-
feature learning for multiclass diagnosis of Alzhei- puted tomography scans of the head with clinical
mer’s disease. IEEE Trans Biomed Eng. workflow integration. NPJ Digit Med. 2018;1(1):9.
2015;62(4):1132–40.
51. Chilamkurthy S, Ghosh R, Tanamala S, et al. Deep
39. Suk H-I, Lee S-W, Shen D. Alzheimer’s disease learning algorithms for detection of critical findings
neuroimaging initiative. Latent feature representa- in head CT scans: a retrospective study. Lancet.
tion with stacked auto-encoder for AD/MCI diag- 2018;392(10162):2388–96.
nosis. Brain Struct Funct. 2015;220(2):841–59.
52. Lee H, Yune S, Mansouri M, et al. An explainable
40. Sarraf S, Tofighi G. Classification of Alzheimer’s deep-learning algorithm for the detection of acute
disease using fMRI data and deep learning convo- intracranial haemorrhage from small datasets. Nat
lutional neural networks. 2016. http://arxiv.org/ Biomed Eng. 2018;5:6. https://doi.org/10.1038/
abs/1603.08631. s41551-018-0324-9.

41. Suk H-I, Lee S-W, Shen D. Alzheimer’s disease 53. Wachinger C, Reuter M, Klein T. DeepNAT: deep
neuroimaging initiative. Deep sparse multi-task convolutional neural network for segmenting neu-
learning for feature selection in Alzheimer’s disease roanatomy. Neuroimage. 2018;15(170):434–45.
diagnosis. Brain Struct Funct. 2016;221(5):2569–87.
54. Ohgaki H, Kleihues P. Population-based studies on
42. Valliani A, Soni A. Deep residual nets for improved incidence, survival rates, and genetic alterations in
Alzheimer’s diagnosis. In: BCB. Boston, MA. 2017. astrocytic and oligodendroglial gliomas. J Neu-
p. 615. ropathol Exp Neurol. 2005;64(6):479–89.

43. Payan A, Montana G. Predicting Alzheimer’s dis- 55. Holland EC. Progenitor cells and glioma formation.
ease: a neuroimaging study with 3D convolutional Curr Opin Neurol. 2001;14(6):683–8.
364 Neurol Ther (2019) 8:351–365

56. Fischl B, Salat DH, Busa E, Albert M, Dieterich M, 68. Montufar GF, Pascanu R, Cho K, Bengio Y. On the
Haselgrove C, et al. Whole brain segmentation: number of linear regions of deep neural networks.
automated labeling of neuroanatomical structures In: Ghahramani Z, Welling M, Cortes C, Lawrence
in the human brain. Neuron. 2002;33(3):341–55. ND, Weinberger KQ, editors. Advances in neural
information processing systems, vol. 27. Red Hook:
57. Landman B, Warfield S. MICCAI 2012 workshop on Curran Associates, Inc.; 2014:2924–32.
multi-atlas labeling. In: Medical image computing
and computer assisted intervention conference. 69. Iidaka T. Resting state functional magnetic reso-
Nice, France. October 1–5, 2012. nance imaging and neural network classified autism
and control. Cortex. 2015;63:55–67.
58. Livne M, Rieger J, Aydin OU, et al. A U-Net deep
learning framework for high performance vessel 70. Chen H, Duan X, Liu F, et al. Multivariate classifi-
segmentation in patients with cerebrovascular dis- cation of autism spectrum disorder using fre-
ease. Front Neurosci. 2019;28(13):97. quency-specific resting-state functional
connectivity—a multi-center study. Prog Neu-
59. Loftis JM, Huckans M, Morasco BJ. Neuroimmune ropsychopharmacol Biol Psychiatry.
mechanisms of cytokine-induced depression: cur- 2016;4(64):1–9.
rent theories and novel treatment strategies. Neu-
robiol Dis. 2010;37(3):519–33. 71. Kuang D, Guo X, An X, Zhao Y, He L. Discrimina-
tion of ADHD based on fMRI data with deep belief
60. Menard C, Pfau ML, Hodes GE, et al. Social stress network. Intelligent computing in bioinformatics.
induces neurovascular pathology promoting New York: Springer International Publishing;
depression. Nat Neurosci. 2017;20(12):1752–60. 2014:225–32.

61. Lian C, Zhang J, Liu M, et al. Multi-channel multi- 72. Tjepkema-Cloostermans MC, de Carvalho RCV, van
scale fully convolutional network for 3D perivas- Putten MJAM. Deep learning for detection of focal
cular spaces segmentation in 7T MR images. Med epileptiform discharges from scalp EEG recordings.
Image Anal. 2018;46:106–17. Clin Neurophysiol. 2018;129(10):2191–6.

62. Jeong Y, Rachmadi MF, Valdés-Hernández MDC, 73. Tsiouris JL, Pezoulas VC, Zervakis M, Konitsiotis S,
Komura T. Dilated saliency U-Net for white matter Koutsouris DD, Fotiadis DI. A long short-term
hyperintensities segmentation using irregularity memory deep learning network for the prediction
age map. Front Aging Neurosci. 2019;27(11):150. of epileptic seizures using EEG signals. Comput Biol
Med. 2018;1(99):24–37.
63. Gootjes L, Teipel SJ, Zebuhr Y, et al. Regional dis-
tribution of white matter hyperintensities in vas- 74. Acharya UR, Oh SL, Hagiwara Y, Tan JH, Adeli H.
cular dementia, Alzheimer’s disease and healthy Deep convolutional neural network for the auto-
aging. Dement Geriatr Cogn Disord. mated detection and diagnosis of seizure using EEG
2004;18(2):180–8. signals. Comput Biol Med. 2018;1(100):270–8.

64. Karargyros A, Syeda-Mahmood T. Saliency U-Net: A 75. Truong ND, Nguyen AD, Kuhlmann L, Bonyadi MR,
regional saliency map-driven hybrid deep learning Yang J, Kavehei O. A generalised seizure prediction
network for anomaly segmentation. In: Medical with convolutional neural networks for intracranial
imaging 2018: computer-aided diagnosis. Interna- and scalp electroencephalogram data analysis.
tional Society for Optics and Photonics. Houston, 2017. http://arxiv.org/abs/1707.01976.
TX. 2018. 105751T.
76. Khan H, Marcuse L, Fields M, Swann K, Yener B.
65. Kuang D, He L. Classification on ADHD with deep Focal onset seizure prediction using convolutional
learning. In: 2014 international conference on networks. IEEE Trans Biomed Eng.
cloud computing and big data. Wuhan, China. 2018;65(9):2109–18.
2014. pp. 27–32. http://ieeexplore.ieee.org.
77. Yousefi S, Amrollahi F, Amgad M, et al. Predicting
66. Suk H-I, Wee C-Y, Lee S-W, Shen D. State-space clinical outcomes from large scale cancer genomic
model with deep learning for functional dynamics profiles with deep survival models. Sci Rep.
estimation in resting-state fMRI. Neuroimage. 2017;7(1):11707.
2016;1(129):292–307.
78. Zhou J, Park CY, Theesfeld CL, et al. Whole-genome
67. Meszlényi RJ, Buza K, Vidnyánszky Z. Resting state deep-learning analysis identifies contribution of
fMRI functional connectivity-based classification noncoding mutations to autism risk. Nat Genet.
using a convolutional neural network architecture. 2019;51(6):973–80.
Front Neuroinform. 2017;17(11):61.
Neurol Ther (2019) 8:351–365 365

79. Buda M, Saha A, Mazurowski MA. Association of chest radiographs: a cross-sectional study. PLoS
genomic subtypes of lower-grade gliomas with Med. 2018;15(11):e1002683.
shape features automatically extracted by a deep
learning algorithm. Comput Biol Med. 82. Zech JR, Badgeley MA, Liu M, Costa AB, Titano JJ,
2019;109:218–25. Oermann EK. Confounding variables can degrade
generalization performance of radiological deep
80. Mobadersany P, Yousefi S, Amgad M, et al. Pre- learning models. 2018. http://arxiv.org/abs/1807.
dicting cancer outcomes from histology and geno- 00431.
mics using convolutional networks. Proc Natl Acad
Sci USA. 2018;115(13):E2970–9. 83. Obermeyer Z, Mullainathan S. Dissecting racial bias
in an algorithm that guides health decisions for 70
81. Zech JR, Badgeley MA, Liu M, Costa AB, Titano JJ, million people. In: Proceedings of conference on
Oermann EK. Variable generalization performance fairness, accountability, and transparency. New
of a deep learning model to detect pneumonia in York: ACM; 2019. p. 89.