A TECHNICAL SEMINAR REPORT ON

Deep Learning for Multigrade Brain Tumor

Classification in Smart Healthcare Systems: A Prospective Survey
A dissertation submitted in partial fulfilment of the
Requirements for the award of the degree of

BACHELOR OF TECHNOLOGY
in

INFORMATION TECHNOLOGY
Submitted by

MOHAMMAD ABDUL RAHIL (20B81A1201)

Under the esteemed guidance of

Mr. A Seetharam Nagesh

Associate Professor, IT Department
CVR College of Engineering

DEPARTMENT OF INFORMATION TECHNOLOGY

CVR COLLEGE OF ENGINEERING

ACCREDITED BY NBA, AICTE & Affiliated to JNTU-H
Vastunagar, Mangalpalli (V), Ibrahimpatnam (M), R.R. District, PIN-501510
2023-2024
 DEPARTMENT OF INFORMATION TECHNOLOGY
CERTIFICATE

This is to certify that the Technical Seminar entitled “Deep Learning for Multigrade
Brain Tumor Classification in Smart Healthcare Systems: A Prospective Survey ”is a
bonafide work done and submitted by MOHAMMAD ABDUL RAHIL (20B81A1201) during
the academic year 2023-2024, in partial fulfilment of requirement for the award of Bachelor of
Technology degree in Information Technology from Jawaharlal Nehru Technological University
Hyderabad, is a bonafide record of work carried out by them under my guidance and supervision.

Certified further that to my best of the knowledge, the work in this dissertation has not
been submitted to any other institution for the award of any degree or diploma.

INTERNAL GUIDE HEAD OF THE DEPARTMENT

Mr. A Seetharam Nagesh Dr. Bipin Bihari Jayasingh
Associate Professor, IT Department Head and Professor, IT Department

TECHNICAL SEMINAR COORDINATOR

Mr. C.V.S Satyamurty
Associate Professor, IT Department
 ACKNOWLEDGEMENT
The satisfaction of completing this project would be incomplete without mentioning our
gratitude towards all the people who have supported us. Constant guidance and encouragement
have been instrumental in the completion of this technical seminar.
First and foremost, we thank the Chairman, Principal, Vice Principal for availing
infrastructural facilities to complete the technical seminar in time.

We offer our sincere gratitude to our internal guide Mr. A Seetharam Nagesh, Associate
Professor, IT Department, CVR College of Engineering for her immense support, timely co-
operation and valuable advice throughout the course of our technical seminar.

We would like to thank the Head of Department, Professor Dr. Bipin Bihari Jayasingh,
for his meticulous care and cooperation throughout the technical seminar.
We are thankful, Mr. C.V.S Satyamurty Technical seminar Coordinator, Associate
Professor, IT Department, CVR College of Engineering for his supportive guidelines and for
having provided the necessary help for carrying forward this technical seminar without any
obstacles and hindrances.
We also thank the Seminar Review Committee Members for their valuable suggestions.
 DECLARATION

We hereby declare that the project report entitled “Deep Learning for Multigrade Brain
Tumor Classification in Smart Healthcare Systems: A Prospective Survey ”is an original
work done and submitted to IT Department, CVR College of Engineering, affiliated to
Jawaharlal Nehru Technological University Hyderabad, Hyderabad in partial fulfilment of the
requirement for the award of Bachelor of Technology in Information Technology and it is a
record of bonafide project work carried out by us under the guidance of Mr. A Seetharam
Nagesh, Associate Professor, Department of Information Technology.

We further declare that the work reported in this technical seminar has not been submitted,
either in part or in full, for the award of any other degree or diploma in this institute or any other
Institute or University.

_________________________

Signature of the Student

(Mohammad Abdul Rahil)

(20B81A1201)
 ABSTRACT
Brain tumor is one of the most dangerous cancers in people of all ages, and its grade recognition
is a challenging problem for radiologists in health monitoring and automated diagnosis. Recently,
numerous methods based on deep learning have been presented in the literature for brain tumor
classification (BTC) in order to assist radiologists for a better diagnostic analysis. In this
overview, we present an in-depth review of the surveys published so far and recent deep learning-
based methods for BTC. Our survey covers the main steps of deep learning-based BTC methods,
including preprocessing, features extraction, and classification, along with their achievements
and limitations. We also investigate the stateof- the-art convolutional neural network models for
BTC by performing extensive experiments using transfer learning with and without data
augmentation. Furthermore, this overview describes available benchmark data sets used for the
evaluation of BTC. Finally, this survey does not only look into the past literature on the topic but
also steps on it to delve into the future of this area and enumerates some research directions that
should be followed in the future, especially for personalized and smart healthcare
 Table of Contents

Content Pg. No

1. Introduction………………………………………………….…. 1
2. Motivation……………….………………………………….… 3
3. Literature survey …………………………………...…………. 5
4. Methodology……………………………………...……………... 13
5. CHALLENGES, RECOMMENDATION…………………………………..18
6. Conclusion………………………………………………...…… 20

References………………………...………………………….... 21
 1. INTRODUCTION
A brain tumor is a mass or growth of abnormal cells in the brain. These tumors
can be benign (non-cancerous) or malignant (cancerous) and can develop in different
parts of the brain, leading to various symptoms depending on their location and size.
Symptoms may include headaches, seizures, changes in vision or hearing, cognitive
impairment, and motor function difficulties. Diagnosis and treatment of brain
tumors are complex processes that often require a multidisciplinary approach
involving neurologists, neurosurgeons, oncologists, and radiologists.
In the past few decades, advancements in image processing and computer vision
have revolutionized the diagnosis and treatment of brain tumors. Automated
diagnostic processes leveraging these technologies have provided invaluable
support to medical professionals, offering a second opinion and assisting in various
diagnostic procedures. This has significantly improved the accuracy and efficiency
of tumor detection and classification, particularly in the early stages when timely
intervention is crucial.
Among all diseases, cancer, including brain tumors, is considered one of the most
threatening due to its potentially fatal nature. Traditionally, the diagnosis of tumors
relied heavily on manual analysis of medical images by specialists. However,
manual identification of tumor signs is prone to errors and can be time-consuming.
With the advent of computer vision techniques, medical imaging analysis has
become more sophisticated and safer. These techniques involve the use of software
with built-in algorithms for tumor detection, classification, and segmentation,
thereby aiding in accurate diagnosis and treatment planning.
Early detection and classification of brain tumors are critical for effective
treatment and improved patient outcomes. Computer-aided diagnosis (CAD) or
brain tumor classification (BTC) methods have emerged as essential tools in
supporting radiologists in visualizing and defining tumor types. These automated
processes play a vital role in reducing the need for invasive procedures such as
surgery, thereby minimizing risks to patients.
Traditional BTC methods rely on low-level features and statistical learning
approaches for tumor classification. They involve preprocessing steps such as
contrast enhancement, image sharpening, and edge detection to estimate tumor
boundaries and localization. In contrast, deep learning-based methods leverage
convolutional neural networks (CNNs) and require significantly fewer
preprocessing steps. CNNs have demonstrated higher accuracy compared to
traditional approaches, making them suitable for real-world clinical practice.

1
 CNN-based BTC methods follow a three-step process involving preprocessing,
training, and testing. Preprocessing includes noise removal and segmentation to
isolate tumors from medical images. During training, CNN classifiers learn patterns
of different tumor grades from labelled training data. Testing involves feature
extraction from query images and prediction of tumor class/grade based on the
trained classifier. The accuracy of CNN classifiers is significantly higher compared
to traditional approaches, making them indispensable tools in clinical practice.
One of the key advantages of deep learning-based BTC methods is their ability
to generalize well to unseen data. This means that once trained on a diverse dataset
of medical images, CNN models can effectively analyse new images and provide
accurate predictions, even for cases that differ from those encountered during
training. This generalization ability is crucial for ensuring the robustness and
reliability of BTC systems in real-world clinical settings.
Moreover, deep learning methods offer scalability, allowing researchers to
develop more complex models capable of handling increasingly large and diverse
datasets. This scalability is particularly advantageous in the field of medical
imaging, where the volume and complexity of data continue to grow rapidly. By
harnessing the power of deep learning, researchers can build models that can
effectively analyse and interpret large volumes of medical images, enabling faster
and more accurate diagnoses.
Another significant benefit of deep learning-based BTC methods is their
potential for integration with other emerging technologies, such as smart healthcare
systems and telemedicine platforms. By leveraging deep learning models for
automated tumor detection and classification, healthcare providers can enhance the
efficiency and accessibility of medical services, particularly in remote or
underserved areas. This integration of deep learning-based BTC methods into smart
healthcare ecosystems holds promise for improving patient outcomes and reducing
healthcare disparities.

2
 2. MOTIVATION
The motivation to choose deep learning for multigrade brain tumor classification
in smart healthcare systems stems from the urgent need for more efficient and
accurate diagnostic tools in the field of medical imaging. Brain tumours represent a
significant health challenge due to their potentially life-threatening nature, making
early detection and classification crucial for effective treatment and patient
outcomes. Traditional diagnostic methods, while effective to some extent, often rely
on manual analysis by specialists, which can be time-consuming, subjective, and
prone to human error.
In recent years, the rapid advancements in deep learning, particularly
convolutional neural networks (CNNs), have revolutionized medical imaging by
enabling automated and highly accurate analysis of medical images. Deep learning
models have demonstrated remarkable capabilities in extracting complex features
from images and making accurate predictions, making them well-suited for tasks
such as brain tumor classification. By leveraging deep learning techniques,
researchers aim to develop robust and reliable diagnostic systems that can assist
healthcare providers in identifying and classifying brain tumors with unprecedented
accuracy and efficiency.
Furthermore, the scalability and generalization abilities of deep learning models
make them particularly well-suited for deployment in smart healthcare systems.
These systems leverage technologies such as cloud computing, edge computing, and
IoT devices to provide personalized and accessible healthcare services to patients.
By integrating deep learning-based brain tumor classification algorithms into smart
healthcare platforms, healthcare providers can enhance the efficiency and
accessibility of diagnostic services, particularly in remote or underserved areas
where access to specialized medical expertise may be limited.
Moreover, deep learning-based approaches offer the potential for continuous
improvement and refinement through the use of large-scale datasets and iterative
training processes. By training deep learning models on diverse datasets containing
annotated medical images, researchers can fine-tune the models to improve their
accuracy and robustness over time. This iterative approach to model development
allows for ongoing optimization and adaptation to new challenges and variations in
medical imaging data.

3
 The adoption of deep learning for multigrade brain tumor classification also
aligns with the broader trend towards digitization and automation in healthcare. As
medical imaging technologies continue to evolve and generate increasingly large
volumes of data, there is a growing need for automated tools that can assist
healthcare providers in analysing and interpreting this data quickly and accurately.
Deep learning-based approaches offer a scalable and efficient solution to this
challenge, enabling the development of automated diagnostic systems that can
handle the complexities of medical imaging data with ease.
Furthermore, the integration of deep learning-based brain tumor classification
algorithms into smart healthcare systems holds promise for improving patient
outcomes and reducing healthcare disparities. By providing timely and accurate
diagnoses, these systems can facilitate early intervention and personalized treatment
planning, leading to better outcomes for patients with brain tumours. Additionally,
the accessibility of smart healthcare systems can help overcome barriers to
healthcare access, particularly for patients in remote or resource-limited settings
who may not have access to specialized medical expertise.
Furthermore, the utilization of deep learning for multigrade brain tumor
classification holds immense potential to revolutionize the field of medical imaging
diagnostics. With its ability to automatically extract intricate features from medical
images, deep learning transcends the limitations of traditional manual analysis
methods, offering a more objective and consistent approach to tumor classification.
By leveraging vast amounts of labelled data and sophisticated neural network
architectures, deep learning models can uncover subtle patterns and nuances in
medical images that may evade human perception, thereby enhancing the accuracy
and reliability of tumor classification.
Moreover, the advent of smart healthcare systems presents an opportune moment
to integrate deep learning-based brain tumor classification algorithms into real-
world clinical practice. These systems leverage cutting-edge technologies such as
artificial intelligence, cloud computing, and interconnected devices to deliver
personalized and efficient healthcare solutions. By incorporating deep learning
models into the fabric of smart healthcare infrastructure, healthcare providers can
streamline the diagnostic process, reduce diagnostic errors, and improve patient
outcomes, all while optimizing resource allocation and healthcare delivery
efficiency.

4
 3. LITERATURE SURVEY

The paper titled [1] introduces a novel approach for tumor classification based
on multi view biological data. The authors propose a method that integrates supervised
discriminative sparse principal component analysis (PCA) with gene selection to
effectively classify tumor using diverse biological data sources. In the proposed method,
sparse PCA is employed to reduce the dimensionality of the multi view biological data
while preserving discriminative information relevant to tumor classification. Unlike
traditional PCA, which emphasizes data reconstruction, sparse PCA focuses on
selecting a subset of informative features. Moreover, the method incorporates
supervision to guide the feature selection process, ensuring that the selected features are
highly discriminative for tumor classification.

One of the key contributions of the paper is the integration of multiple biological
data views, such as gene expression data, methylation data, and clinical data, into a
unified framework for tumor classification. By leveraging information from diverse
biological perspectives, the proposed method aims to improve the accuracy and
robustness of tumor classification models.

The experimental results presented in the paper demonstrate the effectiveness of

the proposed approach compared to existing methods. The method achieves promising
results in terms of tumor classification accuracy and outperforms other techniques,
especially when dealing with high-dimensional and noisy biological data.

Overall, the paper provides valuable insights into the development of advanced
techniques for tumor classification using Multiview biological data. By combining
sparse PCA with supervised learning and integrating information from multiple
biological perspectives, the proposed method offers a promising approach for accurate
and reliable tumor classification, with potential applications in biomedical research and
clinical practice.

The paper titled [2] explores the application of deep learning techniques in the
domain of facial expression recognition, particularly for use in social robots. The
authors delve into the significance of accurately interpreting human facial expressions
for social robots to effectively interact with humans in various contexts.

The study focuses on leveraging deep learning algorithms to enable social robots
to recognize and interpret facial expressions in real-time. By analysing facial cues, these
robots can discern emotions such as happiness, sadness, anger, and surprise, allowing
them to respond appropriately to human interactions. The research aims to enhance the
capabilities of social robots to engage with humans in natural and intuitive ways,
fostering more seamless human-robot interactions.

5
 The authors discuss the methodology employed, which involves training deep
neural networks on large datasets of facial images annotated with corresponding
expressions. Through this training process, the neural networks learn to extract
meaningful features from facial images and make accurate predictions about the
associated emotions. The study evaluates the performance of various deep learning
models and explores techniques to optimize their accuracy and efficiency.

Overall, the paper highlights the potential of deep learning in advancing the field
of social robotics by enabling robots to perceive and interpret human emotions through
facial expressions. By integrating sophisticated facial expression recognition
capabilities, social robots can better understand and respond to human emotions,
thereby enhancing their effectiveness in various social and interactive settings.

The paper titled [3] presents a method for detecting and classifying weld defects
using machine vision techniques. The authors address the critical need for accurate and
efficient detection of weld defects in industrial settings, where the quality of welds
directly impacts the safety and integrity of structures.

The proposed method leverages machine vision technology to analyse images of

welds and automatically identify various types of defects. The authors highlight the
importance of detecting defects such as cracks, porosity, and incomplete fusion early in
the welding process to prevent structural failures and ensure product quality.

Key components of the proposed method include image preprocessing

techniques to enhance the quality of input images, feature extraction methods to capture
relevant information from the images, and machine learning algorithms for defect
classification. The authors emphasize the effectiveness of combining advanced image
processing algorithms with machine learning models to achieve accurate and reliable
defect detection and classification.

Experimental results demonstrate the efficacy of the proposed method in

accurately detecting and classifying weld defects across different types of welds and
varying environmental conditions. The method shows promising performance metrics,
including high detection accuracy and low false positive rates, indicating its potential
for practical deployment in industrial settings.

Overall, the paper contributes to the field of machine vision-based defect

detection by presenting a comprehensive approach tailored specifically for weld
inspection applications. The proposed method offers a viable solution to the challenges
associated with manual inspection methods, paving the way for improved quality
control and safety in welding processes.

6
 The paper [4] explores the applications of deep learning and reinforcement
learning in the context of biological data analysis. The authors provide a comprehensive
overview of how these advanced machine learning techniques can be utilized to extract
insights and patterns from biological datasets, thereby advancing our understanding of
various biological processes.

One key aspect of the paper is its discussion on the integration of deep learning
methods with biological data analysis. Deep learning algorithms, characterized by their
ability to automatically learn hierarchical representations from raw data, offer powerful
tools for feature extraction and pattern recognition in complex biological datasets. By
leveraging techniques such as convolutional neural networks (CNNs) and recurrent
neural networks (RNNs), researchers can effectively analyse diverse types of biological
data, including genomic sequences, protein structures, and medical imaging data.

Additionally, the paper highlights the emerging role of reinforcement learning

(RL) in biological data analysis. RL, a branch of machine learning concerned with
decision-making and sequential behavior, offers a novel approach to optimizing
experimental design and guiding data-driven exploration in biological research. By
formulating biological problems as RL tasks, researchers can train agents to interact
with biological systems, adaptively learn from feedback, and discover optimal strategies
for experimentation and analysis.

Overall, Mahmud et al. provide valuable insights into the potential applications
of deep learning and reinforcement learning in the field of biology. The paper
underscores the importance of integrating advanced machine learning techniques with
biological research, offering new avenues for discovery and innovation in the life
sciences.

The paper titled [5] presents a novel approach to automatic sleep staging using
advanced neural network techniques. Authored by P. Christos, C. A. Frontside, P. T.
Gigli, P. D. Abidis, and C. Kourtidou-Papadeli, the study focuses on leveraging
convolutional neural networks (CNNs) and cortical connectivity images for accurate
sleep stage classification.

The main objective of the research is to develop a robust and efficient system for
automatic sleep staging, which plays a crucial role in diagnosing sleep disorders and
monitoring sleep patterns. Traditional methods for sleep staging often require manual
interpretation of polysomnography (PSG) recordings, which can be time-consuming
and prone to human error. In contrast, the proposed approach aims to automate this
process using CNNs and cortical connectivity images derived from
electroencephalogram (EEG) signals.

7
 The paper introduces a novel CNN architecture designed specifically for
analysing cortical connectivity images, which capture the functional connectivity
patterns within the brain during different sleep stages. By training the CNN on a large
dataset of labelled cortical connectivity images, the researchers demonstrate the
effectiveness of their approach in accurately classifying sleep stages.

The results of the study show promising performance in automatic sleep staging,
with the proposed CNN model achieving high accuracy in distinguishing between
different sleep stages. The use of cortical connectivity images as input features allows
the model to capture intricate patterns in brain activity associated with different sleep
stages, leading to improved classification accuracy compared to traditional methods.

Overall, the paper contributes to the field of sleep research by presenting a novel
approach to automatic sleep staging using CNNs and cortical connectivity images. The
proposed method has the potential to streamline the process of sleep stage classification,
facilitating more efficient diagnosis and treatment of sleep disorders.

In the paper titled [6] authored by Y. Shi, H.-I. Suk, Y. Gao, S.-W. Lee, and D.
Shen, the authors propose a novel approach for the diagnosis of Alzheimer's disease
(AD) using multimodal data. Alzheimer's disease is a neurodegenerative disorder
characterized by cognitive decline, and early and accurate diagnosis is crucial for
effective management and intervention.

The authors leverage coupled interaction between different modalities of data,

such as structural magnetic resonance imaging (MRI) and positron emission
tomography (PET), to improve the accuracy of AD diagnosis. By integrating
information from multiple sources, including anatomical features and molecular
biomarkers, the proposed method aims to capture complementary information and
enhance the discriminative power of the diagnostic model.

The paper introduces a framework that jointly analyses multimodal data through
coupled interaction, thereby enabling the model to exploit the inherent relationships
between different data modalities. The proposed method incorporates advanced
machine learning techniques, such as deep learning and neural networks, to effectively
learn complex patterns and features from the multimodal data.

Experimental results demonstrate the effectiveness of the proposed approach in

improving the accuracy of Alzheimer's disease diagnosis compared to conventional
single-modality methods. The coupled interaction framework achieves promising
results in terms of classification performance, indicating its potential utility in clinical
settings for early detection and diagnosis of Alzheimer's disease.

8
 Overall, the paper contributes to the growing body of research in the field of
medical imaging and machine learning by providing a novel approach for multimodal
Alzheimer's disease diagnosis that leverages coupled interaction between different data
modalities to enhance diagnostic accuracy.

The paper titled [7] presents a novel approach for classifying biomedical figures
using a stacked deep autoencoder model. The authors address the challenge of
effectively categorizing complex biomedical images, which often contain intricate
structures and subtle features that make classification challenging.

The proposed model utilizes a stacked deep autoencoder architecture, which is a

type of artificial neural network known for its ability to learn hierarchical
representations of data. By leveraging this architecture, the model aims to automatically
extract meaningful features from biomedical images, enabling accurate classification
into predefined categories.

The paper describes the methodology used to train and evaluate the stacked deep
autoencoder model, including details on the dataset used for experimentation and the
performance metrics employed for evaluation. The authors likely detail the training
process, including parameter optimization and model validation techniques, to ensure
robust performance.

Results from experiments conducted using the proposed model are likely
presented and discussed, highlighting its effectiveness in accurately classifying
biomedical figures. This may include comparisons with existing methods or alternative
architectures to demonstrate the superiority of the stacked deep autoencoder model in
terms of classification accuracy and computational efficiency.

Overall, the paper likely concludes by discussing the implications of the

proposed approach for biomedical research and applications, as well as suggesting
potential avenues for future research to further improve the performance and
applicability of the model.

The paper titled [8] presents a novel approach for extracting key frames from
hysteroscopy videos using a visual attention-driven framework. The authors propose a
method that leverages visual attention mechanisms to identify the most informative
frames in the videos, thereby facilitating efficient analysis and interpretation by medical
professionals.

The framework outlined in the paper integrates techniques from visual attention
modelling with computational methods for key frame extraction. By simulating the
human visual system's ability to prioritize relevant information, the proposed method
identifies regions of interest within the hysteroscopy videos based on saliency and

9
motion cues. These regions are then used to select key frames that encapsulate the
essential diagnostic information contained within the video sequence.

The efficacy of the proposed framework is evaluated through experiments

conducted on a dataset of hysteroscopy videos. The results demonstrate that the visual
attention-driven approach outperforms traditional methods for key frame extraction in
terms of accuracy and efficiency. By focusing computational resources on the most
informative frames, the proposed method enables more streamlined analysis of
hysteroscopy videos, potentially leading to improved diagnostic outcomes and
enhanced clinical decision-making.

Overall, the paper contributes to the field of medical image analysis by

introducing a novel framework for key frame extraction from hysteroscopy videos. The
integration of visual attention mechanisms enhances the efficiency and effectiveness of
the analysis process, offering valuable insights for medical professionals and
researchers in the field of gynaecology and obstetrics.

The paper titled [9] authored by M. Sajjad et al. and published in IEEE Access
in 2017, focuses on advancing healthcare services by automating the classification and
segmentation of leukocytes in microscopic blood smear images. This innovation is
particularly valuable in smart city environments, where efficient and accessible
healthcare solutions are essential for urban populations.

The study outlines a comprehensive approach that combines image

preprocessing, feature extraction, and machine learning techniques to achieve accurate
leukocyte classification and segmentation. By leveraging advanced image processing
algorithms, the system enhances the quality of blood smear images, facilitating the
subsequent analysis process. Machine learning algorithms, including support vector
machines (SVM) and artificial neural networks (ANN), are trained on annotated
datasets to learn the distinguishing features of different leukocyte types.

A key aspect of the research is its emphasis on resource-awareness in healthcare

services. In the context of smart cities, where computational resources may be limited,
it is crucial to develop efficient and scalable solutions. The proposed system is designed
to optimize resource utilization while maintaining high levels of accuracy and
reliability. By adopting a resource-aware approach, the researchers aim to ensure the
practical feasibility and sustainability of the healthcare service in smart city
environments.

The paper presents the results of experiments conducted to evaluate the

performance of the proposed system. Through rigorous testing and evaluation, the
researchers demonstrate the effectiveness of their approach in accurately classifying and
segmenting leukocytes in microscopic blood smear images. Key metrics such as

10
accuracy, sensitivity, and specificity are used to assess the system's performance,
providing valuable insights into its capabilities and potential applications in real-world
healthcare settings.

Overall, the research contributes to the advancement of automated healthcare

services by addressing the challenges associated with leukocyte analysis in microscopic
blood smears. By combining image processing techniques with machine learning
algorithms, the proposed system offers a promising solution for enhancing medical
diagnostics in smart city environments, ultimately improving the quality and
accessibility of healthcare for urban populations.

The paper titled [10] by Mehmood et al. presents the development and
implementation of a computerized decision support system (CDSS) aimed at analysing
and visualizing brain tumors in three dimensions (3D). The system is designed to assist
medical professionals in diagnosing brain tumors more accurately and efficiently.

The authors begin by highlighting the importance of accurate diagnosis and

visualization in the context of brain tumors, emphasizing the critical role that
technology can play in aiding medical professionals in this task. They identify the
limitations of traditional manual analysis methods and propose the use of computerized
systems to overcome these challenges.

The paper outlines the design and functionality of the proposed CDSS, which
includes modules for image preprocessing, feature extraction, tumor classification, and
3D visualization. The system utilizes advanced image processing techniques and
machine learning algorithms to extract relevant features from medical images and
classify brain tumors into different categories based on these features.

One of the key features of the CDSS is its ability to visualize brain tumors in 3D,
providing medical professionals with a more comprehensive understanding of tumor
morphology and spatial relationships. The authors discuss the benefits of 3D
visualization in enhancing diagnostic accuracy and treatment planning.

The effectiveness of the proposed CDSS is evaluated through experimental

validation, where it is compared against traditional manual analysis methods. The
results demonstrate the superiority of the CDSS in terms of accuracy, efficiency, and
reliability, thus highlighting its potential as a valuable tool in the diagnosis and
visualization of brain tumors.

Overall, the paper presents a comprehensive overview of the development and

implementation of a computerized decision support system for brain tumor analysis and
3D visualization, offering promising results and insights into the potential applications
of technology in the field of medical imaging diagnostics.

11
TABLE 1. DETAILED DESCRIPTION OF THE DEEP LEARNING BASED METHODS WITH THEIR
RESPECTIVE SEGMENTATION, FEATURES, CLASSIFIERS, AND TOTAL NUMBER OF CLASSIFIED
CLASSES

12
 4. METHOLOGY

4.1 BTC Method

In the realm of Brain Tumor Classification (BTC), various methodologies have been
developed, aiming to provide accurate and efficient diagnosis and classification of brain
tumors. The general pipeline of BTC involves the segmentation of the tumor region as the
primary step, which is predominantly targeted by existing segmentation techniques. Some
methods utilize end-to-end models for both segmentation and classification, while others focus
solely on BTC approaches or use segmentation followed by classification.

Several tumor segmentation methods are employed as the initial step prior to
classification in the reviewed literature. For example, Akkus et al. utilized a three-step process
involving image registration, tumor segmentation, and classification using a convolutional
neural network (CNN) model. Data augmentation techniques are often employed to balance
data distribution and enhance classification performance. Similarly, other studies employ
different segmentation techniques and feature extraction methods before passing the features
to a classifier for tumor classification.

Certain methods concentrate solely on BTC, addressing challenges such as the need for
large volumes of training data and the capability to handle image transformations effectively.
For instance, Afshar et al. explored Capsule Networks (Caps Nets) as an alternative to CNNs
for BTC, aiming to achieve maximum accuracy while addressing issues such as overfitting and
image background sensitivity. The study highlights the effectiveness of Caps Nets for BTC but
also acknowledges their sensitivity to image background variations.

Other methods proposed multiscale CNN models or utilized pretrained CNN models
for tumor classification. Ge et al. introduced a 3D multiscale CNN model for glioma tumor
classification, incorporating a feature fusion scheme to enhance tumor region refinement.
Decoyer et al. employed a pretrained VGG model for feature extraction and classification,
while Pereira et al. utilized a 3D-U-Net model for tumor segmentation followed by
classification using a proposed CNN model.

13
A. Multigrade Brain Tumor Data Set
There are two variants of this data set: the original one that consists of 121 MRI
instances and the augmented data set, containing 3630 MRI instances created from the original
images. The overall data set is divided into four different grades according to the standard
classification of WHO tumors of the Central Nervous System [77]. The overall distribution and
statistics of this data set are given in Table III, and sample images are visualized in Fig. 5(a).The
information about data augmentation and other necessary details are given in [32].

B. Brain Tumor Public Data Set

This data set was captured from two different hospitals in China in the duration of
2005–2010. It consists of 3064 T1-weighed CE-MRI slices, collected from 233 different
patients. The size of each slice in this data set is 512 × 512, with 6 and 1 mm, slice thickness
and gap, respectively. The tumor region inside each slice is segmented manually by three
experienced radiologists. This data set is divided into three classes, i.e., Meningiomas,
Gliomas, and Pituitary tumors. The complete statistics of this data set are given in Table IV,
while representative slices from each class are shown in Fig. 5(b).

C. Cancer Imaging Archive (TCIA)

This repository contains several collections of cancer imagery, but only a few of them are
related to our problem of BTC. The concerned group with our problem in this repository is
BRAIN-DSC-MRI, which contains two types of brain tumors: low- and high-grade gliomas.
The data is collected from 49 patients of different ages.

D. BRATS 2015
The BRATS 2015 data set is created for brain tumor segmentation. However, in several
contributions [78]–[80], it has been used for tumor classification. It consists of two types of
tumors: low- and high-grade gliomas. The overall data set consists of 274 MR scans with 220
and 54 for high- and low-grade glioma, respectively. The MRI scanning is performed using
four modalities: T1, T1c, T2, and Flair, with an image size of 240 × 240 × 155.

E. Harvard (AANLIB)
The whole-brain Atlas or ANNLIB is an online repository for MRI of the Central Nervous
System. This database is available online, consisting of more than 13 000 brain MRIs of 30
different cases. These MRIs contain a large variety of normal and tumor images including
different types of stroke or brain attacks, several types of gliomas, Alzheimer’s, and infectious
diseases.

F. Internet Brain Segmentation Repository

IBSR is an open-source repository for brain tumor segmentation and classification. This data
set consists of 18 T1 3-D MRI scans. Each of the MRI scans contains 60–65 slices, with a
resolution of 256 × 256 pixels. The data is collected from 14 male and four female patients
aged seven to 71 years, covering a large variability of brain anatomies.

14
4.2 TRANSFER LEARNING FOR MEDICAL IMAGE ANALYSIS
Due to the advancement in deep learning, transfer learning techniques have been
integral to almost every field of computer vision, i.e., multimedia [81], surveillance [82], and
medical [83]. Among these domains, transfer learning in medical imaging is the most
prominent, where the weights of standard models, trained on medical images or natural image
classification data sets, particularly ImageNet [84], are fine-tuned on medical imaging data.
This transfer learning process is adopted in almost every modality of medical imaging,
including X-rays, CT scans, pathological images, positron emission tomography (PET), and
MRI [85]. In this section, our main target is to review transfer learning techniques based on
MR images, especially brain tumor segmentation, classification, and retrieval. The detailed
descriptions of each method including publication year, pretrained model, and data set with its
main target, are given in Table V. The range of these methods is from 2015 to 2019, starting
from traditional machine learning techniques to state-of-the art deep learning models. In
addition, we also review a few transfers learning methods, focusing on breast cancer
recognition and prostate cancer classification, to show the importance of transfer learning in
the medical imaging domain, i.e., MR images.

4.3 COMPARATIVE STUDY OF DIFFERENT CNNS

In Section III, various CNN architectures and their variants, such as VGG-16, VGG-
19, and Caps Nets, were discussed in the context of Brain Tumor Classification (BTC).
However, the baseline and most popular CNN architectures have not been thoroughly explored
for this problem. Additionally, some existing studies have relied on a single dataset for their
experiments and validation, limiting the generalizability of their findings. Therefore, it is
essential to investigate both baseline and recent CNN models for BTC using multiple datasets.
To address this gap, this section compares several state-of-the-art CNNs for BTC using two
datasets: Data set 1 and Data set 2.

The CNNs selected for comparison include AlexNet, Google Net, VGG, Squeezenet,
Mobile Net, and ResNet. These models were evaluated using a NVidia GeForce TITAN X
(Pascal) GPU and the Caffe deep learning framework with NVidia DIGITS for evaluation.
During the training process, various parameters such as the number of epochs, learning rate,
solver type, and SoftMax loss function were considered to optimize model performance.

The results of the experiments, presented in Table VI, include three evaluation metrics:
accuracy, frames per second (fps), and model size. Accuracy measures the correct predictions
of each BTC approach, fps indicates the processing speed of each method, and model size
reflects the memory requirements for deploying the final prediction architecture. While
accuracy is the primary metric for comparison, fps and model size are also considered to
evaluate practicality and efficiency.

From the results, it is observed that Squeezenet achieved the best fps and model size
due to its efficiency. However, it showed signs of overfitting, being biased toward a single class
on both datasets. AlexNet and Google Net achieved similar results, but their accuracy was not
sufficiently high for consideration in critical applications like CAD systems. Similarly, Mobile
Net and ResNet exhibited lower accuracy and are not deemed trustworthy for real-world BTC
systems.

15
 VggNet emerged as the top performer in terms of accuracy, outperforming other CNNs
considered. Although it had a similar average fps to other models, its model size was larger.
These findings offer valuable insights for industry and hospitals in selecting suitable methods
based on their requirements, accuracy needs, deployment environment, and other constraints.
Overall, the comparative analysis provides a comprehensive understanding of the strengths and
limitations of different CNN architectures for BTC.

16
17
 CHALLENGES, RECOMMENDATIONS, AND FUTURE
RESEARCH DIRECTIONS

Processing and analysing MRI data of brain tumors are among the most challenging
tasks for computer vision scientists. MRI is an advanced technique for producing high-quality
images of the human body parts [115]. It plays a key role in processing and detecting the right
stage and in deciding the correct therapy for the tumor-infected person. To accomplish this
task, researchers have proposed many automatic techniques by using MRI (T1, T2, and FLAIR)
[116], [117]. The main reason is that these MRIs are not affected by radiations, and their
contrast is better compared with other modalities. This point is taken into consideration in
medical image analysis, as several parameters, including similarity measures, modality, image
contents, transformation, implementation, and optimization of algorithms, affect the
performance. Similarly, the selection of a machine learning method for BTC is also a crucial
step, requiring a careful assessment. Most machine learning methods resort to features
extracted from images via traditional strategies, followed by their classification. This process
can be overly complex and time-consuming if the extracted features are highly dimensional.
Other problems associated with machine learning approaches include the diversity of classes
and challenges associated with distance measurement between images. It can also be observed
that medical images are usually affected by low-light contrast, deteriorating their quality, which
consequently affects the classification accuracy. Recently, methods relying on learned
representations (deep learning, especially CNNs) have gained momentum for BTC problems
at the expense of handcrafted feature-based methods [41]. Despite their strength and huge
popularity, CNN-based methods encounter many challenges. For instance, they require a huge
amount of data for training, which can be either not available for each domain, or it can be very
challenging to get the desired accuracy for a target problem [118].

A. Public Availability of BTC Data Sets

The main issue with the BTC literature is the scarcity of public data sets. Many researchers
are passionate to work in this field, but there are still a severely limited number of publicly
available data repositories. This restricts the experimentation and testing of new BTC methods
and their maturity compared with other domains, where data sets can be accessed freely (e.g.,
ImageNet data set [120]). As mentioned previously, the major requirement of any deep learning
model is the huge amount of annotated data for achieving better accuracy scores.
Unfortunately, most of the existing BTC data sets contain a limited number of images for each
class, resulting in lower accuracy levels for the implemented methods. For this reason, we used
data augmentation in our experiments in Section VI.

B. End-to-End Deep Learning Models

Although the majority of the recent techniques are based on deep learning, they use different
models for detection, classification, and segmentation, as discussed in Section III. This
increases the computational complexity of the implemented methods, making them less
suitable for their consideration in clinical practice. Currently, there is no end-to-end deep
learning model, which can detect a tumor in the input MRI image, segment it, and classify its
nature as a final output. Thus, both industry and academia are highly encouraged to further
investigate deep learning models for the problem of BTC in this context. This can greatly
reduce the overall running time of the target BTC model, ultimately matching the practical
constraints of smart healthcare and clinical practice.

18
 C. Edge Intelligence for MRI Data Analysis

Edge intelligence is used on a wide scale in different domains [121], [122] because of its
numerous advantages, such as reduced bandwidth and threats, minimal latency, improved
consistency, compliance, and lower cost. For instance, Chen et al. [123] presented a deep
learning and edge intelligence-assisted system for distributed video surveillance applications.
Their system processes data at network edges instead of network centres, reducing
communication overhead, thereby providing accurate video analysis results with elastic and
scalable computing power. Similarly, Pace et al. [124] proposed “Body Edge,” a framework
for supporting healthcare applications in industry 4.0 with the main focus on reducing the data
load toward the Internet. In the field of BTC, the MRI data are normally collected in the Digital
Imaging and Communications in Medicine (DICOM) format, which is manually converted into
slices for further analysis. This process is time consuming and tedious with comparatively
higher changes of errors. Through edge intelligence, the DICOM images can be automatically
processed over the capturing device for efficient analysis with better accuracy. Currently, there
is no such a concept of edge intelligence for the specific problem of BTC, which can be a new
trend for further research in this domain.

D. Merging Fog and Cloud Computing With Federated

Learning: A New Dawn for BTC Fog computing is an extension of cloud computing performed
over the edge through a distributed network. Fog computing makes it easy to facilitate the
regular processing and generates the output fast enough by using capabilities of edge network
[125]. In the medical field, the data collected from a specific MRI capturing device should be
quickly formulated and processed over different cloud and fog layers for efficient analysis.
This can be possible by exploring fog and cloud computing with an extensive investigation of
the new emerging framework of AI “Federated Learning” (FA) for BTC in future studies. In
FA architecture, models use the distributed mobile/edge devices for computation due to their
recently improved capabilities for executing a machine learning model. Using this hybrid
computing platform, a model can be improved by raining it locally via the data collected by
the concerned edge device, and the changes in terms of model parameters and weights can be
reported to cloud through a secure communication link, e.g., homomorphic encryption as
employed by Feng et al. [126] for outsourcing big data in the federated cloud environment.
The most important aspect of FA is the preservation of user’s privacy, which is utterly
important in the medical domain. In the case of BTC, the output of brain tumors, classified into
various grades, will be generated directly over the cloud or fog, making the overall process
smarter and more feasible compared with manual lengthy processes.

19
 CONCLUSION

Considering the recent development in the domain of BTC and the limitations of existing
studies, we presented a comprehensive survey of deep learning-based BTC methods. Deep
learning technologies accurately assist radiologists in predicting the tumor regions and further
classifying them into their respective types. Many researchers contributed to the field of BTC,
but many challenges remain therein. Therefore, we conducted this study to provide the overall
literature of deep learning-based BTC methods in a single survey and to draw the attention of
both academia and industry toward the necessary development in this domain. This article
comprehensively discussed all deep learning-based BTC methods, with their achievements and
weaknesses, followed by complete information about the existing publicly available data sets
with their respective resources. In order to empirically inform the conclusions drawn from our
literature study, we experimentally analyse various deep learning models by performing
extensive experiments over BTC data sets and highlighted the suitable option for consideration
in smart health care. Finally, this study highlighted key challenges, such as lack of public data
sets and end-to-end deep learning models, and suggested detailed directions for further research
in BTC domain, i.e., exploring edge/fog/cloud computing with FA, advanced data-enrichment
techniques, model confidence and explainability, IoMT, and deep investigation of sequential
and transfer learning strategies. This can increase the maturity level of BTC methods with
better applicability for commercial clinical applications and their smooth integration with smart
healthcare. The brain is an intriguing system whose complexity demands sophisticated means
to understand and characterize its behavior. The unrivalled learning capability of deep learning
models has made them the standard choice to detect and classify brain tumors from MRI
images and other monitored data alike, spawning a flurry of research activity overviewed in
this survey. We hope that the numerous research paths outlined in our overview will serve as
supportive material for the research community currently working on this field and a
stimulating read for newcomers to this domain.

20
 REFERENCES

[1] C. -M. Feng, Y. Xu, J. -X. Liu, Y. -L. Gao and C. -H. Zheng, "Supervised Discriminative
Sparse PCA for Com-Characteristic Gene Selection and Tumor Classification on Multiview
Biological Data," in IEEE Transactions on Neural Networks and Learning Systems, vol. 30,
no. 10, pp. 2926-2937, Oct. 2019, doi: 10.1109/TNNLS.2019.2893190.

[2]J. Sun, C. Li, X. -J. Wu, V. Palade and W. Fang, "An Effective Method of Weld Defect
Detection and Classification Based on Machine Vision," in IEEE Transactions on Industrial
Informatics, vol. 15, no. 12, pp. 6322-6333, Dec. 2019, doi: 10.1109/TII.2019.2896357.

[3] M. Mahmud, M. S. Kaiser, A. Hussain and S. Vassanelli, "Applications of Deep Learning

and Reinforcement Learning to Biological Data," in IEEE Transactions on Neural Networks
and Learning Systems, vol. 29, no. 6, pp. 2063-2079, June 2018, doi:
10.1109/TNNLS.2018.2790388.

[4] P. Chriskos, C. A. Frantzidis, P. T. Gkivogkli, P. D. Bamidis and C. Kourtidou-Papadeli,

"Automatic Sleep Staging Employing Convolutional Neural Networks and Cortical
Connectivity Images," in IEEE Transactions on Neural Networks and Learning Systems, vol.
31, no. 1, pp. 113-123, Jan. 2020, doi: 10.1109/TNNLS.2019.2899781.

[5] A. Arnaud, F. Forbes, N. Coquery, N. Collomb, B. Lemasson and E. L. Barbier, "Fully

Automatic Lesion Localization and Characterization: Application to Brain Tumors Using
Multiparametric Quantitative MRI Data," in IEEE Transactions on Medical Imaging, vol. 37,
no. 7, pp. 1678-1689, July 2018, doi: 10.1109/TMI.2018.2794918.

[5] F. Liao, M. Liang, Z. Li, X. Hu and S. Song, "Evaluate the Malignancy of Pulmonary
Nodules Using the 3-D Deep Leaky Noisy-OR Network," in IEEE Transactions on Neural
Networks and Learning Systems, vol. 30, no. 11, pp. 3484-3495, Nov. 2019, doi:
10.1109/TNNLS.2019.2892409.

[6] D. Nie, L. Wang, Y. Gao, J. Lian and D. Shen, "STRAINet: Spatially Varying sTochastic
Residual AdversarIal Networks for MRI Pelvic Organ Segmentation," in IEEE Transactions
on Neural Networks and Learning Systems, vol. 30, no. 5, pp. 1552-1564, May 2019, doi:
10.1109/TNNLS.2018.2870182.

[7] C. Ma, G. Luo and K. Wang, "Concatenated and Connected Random Forests With
Multiscale Patch Driven Active Contour Model for Automated Brain Tumor Segmentation of
MR Images," in IEEE Transactions on Medical Imaging, vol. 37, no. 8, pp. 1943-1954, Aug.
2018, doi: 10.1109/TMI.2018.2805821.

[8] Z. Tang, S. Ahmad, P. -T. Yap and D. Shen, "Multi-Atlas Segmentation of MR Tumor
Brain Images Using Low-Rank Based Image Recovery," in IEEE Transactions on Medical
Imaging, vol. 37, no. 10, pp. 2224-2235, Oct. 2018, doi: 10.1109/TMI.2018.2824243.

[9] N. M. Balasooriya and R. D. Nawarathna, "A sophisticated convolutional neural network

model for brain tumor classification," 2017 IEEE International Conference on Industrial and
Information Systems (ICIIS), Peradeniya, Sri Lanka, 2017, pp. 1-5, doi:
10.1109/ICIINFS.2017.8300364

21