Brain Tumor Detection From MRI Image Using YOLOV3

Mini Project report submitted to

Central Institute of Technology, Kokrajhar

in partial fulfillment for the award of the degree of

Bachelor of Technology

in

Computer Science and Engineering

by

Manika Debnath,Bhagyasri Bora,Tilok Pegu

(GAU-C-17/079, GAU-C-17/074, GAU-C-16/051)

Under the supervision of

Mr. Mithun Karmakar (Assistant Professor)

Computer Science and Engineering

Central Institute of Technology, Kokrajhar

5th Semester, 2019

26 November, 2019
 DECLARATION

We certify that

(a) The work contained in this report has been done by us under the guidance
of our supervisor.

(b) The work has not been submitted to any other Institute for any degree or
diploma.

(c) We have conformed to the norms and guidelines given in the Ethical Code
of Conduct of the Institute.

(d) Whenever we have used materials (data, theoretical analysis, gures, and text)

from other sources, we have given due credit to them by citing them in the text of

the thesis and giving their details in the references. Further, we have taken

permission from the copyright owners of the sources, whenever necessary.

Date: November 26, 2019

Manika Debnath, Bhagyasri Bora, Tilok Pegu

Gau-C-17/079, Gau-C-17/074, Gau-C-16/051

Place: Kokrajhar
 COMPUTER SCIENCE AND ENGINEERING
CENTRAL INSTITUTE OF TECHNOLOGY,
KOKRAJHAR
KOKRAJHAR - 783370, INDIA

CERTIFICATE

This is to certify that the project report entitled “Brain Tumor Detection from
Magnetic Resonance Imaging using YOLOV3" submitted by Manika Debnath,
Bhagyasri Bora, Tilok Pegu (Roll No. GAU-C-17/079, GAU-C-17/074, GAU-C-
16/051) to Central Institute of Technology, Kokrajhar towards partial fulfillment of
requirements for the award of degree of Bachelor of Technology in Computer
Science and Engineering is a record of bona fide work carried out by them under
my supervision and guidance during 5th Semester, 2019.

Mr. Mithun Karmakar (Assistant

Professor)
Date:November 26, 2019 Computer Science and Engineering
Place: Kokrajhar Central Institute of Technology, Kokrajhar
Kokrajhar - 783370, India
 COMPUTER SCIENCE AND ENGINEERING
CENTRAL INSTITUTE OF TECHNOLOGY,
KOKRAJHAR
KOKRAJHAR - 783370, INDIA

BONAFIDE CERTIFICATE

This is to certify that the project titled “Brain Tumor Detection from Magnetic
Resonance Imaging using YOLO V3" is a bona fide record of the work done by
Manika Debnath(GAU-C-17/079), Bhagyasri Bora(GAU-C-17/074) and Tilok
Pegu(GAU-C-16/051) towards partial fulfillment of requirements for the award of
degree of Bachelor of Technology in Computer Science and Engineering of the
CENTRAL INSTITUTE OF TECHNOLOGY, KOKRAJHAR, during the year
2019.

Mr. Ranjan Maity Dr.Hemanta kumar kalita

Assistant Professor Head of the Department
Project Coordinator

Examiner
 Abstract

Brain Tumor is a brain disease, causing pain and disability in older people,
and it can be characterized by progressive degradation of the brain
membrane. Tumor can affect the brain from functioning as it should.
Detection and progress monitoring of Brain Tumor can be done by
measuring pre-structural and structural changes associated with the
tissues. This project provides a step towards a solution by fully automating
the extraction of tumor in brain MRIs using YOLO to reduce the data
complexity. These results show an accuracy of 98% detection of tumor
when compared to the work of previous studies. This project concludes
with a proof of concept estimating the Tumor of individuals using the
generated annotations. The results with more weights produce more
accuracy.
 Acknowledgements
We would like to express our gratitude and appreciation to all those who gave us the opportunity
to complete this report. A special thanks to our project coordinator, Mr. Ranjan Maity ,Asst.
Professor,CSE Department,CIT Kokrajhar, whose help, stimulating suggestions and
encouragement, helped us to coordinate our project especially in writing this report.

We would also like to acknowledge with much appreciation the crucial role of the sta of
Computer Science and Engineering Lab, who gave the permission to use all required machinery
and the necessary material to complete the project.

Last but not least, many thanks go to the head of the project, Mr. Mithun Karmakar, Asst.
Professor,CSE Department,CIT Kokrajhar who have given his full e ort in guiding the team in
achieving the goal as well as his encouragement to maintain our progress in track. We would
like to appreciate the guidance given by other supervisor as well as the panels especially in our
project presentation that has improved our presentation skills by their comment and tips.

Manika Debnath
Bhagyasri Bora
Tilok Pegu
Contents

1 Introduction 1
1.1 Field and Context . . . . . . . . . . . . . . . . . . . . . . . . 1
1.2 Research Problem . . . . . . . . . . . . . . . . . . . . . . . . 2
1.3 Focus of the project . . . . . . . . . . . . . . . . . . . . . . . . 3

2 Background 4
2.1 Medical Knowledge . . . . . . . . . . . . . . . . . . . . . . . . 5
2.1.1 Magnetic Resonance Imaging . . . . . . . . . . . . . . 5
2.1.2 Brain Tumor . . . . . . . . . . . . . . . . . . . . . . 6
2.2 Computer Science . . . . . . . . . . . . . . . . . . . . . . . . 6
2.2.1 Artificial Intelligence . . . . . . . . . . . . . . . . . . . 7
2.2.2 Machine Learning . . . . . . . . . . . . . . . . . . . . 7
2.2.3 Artificial Neural Networks . . . . . . . . . . . . . . . 8
2.2.4 Tiny YOLO V3 . . . . . . . . . . . . . . . . . . . . . . . . . . 9

3 Methods 11
3.1 Setup . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11
3.2 Data Set . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11
3.3 Preprocessing . . . . . . . . . . . . . . . . . . . . . . . . . . . 12
3.3.1 Training, Validation, and Testing Sets . . . . . . . . . 12
3.3.3 2D and 3D data . . . . . . . . . . . . . . . . . . . . . 14
3.3.4 Annotation . . . . . . . . . . . . . . . . . . . 15
3.4 Architecture . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15
3.4.1 Channels, Growth Rate, and Depth . . . . . . . . . . 16
3.4.2 Activation Functions . . . . . . . . . . . . . . . . . . . 17
3.5 Training . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 18
3.5.1 Metrics and Loss Functions . . . . . . . . . . . . . . . 18
3.5.2 Batch Size . . . . . . . . . . . . . . . . . . . . . . . . . 19
3.5.4 Learning Rate Policy . . . . . . . . . . . . . . . . . . . 20
3.5.5 Early Stopping . . . . . . . . . . . . . . . . . . . . . . 20
3.6 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 20

4 Results 22
4.2 Visual Evaluation . . . . . . . . . . . . . . . . . . . . . . . . . 23

5 Discussion 27

6 Conclusion 27
1 Introduction

Recent advances in artificial intelligence have led to fully automated work-

flows that often exceed human performance. State of the art neural networks
can classify images into thousands of categories more accurately and magni-
tudes faster than humans [1]. They translate texts from multiple languages
[2], drive cars autonomously through cities [3], and detect malware in com-
puter systems [4]. In most of these cases, they have been trained on tens of
thousands, or even millions, of data samples. Neural networks have also
found great success in the field of medical image analysis where data sets
are often much smaller. Although the same techniques can be applied, one is
often confronted with a different set of challenges.

1.1 Field and Context

A brain tumour is a collection, or mass, of abnormal cells in your brain. Your

skull, which encloses your brain, is very rigid. Any growth inside such a
restricted space can cause problems. Brain tumours can be cancerous
(malignant) or noncancerous (benign). People can get brain tumours at any
age. As we get older our risk of brain tumours increases. But there are many
different types of brain tumour and some are more common in younger
people. Primary brain tumours begin when normal cells acquire errors
(mutations) in their DNA. These mutations allow cells to grow and divide at
increased rates and to continue living when healthy cells would die. The
result is a mass of abnormal cells, which forms a tumour. In most people with
primary brain tumours, the cause of the tumour is not clear. But doctors have
identified some factors that may increase your risk of a brain tumour.
 Exposure to radiation- People who have been exposed to a type of
radiation called ionizing radiation have an increased risk of brain tumour.
Examples of ionizing radiation include radiation therapy used to treat
cancer and radiation exposure caused by atomic bombs.
 Family history of brain tumours- A small portion of brain tumours
occurs in people with a family history of brain tumours or a family history
of genetic syndromes that increase the risk of brain tumours.
 Fig1:Brain Tumor in Human Brain

In general, diagnosing a brain tumor usually begins with magnetic resonance

imaging (MRI). Once MRI shows that there is a tumor in the brain, the most
common way to determine the type of brain tumor is to look at the results
from a sample of tissue after a biopsy or surgery.
A primary brain or spinal cord tumor is a tumor that starts in the brain or
spinal cord. This year, an estimated 23,820 adults (13,410 men and 10,410
women) in the United States will be diagnosed with primary cancerous
tumors of the brain and spinal cord. Brain tumors account for 85% to 90% of
all primary CNS tumors. Also, about 3,720 children under the age of 15 is
diagnosed with a brain or CNS tumor this year. Brain and other nervous
system cancer is the 10th leading cause of death for men and women. It is
estimated that 17,760 adults (9,910 men and 7,850 women) will die from
primary cancerous brain and CNS tumors this year. It is important to
remember that statistics on the survival rates for people with a brain tumor
are an estimate. The estimate comes from annual data based on the number
of people with this tumor in the United States. Also, experts measure the
survival statistics every 5 years. So the estimate may not show the results of
better diagnosis or treatment available for less than 5 years.

1.2 Research Problem

The segmentation of brain tumor and the surrounding brain membrane is a

problem which has gained considerable importance in recent years.
Segmentation of brain membrane from medical images is an important
perspective to researchers and clinicians for the development of biomarkers,
surgery procedures, radiation therapy and radiosurgery can be applied for
the treatment. Segmentations of brain are required for computer-based
surgical planning of surgery.

Neural networks are known to be feature selectors, meaning that they will
learn to extract information that is relevant to the task [9]. This assumes that
the size of the data set and the complexity of the problem enable the network
to find correlating features. Based on a series of tests, it was not possible to
create a stable algorithm that would predict the OA of an individual based on
their knee MRI. A brain tumor detection is required to reduce the MR images
to only relevant information in the frame. Since methods like region growing
 lead to insufficient results due to similar intensities of other tissue, time-
consuming manual modifications are needed.
1.3 Focus of this Project
The goal of this project is to develop a fully automated non-contrast MRI
application for detection of brain tumor. The proposed approach is based on
Tiny YOLO V3 to achieve a robust and accurate segmentation of even highly
pathological brain structures. In this project we used Deep learning approach
based on Darknet-53 Architecture and for implementing and performing this
approach using Keras and Tenser-Flow(google colab) framework in python
Language. YOLO will be the base tool for this study because it’s the one of

the fastest and most accurate feature extractor architecture.

Figure 1.2. (a) Original image, (b) Detected Image

2 Background

This chapter provides an overview of prerequisites that led to the main task
of this project. It features a section for the necessary medical knowledge as
well as a hierarchical derivation to the appropriate branch of computer
science.

2.1 Medical Knowledge

This section will briefly explain how the imaging technology works that was
used to create the data set, as well as provide basic information about the
structure and growing process of growth plates in the human body.

2.1.1 Magnetic Resonance Imaging

Magnetic Resonance Imaging (MRI) uses magnetic fields and radio frequen
cies to probe tissue structure. In contrast to X-ray and CT scan which require
the exposure of ionizing radiation, MRI is a noninvasive imaging method. As
such it has become an essential diagnostic imaging modality in the medical
field [13].

96% of the human body is made up of hydrogen, oxygen, carbon, and nitrogen, all of
a spin. Due to the laws of electromagnetic induction, the motion of unbalanced charge produces
a magnetic field around itself. Hydrogen is the element used in MRI because its solitary
proton in the nucleus gives it a relatively large magnetic moment [13].
The positively charged hydrogen particles in water produce a signal when exposed to a
strong external magnetic field. This field is supplied by a magnet in the MRI machine
aligning the magnetic field of the hydrogen atoms to its own. Gradient coils are used to
cause a linear change in the strength of this field. By alternating the current of these
coils on the x, y and z axes, it is possible to calculate a three-dimensional image of the
tissue [13].

2.1.2 Brain Tumor:

A brain tumor is a mass or growth of abnormal cells in your brain.

Many different types of brain tumors exist. Some brain tumors are
noncancerous (benign), and some brain tumors are cancerous (malignant).
Brain tumors can begin in your brain (primary brain tumors), or cancer can
begin in other parts of your body and spread to your brain (secondary, or
metastatic, brain tumors).
How quickly a brain tumor grows can vary greatly. The growth rate as well as
location of a brain tumor determines how it will affect the function of your
nervous system.
Brain tumor treatment options depend on the type of brain tumor you have,
as well as its size and location. Treatment may include some combination
of surgery, radiationtherapy and chemotherapy.
Dexamethasone and furosemide are medications that may be used to
decrease swelling around the tumor. Some tumors grow gradually, requiring
only monitoring and possibly needing no further intervention. Treatments that
use a person's immune system are being studied. Outcome varies
considerably depending on the type of tumor and how far it has spread at
diagnosis. Although benign tumors only grow in one area, they may still be
life-threatening due to their location. Glioblastomas usually have very poor
outcomes, while meningioma’s usually have good outcomes. The
average five-year survival rate for all brain cancers in the United States is
33%.
Secondary, or metastatic, brain tumors are about four times more common
than primary brain tumors, with about half of metastases coming from lung
cancer. Primary brain tumors occur in around 250,000 people a year globally,
making up less than 2% of cancers. In children younger than 15, brain tumors
are second only to acute lymphoblastic leukaemia as the most common form
of cancer. In Australia, the average lifetime economic cost of a case of brain
cancer is $1.9 million, the greatest of any type of cancer.

2.2 Computer Science

The primary task of this project revolves around a particular branch of com-
puter science problems. This section provides a path to where most of the
work will take place. Figure 1 visualizes the hierarchical relationship of the
following subsections.
2
.
2
.
1

F
i
g
2
:
H
Fig2:hirerachical relationship of relevant topics

Artificial Intelligence

Artificial Intelligence (AI) is understood as the effort of automating a given

task that typically needs human intelligence to solve. The history of AI started
in the 1950s, where a particular type called "symbolic AI" gained popularity. It
was believed that human level intelligence could be achieved through hard-
coded rules that programmers specified [17].

Taking a complex problem like playing chess and continuing to break it into
smaller problems, until they can be solved with known logic. While it was
effective for certain tasks, fuzzy problems like image classification, speech
recognition or language translation were difficult to tackle. Over the years a
new approach was found, which today is referred to as machine learning
(ML).

2.2.2 Machine Learning

The concept of classical programming is that an engineer defines a set of

rules, called an algorithm, which uses input data to calculate some form of
output data [17].

Figure 3: Classical programming pipeline

 A machine learning algorithm is an algorithm that can learn from data [18]. It
can be used to calculate these rules automatically, so they do not have to be
specified by hand. Three components are needed for such an approach.

• Input data the algorithm is supposed to transform

• Output data the algorithm is supposed to predict

• A measurement to validate the performance of a prediction

It works by feeding input and output data into a pipeline, which will learn to transform one into the
other. With the advantage that no explicit pro-gramming is needed to generate the rules, comes
the disadvantage that prior input and output data is required for the initial learning process

Figure 4: Machine learning pipeline

Machine learning may be applied as an effective method if it is not feasible or

possible to define an algorithm by hand and sufficient data is available for
training. How much “sufficient” is depends on factors like the type of task, the
complexity of the data, the uniformity of the data, the type of machine
learning algorithm and others.

There are different subparts to machine learning like supervised and

unsuper-vised learning. Supervised learning is used when it is clear what the
output data looks like, whereas unsupervised learning can help to find
unknown patterns in the data. Examples of supervised learning techniques
include lin-ear regression, naive Bayes, support vector machines, decision
trees, random forests, gradient boosting and artificial neural networks
(ANNs). Since the primary interest of this study revolves around ANNs, this
will be the focus of following chapters.
2.2.3 Artificial Neural Networks

Artificial neural networks are inspired by neurobiological concepts of the

human brain. However, they are not models of the human brain. There is no
evidence to support that the brain implements anything like the learning
mechanisms used in ANNs [17]. Artificial neural networks are mathematical
frameworks for learning representations from data and one of, if not the only
tool in deep learning (DL) — a subset of machine learning.

The origin of the term deep learning is twofold. On the one hand, it refers to
the fact that ANNs can learn "deep" hierarchical information from data. On
the other, it describes that they show multiple layers of depth within their
architecture. ANNs are as old as machine learning itself, but only gained a lot
of their popularity in recent years [17].

Figure 2.2.2: An ANN with 3 input nodes, 2 hidden layers with 4 nodes each
and 2 output nodes

These dense layers inside artificial neural networks contain entities called
nodes that are jointly linked with one another. Every node in a dense layer is
connected to each node in the previous and following layer. This structure
gives ANNs the capacity to approximate complex mathematical functions and
learn global representations in the data. These nodes, also called
parameters, can range to hundreds of millions depending on the task [19].
2.2.4 You Only Look Once-

Current detection systems repurpose classifiers to perform detection. To

detect an object, these systems take a classifier for that object and evaluate it
at various locations and scales in a test image. Systems like deformable parts
models (DPM) use a sliding window approach where the classifier is run at
evenly spaced locations over the entire image. More recent approaches like
R-CNN use region proposal methods to first generate potential bounding
boxes in an image and then run a classifier on these proposed boxes. After
classification, post-processing is used to refine the bounding boxes, eliminate
duplicate detections, and rescore the boxes based on other objects in the
scene [13]. These complex pipelines are slow and hard to optimize because
each individual component must be trained separately.
YOLO is refreshingly simple. A single convolutional network simultaneously
predicts multiple bounding boxes and class probabilities for those boxes.
YOLO trains on full images and directly optimizes detection performance.
This unified model has several benefits over traditional methods of object
detection. First, YOLO is extremely fast. Since we frame detection as a
regression problem we don’t need a complex pipeline. We simply run our
neural network on a new image at test time to predict detections. Our base
network runs at 45 frames per second with no batch processing on a Titan X
GPU and a fast version runs at more than 150 fps. This means we can
process streaming video in real-time with less than 25 milliseconds of latency.
Furthermore, YOLO achieves more than twice the mean average precision of
other real-time systems.
Second, YOLO reasons globally about the image when 1 arXiv:1506.02640v5
[cs.CV] 9 May 2016 making predictions. Unlike sliding window and region
proposal-based techniques, YOLO sees the entire image during training and
test time so it implicitly encodes contextual information about classes as well
as their appearance. Fast R-CNN, a top detection method, mistakes
background patches in an image for objects because it can’t see the larger
context. YOLO makes less than half the number of background errors
compared to Fast R-CNN. Third, YOLO learns generalizable representations
of objects. When trained on natural images and tested on artwork, YOLO
outperforms top detection methods like DPM and R-CNN by a wide margin.
Since YOLO is highly generalizable it is less likely to break down when
applied to new domains or unexpected inputs. YOLO still lags behind state-
of-the-art detection systems in accuracy. While it can quickly identify objects
in images it struggles to precisely localize some objects, especially small
ones. We examine these tradeoffs further in our experiments.
3 Methodology

The purpose of this chapter is twofold. It will discuss the design and process-
ing decisions that were made for the development of this project, while also
giving critical insight into how these applied technologies work. The sections
for setup and data set describe the working environment for this study. Pre-
processing, architecture, training and post processing show in chronological
order how the development was addressed.
3.1 Setup

The workstation included an Intel i5 processor, 16GB of RAM and most

importantly a NVIDIA GeForce GTX1060/6GB. Neural Networks can be
trained more efficiently on GPUs than CPUs. This is because the simpler but
highly parallelized architecture of graphics chips plays in favor for the needed
calculations in deep learning [22]. The computer ran Ubuntu 16.04 with the
NVIDIA CUDA and cuDNN li-braries installed to take advantage of the GPU.
As the primary programming language Python 3 was chosen due to its simple
syntax and popularity in the deep learning field. Keras was used as the
framework for training models because its high level syntax allows fast
prototyping and testing.The development environment was a google colab,
which allowed a flexible execution of code snippets instead of running the
entire program for every single change.

The processing of medical image data needed a library that could handle
these formats. PyDicom, SikitLearn is a Python libraries written in C++. It
includes many tools for image processing and is especially popular in the
medical field. Other libraries were also used for smaller tasks.

3.2 Data Set

The data set is a collection MRIs of Human Brain. The number of available
samples grew during the project. For most of the development time, it
included 253 samples that came multiple MRIs sources. All images were
provided in the JPEG file format. It features lossless compression, as well as
technical Meta information about each image. Data featured a sagittal
perspective.

Source Prospective View Samples Maps Resolution

MRIs No various 253 0 various

Table 2: Details of the data set separated by source

 Figure3.2: Six different slices of one 3D MRI
The images above some slices of 3D MRI of various different peoples. Some
MR Images have Tumor in them while other don’t. Also the sizes differs
indicating different stages of the Brain Tumor.

3.3 Preprocessing

The parameters in a Neural Network commonly range from tens of thousands

to hundreds of millions. This complexity allows the model to learn on its own
what features of an image are relevant for any given task. It works in
conjunction with the fact that high volumes of data are available for the
training. Because of the small data set that was available for this study,
several types of preprocessing were applied to the images. For the most part,
these techniques remove irrelevant information and reduce variance between
multiple samples. Other preparation methods experimented.

3.3.1 Training and Testing Sets

The data was split into two subsets of which each was used for a different
purpose (training and testing). The training set is commonly the largest of the
two and contains the data that is applied to the actual learning process. It’s
the only portion of the data the network will draw direct conclusions from.

The second subset is referred to as the testing data, it’s only used once in
the very end, to give a final score. The idea is that by building a model based
on the validation results a certain amount of information bleed occurs, where
the network will implicitly learn from the validation data. In order to prevent
biased results, the testing data is used as a last performance reference.
• Training Set: 90% of the data

• Test Set: 10% of the data

3.3.2 Annotation

For creating a ground truth bounding boxes we perform annotation over our
dataset

Yolo V3 predict B regressions for B bounding boxes. Only one of the B

regressors is trained at each positive position, the one that predicts a box that
is closest to the ground truth box, so that there is a reinforcement of this
predictor, and a specialization of each regressor. In Yolo V3, this
specialization is ‘assisted’ with predefined anchors. The predefined anchors
are chosen to be as representative as possible of the ground truth boxes, with
the following K-means clustering algorithm to define them:
 all ground-truth bounding boxes are centred on (0,0)
 the algorithm initiates 5 centroids by drawing randomly 5 of the
ground-truth bounding boxes
 then, the following two steps are alternated:
 each ground truth box is assigned to one of the centroid, using as
distance measure the IOU, in order to get 5 clusters or groups of
ground-truth bounding boxes
 new centroids are computed by taking the box inside each cluster that
minimizes the mean IOU with all other boxes inside the cluster

Figure 3.3.2: Examples of slices with their ground truth bounding box
 3.4 Architecture-

The YOLO design enables end-to-end training and real-time speeds while
maintaining high average precision. Our system divides the input image into
an S × S grid. If the center of an object falls into a grid cell, that grid cell is
responsible for detecting that object. Each grid cell predicts B bounding
boxes and confidence scores for those boxes. These confidence scores
reflect how confident the model is that the box contains an object and also
how accurate it thinks the box is that it predicts. Formally we define
confidence as Pr(Object) ∗ IOU truth predict . If no object exists in that cell,
the confidence scores should be zero. Otherwise we want the confidence
score to equal the intersection over union (IOU) between the predicted box
and the ground truth. Each bounding box consists of 5 predictions: x, y, w, h,
and confidence. The (x, y) coordinates represent the center of the box
relative to the bounds of the grid cell. The width and height are predicted
relative to the whole image. Finally the confidence prediction represents the
IOU between the predicted box and any ground truth box. Each grid cell also
predicts C conditional class probabilities, Pr(Classi |Object). These
probabilities are conditioned on the grid cell containing an object. We only
predict one set of class probabilities per grid cell, regardless of the number of
boxes B. At test time we multiply the conditional class probabilities and the
individual box confidence predictions, Pr(Classi|Object) ∗ Pr(Object) ∗
IOUtruth pred = Pr(Classi) ∗ IOUtruth pred (1) which gives us class-specific
confidence scores for each box. These scores encode both the probability of
that class appearing in the box and how well the predicted box fits the object.
For evaluating YOLO on PASCAL VOC, we use S = 7, B = 2. PASCAL VOC
has 20 labelled classes so C = 20. Our final prediction is a 7 × 7 × 30 tensor
the number of boxes B. At test time we multiply the conditional class probabilities and
the individual box confidence predictions, Pr(Classi|Object) ∗ Pr(Object) ∗ IOUtruth pred
= Pr(Classi) ∗ IOUtruth pred (1) which gives us class-specific confidence scores for
each box. These scores encode both the probability of that class appearing in the box
and how well the
predicted box.
 f
i
t
s

t
h
e

F
i
Figure 3.4.1: The Architecture. Our detection network has 24 convolutional
layers followed by 2 fully connected layers. Alternating 1 × 1 convolutional
layers reduce the features space from preceding layers. We pre-train the
convolutional layers on the ImageNet classification task at half the resolution
(224 × 224 input image) and then double the resolution for detection.

fits the object. For evaluating YOLO on PASCAL VOC, we use S = 7, B = 2.

PASCAL VOC has 20 labelled classes so C = 20. Our final prediction is a 7 ×
7 × 30 tensor.

Fast YOLO fewer convolutional layers (9 instead of 24) and fewer filters
in those layers. The network pipeline is summarized like below.

Therefore, we can see that, the input image goes through the network once
and then objects can be detected. And we can have end-to-end learning.

3.4.1 Channels, learning Rate, and Depth

The number of parameters in a neural network has a high correlation with its
learning capacity. By adding more nodes that can be adjusted during training,
the model can approximate a more complex function that transforms the
input into the output. The downside is that a larger parameter count will also
increase the possibility of overfitting the data. A convention in the field of
CNNs is to gradually increase the number of channels, while the spatial
resolution is reduced due to the use of MaxPooling [21][25][26][27]. U-Net
also shows this behavior on the left side of its architecture.

A test was set up that compared the original U-Net against five smaller
variants. They varied in the total number of parameters and how the number
of channels changed from one layer to the next.

Training U-Net was very slow in two regards. Each training step lasted six
times longer compared to the smallest model, while it also took 25 times
more training steps to reach the same results. Since it was also overfitting to
a significant amount, it was not trained until the end. The second largest
model E also overfitted on the training data and never achieved comparable
results to the other four models.

Model D started with eight channels which were then increased by a factor of 2,
whereas B started out larger but only increased the channels by a rate of 1.5. Although
being the smaller model, B showed a higher accuracy in the end. It was concluded that
the initial number of output channels has a larger effect on the result than the growth
rate.

This theory was supported by the results from A and C, both of which kept
their initial number of channels throughout the network. These two showed
the highest scores in comparison to the other models while maintaining their
relatively small size. Since their results were identical, model A was kept
because of its faster training speed. It was interesting to see that the smallest
model performed best across the candidates.

The original U-Net featured a depth of 5 convolutional blocks until reaching

the bottom of the "U"-shape. It was also investigated that a depth of 6 would
not improve performance, while a depth of 4 decreased the accuracy.
Therefore, Model A was left unchanged.

3.4.4 Activation Functions

Activation functions sit between layers in a network to introduce a non-

linearity. Otherwise, the operations could be to a simple linear transformation
and remove the benefits of building a deep model. The Leaky ReLUs are one
attempt to fix the “dying ReLU” problem. Instead of the function being zero
when x<0, a leaky ReLU will instead have a small negative slope (of 0.01 or
 so). That is, the function computes f(x)=1(x<0)(αx)+1(x>=0)(x)
f(x)=1(x<0)(αx)+1(x>=0)(x) ) where α is a small constant. Some people
report success with this form of activation function, but the results are not
always consistent. The slope in the negative region can also be made into
a parameter of each neuron, as seen in PReLU neurons, introduced
in Delving Deep into Rectifiers, by Kaiming He et al., 2015. However, the
consistency of the benefit across tasks is presently unclear.

3.5 Training

The training is where the actual learning takes place, and its process is
inspired by how humans improve their performance on tasks. When opposed
to a difficult question the first step would be a mere guess of what the answer
might be. In a second phase, this information is compared to the right answer
and processed by the brain accordingly.

Similarly, the training of a neural network consists of two parts. In the first
step, the input is fed forward through the network, and an answer prediction
is made. In the very beginning, this is just a random guess because ANNs
are initialized with random values.

Figure 14: The iterative process of forward and backward propagation

The prediction is then compared to the right answer, and the error is cal-
culated. This metric can be fed back through the network to optimize the
parameters in the model. These two steps are called forward and backward
propagation.

3.5.1 Metrics and Loss Functions

Metrics are used in deep learning to measure the performance of a model.

For example, the accuracy is often chosen to describe how well a neural
network is doing on a classification task. Intersection over Union is an
evaluation metric used to measure the accuracy of an object detector on a
particular dataset. Any algorithm that provides predicted bounding boxes as
output can be evaluated using IoU. More formally, in order to apply
Intersection over Union to evaluate an (arbitrary) object detector we need:
 1. The ground-truth bounding boxes (i.e., the hand labeled bounding boxes
from the testing set that specify where in the image our object is).
2. The predicted bounding boxes from our model.

Figure 14: Computing the intersection over union

Examining this equation you can see that Intersection over Union is simply a
ratio. In the numerator we compute the area of overlap between the
predicted
Bounding box and the ground-truth bounding box. The denominator is
the area of union, or more simply, the area encompassed by both the
predicted bounding box and the ground-truth bounding box.
Dividing the area of overlap by the area of union yields our final score — the
Intersection over Union.

Figure 14: An example of computing Intersection over Union

for various bounding box

The loss function used in our model is Binary cross-entropy loss function. It’s a multi label
classification. Also called Sigmoid Cross-Entropy loss. It is a Sigmoid activation plus
a Cross-Entropy loss. It is independent for each vector component (class), meaning that the
loss computed for every CNN output vector component is not affected by other component
values. That’s why it is used for multi-label classification, were the insight of an element
belonging to a certain class should not influence the decision for another class. It’s called Binary
Cross-Entropy Loss because it sets up a binary classification problem
between C′=2C′=2 classes for every class in CC, as explained above. So when using this Loss,
the formulation of Cross Entroypy Loss for binary problems is often used:
3.5.3 Batch Size

The number of random samples per training step is referred to as the batch
size. In the past, it was believed that larger batches led to something called
the generalization gap [39], where the accuracy of a model would drop if it
was trained on unusually large batches. Recent work [40] suggests other
reasons for this decrease in accuracy. While common batch sizes range from
32 to 256, Goyal et al. showed accurate results using 8192 images per batch
when training a model on ImageNet [41].

Using batches smaller than 32 samples can introduce a different kind of

problem. Having too few data points that don’t represent the mean of the
data well, may lead to slow and unstable training. It is impractical (and
unnecessary) to use all images in the training set at once to update the
weights. So, a small subset of images is used in one iteration, and this
subset is called the batch size. We kept our batch size as 1. It means 64
images are used in one iteration to update the parameters of the neural
network. Even if there we don’t have a GPU with enough memory to use a
batch size of 64. Fortunately, Darknet allows you to specify a variable
called subdivisions that lets you process a fraction of the batch size at one
time on your GPU. The subdivisions parameter should be multiples of 2(e.g.
2, 4, 8, 16).

3.5.4 Learning Rate Policy

One full iteration over the training samples is referred to as an epoch, and the
learning rate policy describes how the learning rate is changed from one
epoch to another. With the introduction of adaptive optimizers like Adam,
there has been a lower emphasize on this topic because the learning rate is
modified during training. Even though this reduces the number of possible
defects, training time can often be saved with the right initial learning rate.
Ten epochs were run at different learning rates to compare initial results and
to examine the point at which the model wouldn’t converge at all. 0.002 was
the highest rate at which the model started training, but 0.001 resulted in the
best score. Adding a decay that reduces the learning rate manually over time
did not improve the results with the use of Adam.

3.5.5 Early Stopping

Neural networks will continuously minimize the loss on the training set. This
result needs to be validated on data the network hasn’t seen before. At a
certain point during training, the performance on the validation set will start to
decrease because the model is overfitting on the training data. The number
of iterations to reach this point is dependent on many IoU, as well as the
random values the network has been initialized with. As such, it’s difficult to
calculate how long the training will need to reach its peak.
Early stopping is a simple technique that will end the training process as
soon as the model stops improving on the validation data. To do this, a
patience value is defined how long the network should continue training after
the score has stopped increasing. This is important because not every epoch
will lead to a new best score on the validation data.

4 Results

The results in this chapter are based on data the network has not been
trained with. It was also made sure that there was no overlap in images of
people that were recorded multiple times. The network architecture was
developed using yolo V3 and darknet-53 architecture.

Epoch
Data Epoch 1 Epoch 50 Epoch 100 150 Epoch 250
Training Set 0.001 0.493 0.766 0.923 0.981
Test Set - - - - 0.980

The 98% mark is reached on the validation data, but only on the final
evaluation of the test set. Towards the end, the performance of the training
data pulls slightly ahead of the validation results.

4.1 Visual Evaluation

The visual evaluation will focus on the results of the combined network since
its performance is on par with the one channel model while offering more
information about the associated bones.
Network is tested tested after some iteration we stopped our first iteration at
1000 iteration and then iterated till 3000 iteration.
Figure 4.1.0: Input, prediction, this was the result after1000 iteration

Figure 4.1.1: Input, prediction, prediction after 3000th iteration of the network.

In 1000th iteration we are getting a pretty good result, we iterated till 3000 iteration but
no major difference was found visually but the confidence level increased by a
significant amount.
5 Discussion

Yolo V3 networks proved to be a very effective measure for the object

detection of brain-tumour in MRIs. They show better performance than the
ray casting technique and multi-atlas registration which were used in previous
studies. However, the underlying data sets were only similar and not
identical, which could have also led to differences in the results.

The greatest limitation of this study was time. Three weeks were spent on the
programming aspect and trying out different sets of parameters. Each new
idea came with several hours of training to verify the performance. As such,
only popular techniques in the deep learning field were applied. The training
of the model takes at least 8 hours for a good prediction so it was a limitation
as sometimes the weights we trained predicted wrong results. Another three
weeks were invested in writing this project to complete everything by the
deadline. Spending more time on data analysis, fine tuning parameters and
trying entirely new architectures might open up new possibilities.

However, more time could be spent on restructuring the entire architecture to

benefit the needs of these types of convolutions.

6 Conclusion

This project presented the development of tumour detection based on

convolutional neural networks. It shows an excellent performance of 98%
accuracy for detection of brain tumour. It is robust against noise and
adaptable to changes in resolution due to its fully convolutional structure.

References

[1] Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Delving
Deep into Rectifiers: Surpassing Human-Level Performance on
ImageNet Classification. feb 2015.

[2] Yonghui Wu, Mike Schuster, Zhifeng Chen, Quoc V. Le, Mohammad
Norouzi, Wolfgang Macherey, Maxim Krikun, Yuan Cao, Qin Gao, Klaus
Macherey, Jeff Klingner, Apurva Shah, Melvin Johnson, Xiaobing Liu,
Łukasz Kaiser, Stephan Gouws, Yoshikiyo Kato, Taku Kudo, Hideto
Kazawa, Keith Stevens, George Kurian, Nishant Patil, Wei Wang, Cliff
Young, Jason Smith, Jason Riesa, Alex Rudnick, Oriol Vinyals, Greg
Corrado, Macduff Hughes, and Jeffrey Dean. Google’s Neural Machine
Translation System: Bridging the Gap between Human and Machine
Translation. sep 2016.

[3] Mariusz Bojarski, Philip Yeres, Anna Choromanska, Krzysztof Choro-

manski, Bernhard Firner, Lawrence Jackel, and Urs Muller. Explaining
How a Deep Neural Network Trained with End-to-End Learning Steers a
Car. apr 2017.
 [4] Joshua Saxe and Konstantin Berlin. Deep Neural Network Based Mal-
ware Detection Using Two Dimensional Binary Program Features. aug
2015.

[5] European Asylum Support Office. Age assessment practice in Europe.

2013.

[6] World Health Organization. Ionizing radiation, health effects and pro-
tective measures, 2016.

[7] Dennis Säring, Markus Mauer, and Eilin Jopp. Klassifikation des Ver-
schlussgrades der Epiphyse der proximalen Tibia zur Altersbestimmung.
pages 60–65. Springer, Berlin, Heidelberg, 2014.

[8] Eilin. Jopp. Methoden zur Alters- und Geschlechtsbestimmung auf dem
Pruefstand - eine rechtsmedizinische empirische Studie. Kovac, 2007.

[9] R Setiono and H Liu. Neural-network feature selector. IEEE Transac-

tions on Neural Networks, 8(3):654–662, may 1997.

[10] Pierre Dodin, Johanne Martel-Pelletier, Jean-Pierre Pelletier, and Fran-

cois Abram. A fully automated human knee 3D MRI bone segmentation
using the ray casting technique. pages 1413–1424, 2011.

[11] Erik B Dam, Martin Lillholm, Joselene Marques, and Mads Nielsen.
Automatic segmentation of high-and low-field knee MRIs using knee
image quantification with data from the osteoarthritis initiative.

[12] Darko Stern, Thomas Ebner, Horst Bischof, Sabine

Grassegger, Thomas Ehammer, and Martin Urschler. Fully automatic
bone age estimation from left hand MR images. Medical image
computing and computer-assisted intervention : MICCAI ... International
Conference on Medical Image Computing and Computer-Assisted
Intervention, 17(Pt 2):220– 227, 2014.

[13] Catherine Westbrook. MRI at A Glance. Wiley Blackwell, 3 edition, 2016.

[14] Gerhard Aumüller, Gabriela Aust, Andreas Doll, Jürgen Engele, Joachim
Kirsch, Siegfried Mense, and Laurenz Wurzinger. Anatomie. Number 2.
2010.

[15] David E. Attarian. Your Bones: What are growth plates?, 2013.

[16] Markus Auf der Mauer. Automated Quantification of the Growth Plate of
the Proximal Tibia for the Age Assessment in 3D MR Images Using a
Fuzzy-Logic Classification Approach. PhD thesis, 2015.

[17] Francois Chollet. Deep Learning With Python, volume 1. 2017.

[18] Ian Goodfellow, Yoshua Bengio, and Aaron Courville. Deep Learning.
The MIT Press, 2016.

[19] Karen Simonyan and Andrew Zisserman. Very Deep Convolutional

Networks for Large-Scale Image Recognition. sep 2014.

[20] Y. Le Cun, B. Boser, J. S. Denker, D. Henderson, R. E. Howard, and W.

Hubbard. Handwritten Digit Recognition with a Back-Propagation
Network. Advances in Neural Information Processing Systems, pages
396–404, 1990.

[21] Alex Krizhevsky, Ilya Sutskever, and Geoffrey E Hinton. ImageNet

Classification with Deep Convolutional Neural Networks.

[22] NVIDIA. GPU vs CPU? What is GPU Computing?

[23] Nicholas J. Tustison, Brian B. Avants, Philip A. Cook, Yuanjie Zheng,

Alexander Egan, Paul A. Yushkevich, and James C. Gee. N4ITK: Im-
proved N3 bias correction. IEEE Transactions on Medical Imaging,
29(6):1310–1320, 2010.

[24] Özgün Çiçek, Ahmed Abdulkadir, Soeren S. Lienkamp, Thomas Brox,

and Olaf Ronneberger. 3D U-net: Learning dense volumetric segmen-
tation from sparse annotation. In Lecture Notes in Computer Science
(including subseries Lecture Notes in Artificial Intelligence and Lecture
Notes in Bioinformatics), volume 9901 LNCS, pages 424–432, jun 2016.

[25] Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun.
Deep Resid-ual Learning for Image Recognition. dec 2015.

[26] Forrest N. Iandola, Song Han, Matthew W. Moskewicz, Khalid Ashraf,

William J. Dally, and Kurt Keutzer. SqueezeNet: AlexNet-level accuracy
with 50x fewer parameters and. feb 2016.

[27] Olaf Ronneberger, Philipp Fischer, and Thomas Brox. U-Net: Convo-
lutional Networks for Biomedical Image Segmentation. Miccai, pages
234–241, may 2015.

[28] Liang-Chieh Chen, George Papandreou, Florian Schroff, and Hartwig

Adam. Rethinking Atrous Convolution for Semantic Image Segmenta-
tion. jun 2017.

[29] Hengshuang Zhao, Jianping Shi, Xiaojuan Qi, Xiaogang Wang, and
Jiaya Jia. Pyramid Scene Parsing Network. dec 2016.

[30] Guosheng Lin, Anton Milan, Chunhua Shen, and Ian Reid. RefineNet:
Multi-Path Refinement Networks for High-Resolution Semantic Segmen-
tation. nov 2016.
[31] Liang-Chieh Chen, George Papandreou, Iasonas Kokkinos, Kevin Mur-
phy, and Alan L. Yuille. DeepLab: Semantic Image Segmentation with
Deep Convolutional Nets, Atrous Convolution, and Fully Connected
CRFs. jun 2016.

[32] Vladimir Nekrasov, Janghoon Ju, and Jaesik Choi. Global Deconvolu-
tional Networks for Semantic Segmentation. feb 2016.

[33] Fausto Milletari, Nassir Navab, and Seyed Ahmad Ahmadi. V-Net: Fully
convolutional neural networks for volumetric medical image segmenta-
tion. In Proceedings - 2016 4th International Conference on 3D Vision,
3DV 2016, pages 565–571, jun 2016.

[34] Vijay Badrinarayanan, Alex Kendall, and Roberto Cipolla. SegNet: A

Deep Convolutional Encoder-Decoder Architecture for Image Segmen-
tation. nov 2015.

[35] Nitish Srivastava, Geoffrey Hinton, Alex Krizhevsky, Ilya Sutskever, and
Ruslan Salakhutdinov. Dropout: A Simple Way to Prevent Neural
Networks from Overfitting. Journal of Machine Learning Research,
15:1929–1958, 2014.

[36] Vinod Nair and Geoffrey E Hinton. Rectified Linear Units Improve
Restricted Boltzmann Machines.

[37] Djork-Arné Clevert, Thomas Unterthiner, and Sepp Hochreiter. Fast and
Accurate Deep Network Learning by Exponential Linear Units (ELUs).
nov 2015.

[38] Diederik P. Kingma and Jimmy Ba. Adam: A Method for

Stochastic Optimization. dec 2014.

[39] Nitish Shirish Keskar, Dheevatsa Mudigere, Jorge Nocedal, Mikhail

Smelyanskiy, and Ping Tak Peter Tang. On Large-Batch Training for
Deep Learning: Generalization Gap and Sharp Minima. sep 2016.

[40] Elad Hoffer, Itay Hubara, and Daniel Soudry. Train longer, generalize
better: closing the generalization gap in large batch training of neural
networks. may 2017.

[41] Priya Goyal, Piotr Dollár, Ross Girshick, Pieter Noordhuis, Lukasz
Wesolowski, Aapo Kyrola, Andrew Tulloch, Yangqing Jia, and Kaiming
He. Accurate, Large Minibatch SGD: Training ImageNet in 1 Hour. jun
2017.

[42] Adhish Prasoon, Kersten Petersen, Christian Igel, François Francois

Lauze, Erik Dam, and Mads Nielsen. Deep Feature Learning for Knee
Cartilage Segmentation Using a Triplanar Convolutional Neural Net-
work. 2013.
VII