Melanoma skin cancer detection using deep learning and

classical machine learning techniques: A hybrid

approach
Jinen Daghrir, Lotfi Tlig, Moez Bouchouicha, Mounir Sayadi

To cite this version:

Jinen Daghrir, Lotfi Tlig, Moez Bouchouicha, Mounir Sayadi. Melanoma skin cancer detection using
deep learning and classical machine learning techniques: A hybrid approach. International Conference
on Advanced Technologies for Signal and Image Processing, Sep 2020, Sfax, Tunisia. �10.1109/AT-
SIP49331.2020.9231544�. �hal-03172718�

HAL Id: hal-03172718

https://hal.science/hal-03172718v1
Submitted on 17 Mar 2021

HAL is a multi-disciplinary open access L’archive ouverte pluridisciplinaire HAL, est

archive for the deposit and dissemination of sci- destinée au dépôt et à la diffusion de documents
entific research documents, whether they are pub- scientifiques de niveau recherche, publiés ou non,
lished or not. The documents may come from émanant des établissements d’enseignement et de
teaching and research institutions in France or recherche français ou étrangers, des laboratoires
abroad, or from public or private research centers. publics ou privés.
 Melanoma skin cancer detection using deep
learning and classical machine learning techniques:
A hybrid approach
Jinen Daghrir 1,2 , Lotfi Tlig 2 , Moez Bouchouicha 3 , Mounir Sayadi 2
1
Université de Sousse, ISITCom, 4011, Hammam Sousse, Tunisia
2
Université de Tunis, ENSIT, Laboratory SIME, Tunisia
3
Aix Marseille Univ, Université de Toulon, CNRS, LIS, Toulon, France

Abstract—Melanoma is considered as one of the fatal cancer stage. Self examination is so vital even if you have carefully
in the world, this form of skin cancer may spread to other parts protected your skin from ultraviolet radiations. Thus, people
of the body in case that it has not been diagnosed in an early should examine their skins head to toe regularly, looking for
stage. Thus, the medical field has known a great evolution with
the use of automated diagnosis systems that can help doctors and any lesions that might be turned into melanoma . Self-exams
even normal people to determine a certain kind of disease. In can help you identify skin cancers in an early stage, when
this matter, we introduce a hybrid method for melanoma skin the odds of curing them are completely high. Yet, physicians
cancer detection that can be used to examine any suspicious encourage people to routinely do self-examination if they
lesion. Our proposed system rely on the prediction of three notice any suspicious-looking lesions. ”when in doubt, cut it
different methods: A convolutional neural network and two
classical machine learning classifiers trained with a set of features out”, says Darrick Antell, MD, a board-certified plastic and
describing the borders, texture and the color of a skin lesion. reconstructive surgeon practicing in New York.
These methods are then combined to improve their performances In this matter, it becomes vital to use supportive imaging
using majority voting. The experiments have shown that using techniques that have been shown to improve and facilitate
the three methods together, gives the highest accuracy level. the diagnosis process. These techniques are build based on
Index Terms—Melanoma detection, deep learning, classical
machine learning, data fusion, majority voting strategies invented by physicians to capture the melanoma at
an early stage. Automatic detection of melanoma is comprised
of a variety of steps including preprocessing, extraction of
I. I NTRODUCTION
interested area, post-processing, and lesion inspection. These
Melanoma the deadliest form of skin cancer, is considered steps look like the steps of the classical pattern recognition
as the less common form of skin cancers but it is the most fatal. system, where the main design steps of such system are: image
As mentioned above, it can quickly spread to other parts of the acquisition, image processing and lesion segmentation, and
body. Melanoma arises through malignant transformation of classification methodology of the lesion in question [14]. In
melanocytes which are derived from the neural crest neoplasia this paper, we present a new method for melanoma detection
[7]. Melanoma causes 55 500 cancer deaths annually which is for a better self-examination.
0.7% of all cancer deaths. The incidence and mortality rates In this paper, the first section gives a brief description of some
of melanoma differ from one country to another due to the existing work and the visual cues that are used by physicians
variation of ethnic and racial groups [15]. Established risk for melanoma detection. The second section is reserved to
factors for melanoma include ultraviolet radiation, life in low introduce the proposed method. All the conducted experiments
geographic latitudes, high alcohol consumption, consuming and the results evaluating the performance of the proposed
fatty foods, the presence of melanocytic or dysplastic naevi, method are given after that. In the last section, a discussion
a family or personal history of melanoma, phenotypic charac- and conclusion are given.
teristics including fair hair, eye, and skin colors [13].
The incidence and mortality of melanoma are related to the II. R ELATED WORK
development index (HDI), The increase in the HDI index One typical and commonly used clue for melanoma di-
increases access to health services and early detection of agnosis is the ABCDE signs which is a useful indicator
disease and treatment of the disease at an early stage, thereby for melanoma. This feature can differentiate a malignant
reducing mortality [13]. Yet , statistics shows that the 5-year melanoma from a benign skin lesion based on five char-
relative survival rate for people who has been diagnosed with acteristics, namely asymmetry (A), border irregularity (B),
melanoma in an early stage is about 98%. However, about color variability (C), diameter greater than 6 mm (D), and
20% to 50% of people having melanoma in advanced stage evolution (E) or any kind of change. Melanoma detection
will be alive 5 years after diagnosis [15]. typically involves several processes: image acquisition and
Since, it is important that the melanoma is caught at an early preprocessing, lesion segmentation, lesion characterisation and
finally lesion classification. The exact order in which these
processes are applied varies from one method to another,
however, some of them can skip or add other processes, or
use some of them in a hybrid way.
The first step in expert systems used for melanoma detection
involves the acquisition of the tissue digital image. The most
common techniques used for this purpose are the epiluminence
microscopy(ELM), it is also known as dermoscopy [9]. Yet,
smartphone cameras are becoming more and more capable and Fig. 1. RGB color channels of a skin lesion.
effective to capture skin lesions, the resolution of the image
has improved over years which can encourage experts to use
smartphone cameras to clinically imaged lesions rather than of the segmentation step, deep learning automatically detects
using traditional acquisition methods [11]. In addition this help the skin lesion and learns a feature representation from a large
people to easily do self-examination in order to inspect any number of skin images.
suspicious lesion. In paper [10], an automatic detection of melanoma lesion on
Most of the existing systems use the dermoscopic images skin images has been proposed based on the concept of deep
which are proved to be very efficient in melonoma inspection. learning. The results shows that deep learning using CNN
These systems can use image processing and classical machine is able to detect the melanoma lesion efficiently when using
learning techniques [11], [3], [6] or more sophisticated and CNN with 15 × 15 training input size. This neural network
powerful technique which is the deep learning [10], [18]. architecture can accurately detect melanoma or any kind of
After the image is taken, a lesion segmentation is conducted skin lesions though.
in order to isolate the pathological skin lesions from the
III. P ROPOSED METHOD
surrounding healthy skin. This step is considered to be critical
step in the whole process of melanoma detection. One inter- Generally people tend to see doctors when the melanoma
esting and recent method have been proposed by Uzma Jamil is in its last stage and when it becomes difficult to be treated.
et al. [11], which consists on separating the foreground (the People normally consider skin lesion as normal diseases such
lesion) from the background by filtering the image with the any environment infection. Thus it becomes a necessity to
gradient magnitude. do self-examination of any suspicious lesion. Nowadays the
Then, a set of characteristics are extracted. Generally, these medical fields depend not only on the doctor diagnosis but
primitives will represent only the ABCD rule which describes also upon computer-aided diagnosis. Digital image process-
the color, the border and the texture looking for some differ- ing makes dermatologist and patients life easier. It helps in
ential structures [14]. Studying the texture of a skin lesion can diagnosing the lesion area without any physical contact with
bring out many characteristics which can be extracted using skin, in this section we present our proposed methods that help
different kind of texture descriptors, such as SIFT feature, people and doctors in accurately diagnosing melanoma.
SURF and HOG [12]. However in [17], the authors have One typical problem in melanoma detection is to separate the
used color in addition to the textural features. The CN (Color skin lesion from the healthy skin. The segmentation step is
Name) features which are linguistic color labels representing incredibly sensitive to the output of the preprocessing phase.
11 different colors, are used to detect the presence of the Thus improving the image analysis and preprocessing by
black and blue colors in a lesion. The blue-black color is removing the artifacts will certainly improve the segmenta-
another typical feature that is used by physicians to recognize tion process. Artifacts may occlude some information of the
melanoma. It is defined as the presence of a combination of inspected lesion. Artifacts can be hair pixels which should be
blue and black pigmented areas involving at least 10% of the removed for a better segmentation and lesion analysis. In fact,
lesion surface [2]. in the literature many hair removal systems are introduced to
Once the features are extracted, a training phase is needed deal with that issue.
to create a model that differentiate between malignant and The preprocessing method for the hair detection proposed in
benign lesions. Researchers have widely used the support [1] is adopted. A minor changes was introduced. The proposed
vector machines (SVMs), due to their effectiveness and low hair removal system involves several steps which are described
complexity in generating an accurate model [3], [8]. However, below. First, a color enhancement technique based on the blue
in [6] authors proposed a comparison between a K-Nearest component of the RGB color channels is selected for the rest of
Neighbor (KNN) classifier and Artificial Neural Network processing which gives better results of both hair removal and
(ANN) which prove to be more accurate than the KNN, since segmentation. Fig. 1 shows the differences of gray level values
they iteratively adjust their connection weights by minimizing over the 3 RGB color channels. Detecting hair lines from the
the error between the ground truth and the predicted outputs. dermoscopic images is done based on the 2-D derivatives of
Yet, involving the concept of deep learning has become a trend Gaussian(DOG) of the blue component of the images. The
in melanoma detection systems. Different from the traditional DOG accurately detects lines of the hair in four different
pattern recognition systems, which heavily rely on the result directions. Thresholding is then applied to separate the hair
 Fig. 2. Hair lines detection using DOG filter.

Fig. 3. Skin lesion segmentation using thresholding. Fig. 5. Skin lesion segmentation using MorphGAC.

lines from the background. Otsu method is used. Finally,

to enhance and refine hair lines, a morphological dilation is different averages. The advantage of MorphACWE is that
introduced(see Fig. 2). The hair mask is filtered one more to the contour of the skin lesion is unnecessarily defined( see
remove the unwanted objects by eliminating all the objects that Fig. 5). However, the search area for the lesion is minimized
has circular shape and have an area smaller than a threshold. A to make faster and relevant this process. The dimensionality
lesion segmentation process was done to initially recognize the reduction is done based on the result of the initial segmentation
lesion’s borders, this step is done by thresholding the images which uses thresholding. After extracting the skin lesion,
and selecting the region with the largest area(see Fig. 3). This lowlevel features from three visual components: the texture (
step can approximately give us an information about those SIFT, HOG) descriptors, color (Color Name (CN), colorSIFT)
pixels belonging to the skin lesion. After that, a pixel classifi- features and other features describing the border of the lesion(
cation is done using hierarchical clustering to associate those convexity, circularity, irregularity index) ,are extracted and
pixels that does not belong to hair lines neither to the skin used as descriptors characterizing both benign and malignant
lesion. Based on that clustering, a new hair mask is defined lesions. For the training process, an SVM classifier is trained.
and an inpainting process is done. The inpainting algorithm Also, we used a KNN classifier which find the K nearest
used at that stage is the one proposed by [16] which repairs neighbors of an inspected lesion and based on the nature of
the hair line pixels of the images based on the neighboring the majority of neighbors, the lesion is classified whether a
pixels( see Fig. 4). Still, we need to ameliorate these methods malignant or benign.
by exploring different hair removal methods together. For the However, recently many methods using CNNs have been pro-
lesion segmentation, we used morphological snakes which are posed for solving this kind of problem. For that, we tested our
faster and numerically more stable than the traditional snakes. chosen dataset using a CNN( Convolutional Neural Network
There are two Morphological Snakes methods: Morphological ) architecture. The experiments have been performed using
Geodesic Active Contours (MorphGAC) and Morphological a training input size of 512 lesion images. However, CNNs
Active Contours without Edges ( MorphACWE). have to be trained using a huge dataset, This problem has
MorphACWE [5] works well when the pixel values of the been solved using the data augmentation which is a technique
inside and the outside regions of the object to segment have that can be used to artificially expand the size of a training
lesion images to efficiently train the model.
The proposed CNN architecture takes as inputs the whole
images sized 124 × 124 pixels, a preprocessing is needed
before training the model. The CNN architecture is composed
of 9 layers( see Fig. 6). Three convolution layers are used(
filter size is 3 × 3) with ReLU activation function, proceeded
by three spatial max pooling layers which down-samples the
feature maps. The convolution layers aim to learn a feature
representation of the input intensity patch, which typically
consists of a combination of linear and nonlinear operations,
Fig. 4. Final hair mask and image preprocessing result. i.e., convolution operation and activation function. The best
 Fig. 6. The CNN architecture.

solution for reducing overfitting is to obtain more training

data which is not practical in medical imaging due to the
lack of labeled data. The other solutions include regularization
with dropout which is a recently introduced regularization
technique. Thus, we introduced to the CNN model two dropout
layers. After extracting the features by the convolution layers
and down-sampled by the pooling layers, they are mapped by
a fully connected layer with a softmax activation function. The
final outputs of the network is the probabilities of a lesion to
be either malignant or benign. Fig. 7. Image samples of melanoma skin cancer from the ISIC dataset.

IV. E XPERIMENTS AND RESULTS TABLE I

THE ACCURACY VALUES OF USING THE DIFFERENT METHODS
The experiments were conducted using a public dataset
Methods KNN SVM CNN Majority voting
which is collected from the ISIC(International Skin Imaging Accuracy 57.3% 71.8% 85.5% 88.4%
Collaboration) archive, it contains more than 23000 images
of melanoma. We chose to work with only 640 skin lesion
images, containing benign and malignant lesions. They were
collected from a dermatoscopy tool. 512 of these images will we obtained the lowest accuracy considering the 5 nearest
be used as a training set, the rest as a testing set. Fig. 7 neighbors only. KNN barely can identify malignant skin
shows some examples of the skin lesion images. The CNN lesions since it is sensitive to outliers. However, SVM classifier
was trained using 640 images of size 124 × 124 using the performs better than KNN due to its efficiency and adaptabil-
3 different methods. The CNN was trained for 10 epochs. It ity. Although an SVM classifier achieved a quiet performance,
is shown that the CNN has the highest performance over the still the CNN is considered as a more powerful and robust
other two methods, although, classical machine learning and tools for identifying melanoma skin cancer. Yet, fusing the
image processing techniques has a few advantages and could decision of all these systems obviously can ameliorate their
also detect melanoma when a CNN can not. In this matter, a performance, the idea is to take the decision not based on one
result aggregation of the different method is used in order to single result but on many results.
ameliorate the performance of the melanoma detection system.
V. C ONCLUSION AND DISCUSSION
Actually, the fusion of multiple model prediction has appeared
as an accurate strategy to ameliorate the performance of many Melanoma is the deadliest form of skin cancer, can be a non
pattern recognition systems, rather than relying on one single life-threatening cancer if it is diagnosed at an early stage. Thus,
model prediction. In this paper, we use the majority voting it becomes vital to use supportive imaging techniques that have
approach which is one of the basic and intuitive approaches. It been shown to improve and facilitate the diagnosis process.
assigns a sample based on the most frequent class assignment These techniques are build based on strategies invented by
[4]. physicians to capture the melanoma at an early stage. We
Table. I summarizes the results of the three proposed method Have introduced a hybrid method for melanoma skin cancer
as well as the result of fusing them. Using the KNN classifier, detection that can be used to examine any suspicious lesion
in this paper. Our proposed system rely on combining the [13] Zaher Khazaei, F Ghorat, AM Jarrahi, HA Adineh, M Sohrabivafa,
prediction of three different methods using majority voting. and E Goodarzi. Global incidence and mortality of skin cancer by
histological subtype and its relationship with the human development
Most of the systems rely only on using two rules defined index (hdi); an ecology study in 2018. World Cancer Research Journal,
by physicians: the ABCD and the Blue-Black rule, which 6:13, 2019.
are shown to have some limitations and proved ineffective [14] Ilias Maglogiannis and Charalampos N Doukas. Overview of advanced
computer vision systems for skin lesions characterization. IEEE trans-
in some cases. Another concept was proposed which is the actions on information technology in biomedicine, 13(5):721–733, 2009.
ugly duckling. The idea is to observe not only the morphology [15] Dirk Schadendorf, Alexander CJ van Akkooi, Carola Berking, Klaus G
of the lesion in question, but also to compare it to that of Griewank, Ralf Gutzmer, Axel Hauschild, Andreas Stang, Alexander
Roesch, and Selma Ugurel. Melanoma. The Lancet, 392(10151):971–
surrounding lesions, looking for an outlier in the background 984, 2018.
of similar-appearing moles. This clue was clinically proved [16] Alexandru Telea. An image inpainting technique based on the fast
to be a good criteria to detect melanoma, but it has not been marching method. Journal of graphics tools, 9(1):23–34, 2004.
[17] Jufeng Yang, Xiaoxiao Sun, Jie Liang, and Paul L Rosin. Clinical skin
explored in the automatic melanoma detection systems yet. lesion diagnosis using representations inspired by dermatologist criteria.
Thus, an idea of finding the weird skin lesion among a set of In Proceedings of the IEEE Conference on Computer Vision and Pattern
lesions could be explored in a further study. Those who work Recognition, pages 1258–1266, 2018.
[18] Chanki Yu, Sejung Yang, Wonoh Kim, Jinwoong Jung, Kee-Yang
on the field of medical imaging has been suffered from a very Chung, Sang Wook Lee, and Byungho Oh. Acral melanoma detection
serious issue which is the lack of enough labeled data to train using a convolutional neural network for dermoscopy images. PloS one,
their systems. Thus, the use of semi-supervised learning could 13(3), 2018.
be used to cover this issue.

R EFERENCES

[1] Qaisar Abbas, M Emre Celebi, and Irene Fondón Garcı́a. Hair removal
methods: a comparative study for dermoscopy images. Biomedical
Signal Processing and Control, 6(4):395–404, 2011.
[2] Giuseppe Argenziano, Caterina Longo, A Cameron, S Cavicchini, J-Y
Gourhant, A Lallas, I McColl, C Rosendahl, L Thomas, D Tiodorovic-
Zivkovic, et al. Blue-black rule: a simple dermoscopic clue to recog-
nize pigmented nodular melanoma. British Journal of Dermatology,
165(6):1251–1255, 2011.
[3] Samy Bakheet. An svm framework for malignant melanoma detection
based on optimized hog features. Computation, 5(1):4, 2017.
[4] D Ballabio, R Todeschini, and V Consonni. Recent advances in high-
level fusion methods to classify multiple analytical chemical data. In
Data Handling in Science and Technology, volume 31, pages 129–155.
Elsevier, 2019.
[5] Tony F Chan and Luminita A Vese. Active contours without edges.
IEEE Transactions on image processing, 10(2):266–277, 2001.
[6] Fekrache Dalila, Ameur Zohra, Kasmi Reda, and Cherifi Hocine. Seg-
mentation and classification of melanoma and benign skin lesions. Optik,
140:749–761, 2017.
[7] Florentia Dimitriou, Regina Krattinger, Egle Ramelyte, Marjam J
Barysch, Sara Micaletto, Reinhard Dummer, and Simone M Goldinger.
The world of melanoma: epidemiologic, genetic, and anatomic dif-
ferences of melanoma across the globe. Current oncology reports,
20(11):87, 2018.
[8] Diwakar Gautam and Mushtaq Ahmed. Melanoma detection and
classification using svm based decision support system. In 2015 Annual
IEEE India Conference (INDICON), pages 1–6. IEEE, 2015.
[9] Tsong-Long Hwang, Woan-Ruoh Lee, Shu-Chiou Hua, and Jia-You
Fang. Cisplatin encapsulated in phosphatidylethanolamine liposomes
enhances the in vitro cytotoxicity and in vivo intratumor drug accumu-
lation against melanomas. Journal of dermatological science, 46(1):11–
20, 2007.
[10] Ashwin R Jadhav, Arun G Ghontale, and Vimal K Shrivastava. Segmen-
tation and border detection of melanoma lesions using convolutional
neural network and svm. In Computational Intelligence: Theories,
Applications and Future Directions-Volume I, pages 97–108. Springer,
2019.
[11] Uzma Jamil, Asma Sajid, Majid Hussain, Omer Aldabbas, Afshan Alam,
and M Umair Shafiq. Melanoma segmentation using bio-medical image
analysis for smarter mobile healthcare. Journal of Ambient Intelligence
and Humanized Computing, 10(10):4099–4120, 2019.
[12] JC Kavitha, A Suruliandi, D Nagarajan, and T Nadu. Melanoma
detection in dermoscopic images using global and local feature extrac-
tion. International Journal of Multimedia and Ubiquitous Engineering,
12(5):19–28, 2017.