Application of Fractional Order ABC and GA For Neural Network Training and Clustering Process
Application of Fractional Order ABC and GA For Neural Network Training and Clustering Process
5 (2015) 900-913
G. Lavanya
Research Scholar,Anna University Regional Centre, Madurai
Tamilnadu, India
lavanya0283@gmail.com
S. Srinivasan
Professor & Head, Department of Computer Science and Engineering,
Anna University Regional Centre, Madurai
Tamilnadu, India
srinisss@yahoo.com
Abstract
Literature presents several search algorithms to find an item with specified properties from a search space defined
by a mathematical formula or procedure. One of the widely accepted algorithms is optimization algorithm which
can find the optimal element within a certain period of time if the search space is defined. Recent works formulate
several problems as optimization problems which were then solved by many optimization algorithms. Accordingly,
in a previous paper, a hybrid optimization algorithm, called FAGA was proposed using fractional order Artificial
Bee Colony (ABC) and Genetic Algorithm (GA) for optimization to solve the existing benchmark problems. In this
paper, we have planned to apply the FAGA algorithm to well defined-real time problems of neural network training
and the clustering process. Through neural network training, data classification will be done by making use of
FAGA algorithm as neural network training procedure. Similarly, medical image segmentation will be done using
clustering process through FAGA algorithm. The performance of the FAGA algorithm in those two processes will
be evaluated with different evaluation metrics and the comparison of the FAGA algorithm will be also carried out
with the existing ABC and genetic algorithm.
1
efficient for solving problems of optimization. Swarm problem performs better than the orthodox techniques in
Intelligence is a theory that deals with the significance an efficient way.GA evolve as a superior from orthodox
of team-work regarding social insect colonies, different methods and optimization methods in four following
animal societies, or the association of species to their ways: a) GA search is quite distinguished from a
environment, to propose methods or to distributed population of points. So, it will not get blocked in local
problem-solving devices. Acquiring the features as well optimal solution like in orthodox techniques, which
as manners of organisms, many methods of improving make search with respect to a single point. b) GA
the optimization deals with swarm intelligence, are applies probabilistic method of selection rather than
introduced one after another. Also, many applications of deterministic ones. c) GA deals with Chromosome,
optimization algorithms, related to swarm intelligence hence is encoded type of potential solutions’
2, 3, and 4
are introduced continuously, along with several parameters, not the parameters themselves. d) GA
methods for solving optimization issues have been depends on fitness score that is acquired from objective
5
developed. Furthermore, several orthodox and functions, without other derivative or auxiliary
20
experimental approaches have become quite significant. information.
Within some of the experimental approach, a significant In recent times, ABC algorithm has been suggested
theory, which is a method related to social behavior of regarding optimization problems. According to the ABC
6, 7
few creatures. Swarms inherently utilize forms of algorithm, bee's job is to find appropriate flowers and to
decentralized control and self-organization to locate suitable place of flowers. This total procedure is
8
accomplish their goals. SI is the combined behavior of divided in an asynchronous process, which means the
decentralized, self-organized natural or artificial bees communicate with each other, after one find the
21
systems. food source, by dancing on the located source of food.
SI systems are consisted of a group of simple agents Some significant benefit of this algorithm is: a) the
9
communicating within them and with the environment. formation of the algorithm is ideal for distinguished
The agents do not get any instruction from any control approach, hence less time consuming, b) Dependable,
system, they follow simple protocol, by interacting since it enables adjustments and considers specific
22
within themselves and results in the emergence of knowledge of the problem by observing nature. A
intelligence global behavior which is unknown to the hybrid optimization algorithm FAGA is presented in
10, 11
individual agents. SI examined the combined Ref. 23 using fractional order ABC and GA. The FAGA
behavior of systems composed of several individuals solves the existing problems faced by the exiting
communicating locally with each other and with their optimization problem such as convergence problem,
environment. Recent studies have aimed on the meta- local minimization problem, low optimal speed and
experimental theory, such as, Ant Colony Optimization time.
12
(ACO), Particle Swarm Optimization (PSO) and ABC The FAGA algorithm has four phases such as,
13
. It has been proved, that these algorithms have the employee bee, onlooker bee, mutation and scout bee. In
ability to offer enhanced solutions than conventional employee bee phase, neighbor solution is generated
algorithms. By borrowing ideas with respect to nature based on ABC algorithm. Then, in onlooker bee, the
swarming behavior in computer methodologies, probability is used to select a solution and new solution
methods obtained for hard optimization problems is generated based on fractional calculus-dependent
14, 15
become robust, scalable and easily distributed. neighbor solution. The mutation operation of GA is
Some of the essential swarm methods are: PSO, ACO, used in the mutation module and then the scout bee
ABC, Consultant-Guided Search (CGS) and many phase is carried out. A genetic algorithm is generally
more. Again, the complete swarm indicates intelligent said to converge when there is no critical change in the
behavior, serving ultimate solutions for complex estimations of fitness of the population from one
problems like, predator evasion and shortest path generation to the next. Both ABC and GA have local
16, 17
finding . optimization problem. The main features of FAGA are
GA is an orthodox optimization algorithm related to that it directly influences the speed of convergence of
natural selection and the mechanism of population the algorithm and it also permits to understand the local
18,19
genetics. These techniques for solving optimization and search capability. FAGA has potential for
interpreting the evolution of the algorithm and to control image segmentation and data classification and the third
its convergence and for the entire optimization problem section delineates the results obtained for our technique
which existed in the conventional method; FAGA has compared with the existing techniques and the last
the ability to provide results. section concludes our technique.
There are several works available in the literature
but some of those significant research works are 2. Proposed Application of Fractional Order
reviewed here. One can deduce from Ref. 24 that the ABC and GA for neural network training
efficiency of differential evolution (DE), PSO, and and Clustering Process
Evolutionary Algorithms (EAs), which can be applied This section shows the proposed application of
as numerical optimization methods. A well-organized Fractional Order ABC and GA (FAGA) for neural
approach for adapting control parameter settings related network training and clustering process. The clustering
25
with DE has been introduced by Janez Brest et al. process based on FAGA is done for image
The efficiency of the ABC with GA, PSO and PS-EA, segmentation; and the training of neural network based
which were swarm intelligence and population related on FAGA is done for data classification. The FAGA [is
a hybrid algorithm of fractional order Artificial Bee
methods as the ABC algorithm were verified by Dervis 23
26 Colony (ABC) and Genetic Algorithm (GA). The
Karaboga and Bahriye Basturk. A Discrete Particle
FAGA has four phases namely, employee bee phase,
Swarm Optimization (DPSO) technique to solve the
onlooker bee phase, mutation phase and scout bee
NP-hard single machine total weighted tardiness phase. In Ref. 23 we proposed FAGA and obtained
scheduling problem in presence of sequence-dependent better minimization and convergence rate using
setup times was suggested by. Davide Anghinolfi et al. different functions as fitness calculation. Here, the
27
Wu Deng et al. have introduced a dual-stage hybrid FAGA is applied for data classification and image
swarm intelligence optimization algorithm known as segmentation. The process of FAGA is as follows:
GA-PSO-ACO algorithm, which unites the theory of
Employee Bee Phase
GA, PSO and ACO related to evaluating the traveling
2 7KH FRORQ\ RI DUWL¿FLDO EHHV FRQWDLQV WKUHH JURXSV RI
salesman problem. A hybrid optimization algorithm bees: employed bees, onlookers and scouts. A bee
related to Cat Swarm Optimization (CSO) and ABC was waiting on the dance area for making decision to choose
1
suggested by Pei-Wei Tsai et al. Min-Yuan Cheng and a food source is called an onlooker and a bee going to
Li-Chuan Lien have suggested an optimization hybrid the food source visited by it previously is employed bee.
swarm algorithm the particle bee algorithm (PBA), that A bee carrying out random search is called a scout. For
take knowledge from the intelligent swarming behavior every food source, there is only one employed bee. In
of honeybees and birds, to integrate BA global search other words, the number of employed bees is equal to
28 the number of food sources around the hive. The
ability along with the local search advantages of PSO.
employed bee whose food source is exhausted by the
In this paper, we apply the FAGA algorithm for employed and onlooker bees becomes a scout. In each
image segmentation and data classification. The image cycle, the search consists of four steps: sending the
segmentation is done by converting the RGB image into employed bees onto the food sources and then
grey and generating number of solutions randomly and measuring their nectar amounts; selecting of the food
by processing each solution using FAGA algorithm. The sources by the onlookers after sharing the information
FAGA give a best solution based on best fitness and the of employed bees and determining the nectar amount of
Centroid in the best solution is used to cluster the pixels. the foods; mutation operation; determining the scout
bees and then sending them onto possible food sources.
The clustered pixels would be formed as segmented
Here, the position of a food source represents a possible
image. The data classification is done by training the solution of the optimization problem and the nectar
neural network. The neural network uses weights DPRXQWRIDIRRGVRXUFHFRUUHVSRQGVWRWKH¿WQHVVRIWKH
between the neurons in each layer. The weights in the associated solution. The number of the employed bees
neural networks are assigned, while training is based on or the onlooker bees is equal to the number of solutions
the FAGA algorithm. The neural network trained using in the population. In employee bee phase, neighbor
FAGA algorithm would classify the data in a dataset. solution is generated based on ABC algorithm.
The structure of the paper is organized as follows: the At the initialization stage, a set of food source
second section explained our application of FAGA on positions are randomly selected by the employed bees
and their nectar amounts are determined. Then, these
bees come into the hive and share the nectar information by z j ,k and the new position is represented by y j ,k ,
of the sources with the onlooker bees waiting on the which is defined by the equation below:
dance area within the hive. Initially, ABC generates a z j ,k y j ,k j ,k ( y j ,k y i,k ), i z j
(2)
randomly distributed initial population represented by
Where, i {1,2,..., N p } and k {1,2,..., M } . M j, k
popini having N p solutions where each solution is the
LVDUDQGRPQXPEHULQWKHUDQJHí:KLFKFRQWUROV
food source position and N p is the population size.
the production of a neighbor food source position
Each solution is represented by g j , where 1 d j d N p is
around y j ,k DQG WKH PRGL¿cation represents the
a M-dimensional vector, where M is the number of
comparison of the neighbor food positions visually by
optimization parameters taken into consideration. After
the bee. The position update equation shows that as the
initialization, the population of the positions is subjected
difference between the parameters of the y j , k and yi , k
to repeated cycles of the search processes of the
decreases, the perturbation on the position y j , k also
employed bees, the onlooker bees and scout bees.
decreases. Thus, as the search approaches to the
optimum solution in the search space, the step length is
Onlooker Bee Phase adaptively reduced.
In this phase, selection of the food sources by the Fractional Calculus (FC) is a natural extension of
onlookers after receiving the information of employed the classical mathematics and extents the possibility of
bees and generation of new solution based on fractional taking real number powers or even complex number
calculus is carried out. The onlooker bee prefers a food
powers of the differentiation operator or the integration
source area depending on the nectar information
distributed by the employed bees on the dance area. As operator. In our case, fractional calculus is used for
the nectar amount of a food source increases, the position updation step. Rearranging the position
probability with which that food source is chosen by an updation step, we have equation as:
onlooker increases, too. Hence, the dance of employed
bees carrying higher nectar recruits the onlookers for the
z j ,k y j ,k M j ,k ( y j ,k y i ,k )
(3)
food source areas with higher nectar amount. An In the above equation z j , k is the position update
onlooker bee chooses a food source depending on the
probability value associated with that food source Pj from y j ,k in the previous step, representing in the time
given by the expression: domain, we can write y j ,k as zt when z j ,k is taken as
Fj
Pj
NP z t 1 . Hence the above equation is formed as:
¦ Fk z t 1 z t M j ,k ( y j ,k y i ,k )
k 1 (1) (4)
F
In the above equation j LV WKH ¿WQHVV YDOXH RI WKH z t 1 zt is the discrete version of the
The left side
j th solution evaluated by its employed bee, which is
proportional to the nectar amount of the food source in derivative of order D 1 . Hence the equation is formed
the position j and N p is the number of food sources as:
which is equal to the number of employed bees. An D D [ z t 1 ] M j ,k ( y j ,k y i ,k )
(5)
onlooker bee evaluates the nectar information taken Here, by discrete time approximation (taking first
from all employed bees and chooses a food source with four terms), the equation is formed as:
a probability related to its nectar amount and then, she
1 1 1
SURGXFHVDPRGL¿FDWLRQRQWKHSRVLWLRQVROXWLRQLQKHU z t 1 D z t D z t 1 D (1 D ) z t 2 D (1 D )( 2 D ) z t 3 M j ,k ( y j ,k y i ,k )
2 6 24
memory and checks the nectar amount of the candidate
source (solution). Providing that its nectar is higher than (6)
that of the previous one, the bee memorizes the new Rearranging, we have the updated position equation
position and forgets the old one. In other words, a as:
greedy selection mechanism is employed as the 1 1 1
z t 1 D z t D z t 1 D (1 D ) z t 2 D (1 D )( 2 D ) z t 3 M j ,k ( y j ,k y i ,k )
2 6 24
selection operation between the old and the current food
sources. The position update is made with the use of (7)
fractional calculus. Let the old position be represented
dimension value, the RGB image is converted to grey assumption but actually we are not merging. This is
image. The adjoined pixels are merged (by assumption) done because while considering a pixel we have to take
with each pixel of the image for clustering process. the neighboring pixels too. Fig.2 shows the merging of
Initially, the number of clusters required is set by the adjoining pixels of an image.
user and based on it the initial solution is generated to
process the FAGA. The image given as input is then
segmented using FAGA.
compared the minimization and convergence rate with changed and the fitness is calculated for all the solution
other algorithms. In this paper, the fitness is calculated in each phase. The best solution identified at each phase
as follows: is stored for further process. The best solution is based
3 on best fitness (nectar amount). The solution that
F j IS ¦ dist C , PX
k nm obtained least fitness value is judged as best solution
k 1 (9) since a good cluster requires less distance between the
centroid and the elements in it. The FAGA is then
repeated using the solutions obtained at the final phase
In the above equation F j is the fitness of j th until the number of iterations set. At the end of the
solution where j N1toN p ; and Ck is the k th iteration, the best solutions stored are compared with
each other and the centroids of the best solution among
centroid of j th solution and k varies from 1 to 3 since them are chosen to cluster the pixels of the image. The
we take three centroids; and PX is the pixels in the clustered pixels are then formed as segmented image.
image; and IS represents image segmentation. The
above equation is explained as follows: if we consider 2.2. Application of FAGA for Data Classification
the first solution, the distance between each centroid of This section delineates the application of FAGA for data
the first solution and each pixel of the image is taken classification. Here a neural network which is trained
separately and summed to get the fitness value of it. based on FAGA algorithm is used to classify the
dataset. In neural network the weight values between
2.1.4. Segmentation the neurons of each layer is fixed by the FAGA
The FAGA algorithm has different phases and the algorithm. Fig.4 shows a sample neural network.
solutions generated are subjected to all the phases of the
FAGA algorithm. In each phase, the solutions would get
The neural networks would have three layers which In between each neuron on each layer, there would have
are input layer, hidden layer and output layer. The weight values which are used to train the neural
inputs are given to the neurons in the input layer and the network. Initially, the weight values are random
output is obtained from the neuron in the output layer. numbers and later the weight values are changed during
training using certain algorithm. Here we used the generated that have different weight values. Fig.5 shows
FAGA to fix the weight values between each layer. To the sample initial solution generated for assigning the
apply FAGA initially numbers of solutions are weight values of neural network using FAGA.
Fig.5 explains as follows: the {N1, N2, …., Np} The calculation of fitness is done on each phase in
represents the number of solutions generated to train the FAGA process and the best solution is saved. The
neural network; and the {W1A, W1B, …., WCO} FAGA is repeated until the iteration numbers set using
represents the weights in the neural network; and V the solutions obtained in last phase as input to next
represents the corresponding weight values for N and iteration. The weight values in the best solution
W. These solutions are processed using FAGA and the obtained at last iteration is assigned to the neural
best solution is chosen based on FAGA. network for data classification.
Input: Image
Output: Segmented Image
1. Start
2. Get the input image
3. Convert RGB to Grey
4. For each pixel
5. Merge (by assumption) neighboring pixels
6. End for
7. Set the number of centroids
8. Generate initial solutions that have centroids on each solution
9. For each solution
10. Apply FAGA
11. For each phase of FAGA
12. Save the best solution based on fitness calculation of image segmentation
13. End for
14. End for
15. Take the overall best solution for further process
16. Set the centroids of the best solution to cluster
17. The clustered pixels are formed as segmented image
18. Stop
Input: Dataset
Output: Classified data
1. Start
2. Generate initial solutions randomly that have weight values to train neural network
3. For each solution
4. Apply FAGA
5. For each phase of FAGA
6. Save the best solution based on fitness calculation of neural network training
7. End for
8. End for
9. Take the overall best solution for further process
10. Assign the values in the best solution as the weight values for neural network
11. Give the dataset as input to the trained neural network
12. The trained neural network provide classified data
13. Stop
RAM and Windows 7 OS. The implementation is done The Lung Cancer dataset has fifty six attributes and
on Matlab R2013a. thirty two instances.
In Table 1, the black colored region in normal brain region; and the green colored region is the segmented
tumor images is the background region; and the grey brain region; and the pink colored region in the brain is
colored region in normal brain tumor images is the brain the segmented tumor region. Table.2 shows the
region; and the white colored region in the brain is the performance comparison using first image in terms of
tumor. In the segmented brain tumor images of Table.1, accuracy.
the blue colored region is the segmented background
The accuracies obtained for our technique compared using first image, our technique performed better in
to the other existing techniques is shown in Table.2 and terms of non-tumor segmentation and tumor
the accuracy is evaluated for three regions which are segmentation compared to the other existing techniques.
brain region (non tumor region), tumor region and Overall if we take the average of the accuracy of the
background region. Here our technique FAGA is three regions (non-tumor, tumor and background), our
compared with the existing techniques which are ABC, technique performed better than the other techniques.
GA and ABC-GA. While comparing the performance
Table 3 shows the accuracies obtained using second of tumor region segmentation, our technique performed
image. Here in terms of non-tumor region better compared to other techniques. While comparing
segmentation, our technique and GA technique the average accuracy of the three regions, our technique
performed better than the other techniques; and in terms performed better than the other algorithms.
GA 1 0.619632 1 0.873211
Table 4 shows the accuracies obtained using third segmentation, our technique performed better compared
image. Here in terms of non-tumor region segmentation, to other techniques. When the average of all the three
our technique and the GA technique performed better regions is compared, our technique performed better
than other algorithms; and in terms of tumor region than other techniques.
Table 6 shows the performance comparison using the x-axis denotes seventy percentages of the data is
fifth image. Here in terms of non-tumor region used for training and thirty percentages is used for
segmentation and tumor region segmentation, the testing; and the 80.2 represents eighty percentages of
accuracy of our proposed technique is better compared data is used for training and twenty percentages for
to other techniques. When comparing the overall testing; and the 90.1 represents ninety percentages of
performance, the proposed technique is better than other data is used for training and ten percentages is used for
techniques. testing. From Fig.8, it is clear that when increasing the
percentage of testing data, the accuracy of FAGA is
increasing from 60% to 70%. Thus it is clear that the
3.3. Performance based on Data Classification accuracy of FAGA is more compared to that of other
This section shows the performance of our technique method. Here in Fig.8 for all the three varied
based on data classification compared to the existing percentages of PID dataset, our technique FAGA
techniques. To compare the performance based on data performed better compared to other algorithms.
classification, we used three datasets which are PID,
Heart Disease and Lung Cancer for experimentation.
Fig.8 shows the accuracy obtained using PID dataset. 80
60
Accuracy
80
70 40 ABC
60
ABC GA
Accuracy
50 20
40 FAGA
30 GA 0
20 70,3 80,2 90,1
10 FAGA % of Training and Testing
0 (Proposed)
70,3 80,2 90,1
Fig.9. Accuracy using Heart Disease dataset
% of Training and Testing
Fig.9 shows the accuracy obtained for our proposed
technique compared to the existing techniques using
Fig.8. Accuracy using PID dataset Heart Disease dataset. Here, the proposed technique
classified the dataset better than the other techniques for
In Fig.8 the accuracy obtained for the proposed all the varied percentages of dataset. From Fig.9, it is
technique is compared with the existing techniques by clear that there is a slight change in the accuracy of
varying the datasets for training and testing. The 70.3 in FAGA from 70 to 75%, as the training data increases.
Animals by Social Behavior for the Traveling Salesman International Arab Journal of Information Technology,
Problem, International Journal of Combinatorial Vol. 13, No.3. (Published Online)
Optimization Problems and Informatics (2012) vol. 3 no. 24. J. Vesterstrom and R. Thomsen, A Comparative Study of
3 104-123. Differential Evolution, Particle Swarm Optimization, and
17. B. Shivakumar and T. Amudha, A Novel Nature-inspired Evolutionary Algorithms on Numerical Benchmark
Algorithm to solve Complex Generalized Assignment Problems, Congress on Evolutionary Computation (2004)
Problems, International Journal of Research and vol. 2 1980-1987.
Innovation in Computer Engineering (2012) vol. 2 no. 3 25. J. Brest, S.W. Greiner, B. Boskovic, M. Mernik and V.
280-284. Zumer, Self-Adapting Control Parameters in Differential
18. J. Holland, Adaptation in natural and artificial systems Evolution: A Comparative Study on Numerical
(University of Michigan Press Ann Arbor, Mich. 1975). Benchmark Problems, IEEE Transactions on
19. D. Goldberg, Genetic algorithms in search, optimization Evolutionary Computation (2006) vol. 10 no. 6.
and machine Learning, (Addison-Wesley Publishing Co., 26. D. Karaboga and B. Basturk, A powerful and efficient
Inc., Reading, Mass. 1989). algorithm for numerical function optimization: artificial
20. R. Sharapov, Genetic Algorithms: Basic Ideas, (Variants bee colony (ABC) algorithm, J Glob Optim (2007) vol.
and Analysis, Vision Systems: Segmentation and Pattern 39 459–471.
Recognition, 2007). 27. D. Anghinolfi and M. Paolucci, A new discrete particle
21. T. Singh and Z.M. Sandhu, An Approach in the Software swarm optimization approach for the single-machine total
Testing Environment using Artificial Bee Colony (ABC) weighted tardiness scheduling problem with sequence-
Optimization, International Journal of Computer dependent setup times, European Journal of Operational
Applications (0975 – 8887) (2012) vol. 58, no. 21. Research (2009) vol. 193 no. 1 73–85.
22. E. Gerhardt E and H. Gomes, Artificial Bee Colony 28. M. Cheng and L. Lien, hybrid swarm intelligence based
(ABC) Algorithm for Engineering Optimization particle-bee algorithm for construction site layout
Problems, 3rd International Conference on Engineering optimization, Journal Expert Systems with Applications
Optimization, (2012). (2012) vol. 39 no. 10.
23. G .Lavanya and S. Srinivasan, FAGA: Hybridization of
Fractional Order ABC and GA for Optimization",