Performance and Comparison of Various Machine

Learning Classifier for Sentiment Analysis on

Amazon Dataset
*
S.Madhusudhanan,
Department of Computer Science and Kannamma.R Jyothi N.M,
Engineering Department of Artificial intelligence Department of Computer Science and
Rajalakshmi Engineering College and Data Science Engineering, Koneru Lakshmaiah
Thandalam,Tamilnadu,India Prathyusha Engineering College Education Foundation, Vaddeswaram,
ssmadhu80@gmail.com Thiruvallur,Tamilnadu,India Guntur (DT), Andhrapradesh,India
Ssmadhu80@gmail.com Rangaraju Institute of Engineering and
N.Arun Vignesh Technology,Hyderabad
Department of Electronics and
Communication Engineering, Gokaraju
Abstract—The advent of web 2.0 provides an ideal platform opinions, emotions, and perspectives in the given text.
for individuals to share their ideas, opinions, and feelings. Web
Opinion Mining / Sentiment Analysis is a text mining task that
Sentiment Analysis requires considerably more knowledge of
aims to build a system that automatically extracts, identifies, the natural language than text examination and subjective
and categorizes user opinions from natural language text, user
created material, or user generated media. In this work, analysis [20]. The previous algorithm considers the only
performance of different classifiers was analyzed and frequent occurrence of the words in a document but fails to
compared. SVM with 96% and Logistic Regression with 94%
on Electrnics review dataset, were outpermed well with best identify the target opinion especially when messages are
classification accuracy compared with all other classifiers. short. It is the process of identifying subjective information
Keywords: Machine Learning; Sentiment Analysis; as positive or negative. It helps the business organization and
Classification; Bayesian; Support Vector Machine; Random
Forest. individual to make the decision. Recently a lot of research

1 INTRODUCTION works focus on the automatic identification of sentiment by

In Data Science, Artificial intelligence (AI) has become more using machine learning algorithms.[21][22].
popular and focused on recent research work. Machine In this paper, six popular classification techniques namely
Learning (ML) is a part of AI for various applications to Logistic Regression (LR), Support Vector Machine (SVM),
build a classification model [9]. During the training phase, Random Forest (RF), Naive Bayes (NB), Decision Tree
Features and patterns are identified and classifiers are trained (DT), and K Nearest Neighbors (KNN) are analysed and
based on the features set. The Classifier predicted the test by compared its performance in sentiment analysis.
using training data [13].ML is used in day-to-day activities.
E.g., the railway reservation system uses classification
algorithm to predict seat confirmation probability, product 2 RELATED WORK

recommendations in the e-commerce website are based on

The authors Shyamasundar L B., Jhansi Rani P, proposed
purchase history [11][12]. Prediction of words in a Google
feature selection method called unigram and bigram with χ2
search engine, diagnosis based on medical image analysis, (Chi-Square) and Singular Value Decomposition for
dynamic fare in airlines, and so on. In ML, mathematical
dimensionality reduction with four types of scaling methods
model like probability and statistics are used to predict the
and machine learning classifier is used with accuracy rate of
patterns, facts, associations, and trends. In the decision-
84.14%. [21].
making process, to recognize unseen patterns ML techniques
The authors Morinaga et.al performed sentiment
are used. Classification is task which is a part of ML. It is a
classification on product review dataset using SVM and
process to build a model that separates test data label based
Naïve Bayes Classification methods by constructing Bag-of-
on the training phase [11][7][8][10]. Sentiment analysis is
Word Model[13].
one of the applications of text mining that purpose to detect

XXX-X-XXXX-XXXX-X/XX/$XX.00 ©20XX IEEE

 [ () ]
Abbasi et.al [1] proposed hybrid method which combines b
P ∗P ( a )
information gain and entropy weighted genetic algorithm a
(EWGA) for feature selection with SVM classification. The P ()a
b
=
P ( b)
proposed method would product better accuracy of 91% for (1)
movie review data set.
The problem of constructing a domain-dependent sentiment Where P ( ab ) is posterior probability, P ( ba )
is conditional

lexicon with supervised learning method was explored for

probability of b given a known as likelihood probability, P(a)
sentiment classification is implemented by the authors
is a prior probability of a.
Yassine et.al[23]. They utilized unigram and bigram feature
selection techniques with SVM classification algorithm and 3.2.3 SUPPORT VECTOR MACHINE (SVM)
achieved 88% accuracy. SVM examines data, describes decision boundary points, and
To improve the performance of sentiment classification, the uses the segments for the estimation, which are acted in the
author Hung and H. Lin [9] reevaluates objective sentiment input space. In the ML algorithm, SVM outperforms well for
words in SentiWordNet. They used Term frequency with classification [22]. The essential data is converted into two
SVM algorithm to analyse the performance.They achieved vectors of size m. As of now, every datum (passed on as a
accuracy of 77.9% . vector) is referenced into a class. By then, the machine
perceives the separation between two labels which is far from
3 METHOD
any space on the trained data. Support Vectors are essentially
3.1 DATA SET
the coordinates of individual perception.[24][13][17]
Sentiment analysis uses many opinion datasets for
classification. The popular movie review dataset which is 3.2.4 DECISION TREE (DT)
available online 1
[4][5][14]. Data repositories such as The decision tree works by creating a binary tree having a
Kaggle, Data.world, UCI, ASU, Open Knowledge Labs, condition on internal nods and class labels on the leaf nodes.
Kdnuggets, and SNAP. [21] First Information Gain (IG) and entropy are calculated
on each attribute, the attribute with maximum IG is selected
3.2 CLASSIFICATI`ON ALGORITHMS
as a split node. Now the new attributes are labeled and
3.2.1 LOGISTIC REGRESSION (LR)
calculate IG and entropy for each node repeatedly to form the
The most popular supervised algorithm is used to envisage
internode with the leaf node. Leaf node contains the majority
the probability of the target variable. The logistic function is
label [15][18].
used to calculate likelihood probability for independent
variable. This function is called relation function since it 3.2.5 RANDOM FOREST (RF)
relates changes in the independent variables to a higher (or RF classifier is based collection of tree-structured classifier
lower) probability of occurrence of the event by the [2][19]. Many Decision trees predict decision on dataset and
dependent variable [22] [8]. final decision is done by majority voting process. Classifier

3.2.2 NAIVE BAYES CLASSIFIER (NB) predicts the result on the data set. The working principle of

This method uses the Bayes theorem to calculate the most RF are discussed below:
frequent class for the new instance. It can handle some Step-1: Choose random T input points.
features or classes [5][6][16][22]. The Bayes hypothesis is a Step-2: Build the decision tree for the input points.
method to calculate difference between likelihoods P (a | b) Step-3: Forecast the decision which comprise two or more
from P(a), P(b) and P (b | a). likelihoods P (a | b) from P(a), Decision trees.
P(b) and P (b | a). The formula to calculate probability is as Step-4: Repeat Step 1 & 2.
follows : Step-5: Predictions of each decision tree are involved in the
testing data; RF chosen the final decision based on majority
voting process.

1
http://www.cs.cornell.edu/People/pabo/movie-review-data
3.2.6 K NEAREST NEIGHBORS (KNN) Random 82 40.25 41.84 0.81 0.84
The K-Nearest-Neighbors (KNN) is a straightforward and Forest

powerful classification method. In this method, data are Naive 74 29.35 43.20 0.84 0.58
Bayes
arranged by a dominant part vote of its Neighbors. It
Decision 72.43 30.94 41.49 0.79 0.62
acknowledges the new data similarity and is assigned to the
Tree
available similar category. For classification of test data, it
KNN 50.54 18.97 31.57 0.51 0.38
retrieves the stored information, based on that it assigns the
In above Table2, Logistic Regression classifier
class to test data.[3] It is also known as lazy learners, because
outperforms maximum accuracy.
of the storing the data and retrieves it for classification. [20]
4.2 DATASET-II Amazon Electronics Product Review dataset
4 EXPERIMENTS
[14]. The performance of various classifiers is shown in
In this work, datasets are collected from the repository [3][4].
Table3.
The Python software is utilized for implementation.
Confusion matrixes are generated to find the correctly
classified class. The same environment variable is used for TABLE3: EXPERIMENT RESULT OF ELECTRONICS
all algorithms. Algorith Classific TP FP Preci Reca
m ation Rate Rate sion ll
4.1 DATASET-I: Clothing, Shoes, and Jewelry [14]. Rating 5 Accuracy (in%) (in %)
and 4 are considered as positive reviews and 1 and 2 are (in %)
Logistic 94 47.58 46.74 .96 .93
considered as negative reviews. Rating 3 is neglected. 25000 Regressi
reviews of positive and negative are selected randomly and on
make it balanced. 80:20 for training and testing the dataset. SVM 96 47.89 47.72 .96 .95
Similar procedure is followed for all data set. Table 1 shows Random 92 47.80 43.86 .89 .95
Dataset Statistics. The performance of various classifiers is Forest

shown in Naive 79 47.12 37.70 .72 .94

Bayes
TABLE 1: STATISTICS OF DATASET Decision 73 25.21 47.58 .92 .50
Dataset Total Positive Negative Tree
Revies Review Review
Clothing, Shoes and 278677 221597 26655 KNN 84 45.07 39.20 .81 .90
Jewelry
In above table3, SVM classifier outperforms maximum
Electronics 308277 230674 77603
accuracy.
Health and Personal Care 346355 279801 33300

Sports and Outdoors 296337 253017 19249

4.3 DATASET-III Amazon Health and Personal Care Review
dataset [14]. The performance of various classifiers is shown
Toys and Games 167597 140235 11005
in Table4.

TABLE4: EXPERIMENT RESULT OF HEALTH AND PERSONAL CARE

TABLE2: EXPERIMENT RESULT OF CLOTHING, SHOES AND JEWELLERY
Algorithm Classification TP FP Precision Recall
Algorith Classifica TP FP Prec Reca Accuracy Rate Rate
m tion Rate Rate ision ll (in %) (in%) (in
%)
Accuracy (in%) (in %)
Logistic 84 42.13 41.98 .84 .84
(in %) Regression
Logistic 88 44.23 44.12 0.88 0.88
Regressi SVM 84 41.92 41.81 .84 .83
on Random 74 37.00 37.40 .75 .74
Forest
SVM 87.88 44.06 43.82 0.88 0.88
Naive 66 24.58 41.34 .75 .49

1
http://www.cs.cornell.edu/People/pabo/movie-review-data
 Bayes Fig1: Various Classifier with different dataset

Decision 60 14.30 45.93 .79 .28

Tree

KNN 50 22.95 27.13 .50 .46

In above Table4, SVM and Logistic Regression classifiers

outperforms maximum accuracy.
4.4 DATASET-IV Amazon Sports and Outdoors Review
dataset [14]. The performance of various classifiers is shown
in Table5.
TABLE5: EXPERIMENT RESULT OF SPORTS AND OUTDOORS
Algorithm Classificati TP FP Rate Precis Recall
on Rate (in %) ion
Accuracy (in%)
(in %)
Logistic 85 43.35 41.67 .85 .85
Regressi
on
SVM 84.8 42.95 41.83 .85 .85
Random 76.3 38.02 38.28 .78 .75
Forest
Naive 67 24.63 42.22 .78 .48
Bayes
Decision 62 31.37 35.90 .70 .62
Tree
KNN 50 22.80 27.55 .51 .45
In above Table5, Logistic Regression classifiers outperforms 5 CONCLUSION
maximum accuracy. In this work, the performance of various classifications with
different datasets is evaluated and compared. Feature
3.5 DATASET-V Amazon Toys and Games Review dataset selection technique Term Frequency and Inverse Document
[14]. The performance of various classifiers is shown in Frequency (TF-IDF) is used to convert the dataset into
Table6. frequency matrix for selecting the best feature. From the
TABLE6: EXPERIMENT RESULT OF TOYS AND GAMES
Algorithm Classificati TP FP Rate Prec Recal result we conclude that classifiers SVM and LR produced
on Rate (in %) ision l best classification accuracy compared with all other
Accuracy (in%) classifiers. Different feature selection techniques need to be
(in %) implemented in the future.
Logistic 88 45.02 43.11 .88 .88
Regression 6 REFERENCE
SVM 88 44.82 42.89 .88 .88 [1] Abbasi, H. Chen, and A. Salem, “Sentiment Analysis in Multiple
Random 79 40.00 39.09 .80 .78 Languages: Feature Selection for Opinion classification in Web
Forest Forums,” ACM Trans. Information Systems, vol. 26, no. 3, pp. 1-34,
(2008).
Naive 68 25.64 42.73 .80 .50
[2] Anjali Devi, S., Sapkota, P., Rohit Kumar, K., Pooja, S., Sandeep,
Bayes M.S.,” Comparison of classification algorithms on twitter data using
Decision 68 28.70 39.61 .75 .56 sentiment analysis”, International Journal of Advanced Trends in
Tree Computer Science and Engineering, Vol. 9, Issue-5, (2020), pp:8170-
8173
KNN 49 12.70 36.02 .49 .25
[3] Bahrawi,“Sentiment Analysis using Random Forest Algorithm Online
Social Media Based”, Journal of Information Technology and its
In above Table4, SVM and Logistic Regression classifiers Utilization, Vol. 2, Issue 2, (2019),29-33
[4] Pang and L. Lee,“A Sentimental Education: Sentiment Analysis Using
outperforms maximum and same accuracy. Evaluation and Subjectivity Summarization Based on Minimum Cuts,” Proc. 42th
Ann. Meeting on Assoc. for Computational Linguistics (ACL),
Comparison of different Classification algorithms are (2004), pp. 271-278.
depicted in graph on Figure 1: [5] Bo Pang and Lillian Lee,“Opinion mining and sentiment analysis”
Foundations and Trends in Information Retrieval, Vol. 2, No 1-2,
(2008), pp 1–135
[6] Bo Pang, Lillian Lee, and Shivakumar Vaithyanathan,“Thumbs up?
Sentiment Classification using Machine Learning Techniques”,
Proceedings of EMNLP, (2002).
[7] B. Liu, M. Hu, and J. Cheng,“Opinion Observer: Analyzing and
Comparing Opinions on the Web,” Proc. Int’l Conf. World Wide
Web, (2005), pp. 342-351.
[8] Bulusu, A., Sucharita, V.Research ,“Research on machine learning
techniques for POS tagging in NLP”, International Journal of Recent
Technology and Engineering,Vol. 8,,Issue 1, Special Issue- 4,(2019).
[9] Hung and H. Lin, "Using Objective Words in SentiWordNet to
Improve Word-of-Mouth Sentiment Classification," in IEEE

1
http://www.cs.cornell.edu/People/pabo/movie-review-data
 Intelligent Systems, vol. 28, no. 2, pp. 47-54, March-April (2013),
doi: 10.1109/MIS.2013.1.
[10] F. Alattar, K. Shaalan,“Survey on Opinion Reason Mining and
Interpreting Sentiment Variations”,IEEE Access, Volume 9, (2021).
[11] Karthikeyan, C., Sahaya, A.N.A., Anandan, P., Prabha, R., Mohan,
D., Vijendra, B.D,“Predicting Stock Prices Using Machine Learning
Techniques”,Proceedings of the 6th International Conference on
Inventive Computation Technologies, ICICT 2021, .(2021).
[12] KoyelChakraborty; Siddhartha Bhattacharyya; Rajib Bag,“A Survey
of Sentiment Analysis from Social Media Data”, IEEE Transactions
on Computational Social Systems, Volume: 7, Issue: 2, (2020).
[13] Morinaga, S., Yamanishi, K., Tateishi, K. and Fukushima, T.,
(2002), “Mining product reputations on the web”, in Proceeding of the
eighth ACM SIGKDD, international conference on Knowledge
discovery and data mining (pp. 341-349). ACM.
[14] Ni, J., Li, J. & McAuley, J.,“Justifying recommendations using
distantly-labeled reviews and fine-grained aspects”, In Proc. 2019
Conference on Empirical Methods in Natural Language Processing
and the 9th International Joint Conference on Natural Language
Processing (EMNLP-IJCNLP), 188–197, (2019)
[15] Qixuan Hou; Meng Han; Zhipeng Cai,” Survey on data analysis in
social media: A practical application aspect” Big Data Mining and
Analytics , Volume: 3, Issue: 4, (2020).
[16] Sajana, T., Narasingarao, M. R.,“Classification of Imbalanced
Malaria Disease Using Naïve Bayesian Algorithm”, International
Journal of Engineering & Technology,7(2.7) ,786-790,(2018)
[17] S Sakhare, N.N., Sagar Imambi,“Performance analysis of regression-
based machine learning techniques for prediction of stock market
movement" International Journal of Recent Technology and
Engineering 7 (6), 655-662, (2019)
[18] Sanjay Bhargav, P., Nagarjuna Reddy, G., Ravi Chand, R.V., Pujitha,
K., Mathur, A., “Sentiment analysis for hotel rating using machine
learning algorithms” International Journal of Innovative Technology
and Exploring Engineering, Vol. 8,Issue.6,pp 1225-1228.
[19] Shaozhong Zhang; Haidong Zhong,“Mining Users Trust from E-
Commerce Reviews Based on Sentiment Similarity Analysis”, IEEE
Access ,Volume: 7, Page(s): 13523 – 13535, (2019).
[20] Surbhi Bhatia,“A Comparative Study of Opinion Summarization
Techniques”, IEEE Transactions on Computational Social Systems,
Vol. 8, No. 1, (2021).
[21] Shyamasundar L B., Jhansi Rani P, “A Multiple-Layer Machine
Learning Architecture for Improved Accuracy in Sentiment Analysis”,
The Computer Journal , Volume: 63, Issue: 1, Jan. 2020,pp 395 – [22]
409,(2020).
[23] Vavilapalli, S.S., Reddykorepu, P., Saggam, S., Pentyala, M., Devi,
S.A,” Summarizing Sentiment Analysis on Movie Critics Data”,
Proceedings of the 6th International Conference on Inventive
Computation Technologies, ICICT 2021, (2021).
[24] Yassine Al-Amrani,Mohamed Lazaar, Kamal Eddine
lkadiri,“Sentiment Analysis using supervised classification
algorithms”, Proceedings of the 2nd international Conference on Big
Data, Cloud and Applications, Association for Computing
Machinery, Article No.: 61, Pages 1–8,(2017).
[25] You Li, Yuming Lin, Jingwei Zhang and Guoyong Cai, Constructing
Domain-Dependent Sentiment Lexicons Automatically for Sentiment
Analysis. Information Technology Journal, 12: 990-996, (2013).

1
http://www.cs.cornell.edu/People/pabo/movie-review-data