CONFUSION MATRIX

Dr. Chiradeep Mukherjee

Department of CST and CSIT
UEM Kolkata
 What is Classification?
Classification is the process of classifying data into different categories based on
some of their common characteristics.

CLASSIFIER
 Need for Confusion Matrix
Classification Models have multiple output categories. Most error measures will
tell us the total error in our model, but we cannot use it to find out individual
instances of error in our model.

TOTAL
ERROR
INPUT CLASSIFIER ERROR
MEASURE SCORE

MULTIPLE OUTPUT
LABELS/CATEGORIES
 Need for Confusion Matrix
During classification, we also have to overcome the limitations of accuracy. Accuracy can be misleading
for classification problems. If there is a significant class imbalance, a model might predict the majority
class for all cases and have a high accuracy score.

INBOX

Our model predicted 8/10

CLASSIFIER instances and will have an
SPAM accuracy of 80%. BUT IS IT
CLASSIFYING
CORRECTLY?
 What is Confusion Matrix?
• Confusion matrix is a summarised table of the number of correct and incorrect
predictions yielded by a classifier (or a classification model) for binary classification
tasks.
• A Confusion matrix is an N x N matrix used for evaluating the performance of a
classification model, where N is the number of target classes.
• The confusion matrix is a tool for predictive analysis In machine learning. In order to
check the performance of a classification based machine learning model, the
confusion matrix is deployed.

The confusion matrix is in the form

of a square matrix where the
column represents the actual values
and the row depicts the predicted
value of the model and vice versa.
 The Meaning of False Positive and False Negative
HEALTHCARE SECTOR
ACTUAL

TRUE FALSE
POSITIVE POSITIVE

PREDICTED
HEART DISEASE PREDICTION

FALSE NEGATIVE: Patients having heart

disease but classified as not having heart
disease. FALSE TRUE
NEGATIVE NEGATIVE
FALSE POSITIVE: Patients not having
heart disease but classified as having heart
disease.
 What is Confusion Matrix?

ACTUAL ACTUAL
POSITIVE NEGATIVE

PREDICTED TRUE FALSE

POSITIVE POSITIVE POSITIVE

PREDICTED FALSE TRUE

NEGATIVE NEGATIVE NEGATIVE
Confusion Matrix Example 1
Suppose we have 300 samples in DATASET.

CLASSIFY
THE INCOME
INTO
i) >50k
ii) <=50k
 Confusion Matrix Example

CLASSIFIER MODEL OUTPUT

150 0 150 10

0 150 20 120

IDEAL CASE MODEL

OUTPUT
 UNDERSTANDING TP, TN, FP, FN
CLASSIFIER MODEL OUTPUT True Positive (TP): 150 having Income >50k
and were correctly classified

True Negative (TN): 120 not having Income

>50k and were correctly classified
150 10
False Negative (FN): 20 having income >50k
but algorithm wrongly classifies as <=50k
20 150
False Positive (FP): 10 not having income >50k
but algorithm classifies as >50k
MODEL
OUTPUT
METRICS OF CONFUSION MATRIX
Accuracy: The proportion of total Recall: The measure of
number of predictions that were identifying True Positives.
correct.

F1-Score: Harmonic Mean

Precision: The proportion of of the precision and recall
correct positive predictions. of a classifier.

2XPrecisionXRecall
Precision + Recall
 CALCULATIONS FOR OUR EXAMPLE
CLASSIFIER MODEL OUTPUT
Accuracy: The proportion of total
number of predictions that were
correct.

150 10

20 120
150+120
Accuracy:
150+120+20+10
MODEL
Accuracy: = (270/300) = 0.9 OR 90% OUTPUT
 CALCULATIONS FOR OUR EXAMPLES
Precision: The proportion of CLASSIFIER MODEL OUTPUT
correct positive predictions.

150 10

150
Precision:
150+10 20 120

Precision: = 0.93 OR 93% MODEL

OUTPUT
 CALCULATIONS FOR OUR EXAMPLES
Recall: The measure of CLASSIFIER MODEL OUTPUT
identifying True Positives.

150 10

150
Recall:
150+20 20 120

Recall: = 0.88 OR 88% MODEL

OUTPUT
 CALCULATIONS FOR OUR EXAMPLES
F1-Score: Harmonic Mean CLASSIFIER MODEL OUTPUT
of the precision and recall
of a classifier.

2XPrecisionXRecall
Precision + Recall 150 10

2*0.93*0.88
F1-Score:
0.93+0.88 20 120

F1-Score: = 0.90 OR 90% MODEL

OUTPUT
 Confusion Matrix Example 2

GENTOO ADELIE CHINSTRAP

ADELIE CHINSTRAP GENTOO

ADELIE 39 0 2
FIND THE
ACCURACY,
CHINSTRAP 12 10 1
PRECISION, RECALL
AND F1 SCORE.
GENTOO 6 1 29
 Confusion Matrix Example 2

ADELIE

TP = 39 (Cell 1)
ADELIE CHINSTRAP GENTOO

FN = 0+2 = 2 0 2
ADELIE 39
(Cell 2 + Cell 3)

FP = 12+6 (Cell 4 + Cell 7) CHINSTRAP 12 10 1

TN = 10+1+1+29= 41 (Cell 5
+ Cell6 + Cell 8 + Cell9 ) GENTOO 6 1 29
 Confusion Matrix Example 2
Accuracy: The proportion of total
number of predictions that were
correct.
ADELIE CHINSTRAP GENTOO

ADELIE 39 0 2

39+41 10
Accuracy: CHINSTRAP 12 1
39+18+41+2

Accuracy: = (80/100) = 0.8 OR GENTOO 6 1 29

80%
 CALCULATIONS FOR OUR EXAMPLES
Precision: The proportion of
correct positive predictions.

ADELIE CHINSTRAP GENTOO

ADELIE 39 0 2
39
Precision:
39+18
CHINSTRAP 12 10 1

Precision : = 0.684 OR 68.4%

GENTOO 6 1 29
 CALCULATIONS FOR OUR EXAMPLES
Recall: The measure of
identifying True Positives.

ADELIE CHINSTRAP GENTOO

ADELIE 39 0 2
39
Recall:
39+2
CHINSTRAP 12 10 1

Recall: = 0.951 OR 95.1%

GENTOO 6 1 29
 KEY POINTS FOR CONFUSION MATRIX
Evaluation : Provides a holistic view of the classification model.

Imbalanced Data Set: Confusion Matrix good evaluation matrix for

imbalanced dataset.

Precision and Recall: Metrics to identify the correct predictions.

Binary and Multiclass Classification: Confusion Matrix is suitable

for both.