ABSTRACT

Automated Teller Machines are widely used nowadays by people. But It’s hard to
carry their ATM card everywhere, people may forget to have their ATM card or
forget their PIN number. The ATM card may get damaged and users
can have a situation where they can’t get access to their money. In our proposal,
use of biometrics for authentication instead of PIN and ATM card is encouraged.
Here, The Face ID is preferred to high priority, as the combination of these
biometrics proved to be the best among the identification and verification
techniques. The implementation of ATM machines comes with the issue of being
accessed by illegitimate users with valid authentication code. This project
provides service to the user only when the user is legitimate or the user is verified
by the legitimate user of the ATM card. The users are verified by comparing the
image taken in front of the ATM machine, to the images which are present in the
database. If the user is legitimate the new image is used to train the model for
further accuracy. This system uses openCV to process the image being obtained
and Haar Cascade Classifier to detect the faces in the image. The face recognition
is done using Local Binary Pattern.

2
 Chapter 1

Introduction

An Automatic Teller Machine (ATM) is a computerized machine that is

used to withdraw cash from a customer’s respective bank account. As
financial users prefer ATM for cash withdrawals, cash deposits and many
other transaction, the banks are focusing a lot over the security of
ATMs.Hence ATM should be pro- tected properly from the criminal
activities or from any unwanted things.

Due to rapid development in science and technology, upcoming

innovations are being built-up with strong security. But on the other hand,
threats are also being posed to destroy this security level. Though
enhancement in automation has made a positive impact overall, various
financial institutions like banks and applications like ATM are still
subjected to thefts and frauds. The existing ATM model uses a card and a
PIN which gives rise to increase in attacks in the form of stolen cards, or
due to statically assigned PINs, duplicity of cards and vari- ous other
threats.Then another major problem is hacking of PIN. There are other
fraudulent attacks like eavesdropping, spoofing , brute force attacks,
blackmail- ing the user. In the worst case there can also be ATM machine
Robbery.

2
 To overcome these problems,the project ’ATM Security system based on
Face recognition, PIN and OTP’ consists of conventional features ie is
Personal Iden- tification Number (PIN) along with additional features like
face recognition and one-time password (OTP) is used. Database holds
information about a user’s ac- count details, images of his/her face and a
mobile number which will improve security to a large extent.

First, the user will come to the ATM machine and a live image is
captured through the Web Camera interfaced with System defining as the
ATM sys- tem, which is compared with the images stored in the database. If
the face is recognised, then the user is notified to type the PIN. If the PIN
matches, an OTP will be sent to the corresponding registered mobile
number. If the user correctly enters the OTP, the transaction can proceed.
Therefore, the combination of face recognition algorithm,PIN and an OTP
drastically reduces the chances of fraud. Inorder to obtain better accuracy
deep learning based linear discrimint classifica- tion method is utilized. And
excecuted the same in OS.

2
 Chapter 2

Literature Survey

Mohsin Karovaliya in paper [2] proposes Eigenface based method for

the face recognition. The model shows the qualitative analysis of algorithms
used based on the metrics of existing algorithms. According to the statistics,
PCA based face recognition is very accurate, requires less computation time
and less storage space as trainee images are stored in the form of their
projections on a reduced basis.The drawback of using Eigenface based
method is that it can sometimes be spoofed by the means of fake masks or
photos of an account holder. To overcome this prob- lem 3D face
recognition methods can be used. However, its computation cost is high.

The paper [3] suggests a vibration sensor which senses vibrations

produced from ATM machine whenever robbery occurs. This system uses
ARM controller based embedded system to process real time data collected
using the vibration sensor. Once the vibration is sensed the beep sound will
occur from the buzzer.

DC Motor is used for closing the door of ATM. Some other additional
security measures are used. This will prevent the robbery and the person

4
involving in robbery can be easily caught. Software implementation is
deployed using two software packages, first one is the Keil Vision 3.0.
Second one is the Flash magic simulator. Keil Vision Debugger accurately
simulates on-chip peripherals. This system helps in rapid reaction and
minimization of loss by detecting the ATM machine at real-time when it has
been stolen can be found through GSM technol- ogy.

In paper [4], the finger print recognition is done with curvelet transform
by finding the Euclidean distance between the two corresponding finger
codes. The test finger code is compared with the entire finger codes in the
database. If it
matches, an OTP will be send to the corresponding registered mobile
number.Preprocessing is done by using built in mathlab function imread. Histogram
equalization method
helps in improving the global contrast of an image by moving slightly the
inten- sity distribution on a histogram. This permits areas of lower local
contrast to gain a higher contrast without affecting the global contrast.
Histogram equalization fulfill this by effectively spreading out the most
frequent intensity values. The feature extraction can be done using curvelet
transform and by using FFT.

The ATM security is enhanced by adding GSM module to generate OTP

in pa- per [5]. When there is a network problem in GSM technology, instead
of GSM here implements Bluetooth connection with ATM, Which generates
OTP refer- ence through user mobile. GSM modems can be a quick and
5
efficient way to get

started with SMS, because a special subscription to an SMS service

provider is not required. A GSM modem could also be a standard GSM
mobile phone with the appropriate cable and software driver to connect to a
serial port or USB port

The paper [6] suggests fisher faces algorithm for face recognition.
Compared to globality based supervised dimensionality reduction methods
such as Fisher Discriminant Analysis (FDA), locality based ones including
Local Fisher Discrim- inant Analysis (LFDA) have attracted increasing
interests since they aim to pre- serve the intrinsic data structures and are
able to handle multimodally distributed data. However, both FDA and
LFDA are usually solved via a ratio trace form to approximate the trace
ratio, which is the Fisher’s original objective criterion.

In paper, [7] Deep Learning method is introduced with as a part of

learning based strategy to provide a complete analysis about the face
samples present in the system. The visible general problems in face
recognition are fraudulent faces and the factors affecting recognition
accuracy such as noise, diversions in the an- gle, poses and expression.
These problems are the main cause for system to lose its perfection, here it
improves the accuracy of recognition by keeping the track of history
information about the faces arriving as an input. The experimental results

6
 acquired on YALE and ORL database shows that this is an efficient method.

One Touch Multi-banking Transaction ATM System using Biometric and

GSM Authentication is implemented in paper [8] The account details of
user are stored on cloud in a centralized manner. The account details of all
the bank accounts of

account holder are displayed. User needs to select one of the bank accounts for the
transaction. If user wants to transfer the money or debit the money from ac- count
he will get OTP on registered mobile number. The GSM module generates OTP
for enforce authenticate transaction from the bank side.

7
 Chapter 3

Design Phase

The system contains System board as the main processor. The system is
used to contain the dedicated operating system which is compatible with the
System board. The Human face is captured by the Web Camera which can
be directly interfaced with the System board. The monitor displays the mes-
sages for user interface.

3.1 System Flow

8
9
3.2 System Overview

3.2.1 Face Detection and Face

Recognition Face Detection

The algorithm needs a lot of positive images (images of faces) and negative
im- ages (images without faces) to train the classifier. Then it is required to
extract features from it. Features are nothing but numerical information
extracted from the images that can be used to distinguish one image from
another; for example, histogram (distribution of intensity values) is one of
the features that can be used to define several characteristics of an image
even without looking at the image, such as dark or bright image, the
intensity range of the image, contrast, and so on. Using Haar features is a
efficient method for face detection. These features are just like the
convolution kernel. The convolution can be summarized by locating a
Webxel from the image, then crop out a sub-image with the selected
Webxel as the center from the source image with the same size as the
convolution kernel. Cal- culate an element-wise product between the values
of the kernel and sub- image. Add the result of the product. Put the resultant
value into the new image at the same place where you Webcked up the
Webxel location.
Each feature is a single value obtained by subtracting the sum of the
Webxels under the white rectangle from the sum of the Webxels under the
black rectangle. Now, all possible sizes and locations of each kernel are
used to calculate plenty of features. Each feature calculation, requires to
find the sum of the Webxels under the white and black rectangles. The
concept of integral image is very useful to solve this. Integral images are
those images in which the Webxel value at any (x,y) location is the sum of
the all Webxel values present before the current Webxel.

Figure 3.3: Haar cascade[1]

The first feature selected seems to focus on the property that the region of
the eyes is often darker than the region of the nose and cheeks. The second
feature selected relies on the property that the eyes are darker than the
bridge of the nose.
Face Recognition

Face recognition is an easy task for humans. Face recognition based on the
geo- metric features of face is probably the most intuitive approach to face
recognition. One of the first automated face recognition systems was marker
points (position of eyes, ears, nose etc.) were used to build a feature vector
(distance between the points, angle between them etc). The recognition was
performed by calculating the euclidean distance between feature vectors of
a probe and reference image. Some of the latest work on geometric face
recognition was, a 22-dimensional fea- ture vector and experiments on large
datasets have shown that geometrical fea- tures alone may not carry enough
information for face recognition.

I. Method one based on machine learning

Machine learning (ML) is the scientific study of algorithms and statistical

models that computer systems use to effectively perform a specific task
without using explicit instructions, relying on patterns and inference
instead. It is seen as a sub- set of artificial intelligence. Machine learning
algorithms build a mathematical model based on sample data, known as
"training data", in order to make pre- dictions or decisions without being
explicitly programmed to perform the task. Machine learning algorithms
are used in a wide variety of applications, such as email filtering, and
computer vision, where it is infeasible to develop an algorithm of specific
instructions for performing the task
Machine learning uses types of automated algorithms which learn to predict
fu- ture decisions and model and model functions using data fed to it.

The Eigenfaces and Fisherfaces methods took a holistic approach to face

recog- nition. Recently various methods for a local feature extraction
emerged. To avoid the high-dimensionality of the input data only local
regions of an image are de- scribed, the extracted features are hopefully
more robust against partial occlusion, illumination and small sample size.
Algorithms used for a local feature extraction are Gabor Wavelets , Discrete
Cosinus Transform and Local Binary Patterns.
Mainly there are three easy steps to computer coding facial recognition,
which are similar to the steps that human brain use for recognizing faces.
These steps are:

(1) Data Gathering: Gather face data (face images in this case) of the
people you want to identify.
(2) Train the Recognizer: Feed that face data and respective names of each
face to the recognizer so that it can learn.

(3) Recognition: Feed new faces of that people and see if the face
recognizer, just trained before, recognizes them.

OpenCV has two built-in face recognizers. The names of those face
recognizers are: EigenFaces and FisherFaces.

(i) EigenFaces Face Recognizer Algorithm

In this algorithm, a facial image is a point from a high-dimensional image

space and a lower-dimensional representation is found, where classification
becomes easy. The lower-dimensional subspace is found with Principal
Component Anal- ysis(PCA), which identifies the axes with maximum
variance. While this kind of transformation is optimal from a
reconstruction standpoint, it doesn’t take any class labels into account.
Imagine a situation where the variance is generated from external sources,
let it be light. The axes with maximum variance do not necessar- ily contain
any discriminative information at all, hence a classification becomes
impossible. So a class-specific projection with a Linear Discriminant
Analysis was applied to face recognition. The basic idea is to minimize the
variance within a class, while maximizing the variance between the classes
at the same time.

This algorithm considers the fact that not all parts of a face are equally
im- portant or useful for face recognition. Indeed, when you look at
someone, you recognize that person by his distinct features, like the eyes,
nose, cheeks or fore- head; and how they vary respect to each other. Focus
is on the areas of maximum change. For example, from the eyes to the nose
there is a significant change, and same applies from the nose to the mouth.
When multiple faces are given, compar- ison is done by looking at these
areas, because by catching the maximum variation among faces, they help
to differentiate one face from the other. This is how EigenFaces recognizer
works. It looks at all the training images of all the people as a whole and
tries to extract the components which are relevant and useful and discards
the rest. These important features are called principal components.

So, EigenFaces recognizer trains itself by extracting principal

components, but it also keeps a record of which ones belong to which
person. Thus, whenever a new image is introduced to the algorithm, it
repeats the same process as follows: Extract the principal components from
the new Webcture. Compare those features with the list of elements stored
during training. Find the ones with the best match. Return the ‘person’ label
associated with that best match component. In simple words, it’s a game of
matching.However, one thing to note in above image is that EigenFaces
algorithm also considers illumination as an important feature. In
consequence, lights and shadows are Webcked up by EigenFaces, which
classifies them as representing a ‘face’.Face recognition Webcks up on
human things, dominated by shapes and shadows: two eyes, a nose, a
mouth.

(ii) FisherFaces Face Recognizer Algorithm

This algorithm is an improved version of the Eigenfaces. Eigenfaces looks

at all the training faces of all the people at once and finds principal
components from all of them combined. By doing that, it doesn’t focus on
the features that dis- criminate one individual from another. Instead, it
concentrates on the ones that represent all the faces of all the people in the
training data, as a whole. Since EigenFaces also finds illumination as a
useful component, it will find this varia- tion very relevant for face
recognition and may discard the features of the other people’s faces,
considering them less useful. In the end, the variance that Eigen- Faces has
extracted represents just one individual’s facial features. This can be done
by tunning EigenFaces so that it extracts useful features from the faces of
each person separately instead of extracting them from all the faces
combined. In this way, even if one person has high illumination changes, it
will not affect the other people’s features extraction process.

The Principal Component Analysis (PCA), which is the core of the

Eigenfaces method, finds a linear combination of features that maximizes
the total variance in data. While this is clearly a powerful way to represent
data, it doesn’t consider any classes and so a lot of discriminative
information may be lost when throwing components away.

The Linear Discriminant Analysis performs a class-specific

dimensionality re- duction. In order to find the combination of features that
separates best between classes the Linear Discriminant Analysis maximizes
the ratio of between-classes to within-classes scatter, instead of maximizing
the overall scatter. The idea is simple: same classes should cluster tightly
together, while different classes are as far away as possible from each other
in the lower-dimensional representation.

Precisely, FisherFaces face recognizer algorithm extracts principal

components that differentiate one person from the others. In that sense, an
individual’s com- ponents do not dominate (become more useful) over the
 others.Below is an image of principal components using FisherFaces
algorithm.

One thing to note here is that FisherFaces only prevents features of one
person from becoming dominant, but it still considers illumination changes
as a useful feature. But light variation is not a useful feature to extract as it
is not part of the actual face, another face recognizer Algorithm must be
used.

Machine Learning yielded about 75% accuracy for face recognition.

Moreover the model was very sensitive to lighting conditions.
Hence a new method was designed for face recognition and that is deep
learning. This model gives an accuracy upto 95% .
Deep learning (also known as deep structured learning or hierarchical
learning) is part of a broader family of machine learning methods based on
artificial neural networks.
In the case of machine learning, the algorithm needs to be told how to make
an accurate prediction by providing it with more information, whereas, in
the case of deep learning, the algorithm is able to learn that through its own
data processing. It is similar to how a human being would identify
something, think about it, and then draw any kind of conclusion.Deep
learning interprets data features and its relationships using neural networks
which pass the relevant information through several stages of data
processing.
The key here is to get a deep Convolutional Neural Network(CNN) to
produce a bunch of numbers that describe a face (known as face encodings).
When two dif- ferent images of the same person are passed to the network,
the network should return similar outputs (i.e. closer numbers) for both
images, whereas when im- ages of two different people are passed, the
network should return very differ- ent outputs for the two images. This
means that the neural network needs to be trained to automatically identify
different features of faces and calculate numbers based on that.

3.2.2 WEBN Matching

A Personal Identification Number (WEBN), is a numeric or alpha-

numeric pass- word used in the process of authenticating a user accessing a
system.
The WEBN used is a 4-digit number. The details of user along with their
WEBNs are already stored in the database. Once the user enters the WEBN,
it is checked with WEBN in the database. If the WEBN matches with the
user face and details, then sys- tem proceeds to next step. The user has three
chances of entering WEBN. If WEBN is not matched even the third time,
then the account will be temporarily blocked to ensure safety to the real user
indicating that the person who arrived at the ATM is probably a robber.
 CHAPTER 4
SOFTWARE DESIGN

Table of Contents
1.OpenCV-Python
 Overview
 Installation
2. Images as Arrays
 Binary Image
 Grayscale Image
 Colored Image
3. Images and OpenCV
 Importing Images in OpenCV
 Savings images
 Basic Operations on Images
4. Face Detection
 Overview
 Haar feature-based cascade classifiers
 Face Detection with OpenCV-Python
OpenCV-Python
Overview

OpenCV was started at Intel in the year 1999 by Gary Bradsky. The first release came
a little later in the year 2000. OpenCV essentially stands for Open Source Computer
Vision Library.Although it is written in optimized C/C++, it has interfaces for
Python and Java along with C++. OpenCV boasts of an active user base all over the
world with its use increasing day by day due to the surge in computer vision
applications.OpenCV-Python is the python API for OpenCV. You can think of it as a
python wrapper around the C++ implementation of OpenCV. OpenCV-Python is not
only fast (since the background consists of code written in C/C++) but is also easy to
code and deploy(due to the Python wrapper in foreground). This makes it a great
choice to perform computationally intensive programs.

Installation
OpenCV-Python supports all the leading platforms like Mac OS, Linux, and
Windows. It can be installed in either of the following ways:
Unofficial pre-built OpenCV packages for Python.
Packages for standard desktop environments (Windows, macOS, almost any
GNU/Linux distribution) run pip install opencv-python if you need only main
modules run pip install opencv-contrib-python if you need both main and contrib
module (check extra modules listing from OpenCV documentation)
You can either use Jupyter notebooks or any Python IDE of your choice for writing
the scripts. Images as Arrays
An image is nothing but a standard Numpy array containing pixels of data points.
More the number of pixels in an image, the better is its resolution. You can think of
pixels to be tiny blocks of information arranged in the form of a 2 D grid, and the
depth of a pixel refers to the color information present in it. In order to be processed
by a computer, an image needs to be converted into a binary form. The color of an
image can be calculated as follows:
Number of colors/ shades = 2^bpp where bpp represents bits per pixel.
Naturally, more the number of bits/pixels, more possible colors in the images. The
following table shows the relationship more clearly.

Let us now have a look at the representation of the different kinds of images:
Binary Image
A binary image consists of 1 bit/pixel and so can have only two possible colors, i.e.,
black or white. Black is represented by the value 0 while 1 represents white.
 Fig 11 Binary image
Grayscale image
A grayscale image consists of 8 bits per pixel. This means it can have 256 different
shades where 0 pixels will represent black color while 255 denotes white. For
example, the image below shows a grayscale image represented in the form of an
array. A grayscale image has only 1 channel where the channel represents dimension

Fig 12 Grayscale image

Colored image
Colored images are represented as a combination of Red, Blue, and Green, and all
the other colors can be achieved by mixing these primary colors in correct
proportions.

Fig 13 Colored image

A colored image also consists of 8 bits per pixel. As a result, 256 different shades of
colors can be represented with 0 denoting black and 255 white. Let us look at the
famous colored image of a mandrill which has been cited in many image processing
examples

If we were to check the shape of the image above, we would get:

Shape
(288, 288, 3)
288: Pixel width
288: Pixel height
3: color channel
This means we can represent the above image in the form of a three-dimensional
array.

Images and OpenCV

Before we jump into the process of face detection, let us learn some basics about
working with OpenCV. In this section we will perform simple operations on images
using OpenCV like opening images, drawing simple shapes on images and
interacting with images through callbacks. This is necessary to create a foundation
before we move towards the advanced stuff.
Importing Images in OpenCV
Using Jupyter notebooks
Steps:
Import the necessary libraries
import numpy as np
import cv2
import matplotlib.pyplot as plt
%matplotlib inline
Read in the image using the imread function. We will be using the colored 'mandrill'
image for demonstration purpose.
img_raw = cv2.imread('image.jpg')
The type and shape of the array.
type(img_raw)
numpy.ndarray
img_raw.shape
(1300, 1950, 3)
Thus, the .png image gets transformed into a numpy array with a shape of 1300x1950
and has 3 channels.
#viewing the image
plt.imshow(img_raw)
What we get as an output is a bit different concerning color. We expected a bright
colored image but what we obtain is an image with some bluish tinge. That happens
because OpenCV and matplotlib have different orders of primary colors. Whereas
OpenCV reads images in the form of BGR, matplotlib, on the other hand, follows the
order of RGB. Thus, when we read a file through OpenCV, we read it as if it contains
channels in the order of blue, green and red.
However, when we display the image using matplotlib, the red and blue channel
gets swapped and hence the blue tinge. To avoid this issue, we will transform the
channel to how matplotlib expects it to be using the cvtColor function.
img = cv2.cvtColor(img_raw, cv2.COLOR_BGR2RGB)
plt.imshow(img_rgb)

Using Python Scripts

Jupyter Notebooks are great for learning, but when dealing with complex images
and videos, we need to display them in their own separate windows. In this section,
we will be executing the code as a .py file. You can use Pycharm, Sublime or any IDE
of your choice to run the script below.
import cv2
img = cv2.imread('image.jpg')
while True:
cv2.imshow('mandrill',img)
if cv2.waitKey(1) & 0xFF == 27:
break
cv2.destroyAllWindows()
In this code, we have a condition, and the image will only be shown if the condition
is true. Also, to break the loop, we will have two conditions to fulfill:
The cv2.waitKey() is a keyboard binding function. Its argument is the time in
milliseconds. The function waits for specified milliseconds for any keyboard event. If
you press any key in that time, the program continues.The second condition pertains
to the pressing of the Escape key on the keyboard. Thus, if 1
millisecond has passed and the escape key is pressed, the loop will break and
program stops. cv2.destroyAllWindows() simply destroys all the windows we
created. If you want to destroy any specific window, use the function
cv2.destroyWindow() where you pass the exact window name as the argument.
5.1.3.2 Savings images
The images can be saved in the working directory as follows:
cv2.imwrite('final_image.png',img)
Where the final_image is the name of the image to be saved.

5.1.3.3 Basic Operations on Images

In this section, we will learn how we can draw various shapes on an existing image
to get a flavor of working with OpenCV.
Drawing on Images
Begin by importing necessary libraries.
import numpy as np
import matplotlib.pyplot as plt
%matplotlib inline
import cv2
Create a black image which will act as a template.
image_blank = np.zeros(shape=(512,512,3),dtype=np.int16)
Display the black image.
plt.imshow(image_blank)

Function & Attributes

The generalised function for drawing shapes on images is:
cv2.shape(line, rectangle etc)(image,Pt1,Pt2,color,thickness)
There are some common arguments which are passed in function to draw shapes on
images:
Image on which shapes are to be drawn co-ordinates of the shape to be drawn from
Pt1(top left) to Pt2(bottom right)
Color: The color of the shape that is to be drawn. It is passed as a tuple, eg: (255,0,0) .
For grayscale, it will be the scale of brightness.The thickness of the geometrical
figure.
1. Straight Line
Drawing a straight line across an image requires specifying the points, through
which the line will pass.
# Draw a diagonal red line with thickness of 5 px
line_red = cv2.line(img,(0,0),(511,511),(255,0,0),5)
plt.imshow(line_red)

# Draw a diagonal green line with thickness of 5 px

line_green = cv2.line(img,(0,0),(511,511),(0,255,0),5)
plt.imshow(line_green)

2. Rectangle
For a rectangle, we need to specify the top left and the bottom right coordinates.
#Draw a blue rectangle with a thickness of 5 px
rectangle= cv2.rectangle(img,(384,0),(510,128),(0,0,255),5)
plt.imshow(rectangle)
3. Circle
For a circle, we need to pass its center coordinates and radius value. Let us draw a
circle inside the rectangle drawn above

img = cv2.circle(img,(447,63), 63, (0,0,255), -1)

# -1 corresponds to a filled circle
plt.imshow(circle)
Writing on Images
Adding text to images is also similar to drawing shapes on them. But you need to
specify certain arguments before doing so:
Text to be written coordinates of the text. The text on an image begins from the
bottom left direction.
Font type and scale. Other attributes like color, thickness and line type. Normally the
line type that is used is
lineType = cv2.LINE_AA
font = cv2.FONT_HERSHEY_SIMPLEX
text=cv2.putText(img,'OpenCV',(10,500),font,4,(255,255,255),2,cv2.LINE_AA)
plt.imshow(text)
These were the minor operations that can be done on images using OpenCV. Feel
free to experiment with the shapes and text.
Face Detection
Overview
Face detection is a technique that identifies or locates human faces in digital images.
A typical example of face detection occurs when we take photographs through our
smartphones, and it instantly detects faces in the picture. Face detection is different
from Face recognition. Face detection detects merely the presence of faces in an
image while facial recognition involves identifying whose face it is. In this article, we
shall only be dealing with the former.
Face detection is performed by using classifiers. A classifier is essentially an
algorithm that decides whether a given image is positive(face) or negative(not a
face). A classifier needs to be trained on thousands of images with and without faces.
Fortunately, OpenCV already has two pre-trained face detection classifiers, which
can readily be used in a program.
The two classifiers are:
 Haar Classifier and
 Local Binary Pattern(LBP) classifier.
In this article, however, we will only discuss the Haar Classifier.
5.1.4.2 Haar feature-based cascade classifiers
Haar-like features are digital image features used in object recognition. They owe
their name to their intuitive similarity with Haar wavelets and were used in the first
real-time face detector. Paul Viola and Michael Jones in their paper titled "Rapid
Object Detection using a Boosted Cascade of Simple Features" used the idea of Haar-
feature classifier based on the Haar wavelets. This classifier is widely used for tasks
like face detection in computer vision industry.Haar cascade classifier employs a
machine learning approach for visual object detection which is capable of processing
images extremely rapidly and achieving high detection rates.
This can be attributed to three main reasons:
 Haar classifier employs 'Integral Image' concept which allows the features used
by the detector to be computed very quickly.
 The learning algorithm is based on AdaBoost. It selects a small number of
important features from a large set and gives highly efficient classifiers.
 More complex classifiers are combined to form a 'cascade' which discard any
non-face regions in an image, thereby spending more computation on
promising object-like regions.
Let us now try and understand how the algorithm works on images in steps:
1. 'Haar features' extraction
After the tremendous amount of training data (in the form of images) is fed into the
system, the classifier begins by extracting Haar features from each image. Haar
Features are kind of convolution kernels which primarily detect whether a suitable
feature is present on an image or not. Some examples of Haar features are mentioned
below:

These Haar Features are like windows and are placed upon images to compute a
single feature. The feature is essentially a single value obtained by subtracting the
sum of the pixels under the white region and that under the black. The process can
be easily visualized in the example below.
For demonstration purpose, let's say we are only extracting two features, hence we
have only two windows here.
 The first feature relies on the point that the eye region is darker than the
adjacent cheeks and nose region.
 The second feature focuses on the fact that eyes are kind of darker as
compared to the bridge of the nose.
Thus, when the feature window moves over the eyes, it will calculate a single value.
This value will then be compared to some threshold and if it passes that it will
conclude that there is an edge here or some positive feature.
2. 'Integral Images' concept
The algorithm proposed by Viola Jones uses a 24X24 base window size, and that
would result in more than 180,000 features being calculated in this window. Imagine
calculating the pixel difference for all the features? The solution devised for this
computationally intensive process is to go for the Integral Image concept. The
integral image means that to find the sum of all pixels under any rectangle, we
simply need the four corner values.
This means, to calculate the sum of pixels in any feature window, we do not need to
sum them up individually. All we need is to calculate the integral image using the 4
corner values.
The example below will make the process transparent.
3. 'Adaboost' : to improve classifier accuracy
As pointed out above, more than 180,000 features values result within a 24X24
window.However, not all features are useful for identifying a face. To only select the
best feature out of the entire chunk, a machine learning algorithm called Adaboost is
used. What it essentially does is that it selects only those features that help to
improve the classifier accuracy. It does so by constructing a strong classifier which is
a linear combination of a number of weak classifiers. This reduces the amount of
features drastically to around 6000 from around
180,000.
4. Using 'Cascade of Classifiers'
Another way by which Viola Jones ensured that the algorithm performs fast is by
employing a cascade of classifiers. The cascade classifier essentially consists of stages
where each stage consists of a strong classifier. This is beneficial since it eliminates
the need to apply all features at once on a window. Rather, it groups the features into
separate sub-windows and the classifier at each stage determines whether or not the
sub-window is a face. In case it is not, the sub-window is discarded along with the
features in that window. If the sub-window
moves past the classifier, it continues to the next stage where the second stage of
features is applied. The process can be understood with the help of the diagram
below.
The Paul- Viola algorithm can be visualized as follows:
Face Detection with OpenCV-Python
Now we have a fair idea about the intuition and the process behind Face recognition.
Let us now use OpenCV library to detect faces in an image.
Load the necessary Libraries
import numpy as np
import cv2
import matplotlib.pyplot as plt
%matplotlib inline
Loading the image to be tested in grayscale
We shall be using the image below:

#Loading the image to be tested

test_image = cv2.imread('data/baby1.jpg')
#Converting to grayscale
test_image_gray = cv2.cvtColor(test_image, cv2.COLOR_BGR2GRAY)
# Displaying the grayscale image
plt.imshow(test_image_gray, cmap='gray')

Since we know that OpenCV loads an image in BGR format, so we need to convert it
into R
Since we know that OpenCV loads an image in BGR format, so we need to convert it
into RBG
format to be able to display its true colors. Let us write a small function for that.
def convertToRGB(image):
return cv2.cvtColor(image, cv2.COLOR_BGR2RGB)
Haar cascade files
OpenCV comes with a lot of pre-trained classifiers. For instance, there are classifiers
for smile, eyes, face, etc. These come in the form of xml files and are located in the
opencv/data/haarcascades/ folder. However, to make things simple, you can also
access them from here. Download the xml files and place them in the data folder in
the same working directory as the jupyter notebook.
Loading the classifier for frontal face
Face detection
We shall be using the detectMultiscale module of the classifier. This function will
return a rectangle with coordinates(x,y,w,h) around the detected face. This function
has two important parameters which have to be tuned according to the data.
haar_cascade_face=cv2.CascadeClassifier('data/haarcascade/haarcascade_frontalfac
e_descalefactor
In a group photo, there may be some faces which are near the camera than
others. Naturally, such faces would appear more prominent than the ones behind.
This factor compensates for that min Neighbors This parameter specifies the number
of neighbors a rectangle should have to be called a face.
Our next step is to loop over all the coordinates it returned and draw rectangles
around them using Open CV. We will be drawing a green rectangle with a thickness
of 2
for (x,y,w,h) in faces_rects:
cv2.rectangle(test_image, (x, y), (x+w, y+h), (0, 255, 0), 2)
Finally, we shall display the original image in colored to see if the face has been
detected correctly or not.
#convert image to RGB and show image
plt.imshow(convertToRGB(test_image))
faces_rects = haar_cascade_face.detectMultiScale(test_image_gray, scaleFactor = 1.2,
mi
# Let us print the no. of faces found
print('Faces found: ', len(faces_rects))
Faces found: 1

Here, it is. We have successfully detected the face of the baby in the picture. Let us
now create a generalized function for the entire face detection process.
Face Detection with generalized function
Testing the function on new image
This time test image is as follows:
def detect_faces(cascade, test_image, scaleFactor = 1.1):
# create a copy of the image to prevent any changes to the original one.
image_copy = test_image.copy()
#convert the test image to gray scale as opencv face detector expects gray images
gray_image = cv2.cvtColor(image_copy, cv2.COLOR_BGR2GRAY)
# Applying the haar classi_er to detect faces
faces_rect = cascade.detectMultiScale(gray_image, scaleFactor=scaleFactor,
minNeigh
for (x, y, w, h) in faces_rect:
cv2.rectangle(image_copy, (x, y), (x+w, y+h), (0, 255, 0), 15)
return image_copy
TESTING THE FUNCTION ON NEW IMAGE:

#loading image
test_image2 = cv2.imread('baby2.jpg')
# Converting to grayscale
test_image_gray = cv2.cvtColor(test_image, cv2.COLOR_BGR2GRAY)
# Displaying grayscale image
plt.imshow(test_image_gray, cmap='gray')
#call the function to detect faces
faces = detect_faces(haar_face_cascade, test_image2)
#convert to RGB and display image
plt.imshow(convertToRGB(faces))

Testing the function on a group image

Let us now see if the function works well on a group photograph or not. We shall be
using the picture below for our purpose.
Image: The Indian Women's Cricket Team.
 Chapter 5

Implementation Phase

4.1.1 Ubuntu

Ubuntu is a Debian-based computer operating system for System. Since

2015, it has been officially provided by the System Foundation as the
primary operating system for the family of System single-board computers.

The operating system is still under active development. Ubuntu is highly

optimized for the System line’s low-performance ARM CPUs. Ubuntu uses
WEBXEL(Web Improved X-Window Environment, Lightweight), as its
main desktop environment as of the latest update. It is composed of a
modified LXDE desktop environment and the Open box stacking window
manager with a new theme and few other changes. The distribution is
shipped with a copy of computer algebra program Mathematica and a
version of Minecraft called Minecraft Web as well as a lightweight version
of Chromium as of the latest version.

4.1.2 OpenCV
 Open Source Computer Vision Library is released under a BSD license
and hence it’s free for both academic and commercial use. It has C++, C,
Python and Java interfaces and supports Windows, Linux, Mac OS, iOS
and Android. OpenCV was designed for computational efficiency and with
a strong focus on real-time applications. Written in optimized C/C++, the
library can take advan- tage of multi-core processing. Enabled with
OpenCL, it can take advantage of the hardware acceleration of the
underlying heterogeneous compute platform.

One of OpenCV’s goal is to provide a simple-to-use computer vision

infrastruc- ture that helps people build fairly sophisticated vision
applications quickly. The OpenCV library contains over 500 functions that
span many areas in vision, in- cluding factory product inspection, medical
imaging, security, user interface, cam- era calibration, stereo vision, and
robotics. Because computer vision and machin learning often go hand-in-
hand, OpenCV also contains a full, general purpose Machine learning
Library (MLL). This sublibrary is focused on statistical pattern recognition
and clustering. The MLL is highly useful for the vision tasks that are at the
core of OpenCV’s mission, but it is general enough to be used for any ma-
chine learning problem.
Version used: OpenCV 4.0.0

4.1.3 Python
 Python is an interpreted, object-oriented, high-level programming
language with dynamic semantics. Its high-level built in data structures,
combined with dynamic tyWebng and dynamic binding, make it very
attractive for RaWebd Applica- tion Development, as well as for use as a
scripting or glue language to connect existing components together.
Python’s simple, easy to learn syntax emphasizes readability and therefore
reduces the cost of program maintenance. Python sup- ports modules and
packages, which encourages program modularity and code reuse. The
Python interpreter and the extensive standard library are available in source
or binary form without charge for all major platforms, and can be freely
distributed.
Used Python for Face Detection codes in
openCV. Version used: Python 3.7.2

Deep learning was implemented using additional softwares libraries

TensorFlow,Keras,OpenCV,Matplotlib and scikitlearn installed in it.

4.1.4 Keras

Keras is an open-source neural-network library written in Python. It is ca-

pable of running on top of TensorFlow, Microsoft Cognitive Toolkit,
Theano, or PlaidML. Designed to enable fast experimentation with deep
neural networks, it focuses on being user-friendly, modular, and
extensible.Keras helps in removing complexities of tensorflow. Keras is run
with tensorflow in the backend.

4.1.5 Matplotlib

Matplotlib is a plotting library for the Python programming language and

its numerical mathematics extension NumPy. It provides an object-oriented
AWEB for embedding plots into applications using general-purpose GUI
toolkits like Tkin- ter, wxPython, Qt, or GTK+.

4.1.6 Scikit-learn

Scikit-learn (formerly scikits.learn) is a free software machine learning

library for the Python programming language.It features various
classification, regres- sion and clustering algorithms including support
vector machines, random forests, gradient boosting, k-means and DBSCAN,
and is designed to interoperate with the Python numerical and scientific
libraries NumPy and SciPy.
Scikit-image and scikit-learn were used.

4.2 Implementation Results

4.2.1 Phase 1: Face Detection

Face Detection phase was done using OpenCV with the help of python
pro- gramming language. Face Detection was implemented in Windows
Operating System as an initial stage using the Webcamera of the Laptop.
So, during image capture using Webcamera, the computer detected the
available faces from the im- age, using Haar Algorithm.

Further, it was implemented onto System hardware using the Web Camera

,which is interfaced with the System

4.2.2 Phase 2: Creation of Image Database

Using the Face detection concept, we created a face image database of

60 peo- ple with 100 images each, by entering the name of each person
during image cap- ture. So, the 100 images of each person will be saved
using the names entered. Images were captured in different lighting
conditions. The images captured was performed with an Image
Enhancement technique called Sharpening. The sharp- ened images were
saved in JPEG format with dimensions 120X120.

The Face images were trained using Java codes.

4.2.3 Phase 3: Face Recogntion

We initially implemented Face Recognition by displaying the corresponding

 names of the detected faces (from the image dataset) and the Confidence
value in per- centage, as shown in Figure 4.6. Face recognition has been
implemented using Fisher Faces algorithm and using the concept of
confusion matrix, the accuracy was measured as 80 percent.
To improve face recognition accuracy and provide an efficient security
layer for the ATM system, we came into the idea of using Deep Learning.
The First GUI Frame of our ATM system is as shown . The Frame has
the following Options:
1. Start Cam: This button allows to start camera device, which hereby is the
Web Camera. Then it proceeds with the Face Detection Phase. During Face
Detection, it allows to detect only one face at a time.
2. Load Image: This button allows to load the face image, which has been
detected during Face Detection, in the square vacancy provided on the top
left portion of the frame.
3. Identify User: This button allows to proceed with the Face Recognition
Phase. The loaded image on the box, is verified with the image Database
created and displays the username of the corresponding user in the ’User
Name’ label.
 Chapter 6

Conclusions and Future Scope

6.1Conclusion

Facial recognition has proven to be one of the most secure methods of all
biometric systems to a point for high level security and to avoid ATM
robberies and provide security for ATM.

In the proposed project, it replaces the traditional ATM system. It has

advantages such as saves manufacturing cost of cards and overcomes
drawbacks of the traditional system like carrying the ATM card, losing of
card, fraud calls related to ATM card, etc.

With new improved techniques in the field of artificial Intelligence that

help eliminate more disturbances and distortions, the rate of effectiveness of
the system
can be improved.

6.1Future Scope
This project can be used for real time security applications like in ATM
security systems, military applications, high security companies. This can
also be used in bank locker access. Lighting provided to the system is a key
factor to be taken care of. Usage of high-speed computers can improve the
efficiency.
Bibliography

[1]J.J.Patoliya, M.M. Desai, "Face Detection based ATM Security System

using Embedded Linux Platform ", 2nd International Conference for
Convergence in Technology (I2CT), 2017.

[2]M.Karovaliyaa, S.Karediab, S.Ozac, Dr.D.R.Kalbande, "Enhanced

security for ATM machine with OTP and Facial recognition
features",International Conference on Advanced Computing Tech-
nologies and Applications (ICACTA), 2015.

[3]Sivakumar T. 1 , G. Askok 2 , k. S. Venuprathap, "Design and

Implementation of Security Based ATM theft Monitoring system",
International Journal of Engineering Inventions, Volume 3, Issue 1,
2013.

[4]C. Bhosale, P. Dere, C. Jadhav, "ATM security using face and

fingerprint recognition", Interna- tional Journal of Research in
Engineering, Technology and Science, Volume VII, Special Issue, Feb
2017.

[5]Manoj V , M. Sankar R , Sasipriya S , U. Devi E, Devika T , "Multi

Authentication ATM Theft Prevention Using iBeacon", International
Research Journal of Engineering and Technology (IRJET).

[6]L. Wang,H. Ji, Y. Shi, " Face recognition using maximum local fisher discriminant
analysis",
40
 18th IEEE International Conference on Image Processing, 2011.

[7]K.Shailaja and Dr.B.Anuradha, "Effective Face Recognition using Deep

Learning based Linear Discriminant Classification ", IEEE International
Conference on Computational Intelligence and Computing Research,
2016.

40
[8]H. R. Babaei, O. Molalapata and A.H.Y Akbar Pandor, "Face
Recognition Application for Au- tomatic Teller Machines (ATM)",
International Conference on Information and Knowledge Manage- ment
(ICIKM), 2012.

[9]https://docs.opencv.org/2.4/modules/contrib/doc/facerec/facerec-
tutorial.htmlface- recognition

[10] https://www.superdatascience.com/opencv-face-
recognition/ [11]https://www.rankred.com/face-
recognition-algorithms-techniques/
[12] https://www.pyimagesearch.com/2018/06/18/face-recognition-with-
opencv-python-and- deep-learning/

[13] https://towardsdatascience.com/facial-recognition-using-deep-learning-a74e9059a150
CODING

import face_recognition
import cv2
from time import sleep
import os
#os.system("python steganography.py unmerge --img=res/Decrypted.png
--output=output2.png")
READ_API_KEY=' '
CHANNEL_ID= ' '
c=["ARUL","PRIYA","","","","","","","",""]
b=[5000,10000]

def checkface():
name=""
top = 4
right = 4
bottom = 4
left = 4
image_1 = face_recognition.load_image_file("1.jpg")
image_1_face_encoding = face_recognition.face_encodings(image_1)[0]
image_2 = face_recognition.load_image_file("output2.png")
image_2_face_encoding = face_recognition.face_encodings(image_2)[0]

known_face_encodings = [

image_1_face_encoding,
image_2_face_encoding,
]
known_face_names = [

"0","1"
]

face_locations = []
face_encodings = []
face_names = []
video_capture = cv2.VideoCapture(0)
process_this_frame = True
for ig in range(0,50):
ret, frame = video_capture.read()
small_frame = cv2.resize(frame, (0, 0), fx=0.25, fy=0.25)
rgb_small_frame = small_frame[:, :, ::-1]

if process_this_frame:
face_locations = face_recognition.face_locations(rgb_small_frame)
 face_encodings = face_recognition.face_encodings(rgb_small_frame,
face_locations)
for face_encoding in face_encodings:
matches = face_recognition.compare_faces(known_face_encodings,
face_encoding)
name = "Unknown"
if True in matches:
first_match_index = matches.index(True)
name = known_face_names[first_match_index]
face_names.append(name)
process_this_frame = not process_this_frame
for (top, right, bottom, left), name in zip(face_locations, face_names):
top *= 4
right *= 4
bottom *= 4
left *= 4
cv2.rectangle(frame, (left, top), (right, bottom), (0, 0, 255), 2)
cv2.rectangle(frame, (left, bottom - 35), (right, bottom), (0, 0, 255),
cv2.FILLED)
font = cv2.FONT_HERSHEY_DUPLEX
cv2.putText(frame, name, (left + 6, bottom - 6), font, 1.0, (255, 255, 255), 1)
cv2.imshow('Video', frame)
cv2.waitKey(1)
video_capture.release()
cv2.destroyAllWindows()
 return name

while True:
s=input("ENTER 1 FOR FACE TRANSACTION \n ENTER 2 FOR OTHER
TRANSACTION")
if s==1:
a=checkface()
print(a)
if a=="":
print("YOUR FACE IS NOT RECOGNISED")
else:
print("SUCESSFULLY AUTHENTICATED")
v=int(a)
print(c[v])
#print(b[v])
t=input("ENTER THE AMOUNT TO TAKE:")
if t<b[v]:
print("AMOUNT PROCESSED SUCESSFULLY")
b[v]=b[v]-t
print("Your Balance amount is")
print(b[v])
else:
o=raw_input("ENTER SECRET PIN:")
TS = urllib2.urlopen("https://api.thingspeak.com/channels/994271/feeds.json?
api_key=OULT2DMARL34REDR&results=1")

response = TS.read()
data=json.loads(response)
#b = data['channel']['field1']
b=data['feeds'][0]['field1']
u=b[0:4]
if o in u:
print(b[4:])
print("processed Sucessfully")