Final Major Documentation
Final Major Documentation
Bachelor of Technology
In
COMPUTER SCIENCE & ENGINEERING (AIML)
By
ACE
Engineering College
An Autonomous Institution
(NBA ACCREDITED B. TECH COURSES: EEE, ECE, MECH, CIVIL & CSE, ACCORDED NAAC ‘A’ GRADE)
CERTIFICATE
This is to certify that the Project work entitled “Detection of Face Morphing Attacks”,
submitted by K. Tejavardhan Reddy (20AG1A6623), P. Abhishek (21AG5A6604), M.
Laya (20AG1A6642) towards partial fulfilment for the award of Bachelor of Technology in
Computer Science & Engineering (AIML) to the Jawaharlal Nehru Technological University
Hyderabad, is a record of bonafide work done by them. The results embodied in the work are
not submitted to any other University or Institute for award of any degree or diploma.
EXTERNAL EXAMINER
ACKNOWLEDGMENT
We would like to express our gratitude to all the people behind the screen who have helped us
transform an idea into a real time application.We would like to express our heart-felt gratitude
to my parents without whom. We would not have been privileged to achieve and fulfill our
dreams.
A special thanks to our Secretary, Prof. Y. V. GOPALA KRISHNA MURTHY, for having
founded such an esteemed institution. We are also grateful to our beloved principal, Dr. B. L.
RAJU for permitting us to carry out this project.
We profoundly thank Dr. KAVITHA SOPPARI, Head of the Department of CSE (Artificial
Intelligence & Machine Learning), who has been an excellent guide and a great source of
inspiration forourwork.
We extremely thank Dr. KAVITHA SOPPARI, Head of the Department of CSE (Artificial
Intelligence & Machine Learning, Major Project coordinator, who helped us in all the way in
fulfilling of all aspects in completion of our Major Project.
We are very thankful to our internal guide Mrs. Y. VANITHA, Assistant Professor, who has been
an excellent and given continuous support for the Completion of our project work.
The satisfaction and euphoria that accompany the successful completion of the task would be
great, but incomplete without the mention of the people who made it possible, whose constant
guidance and encouragement crown all the efforts with success. In this context, we would like to
thank all the other staff members, both teaching and non-teaching, who have extended their
timely help and eased our task.
DECLARATION
This is to certify that the work reported in the present project titled “DETECTION OF FACE-
MORPHING ATTACKS” is a record work done by us in the Department of CSE (Artificial
Intelligence & Machine Learning), ACE Engineering College.
No part of the thesis is copied from books/journals/internet and whenever the portion is taken, the
same has been duly referred in the text; the reported are based on the project work done entirely by us
not copied from any other source.
ABSTRACT
Face morphing attacks are a growing concern in the digital world, with the
the use of deep learning techniques for both generating and detecting face morphing
neural networks and auto encoders to learn the underlying facial features and
genuine ones. The proposed framework achieves high accuracy in detecting face
morphing attacks, even in cases where the morphed images are visually similar to
the genuine ones. The project highlights the potential of deep learning-based
approaches for addressing the problem of face morphing attacks and provides
LIST OF CONTENTS
S NO. TITLE PAGE NO.
ABSTRACT i
LIST OF CONTENTS ii
LIST OF FIGURES iv
LIST OF TABLES v
1 INTRODUCTION 1
1.1 BACKGROUND 2
1.2 OBJECTIVE 3
1.3 PROBLEM STATEMENT 4
2 LITERATURE SURVEY 5
3 SYSTEM REQUIREMENTS 8
3.1 Hardware Requirements 8
3.2 Software Requirements 8
3.2.1 Python 9
3.2.2 Visual Studio Code 9
3.3.3 Django 10
4 SYSTEM ANALYSIS 11
4.1 EXISTING SYSTEM 11
4.2 LIMITATIONS 11
4.3 EXISTING SYSTEM ARCHITECTURE 12
5 PROPOSED SYSTEM 13
5.1 CREATION OF DATASET 13
5.2 MORPH 3 IMAGES 14
5.3 ARCHITECTURE 15
6 SYSTEM DESIGN 17
6.1 DEEP LEARNING MODEL-FACENET 17
6.2 MTCNN 19
6.3 SVM vs NAIVE BAYES 20
6.4 SVM 21
6.5 PARAMETRS IN FEATURE COMBINATION 22
6.5.1 SSIM 22
6.5.2 COSINE DISTANCE 22
6.6 METHODOLOGY 23
6.6.1 Generation of Face Morphing 23
6.6.2 Detection of Face Morphing 25
7 INPUT AND OUTPUT DESIGN OF WEB PAGE 28
7.1 INPUT DESIGN 28
7.2 OUTPUT DESIGN 28
LIST OF FIGURES
S. No TITLE PAGE NO
Figure 1.1 Subject and Morph 2
Figure 4.3.1 Architecture of Existing Method 12
Figure 5.1.1 Dataset 14
Figure 5.3.1 Block Diagram for Detection 15
Figure 6.1.1 Block Diagram of FaceNet Architechture 18
Figure 6.2.1 Block Diagram of MTCNN 19
Figure 6.6.1.1 Detecting landmarks for subject 1 24
Figure 6.6.1.2 Detecting landmarks for subject 2 25
Figure 6.6.2.1 Formation of Delaunay triangles using detected features for 26
subject1
Figure 6.6.2.2 Formation of Delaunay triangles using detected features for 26
subject2
Figure 7.3.1.1 Class Diagram 30
Figure 7.3.2.1 Admin Sequence Diagram 31
Figure 7.3.2.2 User Sequence Diagram 31
Figure 7.3.3.1 Admin UseCase Diagram 32
Figure 7.3.3.2 User UseCase Diagram 33
Figure 7.3.4.1 Activity Diagram for Register Page 34
Figure 7.3.4.2 Activity Diagram for Register Page 34
Figure 7.3.5.1 Deployment Diagram 35
Figure 10.1 Homepage for the Website created 42
Figure 10.2 About Face Morphing Attacks 42
Figure 10.3 Morphed Image 43
Figure 10.4 Result to Detect Morph Attack 43
Figure 10.5 Application Management 44
Figure 10.6 Morphed Attacks 44
Figure 10.7 Verified Applications 45
Figure 10.8 Morph Attack Analysis 45
LIST OF TABLES
S. No TITLE PAGE NO
CHAPTER 1
INTRODUCTION
Face morphing assaults use facial image manipulation to combine two distinct
people's faces into one image. These assaults can be utilised for political influence,
identity theft, and other types of fraud or deception. A variety of techniques, including
picture warping, mixing, and texture creation, can be employed to create face morphs.
Machine learning models and forensic analysis techniques, which commonly use counters
and pixel values, can be used to detect face morphing attacks. Nevertheless, it might be
difficult to spot face morphing attacks because morphs may be made to deceive detection
systems and frequently look genuine. These morphing attacks have been documented in
the various fields providing threat in real world scenarios such as political campaigns,
social media, and identity theft cases. It is expected that continued study and the creation
of more advanced detection systems will be necessary for the effective detection and
prevention of face morphing attacks.
The number of people that travel has significantly increased as a result of these
numerous transportation options. Manual travel document and facial identity verification
is impossible with such a huge mobility population. Therefore, the authentication and
approval of passports is done by an automated border control system. There are now
border control systems installed in more than 180 airports worldwide. [1]. Face
biometrics are frequently used in border control applications to secure people identity,
where a person's identity is confirmed using either a passport or an identification card
given by the government of the concerned nation. Although in a small number of
countries the face photo for the passport is taken under strict supervision in a reputable
authority structure, a face photo must be submitted by the applicant in the great majority
of nations. Therefore, applicants may use morphing techniques to create any facial image
that more closely resembles the applicant original face. [2]
into a single composite image that combines the features of both facial photographs and
deceives border control biometric verification systems. To ensure the reliability of such
systems, it is crucial to detect this form of fraud. Attacks that morph pose a serious
concern, particularly to border security [3]. Anti-spoofing methods are the focus of
numerous recent research studies. The tremendous difficulty of being recognised makes
morphing attacks one of the most hazardous types of attacks. It can be performed by the
use of different morphing methods using facial images captured during morphing attacks.
In any event, this work primary focus is facial morphing because it is the most harmful
and challenging to detect in Automatic Border Control systems. [4]
It was discovered that an original identity theft scenario can readily fool ABC's
FRS. The fundamental idea behind these attacks is that a malicious morphed face image is
an amalgam of at least two actual facial photos to look like a large number of different
real people, such as an accomplice to a crime. The modified picture is then registered
with the FRS as an identification template. In a successful attack, the perpetrator and any
helper can be similar to the images data kept in the system which recognizes faces. It
indicates that a desired lawbreaker who is not allowed to travel out of the city can use
facial image morphing to combine the faces of two accomplices to have a valid passport.
The altered facial picture samples are displayed. [5].
1.1 Background
Following the introduction of this concept, some inquiries about the commercial
FRS's susceptibility to morphing face attacks have been made. It has been established that
the current FRS is seriously threatened by face morphing attacks. Some face morphing
1.2 Objective
The objective of this project is to analyse the photos to discover any anomalies or
artefacts that might suggest that the image has been altered is necessary to detect face
morphing assaults for morph 3 images. Numerous methods, like comparing face
landmarks, examining image gradients, and checking for differences in skin texture and
lighting, can be used to do this. [1]
This detection method is to accurately differentiate between real facial photos and altered
ones. This will improve the security of numerous systems and applications that use face
recognition technology while also assisting in the prevention of various forms of fraud
and identity theft. [7]
The use of face morphing technology in unlawful activities has become a growing
concern in recent years, particularly in the creation of deepfakes. Deepfakes are videos or
images that have been manipulated to show a person doing or saying something that they
did not actually do or say. This can be achieved through face morphing techniques, which
allow a person's face to be superimposed onto another person's body or used to create a
completely fake image. [9]
Some of the potential unlawful activities that face morphing can be used for include:
1. Fraud: Face morphing can be used to create fake identities, such as for opening bank
accounts or applying for loans. This can lead to financial fraud and identity theft.
2. Morphing images: Face morphing can be used to create fake images or videos of
people in compromising situations, which can then be shared without their consent. This
can be used to harass or blackmail individuals.
3. Misinformation and propaganda: Face morphing can be used to create fake news
stories or propaganda by manipulating images or videos of public figures or events. This
can be used to influence public opinion or spread false information.
Overall, the use of face morphing technology in unlawful activities is a serious concern
and highlights the need for increased awareness and regulation of this technology. It is
important to be vigilant about the potential for misuse and take steps to protect
individuals from the negative consequences of face morphing.
CHAPTER 2
LITERATURE SURVEY
Limitation: Images with participants who were expressing expression, posture, or some
degree of occlusion had a worse time being accurately restored. They only used morph-2
photos fewer morphing resources. [1]
This research presents a conceptual categorization, criteria for evaluating such strategies,
and a thorough literature review of pertinent works. Along with outstanding problems and
difficulties in the field, the methodology surveyed the technical issues and trade-offs are
also investigated.
Limitation: The observed assault detection accuracy using facial picture morphing has
not yet been generalised to datasets that include a wide range of capture situations
encountered in the real world. There are several unresolved problems and obstacles in
face morphing and assault detection research, such as quality and comparability. [9][10]
In this study, deep face representations are used to examine methods for identifying
differential morphing attacks. It is demonstrated that deep face representation-based
approaches offer incredibly high detection sensitivity and robustness with regard to
different processings. The methodologies flaws are then examined in detail.
Limitation: Superior morphs which are of high calibre and morphs that are strikingly
similar to live-picture capture weren't appropriately categorised. The misclassification of
photographs was also aided by differences in facial expression, headgear, eyewear,
illumination, and focus: Lack of a database that accurately represents a real-world
circumstance. There were just morph-2 photos used.
Limitation: The article does not provide a direct comparison of the different image
morphing techniques discussed. It only presents the advantages and disadvantages of each
technique separately, without comparing them to each other.
The de-morphing architecture put forth in this research is built on a machine learning
based convolutional neural network (CNN) architecture. This method depends on the
usage of two pictures: the passport photo that may have been altered and the person's live
picture in the system for Automatic Border Control. The demorphing process aims to
decode the picture. In morph attack detection, accuracy of 98.1% to 98.7% was
attained.[4]
Limitation: In this study, lighting, posture, expressions, and variety of images databases
weren’t taken into account. We only used morph-2 photos. Fewer morphing resources.[1]
Title: Accurate and robust neural networks for face morphing attack detection
In order to improve robustness and generality, they suggest neural network training
strategies in this research that depends on various modifications of the data that must be
trained. For instance, consider attacks using morphing faces, the neural network's ability
to access certain types and locations of information is strictly constrained and their
impacts on precision, generality, and resistance to black-box and semantic attacks are
studied.[3]
Limitation: Only carefully chosen photos with neutral expression, illumination, and
position were used for training and testing. Only photographs with consistent expression
and lighting in the stance were taken into account, hence only those images' accuracy was
achieved. Images that display variability in poses, expression, stance, illumination, and
other aspects like eyewear, headgear, age must undergo testing.
The suggested method depends on two D-CNNs that are fused at the feature level.
(VGG19 and AlexNet), which have been specifically enhanced using a database of
morphing facial images. The suggested solution is thoroughly tested on the recently
created database using altered face from digital and print scanning photographs that
correlate to real-life scenarios and morphed data.
Limitation: The model's accuracy has not been demonstrated on datasets with a wide
range of feature sets because a neutral-position dataset of a typical face expression , pose,
and illumination was used. Images that display variability in position, expression and
other elements like headgear and face mask must undergo testing. They only used morph-
2 photos. Fewer morphing resources. [1]
CHAPTER 3
SYSTEM REQUIREMENTS
Ram 1 Gb 4 Gb
The Table 3.1 shows the hardware requirements needed to perform the project.
Table 3.2 displays the software specifications required for the project comprising of
development software, programming languages and their versions to be installed.
3.2.1 Python
Python is the language used in this project for programming. A high-level, object-
oriented, interpreted programming language is Python. Python is normally used as an
internet software scripting language. Python is garbage-accumulated and dynamically
typed. Python adheres to OOPs principles. The number one intention of Python OOP is to
generate reusable code. Python adheres to essential OOPs standards consisting of class,
object, technique, polymorphism, encapsulation, facts abstraction, and inheritance.
Python is used for numerous responsibilities inclusive of software program development,
back-quit development, facts science, and writing machine scripts. Python gives guide for
modules and applications, which promotes programme modularity. [15]
● Pip: Pip is a package management system used to install and manage software
packages written in Python. It stands for "Pip Installs Packages" or "Pip Installs Python"
Microsoft created Visual Studio Code, a free and open-source code editor. It works with
Windows, macOS, and Linux and supports a wide range of programming languages,
including C++, Java, Python, and JavaScript, among others. It has a user-friendly
interface and is highly customizable to meet the needs of developers. Visual Studio Code
is a powerful tool for creating projects across a wide range of programming languages
and platforms. Here are some key features that make it an excellent choice for project
development:
Integrated Terminal: The integrated terminal in Visual Studio Code enables programmers
to execute commands and scripts without leaving the editor.
Code Navigation: Visual Studio Code makes it easy to navigate code with features such
as go-to-definition, find all references, and definition.
Debugging: Visual Studio Code comes with built-in support for debugging, which helps
developers find and fix issues in their code more quickly. [16]
3.2.3 DJANGO
Developers may quickly design web apps using the high-level Python web
framework Django. It follows the model-view-controller (MVC) architectural pattern and
provides a lot of built-in features and tools for handling common web development tasks,
such as URL routing, database access, user authentication, and template rendering.
URL routing: Django's robust URL routing mechanism enables developers to assign
URLs to views or processes that produce answers. This makes it easy to create user-
friendly URLs for your application.[17]
User authentication: Django provides a built-in authentication system that includes user
registration, login, and logout functionality. It also supports authentication via third-party
providers, such as Google and Facebook.
Django's emphasis on security is one of its main benefits. Common web application flaws
such cross-site scripting (XSS), cross-site request forgery (CSRF), and SQL injection and
clickjacking are all covered by Django's built-in security. [18]
CHAPTER 4
SYSTEM ANALYSIS
The issue of detection of morphed image attacks has received a lot of focus
recently from the scientific community. To accurately detect morph attacks, various
studies have been done in this area and various methods have been used. Since it is
difficult to find enough morph photos for research purposes, a variety of face databases
are used to create morph image databases. Features such as expression variety, head gear,
facial hair, race, eye protection, eyes and different postures are insufficiently or never
used in several research. [3][9]
a) Single MAD: It is using just one image (no reference). There is only one altered image
offered [6][1]. There are two techniques to use deep neural networks to find morphs. In
the beginning, either a new network or an existing one is established is retrained for the
MAD job. Theoretically, any artefact can be detected by deep neural nets [6]. When an
image is morphed, some artefacts remain that can be seen and used to detect the morph.
b) Differential MAD, which uses two pictures. Both the altered image and an additional
one are used. This is a common occurrence in ABC systems [1][4]. If two images of
identities are given, differential MAD typically provides solutions for ABC systems that
are comparable to them. Two pictures are needed for differential MAD to identify
morphing [3]. By estimating differences between features like single picture algorithms
can be transformed into differential MAD algorithms. [6]
4.2 LIMITATIONS
Real-world images pose a problem for both the differential and no-reference MAD
techniques. The actual altered photos have frequently been printed and scanned before
this last step. A passport's image is embedded there. As a result, MAD algorithms can no
longer recognise distorted photos. This issue is also examined in the research previously
cited. [9][12]
The second group of algorithms tries to halt the morphing procedure. The basic idea is
that if there are two or three people represented in the image that is morphed and the
trustworthy live capture is taken out of the possibly distorted reference, the other subject's
identity will become more prominent and result in poorer scores for facial recognition.
Nothing remains in a photograph if there is only one subject there. The so-called
demorphing algorithm, a technique that does direct reversal, was put forth in [9]. In
addition, as an alternative to the explicit de-morphing technique, a Generative Adversarial
Network (GAN)-based demorphing strategy is put forth. [5] [6]
In the realm of morph picture detection, several research papers have reported positive
detection findings, although these studies evaluate datasets having few variations and lack
real-world settings. In several research, characteristics like age, race, facial hair,
headgear, eyewear, different postures, expression, and illumination are either under- or
not at all considered [3][6]. A similar small number of databases are required to detect
morph attacks. Another really serious issue exists with the current morph detection
datasets. These datasets simply took into account the morph of two individuals (morph-2
photos), making morph recognition simple [5][9]. Additionally, to automatically create
morphed images, programming of poor quality depends on morphing tools like
FaceFusion, FaceMorpher, FotoMorph and OpenCV is employed. The majority of these
images are easy for a human to spot through visual inspection. Because criminals rarely
employ these methods, the situations depicted do not reflect real-world events. Despite
having very high detection rates when tested on datasets with the aforementioned
limitations, these techniques will not perform well in real-world situations.
CHAPTER 5
PROPOSED SYSTEM
Attack samples developed for research databases may not be the same as attack samples
from the actual world. The creation of a large number of attack samples, which can be
done automatically, is required to provide meaningful evaluation findings.
• The first public dataset was developed by Biometrix. It is based on the NIST FERET
database and offers 1,082 samples.
• FRLL-Morphs by IDAP is another publicly available morph dataset. It was created
with WebMorpher, StyleGAN2, OpenCV and FaceMoprher, thus offering 4 types of
sample material.
A method must be utilised to enhance the deep learning models performance such that
the models architecture has been appropriately improved and modified to attain the best
outcomes. As a result, the primary aim of this research is to create a database that would
aid the model in learning and enhancing its performance. The FERET, FRGC, and FRLL
datasets were used to create morphs for the photos using the OpenCV and FaceMorpher
programming tools, which were obtained from prior studies. [1]
2. Morph-2, which simply combines the facial images of two people to produce images.
During the process of developing and evaluating in the name of testing and training , one
of the two input photos are live-captured images and the original photographs that were
provided. The quantity of live photos varies depending on the subject. In order to make
this detection of morphed as realistic as possible, the maximum number of live photos
captured is employed for each subject to add varying difficulties and variety to the
teaching and testing.
Images which are retrieved same database were utilised to create morph images because
databases have varied characteristics and different types of attributes. In each database,
training and testing sets have been created. As various features like variation in lighting
effects known as illumination cause artefacts in morphs and lower the quality of morphs,
no attempt has been made to produce morphs by combining photos from various
databases. Comparable facial photos are modified to produce better results because they
have comparable facial structure and features.
The morph 3 image which is the combination of three subjects has multiple
benefits when compared to the images with two subjects i.e. morph 2 image. As a result
of the third image (an extra image) is also considered as the one of the subject to perform
morph serving to ensure a seamless transition between the images. This group of three-
subject morphing photos displays a more realistic portrayal of a human face. By
combining three photographs, colour tone [11] is also correctly normalised, harsh or
sudden differences between different photos, as well as artefacts, are considerably
reduced. The more realistic and natural-looking Morph-3 images are better that they are
good at deceiving the human inspection. Because morph-3 photographs have only
recently been developed and launched in this sector of morphed image recognition, there
isn't any research analysing the quality assessment of morph 3 photos. [1]
In order to evaluate and estimate the standard quality, the various parameters such as
SSIM, PSNR of the morph 3 pictures are employed in this research, the morph 2 and
morph-3 image quality were compared using the SER-FIQ technique [1]. It was found
that the face characteristics in morph-3 photographs are of higher quality in the great
majority of cases. The primary objective of creating a morph is to mislead the recognition
system, and since morph-3 photos are of higher quality, there is a greater likelihood that
they will match with the associated live images and succeed in doing so.
It's also crucial to emphasise that the crooks who produce the morphed
photographs are not restricted to employing a specific sort of software or editing
techniques. It is important to have knowledge about various morphing techniques. Their
primary objective is to trick the identification system by employing all tools and
techniques they have access to, therefore they do not follow any set guidelines for solely
producing morph-2 photos.
5.3 ARCHITECTURE
In this study, a reliable detection method is proposed that can account for age, lighting,
eye, and headgear variations. A classifier and a feature extractor based on deep learning
are used. To improve the detection outcomes, image enhancement and feature
combination are also suggested. [10]
In this work, a distinctive and varied morphing database is manually constructed utilising
expert software. This work includes morphed photos made from two and three subjects.
On the newly created database, the use of extraction of features with deep learning model
like FaceNet and an artificial intelligence-based classifier like SVM forms the basis of a
contemporary morph detection algorithm.
A threshold is defined to determine the decision boundary between genuine and morphed
faces. Images with predicted likelihoods above the threshold are classified as morphed,
while those below it are considered genuine.
This study's goal is to evaluate the suggested morph attack detection model's performance
on various morphed and unmorphed image types. [1]
The generation and detection techniques continually evolve as there is a need to develop
new algorithms and strategies to improve the quality of morphing attacks and their
detection.
CHAPTER 6
SYSTEM DESIGN
FaceNet is a deep learning model for face recognition that was developed by Google
researchers. One of the key features of FaceNet is its ability to perform face recognition
using deep learning-based feature extraction techniques.
In FaceNet, face images are first preprocessed to align and normalize the face region,
followed by being fed into a convolutional neural network with deep learning (CNN)
that has been taught to extract key features from facial images. The CNN in FaceNet
consists of multiple layers that progressively learn more complex and abstract features,
culminating in a layer that outputs a fixed-length vector representation, or embedding, for
each input face image.[19]
The embeddings produced by FaceNet are highly discriminative and can be used for
various identifying faces tasks, such as facial identification and confirmation. One of the
advantages of using deep learning-based feature extraction techniques like FaceNet is that
they can learn highly robust and discriminative representations of faces that are less
sensitive to variations in lighting, pose, and facial expression.
In practice, FaceNet can be used to train a face recognition system by first feeding a large
dataset of labelled face images through the CNN to obtain their embeddings, and then
using these embeddings to train a classifier or similarity metric that can compare new face
images to the ones in the dataset. FaceNet has attained cutting-edge results on a number
of metrics for facial recognition, proving the usefulness of deep learning-based feature
extraction methods for this task. [20]
The FaceNet model takes as input a face image and outputs a fixed-length vector
representation, or embedding, that captures the most important features of the face. The
model consists of the following components:
Input Image: The input to the FaceNet model is a face picture. The image is first
preprocessed to align and normalize the face region.
Preprocessing: Identifying the face in the raw image is a stage in the preprocessing
process and aligning it to a canonical pose. This helps to reduce alterations in the lighting,
pose, and facial expression, which can affect the quality of the embeddings.
The MTCNN model consists of three stages, each of which performs a different task in
the face detection process:
First stage [P-Net]: The first stage of MTCNN is a convolutional neural network (CNN)
that performs candidate face region proposals using a sliding window approach. This
stage is designed to efficiently filter out most of the non-face regions in the image.[1]
Second stage[R-Net]: The second stage of MTCNN is also a CNN that performs fine-
grained face region proposals. This stage refines the candidate face regions from the first
stage by using more complex features and a bounding box regression algorithm. The
second stage is also able to filter out false positives.
Third stage[O-Net]: The third stage of MTCNN is a CNN that performs facial landmark
detection. This stage takes the refined face regions from the second stage and detects
facial characteristics i.e. landmarks such as the eyebrows, nose, lips and mouth. These
landmarks will be used to align the face regions for further processing, such as face
recognition.[21]
On the other hand, Naive Bayes is a statistical classifier that determines probability that a
sample is a member of a particular class by applying the Bayes theorem. The foundation
of Naive Bayes is the idea that all features are independent of one another, which can
simplify the computation and reduce the risk of overfitting. Naive Bayes is often used in
text classification and spam filtering, but it has also been used for face morphing
detection.
In face morphing detection, SVM and Naive Bayes can both be used to classify whether a
given picture is a morphed image or a genuine picture. SVM can be effective when the
feature space is high-dimensional, and it has been shown to achieve high accuracy in face
morphing detection tasks. [23]
The choice between SVM and Naive Bayes depends on the specific task and the
characteristics of the data. Both algorithms can be effective in face morphing detection,
and the performance of each algorithm can depend on factors such as the size and quality
of the training data, and the choice of features.
In this paper SVM is proposed to be the classifier as they have also been shown to be
robust to noise and outliers, which can be beneficial in face morphing detection where the
input images may have varying degrees of distortion.[23]
The main advantage of SVM is its ability to generalize well to new data, which
makes it particularly useful in real-world applications. Several tasks, including
categorisation of text, picture classification, and bioinformatics, have demonstrated the
effectiveness of SVM. By altering the goal function to reduce the discrepancy between
the expected and actual output values, SVM can also be utilised for regression tasks. The
SVM classifier consists of following components. [24]
Data Input: The SVM classifier takes a set of input data points (also called instances or
examples) as input. Each data point has a class label and a vector of features that describe
it.
Feature Extraction: Prior to inputting the data to the SVM classifier, feature extraction
techniques may be used to extract relevant characteristics and landmarks from the input
data.
Training: In the training phase, the SVM algorithm learns a decision boundary that
separates the different classes. The SVM algorithm looks for the optimum hyperplane to
maximise the distance between nearby points from each class, also known as support
vectors. The training process involves finding the optimal values for the model
characteristics that reduce the classification error, including the hyperplane's weights and
biases.
Testing: Once the SVM classifier has been trained, it can be used to classify new input
data points by evaluating which side of the hyperplane the input data point falls on.
Output: The SVM classifier outputs a predicted class label for each input data point based
on its position relative to the learned decision boundary.[25]
The reason SSIM is preferred over other similarity metrics is that it takes into account the
structural information of the images being compared. This means that SSIM can capture
both the pixel-level similarity between images as well as the structural similarity in terms
of their texture, contrast, and spatial layout. This is important for detecting face morphing
attacks, as these attacks often introduce structural distortions into the morphed image that
can be detected using SSIM. [26]
Another advantage of using SSIM is that it is able to detect subtle changes in image
quality that may not be easily visible to the human eye. This is important for detecting
face morphing attacks, as these attacks may introduce subtle changes in the morphed
image that are difficult to detect using other image quality assessment methods
The use of SSIM in face morphing attacks is based on its ability to accurately capture
both the pixel-level similarity and structural similarity between images, as well as its
ability to detect subtle changes in image quality that may indicate the presence of a face
morphing attack [27]
The cosine distance is preferred over other trigonometric ratios such as sine or tangent is
that it is less sensitive to changes in the magnitude of the vectors being compared. This
means that even if the magnitudes of the feature vectors extracted from two faces are
different, the cosine distance measure can still accurately capture the similarity between
the features.
The another advantage of using cosine distance is that it is computationally efficient and
easy to calculate. It is a simple measure that only involves a dot product operation
between the two vectors being compared, making it a fast and efficient method for
comparing large sets of feature vectors.[29]
6.6 METHODOLOGY
There are many ways to perform Generation and Detection of Face Morph
Face image acquisition: Collect a dataset of facial images, including genuine images and
morphed images, with various types of morphing techniques applied.
Feature extraction: Extract features from the facial images, such as geometric features,
texture features, or frequency domain features.
Morphing image generation: Generate morphing images using various techniques, such
as image warping, averaging, or deep learning-based methods.
Detection of morphing attacks: Apply the trained model to detect morphed images in
test datasets.
Generating a good morph with the best algorithm for morph3 involves a number of steps,
including selecting appropriate images, detecting and aligning facial landmarks,
morphing the images, and blending the resulting morphed image.
1. Select two images that have similar facial features, such as the same person in
different poses or with different expressions.
In this project, the deep learning techniques are used to detect the morphed image.
Face detection using MTCNN: MTCNN is a deep learning algorithm that detects faces
in an image. In this stage, the images are given as the input one which is the live captured
image and other the potential morph.
The facial region of the input image is extracted using this approach. MTCNN is capable
of detecting faces at different scales, orientations, and positions in an image, which makes
it a useful tool for various face-related applications. Face extraction using MTCNN from
input photos (morph and live recorded). The next stage input requires the photos that are
of dimensions 160x160x3 for analysing the features of the source image.
Face representation using FaceNet: FaceNet is a deep learning algorithm that generates
a high-dimensional face-specific feature vector. In this stage, the algorithm is used to
extract a feature vector from the face region detected in the previous stage. In this stage,
the face feature recognition component is also incorporated to identify the individual in
the face image. This can help in identifying any anomalies in the feature vector that may
be caused by morphing.
Fig 6.6.2.1 Formation of Delaunay triangles using the detected features for subject 1
Fig 6.6.2.2 Formation of Delaunay triangles using the detected features for subject 2
The Fig 6.6.2.1 and 6.3.2.2 depicts the Delaunay triangulation ensures that no points are
contained within the circumcircle of any triangle, resulting in a more evenly distributed
mesh.
Additionally, deploying forensic tools such as error level analysis, frequency domain
analysis, and splicing detection can help identify if the image has been manipulated.
These tools can help detect changes in colour, contrast, and image quality that may not be
noticeable to the naked eye.
Morphing detection using feature combination: In this stage, the feature vectors
generated by FaceNet are combined with additional parameters such as SSIM and cosine
distance to improve the accuracy of morphing detection. The cosine distance measures
the separation between two feature vectors, whereas the SSIM measures the similarity
between two images. By incorporating these parameters, the algorithm can better
distinguish between morphed and non-morphed faces.
An average is used to integrate the SSIM and cosine distance scores. Using the lowest
SSIM similarity score and cosine combination, the live image is merged with the
characteristics of a potential altered image.[1]
Classification using SVM: Finally, the SVM is trained on the combined feature vector to
classify the given inputs. After classifying the input images it produces the output as
either morphed or a valid image which the passport of the user valid.
CHAPTER 7
A connection between the user and the information system is created through the input
design. Creating specifications and procedures for data preparation is part of the process
of converting transaction data into a format that can be processed. To do this, users can
input the information directly into the system, or a computer can read the information
from a written or printed document. The input process is created with a view on reducing
the amount of input required, minimising errors, delays, and superfluous phases, as well
as keeping a simple workflow.
It is done by creating user-friendly displays that are simple to use when entering large
volumes of data. The goal of input design is to eliminate errors and simplify data entry.
Any data manipulations are feasible due to the design of the data entry panel.
Additionally, it provides the ability to view records. The accuracy of the data entered will
be examined. Screens can help with data entering. When necessary, the appropriate
notifications are delivered to keep the user from getting soon enveloped in maize. As a
result, the goal of input design is to provide an intuitive input arrangement.
A quality output has a clear presentation of the information and complies with the end
user's needs. The means through which processing outcomes are communicated to users
and other systems are through the outputs of any system. The method of information
displacement for both immediate demand and the hard copy output is chosen during
output design. From these 20 sources, the user is provided with the most important and
direct information. A clear display of the information and compliance with the end user's
requirements are characteristics of a quality output. The means through which processing
outcomes are communicated to users and other systems are through the outputs of any
system. During output design, the mechanism of information displacement is selected for
both the hard copy output and the immediate demand. From these 20 sources, the user is
provided with the most important and direct information.
The term "UML" stands for "Unified Modelling Language." UML is a general-purpose
modelling language that is standardised for use in the field of object-oriented software
engineering. The Object Management Team is responsible for the normalize and creating
it.
The objective is for UML to become the standard modelling language for object-oriented
software. UML has two fundamental parts in its ongoing structure the metamodel and the
annotation. Specific types of methods or strategies may be summed to or associated with
UML later on.
A class diagram is a type of UML diagram that depicts the classes, traits, and procedures
of a given class and relationships in a software system to depict the system's structure.
Class diagrams are commonly used in object-oriented programming to help developers
visualize and understand the structure of a software system.Class diagrams can be used to
help with software design, testing, and documentation. They offer a graphic depiction of a
software system's architecture, which can assist designers in spotting possible problems
and enhancing the system's overall quality. Using class diagrams to automatically
generate code can also save time and decrease the risk of mistakes.
In this diagram, the objects or components of the system are represented by vertical lines
called lifelines. The lifelines are arranged in a chronological order from top to bottom,
representing the sequence of events in the system. The interactions between the lifelines
are depicted by arrows or messages, which indicate the order in which messages are sent
between the components or objects, in order to demonstrate the messages that are
exchanged.
A use case diagram is a type of UML diagram that represents the functional requirements
of a software system by modelling its use cases and actors. Actors describe the roles
played by those users or systems, whereas use cases depict the interactions between the
system and its users or external systems.
In a use case diagram, the actors are represented by stick figures, and the use case is
represented by ovals or ellipses. The use cases and actors are connected by lines called
associations, which represent the interactions between them.
A specific kind of UML diagram known as a "activity diagram" shows how a software
system or business process's activities or actions move. It is used to model complex
workflows, business processes, and software algorithms.
In an activity diagram, the activities or actions are represented by nodes, while the
transitions or flow of control between the nodes are represented by arrows. The nodes can
include actions, decisions, and branching paths that determine the flow of the activity.
A specific type of diagram called a deployment diagram shows the real hardware that the
software system will be installed on. It also manages how the programme is put into
action on the underlying hardware. It links system software components to the underlying
hardware.
CHAPTER 8
IMPLEMENTATION
8.1 LIBRARIES
Asgiref: asgiref is a Python library that provides a framework-agnostic base layer for
building asynchronous web applications. ASGI is a standard interface between web
servers, frameworks, and applications for building asynchronous web applications in
Python. Asgiref provides a set of utility functions and classes that make it easier to build
ASGI-compatible applications and servers.
Cycler: Cycler is a Python library that provides a way to create and manipulate cyclic
lists or sequences. It is commonly used for generating colors, markers, and line styles for
data visualization in Matplotlib, a popular data visualization library for Python.Cycler
allows you to create and combine different cycles, such as a color cycle or a linestyle
cycle, to create complex and visually appealing plots. These cycles can be defined as
Python lists or tuples and then passed to Matplotlib functions to control the appearance of
plots.
Imageio: Python's Imageio library offers an intuitive user interface for reading and
writing a range of image and video file types. It is designed to be lightweight, fast, and
efficient, making it a popular choice for working with image and video data in scientific
research, machine learning, computer vision, and other fields. Imageio supports a wide
range of file formats, including common image formats like JPEG, PNG, and GIF, as well
as video formats like AVI, MPEG, and MP4.
Numpy: NumPy is a Python package that offers a strong and effective approach to
interact with numerical data arrays and matrices. It is widely used in scientific computing,
data analysis, and machine learning applications. NumPy arrays can have any number of
dimensions, allowing you to represent complex data structures like images
Pillow: Pillow is a popular Python library for working with digital images. It provides a
wide range of functions for opening, manipulating, and saving many common image file
formats.Pillow provides a simple and consistent API for working with images, making it
easy to resize, crop, rotate, filter, and otherwise modify images in a variety of ways.
SQLparse: SQLparse is a Python library that provides a simple and efficient way to parse
and analyze SQL queries. It can be used to extract information from SQL queries, such as
tables, columns, and expressions, and to analyze the structure and syntax of SQL queries
for validation, optimization, or other purposes.
Six: Python's Six library offers a straightforward approach to create code that is
compatible with both Python 2 and Python 3.Six provides a range of functions and
utilities for handling differences between Python 2 and Python 3, including functions for
handling differences in string types and formatting, functions for handling differences in
module imports and names.
Tifffile: Tifffile is a Python library for reading and writing image data in the Tagged
Image File Format (TIFF). TIFF is a widely used format for storing image data, especially
in scientific and medical imaging applications. Tifffile provides a simple and efficient
way to read and write TIFF files in Python. The key benefits of Tifffile is its efficiency.
Scikit-image: Scikit-image is a Python package used for image processing and computer
vision tasks. It provides a range of functions and algorithms for performing common
image processing tasks, including filtering, segmentation, feature detection, and more.
Scikit-image is built on top of the NumPy and SciPy libraries, making it easy to integrate
with other scientific computing tools in Python
8.2 CODE
FaceMatch.py
import cv2
from skimage.metrics import structural_similarity as ssim
FaceMorph.py
import pywt
import numpy as np
import matplotlib.pyplot as plt
import random
import cv2
cA = (cA1+cA2+cA3)/3
cH = (cH1 +cH2+cH3)/3
cV = (cV1+cV2+cV3)/3
cD = (cD1+cD2+cD3)/3
def facemorph(img1,img2,img3):
# Params
FUSION_METHOD = 'mean' # Can be 'min' || 'max || anything you choose
according theory
img1 = 'media/'+str(img1)
img2 = 'media/'+str(img2)
img3 = 'media/'+str(img3)
# Read the two image
I1 = cv2.imread(img1)
I2 = cv2.imread(img2)
I3 = cv2.imread(img3)
print (I1.shape)
print (I2.shape)
print (I3.shape)
## Seperating channels
iR1 = I1.copy()
iR1[:,:,1] = iR1[:,:,2] = 0
iR2 = I2.copy()
iR2[:,:,1] = iR2[:,:,2] = 0
iR3 = I3.copy()
iR3[:,:,1] = iR3[:,:,2] = 0
iG1 = I1.copy()
iG1[:,:,0] = iG1[:,:,2] = 0
iG2 = I2.copy()
iG2[:,:,0] = iG2[:,:,2] = 0
iG3 = I3.copy()
iG3[:,:,0] = iG3[:,:,2] = 0
iB1 = I1.copy()
iB1[:,:,0] = iB1[:,:,1] = 0
iB2 = I2.copy()
iB2[:,:,0] = iB2[:,:,1] = 0
iB3 = I3.copy()
iB3[:,:,0] = iB3[:,:,1] = 0
outImage = I1.copy()
outImage[:,:,0] = outImage[:,:,1] = outImage[:,:,2] = 0
outImage[:,:,0] = outImageR[:,:,0]
outImage[:,:,1] = outImageG[:,:,1]
outImage[:,:,2] = outImageB[:,:,2]
outImage = np.multiply(np.divide(outImage -
np.min(outImage),(np.max(outImage) - np.min(outImage))),255)
outImage = outImage.astype(np.uint8)
x = random.randint(1000, 2000)
# cv2.imwrite('morphed2.jpg',outImage)
return outImage
Facemorphackup.py
import pywt
import numpy as np
import matplotlib.pyplot as plt
import random
import cv2
cA = (cA1+cA2)/2
cH = (cH1 +cH2)/2
cV = (cV1+cV2)/2
cD = (cD1+cD2)/2
fincoC = cA, (cH,cV,cD)
outImageC = pywt.idwt2(fincoC, 'db5', mode = 'periodization')
outImageC = cv2.resize(outImageC,(shape[0],shape[1]))
return outImageC
def facemorph(img1,img2):
# Params
FUSION_METHOD = 'mean' # Can be 'min' || 'max || anything you choose
according theory
img1 = 'media/'+str(img1)
img2 = 'media/'+str(img2)
# Read the two image
I1 = cv2.imread(img1)
I2 = cv2.imread(img2)
print (I1.shape)
print (I2.shape)
## Seperating channels
iR1 = I1.copy()
iR1[:,:,1] = iR1[:,:,2] = 0
iR2 = I2.copy()
iR2[:,:,1] = iR2[:,:,2] = 0
iG1 = I1.copy()
iG1[:,:,0] = iG1[:,:,2] = 0
iG2 = I2.copy()
iG2[:,:,0] = iG2[:,:,2] = 0
iB1 = I1.copy()
iB1[:,:,0] = iB1[:,:,1] = 0
iB2 = I2.copy()
iB2[:,:,0] = iB2[:,:,1] = 0
outImage = I1.copy()
outImage[:,:,0] = outImage[:,:,1] = outImage[:,:,2] = 0
outImage[:,:,0] = outImageR[:,:,0]
outImage[:,:,1] = outImageG[:,:,1]
outImage[:,:,2] = outImageB[:,:,2]
outImage = np.multiply(np.divide(outImage -
np.min(outImage),(np.max(outImage) - np.min(outImage))),255)
outImage = outImage.astype(np.uint8)
x = random.randint(1000, 2000)
# cv2.imwrite('morphed2.jpg',outImage)
return outImage
CHAPTER 9
TESTING
The purpose of testing is to discover errors. Testing is the process of trying to discover every
conceivable fault or weakness in a work product. It provides a way to check the functionality of
components, sub assemblies, assemblies and/or a finished product It is the process of exercising
software with the intent of ensuring that the Software system meets its requirements and user
expectations and does not fail in an unacceptable manner. There are various types of test. Each test
type addresses a specific testing requirement.
⚫ TYPES OF TESTS
⚫ White Box Testing
White Box Testing is a testing in which in which the software tester has knowledge of the
inner workings, structure and language of the software, or at least its purpose. It is purpose. It is used
to test areas that cannot be reached from a black box level.
⚫ Black Box Testing
Black Box Testing is testing the software without any knowledge of the inner workings,
structure or language of the module being tested. Black box tests, as most other kinds of tests, must be
written from a definitive source document, such as specification or requirements document, such as
specification or requirements document. It is a testing in which the software under test is treated, as a
black box .you cannot “see” into it. The test provides inputs and responds to outputs without
considering how the software works.
⚫ Unit Testing
Unit testing is usually conducted as part of a combined code and unit test phase of the software
lifecycle, although it is not uncommon for coding and unit testing to be conducted as two distinct
phases.
CHAPTER 10
RESULTS
Fig 9.1 depicts the title of a website's home page, which includes connections to other
pages on the site.
Fig 9.2 gives the description of the project about Face Morphing Attacks,the techniques
used to perform the project and usage of different softwares
When two or more images are given as the input it undergoes the generation process and
produces the Morphed image. Fig 9.3 shows the example of a morphed image.
When the admin logins through his credentials he can see the users kept application for
the passport photo verification. Fig 9.4 shows the application to verify.
The dashboard Fig 9.5 consists of the information about the applicants like registered
users , pending applications for verification ,verified and invalid application.
Fig 9.6 shows the morphed attacks that the applicant morphed his image to pass through
the Automatic Border Control
The verified applications are the applications which doesnot contain any morphed image
and the apllicant is genuine . Fig 9.7 shows the verified applications.
The graph analysis of the projects shown in the Fig 9.8 which gives information about the
attacks and verified users in graphical representation.
CHAPTER 11
10.1 CONCLUSION:
It can be concluded that face morphing attacks are a significant threat to digital security
and privacy, and there is a growing need for effective countermeasures to prevent their
misuse. The generation process involved the use of deep learning models to morph two or more
faces to create a single image, while the detection process focused on identifying the
inconsistencies in the facial features of a morphed image compared to genuine images
Through this project, we explored different techniques for generating and detecting face
morphing attacks. The project demonstrated the potential of deep learning-based approaches for
both generating and detecting face morphing attacks. The performance of morph-3 identification
dramatically improved once the morph 3 photos produced using superior tools are used for
training. Additionally, we investigated the effectiveness of other machine learning based
classifiers, and the best outcomes are produced by SVM. Following an analysis of various feature
combining strategies, feature concatenation emerged as the most effective method for morph
identification. In terms of age and lighting, the proposed model provides better results. [1]
Face morphing attacks are becoming increasingly common with the proliferation of social
media and online identity verification systems. As such, there is a need for effective
methods to detect and prevent such attacks. The project can be extended to explore more
advanced techniques for generating and detecting face morphing attacks, such as the use
of generative adversarial networks (GANs) or the integration of facial recognition
technologies. The project can also be extended to develop practical solutions and tools to
protect against face morphing attacks in real-world scenarios.
CHAPTER 12
REFERENCES
[1] Muhammad Hamza , Samabia Tehsin, Hanen Karamti, “Generation and Detection
Face Morphing Attacks” IEEE Access Volume 10,2022
[2] Christoph Busch Norwegian University of Science and Technology (NTNU), Norway
,”Single Image Face Morphing Attack Detection Using Ensemble of Features” ,IEEE
Xplore,2020
[3] C. Seibold, W. Samek, A. Hilsmann, and P. Eisert, ‘‘Accurate and robust neural
networks for face morphing attack detection,’’ J. Inf. Secur. Appl., vol. 53, Aug. 2020,
Art. no. 102526.
[5] F. Peng, L.-B. Zhang, and M. Long, ‘‘FD-GAN: Face de-morphing generative
adversarial network for restoring accomplice’s facial image,’’ IEEE Access, vol. 7, pp.
75122–75131, 2019.
[6] U. Scherhag, C. Rathgeb, J. Merkle, and C. Busch, ‘‘Deep face representations for
differential morphing attack detection,’’ IEEE Trans. Inf. Forensics Security, vol. 15, pp.
3625–3639, 2020.
[7] G. Wolberg, ‘‘Image morphing: A survey,’’ Vis. Comput., vol. 14, no. 8, pp. 360–
372, 1998.
[8] D. B. Smythe, ‘‘A two-pass mesh warping algorithm for object transformation and
image interpolation,’’ Rapport Technique, vol. 1030, p. 31, Mar. 1990.
[9] M. Ferrara, A. Franco, and D. Maltoni, ‘‘Face demorphing,’’ IEEE Trans. Inf.
Forensics Security, vol. 13, no. 4, pp. 1008–1017, Apr. 2018.
[15] https://en.wikipedia.org/wiki/Python_(programming_language)
[16] https://code.visualstudio.com/docs/python/python-tutorial
[17] https://djangoforbeginners.com/
[18] https://djangopackages.org/
[20] FaceNet: A Unified Embedding for Face Recognition and Clustering (the original
research paper): https://arxiv.org/abs/1503.03832
[25] Support Vector Machines (SVM) for Face Recognition: A Comprehensive Review:
https://www.mdpi.com/1424-8220/19/20/4362
[28] https://www.researchgate.net/publication/283109274_Face_Recognition_using_Cosi
ne_Distance_Measure_and_Nearest_Neighbor_Classifier
CHAPTER 13
APPENDICES