Facial landmark detection for face

alignment and recognition

B.Muni Rakesh 41130083

B.Dinesh kumar Reddy 41130071
 AGENDA
• Abstract
• Objective(s)
• Literature Survey
• Inferences from Literature Survey
• Proposed System
• Conclusion
• References
 ABSTRACT
Facial landmark detection is a key component of the face recognition
pipeline as well as facial attribute analysis and face verification. Recently convolutional
neural network-based face alignment methods have achieved significant improvement,
but occlusion is still a major source of a hurdle to achieve good accuracy. • In this paper,
we introduce the Attentioned distillation module in our previous work Occlusion-
adaptive Deep Network model, to improve performance. In this model, the occlusion
probability of each position in high-level features are inferred by a distillation module. It
can be learnt automatically in the process of estimating the relationship between facial
appearance and facial shape. The occlusion probability serves as the adaptive weight on
high-level features to reduce the impact of occlusion and obtain clean feature
representation. • Nevertheless, the clean feature representation cannot represent the
holistic face due to the missing semantic features. To obtain exhaustive and complete
feature representation, it is vital that we leverage a low-rank learning module to recover
lost features. Considering that facial geometric characteristics are conducive to the low-
rank module to recover lost features, the role of the geometry-aware module is, to
excavate geometric relationships between different facial components. The role of
Attentioned distillation module is, to get rich feature representation and model occlusion.
To improve feature representation, we used channel-wise attention and spatial attention.
Experimental results show that our method performs better than existing methods.
 OBJECTIVE(S)
• The objective of this system is to improve facial
landmark detection, particularly in challenging
situations where parts of the face are obscured by
occlusions like masks, sunglasses, or hair. By
enhancing the model's ability to focus on visible
facial features and recover missing details, the
system aims to achieve more accurate and reliable
face recognition, facial attribute analysis, and face
verification, even under difficult conditions.
 LITERATURE SURVEY(1/5)
1. Facial Landmark Detection Algorithm in Complex Scenes
Authors: Haoqi Gao, Xing Yang, Yihua Hu, Haoli Xu, Zhenyu Liang, Bingwen
Wang, HuiqingXi ang, Zhiyang Hu, Shulong Hu / 2024
Content: This paper addresses the challenges of facial landmark
detection in complex environments with varying lighting, large
head poses, and occlusions, which traditional Convolutional
Neural Networks (CNNs) struggle with. The authors propose an
improved approach using joint CNNs to enhance the accuracy of
facial landmark estimation. Their method is tested on
challenging datasets such as Menpo and COFW and
demonstrates superior performance, especially in handling
severe occlusions and large-angle poses.
 LITERATURE SURVEY(2/5)

2. Benchmarking Shadow Removal for Facial Landmark Detection

Authors: Ian Fu, Qing Guo, Felix Juefei-Xu, Hongkai Yu, Yang Liu, wang
Content: This paper investigates the impact of shadow removal on facial
landmark detection. Shadows from light sources can significantly affect
landmark detection accuracy. The authors introduce a new benchmark,
SHAREL, designed to evaluate how shadow removal techniques affect facial
landmark detection. They also propose an adversarial shadow attack method
to test the robustness of systems. The study shows that shadow removal
improves detection accuracy, providing useful insights for enhancing systems
under shadowed conditions.
 LITERATURE SURVEY(3/5)

PyraMoT: A Novel Framework for Enhanced Facial Thermal Landmarks

Detection

Authors: Kais Riani, Salem Sharak, Mohamed Abouelenien / 2024

Content: This paper presents PyraMoT, a novel framework designed for
detecting facial landmarks in thermal images. It utilizes the D5050 dataset
with extensive annotations and combines a customized encoder-decoder
network with Feature Pyramid Networks and MobileNetV2. The proposed
method outperforms existing techniques in thermal facial landmark
detection, offering enhanced performance and greater robustness.
 LITERATURE SURVEY(4/5)

Exploring New Frontiers in Facial Expression Recognition: Dual DenseNet-

201 and Landmark Distance Analytics in the Wild

Authors: Abdullahi Mohamed Hassan, Xiaojun Zhou, Laeeq Aslam / 2024

Content: This paper introduces a method for facial expression recognition
(FER) that integrates landmark distance data with traditional facial image
analysis. The authors use dual DenseNet-201 models to extract features from
both the facial images and distances between landmarks. These features are
combined using a multi-head attention mechanism within a transformer
model. This approach shows significant improvements in recognizing "Sad"
expressions and demonstrates its potential for applications in psychology,
surveillance, and interactive systems.
 INFERENCES FROM LITERATURE SURVEY
• Existing facial landmark detection systems, particularly those
relying on traditional image processing techniques like Active
Shape Models (ASM), perform well in controlled settings but
struggle with real-world challenges.
• Facial occlusions, such as glasses or masks, lead to reduced
accuracy and hinder the detection of key features like the eyes,
nose, and mouth.
• This limitation underscores a critical gap in the system's
ability to handle partial obstructions, compromising its
reliability.
• To address these shortcomings, advanced solutions capable of
robustly detecting facial landmarks in diverse and occluded
environments are needed
 PROPOSED SYSTEM (1/5)
• The proposed system improves facial landmark detection using an
Occlusion-adaptive Deep Network model, which enhances face
alignment and recognition even when parts of the face are covered by
masks, glasses, or other objects. This advanced model addresses the
limitations of existing systems by incorporating a specialized module
that adapts to occlusions, allowing the system to focus on visible
facial features and minimize the impact of hidden areas. It also uses
sophisticated techniques to better capture and represent facial
landmarks, ensuring accurate detection despite partial obstructions.
Overall, the model offers a more reliable and precise solution for real-
world applications where faces are frequently partially covered or
viewed from different angles.
Advantages of Proposed System
• Works with Different Angles
• Consistent Performance
• More Accurate
PROPOSED SYSTEM(2/5)
System Architecture
 PROPOSED SYSTEM(3/5)
• Input data acqusition system
• Preprocessing
• Video to fps
• FPS image processing
• segmentation
• Classification
Results and Discussion (1/3)
 Results and discussion
• 1 st module Face Detection: Locate the face, note the
coordinates of each face locate,d and draw a
bounding box around every faces.
• 2nd module Face Alignments. Normalize the faces in
order to attain fast training.
• 3rd module Feature Extraction. Local feature
extraction from facial pictures for training, this step is
performed differently by different algorithms.
• 4th module Face Recognition. Match the input face
with one or more known faces in our dataset.
 CONCLUSION
The Occlusion-adaptive Deep Network (ODN) model represents a
significant advancement in facial landmark detection. By effectively handling
occlusions, enhancing feature representation, and recovering missing details,
the ODN model provides more accurate and reliable facial landmark
detection, even when parts of the face are obscured. This improved accuracy
makes it particularly valuable for real-world applications where faces are
often partially covered or viewed from different angles. Overall, the ODN
model offers a robust solution for face alignment and recognition, surpassing
previous methods in its ability to handle challenging conditions and deliver
consistent results.
 REFERENCES
1. J. Deng, J. Guo, E. Ververas, I. Kotsia and S. Zafeiriou, "Retinaface: Single-shot multi-level face
localisation in the wild", Proceedings of the IEEE/CVF Conference on Computer Vision and
Pattern Recognition, pp. 5203-5212, 2020.
2. G. Haoqi and O. Koichi, "Improvements over Coordinate Regression Approach for Large-Scale
Face Alignment", IIEEJ Transactions on Image Electronics and Visual Computing, vol. 10, pp.
127-135, 2022.
3. H. Jin, S. Liao and L. Shao, "Pixel-in-pixel net: Towards efficient facial landmark detection in the
wild", International Journal of Computer Vision, vol. 129, pp. 3174-3194, 2021
4. Y. Xu, W. Yan, G. Yang, J. Luo, T. Li and J. He, "CenterFace: joint face detection and alignment
using face as point", Scientific Programming, pp. 1-8, 2020
5. S. Gong, X. Liu and A. K. Jain, "Jointly de-biasing face recognition and demographic attribute
estimation", European Conference, pp. 330-347, 2020
6. R. Valle, J. M. Buenaposada and L. Baumela, "Cascade of encoder-decoder CNNs with learned
coordinates regressor for robust facial landmarks detection", Pattern Recognition Letters, vol.
136, no. 136, pp. 326-332, 2020
7. J. Deng, G. Trigeorgis, Y. Zhou and S Zafeiriou, "Joint multi-view face alignment in the wild",
IEEE Transactions on Image Processing, vol. 28, pp. 3636-3648, 2019.
8. F. J. Chang, A. Tuan Tran, T. Hassner, I. Masi, R. Nevatia and G. Medioni, "Faceposenet: Making
a case for landmark-free face alignment", Proceedings of the IEEE International Conference on
Computer Vision Workshops, pp. 1599-1608, 2019.
THANK YOU