Objective 0 1 Face Frontalization , DLIB, landmarks, Face recognition Sai Srujan A V V R Siddhartha Engineering College , Kanuru, Vijayawada, 520007 , India 25 30

The technique of synthesizing a frontal image of a face from its non-frontal perspective is known as face frontalization. Face recognitionsystems employ frontalization to improve their accuracy. Even while contemporary approaches in face recognition boast accuracy that in certain situations exceeds that of humans, recognition systems' performance degrades when a profile view of faces is provided as input. Synthesizing frontal views of faces before recognition is one technique to address this problem. The DLIB library is used to create a frontal face outline in this example. The landmarks that are discovered during the procedure are the most important factor in obtaining a frontal facial outline.

 1. Introduction

Due to the advancements in face recognition technology, people are increasingly expecting the system to be used in uncontrolled environments. This refers to the lack of interference from the recognized individual throughout the process. The ability to reliably determine an individual's identity is something that has been achieved with the use of various facial recognition techniques. Although this may not be the most practical application in real time, it will serve as a base for the future models. 1.1.

 Advantages

the domain in the future. 1.2.

2022 Copyright for this paper by its authors. 1.3.

 Scope

The objective of the project is to develop a model which helps in generating an outline of the frontal view of an input image.

There is tremendous scope for this project. This system can be effectively used when there is a need to improve security and helps the police and crimes investigation departments in getting a clear view of the suspect.

 2. Literature Survey

In [ 1 ] they presented a method for synthesis of faces with partial or complete occlusion using a BoostGAN network. It achieves this through a combination of various assumptions and methods.

In [ 2 ] the authors proposed a feature- improving GAN that takes advantage of the inherent mapping between the profile and the frontal face. The resulting module, which is called FeatureMapping Block, can map the features of the profile face to the frontal space. The authors then built a compact module called Feature-Mapping Block that maps the features of the profile face into the frontal space. It is capable of distinguishing the features of profile face from those of ground-true frontal face images.

In the [ 3 ] paper authors describe a multi- degeneration face restoration model called MDFR. It combines a dual-agent generator and a pre-guided discriminator to generate high-quality faces. The paper introduces the 3D-based Pose Normalization Module, which helps guide the learning of face frontalization. It consists of a Face Restoration sub-Net and a Face Frontalization sub-Net.

Here in [ 4 ] the paper presents a dual- attention network that aims to achieve photo-realistic face frontalization. It combines a self-attention-based generator and a discriminator to generate better feature representations.

In [ 5 ] the paper presents a disentangled representation learning-generative adversarial network (DR-GAN) that can learn a discriminative or generative representation. The discriminator provides a set of rules that disentangle the face variation from other face variations, and the code provided to the decoder enables the discriminator to estimate the pose.

In [ 6 ] the paper, the proposed cross-face GAN learns the mapping between the faces in image space. The resulting deep representation is then used to generate the frontal-view faces.

In [ 7 ] the paper proposes a Pose-weighted generation network called PWGAN that learns face pose information from an input image. It uses this information to generate better-looking results.

In [ 8 ] the authors proposed a framework, called Pose-Weight Generative Adversarial Network, or PWGAN, learns face pose information by combining fusion features and pose features. It can generate better- generation effect by learning more about facial features.

This [ 9 ] paper proposes a hybrid method that combines the 3-D model-based face pose normalization with the SDAE deep network. It is performed through three consecutive stages. After the facial landmark points have been aligned, the next step is to feed 2-D images into the 3D pose normalization stage. This step involves estimating the pose generation and fine- tuning the resulting image.

3. Planned Procedure

This is a pictorial representation of the whole process that is being done in the project. 3.1.

Dataset Used • • • • • • •

The dataset is taken from Kaggle.

Link: https://www.kaggle.com/chinafax/ cfpw-dataset Data: Contains images and fiducials Images: 10 Frontal and 4 Profile images of each 500 individuals.

Contains pair information for Frontal-Frontal Verification and Frontal-Profile Verification FF: 10-fold verification for Frontal-Frontal.

FP: 10-fold verification for Frontal-Profile. 3.2.

 Software Requirements

• • • • •

Python Programming language Windows 10 DLIB

3.3.

 Hardware requirements

RAM: 4 GB

Disk Space: 6 GB 3.4.

 Methodology

The model that is built will be given an input image and the image can contain any number of faces in it. All the faces that are present in that image will be identified and they will be identified by the model which are in the side view. Once all the image identification is done then the model starts its process. Firstly, the outline of the identified image will be generated and displayed to the user and then the landmarks on the outline generated are verified and they will be used in order to generate the frontal outline of the input image.

The model can be basically divided into a few modules in which each module will be used and imported into other one and the final module will have an input image and it will generate the outline. The modules are:

The utility module and this module will have a function which is able to obtain the landmarks of the input image using the DLIB library. A NumPy array with the landmark coordinates will be returned. Now the landmarks that are obtained should be frontalized, for that an array or a list of landmark coordinates is taken and the frontalized version of that will be generated using the DLIB annotation scheme. There will be different landmarks and they can be like landmarks for eye centers and the landmarks for the mirrored images and all of these landmarks will be taken into consideration. Standardizes a face shape by taking into account translation, scaling, and rotation. If a template face is provided, the conventional face is changed such that its facial elements are relocated in accordance with the template face. A line drawing of the face shape is generated which is the outline and it is drawn between the facial landmarks which are given as the input and this line drawing is firstly drawn for the side view which is the input image.

A matrix is filled with the landmarks. The matrix will also have weights which are learnt from a large set of facial landmarks so that this is basically the training for the model so that it will be able to correctly identify the landmarks of our input image.

And lastly a module which is the one that will take an input image and it will also import all the above-mentioned functions in the util and also will read the matrices for the landmarks. This module will take a static image will load it and then detects the face and extracts the landmarks using the util function and then the outline of the frontal face is generated using DLIB. And finally plotting of three images will be done one is the input image and the second is the outline of the input image and the third is the frontal outline of the input image.

The proposed system mainly focuses on generating an outline of the frontalized view of an input image with a sideface. The solution uses a library called the DLIB library to generate a perfect outline of the frontalized image. The DLIB library used helps in the detection of the face in an input image and also helps in generation of an outline of the input image.

Although the model produces a reasonable face generation effect in multi-angle circumstances, the generated image of asymmetric faces could be better, and the generated image's sharpness could be better. Super resolution generation models combined with GAN could be a future study direction.

 6. References [1] Qingyan Duan , Lei Zhang, “ Look More into Occlusion: Realistic Face Frontalization and Recognition with BoostGAN” , IEEE Transactions on neural networks and learning systems , 2021 [2] Changle Rong , Ximgming Zhang, Yubei Lin, Feature-Improving Generative Adversarial Network for Face Frontalization , IEEE access, 2020 [3] Xiaoguang Tu , Jian Zaho, Qiankun Liu, Wenjie Ai, Joint Face Image Restoration and Frontalization for Recognition, IEEE Transactions on circuits and systems for video technology,2021 [4] Yu Yin , Songayo Jiang, Joseph P. Robinson, Yun Fu, Dual-Attention GAN for Large-Pose Face Frontalization , IEEE International conference on automatic face and gesture recognition , 2020 [5] Luan Tarn , Xi Yin, Xiaoming Liu, Representation Learning by Rotating Your Faces , IEEE Transactions on pattern analysis and machine intelligence , 2019 [6] Jinjiu Liu , Qingbao Li , Ming Liu, Tongxin Wei, CP-GAN: A Cross-Pose Profile Face Frontalization Boosting Pose- Invariant Face Recognition , IEEE access, 2020 [7] Jiaxin Ma , Feng Zhou, Multi-pose Face Frontalization based on Pose Weighted GAN , IEEEINFORMATION TECHNOLOGY , NETWORKING CONFERENCE , 2019 [8] Yi Zhang , Keren Fu, Cong Han, Peng Cheng, Identity- and -pose-guided generative adversarial network for face rotation , IEEE conference on computer vision and pattern recognition , 2019 [9] Anil Celik , Nafiz Arica, Enhancing face pose normalization with deep learning , SCIENTIFICAND TECHNOLOGICAL REASEARCH COUNCIL OF TURKEY ( 2019 )