Abstract

In colleges and school's teachers take attendance or students have to sign on to register to mark their presence. Due to human errors, many times students face problems while getting attendance. This paper portrays a system that is capable of storing the information of students by facial recognition of the students present in the class. In this paper various machine learning and deep learning algorithm has been applied like Support Vector Machine (SVM), Decision Tree (DT), and Convolutional Neural Network (CNN) along with some pre-trained models like VGG19 and ResNet50 which is provided by karas application. The model gives a satisfactory result with an accuracy of 96.82 % when applied on the CNN model and gave 96.97 % accuracy when applied on the ResNet50 pre-trained Model. After that the best model has been saved for system implementation based on their performance and applied the detection technique for detect the face of student and maintain their present in the excel sheet.

1. Introduction

There are diverse biometric frameworks dependent on face recognition, iris, fingerprint, palm-print, etc. yet in the most of the cases, facial recognition is utilized as a prominent innovation [1]. The technology which is capable of matching human faces with digital images comes under facial Recognition. Facial Recognition is used in various fields like Security checking, authentication systems in offices, creating databases for various identification documents. It is used in mobile phones and digital cameras for taking better pictures. It is also used in various social networking sites where people upload pictures and edit them [2]. The system mentioned in this paper analyzes different pictures, store it in a database which can further be extracted for Attendance purposes in schools and colleges during live lectures. It will reduce human efforts and human errors. In this system, Attendance is recorded when a human face is detected in the camera and is matched with the database [3]. Machine Learning and Deep Learning are a part of Computer Science which creates a system by learning methods from various sample of data and behaving like a trained Model. The features of the image are the input data. It represents the behavior of the image [4]. 2. Literature Survey

Nirmalya Kar [5] has used two main components for implementation approach computer vision library (OpenCV) and Fast Light Tool Kit (FLTK). One of Compute vision provides a real-time object detection Computer vision library. It also used in machine learning for object detection. And another one FLTK is used for graphical user interface which is developed by Bill Spitz.

2020 Copyright for this paper by its authors.

Sujata G. Bhele [ 6 ] In this paper mostly worked on the machine learning and deep learning models like SVM, ANN, CNN etc. which can help the model to perform the better result, here best model has chosen which can give true results. This paper explained various features extraction technique or algorithm like PCA, LDA etc. In this paper some other techniques have used which has normalize the size of image that is mostly affected the accuracy result.

Riddhi Patel [ 7 ] has explained the summary of the face recognition and discusses the techniques and their working flow. It is also finding the differences between some face recognition models. It highlights the methods which has given good result as compares to other.

Dwi Sunaryono [ 8 ] has given different approaches about the attendance system based on face recognition. In this paper has explained about the how these types of system can helpful in different sectors like company employment, schools and colleges to avoid the mistakes. It has mostly focused on the school or colleges in which most of students has getting their presents by proxy.

Shireesha Chintalapati [ 9 ] has proposed a brief summary about face recognition system and discussed some dimensionality and machine learning algorithm like PCA, LDA for feature extraction or dimensionality reduction and SVM, Decision tree machine learning algorithms. In this paper also mentioned some errors which mentioned the student details who has not present in the class which suggest that add more features to this type of system.

Jomon Joseph [ 10 ] has proposed the brief summary of the face recognition and explained the methods of their working. In this paper the dataset collected by using mobile camera. And used algorithms like SVM, CNN and some features extraction techniques like PCS. In this paper also given some information about MATLAB which is the multi-paradigm programming language and numeric computing environment, which created by MathWorks. MATLAB gives the platform to plotting of various work like functions, data, implementation of algorithm, creation of user interface etc.

3. Data Description

The dataset used in this paper has been prepared through collection of the images of students and some images have been collected from the internet. And after that work has move forward to separate the students’ images in two database folders. Based on the student who is the member of that class and who is not the member of that class and all the images renamed with their name and registration which help the system to confirm the student attendance according to their name and registration number. In dataset the data has different size, shape and colour which can create the difficulties to machine for resolving these types of problem the data has pre-processed in the upcoming parts of working direction [ 11 ].

3.1 Decision Tree Methodology

Decision tree is a supervised machine learning algorithm or technique which can be used for solving both types of problem like classification and regression, but it is moreover preferred for classification. This is one of the easiest and most versatile structures for classification problems. It is fundamentally a tree of decisions that create nodes where branches are the split of the tree. Each node along with the sub-nodes is a decision relying on values of defined variables which end with the classification of every element into any one of the classes. The root is the first variable that divides the dataset and from here everything starts. Every decision is known as the node and the line which connects these decisions are called the branch [ 12 ].

3.2 Support Vector Machine Methodology

SVM is a Supervised Machine Learning Algorithms which can be applied in both types of problem like Classification as well as Regression problems. In majority of cases, it is used for classification problems under Machine Learning.

SVM’s main motive is to create an appropriate line or which can identify n-dimensional space into classes which allow us to keep the new data point under appropriate category later. It handles various linear and non-linear problems and works perfectly on practical problems [ 12 ]. The best limit choose is termed as Hyperplane, which divides the data into classes. Data is applied on sigmoid kernel of SVM algorithm.

3.3 Convolutional Neural Network Methodology

Convolutional neural networks, i.e., ConvNets were first comes to existence in the 1980s. CNN is composed of multiple layers of artificial neurons [ 13 ]. The behavior of each neuron is decided by its weights. It is a classification structure for classifying images into various labeled classes [ 14 ][ 15 ]. The different layers of CNN take out image features and learn to categorize the images. It is one of the types of feed-forward neural network in DL and AI. CNN has the capability to extract the each and every portion of input image, which is known as by name receptive field. It is assigning the weights for each neuron based on the significant role of the receptive field. So that it can be discriminate or find the difference between the importance of neurons from one another [ 12 ].

Figure-3: Convolutional Neural Network Methodology

3.4 VGG-19 Methodology

VGG stands for Visual Geometry Group. VGG-19 is a type of convolutional neural network that has19 layers deep. It is a type of VGG model which contains the 16 convolution layers, 3 fully-connected layer, 5 MaxPooling layers and 1 SoftMax layer. It can be used for the facial recognition purposes. Its weights are easily available along with other frameworks like karas [ 16 ].

3.5 ResNet-50 Methodology

ResNet50 is the type of Resnet model of karas which contains 48 convolutional layers along with 1 MaxPooling and 1 Average Pooling layer. It has 3.8 x 10^9 Floating points operations. This is the most usable Resnet model. It can be also used for computer vision tasks like classification of images, localization of objects, and detection of objects [ 13 ]. This framework can also be applied to non-computational vision tasks to reduce the computational expenses and give them the benefit of depth.

4. Proposed Methodology

ii.

The methodology proposed here for facial recognition and storing the attendance is entirely based on machine learning algorithms. States the flow diagram of the working of our system.

Data Preprocessing

In data pre-processing step the dataset has imported to the working directory with the help of python library i.e., pandas, OS and OpenCV. Then the data has combined or converted to a list directory in which the both dataset students image data and other members who is the not the part of that class has concatenated with the help of OpenCV function [ 17 ]. i. Face Cropping

The main aim to applying the face cropping is that the images contain the face as well as the other part. By using this we can crop the face for system use that can give the better result without interruption any kinds of this type issues. The face cropping has performed in this paper by using OpenCV python package which have the capacity to handle this type of problem [ 13 ].

Image Reshaping iii.

In this part of paper, the image has collected in different shape by using different types of cameras. So, here all the images have converted to equal shape with (224 X 224) resolution which is also suitable for karas application pre-trained model for better result. And after reshaping the whole images has converted to 1-Dimensional array which can be easy to work with the machine learning model [ 18 ].

Remove Noise The proposed on this step to remove the noise from the images that means reduce the noise like common cause, white noise, remove brightness and colour information in images that can visualize easily and make smooth to use in machine learning for better performance.

Feature Extraction

The proposed of this process to extract the features from the image which is most useful for the model implementation the features of any human image like nose, eyes, hear, ear etc. the new reduced data will be have the capability to summarize the most of the features of that image after features extraction. In machine learning there are different feature extraction techniques available like PCA, LDA, ICA etc. [ 19 ].

Splitting Data

In the step of splitting the data has slitted or divided for training and testing format. In training we are train the model by using most of the features that make the system to work with testing data for good result if our machine learning model learns in better way then it may perform good that is more generalize and ability to solve the problem statement. Mainly 70 to 80 percent of data has taken for training the machine and 20 to 30 percent of data has taken for testing the performance and work criteria of the machine [ 20 ].

5. Proposed Work

Here in this paper has loaded the model like SVM, Decision Tree, CNN, VGG19 and ResNet50. And fit all the model one by one using training dataset. Which has taken 75% of data for train the model and 25% data for testing purpose which can evaluate the work performance on the basis of test data.

Result Analysis

In the result analysis the paper has find the confusion matrix of their performance by using test data set and accordingly find the precision, recall, accuracy and f1-score. And compare the result that is predicted by different model and find which model has given better result as compare to another model.

In this paper the support vector machine (SVM) has given 56.70% of accuracy result, Decision tree has given 71.89% of accuracy result, convolutional neural network has given 96.82% of accuracy result, VGG19 which is a pre-trained model has given 96.60% of accuracy result and ResNet50 is also a pre-trained model has given 93.97% of accuracy result [ 14 ].

As the graph shows CNN gives the best F1 score and accuracy with96.82% and 89.00% respectively followed by VGG19 with 96.60% and 78.43% respectively as compared to other models. SVM gives the least result with 56.70% accuracy and 48.89% F1 score.

After comparing the result, the model has been saved for next level implementation for check the working of the project. Here the paper has work with video processing by using OpenCV which will open the virtual camera and detect the face of students [ 19 ]. If there the student is the member of that class according to that the system will display the name and registration number of student and puts their attendance present in the .csv file [ 21 ].

Table 1 Result Analysis

SL No. Model Accuracy F1-Score

1 2 3

Support Vector Machine

Decision Tree

CNN 56.70% 71.89% 96.82% 48.89% 64.20% 89.00% 96.60% 93.97%

Model Accuracy and F1-Score

6. Conclusion

The system proposed in this paper uses OpenCV Computer Vision along with various machine learning and deep learning technique. Facial detection carried out in this paper is done by using OpenCV. It will reduce human errors and will help in reducing human efforts. The collected data is of students which were applied for training and testing purposes. When tested on different models like SVM the accuracy that we got was 56.70% and the F1 score for this machine learning model was 48.89%. When the data was applied on Decision Tree classifier the accuracy was 71.89% with the F1 Score of 64.20%. When the data was applied on CNN it gave a satisfactory result of 96.82% with a very good F1 score of 89.00 %. When the same data was applied on some pre-trained models like Vgg19 and ResNet50, we got the accuracy of 96.60% and 93.97% respectively along with F1 score of 78.43% and 75.78% respectively. In this paper after the implementation in practical use the attendance data can be extracted in the form of .csv file that helps in managing attendance record of the students as per their presence in the class on a daily basis. So, through this result it can be stated that the paper can be used for attendance purpose in various Schools and colleges. Further the paper can be developed for use in employment sector for creating records.

7. References

[1] Samet, Refik, and Muhammed Tanriverdi. "Face recognition-based mobile automatic classroom attendance management system." 2017 International conference on cyberworlds (CW). IEEE, 2017. [2] Patel, Unnati A., and S. Priya. "Development of a student attendance management system using RFID and face recognition: a review." International Journal of Advance Research in Computer Science and Management Studies 2.8 (2014): 109-119. [3] Bah, Serign Modou, and Fang Ming. "An improved face recognition algorithm and its application in attendance management system." Array 5 (2020): 100014. [4] Jayant, Nazare Kanchan, and Surekha Borra. "Attendance management system using hybrid face recognition techniques." 2016 Conference on advances in signal processing (CASP). IEEE, 2016. [5] Kar, Nirmalya, Mrinal Kanti Debbarma, Ashim Saha, and Dwijen Rudra Pal. "Study of implementing automated attendance system using face recognition technique." International Journal of computer and communication engineering 1, no. 2 (2012):100.

References

[6] Sujata G. Bhele and V. H. Mankar “ A Review Paper on Face Recognition Techniques” published in International Journal of Advanced Research in Computer Engineering & Technology (IJARCET).

[7] Riddhi Patel and Shruti B. Yagnik “ A Literature Survey on Face Recognition Techniques” published international Journal of Computer Trends and Technology (IJCTT).

[8] Sunaryono , Dwi, Joko Siswantoro , and Radityo Anggoro . "An android based course attendance system using face recognition . " Journal of King Saud University-Computer and Information Sciences 33.3 ( 2021 ): 304 - 312 .

[9] Chintalapati , Shireesha, and M. V. Raghunadh . "Automated attendance management system based on face recognition algorithms . " 2013 IEEE International Conference on Computational Intelligence and Computing Research . IEEE, 2013 .

[10] Joseph , Jomon, and K. P. Zacharia . "Automatic attendance management system using face recognition." International Journal of Science and Research (IJSR) 2 .11 ( 2013 ): 327 - 330 .

[11] Andric , Milan , et al. "Web application as a support system for records of working time, monitoring business processes and activities of company employees."

[12] Jianke Li , Baojun Zhao , Hui Zhang, Jichao Jiao, “ Face Recognition System Using SVM Classifier and Feature Extraction by PCA and LDA Combination” , IEEE 2009 .

[13] Minjun Wang , Zhihui Wang , Jinlin Li , “ Deep Convolutional Neural Network Applies to Face Recognition in Small and Medium Databases” , 4th International Conference on Systems and Informatics , IEEE 2017 .

[14] Russakovsky , Olga , et al. "Imagenet large scale visual recognition challenge." International Journal of Computer Vision 115 .3 ( 2015 ): 211 - 252 .

[15] S. Chakravarty , B. K. Paikaray , R. Mishra and S. Dash , "Hyperspectral Image Classification using Spectral Angle Mapper," 2021 IEEE International Women in Engineering (WIE) Conference on Electrical and Computer Engineering (WIECON-ECE ), 2021 , pp. 87 - 90 , doi: 10.1109/WIECONECE54711. 2021 . 9829585 .

[16] Pratama , Yohanssen , et al. "Face recognition for presence system by using residual networks-50 architecture." International Journal of Electrical & Computer Engineering (2088-8708) 11.6 ( 2021 ).

[17] Bradski , Gary, and Adrian Kaehler . Learning OpenCV: Computer vision with the OpenCV library . “ O'Reilly Media , Inc." , 2008 .

[18] P. Wagh , S. Patil , J. Chaudhari and R. Thakare , "Attendance System based on Face Recognition using Eigen face and PCA Algorithms," in 2015 International Conference on Green Computing and Internet of Things (ICGCloT) , 2015 .

[19] Qingdong Liang , Wenting Fang, “ College Student Attendance System based on Face Recognition” , IOP Conf. Ser.: Mater. Sci Eng . 466 012107 , 2018 .

[20] Li Cuimei , Qi Zhiliang, Jia Nan, Wu Jianhua, “ Human face detection algorithm via Haar classifier combined with three additional classifiers” , 13th IEEE International Conference on Electronics Measurement & Instruments (ICEMI) , 2017 .

[21] Raghuwanshi , Anshun, and Preeti D. Swami . "An automated classroom attendance system using video-based face recognition . " 2017 2nd IEEE International Conference on Recent Trends in Electronics, Information & Communication Technology (RTEICT) . IEEE, 2017 .

[22] Mohanty , S. N. , Chatterjee , J. M. , Jain , S. , Elngar , A. A. , & Gupta , P . (Eds.). ( 2020 ). Recommender System with Machine Learning and Artificial Intelligence: Practical Tools and Applications in Medical, Agricultural and Other Industries . John Wiley & Sons.

[23] Jain , S. , Sharma , R. K. , Aggarwal , V. , & Kumar , C. ( 2021 ). Human Disease Diagnosis Using Machine Learning . In Intelligent Data Communication Technologies and Internet of Things (pp. 689 - 696 ). Springer, Singapore.