Iris recognition algorithms have recently demonstrated excellent performance in the authentication task. In this paper, we present a technology transfer project for the development and testing of a biometric recognition system under challenging operational conditions. Due to the stringent operational requirements, the design and implementation of the system included a phase of selecting technologically advanced hardware. The lack of corresponding data sets implied a novel acquisition step. The evaluation phase is preliminary as the data set is being expanded for the acquisition of new samples capable of highlighting the system's critical issues. Current samples were acquired in very diferent lighting conditions and in the presence of glasses, which was not yet done in the literature. In addition to the selected hardware, such data allowed us to simulate a realistic environmental context for the project's ifnal prototype.
In this paper, we present the implementation of a prototype for personal recognition achieving high-level performance under such conditions. The project was carried out under commission of the MBDA S.p.A. company. The recognition/authentication system was required to be capable of operating with a high degree of reliability and safety, i.e. returning a limited number of false positives and false negatives and robust to the possibility of spoofing. The iris was chosen as the biometric trait. Iris recognition is nowadays considered a highly reliable and robust method for authentication due to its high uniqueness and invariance over time [1]. However, some application contexts, such as the military ones, may represent “hostile” operating conditions for many biometric systems that may impact on the properties above [2]. As a matter of fact, the intended use case for the development of the prototype introduced some limitations, in particular: the need for the fastest and most accurate recognition rate possible, the occlusion of a large part of the body due to gloves, goggles, balaclava, etc., hostile environmental factors such as lighting and noise and users under emotional stress.
In these stringent contexts of use, we considered the iris as the most suitable thanks to its unique and stable characteristics [3]. The features contained in the iris are, in fact, unique and are diferent even in genetically identical individuals. Furthermore, iris-based recognition systems are characterized by a very high accuracy, a high dificulty in creating artificial replicas, and the requirement for a small area of the body to be uncovered [4].
To comply with the challenging use case above: (i) we selected of the most suitable hardware for stable and fast acquisitions, also requiring little user collaboration; (ii) we selected the most appropriate feature sets and matching system according to the expected user population size; (iii) we captured a novel dataset, called “DIEE Iris” in the following. This data set was acquired according to the operating conditions previously described. Therefore, it allowed to evaluate the accuracy and robustness of the system in relation to the use case and in a realistic application context.
These steps are explained in Sections 3-6, while Section 2 briefly illustrates the recent literature on iris recognition. Section 7 concludes the paper.
An iris-based recognition system relies on the uniqueness of each individual’s iris patterns.
Starting with the image acquisition of the entire region of the eye, the main pre-processing steps are: segmentation, which isolates the region of interest through pupil identification, and normalization, which represents the iris in a dimensionally consistent manner [5]. These steps are followed by the feature extraction phase, although some recent methods avoid it [6].
The segmentation phase is of fundamental importance due to its impact on the final system’s performance. Incorrect segmentation leads to the introduction of noise into the image and modifies the iris’ discriminative patterns. Roughly speaking, segmentation consists in detecting the Region of Interest (ROI) for the subsequent normalization step, namely, the pupil’s border. Over the years, many techniques for locating the iris and the pupil have been explored. These can be divided into handcrafted techniques and deep-learning based techniques. Handcrafted techniques, i.e. traditional segmentation techniques, are often based on the search for edges and circular geometric structures. Among these, the most used are the Daugman technique and the Circular Hough Transform [7]. Since 2017, deep-learning techniques have spread for this pattern recognition task, proving to be particularly accurate [8].
Normalization is the technique of producing fixed dimension iris images by remapping images from Cartesian coordinates to polar coordinates [9]. This phase allows to minimize the diferences due to the dilation and contraction of the pupil.
The feature extraction is the final step in the iris recognition process and it aims to create a compact and representative description of the biometric trait’s characteristics, which will be compared with those stored in the template gallery. Among the most used methods for feature extraction are [10]: Gabor filters (linear filter used for texture analysis), Discrete Wavelet Transform (DWT), Discrete Cosine Transform (DCT), Haar Wavelet Transform (HWT), Local Binary Pattern (LBP) and Principal Component Analysis (PCA). Using the aforementioned segmentation, normalization, and feature extraction methods, the accuracy achieved on stateof-the-art publicly available datasets ranges from 86.67% to 100%.
In recent years, deep learning approaches, such as Convolutional Neural Networks (CNNs) [11], have also been applied to iris classification, achieving significant improvements. Also in this case, the system starts from a dimensionless representation of the iris data, obtained from normalization, and through a deep neural network, a feature vector is obtained, which lies in a hyperspace where the recognition is performed. However, in addition to being computationally complex, deep learning models are sensitive to image contamination and training data scale, posing a challenge to real-time iris authentication [12].
The iris recognition system was implemented to find a good compromise between accuracy and speed of use under the conditions explained in Section 1.
While the system’s architecture is standard for such a recognition task (Fig. 1), our contribution focused on evaluating state-of-the-art techniques for an accurate and fast system, robust to challenging application conditions, such as lighting changes, and occlusions in the user’s appearance. Since ergonomics, recognition speed, minimal invasiveness, acquisition stability, and robustness to changes in lighting were also required, the choice of the sensor was fundamental for obtaining a reliable system.
In the following, we summarized the choices for implementing the prototype made after the hardware and software state-of-the-art analysis.
- Capture device. After analyzing the characteristics of the technologically advanced sensors on the market, the DUAL IRIS IMAGER CMITECH EMX-30 [13] was selected for the prototype. The CMITECH EMX-30 sensor is a dual iris scanner that captures the iris and face images without physical contact. The scanner can adjust to the position of the subject and automatically tilts the sensor part according to the height of the subject. The following characteristics can be highlighted: (i) minimally invasive, as it works completely hands-free, (ii) ultra-high-speed double iris capture, 30 frames per second and (iii) robustness to changes in lighting, as the eye is acquired with the near-infrared (NIR) technology.
The iris images are acquired at about 35 cm from the face and it is possible to place the sensor in a fixed position so that it is the user to approach for the acquisition. Some other sensors, in fact, require to be held by the user resulting in a less comfortable and stable operation. (a) (b) (c) (d)
- Pre-processing. Acquired images are processed to extract the region of interest (ROI), the iris itself in this case, and remove any noise (such as eyelashes and eyelids). Finally, the segmented images are normalized from their polar coordinates to the spatial ones. This last step allows us to obtain fixed-size images, independently of the size of the segmented ones.
Among other iris segmentation techniques, we selected IrisParseNet [14], a pre-trained convolutional neural network for the recognition of the iris region. The output of the network consists of three diferent images: the outer edge of the iris, the pupil, and the inner region iris. These three images are binarized and the largest connected component of each image is selected. Finally, the external and internal edges are approximated using the least-squares circle iftting algorithm, starting from the pixels that were highlighted in the mask. In 2 we have an example of the segmentation, noise removal, and normalization processes.
This method was selected as particularly robust with respect to non-perfectly circular iris shapes, determined by partial iris occlusion. This aspect is decisive in the choice of the segmentation algorithm: in fact, an approach that does not recognize the iris if the eye is squinted results in many segmentation errors in a dynamic and non-collaborative acquisition scenario. These errors carry over to the later stages of the system, resulting in many false negatives. Among state of the art techniques, we also analyzed Daugman’s Integro-diferential Operator and Circular Hough Transform. The first exploits the fact that the diference of the illumination between internal and external pixels to the edge of the iris is maximum, so it behaves as a circular edge detector. The second one is used to isolate features of various shapes within an image, in this case, a circular shape. These two methods, however, have little robustness with respect to not perfectly circular shapes, consequences of the partial occlusion of the iris. This is why our choice fell on IrisParseNet.
- Feature extraction. Features are extracted from the normalized images through specific algorithms, obtaining a feature template for each image. The goal is to obtain a sort of code weakly variable with environmental conditions, and such that the iris’s uniquess is pointed out. This required a trade-of between the code lenght and the user population size.
Specifically, two feature extraction techniques were tested during our experiments: the first is based on Gabor’s filters [ 15] and the second one on the Binarized Statistical Image Features (BSIF) [16]. Both these algorithms have proven to be very efective in describing the uniqueness of iris patterns that characterize the individual; additionally, with the advantage of low computational complexity. Therefore, they fully met the requirements for the project implementation.
A general Gabor filter can be described by this equation: is the wavelength, is the filter’s orientation, is the cosine factor phase, is the parameter that regulates the envelope and specifies the ellipticity of the Gabor function support. On the basis of the state-of-the-art results [17], we decided to adopt a particular variant of such a filter: ( ) = − (( /0))2 )︂ ︂(
2(/ 0))2 Where 0 represents the central frequency of the filter and represents a scale factor. We then applied a convolution between the filter and the segmented image. This operation has been executed individually for each row of pixels, obtaining as output a filtered image like the one in 3.
BSIF involves the construction of a representation of the image, using particular local descriptors [18]. The general algorithm can be described by the following equation: where: ′ = ( ) + ( ) and ′ = − ( ) + ( ) = ∑︁ (, )(, ) =
, where represents the response of the filter when applied to a portion of the image of the same size. This response is binarized considering , equal to 1 if > 0 or equal 0 otherwise.
BSIF computes a binary code string for the pixels of a given image. The value of the code of a pixel is considered as a local descriptor and can be used to build histograms to characterize texture properties within the sub-regions of the image. Each bit in the binary code string is computed by binarizing the response of a linear filter with a zero threshold. The image is divided into non-overlapping regions and a descriptor is calculated for each of these regions. Finally, the descriptors are concatenated for a global description of the iris.
- Matching. During the recognition phase, the input sample is compared with the templates present in the gallery and acquired during the enrolling phase.
As a distance metric, we selected the Hamming Distance [19]. The Hamming distance can
be defined as the number of diferences between the template and the input feature vector,
evaluated through a bit-to-bit XOR operation. In principle, fewer are the diferences, smaller the
distance will be. As shown in the following equation, each element from the first feature vector
is compared to each element from the second feature vector . Then, we sum the number of
diferent elements and extract only the ones that are outside the noise mask:
(
In our case, for each couple, we perform multiple comparisons by shifting the feature vector to the right at each iteration, in order to compensate for possible rotations of the eye [20]. At the end of the procedure, the selected distance will be the smallest one among all the comparisons and it will be normalized between 0 and 1. We chose this procedure as a trade-of between the accuracy and the execution speed requirements.
- Dual iris recognition. Our system includes the option to exploit the information related to both eyes [21]. To this aim, we selected a score level Min after Mean approach [22].
The fusion is carried out at the score level [23]: for each user’s eye, we computed the Hamming distance between it and the others present in the database. Then, we calculate the average value among those distances. We finally select the smallest value.
The minimum requirements identified for this system are a CPU Intel Core i3 and 12 GB of RAM.
According to the project requirements, the final personal recognition system must work under stress conditions, including uncommon application contexts where the user cannot fully cooperate. For this reason, we acquired the DIEE-Iris dataset, explicitely designed to perform this analysis.
As reported in Section 3, we adopted the DUAL IRIS IMAGER CMITECH EMX-30 [13] as capture device for the whole system. This sensor is a dual iris scanner that has been chosen over others because it allows capture without physical contact. It is a very user-friendly sensor because it provides LED-based feedback to the user to help him/her find the correct distance for the acquisition. The sensor acquires images at a distance from the subject of about 35 cm. These details are critical in our application context, which requires fast authentication with the least amount of user efort, such as military applications.
Each acquisition included two images, one for the left eye and one for the right eye of 640 x 480 pixels each. At the moment of this paper writing, the number of acquired users is 31, for a total of 2480 images. However, more subjects will be collected to reach a minimum size of 100 users. Worth noting, this number is comparable with that of existing datasets, such as MMU, MMU2, and UTIRIS [24, 25], whose images were captured with outdated sensors.
The acquisition protocol consisted of four diferent lighting conditions and includes a total of 10 acquisitions for each condition: (i) difused artificial light, (ii) artificial light - illumination from the right, (iii) artificial light - illumination from the left and (iv) difused natural light. With regard to other environmental conditions, we concluded that wearing a bataclava was not a real obstacle for the selected sensor, after a preliminary simulation with some members of the laboratory staf. As a matter of fact, the sensor focused on the eyes only, and in particular on the pupil, as shown in Fig. 4. Therefore, acquisitions wearing such dress were not taken into (a) (b) (c) (d) (e) account. The application context of this experiment foresees the possibility that the user wears glasses and cannot move them for the acquisition. For this reason, for five users, 10 captures per eye were added while wearing eyeglasses. Moreover, to evaluate the robustness to critical variations of ambient lighting for two users we acquired iris in an environment with blue and red artificial lights as shown in Figure 5. Finally, some captures were made by asking users to squint (Fig.4-d). These acquisitions have allowed us to simulate a real application context under particular lighting conditions or health states, thus replicate a possible crisis situation.
The following meta-information has been saved for each image: the (i) user ID, (ii) left or right eye, (iii) acquisition number, (iv) gender, (v) age, and (vi) lighting condition. Beside the DIEE-Iris data set, we tested the performance on three publicly available ones: - MultiMedia University (MMU) [24], made up of 5 images of the right and left irises of 46 subjects for a total of 460 images captured by the LG EOU 2200 iris acquisition sensor. This sensor allows to acquire irises at a distance of 7-25 cm in the NIR spectrum.
- MultiMedia University v2 (MMU2, second version of MMU) [24] , made up of of 995 images captured using the Panasonic BM-ET100US sensor at a distance of 47-53 cm from the subject. Also in this case, the images were acquired in the NIR spectrum. In particular, images of the irises of 100 diferent subjects were collected (5 images per eye).
- University of Tehran IRIS (UTIRIS) [25]: composed of 1540 images of 79 subjects (including both eyes). The dataset includes both images in the visible spectrum, acquired using an RGB camera (Canon EOS 10D) and in the NIR spectrum (ISG Lightwise LW). The sensor-to-subject distance was not specified for this dataset.
These datasets, especially MMU2 and UTIRIS, are particularly challenging as they lack uniformity in the acquisition protocol regarding pose and distance. In fact, acquisition technologies have evolved in recent years and modern sensors support the user in positioning in front of the scanner.
In order to carry out an exhaustive analysis that would consider all possible use cases, four experimental protocols were identified.
• Closed set recognition protocol. The system attempts to idenfity the registered user by comparing his/her irises with all those present in the database. Specifically, 10 test images for each user were compared with as many images of each registered user. This protocol was used to test the system on state-of-the-art datasets described in the previous Section. In addition, the DIEE-Iris was adopted for a more realistic simulation of the challenging application context. This protocol was aimed to focus on the misclassification occuring into a set of subjects generally authorized to the system access, but with diferent levels of access permission. For example, what if the system misclassifies the administrator with a simpler user? In other words, it simulates the minimal level of security that a recognition system must assure at an acceptable rate for the application. The secondary goal was to identify the most suitable feature set for the task between Gabor and BSIFs. • Open set recognition protocol. The system attempts to identify authorized users from unauthorized ones (impostors) by comparing their irises with all those present in the database. Specifically, 10 test images for each user are compared with as many images of each registered user. Of the 31 users in the contextualized dataset, nine are excluded from the system and used as impostors during the testing phase. In this case, the goal is to evaluate the misclassification rate of the impostors and detect those authorized subjects that may potentially represent a security breach if highly confused with impostors. • Authentication protocol. The system verifies the identity declared by the user by comparing his irises with all those present in the template gallery corresponding to the declared ID. Each user declares his identity plus nine other identities. This protocol is much more stringent and allows detecting those users weakly characterized by the selected feature set, thus requiring more than one template to be correctly recognized at an acceptable rate. Specifically, 10 test images for each user are compared with the template gallery belonging to the declared ID. We have considered a template gallery made up of ten images. In a variant of this protocol, the system verifies the claimed identity by comparing the subject’s iris(es) with the related unique acquisition in the template gallery. Specifically, ten test images for each subject are compared with one image belonging to the declared ID. Each user declares his identity plus nineteen other identities. This protocol represents the worst-case scenario, characterized by low or compromised storage resources.
Table 1 shows the comparison numbers genuine-genuine, other registered user-genuine and impostor-genuine for each experimental protocol.
The selected performance parameters are well-known in biometric recognition [26]: (i) False Reject Rate (FRR): percentage of authorized users, incorrectly recognized as impostors. (ii) False Accept Rate (FAR): percentage of impostors, incorrectly recognized as authorized users. (iii) True-Positive Identification Rate (TPIR): percentage of properly recognized authorized users. (iv) Accuracy (ACC): overall percentage of correctly classified samples. The first two are
Closed set Open set Authentication protocol Authentication protocol
genuine-genuine 3100 2200 3100 310 other user-genuine 93000 46200 0 0 impostor-genuine 0 19800 27900 2790 especially useful for the open-set protocol, which involves the presence of impostors. These metrics allow us to determine the extent to which the system is capable of rejecting unauthorized users but also how robust it is to errors.
The closed-set recognition protocol constitutes the optimal case in which the possibility of an impostor attempting to authenticate instead of a genuine user is not considered. The only errors that can occur are due to the rejection of the authorized user (FRR) or the incorrect recognition (1-TPIR).
The system exhibited the lowest performance on MMU2 (
To explain this diference in results, we performed an image quality analysis (Fig.6) on the two MMU datasets (v1 and v2) and on data acquired ad hoc. We used our selected system’s sensor and a second sensor, the Irishield MK 2120U, which captures at a distance of 5 cm away from the eye. The qualities, calculated using the open-source BIQT Framework1, are shown in Figure 6.
The high average quality of the samples acquired with the CMITECH EMX-30 and the Irishield MK 2120U sensors leads us to hypothesize that this is closely linked to the small distances between the sensor and the subject. The low variance of the CMITECH EMX-30 sensor may be due to the stability of the acquisition due to a constant distance and a fixed-position sensor. In fact, the Irishield MK 2120U cannot be used in a fixed position and incorrect positioning of the sensor leads to low-quality images. This aspect is of critical importance for the purposes of the project as it requires a high quality of the acquisition to obtain a robust and reliable system.
The poor quality of the samples of the MMUv2 dataset explains the low performance of the recognition system on these data: in fact, low-quality samples are more likely to be badly segmented, and these errors result in feature vectors not representative of the biometric trait.
The open-set recognition protocol provides for the possibility of an impostor attack. A stringent rejection threshold allows us to protect the system from this eventuality. Since we want to simulate maximum security application contexts, we have calculated the performance metrics with an operational threshold equal to ZeroFAR. This experiment allows us to evaluate how many genuine users are rejected with a stringent rejection threshold, that no impostors can access the system. If this threshold leads to many genuine users rejected, the system would be unusable. Accordingly, we assessed the score threshold value Gabor that was close for both features, namely, 0.40 when using Gabor, and 0.34 when using BSIF. We obtained that using one eye at a time, the FRR was about 3.50% using the latter features set. On the other hand, no false rejects occurred using the Gabor features over the DIEE-Iris. Moreover, the TPIR is 100% for both the feature extraction methods. These tests revealed that only one eye is suficient to obtain a very high performance level under the conditions considered. In our opinion, this is partly merit of the selected sensor, which allowed a very robust set images under the variations explored for the project’s goal. Moreover, the adopted feature sets confirmed what achieved in the previous experiment.
To evaluate the variation of system errors as the rejection threshold varies, we have shown the Receiver Operator Characteristic (ROC) curve in Figure 7 considering recognition with a single eye (c) or the fusion of the two eyes (d). The ROC confirms that for stringent operational thresholds, errors are still low. This is also evident from the distance distributions (a,b), since the two distributions are almost completely separate. Although the tests need to be expanded, the preliminary results with occlusions demonstrate that the presence of eyeglasses does not afect the recognition of the individual through the iris. In particular, the fusion at the score level between the distances of the two eyes makes the recognition system particularly performing in (a) BSIF distributions (b) Gabor distributions (c) ROC single eye (left) (d) ROC score level fusion the tested application conditions.
We noticed that if the gallery contains 10 templates per user for Gabor, no errors occurred. FRR is 0.32% FRR at ZeroFAR when using one eye only.
To simulate an unfavourable condition of use, we tested the system by registering only one template per user (Table 3). Although the case considered is extremely challenging, the results are still acceptable, confirming the robustness and reliability of the system based on iris recognition with NIR sensor and fixed position (the sensor is stationary in front of the user who must do nothing but approach the right distance). These experiments showed that in the case of unrepresentative enrollment data, combining the information collected from the two eyes is advantageous. It is worth highlighting that on a device with CPU Intel Core i9-9900K 3.60GHz, 32 GB RAM, the average time of a single probe-template comparison is 0.01 seconds for Gabor vectors and 0.08 seconds for BSIF ones. The system is therefore fast even with about 10 templates per user.
We presented a fully integrated prototype built on the basis of the best state-of-the-art modules, aimed to work in a specific and challenging scenario requiring precision, speed, minimal invasiveness and robustness to variations in noise, stress and lighting. The selected modules exhibited the best compromise, in particular, between precision and computational complexity, and fully met the other constraints. The iris was the chosen biometric trait for this goal. In particular, the selection of a stable and precise acquisition sensor and of a performing segmentation algorithm based on deep learning has allowed using simple and fast approaches, such as textural algorithms, for the chosen biometric description and matching process. AIbased pre-processing algorithms, as well as simple and compact iris representation, are the system’s core. To evaluate the adherence to the requirements from an empirical viewpoint, a novel data was acquired to simulate realistically the use-case scenario, called “DIEE-Iris”. The acquisition protocol took into account the possible lighting variations expected during system’s operations: difuse artificial light, artificial light coming from the right, artificial light coming from the left, natural light and coloured light. Also, the irises of users wearing glasses and squinting were captured. The DIEE-Iris is still under collection; however, a preliminary set of experiments was already possible. Experiments pointed out the limits of iris recognition systems in unfavourable conditions: they are often linked to the device and the data acquisition methods. However, using sensors that operate in the near-infrared range allowed to manage variations in lighting. In addition, a stable sensor placement optimises the image quality and made easier the probe-template comparison. When it is not possible to obtain a good representativeness of the iris with more templates, independently of the quality, the double-eye acquisition keeps stable or improves the performance. In conclusion, what reported showed that the iris acquisition and processing technology is fully mature to operate in real and challenging application contexts. To complete the analysis, we planned to extend the dataset, including presentation attacks, to evaluate the robustness of this prototype, which is full operating and available in the university lab and in the one of the company, to spoofing attacks.
This work is partially supported by MBDA Italia Spa and by the Italian Ministry of Education, University and Research (MIUR) within the PRIN2017 - BullyBuster - A framework for bullying and cyberbullying action detection by computer vision and artificial intelligence methods and algorithms (CUP: F74I19000370001). The BullyBuster project has been included in the Global Top 100 list of AI projects addressing the 17 UNSDGs (United Nations Strategic Development Goals) by the International Research Center for Artificial Intelligence under the auspices of UNESCO.