Hybrid Iris Segmentation Method Based on CNN and Principal Curvatures

Abstract

In this article the new hybrid iris image segmentation method based on convolutional neural networks and mathematical methods is proposed. Iris boundaries are found using modified Daugman's method. Two UNet-based convolutional neural networks are used for iris mask detection. The first one is used to predict the preliminary iris mask including the areas of the pupil, eyelids and some eyelashes. The second neural network is applied to the enlarged image to specify thin ends of eyelashes. Then the principal curvatures method is used to combine the predicted by neural networks masks and to detect eyelashes correctly. The proposed segmentation algorithm is tested using images from CASIA IrisV4 Interval database. The results of the proposed method are evaluated by the Intersection over Union, Recall and Precision metrics. The average metrics values are 0.922, 0.957 and 0.962, respectively. The proposed hybrid iris image segmentation approach demonstrates an improvement in comparison with the methods that use only neural networks.

Keywords

• Biometrics
• Iris Identification
• Iris Segmentation
• Principal Curvatures
• Convolutional Neural Network

-

Iris recognition is one of the most accurate methods of biometric identification. In order to determine the features which are necessary for person identification by the iris image, the image segmentation must be performed. It includes determination of the inner and outer boundaries of the iris (iris localization) and the areas where eyelids and eyelashes overlap an eye. The result of segmentation is a mask, which is a binary image of the visible iris region. The mask helps to identify areas suitable for further parameterization.

Iris segmentation is an important stage of the iris image preprocessing. Unlike the texture of the iris, the position of eyelashes and eyelids are not constant. Nevertheless, only regular features could be used for the identification, so it is necessary to exclude all the variable parameters in the eye image.

There are some mathematical methods for iris localization: integro-differential Daugman’s method [ 1 ], Hough’s transform-based method [ 2 ], fast method named ”Pulling and Pushing” [ 3 ]. Eyelids and eyelashes are often found as parabolas and noises [ 4, 5 ].

The evolution of the neural networks increases the popularity of their use. Neural network-based methods give usually better results than mathematical methods. The most popular architectures for iris segmentation are: convolutional neural network (CNN) such as U-Net [ 6, 7 ], SegNet [ 8 ]; hierarchical convolutional neural networks (HCNN) and multi-task fully convolutional networks (MTFCN) [ 9 ]; deep neural networks (DNN) [ 10 ]; densely connected fully convolutional networks (IrisDenseNet) [ 11 ]; dense deep convolutional neural networks (DDNet) [ 12 ] and other architectures. However, it is necessary to control the results of neural networks work. Therefore, the use of a hybrid method improves the results of CNN application.

The hybrid iris image segmentation is proposed in the article. The Daugman’s integro-differential method is modified and used for iris localization. To obtain iris mask image two UNet-based convolutional neural networks are used. First CNN is used to predict the preliminary mask of the iris, and the second one is used to specify thin ends of eyelashes. The modified principal curvatures method is used to combine predicted masks and to detect eyelashes correctly. Test results using CASIA IrisV4 Interval database [ 13 ] show the effectiveness of the proposed iris segmentation method. 2

Iris Localization

The iris localization method proposed by John Daugman [ 1 ] is based on the formula: max(r;x0;y0) G (r) where G (r) is a Gaussian function and I(x; y) is an iris image intensity function, integration is carried out along a circular contour with a radius r centered at the point (x0; y0) of the image. Usually different values of smoothing parameter are used for inner and outer iris boundaries detection. For outer iris boundary the integration is carried out not over the entire circle, but only along the right and left arcs, divided into two parts (Fig. 1). The measure of the arc in each of the areas is calculated by the formula [ 14 ]: i = 90 I 4 P Ij j=1 i ; i 2 f1; 2; 3; 4g; (1) (2) where Ii is the average image intensity in the corresponding region.

Iris pupil boundary is close to the circle but it is necessary to precise it for the accurate iris segmentation. The image is translated into a polar coordinate system centered at the pupil center and the Canny filter [ 15 ] is applied to the image (Fig. 2). The comparison of the internal boundaries found by the modified and classical methods is shown in Fig. 3.

The modified Daugman’s integro-differential method accurately determines the inner and outer iris boundaries, but it is not able to determine the areas of eyelids or eyelashes.

Hybrid Iris Segmentation Method Based on CNN and Principal Curvatures 3

3 Iris Segmentation

In order to determine the areas of eyelids and eyelashes, a U-net–based [ 7 ] convolutional neural network is used (Fig. 4). Binary cross-entropy is used as a loss function, the Jacquard measure is used as a metric [ 16 ], the Adam optimizer [ 17 ] for optimization.

Ground truth images are needed to train the neural network. There is the opensource database of labeled images [ 18 ], but eyelashes are not marked there. So, 150 eye images from the Casia IrisV4 Interval database [ 13 ] were manually labeled in related work: 100 for the training set, 27 for the validation and 23 for the test set. The training is conducted through 40 epochs, with 100 iterations in each. To increase the number of images in the training data, augmentations were used – 16 pixels left and right, 14 pixels up and down shifts and rotations up to 5 degrees. The validation set is used after passing through each epoch in order to adjust the learning rate if the accuracy values and loss functions do not improve over six epochs.

However, the neural network trained on the source data is not able to distinguish the thin ends of the eyelashes due to the strong size reduction of the image after the third layer (Fig. 5), while two layers are not enough for the right segmentation. There are several ways to determine the ends of thin eyelashes: to add a dilated convolution layer to increase image size or to use a second neural network with the same architecture, which is trained on the double sized with bicubic interpolation images from the same training dataset. This network is able to determine eyelashes better, some iris features could be recognized as eyelashes. The results of both networks are combined using the curvature method. The image can be represented as a three-dimensional surface, taking the intensity at each point as the value of the z-coordinate [ 19, 20 ]. Let L(x; y) be the Gaussian smoothed image, Lxx(x; y), Lxy(x; y), Lyx(x; y), Lyy(x; y) be its second derivatives. Local characteristics of the image L(x; y) at the point (x; y) can be determined using the Hessian matrix:

H(x; y) =

Lxx(x; y) Lxy(x; y) Lyx(x; y) Lyy(x; y) : (3)

Eigenvalues of the Hessian matrix H(x; y) are denoted by 1; 2, where j 1j > j 2j, and the eigenvectors corresponding to them are 1; 2. The direction and value of the maximum curvature at the point (x; y) will correspond to the vector 1 and the value Hybrid Iris Segmentation Method Based on CNN and Principal Curvatures 5 (a) (b) (c) (d) of 1, the minimum – 2 and 2 [ 20 ]. Vector 2 is directed along the tubular-shaped area, while vector 1 is directed across it. Thus, each next eyelash point can be found by moving from the previous one in the direction of the vector 2 to a distance proportional to 1 with coefficient k. Fig. 6 shows the field of such vectors in the eye image: it is clearly seen that the direction of vector 2 lines up along the eyelashes. However, there are other vectors in the image. To avoid it, the threshold parameter is used.

At each starting eyelash point of the image the eigenvalues and eigenvectors of the Hessian matrix are calculated. If the point conforms to the rule j 1j > , the next step of j 2j the algorithm is applied at a point located in the direction of the vector 2 at a distance proportional to k 1 from the previous point, where the parameter k is: k = max(0:15; min( j 1j j 2j The proposed iris segmentation method (Fig. 7) can be described as follows: first, the iris mask is obtained using the first CNN. After that, using the second neural network, the iris mask of enlarged image is found to refine thin eyelashes. Then, the principal curvature method that consists of three stages is applied to the results of both networks (Fig. 7 d, e, f). 1. At the first stage, only the points with the image intensity values less than a threshold, that were predicted by the first network are taken as the starting points (Fig. 7 d). 1 = 1:7. 2. At the second stage, only the points marked as eyelashes in the image after the first stage are taken as the starting points. This stage was performed to thicken and connect interrupted eyelashes, the intensity of which in the original image is less than the threshold. (Fig. 7 e) 2 = 2:0. 3. At the third stage, only the points recognized as eyelashes by the second network and adjoined to the points from the second stage are taken as the starting points.

This step is performed to extend the thin ends of the eyelashes. (Fig. 7 f) 3 = 1:8. Thus, the image mask with eyelashes is obtained.

Then, the modified Daugman’s algorithm is applied to the original image to obtain the inner and outer iris boundaries. After that, the region of obtained mask, between the outer and inner iris boundaries is taken as iris image mask. 4

Results

The results of the proposed method for the images from the CASIA IrisV4 database are presented in Fig. 8 and Fig. 9. Fig. 8 represents the change of the mask at each stage of the algorithm and comparison of segmentation results with manually marked ground truth segmentation. Green in the last column (Fig. 8 g) indicates the intersection areas, red indicates the areas marked by the proposed algorithm as the iris but not highlighted in the manually marked image, yellow indicates the opposite situation. The results before and after applying the principal curvature method (Fig. 8 d and Fig. 8 e) are evaluated by three metrics: Intersection over Union (IoU), Precision (Prec) and Recall (Rec). The Recall metric value decreases since while clarifying the eyelashes, the areas between them could also be marked as eyelashes that leads to increase in the number of pixels falsely labeled as eyelashes. However, the areas marked as eyelashes

Hybrid Iris Segmentation Method Based on CNN and Principal Curvatures 7 are not used for further biometric identification. For this reason, reducing the number of eyelash points falsely marked as iris points is the priority of the method. The increase of the Precision metric indicates that the principal curvatures method improves the results of the CNN-based method. Fig. 9 shows the overlay of the obtained mask on the original image.

In the article [ 6 ] U-net with different depth values is used for the iris segmentation. The comparison of average values of Precision, Recall and Intersection over Union metrics, gained in this article with the corresponding metrics obtained in [ 6 ] is given in Table 1. In this paper the new hybrid iris segmentation method based on CNNs and principal curvatures method is proposed. Experimental results obtained using Casia IrisV4 Interval database demonstrate that the proposed approach gives good segmentation results. The evaluation of this method by the IoU metric is 92%, which appears to be higher than the results of the methods that use only a neural network with a similar depth.

Hybrid Iris Segmentation Method Based on CNN and Principal Curvatures 9

References

1. Daugman , J.G. : High confidence visual recognition of persons by a test of statistical independence . IEEE transactions on pattern analysis and machine intelligence 15 ( 11 ), 1148 - 1161 ( 1993 )

2. Wildes , R.P. : Iris recognition: an emerging biometric technology . Proceedings of the IEEE 85 ( 9 ), 1348 - 1363 ( 1997 )

3. He , Z. , Tan , T. , Sun , Z. , Qiu , X. : Toward accurate and fast iris segmentation for iris biometrics . IEEE transactions on pattern analysis and machine intelligence 31 ( 9 ), 1670 - 1684 ( 2008 )

4. Min , T.H. , Park , R.H.: Eyelid and eyelash detection method in the normalized iris image using the parabolic hough model and otsu's thresholding method . Pattern recognition letters 30 ( 12 ), 1138 - 1143 ( 2009 )

5. Kong , W. , Zhang, D.: Accurate iris segmentation based on novel reflection and eyelash detection model . In: Proceedings of 2001 International Symposium on Intelligent Multimedia, Video and Speech Processing. ISIMP 2001 (IEEE Cat. No. 01EX489) . pp. 263 - 266 . IEEE ( 2001 )

6. Lozej , J. , Meden , B. , Struc , V. , Peer , P. : End-to-end iris segmentation using u-net . In: 2018 IEEE International Work Conference on Bioinspired Intelligence (IWOBI) . pp. 1 - 6 . IEEE ( 2018 )

7. Ronneberger , O. , Fischer , P. , Brox , T.: U-net: Convolutional networks for biomedical image segmentation . In: International Conference on Medical image computing and computerassisted intervention . pp. 234 - 241 . Springer ( 2015 )

8. Badrinarayanan , V. , Kendall , A. , Cipolla , R.: Segnet: A deep convolutional encoder-decoder architecture for image segmentation . IEEE transactions on pattern analysis and machine intelligence 39 ( 12 ), 2481 - 2495 ( 2017 )

9. Liu , N. , Li , H. , Zhang, M. , Liu , J. , Sun , Z. , Tan , T. : Accurate iris segmentation in noncooperative environments using fully convolutional networks . In: 2016 International Conference on Biometrics (ICB) . pp. 1 - 8 . IEEE ( 2016 )

10. Bazrafkan , S. , Thavalengal , S. , Corcoran , P.: An end to end deep neural network for iris segmentation in unconstrained scenarios . Neural Networks 106 , 79 - 95 ( 2018 )

11. Arsalan , M. , Naqvi , R. , Kim , D. , Nguyen , P. , Owais , M. , Park , K. : Irisdensenet: Robust iris segmentation using densely connected fully convolutional networks in the images by visible light and near-infrared light camera sensors . Sensors 18 ( 5 ), 1501 ( 2018 )

12. Chen , Y. , Zeng , Z. , Hu , F. : End to end robust recognition method for iris using a dense deep convolutional neural network . In: Chinese Conference on Biometric Recognition . pp. 364 - 375 . Springer ( 2019 )

13. Casia iris image database . idealtest.org/dbDetailForUser.do?id=4

14. Pavelyeva , E.A. , Krylov , A.S.: Iris identification algorithm using the most informative iris points . In: International Conference on Computer Graphics and Machine Vision ”Graphicon” . pp. 228 - 231 ( 2009 )

15. Canny , J.: A computational approach to edge detection . IEEE Transactions on pattern analysis and machine intelligence (6) , 679 - 698 ( 1986 )

16. Chen , Y. , Wang , W. , Zeng , Z. , Wang , Y. : An adaptive cnns technology for robust iris segmentation . IEEE Access 7 , 64517 - 64532 ( 2019 )

17. Kingma , D.P. , Ba , J.: Adam: A method for stochastic optimization . arXiv preprint arXiv:1412.6980 ( 2014 )

18. Hofbauer , H. , Alonso-Fernandez , F. , Wild , P. , Bigun , J. , Uhl , A. : A ground truth for iris segmentation . In: 2014 22nd international conference on pattern recognition . pp. 527 - 532 . IEEE ( 2014 )

19. Safronova , E.I. , Pavelyeva , E.A. : Palm vein recognition algorithm using multilobe differential filters . In: International Conference on Computer Graphics and Machine Vision ”Graphicon” . pp. 117 - 121 ( 2019 )

20. Choi , J.H. , Song , W. , Kim , T. , Lee , S.R. , Kim , H.C. : Finger vein extraction using gradient normalization and principal curvature . In: Image Processing: Machine Vision Applications II . vol. 7251 , p. 725111 . International Society for Optics and Photonics ( 2009 )