=Paper=
{{Paper
|id=Vol-2744/paper31
|storemode=property
|title=Hybrid Iris Segmentation Method Based on CNN and Principal Curvatures
|pdfUrl=https://ceur-ws.org/Vol-2744/paper31.pdf
|volume=Vol-2744
|authors=Varvara Tikhonova,Elena Pavelyeva
}}
==Hybrid Iris Segmentation Method Based on CNN and Principal Curvatures==
https://ceur-ws.org/Vol-2744/paper31.pdf
Hybrid Iris Segmentation Method Based on CNN and
Principal Curvatures
Varvara Tikhonova[0000−0002−7386−0250] and Elena Pavelyeva[0000−0002−3249−2156]
Faculty of Computational Mathematics and Cybernetics, Lomonosov Moscow State University,
Moscow, Russia
varvara.a.tikhonova@gmail.com, paveljeva@yandex.ru
Abstract. In this article the new hybrid iris image segmentation method based
on convolutional neural networks and mathematical methods is proposed. Iris
boundaries are found using modified Daugman’s method. Two UNet-based con-
volutional neural networks are used for iris mask detection. The first one is used
to predict the preliminary iris mask including the areas of the pupil, eyelids and
some eyelashes. The second neural network is applied to the enlarged image to
specify thin ends of eyelashes. Then the principal curvatures method is used to
combine the predicted by neural networks masks and to detect eyelashes cor-
rectly. The proposed segmentation algorithm is tested using images from CASIA
IrisV4 Interval database. The results of the proposed method are evaluated by the
Intersection over Union, Recall and Precision metrics. The average metrics values
are 0.922, 0.957 and 0.962, respectively. The proposed hybrid iris image segmen-
tation approach demonstrates an improvement in comparison with the methods
that use only neural networks.
Keywords: Biometrics, Iris Identification, Iris Segmentation, Principal Curva-
tures, Convolutional Neural Network.
1 Introduction
Iris recognition is one of the most accurate methods of biometric identification. In order
to determine the features which are necessary for person identification by the iris image,
the image segmentation must be performed. It includes determination of the inner and
outer boundaries of the iris (iris localization) and the areas where eyelids and eyelashes
overlap an eye. The result of segmentation is a mask, which is a binary image of the
visible iris region. The mask helps to identify areas suitable for further parameterization.
Iris segmentation is an important stage of the iris image preprocessing. Unlike the
texture of the iris, the position of eyelashes and eyelids are not constant. Nevertheless,
only regular features could be used for the identification, so it is necessary to exclude
all the variable parameters in the eye image.
There are some mathematical methods for iris localization: integro-differential Daug-
man’s method [1], Hough’s transform-based method [2], fast method named ”Pulling
and Pushing” [3]. Eyelids and eyelashes are often found as parabolas and noises [4, 5].
Copyright c 2020 for this paper by its authors. Use permitted under Creative Commons
License Attribution 4.0 International (CC BY 4.0).
2 V. Tikhonova, E. Pavelyeva
The evolution of the neural networks increases the popularity of their use. Neu-
ral network-based methods give usually better results than mathematical methods. The
most popular architectures for iris segmentation are: convolutional neural network (CNN)
such as U-Net [6, 7], SegNet [8]; hierarchical convolutional neural networks (HCNN)
and multi-task fully convolutional networks (MTFCN) [9]; deep neural networks (DNN)
[10]; densely connected fully convolutional networks (IrisDenseNet) [11]; dense deep
convolutional neural networks (DDNet) [12] and other architectures. However, it is nec-
essary to control the results of neural networks work. Therefore, the use of a hybrid
method improves the results of CNN application.
The hybrid iris image segmentation is proposed in the article. The Daugman’s
integro-differential method is modified and used for iris localization. To obtain iris mask
image two UNet-based convolutional neural networks are used. First CNN is used to
predict the preliminary mask of the iris, and the second one is used to specify thin
ends of eyelashes. The modified principal curvatures method is used to combine pre-
dicted masks and to detect eyelashes correctly. Test results using CASIA IrisV4 Interval
database [13] show the effectiveness of the proposed iris segmentation method.
2 Iris Localization
The iris localization method proposed by John Daugman [1] is based on the formula:
I
∂ I(x, y)
max(r,x0 ,y0 ) Gσ (r) ∗ , (1)
∂r 2·π·r
r,x0 ,y0
where Gσ (r) is a Gaussian function and I(x, y) is an iris image intensity function, in-
tegration is carried out along a circular contour with a radius r centered at the point
(x0 , y0 ) of the image. Usually different values of smoothing parameter σ are used for
inner and outer iris boundaries detection. For outer iris boundary the integration is car-
ried out not over the entire circle, but only along the right and left arcs, divided into two
parts (Fig. 1). The measure of the arc in each of the areas is calculated by the formula
[14]:
90◦ · Ii
αi = 4 ; i ∈ {1, 2, 3, 4}, (2)
P
Ij
j=1
where Ii is the average image intensity in the corresponding region.
Iris pupil boundary is close to the circle but it is necessary to precise it for the accu-
rate iris segmentation. The image is translated into a polar coordinate system centered
at the pupil center and the Canny filter [15] is applied to the image (Fig. 2). The compar-
ison of the internal boundaries found by the modified and classical methods is shown in
Fig. 3.
The modified Daugman’s integro-differential method accurately determines the in-
ner and outer iris boundaries, but it is not able to determine the areas of eyelids or
eyelashes.
Hybrid Iris Segmentation Method Based on CNN and Principal Curvatures 3
Fig. 1. Divided regions of the image.
Fig. 2. Determination of the inner iris boundary.
Fig. 3. Comparison of iris localization using the classical (red line) and modified (blue line) Daug-
man’s method.
3 Iris Segmentation
3.1 CNN for iris segmentation
In order to determine the areas of eyelids and eyelashes, a U-net–based [7] convolu-
tional neural network is used (Fig. 4). Binary cross-entropy is used as a loss function,
the Jacquard measure is used as a metric [16], the Adam optimizer [17] for optimiza-
tion.
Ground truth images are needed to train the neural network. There is the open-
source database of labeled images [18], but eyelashes are not marked there. So, 150 eye
images from the Casia IrisV4 Interval database [13] were manually labeled in related
work: 100 for the training set, 27 for the validation and 23 for the test set. The training
is conducted through 40 epochs, with 100 iterations in each. To increase the number
4 V. Tikhonova, E. Pavelyeva
of images in the training data, augmentations were used – 16 pixels left and right, 14
pixels up and down shifts and rotations up to 5 degrees. The validation set is used after
passing through each epoch in order to adjust the learning rate if the accuracy values
and loss functions do not improve over six epochs.
Fig. 4. Neural network architecture.
However, the neural network trained on the source data is not able to distinguish the
thin ends of the eyelashes due to the strong size reduction of the image after the third
layer (Fig. 5), while two layers are not enough for the right segmentation. There are
several ways to determine the ends of thin eyelashes: to add a dilated convolution layer
to increase image size or to use a second neural network with the same architecture,
which is trained on the double sized with bicubic interpolation images from the same
training dataset. This network is able to determine eyelashes better, some iris features
could be recognized as eyelashes. The results of both networks are combined using the
curvature method.
3.2 Principal curvatures method
The image can be represented as a three-dimensional surface, taking the intensity at
each point as the value of the z-coordinate [19, 20]. Let L(x, y) be the Gaussian smoothed
image, Lxx (x, y), Lxy (x, y), Lyx (x, y), Lyy (x, y) be its second derivatives. Local char-
acteristics of the image L(x, y) at the point (x, y) can be determined using the Hessian
matrix:
Lxx (x, y) Lxy (x, y)
H(x, y) = . (3)
Lyx (x, y) Lyy (x, y)
Eigenvalues of the Hessian matrix H(x, y) are denoted by λ1 , λ2 , where |λ1 | >
|λ2 |, and the eigenvectors corresponding to them are ν1 , ν2 . The direction and value of
the maximum curvature at the point (x, y) will correspond to the vector ν1 and the value
Hybrid Iris Segmentation Method Based on CNN and Principal Curvatures 5
(a) (b) (c) (d)
Fig. 5. Comparison of masks predicted by the first and second neural networks. (a) is the original
image, (b) is the mask predicted by the first network, (c) is the mask predicted by the second
network, (d) is a comparison of the masks. The matching white areas are shown in green, the
areas that only the first network has identified as the iris are shown in red, the areas that only the
second network has identified as the iris are shown in yellow.
of λ1 , the minimum – ν2 and λ2 [20]. Vector ν2 is directed along the tubular-shaped
area, while vector ν1 is directed across it. Thus, each next eyelash point can be found by
moving from the previous one in the direction of the vector ν2 to a distance proportional
to λ1 with coefficient k. Fig. 6 shows the field of such vectors in the eye image: it is
clearly seen that the direction of vector ν2 lines up along the eyelashes. However, there
are other vectors in the image. To avoid it, the threshold parameter γ is used.
At each starting eyelash point of the image the eigenvalues and eigenvectors of the
|λ1 |
Hessian matrix are calculated. If the point conforms to the rule |λ 2|
> γ, the next step of
the algorithm is applied at a point located in the direction of the vector ν2 at a distance
proportional to k · λ1 from the previous point, where the parameter k is:
|λ1 | 1
k = max(0.15, min( · , 0.7)). (4)
|λ2 | 100
|λ1 |
The algorithm stops if the next step leads to the same point or if the condition |λ 2|
>γ
is not met.
3.3 Hybrid iris segmentation
The proposed iris segmentation method (Fig. 7) can be described as follows: first, the
iris mask is obtained using the first CNN. After that, using the second neural network,
the iris mask of enlarged image is found to refine thin eyelashes. Then, the principal
curvature method that consists of three stages is applied to the results of both networks
(Fig. 7 d, e, f).
6 V. Tikhonova, E. Pavelyeva
Fig. 6. The vector field of the eye image corresponding to vector ν2 , and the length of vectors is
proportional to λ1 .
1. At the first stage, only the points with the image intensity values less than a thresh-
old, that were predicted by the first network are taken as the starting points (Fig. 7
d). γ1 = 1.7.
2. At the second stage, only the points marked as eyelashes in the image after the
first stage are taken as the starting points. This stage was performed to thicken and
connect interrupted eyelashes, the intensity of which in the original image is less
than the threshold. (Fig. 7 e) γ2 = 2.0.
3. At the third stage, only the points recognized as eyelashes by the second network
and adjoined to the points from the second stage are taken as the starting points.
This step is performed to extend the thin ends of the eyelashes. (Fig. 7 f) γ3 = 1.8.
Thus, the image mask with eyelashes is obtained.
Then, the modified Daugman’s algorithm is applied to the original image to obtain
the inner and outer iris boundaries. After that, the region of obtained mask, between the
outer and inner iris boundaries is taken as iris image mask.
4 Results
The results of the proposed method for the images from the CASIA IrisV4 database
are presented in Fig. 8 and Fig. 9. Fig. 8 represents the change of the mask at each
stage of the algorithm and comparison of segmentation results with manually marked
ground truth segmentation. Green in the last column (Fig. 8 g) indicates the intersec-
tion areas, red indicates the areas marked by the proposed algorithm as the iris but not
highlighted in the manually marked image, yellow indicates the opposite situation. The
results before and after applying the principal curvature method (Fig. 8 d and Fig. 8
e) are evaluated by three metrics: Intersection over Union (IoU), Precision (Prec) and
Recall (Rec). The Recall metric value decreases since while clarifying the eyelashes,
the areas between them could also be marked as eyelashes that leads to increase in the
number of pixels falsely labeled as eyelashes. However, the areas marked as eyelashes
Hybrid Iris Segmentation Method Based on CNN and Principal Curvatures 7
Fig. 7. Description of the proposed method: (a) the source image; (b) the mask predicted by
the first neural network; (c) the mask predicted by the second neural network; (d)-(f) stages of
curvature method; (g) the result of iris localization; (h) the result of the method.
are not used for further biometric identification. For this reason, reducing the number of
eyelash points falsely marked as iris points is the priority of the method. The increase
of the Precision metric indicates that the principal curvatures method improves the re-
sults of the CNN-based method. Fig. 9 shows the overlay of the obtained mask on the
original image.
(a) (b) (c) (d) (e) (f) (g)
Fig. 8. The results of the proposed iris segmentation method. (a) the original image, (b) the result
of the modified Daugman’s method, (c) the mask predicted by the first neural network, (d) the
intersection of the mask predicted by neural network and the area selected by the Daugman’s
method, (e) the result of the proposed method: the intersection of the mask specified using the
curvature method and the area selected by the Daugman’s method, (f ) the manually marked iris
image segmentation, (g) the comparison of the proposed and manually marked iris segmentation.
8 V. Tikhonova, E. Pavelyeva
(a) (b) (c)
Fig. 9. The results of the proposed method. (a) the original image, (b) the result of the proposed
iris segmentation method, (c) the overlay of the segmented iris region on the original image.
In the article [6] U-net with different depth values is used for the iris segmentation.
The comparison of average values of Precision, Recall and Intersection over Union
metrics, gained in this article with the corresponding metrics obtained in [6] is given in
Table 1.
Table 1. Comparison to competing method.
Method Database Avg. Precision Avg. Recall Avg .IoU
U-net [6], depth 3 Casia Iris V1 0.937 0.957 0.899
U-net [6], depth 4 Casia Iris V1 0.948 0.955 0.908
U-net [6], depth 5 Casia Iris V1 0.948 0.960 0.912
Proposed method Casia Iris V4 0.962 0.957 0.922
5 Conclusion
In this paper the new hybrid iris segmentation method based on CNNs and principal
curvatures method is proposed. Experimental results obtained using Casia IrisV4 Inter-
val database demonstrate that the proposed approach gives good segmentation results.
The evaluation of this method by the IoU metric is 92%, which appears to be higher
than the results of the methods that use only a neural network with a similar depth.
Hybrid Iris Segmentation Method Based on CNN and Principal Curvatures 9
References
1. Daugman, J.G.: High confidence visual recognition of persons by a test of statistical indepen-
dence. IEEE transactions on pattern analysis and machine intelligence 15(11), 1148–1161
(1993)
2. Wildes, R.P.: Iris recognition: an emerging biometric technology. Proceedings of the IEEE
85(9), 1348–1363 (1997)
3. He, Z., Tan, T., Sun, Z., Qiu, X.: Toward accurate and fast iris segmentation for iris bio-
metrics. IEEE transactions on pattern analysis and machine intelligence 31(9), 1670–1684
(2008)
4. Min, T.H., Park, R.H.: Eyelid and eyelash detection method in the normalized iris image
using the parabolic hough model and otsu’s thresholding method. Pattern recognition letters
30(12), 1138–1143 (2009)
5. Kong, W., Zhang, D.: Accurate iris segmentation based on novel reflection and eyelash de-
tection model. In: Proceedings of 2001 International Symposium on Intelligent Multimedia,
Video and Speech Processing. ISIMP 2001 (IEEE Cat. No. 01EX489). pp. 263–266. IEEE
(2001)
6. Lozej, J., Meden, B., Struc, V., Peer, P.: End-to-end iris segmentation using u-net. In: 2018
IEEE International Work Conference on Bioinspired Intelligence (IWOBI). pp. 1–6. IEEE
(2018)
7. Ronneberger, O., Fischer, P., Brox, T.: U-net: Convolutional networks for biomedical im-
age segmentation. In: International Conference on Medical image computing and computer-
assisted intervention. pp. 234–241. Springer (2015)
8. Badrinarayanan, V., Kendall, A., Cipolla, R.: Segnet: A deep convolutional encoder-decoder
architecture for image segmentation. IEEE transactions on pattern analysis and machine in-
telligence 39(12), 2481–2495 (2017)
9. Liu, N., Li, H., Zhang, M., Liu, J., Sun, Z., Tan, T.: Accurate iris segmentation in non-
cooperative environments using fully convolutional networks. In: 2016 International Confer-
ence on Biometrics (ICB). pp. 1–8. IEEE (2016)
10. Bazrafkan, S., Thavalengal, S., Corcoran, P.: An end to end deep neural network for iris
segmentation in unconstrained scenarios. Neural Networks 106, 79–95 (2018)
11. Arsalan, M., Naqvi, R., Kim, D., Nguyen, P., Owais, M., Park, K.: Irisdensenet: Robust iris
segmentation using densely connected fully convolutional networks in the images by visible
light and near-infrared light camera sensors. Sensors 18(5), 1501 (2018)
12. Chen, Y., Zeng, Z., Hu, F.: End to end robust recognition method for iris using a dense deep
convolutional neural network. In: Chinese Conference on Biometric Recognition. pp. 364–
375. Springer (2019)
13. Casia iris image database. idealtest.org/dbDetailForUser.do?id=4
14. Pavelyeva, E.A., Krylov, A.S.: Iris identification algorithm using the most informative iris
points. In: International Conference on Computer Graphics and Machine Vision ”Graph-
icon”. pp. 228–231 (2009)
15. Canny, J.: A computational approach to edge detection. IEEE Transactions on pattern analy-
sis and machine intelligence (6), 679–698 (1986)
16. Chen, Y., Wang, W., Zeng, Z., Wang, Y.: An adaptive cnns technology for robust iris seg-
mentation. IEEE Access 7, 64517–64532 (2019)
17. Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint
arXiv:1412.6980 (2014)
18. Hofbauer, H., Alonso-Fernandez, F., Wild, P., Bigun, J., Uhl, A.: A ground truth for iris
segmentation. In: 2014 22nd international conference on pattern recognition. pp. 527–532.
IEEE (2014)
10 V. Tikhonova, E. Pavelyeva
19. Safronova, E.I., Pavelyeva, E.A.: Palm vein recognition algorithm using multilobe differen-
tial filters. In: International Conference on Computer Graphics and Machine Vision ”Graph-
icon”. pp. 117–121 (2019)
20. Choi, J.H., Song, W., Kim, T., Lee, S.R., Kim, H.C.: Finger vein extraction using gradient
normalization and principal curvature. In: Image Processing: Machine Vision Applications
II. vol. 7251, p. 725111. International Society for Optics and Photonics (2009)