Unsupervised Palm Vein Image Segmentation*

Abstract

In this article the new hybrid algorithm for palm vein image segmentation using convolutional neural network and principal curvatures is proposed. After palm vein image preprocessing vein structure is detected using unsupervised learning approach based on W-Net architecture, that ties together into a single autoencoder two fully convolutional neural network architectures, each similar to the U-Net. Then segmentation results are improved using principal curvatures technique. Some vein points with highest maximum principal curvature values are selected, and the other vein points are found by moving from starting points along the direction of minimum principal curvature. To obtain the final vein image segmentation the result of intersection of the principal curvaturesbased and neural network-based segmentations is taken. The evaluation of the proposed unsupervised image segmentation method based on palm vein recognition results using multilobe differential filters is given. Test results using CASIA multi-spectral palmprint image database show the effectiveness of the proposed segmentation approach.

Keywords

• Biometrics
• Image Segmentation
• Palm Vein Recognition
• Unsupervised Learning
• Principal Curvatures

Nowadays information security plays crucial role in human life and, as it turned out,

accustomed keys and passwords are not reliable enough. Instead, the biometric characteristics that uniquely identify a person from an entire population based on intrinsic physical or behavioral traits [ 1 ], provide stable and safe data protection. Biometrics recognizes individuals based on these characteristics.

One of the most advanced and progressive personal identification technologies is

palm vein recognition. Veins are usually not visible to others that provides low risk of fake or theft. As for other important advantages, vein patterns are quite unique to the owners, image acquisition does not require physical contact and the system can be made compact. Deoxygenated hemoglobin in the vein blood absorbs near infrared light so infrared camera captures images containing veins.

Palm vein recognition algorithm consists of several steps. Firstly, the region of in

terest (ROI) is extracted and segmentation of ROI image into two classes is performed. Vein points are marked in white while the other points are marked in black. The second step, feature vector extraction, represents the main difference between existing approaches. Since vein recognition is relatively young study, some feature extraction methods could be derived from other biometric recognition algorithms based on statistical information [ 2 ], image key points [ 3, 4, 5 ], subspace-based methods [ 6 ], phase based methods [ 7, 8 ], etc. Some approaches were developed specifically for vein recognition [ 9 ]. The last algorithm step, image matching, is based on feature vector type. At this step the distance between palm vein images is calculated. Much recent work has been focused on employing deep convolutional neural networks (CNN) in biometrics.

Deep learning methods can be applied to any step of palm vein recognition algorithm

[ 10, 11, 12, 13 ].

In this paper we propose a hybrid approach based on unsupervised machine learning and mathematical methods to obtain good vein segmentation. The problem of unsupervised image segmentation is one of the major challenge in computer vision which has been deeply researched. The range of well-known techniques for solving this issue contains normalized-cuts [ 14 ], Markov random field-based methods [ 15 ], CNN-based approaches [ 16 ], etc. However, the results of applying of these methods may include inaccuracy due to specific features of a technique and lack of correct ground truth. So the mathematical methods shall control the results of CNN. In this paper we propose a hybrid segmentation method consisting of two approaches: based on CNN and principal curvatures (Fig.1).

The rest of this paper is organized as follows. In Section 2 the palm vein image prepro

cessing and ROI extraction algorithms are described. The vein structure extraction is described in Section 3 where Subsections 3.1 and 3.2 present principal curvatures and

CNN-based segmentation algorithms, the hybrid approach is described in Subsection

3.3. The evaluation of the proposed unsupervised image segmentation method based on

Unsupervised Palm Vein Image Segmentation 3

multilobe differential filters for palm vein feature extraction is described in Section 4.

The experimental results for images from CASIA multi-spectral palmprint image database [17] are given in Section 5. Finally, Section 6 concludes this paper.

2

Palm vein image preprocessing

The proposed palm vein region of interest (ROI) detection and enhancement scheme is illustrated in Fig. 2.

First, hand boundary is detected by OTSU binarization algorithm [ 18 ] and points between the fingers are found as points where local minimum of Euclidean distances between the center of palm and all points on the hand contour is reached. The points between index and middle fingers, 1, and forth and little fingers, 2, can be taken as landmarks for extraction of square ROI (Fig. 2 d) [ 19 ]. To eliminate the influence of palm rotation, the image is rotated to the angle θ which is the angle between the line 1 2 and the horizontal line. To reduce the non-uniform illumination appearing in palm vein images the background is subtracted and the histogram is stretched (Fig. 2 f). To emphasize vein structure contrast-limited adaptive histogram equalization (CLAHE) technique [ 20 ] is used (Fig. 2 g). After contrast enhancement all image details including noise and glares are sharper. In order to smooth the undesirable details, non-local means (NLM) algorithm [ 21 ] is used to reduce noise (Fig. 2 h). NLM smoothies also veins a little so CLAHE is applied again to obtain distinguishable veins (Fig. 2 i). Fig. 2 shows the ROI of palm vein image and the results of preprocessing algorithm. After preprocessing veins become sharper and more distinguishable [ 22 ]. 3 3.1

Vein structure extraction

Principal curvatures

The next step is the vein structure extraction. Consider an image as a surface in a three

dimensional space, where the brightness value of the pixels is the z-coordinate. We are going to extract vein structure using principal curvatures method [ 22 ].

Let ( , )denote the image intensity at the pixel position, ( , )be the image

gradient vector. Then the normalized gradient after a hard thresholding is defined as: Let 1, 2 be the eigenvalues and 1 , 2 be the corresponding eigenvectors of ( , ), | 1| > | 2|. Then two principal directions, the directions of the maximum and minimum curvatures, are determined by two eigenvectors 1 and 2. Consequently, two eigenvalues 1 , 2 represent the principal curvatures (the curvatures along the principal directions) [ 9 ]. The tubular-shaped regions have maximum principal curvature 1 higher than other regions and vector 1 is directed across tubular direction, vector 2 – along tubular direction [ 23 ].

( , ) , ‖ ( , )‖ ≥ ( , )= {‖ ( , )‖ 0, ‖ ( , ‖ < where γ is a threshold level. In the experiments we use γ = 4. The normalized gradient field contains noisy components so we smooth it with Gaussian function ( , ): Let ( , )= (ℎ ( , ), ℎ ( , )). The local shape characteristics of an image at a point ( , )can be described by the Hessian matrix ( , ). (1) (2) (3)

Unsupervised Palm Vein Image Segmentation 5

In order to catch veins of different widths, consider the set of parameters σ for the

Gauss function: 0, … , −1, where = 10, = 0 ∙ 4√2 , 0 = 2, = 0, 1, … , 9. For each value of the Hessian matrix is constructed, at each point the maximum positive eigenvalue 1 and an eigenvector 2 corresponding to 2 are calculated. Then, at each point of the image, the largest value of 1 over all and the corresponding vector 2 are taken.

We select points with highest maximum principal curvature values as points that

certainly belong to veins. The other vein points can be found [ 22 ] from starting points by moving along direction of vector 2 by | 1|. The results of this approach is shown in Fig.3. As we do not have the ground truth for the task of ROIs segmentation, the unsupervised method is required, so the approach based on W-Net architecture [ 16 ] is proposed. The authors of W-Net method present a new architecture which ties two fully convolutional network (FCN) architectures, each similar to the U-Net [ 24 ], together into a single autoencoder (Fig. 4). The first FCN encodes an input image into a k-way soft segmentation: : ℝ × ×3 → ℝ × × , where × denotes a size of input image, ( ) = ( = )∈ [ 0, 1 ] measures the probability of pixel belonging to class k ( is set of pixels in segment k). The second FCN, decoder, reverses this process, going from the segmentation layer back to a reconstructed image: : ℝ × × → ℝ × ×3 (Fig.4).

The both reconstruction errors of the autoencoder and soft normalized cut loss function on the encoding layer are used during training. The reconstruction loss is standard ‖ , ( for training encoder-decoder architecture and can be defined as: = where is set of pixels in segment k, V is the set of all pixels, and w measures the weight between two pixels.

However, since the argmax function is non-differentiable, it is impossible to calculate the corresponding gradient during backpropagation. Instead, it is proposed to use a soft version of the Ncut loss which is differentiable [ 16 ]: −

( , )= ∑ =1 ( , − )= − ∑ =1

( , ) ( , ) ( , )= − ∑ =1

∑ ∈ , ∈ ( , ) ( = ) ∑ ∈ , ∈ ( , ) ( = ) ( = )= − ∑ = ∑ ∈ ( = )∑ ∈ ( , ) ( = ), ∑ ∈ ( = )∑ ∈ ( , ) where ( = )measures the probability of node u belonging to class k that is directly computed by the encoder. The weight matrix W for − is defined as: , = −‖ ( )− ( )‖22 2 ∗ { −‖ ( )− ( )‖22

2 0 if ‖ ( )− ( )‖2 < , otherwise, where ( )and ( )are the spatial location and pixel value of node i, respectively. Since the size of our ROI images is 128×128 which is smaller than in the original work [ 16 ], the depth of W-Net was decreased in our experiments, as it is shown in Fig. 5. We use : ℝ128×128×1 → ℝ128×128× and : ℝ128×128× → ℝ128×128×1.

(4) (5) (6)

Unsupervised Palm Vein Image Segmentation 7

As vein images have several semantic classes, such as veins with different intensity, background, skin wrinkles, the neural network was applied for the overclustering, = 16. The training dataset contains 120 images. In Fig.6 one can see the results of this approach: the first column in the figure shows input images (Fig. 6 a) while the second one illustrates their reconstruction (Fig. 6 b); the third column presents the result of overclustering (Fig. 6 c). After unification of classes corresponding to veins the obtained vein image binarization is shown in the fourth column (Fig. 6 d). based and CNN-based vein segmentations is taken (Fig. 7).

(a) (b) (c) (d) (MLDF) [ 25 ] that highlight vein branch points (Fig. 8) for palm vein feature extraction and normalized root-mean-square error for feature maps matching [ 22 ]. Mathematically the MLDFs are given as follows: = ∑

1 =1 √2 −( − )2 2 2 − ∑

1 =1 √2 −( − )2 2 2 where the variables µ and σ denote the central positions and the scales of a 2D Gaussian filters respectively, denote the number of positive lobes, and denote the number

Unsupervised Palm Vein Image Segmentation 9

of negative lobes. Constant coefficients and are used to ensure zero sum of the

MLDF. We take the convolution results of the ROI images in vein points obtained after

vein image segmentation with the proposed MLDF kernels to obtain the feature maps of vein images. In order to provide slight translation and rotation invariance matching, the normalized root-mean-square error (NRMSE) [ 26 ] is proposed for feature maps matching [ 22 ].

Given the intra- and interclass vein matching results, the recognition performance is

measured by the following indicators: the distribution of genuine and impostor scores,

False Acceptance Rate (FAR), False Reject Rate (FRR) and Equal Error Rate (EER) − the cross-over error rate when FAR is equal to FRR. Lower EER means higher accuracy of a biometric matcher.

5

Experimental results

Experimental results using CASIA Multi-Spectral Palmprint Image Database [ 17 ] are presented. The database contains 7200 palm images captured from 100 different people using a self-designed multiple spectral imaging device. Each sample contains six palm images which are captured at the same time with six different electromagnetic spectrums. Each hand of any person in the database is represented by six images at one wavelength. In our study the images from CASIA database obtained at 850 nm are taken.

The CNN model was implemented in PyTorch [27] framework and trained for 100

epochs with Google Colaboratory with a batch size of 16 using Adam optimizer [ 28 ] with the learning rate of 0.001. In order to train W-Net we randomly selected 20 hands from the dataset and took all 6 corresponding images, so we got 120 images in training set.

To test the proposed hybrid segmentation method, the recognition results using a part of CASIA database are presented in Fig. 9. Recognition results after image segmentation with principal curvatures and without W-Net based CNN (Fig. 3) is shown in Fig.9 a. Recognition results after image segmentation with W-Net based CNN and without principal curvatures (Fig. 6) is shown in Fig.9 b. Recognition results after proposed hybrid segmentation (Fig. 7) is shown in Fig.9 c.

In this article the new palm vein image segmentation method based on principal curva

tures and unsupervised convolutional neural network is proposed. It is shown that the method based on principal curvatures improve segmentation results obtained by CNN.

Experimental results using CASIA multi-spectral palmprint image database are presented.

Unsupervised Palm Vein Image Segmentation 11

References

1. Jain , A. K. , Bolle , R. , Pankanti , S. : Biometrics: personal identification in networked society , Vol. 479 . Springer Science & Business Media ( 2006 ).

2. Rosdi , B. A. , Shing , C. W. , Suandi , S. A. : Finger vein recognition using local line binary pattern . Sensors 11 ( 12 ), 11357 - 11371 ( 2011 ).

3. Matsuda , Y. , Miura , N. , Nagasaka , A. , Kiyomizu , H. , Miyatake , T. : Finger-vein authentication based on deformation-tolerant feature-point matching . Machine Vision and Applications 27 ( 2 ), 237 - 250 ( 2016 ).

4. Protsenko , M. А. , Pavelyeva , E. A. : Iris image key points descriptors based on phase congruency . ISPRS - International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences 42 ( 2 /W12), 167 - 171 ( 2019 ).

5. Wang , L. , Leedham , G. , Cho , D. S. Y.: Minutiae feature analysis for infrared hand vein pattern biometrics . Pattern recognition 41(3) , 920 - 929 ( 2008 ).

6. Wu , J. D. , Liu , C. T.: Finger-vein pattern identification using principal component analysis and the neural network technique . Expert Systems with Applications 38 ( 5 ), 5423 - 5427 ( 2011 ).

7. Han, W. Y., Lee , J. C. : Palm vein recognition using adaptive Gabor filter . Expert Systems with Applications 39 ( 18 ), 13225 - 13234 ( 2012 ).

8. Pavelyeva , E. A. : Image processing and analysis based on the use of phase information . Computer Optics 42 ( 6 ), 1022 - 1034 , 2018 .

9. Choi , J. H. , Song , W. , Kim , T. , Lee , S. R. , Kim , H. T.: Finger vein extraction using gradient normalization and principal curvature . In: Image Processing: Machine Vision Applications II , pp. 725111 . International Society for Optics and Photonics ( 2019 ).

10. Jha , R. R. , Thapar , D. , Patil , S. M. , Nigam , A. : Ubsegnet: Unified biometric region of interest segmentation network . In: 2017 4th IAPR Asian Conference on Pattern Recognition (ACPR) , pp. 923 - 928 . IEEE ( 2017 ).

11. Lefkovits , S. , Lefkovits , L. , Szilágyi , L. : Applications of different CNN architectures for palm vein identification . In: International Conference on Modeling Decisions for Artificial Intelligence , pp. 295 - 306 . Springer, Cham ( 2019 ).

12. Thapar , D. , Jaswal , G. , Nigam , A. , Kanhangad , V.: PVSNet: Palm Vein Authentication Siamese Network Trained using Triplet Loss and Adaptive Hard Mining by Learning Enforced Domain Specific Features . In: 2019 IEEE 5th International Conference on Identity, Security, and Behavior Analysis (ISBA) , pp. 1 - 8 . IEEE ( 2019 ).

13. Wang , J. , Yang , K. , Pan , Z. , Wang , G. , Li , M. , Li , Y. : Minutiae-Based Weighting Aggregation of Deep Convolutional Features for Vein Recognition . IEEE Access 6 , 61640 - 61650 ( 2018 ).

14. Shi , J. , Malik , J.: Normalized cuts and image segmentation . IEEE Transactions on pattern analysis and machine intelligence 22 ( 8 ), 888 - 905 ( 2000 ).

15. Zhang , Y. , Brady , M. , Smith , S. : Segmentation of brain MR images through a hidden Markov random field model and the expectation-maximization algorithm . IEEE transactions on medical imaging 20(1) , 45 - 57 ( 2001 ).

16. Xia , X. , Kulis , B.: W-Net : A deep model for fully unsupervised image segmentation . arXiv preprint arXiv:1711.08506 ( 2017 ).

17. CASIA Multi-Spectral Palmprint Image Database, http://biometrics.idealtest.org/.

18. Otsu , N.: A threshold selection method from gray-level histograms . IEEE transactions on systems, man, and cybernetics 9 ( 1 ), 62 - 66 ( 1979 ).

19. Lin , C. L. , Chuang , T. C. , Fan , K. C. : Palmprint verification using hierarchical decomposition . Pattern Recognition 38 ( 12 ), 2639 - 2652 ( 2005 ).

20. Zuiderveld , K. : Contrast limited adaptive histogram equalization . Graphics gems , 474 - 485 ( 1994 ).

21. Buades , A. , Coll , B. , Morel , J. M.: A non-local algorithm for image denoising . In: 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05) , vol. 2 , pp. 60 - 65 . IEEE ( 2005 ).

22. Safronova , E. I. , Pavelyeva , E. A. : Palm Vein Recognition Algorithm using Multilobe Differential Filters . In: Proceedings of 29-th International Conference on Computer Graphics and Vision GraphiCon , vol. 1 , pp. 117 - 121 ( 2019 ).

23. Renault , C. , Desvignes , M. , Revenu , M.: 3D curves tracking and its application to cortical sulci detection . In: Proceedings 2000 International Conference on Image Processing (Cat. No. 00CH37101) , vol. 2 , pp. 491 - 494 . IEEE ( 2000 ).

24. Ronneberger , O. , Fischer , P. , Brox , T.: U-net: Convolutional networks for biomedical image segmentation . In: International Conference on Medical image computing and computer-assisted intervention , pp. 234 - 241 . Springer, Cham ( 2015 ).

25. Sun , Z. , Tan , T. : Ordinal measures for iris recognition . IEEE Transactions on pattern analysis and machine intelligence 31 ( 12 ), 2211 - 2226 ( 2008 ).

26. Fienup , J. R. : Invariant error metrics for image reconstruction . Applied optics 36 ( 32 ), 8352 - 8357 ( 1997 ).

27. Paszke , A. et al. Automatic differentiation in pytorch ( 2017 ).

28. Kingma , D. P. , Ba , J. Adam: A method for stochastic optimization . arXiv preprint arXiv:1412.6980 ( 2014 ).