Recent research into the domain of identity document recapture detection demonstrated the capability of the Meijering iflter, a biomedical image processing algorithm, to detect features present in recaptured documents. Manipulation of identity documents using image processing software is a low-cost, high-risk threat to modern financial systems, opening these institutions to fraud through crimes related to identity theft. In this paper we extend the research into the application of biomedical image processing algorithms, including the Meijering filter and the Sato filter. We build support vector machine and decision tree classifiers based on histograms of images generated from these filters and apply some rudimentary feature reduction techniques. The results show that both filters can be applied to this domain, with the Meijering filter slightly outperforming the Sato filter in most tests.
tial that the biomedical algorithm known as the Meijering iflter can be used in the domain of identity document reTraditional Know Your Customer (KYC) channels for fi- capture detection. The Meijering filter [ 4] is a technique nancial institutions are slow, ineficient, and costly [ 1]. designed to assist the analysis of neurite growth of fluoRemote customer onboarding services and electronic rescence images captured using microscopes. As a form Know Your Customer (eKYC) services are viable alter- of texture detection, it was used successfully to detect native to traditional methods and help reduce costs and paper texture by Magee et al. Another biomedical filter, friction experienced by customers signing up for services. known as the Sato filter [ 5] is used to detect and enhance Retail banking institutions are increasingly including tubular and linear structures in medical images. Both eKYC services within mobile banking apps, including the Meijering and Sato filters are built on techniques the ability to open a new account remotely. This can that use the eigenvalues of the Hessian matrix for image present a hole in the security architecture as bad actors enhancement. can easily use modern digital imaging software to manip- The goal of this research is to build on the recent reulate identity documents, exposing financial institutions search by Magee et al. that demonstrated the capability to fraud [2] through simple document recapture attacks. of the Meijering filter as a feature extraction process to The consequences of fraud are significant; in 2022 the UK help detect recaptured identity document images. This National Crime Agency reported that money laundering research improves on the original work by 1) attempting cost the UK economy in the region of “hundreds of billion to improve the data set used by Magee et al., 2) reproduce pounds per year” 1. A recaptured identity document is the original work with the new data set and compare one where a copy is made of a legitimate identity docu- our results with the reported results 3) use a range of ment, possibly altered, and then printed on to paper as a input features to test how these influence the classificahard copy. Document recapture attacks are a low-cost, tion accuracy, 4) test the application of the Sato filter 5) high risk to eKYC services and such presentation attacks train a decision tree classifier using the Meijering and should be detected and immediately rejected. Sato filtered data and 6) compare the classification perRecent work by Magee et al. [3] demonstrated poten- formance of the decision tree and SVM algorithms using the Meijering and Sato filtered data across the range of input features.
APWG.EU Technical Summit and Researchers Sync-Up 2023 (Tech 2023), June 21 22, 2023, Dublin, Ireland $ B00149241@mytudublin.ie (J. Magee); ss@mytudublin.ie (S. Sheridan); ct@mytudublin.ie (C. Thorpe)
© 2023 Copyright for this paper by its authors. Use permitted under Creative Commons License CPWrEooUrckReshdoinpgs IhStpN:/c1e6u1r3-w-0s.o7r3g ACttEribUutRion W4.0oInrtekrnsahtioonpal (PCCroBYce4.0e).dings (CEUR-WS.org) 1https://www.parliament.uk/business/lords/media-centre/houseof-lords-media-notices/2022/november-2022/the-governmentmust-take-the-fight-to-the-fraudsters-by-slowing-down-fasterpayments-and-prosecuting-corporates-for-failure-to-preventfraud/
Attack Detection standard, ISO 30107 2. This publication introduced specific terminology to address error rates for classification algorithms. These terms are Attack Presentation Classification Error Rate (APCER) and Bona Fide Presentation Classification Error Rate (BPCER). Readers familiar with the confusion matrix may not recognise these terms, but they are just synonyms for False Negative Rate (FNR) and False Positive Rate (FPR).
The Sato filter, introduced by Sato et al. [ 5], is a type of image filter used for the detection and enhancement of tubular and linear structures in medical images and is an extension of the widely used vessel enhancement filter known as the Frangi filter [ 6]. It is a multi-scale filter that uses the eigenvalues of the Hessian matrix to enhance linear structures in an image while reducing background noise. While this filter is typically used to detect blood vessels, in this exploratory research we are using it to detect surface texture of recaptured document images.
Introduced by Meijering et al. [4], this technique was designed to assist the analysis of neurite growth of fluorescence images captured using microscopes and is also an extension of the Frangi filter [ 6]. Their algorithm is implemented in 2 phases, it first assigns each pixel in the image a probability of that pixel belonging to a neurite (known as the detection phase), it then it links together the center lines of the neurites to form the neurite tracing (this is known as the tracing phase). Conceptually, this can be considered a form of texture detection where the neurite is the textured object being detected against a lfuorescent background. Texture detection is a technique previously used for document recapture detection [7, 8].
quisition from a mobile device for counterfeit detection. Their research focused on Spanish identity documents but was also used to detect counterfeit bank notes. Their data set was generated through crowd sourcing using a mobile app, individuals were encouraged to submit identity documents and classify them (genuine/not genuine) in the mobile app. The disadvantage of this solution is that it requires model generation per document type, which increases the complexity when the number of document types and issuing countries/institutions increase.
Yang et al. [10] developed a Convolutional Neural Network (CNN) based solution to detect recaptured images. They reused images from existing data sets [11, 12] of general photographs from public sources, not identity documents. Their data set consists of 10,000 genuine and recaptured images. All images are 512 x 512 pixels in size, considerably low resolution compared to the capabilities of modern mobile cameras. Their contribution is to add a Laplician filter into the CNN as an enhancement layer. They report an classification accuracy of 99.74% for images 512 x 512 pixels, while smaller images resulted in a slightly lower accuracy.
Berenguel et al. [9] proposed a Counterfeit Recurrent Comparator (CRC) network design to identify counterfeit documents. The design is based on research on the human perception system [13]. The researchers reused a data set from their own work [8] that consists of identity documents and counterfeit bank notes. The document is split into patches and the CRC network iterates over all patches until the complete document is assessed. Their network performance is compared to PeleeNet[14], outperforming it with a mean AUC score of 0.984.
Chen et al. [15] proposed a Siamese network design to address the detection of recaptured documents. A Siamese network is a neural network design that contains two or more identical components used to find similarities between inputs. Such network designs require samples of genuine documents, as well as recaptured documents, to train the network. To address this problem, they created a database of 320 captured and 2627 recaptured document images based on generated synthetic data. Chen et al. achieved 6.92% APCER and 8.51% BPCER by their proposed network.
Magee et al. [3] investigated the application of a biomedical imaging algorithm, the Meijering filter, to the domain of document recapture detection. The Meijering filter was applied to recaptured document images and histograms were generated from these filtered images as a form of rudimentary feature extraction. A support vector machine was trained using these features as a classification model to distinguish between document images recaptured from a screen and document images recaptured from printed hard copies. Without applying any data cleaning techniques, the results were promising with a mean APCER of 15.45% for an iPhone8 and 29.35% for an iPhone12 mobile device, while both models had a BPCER of approximately 24%. These results are not state-of-the-art, but they do show the potential to use biomedical imaging algorithms in the domain of identity document recaptured detection.
Magee et al. [3] concluded that their data set required cleaning as the source of their recaptured images did not truly represent that of an identity document data set. The source data set was the BID data set [16], a synthetic data set generated based on images of real Brazilian driver licenses. The purpose of BID data set was to assist with document segmentation and OCR, meaning the authors intentionally added variation (diferent background colours, adding noise or bold text) that is not present in genuine identity documents, consistency being a security feature of identity documents. Based on this observation, we undertook an attempt to clean the data set used by Magee et al. and we removed 10 source images from the data set due to noise and replaced them with 8 new images that contained a consistent look and feel in an attempt to reduce the amount of variance in the data set. We plan to recapture more images from the source BID data set to augment our data set in future work. Magee et al. used screen recaptured images as a ground truth data set, something to measure the printed recaptured documents against. As no data set of genuine recaptured identity documents is available, these screen recaptured images act as a proxy for genuine recaptured documents. We also replaced some of the recaptured screen images that contained obvious screen artifacts. This resulted in 22 screen recaptured images being replaced by the iPhone8 and 23 screen recaptured images being replaced by the iPhone12. This efort was only partially successful as it was not possible to obtain recaptures without some visible screen artifacts. A breakdown of the data set after the cleaning exercise is represented in Table 1. transitioned to the screen background on the monitor. These artifacts are not what we expect to see in genuine recaptured documents, therefore all filtered images are cropped to remove these artifacts by removing the bordering 50 pixels from each side of the image. Python scripts were used to process all of the recaptured images and the data processing pipeline is represented in Figure 1. Image processing scripts are written in Python, version 3.8.5, Scikit-Image version 1.2.2 and OpenCV 4.4.0.42. All default parameters in the Scikit-Image library are used when generating the Meijering and Sato images.
Magee et al. used the Support Vector Machine (SVM) classification algorithm to train a classifier to distinguish between screen and printed recaptured images. Previous research has shown the SVM classifier to be useful in this domain [17, 18, 7]. SVM models were generated for each bin number as outlined in Section 3.2. The SVM is used as a binary classifier, therefore a class value of 0 represents a label for screen recaptured images and a class value of 1 represents the label for printed recaptured images. To generate suficient accuracy metrics, we used the same training and testing procedure as outlined by Magee et al., twenty diferent seeded tests were run, each using stratified 10-fold cross validation, resulting in 200 accuracy metrics for each test. The same seeds used by Magee et al. were used in this work. The APCER and BPCER metrics are computed for each test and the average metric across all tests for each device is reported. The SVM was trained using Python 3.8.5 and scikit-learn 1.2.2, all default parameters are used.
3.2. Feature Extraction and Reduction The application of the No Free Lunch theorem [19] tells us that each machine learning algorithm is biased in The feature extraction process used by Magee et al. was its own way. In an efort to avoid being misled by any limited to the histogram intensity values of the Meijer- inherent bias in the SVM algorithm, we use the decision ing filtered images with a bin width value of 1, meaning tree algorithm as a measure of comparison. Decision the greyscale value with 256 grey values produced 256 tree algorithms are easy to explain and can be visualised input features. This represents the highest resolution very easily but may also over-fit training data [ 20]. The histogram possible from a greyscale image. As part of exact same process reported in Section 3.3 is used to this research, we are applying some rudimentary data train a decision tree classification algorithm. We use reduction by using diferent numbers of bins during the three diferent split criteria in this research, they are Gini, histogram generation process. The bins used in this work Entropy and logloss. The APCER and BPCER metrics are are 8, 10, 16, 32, 48, 50, 64, 128 and 256. We are also intro- computed for each test and the average metric across all ducing the Sato filter as a new feature extraction process. tests for each device is reported. The decision tree was The same feature extraction process described above is trained using Python 3.8.5 and Scikit-learn 1.2.2. Except applied to the Sato filtered images. After applying the for the split criterion, all default parameters are used. iflters, the same screen artifacts described by Magee et al. were observed in the screen recaptured images. These are unique artifacts close to the edges of the image, indicating the area where the identity document image limited number of samples in the test data sets, meaning a single misclassification will result in a minimum error rate of 9%. The diference between the Meijering and Sato classification metrics can be down to only 1 more misclassified sample.
3.5. Results of Magee et al. and continue the investigation into the use of biomedical imaging algorithms into the domain of The results of the SVM classification models are displayed identity document recapture detection. In this paper we in Table 2 and the results of the decision tree models are introduced a rudimentary feature reduction technique by shown in Table 3. selecting diferent bin numbers in the histogram genera
The results of both classification algorithms indicate tion process and measuring how the features influenced that models trained on the Meijering filtered data result the model accuracy. We introduced a second biomedical in more accurate models than those trained on Sato data, algorithm, the Sato filter, and applied all the same model but this is not the case across the board. For example, the generation and testing to images produced by this filSVM results in Table 2 show the BPCER for the iPhone12 ter. Finally, we added a new machine learning technique, using only 8 bins is 10% higher for Meijering data com- the decision tree, as a new method to compare perforpared to Sato, which is a considerable diference. The mance against that of the SVM. We have shown that the decision tree results also show that models trained on the decision tree algorithm typically out performs the SVM Meijering filtered data result in more accurate models model when comparing the APCER and BPCER metrics than those trained on Sato data, regardless of the split and they also appear more invariant to changes in the criteria used. Very little variation was observed across number of input features. The results also show that the the 3 diferent split criteria, Gini, Entropy and logloss, Meijering filter typically results in models that provide indicating that each split criteria was invariant to any higher accuracy than those trained using the Sato filter, bias in the data. It is also remarkable how invariant the but this is not across the board and the diferences reldecision tree accuracy metrics are relative to the number atively minor in the context of the small data sets used of bins used, the variance shown by the SVM models for testing (small classification diferences result in large appears much higher. For example, for the Meijering percentage diferences). We strictly controlled the procedata, the iPhone8 SVM model has an APCER of 11.84% dure used in this work, meaning there is no variation in for 8 bins and 17.64% for 256 bins whereas the decision the process used to create and test models based on data tree APCER is 14.55% and 14.83% respectively for the from the Meijering and Sato filters. The exact same data same data. Decision tree models trained on the Meijering set is used, the same random seeds are used to control ifltered data are approximately 5% more accurate that the kFold cross validation process and the same machine those trained on Sato filtered data. However, all these learning algorithm implementations are used to train and test results need to be interpreted in the context of the evaluate the models. As a result of this, we are confident
SVM Performance Statistics that any variation in performance between the two filters for this work. They report APCER 29.35% and BPCER is directly related to the algorithms. 24.05% for the iPhone12 compared to APCER of 27.40%
Results reported by Magee et al., only using 256 bins, and BPCER of 26.28% of this work. The results of this are APCER 15.45% and BPCER 24.40% for the iPhone8 work show degraded classification performance in comcompared to APCER of 17.64% and BPCER of 26.71% parison to Magee et al. in 3 of the 4 results, when using 64 256 bins, although the diference is small. It should be noted that the best classification accuracy metrics obtained in this work are for models trained with less than 256 bins. The diference in results is due to the data cleaning exercise that we undertook. Future work will include more data capture to augment the results obtained using the Meijering and Sato filters. ument counterfeit detection, in: 2017 14th IAPR International Conference on Document Analysis and Recognition (ICDAR), IEEE, 2017, pp. 15–20. [10] P. Yang, R. Ni, Y. Zhao, Recapture image forensics based on laplacian convolutional neural networks, in: Y. Q. Shi, H. J. Kim, F. Perez-Gonzalez, F. Liu (Eds.), Digital Forensics and Watermarking, volume 10082, Springer International Publishing, 2017, pp. 119–128. Series Title: Lecture Notes in Computer
Science. [11] H. Cao, A. C. Kot, Identification of recaptured pho[1] R. Soltani, U. Trang Nguyen, A. An, A new ap- tographs on LCD screens, in: 2010 IEEE Internaproach to client onboarding using self-sovereign tional Conference on Acoustics, Speech and Signal identity and distributed ledger, in: 2018 IEEE Inter- Processing, IEEE, 2010, pp. 1790–1793. national Conference on Internet of Things (iThings) [12] R. Li, R. Ni, Y. Zhao, An efective detection method and IEEE Green Computing and Communications based on physical traits of recaptured images on (GreenCom) and IEEE Cyber, Physical and Social LCD screens, in: Y.-Q. Shi, H. J. Kim, F. PérezComputing (CPSCom) and IEEE Smart Data (Smart- González, I. Echizen (Eds.), Digital-Forensics and Data), IEEE, 2018, pp. 1129–1136. Watermarking, volume 9569, Springer International [2] H. B. Macit, A. Koyun, Tamper detection and re- Publishing, 2016, pp. 107–116. covery on RGB images, in: D. J. Hemanth, U. Kose [13] P. Shyam, S. Gupta, A. Dukkipati, Attentive recur(Eds.), Artificial Intelligence and Applied Mathemat- rent comparators, in: D. Precup, Y. W. Teh (Eds.), ics in Engineering Problems, Springer International Proceedings of the 34th International Conference Publishing, 2020, pp. 972–981. on Machine Learning, volume 70 of Proceedings of [3] J. Magee, S. Sheridan, C. Thorpe, An investigation into the application of the meijering filter for docu- 3M1a8c1h. ine Learning Research, PMLR, 2017, pp. 3173– ment recapture detection. under review for the 12th [14] R. J. Wang, X. Li, C. X. Ling, Pelee: A real-time obinternational conference on intelligent information ject detection system on mobile devices, in: S. Benprocessing (iciip 2023), not yet published., 2023. gio, H. Wallach, H. Larochelle, K. Grauman, N. Cesa[4] E. Meijering, M. Jacob, J.-C. Sarria, P. Steiner, Bianchi, R. Garnett (Eds.), Advances in Neural InH. Hirling, M. Unser, Design and validation of a formation Processing Systems, volume 31, Curran tool for neurite tracing and analysis in fluorescence Associates, Inc., 2018. microscopy images, Cytometry Part A 58A (2004) [15] C. Chen, S. Zhang, F. Lan, J. Huang, Domain167–176. agnostic document authentication against practical [5] Y. Sato, S. Nakajima, N. Shiraga, H. Atsumi, recapturing attacks 17 (2022) 2890–2905.
S. Yoshida, T. Koller, G. Gerig, R. Kikinis, Three- [16] D. S. Soares, R. B. Das Neves Junior, B. L. D. Bezerra, dimensional multi-scale line filter for segmentation BID dataset: a challenge dataset for document proand visualization of curvilinear structures in medi- cessing tasks, in: Anais Estendidos da Conference cal images 2 (1998) 143–168. on Graphics, Patterns and Images (SIBRAPI Esten[6] A. F. Frangi, W. J. Niessen, K. L. Vincken, M. A. dido 2020), Sociedade Brasileira de Computação, Viergever, Multiscale vessel enhancement filter- 2020, pp. 143–146. ing, in: Medical Image Computing and Computer- [17] C.-Y. Yeh, W.-P. Su, S.-J. Lee, Employing multipleAssisted Intervention — MICCAI’98, volume 1496, kernel support vector machines for counterfeit banSpringer Berlin Heidelberg, 1998, pp. 130–137. knote recognition 11 (2011) 1439–1447. [7] X. Hou, T. Zhang, G. Xiong, Y. Zhang, X. Ping, Im- [18] V. Lohweg, J. L. Hofmann, H. Dörksen, R. Hildeage resampling detection based on texture classifi- brand, E. Gillich, J. Hofmann, J. Schaede, Banknote cation 72 (2014-09) 1681–1708. authentication with mobile devices, 2013, p. 866507. [8] A. B. Centeno, O. R. Terrades, J. L. i. Canet, C. C. [19] H. Wolpert, D, G. Macready, W., No free lunch Morales, Evaluation of texture descriptors for val- theorems for optimization, in: IEEE Transactions idation of counterfeit documents, in: 2017 14th on Evolutionary Computation, 1997. IAPR International Conference on Document Anal- [20] G. James, D. Witten, T. Hastie, R. Tibshirani, An ysis and Recognition (ICDAR), IEEE, 2017-11, pp. Introduction to Statistical Learning: with Applica1237–1242. tions in R, Springer Texts in Statistics, Springer US, [9] A. Berenguel, O. R. Terrades, J. Llados, C. Canero, 2021.
E-counterfeit: A mobile-server platform for doc