Proceedings of the 1st International Workshop on Automated Forensic Handwriting Analysis (AFHA) 2011 Evaluation of Local and Global Features for Offline Signature Verification Muhammad Imran Malik∗† , Marcus Liwicki∗ , Andreas Dengel∗† ∗ German Research Center for AI (DFKI GmbH) Knowledge Management Department, Kaiserslautern, Germany {firstname.lastname}@dfki.de † Knowledge-Based Systems Group, Department of Computer Science, University of Kaiserslautern, P.O. Box 3049, 67653 Kaiserslautern, Germany Abstract—In this paper we evaluate the impact of two state- because the questioned signatures and the collected refer- of-the-art offline signature verification systems which are based ence signatures (known) are commonly supplied offline [4]. on local and global features, respectively. It is important to take Therefore, we focused explicitly on the offline signature into account the real world needs of Forensic Handwriting Examiners (FHEs). In forensic scenarios, the FHEs have to verification. make decisions not only about forged and genuine signatures In many recent works signature verification has been but also about disguised signatures, i.e., signatures where the considered as a two-class pattern classification problem [1]. authentic author deliberately tries to hide his/her identity with Here an automated system has to decide whether or not a the purpose of denial at a later stage. The disguised signatures given signature belongs to a referenced authentic author. If play an important role in real forensic cases but are usually neglected in recent literaure. This is the novelty of our study the system could not find enough evidence of a forgery from and the topic of this paper, i.e., investigating the performance the questioned signature feature vector, it simply considers of automated systems on disguised signatures. Two robust the signature as genuine belonging to the referenced au- offline signature verification systems are slightly improved thentic author, otherwise it declares the signature as forged. and evaluated on publicly available data sets from previous However, when talk about the forensic aspect, there is signature verification competitions. The ICDAR 2009 offline signature verification competition dataset and the ICFHR 2010 another equally important class of signatures that also needs 4NSigComp signatures dataset. In our experiments we observed to be identified, i.e., the disguised signatures. that global features are capable of providing good results if only A disguised signature is a signature that is originally a detection of genuine and forged signatures is needed. Local written by the authentic reference author. However, it differs features, however, are much better suited to solve the forensic from the genuine signatures in the authors intent when it was signature verification cases when disguised signatures are also involved. Noteworthy, the system based on local features could written. A genuine signature is written by an author with the outperform all other participants at the ICFHR 4NSigComp intention of being positively identified by some automated 2010. system or by an FHE. A disguised signature, on the other Keywords-signature verification, mixture models, forgeries, hand, is written by the genuine author with the intension disguised signatures, forensic handwriting analysis of denial, that he/she has written that particular signature, later. The purpose of making such disguised signatures can I. I NTRODUCTION be hundreds, e.g., a person trying to withdraw money from Signature verification is in focus of research for decades. his/her own bank account via offline signatures on bank Traditionally, automated signature verification is divided check and trying to deny the signatures after some time, into two broad categories, online and offline signature or even making a false copy of his/her will etc. Potentially verification, depending on the mode of the handwritten whatever the reason is, disguised signatures appear in real input. If both the spatial as well as temporal information world and FHEs have to face them. regarding signatures are available to the systems, verification The category of disguised signatures has been addressed is performed on online data. In the case where temporal during the ICFHR 4NsigComp 2010 [5]. This was the first information is not available and the systems must utilize attempt to include disguised signatures into a signature only the spatial information gleaned through scanned or verification competition. The systems had to decide whether even camera captured documents, verification is performed the author wrote a signature in a natural way, with an on offline data [1], [2], [3]. intension of a disguise, or whether it has been forged by The main motivation of this paper is to study the foren- another writer. sic relevance of signature features and their influence on In this paper we investigate two methods on two bench- verification. Until now online signature verification is not mark data sets. The first method is based on global features, a common type of criminal casework for a forensic expert i.e., a fixed number of features is extracted from signature 26 Proceedings of the 1st International Workshop on Automated Forensic Handwriting Analysis (AFHA) 2011 images. In contrast, the second method uses a local ap- III. AUTOMATED SIGNATURE V ERIFICATION S YSTEMS proach, i.e., the number of features might vary - depending In this section we provide a short description of two state on the size of the signature. The two datasets are taken of the art offline signature verification systems we used in from previous signature verification competitions, i.e., the this study. SigComp09 data set from the ICDAR 2009 [6] and the 4NSigComp10 data set from the ICFHR 2010 [5]. A. Local Features combined with GMM The rest of this paper is organized as follows. Section II This system was originally designed by the authors of this summarizes the two datasets used for this study. Section III paper. A prior version of this system participated already describes the two robust offline signature verification sys- in the ICDAR 2009 signature verification competition and tems we applied. Section IV reports on the experimental achieved good results. It was not considered for participation results and provides a comparative analysis of the results. during the 4NSigComp 2010 since the authors of this papers Section V concludes the paper and gives some ideas for our were among the organizers of this event. Our system uses future work. Gaussian Mixture Models (GMMs) for the classification of the feature vector sequences. For the purpose of complete- II. DATA SETS ness, a short presentation of the system will be given here. A. ICDAR 2009 Signature Verification Competition For more details refer to [7]. The first data set is the training set of the SigComp09 Given a scanned image as an input, first of all binarization competition [6]. This dataset contains 1, 898 signature sam- is performed. Second, the image is normalized with respect ples in all. There are 12 genuine authors – each one of whom to skew, writing width and baseline location. Normalization wrote 5 of his/her genuine signatures, thereby yielding 60 of the baseline location means that the body of the text genuine signatures. 31 forgers were had to forge the genuine line (the part which is located between the upper and the signatures. Each forger contributed 5 forgeries for one writer lower baselines), the ascender part (located above the upper resulting in 155 forged signatures per writer.1 . Note that this baseline), and the descender part (below the lower baseline) dataset had no disguised signatures. is vertically scaled to a predefined size each. Writing width It is important to note that the said data were collected at a normalization is performed by a horizontal scaling operation, forensic institute where real forensic casework is performed. and its purpose is to scale the characters so that they have During dataset generation a special focus was given to the a predefined average width. provision of more and more skilled forgeries since auto- To extract the feature vectors from the normalized images, mated systems performance could vary significantly with a sliding window approach is used. The width of the window how the forgeries were produced [4]. is generally one pixel and nine geometrical features are computed at each window position. Thus an input text line B. ICFHR 2010 Signature Verification Competition is converted into a sequence of feature vectors in a 9- These signatures were originally collected for evaluating dimensional feature space. The nine features correspond to the knowledge of FHEs under supervision of Bryan Found the following geometric quantities. The first three features and Doug Rogers in the years 2002 and 2006, respectively. are concerned with the overall distribution of the pixels in The images were scanned at 600dpi resolution and cropped the sliding window. These are the average gray value of at the Netherlands Forensic Institute. the pixels in the window, the center of gravity, and the The signature collection we used in our evaluation is the second order moment in vertical direction. In addition to original test set of the ICFHR competition. It contains 125 these global features, six local features describing specific signatures for one reference author. Out of this collection, points in the sliding window are used. These include the 25 were the genuine signatures of reference author and locations of the uppermost and lowermost black pixel and remaining 100 were the questioned signatures. These 100 their positions and gradients, determined by using the neigh- questioned signatures comprised 3 genuine signatures; 90 boring windows. Feature number seven is the black to white simulated signatures (written by 34 forgers freehand copying transitions present within the entire window. Feature number the signature characteristics of the referenced author after eight is the number of black-white transitions between the training); and 7 disguised signatures written by the reference uppermost and the lowermost pixel in an image column. author himself/herself with the intention of disguise. Note Finally, the proportion of black pixels to the number of the huge difference between authentic data (3 genuine + 7 pixels between uppermost and lowermost pixels is used. For disguised signatures) vs. simulations (90 signatures). This a detailed description of the features see [8]. did not affect our evaluation since we used the Equal Gaussian Mixture Models [9] have been used to model Error Rate (EER) and relied on the Receiver Operating the handwriting of each person. More specifically, the Characteristic curves (ROC-curves). distribution of feature vectors extracted from a persons 1 22 of these forged signatures were not available so they have been handwriting is modeled by a Gaussian mixture density. For ignored (this results in 1,838 forged signatures in all instead of 1860) a D-dimensional feature vector denoted as x, the mixture 27 Proceedings of the 1st International Workshop on Automated Forensic Handwriting Analysis (AFHA) 2011 density for a given writer (with the corresponding model A ) is defined as: m X p(xkA) = wi pi (x) i=1 In other words, the density is a weighted linear com- bination of M uni-modal Gaussian densities, pi (x), each parameterized by a D × 1 mean vector, and D*D covariance matrix. For further details refer to [10]. B. Global Features combined with kNN Our system is based on the methods introduced in [11]. However, we have modified/optimized it in order to fit in the scenarios presented in the datasets of the two mentioned signature verification competitions. A short summary of the system is given here, for further details consult [11]. Figure 1: ROC on the ICDAR 2009 data First, the signature image is spatially smoothed followed by binarization. In the optimized version of this approach assessing the performance of signature verification systems. we used various combinations of local and global binariza- They are especially suited if there are unequal numbers of tion techniques. After these preprocessing steps following forged and genuine signatures in the dataset as in the case operations were performed. of both the ICDAR 2009 and ICFHR 2010 datasets. Results • Locating the signature image through its bounding box depict that, if only accuracy is used to evaluate signature • Centralizing the signature image to its center of gravity. verification systems, a system that votes by chance may • Partitioning the image horizontally and vertically start- show higher accuracy that in fact is false in context of a ing at center of gravity until it is divided into 64 cells. biometric system. • Finding the size of each cell of the image and normal- On the ICDAR 2009 dataset we performed 5-fold cross izing it with the total number of black pixels it has. validation for each of the systems and generated ROC- This constitutes the first feature vector. curves. Furthermore, we evaluated both the systems on • Calculating the angle that is made by the center point the ICFHR 2010 dataset again using the ROC-curves. The of each cell of the image with its lower right corner to details of these evaluations are presented in the following obtain the second feature vector. sections. • Obtaining a third feature vector by calculating the angle of inclination of each black pixel in a cell to the lower A. Results on the ICDAR 2009 Dataset right corner of its corresponding part of the image. We did 5-fold cross validation in the same way as in [6] Note that the approach divides the signature into 64 small and [7], i.e., for each genuine author we used only four parts, which can be seen as a local feature extraction of his/her genuine signatures to train and then tested the technique. However, since this division is based on a global classifiers. The training set was rotated 5 times. analysis and the number of extracted features is fixed, Figure 1 shows the results of both the systems on the disregarding the length of the signature, this approach is ICDAR 2009 data set. It depicts the average results on all considered as a global approach. Therefore note that a simple signatures by all writers. As shown in Fig. 1 the global disguise attempt would be to add a random character at the features based system outperforms the local features based end of the signature and the global approach would fail while system. The Equal Error Rate (EER) for the global features the local feature extraction would still find many similarities. based system is as low as 20 % whereas for the local features After computing these feature vectors, thresholds are based system it is nearly 36 %. Note that the local features computed using means and variances. Following that, nearest based system also participated in the ICDAR SigComp 2009. neighbor approach is applied to decide on the result of each On the test data it provided an EER of 16 % [6] and was feature vector and finally a voting based classification is among the best classifiers. Since the test set is not publicly made. In the optimized version different voting strategies available, therefore we evaluated our system on the training have been applied that improved the overall performance. data. IV. E VALUATION B. Results on the ICFHR 2010 Dataset For reporting the results we primarily use the ROC- We evaluated both of the systems described in Section III curves according to the evaluation procedure of the ICFHR according to the scenario posed by the ICFHR 4NSigComp 4NSigComp 2010. ROC-curves are a standard procedure of 2010.There, the systems had to present their opinion by 28 Proceedings of the 1st International Workshop on Automated Forensic Handwriting Analysis (AFHA) 2011 Table I: Interpretation of the output Decision Probability Value (D) P >t P t P t) indicated that the questioned signature was most likely a genuine one. A lower value (P ≤ t) indicated that the questioned signature was not genuine, meaning that it was not written by the reference author. A probability value of (P = t) was considered as inconclusive. The decision value D represents the system’s decision about the process by which the questioned signature was most likely generated. A decision value of 1 means that the underlying writing is natural: there is no or not enough evidence of any simulation or disguise attempt and the signature is written by the reference author. The decision value 2 represents that the underlying writing process is unnatural: there is evidence of either a simulation or disguise attempt. Finally, a decision value 3 shows that the system Figure 3: ICFHR 2010 results with disguised signatures is unable to decide if the underlying writing process is natural or unnatural: no decision could be made whether the signature is genuine, simulated, or disguised. signatures the results of one participant were nearly perfect. The output reference showing the various output possi- In order to make our systems’ performance comparable to bilities is provided as Table I. Here a value of P greater those from the ICDAR competition, we present our results in than t with output 1 means correct genuine authorship, with the same manner, i.e., first without considering the disguised output 2, on the other hand, means that the author has signatures and then taking the disguised signatures into made an attempt to disguise her/his identity. If the Decision account as well. Value is 3, then with any value of probability it is simply Figure 2 shows the results when we disregard the dis- inconclusive. Any value of P less than t with decision value guised signatures and consider only the case of forged vs. 2 indicates that the questioned signature is a result of a genuine signatures. The EER of both systems is the same. simulation or disguise process. The final assessment of the However, when considering the area under the curve, the output values is given in Table II. local feature based system is slightly better. As mentioned already, the novel feature of this dataset is The most important aspect of our study is the investigation the inclusion of disguised signatures. Various state-of-the- of the influence of disguised signatures. The results are art systems participated in the competition and aimed at depicted in Figure 3. As shown, the local features based correctly classifying these disguised signatures. All of these GMM system performs significantly better than the global systems failed to correctly detect the disguised signatures. features based system. It has an EER of 20% whereas the The EER of the best system was larger than 50 %. More global feature based system has an EER of nearly 56%. details of these results are provided in [5]. When these Our point here is that, our GMM classifier performed well systems were evaluated without considering the disguised because it was relying exclusively on local features. To 29 Proceedings of the 1st International Workshop on Automated Forensic Handwriting Analysis (AFHA) 2011 consolidate our thinking we also performed experimentation R EFERENCES with the GMM classifier by feeding it with the global [1] R. Plamondon and G. Lorette, “Automatic signature verifica- features (the same global features that are used by our NN tion and writer identification – the state of the art,” Pattern Classifier). The results were worse in this case. The accuracy Recognition, vol. 22, pp. 107–131, 1989. went below 50% and the EER was above 70%. Actually the nature of global features is to have a fixed amount of [2] R. Plamondon and S. N. Srihari, “On-line and off-line hand- writing recognition: A comprehensive survey,” IEEE Transac- features while local features are not fixed. As such our GMM tions on Pattern Analysis and Machine Intelligence, vol. 22, based system also outperforms all the participants of ICFHR pp. 63–84, 2000. 4NsigComp 2010 in this scenario as well. An important point to mention here is that our GMM based system was not [3] D. Impedovo and G. Pirlo, “Automatic signature verification: even optimized to work with disguised signatures explicitly. The state of the art,” IEEE Transactions on Systems, Man, and Cybernetics, Part C (Applications and Reviews), vol. 38, In contrast, it was initially developed as a general-purpose no. 5, pp. 609–635, Sep. 2008. offline writer identification system. We strongly believe that this better performance of our system is attributed to the fact [4] V. L. Blankers, C. E. v. d. Heuvel, K. Y. Franke, and L. G. that it relies on the local features. Vuurpijl. (2009) Call for participation:signature verification competition, on- and offline skilled forgeries. [Online]. Available: http://sigcomp09.arsforensica.org/ V. C ONCLUSION AND F UTURE W ORK [5] M. Liwicki, C. E. van den Heuvel, B. Found, and M. I. Malik, In this paper we have reported on the experiments con- “Forensic signature verification competition 4NSigComp2010 - detection of simulated and disguised signatures,” in 12th ducted to evaluate the impact of local and global features International Conference on Frontiers in Handwriting Recog- on automated signature verification for off-line signatures nition, 2010, pp. 715–720. collected by the FHEs. Two state of the art offline signature verification systems were applied on the datasets of the last [6] V. L. Blankers, C. E. v. d. Heuvel, K. Y. Franke, two signature verification competitions. and L. G. Vuurpijl, “Icdar 2009 signature verification competition,” in Proceedings of the 2009 10th International Our experimental results show that the global featu- Conference on Document Analysis and Recognition, ser. res could produce acceptable results when the traditional ICDAR ’09. Washington, DC, USA: IEEE Computer paradigm of forged vs. genuine authorship is under con- Society, 2009, pp. 1403–1407. [Online]. Available: sideration. The actual power of local features is revealed http://dx.doi.org/10.1109/ICDAR.2009.216 when considering the more realistic scenario which involves [7] M. Liwicki, “Evaluation of novel features and different mod- the presence of disguised signatures among the questioned els for online signature verification in a real-world scenario,” signatures. This has been shown by using the equal error in Proc. 14th Conf. of the Int. Graphonomics Society, 2009, rates achieved by a GMM based offline signature verification pp. 22–25. system that heavily relies on the local features of offline [8] U.-V. Marti and H. Bunke, Using a statistical signature samples. We strongly believe that the main reason language model to improve the performance of an for the good performance of this system is due to the HMM-based cursive handwriting recognition systems. difference that this system is relying on local features. River Edge, NJ, USA: World Scientific Publishing In future we plan to investigate more local features Co., Inc., 2002, pp. 65–90. [Online]. Available: approaches for signature verification. Using novel image http://portal.acm.org/citation.cfm?id=505741.505745 analysis methods like scale-invariant Speeded Up Robust [9] J. Marithoz and S. Bengio, “A comparative study of adapta- Features (SURF) [12] might be an interesting idea as well. tion methods for speaker verification,” 2002. We also plan to combine various offline signature verifica- tion systems based on different global and local features [10] A. Schlapbach, M. Liwicki, and H. Bunke, “A writer identifi- through voting strategies to produce even better results. cation system for on-line whiteboard data,” Pattern Recogn., vol. 41, pp. 2381–2397, July 2008. Furthermore, we plan to perform analyses on data which contains signatures from more reference writers and skilled [11] P. I. S. Dr. Daramola Samuel, “Novel feature extraction tech- forgers. Regarding genuine signatures, large and diverse test nique for off-line signature verification system,” International sets where signatures are produced by different authors un- Journal of Engineering Science and Technology, vol. 2, pp. 3137–3143, 2010. der various different psychological and physical conditions may also yield interesting results. [12] H. Bay, A. Ess, T. Tuytelaars, and L. Van Gool, “Speeded- up robust features (surf),” Comput. Vis. Image Underst., vol. 110, pp. 346–359, June 2008. [Online]. Available: ACKNOWLEDGMENT http://portal.acm.org/citation.cfm?id=1370312.1370556 The work was supported by the ADIWA project. 30