Proceedings of the 1st International Workshop on Automated Forensic Handwriting Analysis (AFHA) 2011 Static Signature Verification by Optical Flow Analysis D. Impedovo, Member, IEEE , and G. Pirlo, Member, IEEE local stability function can be obtained by using DTW to Abstract—This paper presents a new approach for static match a genuine signature against other authentic specimens. signature verification based on optical flow. In the first part of Each matching is used to identify the Direct Matching Points the paper, optical flow is used for estimating local stability of (DMPs), that are unambiguously matched points of the static signatures. In the second part, signature verification is performed by the analysis of optical flow, using an alternating genuine signature. Thus, a DMP can indicate the presence of a decision tree. The experimental tests, carried out on signature of small stable region of the signature, since no significant the GPDS database, demonstrate the validity of this approach and distortion has been locally detected. The local stability of a highlight some direction for further research. point of a signature is determined as the average number of time it is a DMP, when the signature is matched against other Index Terms—Static Signature Verification, Local Stability, genuine signatures. Following this procedure low- and high- Optical Flow. stability regions are identified [7, 8, 9] in the selection of reference signatures [10, 11] and verification strategies [12, I. INTRODUCTION 13]. H ANDWRITTEN signatures occupy a very special place in biometrics. Unlike other biometric traits, handwritten signatures have long been established as the most widespread A client-entropy measure has been also proposed to group and characterize signatures in categories that can be related to signature variability and complexity. The measure, that is means of personal verification. Signatures are generally based on local density estimation by a HMM, can be used to recognized as a legal means of verifying an individual's access whether a signature contains or not enough information identity by administrative and financial institutions. Moreover, to be successfully processed by any verification system [14, verification by signature analysis requires no invasive 15, 16]. measurements and people are familiar with the use of Other types of approaches estimate the stability of a set of signatures in their daily life [1, 2, 3]. common features and the physical characteristics of signatures Unfortunately, a handwritten signature is the result of a which they are most related to, in order to obtain global complex generation process. The rapid writing movement information on signature repeatability which can be used to underlying signing is determined by a motor program stored improve the verification systems [17, 18]. In general, these into the brain of the signer and realized though his/her writing approaches have shown that there is a set of features that system (arm, hand, etc.) and writing devices (paper, pen, etc.). remain stable over long time periods, while there are other Therefore, a signature image strongly depends on the features which change significantly in time [19, 20]. Of course, psychophysical state of the signer and the conditions under since intersession variability is one of the most important which the signature apposition process occurs [4, 5]. causes of the deterioration of verification performances, The net result is that signature variability is one of the most specific parameter-updating approaches have been considered relevant issues that must be faced to develop accurate [18, 19, 20]. signature verification systems. In general, two types of Concerning static signatures, a multiple pattern-matching variability can be distinguished in signing: short-term strategy has been recently proposed to determine - at local variability and long-term variability. Short-term modifications level - the degree of stability of each region of a signature [21, depend on the psychological condition of the writer and on the 22, 23]. In this paper the optical flow is used to estimate the writing conditions. Long-term modifications depend on the local stability of the signature images. In addition, the optical alteration of the physical writing system of the signer (arm and flow is also considered for signature verification, using an hand, etc. ) as well as on the modification of the motor alternate decision tree classifier. The experimental results, program in his/her brain [5, 6] carried out on signatures of the GPDS database, demonstrate In literature, the approaches proposed for the analysis of the validity of the approach with respect to other techniques in local stability are mainly devoted to dynamic signatures. A literature. D. Impedovo and G. Pirlo are with the Dipartimento di Informatica, II. STATIC SIGNATURE ANALYSIS BY OPTICAL FLOW Università degli Studi di Bari, via Orabona 4, 70125 Bari – Italy Two categories of signature verification systems can be (corresponding author - e-mail: pirlo@di.uniba.it). 31 Proceedings of the 1st International Workshop on Automated Forensic Handwriting Analysis (AFHA) 2011 identified, depending on the data acquisition method [1]: static Figure 1 shows an example of Optical Flow: in (a) the (off-line) systems and dynamic (on-line) systems. Static movement of a rectangle over two frames is shown; in (b) the systems perform data acquisition after the writing process has optical flow vectors is reported. been completed. In this case, the signature is represented as a grey level image I(x,y), where I(x,y) denotes the grey level at III. ANALYSIS OF STABILITY OF STATIC SIGNATURES the position (x,y) of the image. The results is that static In the next section, optical flow analysis is applied to the systems involve the treatment of the spatio-luminance analysis of regional stability of static signatures. For this representation of a signature image. Therefore, no dynamic purpose, after the preprocessing phase, in which each signature information is available on the signing process when static is binarized and normalized to a fixed rectangular area, the signatures are considered [1, 2]. Notwithstanding, static identification of the stable regions starts. signature verification is very important for many application In particular, let be: fields, like automatic bank-check processing, insurance form • Igi the set of N genuine signatures of a writer, processing, document validation and so on. When static i=1,2,,…N; signatures are considered, information on local stability is an • [uij(x,y), vij(x,y)]T the optical flow between Igi important parameters for verification aims. In this paper local and Igj . stability is analyzed by optical flow. Optical flow can be Now, if we consider the i-th signature Igi of a signer, for defined as the distribution of apparent velocities of movement each pixel Igi(x,y) we can consider the sets of optical flow of brightness patterns in an image I. As discussed in the vectors defined as: excellent paper of O'Donovan [24], optical flow has been used for a variety of computer vision applications like autonomous Ui ={uij(x,y) | j=1,2,…,N; j≠i } navigation, object tracking, traffic analysis, image segmentation and stabilization. Vi = {vij(x,y) | j=1,2,…,N; j≠i }. In this paper we consider the approach of Horn and Shunck for optical flow estimation [25]. In this case optical flow is The stability (S) of Igi(x,y) can be estimated as: determined through the minimization of the energy functional [25]: S ( I ig ( x, y )) = σ u2 + σ v2 being σu and σv the standard deviation of Ui and Vi, where respectively. • Ix, Iy and It are the derivatives of the image intensity values along the x, y and time dimensions, IV. SIGNATURE STABILITY BY OPTICAL FLOW respectively; • [uij(x,y), vij(x,y)]T is the optical flow vector; Optical flow provides useful information on local dissimilarity among signature images. In this paper this • α is the regularization parameter. information is used for signature verification aims. In In other words, the functional E consists of two terms: the particular, signature verification is performed by an alternating first term is the optical flow constraint equation and the second decision tree (ADT). ADT, that was first introduced by Freund is the smoothness constraint which is multiplied by the and Mason [26], consists of decision nodes and prediction regularization parameter α. nodes. Decision nodes specifies a predicate condition, prediction nodes contain a single number. Classification by an ADT is performed by following all paths for which all decision nodes are true and summing any prediction nodes that are traversed. More precisely, in our approach, let be: • Igi the set of N genuine signatures of a writer, i=1,2,,…N; • Ifp the set of M forgery signatures of a writer, p=1,2,…,M. (a) (b) In the enrollment stage the ADT is trained by using the Fig. 1. Example of Optical Flow. optical flow vectors concerning intra-class and inter-class variability: Horn and Schunk work out the previous minimization • [uij(x,y), vij(x,y)]T the optical flow between Igi and Igj , problem using a digital estimation of the Laplacian for the i,j=1,2,…,N, i≠j (intra-class variability); optical flow gradients, to get a large system with two equations • [uik(x,y), vik(x,y)]T the optical flow between Igi and Igk , for each pixel that can be solved by the Jacobi method [25]. i=1,2,…,N, k=1,2,…,M (inter-class variability). 32 Proceedings of the 1st International Workshop on Automated Forensic Handwriting Analysis (AFHA) 2011 ( a) (b) Fig. 2. Example Analysis of Local Stability. N   = 10 optical flows between genuine signatures and V. EXPERIMENTAL RESULTS 2 The experimental results have been carried out using static N⋅M=20 optical flows between genuine signatures and signatures of the GPDS database. The database contains 16200 forgeries are used for training. For testing, fourteen genuine signatures from 300 individuals: 24 genuine signatures and 30 and fourteen forged signatures are considered. In the testing forgeries for each individual [27]. The result here reported stage, the optical fields [uti(x,y), vti(x,y)]T between the test concerns only twenty-five signers since other experiments are signature It and each genuine signature Igi, i=1,2,…,N, are still in progress. For each signer the stability analysis is computed. Each one of the N optical flows is provided to the performed, according to the approaches described in Section ADT that returns a verification results rti. The N results are III. Figure 2 shows a genuine specimen (a) and the result of the combined according to the majority vote strategy, in order to stability analysis obtained by optical flow (b). High stability define the final verification result for the test signature It. regions are marked by continuous-line rectangles, low stability The results, in terms of Type I - False Rejection Rate (FRR) regions are marked by dotted-line rectangles. In this case the and Type II - False Acceptance Rate (FAR) are reported in stability analysis has been achieved by considering the three Table 1. On average we register a Type I error rate equal to optical flows in Figure 3, obtained by computing the optical 23% and a Type II error rate equal to 20%. Figure 4 shows an flows between the signature in Figure 2a and other three example of optical flow between two genuine specimens. genuine specimens. Figure 5 shows the optical flow between a genuine specimen Signature verification has been carried out by considering, and a forgery. The great amount of deformation is clearly for each signer, N=5 genuine signatures (Igi, i=1,…,5) and visible when the optical flow is performed between a genuine M=4 forgeries (Ifi, i=1,..,4) for training the ADT. Therefore, signature and a forgery. ( a) (b) ( c) Fig. 3. Optical Flows between genuine signatures 33 Proceedings of the 1st International Workshop on Automated Forensic Handwriting Analysis (AFHA) 2011 TABLE I Experimental Results Author PERFORMANCE n. FRR FAR 1 14% 36% 2 0% 0% 3 29% 0% 4 43% 57% 5 29% 14% 6 29% 43% 7 0% 0% 8 57% 14% 9 29% 57% 10 0% 0% Fig. 5. Optical Flow: genuine vs false 11 21% 29% 12 14% 0% 13 29% 50% VI. CONCLUSION 14 21% 14% In this paper optical flow is considered as a tool for static 15 0% 7% signature analysis. In the first part of the paper local stability in static signatures is analyzed by optical flow analysis. In the 16 14% 14% second part, optical flow vectors between test signature and 17 57% 29% genuine specimens are considered to verify the authenticity of 18 43% 36% a test signature, using an alternate decision tree. Some results 19 0% 0% carried out on static signatures extracted from the GPDS 20 21% 7% database demonstrate the new approach is worth consideration 21 14% 0% for further research. Of course, more experimental results are necessary to verify the effectiveness of the proposed approach 22 14% 14% and - in particular - to determine the capability of the Optical 23 57% 36% Flow in recognizing short-term and long-term variability as 24 36% 43% well as for evaluating the extent to which stability depends on 25 14% 7% the signature type and signer characteristics. REFERENCES [1] R. Plamondon and G. Lorette, “Automatic Signature Verification and Writer Identification – The State of the Art”, Pattern Recognition, Vol. 22, No. 2, Jan. 1989, pp. 107-131. [2] D. Impedovo, G. Pirlo, “Automatic Signature Verification – The State of the Art”, IEEE Transactions on Systems, Man and Cybernetics - Part C: Applications and Review, Vol. 38, No. 5, Sept. 2008, pp. 609 – 635. [3] G. Pirlo, “Algorithms for Signature Verification”, in Proc. of NATO- ASI Series Fundamentals in Handwriting Recognition, S. Impedovo (Ed.), Springer-Verlag, Berlin, 1994, pp. 433-454. [4] R. Plamondon, “A Kinematic Theory of Rapid Human Movements: Part I: Movement Representation and generation”, Biological Cybernetics, Vol. 72, No. 4, 1995, pp. 295-307. [5] R. Plamondon, M. Djioua, “A Multi-Level Representation Paradigm for Handwriting Stroke Generation”, Human Movement Science, Vol 25, No. 4-5, 2006, pp. 586-607. [6] S. Impedovo and G. Pirlo, "Verification of Handwritten Signatures: an Overview", Proc. 14th International Conference on Image Analysis and Processing - ICIAP 2007, IEEE Computer Society Press, September, 11- 13, 2007, Modena, Italy, pp. 191-196. [7] G. Congedo, G. Dimauro, S. Impedovo, G. Pirlo, “A new methodology for the measurement of local stability in dynamical signatures”, 4th International Workshop on Frontiers in Handwriting Recognition Fig. 4. Optical Flow: genuine vs genuine (IWFHR-4), Taipei, Taiwan, Dec. 1994, pp. 135-144. [8] G. Dimauro, S. Impedovo, R. Modugno, G. Pirlo, L. Sarcinella, “Analysis of Stability in Hand-Written Dynamic Signatures”, 8th International Workshop on Frontiers in Handwriting Recognition 34 Proceedings of the 1st International Workshop on Automated Forensic Handwriting Analysis (AFHA) 2011 (IWFHR-8), Ontario, Niagara-on-the-Lake, Canada, Aug. 2002, pp. [27] J.F. Vargas, M.A. Ferrer, C.M. Travieso, J.B. Alonso, “Off-line 259-263. Handwritten Signature GPDS-960 Corpus”, Proc. 9th ICDAR, Vol. 2, [9] K. Huang and H. Yan, “Stability and style-variation modeling for on- 23-26 Sept. 2007 pp.764-768. line signature verification”, Pattern Recognition, Vol. 36, No. 10, Oct. 2003, pp. 2253-2270. [10] G. Congedo, G. Dimauro, A.M. Forte, S. Impedovo, G. Pirlo, “Selecting D. Impedovo (IEEE member) received the M.Eng. degree "summa cum Reference Signatures for On-Line Signature Verification”, 8th laude" in Computer Engineering in 2005 and the Ph.D. degree in Computer International Conference on Image Analysis and Processing (ICIAP-8), Engineering in 2009 from the Polytechnic of Bari (Italy). He is, currently, Series: Lecture Notes in Computer Science, Vol. 974, Springer-Verlag with the Department of Computer Science of the University of Bari. His Berlin, Heidelberg, C. Braccini, L. De Floriani and G. Vernazza (Eds.), research interests are in the field of pattern recognition and biometrics San Remo, Italy, Sept. 1995, pp. 521-526. (speaker recognition ad automatic signature verification). He is co-author of [11] V. Di Lecce, G. Dimauro, A. Guerriero, S. Impedovo, G. Pirlo, A. Salzo, more than 20 articles in these fields in both international journals and L. Sarcinella, “Selection of Reference Signatures for Automatic conference proceedings. Signature Verification”, Proc. 5th International Conference on He received "The Distinction" for the best young student presentation in Document Analysis and Recognition (ICDAR-5), Bangalore, India, Sept. May 2009 at the International Conference on Computer Recognition Systems 20-22, 1999, pp. 597-600. (CORES - endorsed by IAPR). He is reviewer for the Elsevier Pattern [12] V. Di Lecce, G. Dimauro, A. Guerriero, S. Impedovo, G. Pirlo, A. Salzo, Recognition journal, IET Journal on Signal Processing and IET Journal on “A Multi-Expert System for Dynamic Signature Verification”, 1st Image Processing and for many International Conferences including ICPR. International Workshop, Multiple Classifier Systems (MCS 2000), Dr. Impedovo is IAPR and IEEE member. Series: Lecture Notes in Computer Science, Springer-Verlag Berlin Heidelberg, J. Kittler and F. Roli (Eds.), Vol. 1857, Cagliari, Italy, June G. Pirlo (IEEE member) received the Computer Science degree “cum laude” 2000, pp. 320-329. in 1986 at the Department of Computer Science of the University of Bari. [13] L. Bovino, S. Impedovo, G. Pirlo, L. Sarcinella, “Multi-Expert Since then he has been carrying out research in the field of pattern recognition Verification of Hand-Written Signatures”, 7th International Conference and image analysis. In 1988 he received a fellowship from IBM. Since 1991 on Document Analysis and Recognition (ICDAR-7), IEEE Computer he has been Assistant Professor at the Department of Computer Science of the Society, Aug. 2003, Edinburgh, Scotland, pp. 932-936. University of Bari, where he is currently Associate Professor. His interests [14] N. Houmani, S. Garcia-Salicetti, B. Dorizzi, "A novel personal entropy cover the areas of biometry, pattern recognition, intelligent systems, computer measure confronted with online signature verification systems' arithmetic, communication and multimedia technologies. performance", Proceedings of the 2nd IEEE International Conference He has developed several scientific projects and published more than 150 on Biometrics: Theory, Applications and Systems (BTAS '08), papers in the field of document analysis and processing, handwriting Washington, DC, USA, September 2008 recognition, automatic signature verification, parallel architectures for [15] S. Garcia-Salicetti, N. Houmani, B. Dorizzi, "A client-entropy measure computing, communication and multimedia technologies for collaborative for on-line signatures", Proceedings of the IEEE Biometrics Symposium work and distance learning. He served as reviewer for many international (BSYM '08), pp. 83-88, Tampa, Fla, USA, September 2008. journals and conferences. Prof. Pirlo is member of the IEEE and of the IAPR - [16] N. Houmani, S. Garcia-Salicetti, B. Dorizzi, “On assessing the International Association for Pattern Recognition TC11 (Technical robustness of pen coordinates, pen pressure and pen inclination to time Committee on “Reading Systems”). He is also in the Governing Board of the variability with personal entropy”, Proc. of IEEE 3rd International Italian Society for e-Learning (SIe-L). Conference on Biometrics: Theory, Applications, and Systems, 2009 (BTAS '09), Washington, DC, Sept. 28-30, 2009, pp. 1 – 6. [17] R. Guest, “Age dependency in handwritten dynamic signature verification systems”, Pattern Recognition Letters, Vol. 27, N. 10, 15 July 2006, pp. 1098-1104. [18] R.M. Guest, “The Repeatability of Signatures”, 9th International Workshop on Frontiers in Handwriting Recognition (IWFHR-9), Kichijoji, Japan, Oct. 2004, pp. 492-497. [19] H. Lei and V. Govindaraju, “A comparative study on the consistency of features in on-line signature verification”, Pattern Recognition Letters, Vol. 26, 2005, pp. 2483-2489. [20] Y. Kato, D. Muramatsu, T. Matsumoto, “A Sequential Monte Carlo Algorithm for Adaptation to Intersession Variability in On-line Signature Verification”, Proc. 10th Int. Workshop on Frontiers in Handwriting Recognition (IWFHR 10), La Baule, France, Oct. 2006. [21] D. Impedovo, R. Modugno, G. Pirlo, E. Stasolla, “Handwritten Signature Verification by Multiple Reference Sets”, Proc. of the 11th International Conference on Frontiers in Handwriting Recognition (ICFHR), 19-21 Aug. 2008. [22] D. Impedovo, G. Pirlo, " On the Measurement of Local Stability of Handwriting - An application to Static Signature Verification ", Proc. of Biometric Measurements and Systems for Security and Medical Applications (BIOMS 2010), September, 9, 2010, Taranto, Italy, IEEE Computer Society Press, pp. 41-44. [23] D. Impedovo, G. Pirlo, E. Stasolla, C.A. Trullo, "Learning Local Correspondences for Static Signature Verification”, Proc. 11th Int. Conf. of the Italian Association for Artificial Intelligence (AI*IA 2009), December 9-12, 2009, Reggio Emilia, Italy. [24] P. O'Donovan, “Optical Flow: Techniques and Applications”, The University of Saskatchewan, T.R. 502425, April 2005. [25] B.K.P. Horn and B.G.Schunck, “Determining Optical Flow”, MIT Press, A.I. Memo n. 572, April 1980. [26] Y. Freund and L. Mason, “The alternating decision tree learning Algorithm”, Proceedings of the Sixteenth International Conference on Machine Learning, Morgan Kaufmann, 199, pp. 124–133. 35