The fusion of different forensic modalities for arriving at a decision of whether the evidence can be attributed to a known individual is considered. Since close similarity and high dimensionality can adversely affect the process, a method of score fusion based on discretization is proposed. It is evaluated considering the signatures and fingerprints. Discretization is performed as a filter to find the unique and discriminatory features of each modality in an individual class before their use in matching. Since fingerprints and signatures are not compatible for direct integration, the idea is to convert the features into the same domain. The features are assigned an appropriate matched score, M S bp which are based to their lowest distance. The final scores are then fed to the fusion, F S bp . The top matches with F S bp less than a predefined threshold value, η are expected to have the true identity. Two standard fusion approaches, namely Mean and Min fusion, are used to benchmark the efficiency of proposed method. The results of these experiments show that the proposed approach produces a significant improvement in the forensic identification rate of fingerprint and signature fusion and this findings support its usefulness.
The goal of forensic analysis is that of determining whether observed evidence can be attributed to an individual. The final decision of forensic analysis can take one of three values: identification/no-conclusion/exclusion. Biometric systems have a similar goal of going from input to conclusion but with different goals and terminology: biometric identification means determining the best match in a closed set of individuals and verification means whether the input and known have the same source. While biometric systems attempt to do the entire process automatically, forensic systems narrow-down the possibilities among a set of individuals with the final decision being made by a human examiner. Automatic tools for forensic analysis have been developed for several forensic modalities including signatures [1], fingerprints [2], handwriting [3], and footwear prints or marks [4]. In both forensic analysis and biometric analysis more than one modality of data can be used to improve accuracy [5], [6]. Examples of the need to combine forensic evidence in forensic analysis are: signature and fingerprints on the same questioned document, pollen found on the clothing of an assailant together with human DNA [7], multiple shoe-prints in a crime scene [8], etc. In this paper we explore how evidence of different modalities can be combined for the forensic decision. Biometric identification systems such as token based and password based identification systems, unimodal identification recognizes a user, by "who the person is", using a one-to many matching process (1:M) rather than by "what the person carries along". Conventional systems suffer from numerous drawbacks such as forgotten password, misplaced ID card, and forgery issues. To address these problems, unimodal based identification was developed and has seen extensive enhancements in reliability and accuracy of identification. However, several studies have shown that the poor quality of image samples or the methodology itself can lead to a significant decreasing in the performance of a unimodal based identification system [9], [10], [11]. The common issues include intra-class variability, spoof attack, non-universality, and noisy data. In order to overcome these difficulties in unimodal identification, multimodal based identification systems (MIS) have been developed. As the name suggests, in an MIS the identification process is based on evidence presented by multiple modality sources from an individual. Such systems are more robust to variations in the sample quality than unimodal systems due to the presence of multiple (and usually independent) pieces of evidence [12]. A key to successful multimodal based system development for forensic identification, is an effective methodology organization and fusion process, capable to integrate and handle important information such as distinctiveness characteristic of an individual. Individual's distinctive characteristics is unique to forensic. Therefore, in this paper, the multi-matched scores based discretization method is proposed for forensic identification of an individual from different modalities. Compared to previous methods, the proposed method is unique in the sense that the extracted features correspond to the individuality of a particular person which are discretized and represented into standard sizes. The method is robust and capable to overcome dimensionality issues without requiring image normalization. The low dimension and standardized features make the design of post-processing phase (classifier or decision) straightforward. Moreover, the clear physical meanings of the discretized features are meaningful and distinctive, and be used in more complex systems (e.g., expert systems for interpretation and inference).
In identification systems, fusion takes into account a set of features that can reflect the individuality and characteristics of the person under consideration. However, it is difficult to extract and select features that are discriminatory, meaningful and important for identification. Different sets of features may have better performance when considering different groups of individuals and therefore, a technique is needed to represent for each sample set of features. In this paper, multimatched scores fusion based discretization is proposed for forensic identification to represent the distinctiveness in multimodalities of an individual.
Extracting and representing relevant features which contains the natural characteristics of an individual is essential for a good performance of the identification algorithms. Existing multimodal based identification systems make the assumptions that each modality feature set from an individual is local, wide-ranging, and static. Thus, these extracted feature sets are commonly fed to individual matching or and classification algorithms directly.
As a result, the identification system becomes more complex, time consuming, and costly because a classifier is needed for each modality. Furthermore, concatenating features from different modalities after the feature extraction method leads to the need of comparing high dimensional, heterogeneous data which is a nontrivial issue. However, much work has been proposed to overcome the dimensional issues in extracted features such as implementation of normalization techniques after extraction. Careful observation and experimental analysis need to be performed in order to improve the performance of identification. Too much of normalization will diminish the originality characteristic of an individual from different modality images. Thus, another process is needed to produce a more discriminative, reliable, unique and informative feature representation to represent these inherently multiple continuous features into standardized discrete features (per individual). This leads to the multi-matched score fusion discretization approach introduced in this paper which is explored in the context of forensic identification of different modalities for distinguishing a true identity of a person.
Discretization is a process whereby a continuous valued variable is represented by a collection of discrete values. It attracted a lot of interest from and work in several different domains [13], [14], [15]. The discretization method introduced here is based on discretization defined in [16].
Given a set of features, the discretization algorithm first computes the size of interval, i.e., it determines its upper and lower bounds. The range is then divided by the number of features which then gives each interval upper and lower approximation. The number of intervals generated is equal to the dimensionality of the feature vectors, maintaining the original number of extracted features from different extraction methods in this study. Subsequently, a single representation value for each interval, or cut, is computed by taking the midpoint of the lower approximation,Approx lower and upper approximation, Approx upper interval. Algorithm 1 shows the discretization steps discussed above.
Algorithm
For signature, the input image is first binarized by adaptive thresholding, followed by morphology operations (i.e., remove and skel) to get the gray level of clean and universe of discourse signature image (UOD) as illustrated in Fig. 1. The UOD of signature is extracted using geometry based extraction approach [17], which is based on 3x3 window concept. The process is done on individual window instead of the whole image to give more information of the signature image icludes the positions of different line structures. Unimodal extraction and the discretization step are illustrated in Table I for signature data for individual 1, and Table II for the fingerprint data for the same individual.
In each of these tables, the feature values are divided into predefined number of bins, which is based on the number of features for each modality image.
In the top portion of these tables, for each bin, the lower and upper values are recorded in columns two and three respectively, and bin, RepV alue, the average of lower and upper values, is recorded in column four. Max and Min values are highlighted in bold face.In the bottom portion of the table, the discretized features for signature and fingerprint are displayed. These tables shows an example of how the actual feature sets from individual are discretized.
As it can be seen from the Table I, the feature values, 35.259 occurs for every column of the nine features for the signature data of the same individual.This means that the first individual is uniquely recognized by this discriminatory value. A similar discussion holds for Table II, where the set of discriminatory values for fingerprint data for first individual, obtained from four different images is 104.
The selected features are the representation values (Discriminatory features, DF of an individual) that describe the unique characteristics of an individual which will be used for matching process. In matching module, the distance between the discretized values with the stored feature values are computed by Euclidean Distance equation as defined in (1).
Where Df bp,i represents ith discretized feature of new modality image meanwhile Df (r) bp,i defines the ith discretized feature of reference modality image in stored template and bp represents either behavioral or phisiological trait of the individual. The ith total number of features extracted from a single modality image is denoted by N. Let X sign = ED sign (x), where X sign = (x 1 , ...x d ) denotes a distance for discretized signature features and Y f inger = ED f inger (y), where Y f inger = (y 1 , ...y d ) is a distance for the discretized fingerprint features. The lowest distance for signature can be denoted as min[ED sign (x)] and lowest distance for fingerprint can be defined as min[ED f inger (y)]. Then, we define the modality features with the lowest distance as match score-1,(M S bp = 1), the second modality features with the second lowest distance as M S bp = 2 and so on. bp here defines either behavioral(i.e., signature) or phisiological(i.e., fingerprint) trait of the individual. Then, the match score, M Sbp is fed to the fusion approach.
After matching, the matched scores of signature and fingerprint are fed to the fusion method. Let X sign =M S sign (1),M S sign (2),...M S sign (n) denotes the computed signature match scores and Y f inger=M S f inger (1),M S f inger (2),...M S f inger (n) defines the computed match scores for fingerprint.In this work, the final fused score, F S bp of the individual are computed using Equation ( 2), where k represents the number of different modalities of an individual. The M S for fingerprint and signature are combined and divided by k to generate a single score which is then compared to a predefined threshold to make the final decision.
Fusion approaches, namely Mean, M eanF S bp and Min, M inF S bp fusion as defined in (3) and ( 4) are chosen for comparisson to show the efficiency of the proposed method on multi-modalities identification.
Finally, the F S bp is forward to next phase for identification.
In identification process of one-to-many matching (1:M), F S bp is compared with the predefined identification threshold, η in order to identify the individual from M individuals. In this work, the identity of a person is identified if,
The performance of this work is performed using ROC curve which consists of Genuine Acceptance Rate (GAR) of a system mapped against the False Acceptance Rate (FAR). In this work, GAR is equal to 1-FRR. Fig. 1 shows the performance of Unimodal identification for signature and fingerprint. Discretization is applied in this experiment. No normalization and fusion methods are implemented. The performance of the identification for both discretized signature and fingerprint and non-discretized dataset is compared. From ROC graph, clearly defines that the use of discretization on the unimodal dataset enhances the overall performance of identification significantly over the performance of identification without discretization. Due to efficiency of the discretization method on unimodal identification, thus, the same technique is applied to multimodal identification in order to improve the accuracy of identification on multiple modalities. Fig. 2 and Fig. 3 below shows the performance of ROC graph for two different fusion methods namely Mean fusion rule and Min method with the implementation of Z-Score normalization and matched scores fusion based discretization approach on multiple modalities. From the ROC graph depicted in Fig. 2, it can be seen that the implementation of the proposed method based discretization on the multi-modalities fusion of signature and fingerprint shows a better performance than the standard signature and fingerprint identification system. At FAR of 0.1%, 1.0%, and 10.0%, the implementation of the proposed method which is based on discretization has a GAR of 96.9%, 98.9%, and 99.9% respectively, where the performance is better than the Z-score normalization and Mean fusion on signature and fingerprint modalities, 93.5%, 93.7%, and 96.4%. Fig. 3 shows the GAR performance on Min fusion based Zscore normalization and the proposed multi-matched score based discretization. Again, in Fig. 3, interestingly, the proposed method based on discretization on signature and fingerprint modalities yields the best performance over the range of FAR. At 0.1%, 1.0%, and 10.0% of FAR, the Min fusion method works the best with proposed method, 95.0%, 97.99%, and 99.40% respectively. Therefore, it can be summarized that the used of discretization and proposed fusion of fingerprint and signature modalities generally performs well over the use of normalization and conventional fusion approaches for personal identification.
A key to successful multimodal based system development for forensic identification, is an effective methodology organization and fusion process, capable to integrate and handle important information such as distinctiveness characteristic of an individual. In this paper, the match scores discretization is proposed and implemented on different modality datasets of an individual. The experiments are done on signature and fingerprint datasets, which consist of 156 students (both female and male) where each student contributes 4 samples of signatures and fingerprint.
Ten features describing the bifurcation and termination points of fingerprint, were extracted using Minutia based extraction approach whereas signature is extracted using Geometry based extraction approach. In matching process, each template-query pair feature sets is compared using Euclidean distance. Two fusion approaches namely Mean and Min fusion are performed to seek for the efficiency of the proposed method in Multimodal identification. The experimental results show that the proposed multi-matched scores discretization perform well on multiple set of individual traits, consequently improving the identification performance.
This work is supported by The Ministry of Higher Education (MOHE) under Research University Grant (GUP) and My-brain15. Authors would especially like to thank Universiti Teknologi Malaysia, Skudai Johor Bahru MALAYSIA for the support and Soft Computing Research Group (SCRG) for their excellent cooperation and contributions to improve this paper.