Keystroke dynamics recognition is a crucial element in enhancing security, enabling personalized user authentication, and supporting various identity verification systems. This study offers a comparative analysis of a nine-variate prediction ellipsoid for normalized data and machine learning algorithms specifically, autoencoder, isolation forest, and one-class support vector machine for keystroke dynamics recognition. Traditional methods often assume a multivariate normal distribution. However, real-world keystroke data typically deviate from this assumption, negatively impacting model performance. To address this, the dataset was normalized using the multivariate Box-Cox transformation, allowing the construction of a decision rule based on a nine-variate prediction ellipsoid for normalized data. The study also includes The results revealed that the application of the Box-Cox transformation significantly enhanced both the accuracy and robustness of the prediction ellipsoid. Although all models demonstrated strong performance, the nine-variate prediction ellipsoid for normalized data consistently outperformed the machine learning algorithms. The study highlights the importance of careful feature selection and multivariate normalizing transformations in keystroke dynamics recognition. Future studies could benefit from broader datasets that include a wider range of user behaviors, such as variations in environmental factors and longer key sequences.
In recent years, keystroke dynamics recognition has become an effective method for biometric
authentication. By analyzing the unique patterns and rhythms individuals demonstrate while typing,
such as keystroke duration and the intervals between key presses [
The keystroke recognition process involves several essential stages to ensure accurate user
authentication. It begins with the collection of a dataset, typically consisting of timestamps for
keypress and key release events. From this raw data, key attributes such as hold times and inter-key
intervals are extracted, which reveal the unique typing behavior of the user. A critical preprocessing
step is the detection and removal of outliers - data points that significantly deviate from the expected
behavior and could otherwise distort the results [
In traditional recognition tasks, classification typically involves assigning an object to one of
several predefined categories. However, in the context of keystroke dynamics, one-class
classification is more frequently employed. Unlike standard classification methods, which rely on a
balanced dataset with both positive and negative examples, one-class classification focuses on
without the need for negative samples. This approach is particularly beneficial in authentication
systems, where the goal is to continuously verify that the current user matches the known profile,
rather than distinguishing between multiple users [
Prediction ellipsoids [
Mathematical modeling techniques are pivotal in the field of keystroke dynamics recognition, aimed
at improving accuracy and reliability. Recent advancements have integrated a range of approaches.
Tree-based models, like random forests [
However, for user authentication systems, one-class classification is more commonly employed
[
The use of prediction ellipsoids relies on the assumption that data conforms to a multivariate
normal distribution [
This study focuses on comparing prediction ellipsoid for normalized data and machine learning algorithms such as OCSVM, IF, and AE, which are widely used and offer distinct approaches to oneclass classification. In the context of keystroke dynamics recognition, accuracy, and efficiency are critical, making it essential to evaluate the effectiveness of different approaches. While prediction ellipsoid offers interpretability and computational efficiency, it can encounter limitations when dealing with non-Gaussian data distributions. On the other hand, machine learning algorithms such as OCSVM, IF, and AE provide alternative techniques, each with its own advantages and challenges when applied to keystroke dynamics.
In keystroke dynamics recognition, the quality and structure of the dataset play a crucial role in determining the performance and accuracy of the applied algorithms. A typical keystroke dynamics dataset records various temporal characteristics of an individual's typing behavior, including metrics such as the duration of key presses and the intervals between consecutive key events.
This study utilizes the CMU keystroke dynamics dataset, which captures detailed typing data dataset records various keystroke timing features in seconds, including how long each key is pressed and the intervals between key presses. Data collection was conducted over eight distinct sessions per subject, with at least one day between sessions. Each session required subjects to type the password 50 times, resulting in 400 samples per individual and a total of 20,400 samples across all participants.
The dataset is organized by subject identifier, session number, repetition count, and 31 timing features. Columns are labeled to reflect specific keystroke metrics: H.key denotes the hold time for a particular key, measuring the duration from key press to release. DD.key1.key2 represents the keydown-keydown interval, i.e., the time between pressing two consecutive keys, while UD.key1.key2 indicates the keyup-keydown interval, measuring the time between releasing one key and pressing the next. Notably, UD times can be negative in some cases, and the sum of H times and UD times corresponds to the DD time for a given digraph.
To simplify the modeling process, this study focuses on 9 key properties, hold time for a particular key, forming the feature vector: X = { H.t, H.i, H.e, H.5, H.R, H.o, H.a, H.n, H.l }.
After extracting feature vectors, the subsequent step involves detecting and removing outliers. This process is crucial because outliers can distort the analysis and undermine the performance of recognition models. By eliminating these anomalies, the dataset is refined, ensuring that the data better reflects typical user behavior, which in turn enhances model training.
One commonly used method for outlier detection is based on the squared Mahalanobis distance
(SMD). However, SMD assumes that the data follows a multivariate normal distribution, which might
not always be the case. To verify this assumption, it is necessary to assess the data's normality
through statistical tests like the Mardia test, which is used in the study [
The Mardia test calculates skewness scaled by /6, which follows a chi-square distribution with ( + 1)( + 2)/6 degrees of freedom, where is the number of variables and is the sample size. Kurtosis is compared to the normal distribution, with a mean of ( + 2) and a variance of 8 ( + 2)/ . By comparing the calculated skewness and kurtosis values with those expected under a normal distribution, the test helps identify significant deviations from multivariate normality. If the data deviates significantly, normalization is required to transform a non-Gaussian vector = 1, 2, … , 9 into a Gaussian vector = 1, 2, … , 9 .
Normalization transformations are essential in data analysis and machine learning, as they help stabilize variance, reduce skewness, and better align data with a multivariate Gaussian distribution. Univariate transformations, such as logarithmic transformations and univariate BCT, are typically applied to individual features. The logarithmic transformation is effective for stabilizing variance in positively skewed data, while the univariate BCT can handle both positive and negative skewness by optimizing a interdependent, and the BCT can be sensitive to outliers due to the complexity of parameter estimation.
A prediction ellipsoid is a multivariate tool used to assess whether a data point belongs to a specific
target class. It operates by calculating the SMD for each point, which forms the left side of the
comparison equation. This distance is then measured against a critical value derived from the
chisquare distribution, which serves as the right side of the equation [
− ̄ ) = 92, 0.005.
The SMD follows a chi-square distribution with degrees of freedom corresponding to the number of features in the data, which in this case is 9. This allows for the calculation of a critical value based on the desired significance level, commonly set at 0.005 for one-class classification tasks. If a data different class. If the SMD falls below the threshold, the point is considered an instance of the target an anomaly, meaning it is likely part of a
In contrast, multivariate transformations like the multivariate BCT consider the relationships between multiple features. The multivariate Box-Cox transformation builds upon the principles of the univariate Box-Cox transformation but applies it across multiple variables at once: = (λ ) = {( λ − 1)/λ , ln( ) , λ ≠ 0; ellipsoid, signifying its membership in the target class.
While it is more computationally demanding, this transformation preserves correlations between
variables, offering a more robust approach to normalizing complex datasets. The multivariate BCT
improves the alignment of data with a multivariate normal distribution by optimizing parameters
through methods such as maximizing the log-likelihood of transformed data, as discussed in the
study [
After normalization, outlier removal is performed iteratively using the SMD method, removing one data point per iteration based on the largest distance. This ensures that the most extreme values are eliminated first, leading to a cleaner and more representative dataset for subsequent analysis. (1) (2) (3)
OCSVM constructs a decision boundary that separates target data from the rest of the feature space
by finding a hyperplane with the maximum margin. This boundary is optimized by maximizing the
distance between the hyperplane and the origin within a high-dimensional feature space. The
OCSVM employs an implicit transformation function, denoted as φ(∙), which is a non-linear
projection evaluated through a kernel function. This kernel function maps the original feature space
into a potentially higher-dimensional one: ( , ) = φ( ) ∙ φ( ) [
Several kernel functions are commonly used in OCSVM. The linear kernel computes dot products in the original feature space, making it ideal for linearly separable data. The polynomial kernel captures non-linear relationships by raising dot products to specific powers, allowing it to model more complex decision boundaries. The radial basis function kernel, using a Gaussian function, effectively captures intricate relationships, particularly in cases where data is not linearly separable. The sigmoid kernel, based on the hyperbolic tangent function, excels at capturing non-linear patterns, making it useful for handling complex relationships between features and classes.
The decision boundary that OCSVM learns is defined by the following equation: ( ) = φ( ) − , where represents the normal vector of the hyperplane, and is the bias term.
OCSVM is formulated as a quadratic optimization problem, aiming to minimize the weight vector while maximizing the margin, subject to specific constraints. The optimization problem can be expressed as: vectors. function is: where ξ are slack variables that account for separation errors, and ν ∈ (0,1] is the regularization parameter, which controls the balance between the number of outliers and the number of support
The optimization problem is typically solved in its dual form, producing a decision function that classifies new data points as either belonging to the target class or as anomalies. The final decision ( ) =
( ( )).
The function returns a positive value for data points belonging to the target class and a negative value for anomalies.
the forest.
following formula:
Unlike traditional methods that rely on modeling target points, IF takes a distinctive approach by
focusing directly on isolating anomalies. This technique works by constructing isolation trees, where
internal nodes represent features and their split values, and leaf nodes represent individual data
points. The construction of isolation trees begins by randomly selecting a feature and a
corresponding split value within its range. This random process continues until each data point is
isolated in its own leaf node or until a specified maximum tree depth is reached [
The anomaly score for each data point is derived based on its average path length using the where (ℎ( )) is the average path length of data point across isolation trees: and ( ) represents the average path length of an unsuccessful search in a binary tree: ( , ) = 2
(ℎ( )) − ( ) , (ℎ( )) = ∑ =1 ℎ ( ) where ( ) = ln( ) + γ, and γ
Data points with shorter path lengths, closer to the root of the tree, are more likely to be anomalies, while those with longer paths are considered targets. Based on the anomaly scores, a threshold is set to classify data points as either anomalies or normal. Points with scores above the threshold are flagged as anomalies, while those below are classified as normal data points.
An autoencoder is a type of artificial neural network designed for learning efficient data representations, dimensionality reduction, and anomaly detection. As an unsupervised learning method, it consists of two key components: an encoder and a decoder.
The primary goal of an autoencoder is to learn a compressed and meaningful representation of
the input data. The encoder's function is to map the input data into latent space, effectively
compressing the data into a lower-dimensional form. This is typically achieved through a series of
layers, where each layer applies non-linear transformations to the input data. The resulting latent
space captures the most relevant features and patterns of the input, condensing its essential
information. The decoder, on the other hand, is tasked with reconstructing the original input data
from its latent space representation. Its architecture generally mirrors that of the encoder, but in
reverse, and it applies a series of non-linear transformations to transform the latent representation
back into the original data format [
During training, the autoencoder aims to minimize reconstruction error, which quantifies the difference between the original input and the reconstructed output. This is typically achieved by optimizing a loss function, such as mean squared error or binary cross-entropy, using gradient-based methods like backpropagation.
In recognition tasks, the autoencoder is trained using only instances of the target class, allowing it to learn the typical patterns and structure of normal data. When the autoencoder encounters new data, it will reconstruct the input with a low error if it belongs to the target class. However, if the input represents an anomaly, the reconstruction error will be higher, as the autoencoder is not wellequipped to accurately reconstruct unfamiliar instances. By establishing a threshold for the reconstruction error, anomalies can be detected and distinguished from normal instances.
In one-class classification, where the objective is to differentiate between target instances and
anomalies, evaluation metrics such as specificity, recall, precision, F1 score, and accuracy are crucial
for assessing model performance [
These metrics are derived from the classification outcomes, which can be categorized into four groups: true positives (TP), representing correctly identified anomalies; false positives (FP), indicating instances mistakenly classified as anomalies; true negatives (TN), denoting correctly identified target instances; and false negatives (FN), reflecting actual anomalies that were misclassified as target instances. proportion of accurately identified target instances out of all target instances: = =
+
Recall gauges the model's ability to detect all actual anomalies, measuring the proportion of true anomalies correctly identified out of all existing anomalies: +
Precision assesses the model's reliability when identifying anomalies, showing the proportion of true anomalies among all instances classified as anomalies: +
F1 score provides a balanced evaluation by calculating the harmonic mean of precision and recall, offering a single metric that accounts for both aspects:
∗ 1 = 2 ∗ .
+
Finally, the accuracy metric measures the overall correctness of the classification, taking both target instances and anomalies into account: + = .
+ + +
After constructing the models, they will be evaluated using these metrics, enabling a comprehensive analysis of their performance.
For the experiments, data with the identifier s015 was randomly selected for analysis, while data from s004 was used as a test set to evaluate the recognition of keystroke dynamics from a different individual. Outlier detection began by assessing whether the s015 dataset adhered to a multivariate normal distribution. The Mardia test revealed significant deviations, as the test statistic for multivariate skewness 1/6, at 391.54, exceeded the chi-square critical value of 215.53 for 165 degrees of freedom at a significance level of 0.005. Similarly, the multivariate kurtosis statistic 2, with a value of 113.32, surpassing the critical value of 102.62 for a mean of 99, variance of 1.98, and significance level of 0.005, indicating non-normality and necessitating further normalization.
Normalization parameters were estimated using the maximum likelihood method, yielding the following estimates for the multivariate BCT: λ̂1 = 0.9939, λ̂2 = 1.3605, λ̂3 = 1.2202, λ̂4 = 1.7521, λ̂5 = 2.2965, λ̂6 = 1.0447, λ̂7 = 1.6466, λ̂8 = 1.3512, λ̂9 = 2.0599.
After applying the nine-variate Box-Cox transformation with components (1), the Mardia test was performed again. The skewness statistic 1/6 was reduced to 212.07, which is below the chisquare threshold of 215.53, but the kurtosis statistic 2 remained slightly elevated at 109.01, still above the critical value of 102.62. Despite some remaining non-normality, primarily due to outliers, the transformed dataset better approximated a multivariate normal distribution, improving the conditions for using SMD.
Subsequently, SMD was computed for each feature vector to identify potential outliers. These distances were compared to the chi-square critical value of 23.59 for 9 degrees of freedom at a 0.005 significance level. Any vectors with SMD exceeding this value were classified as outliers. The most extreme outlier, vector number 295 with an SMD of 37.44, was removed.
This process of outlier removal was iteratively repeated until all extreme points were excluded. After eliminating 6 outliers, the multivariate kurtosis statistic finally fell below the critical value, confirming that outliers had a substantial impact on the dataset's distribution.
Table 1 lists the SMD values and the corresponding indices for each outlier that was removed. This iterative process continued until no further significant outliers were detected, resulting in a refined dataset that was less affected by extreme values.
To mitigate any potential bias related to the order of the data, the final sample was randomly shuffled to ensure an even distribution across the training and test sets. The shuffled data was then split into two equal parts, with 195 vectors in each set.
The training set was utilized to build both the prediction ellipsoid and the machine learning models, allowing them to capture the underlying patterns and relationships within the data. Meanwhile, the test set was reserved to assess the performance of the models on data not previously encountered during training.
The prediction ellipsoid should be constructed using data that follows a normal distribution, so verifying the data's normality is a necessary first step. Based on the Mardia test results, the multivariate distribution of this training sample deviates from normality. The test statistic for multivariate skewness 1/6 is 286.99, exceeding the critical value of 215.53 from the chi-square distribution for 165 degrees of freedom at a 0.005 significance level. Additionally, the test statistic for multivariate kurtosis 2 is 105.43, also exceeding the critical value of 104.19, given a mean of 99, a variance of 4.062, and a 0.005 significance level.
To address this non-normality, the training set is normalized using a nine-variate BCT. The optimal parameters for this transformation were estimated using the maximum likelihood method: λ̂1 = 1.3676, λ̂2 = 1.4807, λ̂3 = 1.078, λ̂4 = 1.7393, λ̂5 = 2.1004, λ̂6 = 1.1498, λ̂7 = 1.566, λ̂8 = 1.1685, λ̂9 = 2.1146.
After applying the BCT with components (1), the normalized training set has a mean vector ̅= {0.70932; 0.66184; 0.86764; -0.57016; -0.47427; 0.81642; 0.62417; -0.81443; -0.47084}. The covariance matrix is presented in Table 4.
The Mardia test performed on the normalized training set indicates conformity with multivariate normality. The test statistic for multivariate skewness 1/6 is 175.47, which is below the critical value of 215.53. Similarly, the test statistic for multivariate kurtosis 2 is 99.76, which does not exceed the critical value of 104.19, confirming that the normalized set follows a multivariate normal distribution.
This section outlines the implementation of various machine learning algorithms used to recognize the keystroke dynamics data. Specifically, we explore the One-Class Support Vector Machine, Isolation Forest, and autoencoder models, each selected for their unique capabilities in anomaly detection and one-class classification.
The OCSVM is implemented in Python using the OneClassSVM object from the scikit-learn library. This implementation allows for the customization of several critical parameters, including determines the acceptable proportion of training errors and establishes an upper limit for the fraction of outliers in the training dataset. The radial basis function kernel was chosen for its flexibility in modeling non-linear relationships among data points. The gamma parameter is set to "auto," allowing its value to be computed automatically based on the inverse of the number of features, influencing the range for each training example; lower values extend the influence while higher values localize it.
The IF algorithm, also implemented through sci-kit-learn [
Additional significant parameters include n_estimators, which denotes the number of decision trees in the forest (set at 100), max_samples, indicating the maximum number of samples per tree (set to 256), and max_features, specifying the maximum number of features for splitting each node (set to 1.0 to utilize all features). To classify a sample as either target or anomalous, we compare the anomaly score against a defined threshold. The scores can range from negative to positive values; negative scores indicate a higher likelihood of being a target, while positive scores suggest a greater probability of being anomalous. The selection of the threshold value is application-dependent; in this analysis, a threshold of 0 yielded optimal results.
For the AE model, we utilized TensorFlow and Keras, leveraging their combined strengths in flexibility, scalability, and ease of use. Keras, as a high-level API for building neural networks atop TensorFlow, simplifies the process of constructing and training models. Meanwhile, TensorFlow provides the essential computational framework, ensuring efficient performance during training and inference.
Before passing the data into the neural network, min-max normalization is applied to each feature
individually, scaling all features to a range of [
The AE architecture consists of an input layer configured to accept a nine-variate representation
of the data. The model includes fully connected layers for encoding and decoding operations. During
the encoding phase, the input data is compressed into a lower-dimensional representation,
progressively reducing dimensionality from 9 to 8 and then to 6, creating a bottleneck in the network
structure. This bottleneck layer compels the model to capture essential features of the input data
while minimizing redundancy [
Each encoding layer employs rectified linear unit ReLU activation functions, introducing
nonlinearity that facilitates the extraction of complex features. The decoding phase reverses this process,
expanding the dimensionality back to 8 and ultimately to the original 9 dimensions, using ReLU
activation functions to retain the learned non-linear relationships. The final layer utilizes a sigmoid
activation function to constrain output values within the range of [
To train the model, we employed the Adam optimizer in conjunction with binary cross-entropy
loss, a standard metric for reconstruction tasks aimed at minimizing the discrepancy between the
original and reconstructed data. The Adam optimizer combines the strengths of AdaGrad and
RMSProp [
Table 5 displays a comparison of the recognition performance among various methods, including the Prediction Ellipsoid for Non-Gaussian Data (PENGD) (1), the Prediction Ellipsoid for Normalized Data (PEND) (7), One-Class Support Vector Machine (OCSVM), Isolation Forest (IF), and Autoencoder (AE).
PENGD PEND OCSVM
IF AE
All models evaluated in this study demonstrate commendable performance in keystroke dynamics recognition. However, the PENGD stands out with the lowest accuracy among the models assessed, indicating that while it can capture some patterns, it may struggle with more complex datasets, particularly due to the challenges posed by non-Gaussian data distributions. On the other hand, both the OCSVM and AE exhibit very good performance across multiple metrics, reflecting their capabilities in identifying true anomalies with high precision and recall. These models effectively leverage their respective architectures to capture intricate relationships within the data, contributing to their robust performance. In contrast, the IF did not perform as well as the other models.
Ultimately, the PEND emerged as the best-performing model, achieving the highest scores across key evaluation metrics. This reinforces the significance of normalization transformations in enhancing prediction ellipsoid models for recognition tasks, particularly in scenarios involving nonGaussian data distributions.
All models in this study exhibit strong performance in keystroke dynamics recognition, but PEND stands out as the best performer. The precision, recall, and F1 score of this model are the highest, demonstrating its ability to handle keystroke dynamics recognition tasks with remarkable accuracy. The performance of OCSVM and AE is also notable, offering very good results, while IF lags slightly behind the others.
The findings underscore that applying the nine-variate BCT played a critical role in boosting model performance, particularly by improving how the models handle non-Gaussian data. Multivariate transformations like BCT take into account the correlations between variables, allowing for a more accurate and comprehensive prediction ellipsoid. This, in turn, enhances the model's ability to identify intricate patterns in the data, improving both its accuracy and reliability.
However, there are certain disadvantages to using prediction ellipsoid for normalized data. A robust model typically requires a dataset of at least 100 instances, which can be a challenge for smaller datasets. Additionally, selecting the most appropriate normalization transformation can be complex, especially for datasets with intricate distributions or a large number of outliers. Another important factor is the choice of significance level, as this can influence the efficiency and reliability of the prediction ellipsoid.
Limitations also arise from the outlier removal process, as deleting 10 outliers during preprocessing may cause the model to miss some underlying patterns in the data. To mitigate this, more advanced normalization techniques, such as the Johnson transformation, could be considered to better align the model with the dataset's distribution, improving its ability to generalize across all relevant data points.
In this paper, the primary aim was to address the challenge posed by non-Gaussian data distributions in the context of biometric identification based on keystroke dynamics. Emphasizing the importance of normalization techniques, specifically the multivariate Box-Cox transformation, to enhance model accuracy with such data.
The dataset used in this study represents a 10-character password length, which may not be optimal for real-world applications. A password length of 20-22 characters, without the use of uppercase characters, is generally considered preferable, as it allows for more comprehensive feature extraction. Beyond keystroke length and character variety, several contextual factors, such as the 30], were not considered in this research. However, these factors could play an important role in biometric identification based on keystroke dynamics.
In future research, a broader dataset that includes data reflecting the impact of environmental factors could be used, along with extended key sequences. The inclusion of these factors would provide a more realistic representation of user behavior. Additionally, the application of other normalization techniques, such as the Johnson transformation, could further enhance model accuracy by addressing the complexity of non-Gaussian distributions.
The focus of this paper was to address the challenges associated with non-Gaussian data distributions in the context of keystroke dynamics recognition. The study compared the performance of prediction ellipsoid models and machine learning algorithms, including OCSVM, IF, and AE. All models demonstrated a high probability of recognition. Notably, the prediction ellipsoid for non-Gaussian data had the lowest accuracy, highlighting the challenges posed by complex datasets. However, by applying the multivariate BCT, the prediction ellipsoid model for normalized data showed significant performance improvements, emphasizing the critical role of normalization when addressing nonGaussian data distributions. The BCT not only improved the overall accuracy but also deepened the understanding of data patterns by considering correlations between variables, ultimately leading to a more precise prediction ellipsoid.
Despite these advancements, the study identified certain limitations and challenges. One significant drawback is the necessity for a large dataset, as constructing a reliable prediction ellipsoid model generally requires at least 100 instances. Furthermore, selecting the optimal normalization transformation remains a complex task, especially when dealing with datasets that contain outliers or exhibit highly intricate distributions. Another challenge lies in determining the appropriate significance level, which directly affects the reliability and efficiency of the prediction ellipsoid.
Looking ahead, future research could expand the dataset to include factors like environmental factors, as well as extended key sequences, to provide a more realistic representation of user behavior.
The incorporation of alternative normalization techniques, such as the Johnson transformation, could further enhance model accuracy by addressing the impact of non-Gaussian data. Further investigation into model complexity and feature selection for both prediction ellipsoid models and machine learning algorithms could offer valuable insights for improving keystroke dynamics recognition.
. arXiv preprint