Face anti-spoofing systems play a crucial role in securing biometric authentication frameworks against presentation attacks. The growing complexity of spoofing techniques demands the development of advanced detection methods that can effectively generalize across various attack forms and environmental conditions. In response to challenges, a new architecture fusing Vision Transformers (ViT), ConvNeXT, and Swin Transformer is proposed for advanced face anti-spoofing. The method combines global contextual modeling with local feature extraction and multi-scale analysis. Detailed evaluations on the OULU-NPU and CASIA-MFSD datasets demonstrate competitive performance across various protocols, with notable improvements in generalization to unseen environmental conditions. Feature space visualizations reveal improved class separability post-fusion, emphasizing the effectiveness of the combined approach. Cross-dataset experiments highlight challenges in domain generalization in bidirectional evaluations between OULU-NPU and CASIA-MFSD. The proposed method advances the state-of-the-art in face anti-spoofing, offering insights into feature fusion strategies and avenues for future research in cross-domain generalization.
Face recognition technologies have rapidly evolutionized and are now essential to various security
systems, from personal devices to large-scale surveillance networks. However, the widespread
adoption of these technologies has also led to the emergence of face spoofing attacks, where
attackers use photos, videos, masks, or other facial representations to deceive recognition systems
[
The fusion of ViT and Convolutional Neural Networks (CNNs) represents a promising route for
addressing these challenges. Vision Transformers, introduced by [
The proposed methodology implies a multi-stream architecture that processes input images
through ViT and uses two parallel pathways. In this context ViT offers the potential to capture
subtle, global features that may be indicative of spoofing attacks. The proposed approach builds
upon recent advancements in face anti-spoofing research, including multi-modal fusion techniques
[
The remainder of this paper is organized as follows: Section 2 provides an overview of related works, the advancements and challenges in the domain of face anti-spoofing. In Section 3, we detail the materials and methods used in our study, including the problem statement, the proposed method, evaluation metrics, loss functions, and datasets. Section 4 presents the experimental results, demonstrating the performance of our approach on various benchmarks. Finally, Section 5 offers a discussion of the findings, their implications, and potential directions for future research.
Face anti-spoofing research has made a big growth transitioning from traditional handcrafted feature-based approaches to deep learning-driven methodologies.
Early face anti-spoofing techniques primarily relied on texture analysis to differentiate between
genuine and spoofed facial presentations. Local Binary Patterns (LBP) and its variants were
extensively employed to capture micro-textural patterns [
The deep learning settled a paradigm shift in face anti-spoofing research. CNNs became the
powerful tools for automatically learning hierarchical features from raw input images. [
The integration of attention mechanisms into face anti-spoofing models has gained significant
interest due to their ability to focus on salient regions. [
A continuous challenge in face anti-spoofing lies in the domain shift between training and
testing distributions. To address this, several works have explored domain generalization
techniques. [
The proposed architecture in Figure 1 consists of ViT, ConvNeXT [
Face anti-spoofing systems aim to differentiate between bona fide facial presentations and fraudulent attempts using various spoofing techniques, such as printed photographs, digital displays, or 3D masks. The challenge lies in capturing both fine-grained textural details and global contextual information while maintaining robustness across diverse environmental conditions and attack modalities. Formally, given an input image , where and represent height, width, and channels respectively, the objective is to learn a function where 0 denotes a spoofed presentation and 1 indicates a genuine facial image.
The proposed architecture comprises three main components. A ViT for global feature extraction, a ConvNeXT module for local feature refinement, and a Swin Transformer for multi-scale feature analysis. The fusion of outputs from ConvNeXT and Swin Transformer yields the final classification decision.
The ViT module partitions the input image I into N non-overlapping patches, each of size . In our experiment since we use 8x8 patching for 224x224x3 image, the N will become 784. (1) is our ViT model. Then is the output of i-th layer, where total number layers is 12 and is the batch size. Out of 12 layers, we use only last 4 layers, since they represent the higher level information. Each of shape
, after concatenation the tensor becomes where DWConv and PointwiseConv represent depthwise and pointwise convolutions, respectively, and GELU is the Gaussian Error Linear Unit activation function.
The Swin Transformer operates on the same concatenated feature maps, employing shifted window-based self-attention to capture multi-scale contextual information (2) (3) (4) (5)
The ConvNeXT module processes the concatenated high-level feature maps from ViT using a series of depthwise separable convolutions and inverted bottleneck layers. For each block i. where Q, K, V are query, key, and value matrices, d is the dimension of queries/keys, and B is the relative position bias. Prior to the Swin Transformer, features were passed through a convolutional layer to reduce the number of parameters, thereby facilitating more efficient training. This approach was necessary because the Swin Transformer, unlike ConvNeXT, is comparatively slower in processing.
The outputs from ConvNeXT and Swin Transformer undergo global average pooling:
BPCER measures the rate at which the system incorrectly classifies a genuine attempt as a spoofing attack. metric reflects the system's ability to correctly recognize legitimate users.
ACER is the average of APCER and BPCER. It is usually used to find a balance between aforementioned metrics: For the loss, we used Binary Cross Entropy Loss: where GAP is Global Average Pooling. The final value before feeding toSigmoid function is so that both have an equal contribution.
Performance metrics used to evaluate the effectiveness of these anti-spoofing systems include
APCER, BPCER, and ACER [
APCER measures the rate at which the system incorrectly classifies a spoofing attack as a genuine attempt. In other words, it is the proportion of spoofing attempts that are wrongly accepted as legitimate by the system. (6) (7) (8) (9) (10) (11) (12) where N - the total number of samples, - true label, - predicted probability.
The proposed method was evaluated on two widely recognized datasets in the face anti-spoofing
domain: OULU-NPU [
OULU-NPU combines 4,950 real access and spoofing videos from 55 subjects, captured using six mobile devices with front-facing cameras. The dataset provides four protocols evaluating generalization across unseen environmental conditions, attack types, input sensors, and a combination thereof. Spoofing attacks include print and video-replay using two printers and two display devices. Environmental variations encompass three sessions with different illumination and background settings.
CASIA-MFSD contains 600 video clips of genuine and attack attempts from 50 subjects. The dataset features three image quality categories: low-quality, normal-quality, and high-quality. Spoofing attacks are categorized into three types: warped photo attacks, cut photo attacks, and video attacks. The dataset was collected under varying illumination conditions and with different digital devices, presenting challenges in terms of image quality and attack diversity. The dataset provides seven scenarios from three main protocols with low, high qualities and last is a mix of all train versus mix of all test samples. For this dataset we used only the 3rd protocol, which is a 7th scenario.
For both datasets we took only every 25th frame and 5 frame in total only. Face detection was
performed using MTCNN [
In this research, the experiments were executed using an NVIDIA RTX 4090 GPU with 24GB of VRAM. Batch size is 16, which took about 17GB memory of GPU. The optimizer is Adam with initial learning rate value 0.00001 by reducing using ReduceLROnPlateau scheduler after each 3 epochs without loss decrease for 20 epochs. For the augmentations we used flipping, rotating, random cropping, blurring and changing the brightness.
The proposed method went through a different evaluations using the OULU-NPU and CASIAMFSD datasets, with performance metrics including ACER, APCER and BPCER. The experimental results are presented in Tables 1, 2, and 3, showing the model's performance across various protocols and cross-dataset scenarios.
1 2 3 4
STDN [
CDCN [
Ours
STDN [
CDCN [
Ours
STDN [
CDCN [
Ours
STDN [
CDCN [
Ours
Table 2 presents the performance on the CASIA-MFSD dataset. The proposed method achieved an APCER of 1.68%, BPCER of 1.68%, and ACER of 1.68%, indicating high performance in detecting presentation attacks while maintaining a low false rejection rate for genuine presentations.
Figure 4 presents a visual representation of the feature spaces generated by different components of the proposed architecture. The top graph illustrates the Principal Component Analysis (PCA) of the ViT output, while the bottom graph depicts the PCA after fusion of ConvNeXT and Swin Transformer blocks. Both visualizations use the same samples from the CASIA-MFSD dataset. The ViT output on top graph provides a relatively clustered distribution of features, with considerable overlap between live faces and various types of spoofing attacks. The feature space lacks clear separation between classes, indicating that the ViT alone struggles to distinguish between genuine and fake presentations consistently. In contrast, the fused output of ConvNeXT and Swin Transformer blocks on the bottom graph demonstrates a markedly improved feature distribution. The live face samples form a distinct cluster, which are shown in blue and gray, well-separated from the spoofing attack samples. The fake face presentations, including paper-based and video replay attacks, are grouped together but distinctly apart from the live face cluster.
Figure 5 illustrates the challenge of domain shift in cross-dataset scenarios. The PCA visualization shows the feature distribution when the model, trained on CASIA-MFSD, is tested on OULU-NPU. The plot reveals four distinct clusters corresponding to live faces and different types of spoofing attacks. Notably, the printed fake faces (orange cluster) are well-separated from other categories, suggesting robust detection of this attack type across datasets. However, the live faces (blue), video replay attacks (red), and another type of printed fake (green) show some overlap, indicating potential challenges in distinguishing these categories in cross-dataset scenarios. The significant shift in feature distribution between the training and testing datasets is evident from the distinct grouping of samples on the right side of the plot.
The experimental results and feature space visualizations provide valuable insights into the performance and characteristics of the proposed face anti-spoofing method. The fusion of ViT, ConvNeXT, and Swin Transformer demonstrates promising capabilities in addressing the challenges of face presentation attack detection across various scenarios.
On the OULU-NPU dataset, the proposed method shows competitive performance across all four protocols. Notably, in Protocol 1, which evaluates generalization to unseen environmental conditions, the method achieves the lowest APCER among all compared approaches, suggesting that the fusion of global and local feature extraction mechanisms effectively mitigates the impact of varying illumination and background conditions. The method's performance in Protocols 2-4 remains competitive, indicating robustness to unseen attack types and input sensors. The high performance on the CASIA-MFSD dataset further confirms the method's effectiveness in handling diverse spoofing techniques and image qualities. The lowest APCER on this dataset is particularly noteworthy, showing a strong ability to detect all presentation attacks without false acceptances.
The PCA visualizations in Figure 4 provide crucial insights into the feature learning process.
The ViT output alone shows limited separation between live faces and spoofing attacks, indicating
that global context modeling is insufficient for robust anti-spoofing. However, the fused output of
ConvNeXT and Swin Transformer blocks demonstrates a marked improvement in class
separability, indicating the importance of combining global contextual information with local
textural features and multi-scale analysis for effective spoofing detection. The clear separation
between live faces and various attack types in the fused feature space aligns with the strong
performance metrics observed on individual datasets. It suggests that the proposed architecture
successfully learns discriminative features that generalize well across different spoofing techniques
and environmental conditions within a single dataset. The cross-dataset experiments reveal both
strengths and limitations of the proposed method. When trained on OULU-NPU and tested on
CASIA-MFSD, the model achieves an ACER, which, while not optimal, indicates some degree of
generalization. On the other hand, training on CASIA-MFSD and testing on OULU-NPU yields a
better ACER, suggesting that the features learned from CASIA-MFSD may be more generalizable.
Figure 5 visualizes the domain shift problem inherent [
While the proposed method demonstrates strong performance within individual datasets, the
cross-dataset results reveal room for improvement in domain generalization. The disparity in
crossdataset performance depending on the training set suggests that the model's generalization
capabilities are influenced by the diversity and characteristics of the training data. Future work
should focus on addressing the domain shift problem, potentially through techniques such as
adversarial domain adaptation or meta-learning approaches [
In summary, the proposed face anti-spoofing method, which combines Vision Transformer, ConvNeXT, and Swin Transformer, demonstrates high results in detecting presentation attacks across diverse scenarios. Experimental evaluations on OULU-NPU and CASIA-MFSD datasets reveal competitive performance, particularly in generalizing to unseen environmental conditions and attack types. Feature space analysis through PCA visualizations shows the importance of fusing global and local feature representations. The clear separation between genuine and spoofed samples in the fused feature space correlates with the strong performance metrics observed on individual datasets. The proposed method offers a balance between capturing fine-grained textures and modeling long-range dependencies, crucial for robust spoofing detection.
However, cross-dataset experiments expose challenges in domain generalization, with varying performance when transferring between OULU-NPU and CASIA-MFSD. Future work should focus on improving cross-dataset generalization through advanced domain adaptation techniques or meta-learning approaches. Additionally, investigating the individual contributions of each architectural component may lead to further optimizations in the fusion strategy. While the proposed method exhibits strong performance within individual datasets, addressing domain shift remains a critical challenge for real-world deployment of face anti-spoofing systems.
During the preparation of this work, the authors used Google Gemini in order to: Grammar and spelling check. After using this tool, the authors reviewed and edited the content as needed and take full responsibility for the publication’s content.