Introduction

Deepfakes and synthetic media are becoming more prevalent and realistic day by day, presenting society with an increasingly urgent challenge, learning to distinguish reality from fiction effectively. These fake content can and are continually being used to spread disinformation, create smear campaigns, and manipulate reality with potentially devastating impacts for anyone who may end up a victim. To contrast this phenomenon, research has been conducted in recent years creating detectors, often based on deep learning techniques, that can classify a piece of content (such as an image) as realistic or fake. Despite many efforts, this discrimination capability is still insufficient today with many open problems in the field of deepfake detection. One example above all is that of generalization [1,2]. In fact, deepfake detectors, although particularly effective in detecting images generated or manipulated by the same methods they are trained on, fail when using different and novel techniques. An-other problem is that of adversarial attacks, strategies of camouflaging traces, enhancing fake content or ad-hoc manipulations designed to fool the detector, which can be used to make detection even more complex. Deepfake detection models must therefore be designed so that they provide a high degree of robustness to possible adversarial attacks and also be able to effectively distinguish deepfakes without raising false alarms. For this reason, AIMH Lab at ISTI CNR has carried out numerous research attempts to explore new innovative techniques to advance this field but also to highlight possible hidden dangers that may damage the efforts made in previous research, representing dangers that detection systems may encounter in the real world. In particular, this paper summarizes the efforts made in [3] and [4]. process, some aspects of the image may change. For example, some previously visible details may become more blurred, totally disappear, or, conversely, be emphasized and brought to light. Deepfake generation algorithms commonly tend to introduce some more or less visible artifacts. In the case of human faces, these artifacts can be, for example, anomalies in pupils, contours of lips, eyes or ears, or accessories. Typically, deepfake detection models learn to recognize the specific anomalies introduced in manipulated images and, because of them, can discriminate between pristine and fake content. In [4], we explored whether Super Resolution techniques can be used as an adversarial attack to camouflage artifacts introduced by deepfake generations approaches.

Model

To do this, we proposed an SR-attack pipeline, whose purpose is to disguise artifacts present in deepfake images while still trying to keep the appearance as unaltered as possible. The pipeline begins with the detection of a face from the deepfake image that is subsequently downscaled by a factor 𝐾 using interpolation techniques. The resulting image is restored to its initial resolution through a super-resolution approach. The resulting face can eventually be reinserted into the original image, resulting in the camouflaged image.

An example of the impact of the proposed SR-attack is shown in Figure 1. As the figure shows, the attack leads to the removal of artifacts introduced by deepfake generation techniques, such as noise around the mouth, and thus makes their detection extremely complex. In fact, to distinguish a counterfeit image from a pristine On the left, an example of manipulated face with a zoom on the artifacts around the mouth. On the right the same face but after the application of the SR-attack. From the zoom on the second one it can be seen how the artifacts are drastically smoothed.

one we often rely on observing these artifacts and oddities that can be introduced by the manipulation process. The fact that super-resolution leads to their removal or attenuation qualifies it as a potentially effective attack against deepfake detectors but also against the human eye itself.

The usage of the SR-attack conduct to a blurring effect on the artifacts introduced in the fake images and this makes them more difficult to detect. This is pretty evident in terms of performance; in fact, the use of the attack drastically degrades the performance of classifiers trained to do deepfake detection.

Table 1 shows the accuracies of a Swin Transformer[6] and a Resnet50 [7] on images manipulated with different techniques, considering them before and after SR-attack. The dataset used is FaceForensics++ [5] and the deepfake generation methods considered are Deepfakes [8], Face2Face [9], FaceSwap [10], FaceShifter [11] and NeuralTextures [12]. This allows us to evaluate the effect of the super-resolution attack on different types of manipulations, thus highlighting on which it is more or less effective. Both the models are trained on the FaceForensics++ training set considering for the construction of the fake class, the same deepfake generation method used for the test.

According to our experiments, for both the considered models, when images are attacked with the proposed approach, there is an increase in False Negative Rate, particularly on some methods namely Face2Face, FaceSwap and FaceShifter. Others, however, are found to be more robust to attack, namely Deepfakes and NeuralTextures on which, however, there is an increase in False Positives. This behavior indicates that in some cases, the use of super-resolution techniques could lead to the elevation of false alarms, leading models to identify legitimately enhanced images through these approaches as deepfakes.

The latter result highlights a problem that could prove crippling to traditional deepfake detectors and could prevent their deployment in the real world. Indeed, it is plausible to think that on social networks it will become increasingly common to improve the quality of one's photos through Super-Resolution techniques. If this were to happen and in parallel the deepfake detectors were unable to understand that these are legitimate images but instead end up mistaking them for deepfakes, the number of false alarms would be such as to prevent their effective deployment on a large scale. It is therefore necessary on the one hand to defend against the malicious use of super-resolution to disguise artifacts introduced by manipulation techniques but also to make detectors robust so that they are able to recognize legitimately augmented images.

In these experiments we used only EDSR [13] as the basis of our attack but the proposed attack can be conducted using different types of SR algorithms (such as BSRGAN [14]), and depending on the peculiarities of each, greater or lesser effectiveness can be achieved on each specific deepfake generation method. The choice of the 𝐾 factor also has an impact in that as it increases, the detectors' errors increase but the quality of the image itself also deteriorates. Therefore, it is crucial to choose the SR algorithm and the value of the 𝐾 factor appropriately in order to achieve maximum effectiveness from the attack.

Future Works

In this section we expose some of the future works we are working on either as extensions of previously presented works or as new applications and solutions for effective deepfake detection.

Robustness of Deepfake Detectors

The fact that Deepfake Detectors are susceptible to the use of SR techniques on images, whether fake or pristine, exposes a serious problem in their use in the real world and therefore requires that more studies be conducted to make them robust to this kind of content. It is necessary to find an effective way to make the models robust to this attack for example by introducing super-resolution as data augmentation during training.

The attack itself can also be further explored and improved by going to identify the optimal 𝐾 value and corresponding SR method as well as experimenting with different strategies for applying the attack, such as focusing on a frame rather than a detected face.

Deepfake Detection without Deepfakes

As stated before, one of the most stringent problems in the field of deepfake detection is that of generalization. Indeed, there is ample evidence that detectors tend to learn to effectively recognize deepfake content obtained through methods used to construct their training set, but fail when they need to classify content obtained through novel techniques. This leads to a total inadequacy of conventional deepfake detectors in being used in the real world. In fact, new deepfake generation techniques are continually being created, and it would be impractical to retrain the model each time to introduce every single possible method. In the context of synthetic images, this tendency of deepfake detectors stems from the fact that each generator introduces a specific fingerprint into the image [15,16,17]. It tends to be invisible to the human eye but involves the presence of structured patterns in the frequency domain (grids, symmetric peaks, halos, etc.).

From the observation of this phenomenon, as a future work we are exploring a new training technique for deepfake detectors that tries to stimulate the model to recognize the presence of structured patterns in the frequency domain and not to learn a specific fingerprint. The preliminary results of this approach can be found in [3].

We propose to reproduce prototype structured patterns inspired by what we observed from the fingerprints actually introduced by generative patterns of various types. These patterns, in frequency, are injected in pristine images and considered as "fake" in the training phase. During the training, we show the model pristine images and others on which a pattern has been applied, indicating them to the model as fakes.

From our preliminary experiments, we demonstrated that models trained on this proto-task, are extremely effective at identifying synthetic images despite never really seeing one in the training phase.

The use of synthetic patterns may also be used to support traditional training of deepfake detectors, introducing them only occasionally and still maintaining deepfakes in the training set. In addition, it will be possible to experiment with a virtually infinite number of patterns by searching for the most effective ones and studying their impact.

Conclusions

Carrying out research in the field of deepfake detection is an increasingly pressing need because of the multitude of techniques that now make it possible to produce synthetic or manipulated content with an increasing degree of credibility. As shown in our recent research, it is critical to explore both innovative methods to try to improve the capability of deepfake detectors looking for new training approaches and techniques. This could be needed to overcome the pressing problem of generalization. On the other hand, it is important to find new solution to face the possible risks and unexpected situations that these models might encounter in the real-world that could invalidate their potential, such as the adversarial attacks.