Twitter, Facebook, and Instagram are just some examples of social media currently used by people to share news with other users worldwide. However, the information widespread through these channels is typically unverified and/or interpreted according to the user's point of view. Accordingly, those means represent the perfect tool to hack user opinions with misleading or false news and make fake news viral. Identifying this malicious information is a crucial but challenging task since fake news can concern diferent topics. Indeed, the detection models learned against a specific domain will exhibit poor performances when tested on a diferent one. In this work, we propose a novel deep learning-based architecture able to mitigate this problem by yielding cross-domain high-level features for addressing this task. Preliminary experimentation conducted on two benchmarks demonstrated the validity of the proposed solution.
Online Web sources and Social media represent the main means for news information dissemination and spreading. In particular, an exponential increase in the use of social media has accelerated information difusion. The speed with which misinformation spreads, alongside social media’s open access content production and dissemination, increases the potential damage, making online platforms major targets for fake news propagation.
Misinformation, in general, often spreads faster and more widely than true news, posing new
risks for democracy and national security, weakening trust in public institutions, and putting at
risk society’s trust in information. As an example, disinformation is polarizing public debate on
topics related to COVID-19; amplifying hate speech; heightening the risk of conflict, violence
and human rights violations; and threatening long-term prospects for advancing democracy,
human rights, and social cohesion. It has been estimated that at least 800 people died and 5800
were admitted to hospital due to false information related to the COVID-19 pandemic, e.g.,
believing alcohol-based cleaning products are a cure for the virus [
While, in the last years, COVID-19 emergency provided a dramatic and pressing example of the paramount importance of increased efectiveness of fake news detection in the health field, recently other crucial issues that require proper communication have started attracting public attention, like the Russia-Ukraine war. Disinformation can be harmful in all these contexts and may lead public opinion to push forward detrimental decisions with huge social and economic costs.
It is, therefore, necessary to limit the impacts of misinformation, as well as develop specific tools and services to allow citizens and the professional community to access reliable and trustworthy information on the Web and Social media.
In this scenario, assessing the veracity and authenticity of news represents a crucial problem that can benefit from recent advances in Artificial Intelligence (AI) and Machine Learning (ML). As this task is time-consuming, expensive, and unfeasible on huge amounts of data produced on the Web, AI-Based tools represent an efective solution to automate the identification of deceptive information by limiting the need for intervention by specialized and trusted professionals.
In particular, the automatic detection of fake news is a relevant problem attracting great
interest from the research community. This problem was traditionally addressed in the literature
as a text classification problem [
However, learning reliable detection models able to identify misinformation requires coping with diferent complex issues. First, an efective solution should allow for handling low-level raw data frequently afected by noise, as the channels used to spread fake news typically allow for sharing only short text (e.g., Twitter). Moreover, the number of labelled training instances is limited; the labelling phase is a dificult and time-consuming task manually performed by domain experts. Finally, fake news can concern diferent topics; therefore, the features leveraged to perform the prediction should be domain independent to handle diferent topics.
Notably, most existing techniques [
Contribution. To address the aforementioned issues, the proposed work introduces an
end-to-end Deep Learning based framework for fake news detection tailored for cross-domain
applications. The adoption of the Deep Learning (DL) paradigm [
artificial neurons are stacked one upon the other), allowing for learning features at diferent abstraction levels to represent raw data.
In more detail, we explore and evaluate diferent deep learning based strategies (mainly Generative Networks) to learn transferable and discriminable feature representations for fake news detection. Moreover, we derive features that are domain-invariant and, thus, benefit the detection of fake news on newly arrived, emergent events for which only a few verified news are available.
The proposed solution is composed of three main components (i.e., neural models) that collaborate to solve two tasks simultaneously: the main one is to recognize fake information, and the second (auxiliary) task aims to produce domain invariant features. Preliminary experimentation conducted on two real datasets shows promising results and encourages further studies. Organization of the paper. The remainder of the paper is structured as follows. In Section 2, we survey recent works concerning the cross-domain fake news detection problem; Section 3 introduces our approach and details the devised neural architecture, and Section 4 showcases numerical results. Finally, Section 5 concludes the paper and outlines possible future research directions.
Numerous studies on automating fake news detection have been proposed in the recent few years. Most studies explore diferent supervised models with diferent modalities (e.g., text, images, and propagation networks) of news records to identify fake news. However, the performances of these existing state-of-the-art detection techniques significantly collapse if the news are coming from diferent domains (e.g., politics, gossip, medicine). In fact, while they perform well in the domain they were trained on (e.g., politics), they perform poorly in other domains (e.g., healthcare), especially for domains that are unseen or rarely seen during training.
News from diferent domains have significantly diferent word usage, specific communities of users involved in the news engagements, and also diferent propagation patterns. Furthermore, the models are biased toward event-specific features. Therefore, to address these challenges, it is key to learn models able to catch cross-domain information.
Cross-domain modeling refers to a model capable of learning from data in a certain source domain and being able to adapt and have a good performance on a diferent target one. A few previous works have attempted to perform fake news detection using cross-domain datasets. In this section, we overviewed some of the most significant approaches proposed in the literature.
In Wang et al. [
In [
In [
Silva et al. [
In [
In [
The majority of the approaches for cross-domain fake news detection mentioned above have two drawbacks: (1) it assumes that the news records from diferent domains arrive sequentially, even though this isn’t always true for real-world streams; and (2) it necessitates knowing the domain of news records, which isn’t always possible; (3) it is not able to capture newly emerging domains and handling temporal changes in domains. In contrast, our method learns cross-domain knowledge of news without being aware of the news’ real domain.
In this section, we provide a detailed description of our approach based on DL to learn a detection
model able to identify malicious/misleading information across diferent domains. The problem
is particularly relevant in situations where fake news emerge in new contexts for which no
prior evidence is available. Most of the existing approaches are designed for specific events
and as a result, inefective for emerging ones [
Our proposal relies on a modular architecture based on feature extraction and embedding, domain generalization, and classification. Essentially, the general framework consists of three main components: feature extractor, fake news detector and domain discriminator.
The feature extractor maps an input into a multidimensional representation , which is a compressed mapping of diferent types of features (such as news content, images, user comments, propagation patterns, and user-news interactions). In our scheme, it is made of two components: the first component is an embedder used to encode the diferent modalities (e.g., Bidirectional Encoder Representations from Transformers (BERT), Graph Neural Networks (GNN), Video Graphics Array (VGA), etc.) for learning representations; while the second component combines all such representations.
In the current implementation, the embedder takes the form of a pre-trained BERT [
NSP MaskLM MaskLM C T1 TBNERTT1 T1’ TN’ E[CLS] E1 … EN E[SEP] E1’ … EM’ CLS Tok1 TokN SEP Tok1 TokM MaskedSentenceA MaskedSentenceB
UnlabeledsentenceAandBpair Pretrained Embedder
(feature extractor) NSP MaskLM MaskLM C T1 TBNERTT1 T1’ TN’ E[CLS] E1 … EN E[SEP] E1’ … EM’ CLS Tok1 TokN SEP Tok1 TokM MaskedSentenceA MaskedSentenceB
UnlabeledsentenceAandBpair Pretrained Embedder (feature extractor)
E m b e d d i n g E m b e d d i n g ex ex Good news,Wuhan’s corona virus can be curedbyone bowlof freshlyboiledgarlic water
Good news,Wuhan’s corona virus can be curedbyone bowlof freshlyboiledgarlic water
News ^ y ^ z z ^ y ^ z z
(fake news detector)
G(ex) composing a sentence is masked, and the model is trained to predict the missing terms by considering the word context, i.e., the terms that precede and follow the masked one. Then, the model is fine-tuned by considering a further task that allows for understanding the relations among the sentences. In our framework, we adopt a BERT instance pre-trained on Wikipedia pages. The fake news detector includes two modules integrated into a single architecture. The first module is a generalization network that maps the combined embedding into a generalized embedding (). The second component is a binary classifier that takes the generalized representation () as input and provides a classification response ˆ. The two networks work in a combined way: the purpose of the generalization network is to remove all domain-dependent features from , in a way that allows the classification module to focus only on domain-invariant features.
The domain discriminator takes the form of the support network , learned on an auxiliary
task to characterize the source domain of the input . The domain discriminator is exploited
in an adversarial framework [
Specifically, we investigate two diferent architectural choices for that enable such mechanisms. The main diference relies on how the self-supervision is combined with adversarial learning: in the first case, a classifier is adopted to model the discriminator while, in the second one, an autoencoder solution is used. Both these architectures are sketched in fig. 1. The first choice (devised in fig. 1a) exploits a pre-trained domain classifier. Here, we assume that given an input ˜, the response (˜) is a label characterizing the source domain. This requires that is trained in order to be able to discriminate on the underlying topic characterizing ˜. In our framework, ˜ is either or (): in the former case, (˜) = and represents the actual source domain; in the latter, (˜) = ˆ and is the predicted one. Then, given , the whole network can be trained using the following loss: ℓCL(, ) =BCE (ˆ, ) + ℓ(, ˆ)
Here, BCE represents the binary cross-entropy loss. The second term in the loss aims at penalizing the overlap between the distributions devised on both and (), through the classifier. This can be achieved by exploiting a combination between Kullback-Leibler divergence and entropy on and ˆ. In the following, we assume that is a binary classifier and implement ℓ to penalize the certainty in classification on (), as opposed to that on : ℓ(, ˆ) = log(|.5 − ˆ|) + log( + ) + log(1 − + ).
The term in the formula bounds the contribution of and avoids the discriminating capabilities on the original input embedding to make the contribution of the generalization network negligible. As we are considering a binary classifier, we use a constant value of 0.5 that represents the max uncertainty for the classification. In practice, the objective is to obtain a generalization that introduces maximum uncertainty in domain discrimination, as opposed to minimum uncertainty in the original embedding.
A major disadvantage of the above learning scheme is that the source domain’s information must be known in advance (as a consequence, the domain discriminator has to be pre-trained). In other words, the dataset has to be labeled with the domain from which the information was extracted. Although this is not necessarily an issue, an unsupervised approach that does not require prior knowledge of the source domain could be better suited. We explore this direction in the alternative architecture (shown in fig. 1b) that combines autoencoders with adversarial approaches for learning unsupervised feature representations about the domain and using them for domain adaptation. The main goal is to leverage more general latent feature representations characterizing a domain using an autoencoder that is trained by minimizing the reconstruction error within the source domain (without the need for the dataset to be labeled). The resulting latent feature is hence exploited in a minimax game between the domain discriminator and the fake news detector.
Formally, given input and a label , we tune the components of the network to map both
and () in the domain [
In practice, learns to generalize the features of by disregarding the components that characterize the source domain. The latter can be trained in an alternate fashion within the same learning framework, without the need to resort to supervised prior information about the domain. This guarantees full supervision and, at the same time, forces the detector to only focus on domain-invariant features. As regard the constant value 2, it represents the max diference between the outputs ( and ()) since we are adopting a tanh activation function.
This section describes preliminary experimentation conducted to test our solution. We introduce
datasets, model parameters, and evaluation protocol, then we describe the adopted evaluation
metrics, and finally, we discuss numerical results and show a qualitative analysis.
Datasets, parameters, and evaluation protocol. In our experimentation, we evaluated
the quality of our solution by considering two real datasets extracted from the FakeNewsNet
data repository [
In Table 1, we report some relevant information concerning the two datasets: the overall number of articles, the vocabulary size, and some statistics on the number of words per article (i.e., average, median, first, and third quartile).
The neural architectures used in our experimentation mainly difer in the number of layers and neurons. Considering the solution depicted in Figure 1a, the Generalization Network is an MLP composed of three fully-connected layers (respectively, instantiated with 768, 256, and 768 neurons), whereas both the classifiers and are composed of two fully-connected
layers (respectively, with 64 and 32 neurons) and an output layer equipped with a sigmoid activation function. As regards the solution shown in Figure 1b, the model takes the form of a Variational Autoencoder (VAE). The encoder is composed of three layers, respectively, with sizes 768, 256, and 64. Symmetrically, the decoder includes the same number of layers and neurons. Also in this case, the classifier includes two hidden layers but respectively, with 768 and 256 neurons and an output layer equipped with a sigmoid activation function. Finally, the Generalization Network is an MLP composed of three layers (instantiated, respectively, with size 768, 256, and 768). Unless otherwise specified, each layer is equipped with a tanh activation function.
The predictive performances exhibited by our solution are compared with a simple baseline neural model, initialized with the same architecture and parameters of the fake news detector component, and fed with the same embedding. In more detail, to evaluate the cross-domain generalization capability of our approach, we adopted the following protocol: each model is trained against a single domain training set and tested against the data from another one.
Lastly, the experiments were executed on an NVidia DGX Station equipped with 4 GPU V100 32GB. The model was learned by optimizing the weights in batches of 32 texts from the training set using the Adam optimizer with a learning rate = 0.001.
Evaluation Metrics. To assess the detection capabilities of the proposed approach, we computed diferent performance metrics for both test cases. First, let us define as the number of positive cases correctly classified, as the number of negative cases incorrectly classified as positive, as the number of positive cases incorrectly classified as negative, and as the number of negative cases correctly classified. Then, the following metrics have been considered: • Accuracy: defined as the fraction of cases correctly classified, i.e., +++ + . • Precision and Recall: used to estimate a system’s detection capability in identifying attacks and avoiding false alarms. Specifically, Precision = + , while Recall = + . • F-Measure: summarizes the overall system performances and is calculated as the harmonic mean of Precision and Recall.
Notably, F-Measure, Precision and Recall are computed by using a Macro-Averaging strategy i.e., the metrics are computed for each class and then averaged. We adopted macro-averaging, which weights all the classes equally, to summarize with a single value the performances of both classes.
Numerical Results. In Table 2, we report the experimental results obtained by comparing our proposed framework with the baseline introduced above. The table shows the results obtained by using the evaluation protocol introduced in the previous paragraph, which consists in learning the model on a specific domain (Training Set) and validating it on another one (Test Set). Our Solution (AE) refers to the proposed architecture adopting an autoencoder as discriminator, while Our Solution (CL) indicates the setting in which the discriminator is a classifier.
Bold values represent the best results obtained on the test set. A cross-validation strategy [21] is used to select the best model in the learning stage.
Basically, the adoption of our solution allows for improving the predictive capability of the fake detector for each configuration and evaluation metric. These preliminary results highlight that the proposed framework permits the extraction of cross-domain features, which improves the generalization capabilities of the detector.
Although the preliminary results are promising, it is worth pointing out that the study has been carried out by exploiting only the textual content of social media posts. The obtained results are in line with the ones achieved by state-of-the-art models that rely only on the textual content of news and mainly address single-domain scenarios, while the proposed solutions generalize across domains.
However, the predictive performances of the detector could further benefit from using social media-rich data, and exploiting not only textual content but also temporal, geographical, and network information, enabling the development of novel fake news detection solutions by integrating data of diferent natures.
In this work, we designed a framework for detecting fake news using the deep learning paradigm that efectively addresses a major issue in this field i.e., recognizing misinformation across diferent domains by exploiting one learning model able to generalize through the domains. The framework combines in a single architecture diferent neural components and tries to solve simultaneously two tasks: (i) extracting cross-domain features, and (ii) learning a detector able to distinguish real/fake news. Specifically, a BERT model pre-trained on Wikipedia data is used to generate an embedding from the news text, then this compressed representation feeds two other neural networks: the former performs the detection while the latter solves an auxiliary task so as to yield cross-domain invariant features.
Future works aim at refining the proposed framework. Part of our ongoing research is devoted to evaluating the proposed architecture on heterogeneous data. Multi-modality is a key approach to improving fake news detection. It is more dificult and more challenging to detect fake news on multi-modal inputs, as it requires not only the evaluation of each modality but also cross-modal connections and the efective combinations of the diferent inputs. This becomes even more challenging when each modality e.g., text or image is credible but the combination creates misinformative content.
We aim to exploit multi-modal data like images, video, news propagation patterns, social context representing the user engagements of news on social media (e.g. the number of followers, hashtags, friendship networks, retweets).
In fact, information such as “from who” and “how many times” the news has been replied, quoted or shared could represent further precious indicators to reveal the malicious nature of the news. In this respect, Graph Neural Networks (GNNs) have proven to be an efective tool to operate on the graph domain and summarize the social network properties into a single graph embedding.
Another method of strengthening multi-modal models for fake news detection is to augment them with features based on the behavior of news producers. News producers copy news stories from each other for easy engagement or to increase the perceived credibility of stories. The idea of exploiting features of media sources has been only partially explored in literature. We plan to address this gap by investigating alternative solutions like introducing a set of veracity features to characterize the sources based on content sharing behavior.
This work has been partially supported by project SERICS (PE00000014) under the NRRP MUR program funded by the EU - NGEU. [20] K. Shu, A. Sliva, S. Wang, J. Tang, H. Liu, Fake news detection on social media: A data mining perspective, ACM SIGKDD Explorations Newsletter 19 (2017) 22–36. [21] C. Schafer, Selecting a classification method by cross-validation, Machine Learning 13 (1993) 135–143.