Problem Statement and Relevance. The Global Risk Report 2024 [1] identifies misinformation and disinformation -on which this work places particular emphasis -as the number one global risk (ranked by severity over the short and long term) for the upcoming two years and on the fifth rank for the next ten years. Geopolitical events and developments are interconnected with information disorders (cf. [2]) and have led to societal polarization. Advancements in artificial intelligence (AI) (e.g., deep fakes) have only accelerate this.
Since 2008 online content sharing platforms (OCSPs) have been on the rise (cf. [3]). But this development and the change of information consumption have also caused problems. Types of information disorder (cf. [2]), as shown in Figure 1, which spread rapidly, have caused economic and societal harm. Examples for the latter are the COVID-19 pandemic or infodemic, like the World Health Organizations (WHO) denoted it alternatively (cf. [4]), the European Medicines Agency (EMA) hacking in 2020 and release of manipulated vaccine data (cf. [5]), as well as the 2021 United States Capitol attack (cf. [6]). However, for voters it is key to have access to authentic information or facts within a networked society (cf. [7]).
Proceedings of the Doctoral Consortium at ISWC 2024, co-located with the 23rd International Semantic Web Conference (ISWC 2024) mark.nicolas.gruensteidl@wu.ac.at (M. N. Gruensteidl) 0009-0000-1427-4748 (M. N. Gruensteidl)
Figure 1: Information Disorder Types (cf. [2]).
That way a monitorial citizen (cf. [8]) can come to a rational choice in the sense that an individual follows his or her rational interest when voting (cf. [9]). However, information disorders make this task more difficult.
Policies and legal regulations have addressed different media content (e.g., text, images, etc.), OCSPs (e.g., data access), and flows of information (e.g., disinformation task forces) while at the same time protecting human rights and fundamental rights (cf. [7,10,11]). However, transparency and trust are open issues (cf. [12]), especially when it comes to solution approaches that should fulfill legal requirements as well as societal expectations (cf. [13]). While critical thinking and media literacy are indeed crucial skills, in order to establish societal resilience against information disorders (cf. [14]), information technology can assist democratic actors such as citizens with information disorder detectors (cf. [15]) and provide meaningful and transparent further context (cf. [16]) to voters for example. Additionally, there exists a lack of (training) data (cf. [17]). The consequence of this are possibly biased natural language processing (NLP) machine learning (ML) and deep learning (DL) models (cf. [18]). Lastly, information disorders are complex regarding their phases, specific types, multiple classes, content variability, rapid dissemination, context sensitivity, origins, dynamic nature, and when they are generated by AI (cf. [19]).
Focus and Contributions. The first focus lies on improving information disorder detection. For that, different detection algorithms (e.g., deep learning, BERT; rule or lexicon-based; linear regression, Support Vector Machine; probabilistic, Naïve Bayes; proximity-based, K-nearest neighbors algorithm -see Figure 3) will be used to evaluate and compare their performance for the task of detecting information disorder types on five selected datasets (see Table 1) with respect to mis-, dis-, and malinformation (depicted in Figure 1). A second research aim is the analysis of potential bias within black box models applying explainable artificial intelligence (XAI) tools. Examples for the latter include LIME (Local interpretable model-agnostic explanations) as well as Anchors (cf. [38]). Further, to enrich the contextual understanding of choices a model makes, a knowledge graph (KG) can be applied. But also existing fairness toolkits for bias detection are relevant for the analysis (cf. [20]). Third, the goal is to investigate to what extent it is possible to detect intent to harm. When it comes to transparency, a KG can add useful context to an intent recognition (IR) model. The overarching goal is to increase trust in AI-based detection models as well as help research, societal, and business stakeholders to cope with information disorders better.
Paper Structure. The rest of this paper is organized as follows: in Section 2 related work is presented and discussed. Afterwards, the research objectives, open issues or challenges, main hypothesis, research questions, methodology, research plan, and data sources are given in Sections 3 and 4. Finally, Section 5 concludes and identifies future work.
Already when the COVID-19 pandemic started, AI fact-checking models were available. But training datasets did not specifically cover COVID-19 misinformation which led to domain analysis issues and limited the application scope of these detection models (cf. [21]). Therefore, domain-specific datasets, that are designed to include the unique characteristics of such events, had to be created (cf. [21,22]). Another problem is that false information tends to spread faster compared to facts. Typical NLP classification models for misinformation detection are based on BERT or modified versions of it and misinformation can be analyzed looking into textual elements like sentiment or veracity (cf. [22]). The overall aim is to tackle it by detecting or validating its content and analyze its dynamics or management (cf. [23]).
Regarding the analysis of the distribution of false information and datasets that are used to train detectors, specific topics or demographics matter (cf. [24]).
Others like Mensio and Alani [25] analyzed who interacts with misinformation. Interestingly, readability is better for false information and the more complex it is the more people believe in it. In this regard, characteristics like sentence and word length are relevant (cf. [26]). Bruel and Alani [27] crafted a reporting tool for dynamic spread and fact-checks. Detection of false claims is a multidisciplinary challenge and especially DL as well as XAI have been researched in computer science. It should be taken into account that combating information disorders is a task that can be tackled via human-centric artificial intelligence (AI) approaches good (cf. [28]). When it comes to large language models (LLMs), Leite et. al. [29] emphasized the time-consuming annotation process and role of credibility signals.
Although there have been significant advancements in terms of NLP-based text analysis tools, reputation evaluation, network data analyses, and imaged-based detection techniques, there are a number of open challenges with respect to the concept drift and dynamic streaming nature of (false) information but also deepfakes represent an arising problem. Explainable lifelong learning concepts could be a remedy (cf. [30]).
From a symbolic AI point of view, semantic web (SW) concepts like ontologies (e.g., OWL) (cf. [31]) and KGs have already been used to detect fake news. Tchechmedjiev et. al. [16] have shown an extensible KG consisting of fact-checked claims that serves to tackle the lack of corpora containing structured (meta) data. Pan et. al. [32] also used KGs for content based fake news detection and showed promising results for detecting false content. Moreover, they found that even incomplete or imprecise KGs are helpful.
Kaliyar et. al. [33] have shown in their experiments that BERT-based fake news detection can reach a high accuracy. Combining such DL models with SW technologies can be done in various ways. Denaux et al. [34] introduced a data model and distributed agents' architecture for composable credibility reviews in context of explainable misinformation detection. Furthermore, Lovera et. al. [35] took a different approach and worked on another hybrid solution. They analyzed the sentiment of short texts by using KGs and DL technologies to classify their sentiment. Traceability but also interpretability via XAI led to results that are more transparent. Such an approach compensates for the weaknesses of a black box model in this regard. Their model outperformed character n-gram solutions.
In particular, adapting original data sources with additional information from semantic resources is prominent for contextual enrichment. Some KGs are commonly used in that regard (e.g., ConceptNet or DBpedia) (cf. [18]). KGs can help to identify relevant events for information disorder detection as certain events are often linked to fake news. Opdahl and Tessem [36] looked into ontologies to find journalistic angles. KGs can also support detecting online toxicity by mitigating possible bias, subjective views, or lacking domain knowledge of annotators (cf. [37]). Keeping in mind that black box models should be explainable, Szczepański et. al. [38] proposed an extension which is based on LIME and Anchors that enriches a model's classification with additional reasoning. XAI techniques like this and KGs for facilitating context-based detection of information disorders like Fane-KG (cf. [39]) are on the rise. Fane-KG represents a KG that is designed to be used specifically for context-based fake news detection. Hani et. al. [39] provide a human as well as machine processable data repository built on semantic standards (OWL and RDF).
Generating a KG for news articles, which can be then used for detecting or checking false information, is challenging. Generally speaking, KG construction consists of three phases: the acquisition, refinement, and evolution of knowledge (cf. [40]). Such created KGs can then be used to improve the detection of information disorders. Mayank et. al. [41] worked on a combination of a NLP and tensor decomposition model. They encoded content from news and embedded KG entities. The latter and their variety could improve their detector.
There is a research trend to combine ML and SW technologies like KGs identifiable (cf. [42]).
Lastly, regarding intent recognition (IR), there are already pretrained BERT-based models that achieve promising and precise results (cf. [43]). In addition, domain-specific KGs can improve the intent classification task (cf. [44]). Zhou et. al. [45] assessed the intent of shared fake news and proposed an influence graph as well as modeled the intent of an individual when spreading fake news. Furthermore, a centralized KG with checked articles could help with queries and add contextual knowledge for fact-checking tasks (cf. [46]).
Starting with the challenges, as outlined in Section 2, AI techniques (i) used to identify information disorders are not transparent enough and (ii) lack in cross-domain applicability. In addition, (iii) AI causes issues if producing deep fakes (cf. [13]). Researchers classify fake news mostly based on intent (e.g., to harm) and content (e.g., text or images). Also, (iv) it is a challenge to label fake news since there are many terms (cf. [2]). According to Farhangian et. al. [47], it may be useful for information disorder detection research purposes to look into multiple feature representations, perspectives, dynamic ensemble models, and a crossdataset evaluation (tackling concept drift). DL and ML can identify domain-specific information disorders but there exists a lack of trust into black box models (cf. [13]). Furthermore, (v) missing data for detection models is a major issue (cf. [17]) and (vi) data validity (e.g., incomplete data) or how humans use AI systems are problems that can increase bias in algorithms (cf. [18]). Combining symbolic and sub-symbolic AI has the potential to fill those weaknesses of current AI systems. Common approaches to mitigate bias are altering the data AI systems are trained on, changing how a model learns, or using a holdout set (not part of training) to adapt the outcome. However, there is still not enough research on identifying bias (cf. [18]). Additionally, combining a NLP DL approach with a KG, using structured and unstructured data, can outperform non-hybrid models (cf. [48]). [49,50]).
Regarding XAI, Yuan et. al. [28] concluded that explainable detection of false information from a user perspective is key and highlighted human-machine communication. As a last obstacle for detecting and analyzing information disorders, there remains (vii) the open issue of classifying it better into mis-, dis-, and malinformation. For this task, automatically created domain-specific KGs can assist during the intent classification process (cf. [44]) and add for example useful contextual information (cf. [46]). For instance, provenance or temporal characteristics of information disorders could be considered in this context.
Therefore, the main hypothesis for this research proposal is summarized below: A more transparent information disorder detection model may be made possible by: (i) a domain-specific cross-dataset analysis for information disorder detection using ML and DL; (ii) the enrichment of ML and DL with knowledge graph-based meaning that facilitates explainability and bias detection; and (iii) intent recognition approaches that can distinguish between mis-, dis-, and malinformation.
The following research questions (RQs) are derived from this hypothesis: RQ1) Which machine and deep learning algorithms are most effective when it comes to information disorder detection? RQ2) To what extent can machine and deep learning explainability and bias detection be facilitated via knowledge graph-based enrichment? RQ3) How can hybrid AI-based approaches be used to better distinguish between the three information disorder types (misinformation, disinformation, and malformation)?
This research project is planned according to the design science research methodology (DSRM), which is an information systems specific research method that aims to develop innovative artifacts (cf. [51]) and consists of six phases as shown in Figure 2 (cf. [49,50]).
The problem and motivation is to increase transparency and trust in AI-based detection models in order to help stakeholders to better cope with information disorders. The primary goal is to improve the detection of information disorders, explainability of models, bias detection as well as mitigation, and intent recognition.
In terms of solution objectives, the plan is to combine ML or DL algorithms with XAI, bias detection, and KGs, in order to understand and mitigate potential bias in information disorder detection models better. However, also for intent recognition, which is needed to distinguish the three information disorder types (misinformation, disinformation, and malinformation) effectively, transparency is key. Addressing the lack of training datasets (cf. [28]), one of the envisaged outcomes and innovative artifacts (cf. [51]) is a new dataset incorporating the three information disorder types with binary as well as non-binary labels. Design and Development aims to create one or more prototypes as artifacts that address the issues highlighted under the solution objectives heading (cf. [50]). Figure 3 shows the research plan. Possible artifacts include (i) statistical insights (accuracy, recall, precision, F1 score), (ii) performance results, (iii) models, (iv) datasets, or (v) KGs. In addition, Table 1 shows five selected datasets which cover different types of information disorders. During demonstration, the artifact is used to address the problem or instances derived from a concrete use case scenario. This can be a case study or experimental setting (cf. [50]). For RQ1, this includes the application of detection model prototypes. Regarding RQ2, KGs and statistical insights from XAI are applied to analyze bias. Third, for RQ3, a new dataset containing mis-, dis-, and malinformation as well as a model are part of the demonstration.
In order to evaluate the artifact, we need to analyze how well it provides a solution to the issue(s) it aims to address. Using selected quantitative metrics (e.g., accuracy, precision, recall, F1 score, etc.), the objectives and goals can be compared as well as analyzed (cf. [50]).
Finally, communication ensures that the problem and its relevance, artifacts and their utility, consequent design, and solutions as well as their effectiveness are presented to our stakeholders. The plan is to publish in highly ranked conferences and journals (cf. [50]).
This proposal aims to improve the detection of information disorders, bias detection and mitigation, intent recognition, and stressed the need for more transparency and explainability concerning AI detection models. We started by investigating the status quo, discussed research questions, described how to approach open issues, and showed the research plan as well as methodology guiding it. A limitation of this work is that it does not address multi-modal detection. Future work includes tackling the research questions with these key objectives: (i) a cross-dataset and detection model comparison; (ii) a bias analysis deploying XAI and KGs; and (iii) the separation of information disorders by intent to harm.
This research is conducted under the supervision of Prof. Sabrina Kirrane.