The continuing spread of disinformation on the Web has caused a great demand for automatic factchecking tools to help institutions and practitioners check the truthiness of web content. Existing literature presents numerous solutions for specific tasks, such as textual fact-checking or image/video truthiness, that, together with the availability of Open Source Intelligence (OSINT) tools and principles, paves the way for new comprehensive solutions. This work introduces a Knowledge Graph-based approach for fact-checking and news debunking. The idea is to map fact-checking workflow activities leveraging OSINT to specific scenarios emerging from Web and social media monitoring. The reference Knowledge Graph is constructed by analyzing sources through Text Mining and Semantic Analysis techniques. Finally, a real case study is carried out to show the applicability of the approach for factchecking purposes.
Fact-checking techniques may be employed to prevent disinformation proliferation and
consequential anxiety among people. In detail, these activities are thought to clarify the
primarily false information presented and thus force the recipient to think more deeply about
the published facts [
Notwithstanding this digital support, proving the truth of content may be a very stressful task to fulfill due to the efort spent by humans in checking multiple features related to the news and choosing which aspect to focus on first or which data to search for to spot distinctive elements for the analysis. Therefore, questions arise: What are the elements (e.g., provenance, source, content, etc.) to be checked first? What kind of data should fact-checkers consider and retrieve to prove fact truthiness? Could the activities be organized in a unique method to perform reliable fact-checking?
This study tries to answer these questions by describing and applying a cognitive approach
depicting the various stages to consider, data to acquire, and diferent OSINT technologies to
perform fact-checking based on previous knowledge about similar facts or events. The approach
leverages a Knowledge Graph [
• The approach’s applicability has been proven through a real example of fact-checking. The rest of the paper is organized as follows: Section 2 provides related work, Section 3 introduces the proposed cognitive approach for fact-checking, and Section 4 demonstrates its potential in a real case study. Conclusions close the paper.
Many approaches explored fact-checking techniques to discover diferent kinds of disinformation
forms. Some works, in particular, focussed on debunking rumors on social networks to analyze
behaviors [
Other types of research focus on designing automated or semi-automated tools for
factchecking, such as [
Some other works employed knowledge to achieve more robust fact-checking, such as [
Contrary to the existing literature, this work proposes a cognitive approach whose Knowledge Graph (KG) suggests suitable actions (among ones in the fact-checking workflow) based on the experience and, at the same time, harvests experts’ feedback to feed the KG. Moreover, the proposed approach is applied to an existing case study to check its suitability in debunking and fact-checking.
This paper proposes a methodology consisting of constructing and maintaining a Knowledge Graph (KG) to suggest suitable fact-checking activities among the ones described by the Factchecking workflow. The KG is fed with data taken from the Web and social media (see Figure 1), as well as experts’ feedback.
The following subsections detail the KG and the workflow used in the proposed methodology.
As already stated, the proposed methodology is based on a Knowledge Graph (KG) supplied by suggestions, doubts, or skepticism noticed on the Web or social media. The idea consists of identifying sources of interest (e.g., Facebook pages, Twitter accounts, etc.) and constantly monitoring their evolution, mainly about specific topics. Relevant recommendations can be extracted by scraping such pages and applying Natural Language Processing (NLP) techniques combined with a semantical analysis. Recommendations conveniently processed and conceptualized populate the KG and give directions for subsequent fact-checking activities. Moreover, the experts’ choices and decisions further contribute to KG’s growth and update. For example, suppose the expert realizes a correlation between a news aspect and a workflow phase: such intuition feeds the graph.
The graph construction leverages NLP techniques and the Debunking Model for the semantical analysis. Debunking Model consists of an ontological model relating the main aspects involved in the workflow to interface and analyze the data collected about the article or post (i.e., container, content, user profiles, etc.). The ontology is built over state-of-the-art ontologies (i.e., OWL 1, OWL time2, etc.) to model knowledge about the people writing and sharing web contents, temporal and spatial information related to the contents and people’s activities, respectively. New classes and properties added to existing ontologies to represent the main aspects involved in fact-checking and debunking activities are depicted in Figure 2 and summarized as follows: • Class Container and its subclass Website represent the web space where contents are shared. • Class Publication represents the posted content that may be an article on a news site (class Article) or a post on a social network (SocialNetwork class). Elements of a post or an article, such as text, photos, links and videos, are represented as instances of Piece of Content. • Class Account represents people that wrote and shared specific content (e.g., an article, a post) on a Website.
Moreover, Figure 2 also demonstrates a simplified example of ontology instantiation based on the case study described in Section 4.
Fact-checking experts, mainly from important news agencies, shared numerous hints aimed
at evaluating content trustworthiness. In particular, the suggested workflow by Urbani [
• The motivation aims to consider what caused the capturing of the piece of content.
Each pillar contributes to a better understanding of the content and its reliability. The following subsections give details about each pillar.
Finding the provenance of content means examining its original form to more easily understand who posted it, when, where, and why. Techniques to discover the original content depend on the type of content. For example, the Reverse Image Search, consisting of searching for the content in large databases (e.g., Google Images), can be a solution for images. In the case of videos, a frame from a video can be subjected to a reverse image search through the Reverse Video Search. In some situations, finding the original content through other strategies 1http://owl.man.ac.uk/2006/07/sssw/people.owl 2https://www.w3.org/TR/owl-time/ is dificult; searching for it in more private and anonymous locations could be helpful. Some examples are Reddit3, 4chan4, Discord5, and, where appropriate, Twitter and Facebook.
The source of content (i.e., owner) can be a valid indication of its reliability. However, since everyone can re-post a piece of content written or captured by someone else on the Internet, it is mandatory to identify its real ”owner”.
Once the first uploader is found, we should understand if the content is coherent with the authors’ geographic position, other shared contents, and so on. In particular, it could be interesting to investigate authors’ social accounts, make reverse image searches of account images, search for shared posts in Google to understand if there is embedded content, and so forth. In addition, checking if declared email addresses are associated with any user (for example, through Skype) could help determine the source’s credibility and, for example, make sure it is not an automated account (i.e., a bot). In this regard, specific techniques can be used (e.g., learning models), or attention can be paid to the volume of daily posts and whether a period of silence is associated with nighttime rest.
Finding the date means determining when the original content was created. A starting point is referring to timestamps associated with a post or file metadata, for example, in the case of image files, the Exif (Exchangeable image file format). However, since these types of metadata are not always available, some further checks could include observing the video/image to understand the period of the year. In this sense, handy tools are: • SunCalc6. It allows viewing the sun’s angle on a specific day at a particular location, which can help identify the time associated with an event in a photo or video. • Wolfram Alpha7. It is a computational knowledge engine that, among other things, enables you to look up the weather for a specific date. This way, a check can be done between the declared date and the weather for that date.
Similar problems with date reconstruction may arise with location identification since geotags are not always available and may not accurately reflect the location provided in the content. Finding specific details in the image or video and doing research using satellite imagery may be helpful for this aim. Examples include looking for squares, signs, flags, banners, etc., to attempt and associate a location with the picture or video. Also, spoken language and clothing could be helpful. However, special attention should be paid to the level of update of images and the latest events in the area by considering the most recent local events (e.g., war or extremely severe weather) that may significantly impact the terrain.
About motivation, it is dificult to identify common hints: the process strictly depends on the considered piece of content. However, in general terms, it could be helpful to find involved people’s afiliations or communities and, where possible, try to speak directly with them. Motivation could be better depicted by extracting the context of the news facts by analyzing comments to the post/news, tweets and other web resources referencing the article considered. 3https://camas.github.io/reddit-search/ 4https://4chansearch.com/ 5https://discord.com/ 6https://www.suncalc.org/ 7https://www.wolframalpha.com/
The KG-based approach described in Section 3 has been demonstrated in a case study based
on an experiment by the German artist Simon Weckert, who walked in empty streets of Berlin
bringing a little wagon containing 99 smartphones, each running a GPS Maps service. The artist
aimed to simulate a fake trafic congestion event by exploiting the high number of devices, their
closeness and the slow movement of the wagon. The experiment was introduced by the artist
in an article called “Google Maps Hack” on his personal blog8 and reported in the book “How
Algorithms Create and Prevent Fake News” [
The objective of the case study is to question some technical aspects of the experiment that the author does not fully describe. Following the approach proposed in Section 3, we tried to extract useful information to understand the feasibility of the experiment. In particular, the case study starts with identifying relevant suggestions about the experiment itself on the Web to realize what pillars of the workflow are bringing up. Moreover, each pillar investigation contributes to populating the Debunking Model, as depicted in Figure 2. Finally, the KG turns on incoherence among findings.
The following subsections detail each pillar analyzed.
News about the experiment appeared on many news sites and blogs; however, we focused on original pictures and information posted on Weckert’s site.
The source of information is the artist himself, who posted information about the experiment on his personal site. However, through cross searches, we also identify his Twitter account9, from which we also find the date of publication of the experiment itself. The tweet posting the experiment had a lot of resonance. In particular, other Twitter users’ comments questioned technical modalities of the experiment. For example, they expressed doubts about adopted devices, Internet connections and attainability (see Figure 3); Weckert did not clarify all factors. Following, this aspect is further examined.
Since the pictures depicting adopted smartphones (see Figure 4) are insuficient to discover device details, we explored the Web to find other helpful information. In a video posted by Arte TV10, we catch the image in Figure 5 from which smartphones are better visible. The first remark regards the diference between smartphone screens in Figure 4 and Figure 5: in this latter, despite the lack of light concerning the experiment execution, screens are darker and less visible.
In order to acquire more information about devices, we searched for similar images and obtained the smartphone model through Google Lens. It is a Huawei Mate 20 Pro, a particularly expensive model released in 2018. Therefore, the hypothesis that Weckert bought all these 8https://www.simonweckert.com/googlemapshacks.html 9https://twitter.com/simon_deliver 10https://www.arte.tv/de/videos/102583-000-A/simon-weckert-arte-tracks/ expensive devices is hard to believe. Instead, many suppose he rented or purchased second-hand devices; anyway, he does not provide further details about them.
Another crucial element to prove experiment truthiness is geo-localization. Since the smartphones are all grouped in the wagon, they have very close GPS coordinates or even the same, which is not the case of a trafic congestion scenario but more similar to an accident with 99 vehicles involved. Therefore, even though fascinating and unreal, the 99-car accident thesis must be rejected since the cart moves and crashed cars do not move. 4.3. Date From Weckert’s website, it is unclear when the experiment was done. Unique available information is the date of the news posting on Twitter (i.e., 02/01/2020). Thus, through the FotoForensics11 tool, we extracted image metadata that, for pictures in Figure 6 (i.e., not subsequently modified), reports October 06, 2019, as the creation date. For images in Figure 7, only the modification date was available, i.e., October 14, 2019.
By a search on Wolphram Alpha, at the given date, the weather is congruent with the conditions in the photos. Moreover, the sun’s direction, evaluated through sunCalc, is congruent with the lights in the pictures.
Regarding location identification, we know a priori that Weckert conducts the experiment in Berlin, Germany. However, in the experiment description, the exact covered distance and the main streets visited are reported as unclear. So, to further analyze the feasibility of the experiment, we realized a focused analysis to reconstruct its path, as described below.
By leveraging picture metadata, we know the chronological sequence of pictures, as depicted in Figure 6. Moreover, from the images in Figure 7, we can extract traversed streets (that, through searches on Google Street View, also match with associated images): • An der Schillingbrücke • Michaelbrücke • Ziegelstraße and Ebertbrücke. • Mittelstraße • Geschwister-Scholl-Straße.
Discovering locations reported in Figure 6’s pictures needs an in-depth analysis made through the synergy between Google Lens and Google Street View. In particular, we can recognize, in chronological order:
The third location in Figure 6 is impossible to identify since there is no recognizable place.
The longest path among cited places is between three and four kilometers (depending on the type of selected path). In this sense, let us notice that on maps shown on Weckert’s site, with trafic mode on, the path is shown as full green lines along the roadside, as reported in the leftmost image in Figure 8. Since this configuration is present in maps showing paths by feet, we assume that Google Maps has detected the smartphones as people walking on the road; otherwise, the result would have been as the one shown in in the right image in Figure 8 (i.e., the smartphones would have been taken as 99 vehicles). Although the possibility of managing fake accounts, according to Google Maps, which collects data during the trip (i.e., the type of device employed and the mean speed), the service would not have mistakenly taken the 99 smartphones as 99 vehicles. Moreover, the hypothesis that the service would have considered the 99 devices as people seems not credible either.
Simon Weckert, through his site, declares his fascination for the digital world and its reflection on social aspects. He aims to evaluate the worth of technology from the perspective of future generations. The artist’s philosophy aligns with the nature of the experiment.
The paper presents a cognitive approach for fact-checking web content based on a Knowledge Graph leveraging OSINT tools and principles and a domain ontology. In particular, the method considers state-of-the-art tips and associated instruments for acquiring and fact-checking information. Acquired data is conveniently annotated and analyzed at each workflow stage to determine inconsistencies between diferent aspects (i.e., website, account profiles, articles and pieces of content), allowing smart fact-checking activities. The proposed approach was applied to a real case study concerning an experiment performed by artist Simon Weckert consisting in tricking Google Maps service with simulated trafic congestion. The case study aims to demonstrate the practical potential of the proposed methodology to support practitioners and fact-checkers in ascertaining the truthiness of web content through a multi-aspect fact analysis.
In the future, it would be interesting to automatize the whole workflow supporting the technical tasks (e.g., KG implementation and querying), as well as content identification, extraction and annotation.
This work was partially supported by project SERICS (PE00000014) under the MUR National Recovery and Resilience Plan funded by the European Union - NextGenerationEU.