=Paper= {{Paper |id=Vol-3825/short6-3 |storemode=property |title=Rogue Algorithms: Using AI to Track the Spread of Disinformation Applications |pdfUrl=https://ceur-ws.org/Vol-3825/short6-3.pdf |volume=Vol-3825 |authors=Jimmy Mulder,Librecht Kuijvenhoven,Stan Meyberg,Stefan Leijnen |dblpUrl=https://dblp.org/rec/conf/hhai/MulderKML24 }} ==Rogue Algorithms: Using AI to Track the Spread of Disinformation Applications == https://ceur-ws.org/Vol-3825/short6-3.pdf

Rogue Algorithms: Using AI to Track the Spread of
Disinformation
Jimmy Mulder1,*, †, Librecht Kuijvenhoven1, †, Stan Meyberg1, Stefan Leijnen1.

1Utrecht University of Applied Sciences, Heidelberglaan 15, 3584 CS, Utrecht, The Netherlands 1

ORCiD ID: Jimmy Mulder http://orcid/org/0000-0001-9681-863X
Stefan Leijnen https://orcid.org/0000-0002-4411-649X

Abstract
Disinformation has become a growing problem in the digital age, and the rise of generative AI will
likely only increase its ubiquitousness. Human fact-checkers are able to qualitatively debunk a
tiny fraction of fake news, but they cannot keep up with the vast amounts of disinformation that
is unleashed every day. There is a demand for automated tools to aid the process of identifying
(potential) disinformation. In this paper we suggest a new quantitative approach, using a
recursive algorithm based on Large Language Models to provide insight into the spread of
disinformation articles. Our program identified the original source for 200.000 articles spread
across more than 7000 websites. This information can be used to assess the trustworthiness of
websites that host news articles.

Keywords
Disinformation, Misinformation, Algorithms, LLM, Explainability

1. Introduction
In their Global Risks Report 2024, the World Economic Forum identified misinformation
and disinformation “as the most severe global risk anticipated over the next two years”,
citing the disruption of electoral processes, growing distrust, and increasingly polarized
views [1]. While the authors warn for the risks of inaction, they also note that “There is a
risk of repression and erosion of rights as authorities seek to crack down on the
proliferation of false information”, highlighting the need for an efficient remedy which
preserves the rights of individuals.
A popular method to curb the effects of disinformation is the employment of so called
‘fact checkers’ [2], who take an article or belief and deconstruct its arguments one by one,
using reputable sources and expert opinions. This is a costly process in terms of manual
labor, and with hundreds of new disinformation articles being published daily (see results)
it is not feasible for fact checkers to keep up. Furthermore, fact checking articles only reach
consumers who visit reliable websites in the first place, reducing their effectiveness [3].
Much is unknown about the role of malicious algorithms in the spread of disinformation.
Researchers have found a significant influence of ‘bots’ on social media [4]; does this apply

1∗ Corresponding author.
†
These authors contributed equally.
jimmy.mulder@hu.nl (J. Mulder); stefan.leijnen@hu.nl (S. Leijnen)
0000-0001-9681-863X (J. Mulder); 0000-0002-4411-649X (S. Leijnen)
© 2024 Copyright for this paper by its authors. Use permitted under Creative Commons License Attribution 4.0 International (CC BY 4.0).

CEUR
ceur-ws.org
Workshop ISSN 1613-0073
Proceedings
to news websites as well? Are malicious programmers using algorithms to autonomously
find and re-post disinformation? Anecdotal evidence such as ‘authors’ publishing more than
a thousand articles per year suggests that these practices occur, but the scale is unknown.
The European AI act [5] requires low-risk applications to be transparent about the
algorithms they employ, and prohibits algorithms to pose as persons; identifying
malpractices can provide an opportunity to legally curb the spread of disinformation while
preserving the right to free speech.
To handle the large volume of disinformation, software solutions capable of
autonomously detecting disinformation are increasingly being researched and deployed
[6]. These algorithms can be divided roughly into two strategies: one focusses on the
message, determining the truth value of an article by looking at its content; the other
focusses on the messenger, by estimating the trustworthiness of the source.
While the first strategy seems fairer for not dismissing an article based on its publisher
alone, it is much harder to guarantee fairness this way. This strategy requires using a Large
Language Model to recognize patterns in disinformation articles, but these LLMs may have
unknown biases [7]. Additionally, data scientists may add their personal biases to a model
when selecting datasets to train a disinformation classifier [8]. After being trained, such a
classifier will almost certainly be a black box, which makes it very difficult explain why a
certain article was deemed disinformation or not. Also, if such a classifier is made public, a
malicious disinformation creator may use it to train a model to fool the classifier, resulting
in a GAN-like competition. And finally, even if ‘debunking’ can be automated, its
effectiveness is still debated [9]. We believe that a responsible AI solution should be
transparent, explainable and free of bias, which cannot be guaranteed with this method.
While the second strategy is less precise in its assessment of any single article, the
algorithms used for this type of program allow for greater transparency and explainability,
which in turn lowers the risk of (unknown) biases and increases objectivity [10]. As we will
demonstrate, a user of our program is able to inspect how news websites interact with each
other to gain a qualitative insight into their trustworthiness. This offers a more responsible
tool to journalists, scientists and internet users who wish to identify potential
misinformation.

2. Method
Our application creates a network graph based on the number of shared articles between
websites. The application consists of three algorithms: one to determine the level of
similarity between two articles, one to scrape websites for articles, and one to find possible
duplicates of articles on other websites. These work together to form our ‘backend’ which
produces data on the similarity and source of online articles; all data analysis (such as
mapping the data into a visual graph) is done post-hoc in a different environment.
At the core of our application lies a Large Language Model (LLM), in our case based on
the multilingual LLM MUSE by Meta [11], which can take two sentences as input and output
their semantic similarity as a value between 0 and 1. In order to calculate the semantic
similarity between two articles, we cross-compare every sentence in article A with every
sentence in article B, and pair those sentences with the highest similarity score. We then
calculate the average similarity across all sentence pairs. The result is a similarity score
between 0 and 1 for the entire article. Qualitative validation revealed that articles which are
word-for-word the same receive a similarity score of 0.9-1.0, as expected. Articles that are
very different score between 0 and 0.5.
The scraper algorithm produces a list of all the articles that a website has published. In
most cases this list can be produced by looking at the sitemap of a website. In cases where
no sitemap is found, a crawler is employed to crawl that specific website and gather as many
links to articles as possible.
Our goal is to provide insight into the spread of disinformation and determine whether
a website mostly produces original content or mostly reproduces content from other
sources. Our third algorithm works by doing this analysis for one starting point (i.e. one
website chosen by the authors) and then recursively adding more websites to the list.
Each recursive ‘round’ does the following:

• For every website on the list that has not been analyzed yet, do the following:
o Use the scraper algorithm to find all articles that were published in the last
two years.
o For each of these articles:
▪ Use a search engine (we used Bing Search) to find five potential
duplicates of this article, based on the title.
▪ Calculate the similarity score between the article and each potential
duplicate, and use this score to determine whether these potential
duplicates are really (semantically) close to identical. The threshold
for two articles to be considered a match was set to a similarity score
of 0.7, based on the distribution of similarity scores, as shown in
figure 1.
In case two articles match, the website that hosted the newly discovered duplicate is
added to the list. In future rounds, this website will then be analyzed in the same way.
Because the algorithm is recursive, it can theoretically run forever. Due to financial
constraints we bounded the runtime of our algorithm to analyze 200.000 articles spread
over more than 7000 websites. At this point there were roughly 70.000 websites and 7.4
million articles left on our ‘to-do’ list.
An interactive network graph was created from the database, showing the connections
between all websites that have at least one article (with sufficient similarity) in common.

3. Results
Figure 2 shows a screenshot of our interactive visual graph which can be viewed at our
website: https://uashogeschoolutrecht.github.io/RogueAlgorithmsVisialisation/. Here
users can view each website as a node, with all the relevant articles attached. Users can also

Figure 1. A histogram of the similarity score distributions. Most potential duplicates
found using bing search are either very similar or very different to the original, with only a
small percentage being in the 0.6-0.8 range. Based on this, we set the threshold for when
we consider an article a (partial) duplicate at 0.7. Articles with a similarity score <0.3 were
consider obviously original and were excluded from the database in order to save
computing power.

Figure 2. A screenshot of our interactive tool. Blue nodes host mostly original articles, red
nodes host mostly duplicates. Nodes and links are clickable, which provides a list of
articles.

click on a connection between two nodes to view the articles that they share, and can
qualitatively inspect these articles to verify the decision of the algorithm. The data for this
graph was acquired in December 2022, and articles may have been deleted since.
Quantitative analysis can be done in a separate environment. Since the runtime of our
experiment was highly constrained for budgetary reasons, the resulting dataset of matching
articles is a semi-random subset of all possible articles. As such, no conclusions can be
drawn about specific actors, hubs or networks. However, some observations are of interest.
Most duplicates are shared within the first day after the original is published, usually
within the first hour. However, there are outliers of up to three years. Another surprising
observation is that although we started our analysis on Dutch websites, the algorithm
quickly found translations in English, Arabic and other languages, creating a diverse dataset
and showcasing the strengths of using a multilingual LLM. In one example, two Dutch
articles (listing eight health benefits of apples) with different wording were both linked to
an English article (which listed ten benefits), illustrating the models robustness.

4. Conclusion & Discussion
We have developed an algorithm that has proven to reliably and resiliently track the
spread of (dis)information across websites on the internet. By employing a multi-lingual
LLM we can detect translations, word replacements and other edits that would normally
obfuscate the link between a source and its duplicates. We determined for every analyzed
website how many of their published articles are original and how many are (mostly)
duplicates of other – and which – websites.
This data could conceivably be used to calculate a kind of trustworthiness index;
websites which copy many articles from untrustworthy sites can themselves be considered
untrustworthy. Articles that are originally published by untrustworthy sources can be
flagged as unreliable when they appear on other websites. These actions can inform internet
users and legal representatives. However, this analysis fell outside the scope of our
research.
The versatility of the used tools allows the application to be used in other ways. For
example, authors and publishers can use this approach to detect plagiarism, although we
did not compare our similarity scores to those given by conventional plagiarism detection
software. Additionally, the LLM could be replaced with an image-based foundation model
(such as Dall-E), which would allow the algorithm to track the spread of images across the
internet. By applying similarity scores to the embeddings of images, such a tool would be
resistant (to some extent) to any tampering with an image.
Within the context of disinformation, we believe that a quantitative analysis may also
reveal qualitative links between websites (e.g. one author writing for multiple publishers).
We have also done a rudimentary topic analysis: in our dataset, the words ‘corona’ and
‘vaccine’ occurred the most often in titles of articles, but ‘war’, ‘climate’ and ‘bitcoin’ were
also popular. A more sophisticated analysis may reveal interesting patterns of thought
within disinformation networks.
Some might consider it a downside that our algorithm completely ignores the content
of articles in determining the trustworthiness of the source. In our view this is one of its
major strengths: rather than reproducing the biases of a few programmers in an algorithm
which determines what is ‘true’, we allow users to qualitatively assess any number of
baselines based on their own expertise. This does mean that the quality of the initial
baselines chosen by the user will have a significant impact on the long-term effectiveness of
the algorithm as it progresses recursively. However, the transparency and explainability of
our method ensure that any questions about the algorithm can be answered, providing a
principled starting point in curbing the spread of disinformation.
Acknowledgements
We thank the Stichting Internet Domeinregistratie Nederland (Dutch Internet Domain
registration Foundation) for providing the funds for this research.

References
[1] World Economic Forum, The Global Risks Report 2024. 2024. [Online]. Available:
www.weforum.org
[2] C. Lim, “Checking how fact-checkers check,” Research and Politics, vol. 5, no. 3, Jul.
2018, doi: 10.1177/2053168018786848.
[3] M. Hameleers and T. G. L. A. van der Meer, “Misinformation and Polarization in a
High-Choice Media Environment: How Effective Are Political Fact-Checkers?,” Communic
Res, vol. 47, no. 2, pp. 227–250, Mar. 2020, doi: 10.1177/0093650218819671.
[4] M. Del Vicario et al., “The spreading of misinformation online,” Proc Natl Acad Sci U
S A, vol. 113, no. 3, pp. 554–559, Jan. 2016, doi: 10.1073/pnas.1517441113.
[5] “European Parliament P9_TA(2024)0138 Artificial Intelligence Act
(COM(2021)0206-C9-0146/2021-2021/0106(COD)) (Ordinary legislative procedure: first
reading),” 2019.
[6] P. Nakov et al., “Automated Fact-Checking for Assisting Human Fact-Checkers,” Mar.
2021, [Online]. Available: http://arxiv.org/abs/2103.07769
[7] A. Abid, M. Farooqi, and J. Zou, “Persistent Anti-Muslim Bias in Large Language
Models,” Jan. 2021, [Online]. Available: http://arxiv.org/abs/2101.05783
[8] C. Winship and R. D. Mare, “Models for Sample Selection Bias,” Annu Rev Sociol, vol.
18, no. 1, pp. 327–350, Aug. 1992, doi: 10.1146/annurev.so.18.080192.001551.
[9] M. pui S. Chan, C. R. Jones, K. Hall Jamieson, and D. Albarracín, “Debunking: A Meta-
Analysis of the Psychological Efficacy of Messages Countering Misinformation,” Psychol Sci,
vol. 28, no. 11, pp. 1531–1546, Nov. 2017, doi: 10.1177/0956797617714579.
[10] European Commission, “High-Level Expert Group on Artificial Intelligence set up by
the European Commission Ethics Guidelines for Trustworthy AI,” 2018. [Online]. Available:
https://digital-strategy.ec.europa.eu/en/library/ethics-guidelines-trustworthy-ai
[11] A. Conneau, G. Lample, M. Ranzato, L. Denoyer, and H. Jégou, “Word Translation
Without Parallel Data,” Oct. 2017, [Online]. Available: http://arxiv.org/abs/1710.04087