<!DOCTYPE article PUBLIC "-//NLM//DTD JATS (Z39.96) Journal Archiving and Interchange DTD v1.0 20120330//EN" "JATS-archivearticle1.dtd">
<article xmlns:xlink="http://www.w3.org/1999/xlink">
  <front>
    <journal-meta>
      <journal-title-group>
        <journal-title>An exploratory analysis, Journal of Business Research</journal-title>
      </journal-title-group>
      <issn pub-type="ppub">1613-0073</issn>
    </journal-meta>
    <article-meta>
      <article-id pub-id-type="doi">10.1016/j.jbusres.2011.07.029</article-id>
      <title-group>
        <article-title>Multilingual Behavioural Study of User Engagement with Disinformation on X</article-title>
      </title-group>
      <contrib-group>
        <contrib contrib-type="author">
          <string-name>Lorella Viola</string-name>
          <email>l.viola@vu.nl</email>
          <xref ref-type="aff" rid="aff0">0</xref>
        </contrib>
        <contrib contrib-type="author">
          <string-name>Workshop</string-name>
        </contrib>
        <contrib contrib-type="editor">
          <string-name>Disinformation, Cultural Influence, Multilingual Analysis, Engagement behaviour</string-name>
        </contrib>
        <aff id="aff0">
          <label>0</label>
          <institution>Vrije Universiteit Amsterdam (VU Amsterdam)</institution>
          ,
          <addr-line>De Boelelaan 1105, 1081 HV Amsterdam</addr-line>
          ,
          <country country="NL">The Netherlands</country>
        </aff>
      </contrib-group>
      <pub-date>
        <year>2018</year>
      </pub-date>
      <volume>66</volume>
      <issue>2013</issue>
      <fpage>105</fpage>
      <lpage>114</lpage>
      <abstract>
        <p>Research into user engagement with disinformation has grown rapidly, identifying factors like emotion, credibility, and content structure. Yet, how linguistic and cultural identity shape engagement remains understudied. This study fills this gap by exploring the dynamics of language and user engagement with disinformation in a multilingual online context. Using a data-set of more than 5,000 disinformation tweets about the human papilloma virus (HPV) vaccine in 30 languages, the analysis investigates the interaction between language, sentiment, and topical themes in shaping engagement with disinformation through metrics such as likes, retweets, quotes, shares, and replies. The regression, sentiment and topic modelling results reveal language-specific trends and cultural and contextual nuances. For example, tweets in Swedish show a strong positive correlation between sentiment and engagement, while German tweets display negative correlations for certain topics, such as vaccine eficacy. These findings indicate that the experience of disinformation is not universal and underscore the importance of analysing it through a multilingual and multicultural lens. The paper ends by ofering actionable insights for practitioners and researchers to improve the understanding of cultural dynamics in global communication, advancing methods to combat online disinformation in complex, multilingual environments such as social media platforms.</p>
      </abstract>
    </article-meta>
  </front>
  <body>
    <sec id="sec-1">
      <title>1. Introduction</title>
      <p>
        Online disinformation—intentionally disseminated false or misleading content causing public harm
[
        <xref ref-type="bibr" rid="ref1">1</xref>
        ]—has surged in recent years, especially after the COVID-19 health crisis. Recognized as a major
threat to personal and public safety, particularly in health, extensive research has focused on analyzing
the structure [
        <xref ref-type="bibr" rid="ref2 ref3">2, 3</xref>
        ], spread [
        <xref ref-type="bibr" rid="ref4">4</xref>
        ], impact [
        <xref ref-type="bibr" rid="ref5 ref6 ref7">5, 6, 7</xref>
        ], and content [
        <xref ref-type="bibr" rid="ref8 ref9">8, 9, 10</xref>
        ] of disinformation, including
conspiracy theories and fake news, and the consequences of health-related misperceptions on behaviour
[11, 12]. Scholars have also examined user interaction and network structure’s impact on disinformation
spread [13, 14] and features through which disinformation persuades individuals [15, 16, 17]. However,
despite rapid growth, the fundamental reasons individuals engage with disinformation remain unclear.
Scholars attribute this uncertainty to fragmented findings and a tendency to see disinformation merely
as the opposite of truth, neglecting its cognitive and subjective aspects [18, 15, 19, 16, 20]. While both
arguments are valid, this study argues that a key gap in the literature is the lack of research addressing
the role of culture and language in driving engagement with disinformation.
      </p>
      <p>This argument is supported by at least two factors. First, most current research focuses
disproportionately on a small number of advanced democracies like the United States and the United Kingdom [21].
A study by Seo &amp; Faris [22] found that 62.8 percent of empirical articles published in communication
journals between 2015 and 2020 used data from the U.S. Authors like Bajaj [21] argue that disinformation
is not experienced universally, and this geographic bias distorts our understanding of disinformation
and undermines mitigation eforts that ignore cultural dynamics.</p>
      <p>Second, specific cultural groups are often targeted by coordinated disinformation campaigns [
23].</p>
      <p>CEUR</p>
      <p>ceur-ws.org</p>
      <p>For example, research found that Russian disinformation campaigns on X exploit racial and political
identities to infiltrate online communities with politically charged content [ 24]. Disinformation also
reaches cultural groups through direct messages. Studies show that WhatsApp disinformation targeted
South Asian voters in North Carolina, while mistranslated content about Joe Biden targeted
Spanishspeaking voters in the U.S. [25].</p>
      <p>The global complexity of the digital communication landscape highlights the need to understand how
diferent cultural and linguistic groups are susceptible to disinformation. This study aims to address this
gap by investigating the role of linguistic and cultural identity in shaping disinformation experiences.
Using a data-set of 5,183 disinformation tweets in 30 languages, the study employs regression analysis,
sentiment analysis, and topic modelling to examine the relationship between language, sentiment, and
topical themes in shaping user engagement with disinformation. The results emphasize the significant
influence of linguistic and cultural factors in shaping individuals’ susceptibility to disinformation,
providing a more nuanced perspective on how people engage with disinformation across multilingual
contexts.</p>
    </sec>
    <sec id="sec-2">
      <title>2. User engagement and disinformation</title>
      <p>The literature on user engagement in online contexts has explored this phenomenon through various
frameworks and methodologies. Engagement is often defined as user-initiated actions that contribute to
the co-creation of value, as posited by Brodie et al. [26]. This broad definition captures the interplay of
behavioural, cognitive, and emotional dimensions, emphasizing the need to explore interaction nuances
and motivations. Shao [27] categorized user interaction into three behaviours: consumption (viewing),
participation (interacting), and production (creating content). Several studies have further diferentiated
between active participation (e.g., liking, commenting, sharing) and passive consumption (e.g., clicking,
watching) [28, 29], particularly on platforms like Facebook, YouTube, and X (formerly Twitter) [30].
Although passive users, or ‘lurkers’, comprise up to 90% of online communities [31, 32], a smaller active
subset significantly shapes the content landscape.</p>
      <p>Recent research has examined the factors driving user engagement with disinformation.
Emotionally charged content, including sensationalized headlines and narratives, has been found to amplify
engagement by exploiting emotions like fear and anger [33]. Visual elements and features like clickbait
and references to specific entities also enhance perceived credibility and emotional resonance, driving
engagement [34, 35]. Other factors, such as source credibility, social media fatigue, and fear of missing
out, have been found to influence the sharing of fake news [ 36, 37], designed to grab attention and
provoke emotions.</p>
      <p>The role of content creators in spreading disinformation is also significant [ 38]. Audiences trust
content from credible sources, with attributes like platform tenure, audience size, and verified profiles
increasing perceived credibility [39, 40]. Emotionally charged content from trusted sources, particularly
those with large followings, amplifies the spread of disinformation [ 41, 42].</p>
      <p>
        This study builds on this literature, proposing that language and culture are central to disinformation’s
ability to leverage emotion and drive engagement. The hypothesis is that subjective emotions like
fear and anger are shaped by one’s culture, values, and experiences [18]. While digital platforms have
amplified these emotions [
        <xref ref-type="bibr" rid="ref4">4, 43</xref>
        ], attributing them solely to technology oversimplifies the issue [ 44].
Disinformation has existed long before the internet, and factors like individual traits, cultural beliefs,
personality, education, and sociopolitical contexts such as populism and distrust in expertise also play
a role [45, 46]. Disinformation content must therefore resonate with audience values and enhance
the virality of its messages. For instance, research links conspiracy theories and cultural stereotypes
with the spread of fake news and hate speech [
        <xref ref-type="bibr" rid="ref6">18, 47, 6, 48</xref>
        ]. This study underscores the importance
of understanding linguistic and cultural dynamics in crafting strategies to combat disinformation and
protect public discourse. By examining the relationship between language, sentiment, and topical
themes, the results will ofer data-driven empirical evidence that highlights the critical role of linguistic
and cultural factors in shaping engagement.
      </p>
    </sec>
    <sec id="sec-3">
      <title>3. The HPV vaccine and disinformation</title>
      <p>In recent years, particularly following the COVID-19 health crisis, health-related disinformation has
surged globally, becoming one of the most significant threats to public well-being. Vaccine hesitancy,
historically influenced by time, location, and type of vaccine, was notably intensified by the 2020
pandemic [49, 44]. A key example of the intersection between disinformation and vaccine hesitancy is
the human papillomavirus (HPV) vaccine Gardasil, licensed by Merck in 2006 [12, 50].</p>
      <p>HPV, the most common sexually transmitted infection globally, includes over 100 strains, with an
estimated 80%-90% of individuals contracting it at some point [51]. Of these, 13 are oncogenic, with
cervical cancer being the most common HPV-related cancer, and the fourth most common cancer in
women worldwide [52, 53]. The vaccine, initially approved only for females aged 9 to 26, led to the
“feminization” of HPV, intertwining the vaccine with issues of gender norms and sexuality, including
perceptions of female virginity and promiscuity [54, 55, 56]. This focus overlooked men, especially
those who have sex with men, which exposed heteronormative biases [56]. By 2022, 125 countries had
included the vaccine in routine programs for girls, compared to only 47 for boys [57], perpetuating
misconceptions that HPV is solely a women’s issue and leaving men less protected.</p>
      <p>The feminization of HPV has made Gardasil a target for anti-vaccine disinformation. Since 2006,
critics have raised concerns about promiscuity, incomplete protection, mandatory vaccination, health
disparities, and unreported adverse efects [ 58, 11, 59]. These concerns have been amplified through
social media and propaganda films like Andi Reiss’s 2018 documentary Sacrificial Virgins: Not for the
Greater Good, which claims the vaccine caused paralysis in two girls. The film sparked controversy in
Australia, leading to an attempted ban on its distribution [60]. Other disinformation materials include
The HPV Vaccine on Trial: Seeking Justice for a Generation Betrayed [61] and a now-retracted article
suggesting the vaccine reduces fertility [62].</p>
      <p>Conspiracy theories surrounding the HPV vaccine have also flourished, blending science, politics,
economics, and gender. In China, for example, some claim the vaccine is a profit-driven tool or a
Western bioweapon targeting the Chinese population [11]. Broader conspiracies accuse governments
and pharmaceutical companies of fabricating vaccine data, claiming infertility or ovarian insuficiency,
or asserting the vaccine is part of a depopulation agenda [49, 63]. These narratives have fueled
public mistrust in health authorities and created a fertile environment for disinformation. Given the
global circulation of these narratives, understanding how diferent users engage with and respond to
disinformation narratives—especially those shaped by cultural and nation-specific contexts—becomes
crucial for analysing disinformation.</p>
    </sec>
    <sec id="sec-4">
      <title>4. Data and methodology</title>
      <p>The data-set covers a total duration of 394 days spanning a time period from 1 August 2022 to 30 August
2023. It contains 5,183 tweets in 30 languages and several attributes such as the tweet texts, the hashtags,
likes, replies, retweets, shares, and quotes count. A description of the data-set is given in Table 3 in
the Appendix. Data retrieval was conducted through targeted queries that extracted tweets containing
specific hashtags, including #HPV, #papillomavirus, and #Gardasil. To isolate disinformation content,
qualitative and quantitative analyses identified frequent additional hashtags, primarily #Agenda2030,
followed by #The GreatReset and #NewWorldOrder. Other commonly used hashtags were
#ProtegeonsNosEnfants (let’s protect our children), #nonsonovaccini (they are not vaccines), #VaccineAdverseEfects ,
#VaccineDeaths. The full list of hashtags is provided in 4 in the Appendix. These hashtags, identified as
being associated with disinformation, fake news, and conspiracy theories [64, 65, 66], were selected as
the basis for the working data-set. The data-set was pseudomysed to remove any identifiable references
and it can be provided upon request to the author.</p>
      <p>This study suggests that the behavioural analysis of user engagement with the selected posts ofers
quantifiable measures of disinformation dissemination and reception. Engagement is here defined
as the sum of likes, shares, retweets, quotes, and replies given to each tweet, as discussed earlier
in the paper. The study assumes that these metrics will indicate the interaction, endorsement and
general dissemination of the audience. After calculating users’ engagement, the analysis proceeds with
computing Latent Dirichlet Allocation (LDA) topic modelling (TM) [67] of the tweets’ content. TM is a
computational, statistical method to discover linguistic patterns in large collections of texts. Based on
distributional semantics theory [68], TM assumes that groups of words purport collective meanings, i.e.,
topics. By calculating the correlation between topic contributions as derived from the LDA model and
engagement, the analysis will determine the relationship between specific topics and how audiences
engage with content. Next, sentiment analysis is applied to the topics and the correlation between
sentiment and engagement is calculated. The aim is to examine whether the tone (positive, negative, or
neutral sentiment) of topics as derived from the tweets influences how audiences interact with them.
Finally, a comprehensive analysis of the relationships between language, sentiment, engagement, and
topics is calculated from the overall correlations and language-specific correlations to understand how
engagement behaviours vary across languages and topics.</p>
    </sec>
    <sec id="sec-5">
      <title>5. Analysis and results</title>
      <sec id="sec-5-1">
        <title>5.1. Preliminary observations</title>
        <p>Before proceeding to the analysis of the correlation between topics and users’ engagement, it is worth
examining whether there is a statistically significant diference in the average engagement for tweets
across various features, such as multimedia content, verified user status, hashtags, and mentions. This
helps identify patterns in how specific features impact the overall engagement. In turn, this analysis
provides an initial overview of users’ behaviour on X. Figures 1-4 show the results. Notable insights are
already emerging. In this analysis, a two-sample independent t-test was conducted for which tweets
were grouped based on the presence or absence of these specific features.</p>
        <p>For each feature, the T-test returned the size of the diference relative to the variation in engagement
values whereas the p-value determined the statistical significance of the diference below the significance
level of 0.05. The results show that tweets with multimedia content (e.g., images, videos, links) have
substantially higher average engagement (p-value=0.03 significant at the 0.05 level) (see 1a), confirming
that visual or interactive elements drive audience interaction. On the other hand, the use of hashtags
does not significantly afect engagement levels. In fact, engagement is slightly lower for tweets with
hashtags, suggesting that hashtags alone are not strong drivers of interaction in this data-set (see
1b). Similarly, tweets with mentions do not significantly difer in engagement from those without
mentions. Again, engagement is marginally lower for tweets with mentions, which may reflect limited
audience interest in direct interactions (see 1c). Finally, tweets by verified users generate dramatically
higher engagement, likely due to their perceived credibility, larger follower base, or higher visibility
in algorithms, as found in the literature and previously discussed here (see 1d). An overview of the
regression analysis results is provided in Table 2 in the Appendix.</p>
      </sec>
      <sec id="sec-5-2">
        <title>5.2. Topic modelling and engagement</title>
        <p>This part of the analysis applies LDA on the tweets’ content and calculates the correlation between
topic contributions and engagement to determine the relationship between specific topics and how
audiences engage with content. LDA was implemented using the LdaModel class from the Gensim
library. The optimal number of topics best_num_topics was determined empirically using coherence
scoring for higher interpretability and topic quality. Coherence scores were calculated across models
with 2 to 10 topics (see 2a). Based on the results, the 5-topic model was selected for analysis, as it
achieved the highest coherence score (0.5419), indicating the best balance between granularity and
interpretability. The 5-topic model was then trained on a bag-of-words representation of the tweets
(corpus), with a fixed random seed ( random_state=42) to ensure reproducibility.</p>
        <p>Model training employed an online variational Bayes algorithm with the update_every parameter
set to 1, enabling incremental updates after each mini-batch of size 100 (chunksize=100). A total of 10
(a) Tweets with media content
(b) Tweets with hashtags
(c) Tweets with mentions
(d) Tweets from verified users
full passes over the corpus were performed (passes=10). The hyperparameter alpha was set to ‘auto’ to
allow automatic asymmetric prior estimation for document-topic distribution, improving the model’s
ability to capture topic sparsity. Additionally, per_word_topics=True enabled the extraction of topic
distributions at the word level for more granular analysis. The top ten LDA model output keywords are
displayed in Table 5 in the Appendix.</p>
        <p>Subsequently, the topic contributions were correlated with engagement metrics using Pearson
correlation to assess how topic prominence influenced user interaction. The resulting correlation
measures a higher contributions from a topic associated with higher engagement (positive correlation)
and a higher contributions from a topic associated with lower engagement (negative correlation). The
correlation results between topics and engagement are displayed in Figure 2b.
(a) LDA coherence scores across models with 2 to 10</p>
        <p>topics
(b) Engagement correlation by topic</p>
        <p>Topic 1, with terms like ‘agenda2030’, ‘climatescam’, ‘greatreset’, and ‘newworldorder’, revolves
around conspiratorial narratives linked to global agendas, environment, and health. This topic seems
to frame climate action, global governance, and figures like Bill Gates as part of a coordinated elite
conspiracy (i.e., ‘their’) to control populations under the guise of progress or sustainability. Topic 2,
including keywords such as ‘video’, ‘pfizer’, ‘información’, focuses on suspicion towards mainstream
narratives, especially pharmaceutical institutions (e.g., Pfizer), and could imply manipulation or
concealment of information at the national (Spanish or Spanish-speaking regions) and international level.
Topic 3 includes terms like ‘plandemia’ (a term combining ‘plan’ and ‘pandemic’ implying that the
pandemic was orchestrated), ‘world’, ‘wefpuppets’, highlights the idea that global institutions (e.g.,
WEF) manipulate world events and public health crises (‘plandemia’), positioning the globalist agenda as
invasive. Topic 4 conveys resistance to perceived authoritarian impositions (e.g., ‘sanitary dictatorships’)
and possibly loss of freedom under the new world order, potentially with regional references like Chile.
It is also the topic more associated with Gardasil and HPV discussions. Finally, Topic 5 conflates public
health (vaccines, COVID-19), geopolitical conflict (Ukraine/Russia), and political leadership (e.g., Giorgia
Meloni) with a polarized moral framing (‘malditaagenda2030’). The emergence of specific phrases
such as Agenda2030, newworldorder, and wefpuppets reflects the blending of health discussions with
political and social issues. This indicates the impact of concerns around these topics on the broader
vaccine discussion.</p>
        <p>In terms of engagement, the Pearson correlation analysis indicates that Topic 4 and Topic 5 exhibit the
highest positive correlation with engagement. Centred on conspiratorial and vaccine-related discussions,
the result indicates significant public interest in pandemic-related conspiracies and globalist narratives.
On the other hand, Topic 1, Topic 2, and Topic 3 show negative correlations with engagement. This
result suggests that while topics such as climate conspiracies and WEF might be prevalent, they are
less efective in driving audience interactions. This may suggest audience saturation towards these
discussions or reduced credibility among mainstream users. In the next part, the analysis examines
how sentiment within each topic impacts engagement to identify whether positive or negative tones
drive interactions.</p>
      </sec>
      <sec id="sec-5-3">
        <title>5.3. Sentiment analysis and topics’ engagement</title>
        <p>This part of the analysis examines the relationships between sentiment, topics, and engagement levels
by exploring correlations, sentiment distribution, and average topic contributions for high- and
lowengagement posts. This will ofer insights about which topics drive more audience interaction in
relation to sentiment. First, sentiment analysis (SA) was performed on the entire data-set using the
XLM-RoBERTa multilingual sentiment model [69]. This model was chosen because it was trained
specifically for Twitter sentiment analysis and supports over 50 languages, including those present in
the data-set.</p>
        <p>After performing SA, the Pearson correlation coeficient was calculated between the topic contribution
(i.e., the degree to which a tweet associated with a specific topic drove engagement, as in previous
step) and the sentiment text polarity (positive, neutral, or negative) (see 3a). A median engagement
value was first used to categorize tweets into tweets with engagement above the median (i.e., high
engagement) and tweets with engagement at or below the median (i.e., low engagement). Afterwards,
mean sentiment values were calculated to examine whether high-engagement tweets are more positive
or negative compared to low-engagement ones. Figure 3b shows the range and variance of sentiment
in each group for each topic. Positive bars represent more positive sentiment overall (i.e., average
sentiment &gt; 0) whereas negative bars indicate more negative sentiment overall (i.e., average sentiment &lt;
0). Colours are used to highlight diferences between high (blue) and low (red) engagement, independent
of the actual sentiment polarity.</p>
        <p>The overall sentiment is positive across all topics. Topic 3 has the highest sentiment suggesting that
discussions around conspiracies the health crisis and global institutions elicit more supportive reactions.
Tweets here may be ironically enthusiastic or rallying support around anti-globalist claims, leading
to a net positive tone. Topic 1 and Topic 2 display that more neutral or slightly negative tones (e.g.,
skepticism towards ‘agenda2030’ or climate issues) resonate more with the audience driving higher
(a) Overall sentiment correlation with engagement per
topic
(b) Sentiment trends by topic of high vs low
engagement
engagement. This suggests that engagement may be driven by controversial or critical discussions.
Topic 4 exhibits the lowest average sentiment for both high- and low-engagement tweets. As this topic
is one of the two that exhibit the highest engagement overall, this indicates that sentiment may not
be a primary driver of engagement for it. In Topic 5, more positive sentiment correlates with higher
engagement indicating that content on geopolitical events, health issues, and political figures may boost
average positivity. This may suggest that audiences respond more to afirmative or supportive tones.</p>
        <p>As the net sentiment remains above zero, the results point to an ironic usage of positive tones, for
example mocking conspiracies, or supportive discourse towards them. Overall, topics referencing global
agendas and conspiracies seem to be discussed in a supportive way which, especially in high-engagement
tweets, may mean overall disinformation endorsement.</p>
      </sec>
      <sec id="sec-5-4">
        <title>5.4. Language and engagement</title>
        <p>This part of the analysis examines possible language-specific diferences that can indicate that audience
responses vary significantly according to cultural or linguistic factors. Conversely, if very weak or no
correlation is found, this may indicate that other factors (e.g., topic relevance) might play a larger role
in driving engagement. The data-set language classification was performed using Fasttext [ 70] which
identified 30 languages. The language distribution across posts is provided in the Appendix. Given
the uneven distribution of languages in the data-set—where Spanish, English, and French collectively
account for approximately 49.4% of all tweets (see Table 1)—average engagement was calculated for the
ten most represented languages to enhance the robustness and generalisability of the findings.</p>
        <p>Language</p>
        <p>Number of posts</p>
        <p>First, the correlation between language and engagement was calculated for these languages, with
ES
EN
FR
DE
NL
IT
PT
TR
CA
SV
results shown in Figure 4. Although the overall correlation is weak (0.030), there are significant
disparities between languages. French (fr) has the highest average engagement, indicating that tweets
in French perform exceptionally well in terms of likes, replies, and retweets. In contrast, languages
like Swedish (sv) and Catalan (ca) show the lowest engagement, while German (de) and Spanish (es)
are in the middle. This may suggest limited content or lower user activity in these languages. The
stark contrast between French and other languages underscores how audience responses vary, with the
near-zero overall correlation indicating that language alone does not systematically drive engagement
across the dataset, though individual languages may exhibit distinct patterns.</p>
        <p>Next, the correlation between sentiment and engagement by language was calculated to examine
cultural diferences. The results, displayed in Figure 5a, show that Swedish (sv) has the highest positive
correlation, meaning more positive sentiment aligns with higher engagement in Swedish tweets.
Languages like Catalan (ca), Dutch (nl), and Spanish (es) also show positive correlations, while Italian
(it), German (de), and French exhibit negative correlations, suggesting that negative sentiment may
attract more engagement in these languages. This implies that more positive sentiment can reduce
engagement, while negative sentiment may drive more attention. A few languages, such as English
and Spanish, show minimal correlation, indicating that sentiment has little efect on engagement in
these cases. This suggests that content topic might be a more significant factor in these languages.
Overall, while sentiment can influence engagement, the efect appears language-specific and modest,
highlighting the complexity of audience dynamics and the need for localized content strategies.
(a) Sentiment and engagement correlation by language.
(b) Polarising topics by language.</p>
        <p>The analysis examines the correlation between topics and engagement across diferent languages,
aiming to identify variations and patterns in potentially polarizing and audience-dependent topics.
Results are shown in Figure 5b, where each bubble represents the relationship between a language and
a topic, with bubble size indicating the strength of the engagement correlation. Larger bubbles signify
stronger correlations, while the colour indicates sentiment: blue for negative and red for positive, with
darker shades representing stronger sentiment polarity.</p>
        <p>The results indicate that engagement with all topics is generally associated with negative sentiment
across most languages. However, some languages show diferent patterns: Swedish (sv) exhibits strong
positive engagement, particularly with Topic 1 (large red bubble), suggesting users react positively
to conspiratorial narratives about climate change and global agendas. Catalan (ca) shows moderate
engagement with slightly positive sentiment for Topics 1 and 3, reflecting discussions on pandemic
conspiracies. Italian (it) and German (de) show medium blue bubbles across all topics, indicating
moderate engagement and negative sentiment, possibly reflecting skepticism towards health-related
authoritarian narratives. Dutch (nl) displays low engagement and sentiment neutrality, with small
bubbles and pale colors. English (en) has very small bubbles, suggesting weak engagement and sentiment
polarity. Finally, French (fr), Turkish (tr), and Portuguese (pt) show moderate engagement, with smaller,
neutral-to-lightly-colored bubbles.</p>
        <p>Overall, the results reveal that reactions to disinformation topics are highly language-specific. For
example, Topic 1 (likely related to broad conspiracies like Agenda 2030) is polarizing in Swedish and Italian
but less so in English and Dutch. Topic 3 triggers negative sentiment in several European languages (it,
de), while Topic 4 sees positive engagement in some communities like Swedish and Portuguese. Catalan
and Swedish emerge as particularly responsive, with strong sentiment and engagement, indicating
these communities are more engaged with disinformation. These findings highlight the importance
of tailoring content strategies to match language-specific sentiment trends and audience’s cultural
identities.</p>
      </sec>
    </sec>
    <sec id="sec-6">
      <title>6. Discussion</title>
      <p>The findings of this study provide valuable insights into how cultural and linguistic factors shape
user engagement with disinformation, especially in multilingual contexts. One key outcome is the
clear evidence that disinformation engagement is not universal; it is influenced by linguistic and
cultural nuances. For example, negative sentiment tweets in French, German, and Italian showed high
engagement, suggesting that negative tones in these languages foster more interaction. In contrast,
Swedish tweets displayed positive sentiment and engagement, revealing a culture-specific response to
certain narratives. This highlights the need for a multicultural approach when examining
disinformationrelated user behaviour.</p>
      <p>The topic modelling and sentiment analyses revealed several key trends. Topics on globalist agendas
(e.g., Topics 1 and 3) and vaccine-related discussions (e.g., Topic 4) saw the highest engagement,
particularly in Swedish and German, indicating strong public interest in health-related disinformation
in these communities. Conversely, broader conspiracy topics like ‘Agenda 2030’ and ‘The Great
Reset’ exhibited high engagement across various languages, suggesting widespread interest in these
narratives. Language-specific trends in sentiment and engagement further highlight cultural and
emotional influences. For example, Swedish and Catalan users responded positively to positive sentiment
content, while German, Italian, and French users showed stronger engagement with skeptical or neutral
sentiment. Furthermore, the study highlighted the role of verified user status and multimedia content
in driving engagement. Tweets by verified users and those with multimedia elements consistently
garnered higher interaction, emphasizing the importance of perceived credibility and visual content in
amplifying disinformation.</p>
      <p>This study provides critical insights for fostering healthier online discourse, empirically validating how
language-specific sentiment, topic relevance, and engagement intersect in multilingual contexts. The
identified language- and culture-specific patterns ofer a foundation for designing targeted mitigation
strategies which can be tailored to align with the emotional and cultural dynamics of diferent linguistic
communities. Additionally, the study’s findings on verified users and multimedia content suggest
strategies for platform moderation. Platforms could leverage visual content and prioritize credible
sources while limiting content from unverified accounts spreading disinformation.</p>
    </sec>
    <sec id="sec-7">
      <title>7. Conclusions</title>
      <p>This study shows how linguistic and cultural factors significantly shape disinformation engagement,
challenging the idea of a universal audience response. Analysing over 5,000 tweets in 30 languages,
the research reveals that disinformation’s efectiveness varies across linguistic and cultural contexts,
influenced by sentiment, topic relevance, and audience demographics. The findings therefore highlight
the need for a multilingual and multicultural approach in disinformation mitigation. Tailoring strategies
to the cultural and linguistic dynamics of diferent communities allows policymakers, researchers,
and social media platforms to promote more informed and resilient public discourse, reducing the
harmful impacts of disinformation. Finally, the emergence of COVID-19, climate change, and Agenda
2030-related topics suggests their influence on the broader vaccine discussion. This indicates that
global issues are reshaping public discourse around vaccines; as discussions become more complex and
interconnected, public health messaging must address a wide range of concerns, with comprehensive
approaches being more efective than isolated messaging.</p>
    </sec>
    <sec id="sec-8">
      <title>8. Limitations</title>
      <p>Despite its comprehensive scope, this study has a number of limitations. First, the data-set has an uneven
distribution of languages, with some languages being overrepresented and others having significantly
fewer tweets. This discrepancy could bias the findings and limit generalizability. Future studies
could address the languages uneven data-set distribution by actively sampling the under-represented
languages.</p>
      <p>Second, the reliance on engagement metrics such as likes, shares, and retweets may not reflect deeper
cognitive or emotional responses. Future studies could incorporate qualitative methods, such as user
interviews and linguistic analysis of the tweets, to gain deeper insights into cognitive and emotional
responses. Finally, future works could investigate the role of platform algorithms in amplifying certain
types of disinformation across diferent linguistic and cultural contexts, to expand the scope of the
analysis and enhance our understanding of how to combat online disinformation.</p>
    </sec>
    <sec id="sec-9">
      <title>Declaration on Generative AI</title>
      <p>During the preparation of this work, the author used GPT-4o in order to: Grammar and spelling check
and to generate images. After using this tool, the author reviewed and edited the content as needed and
takes full responsibility for the publication’s content.
[10] D. Fallis, A Conceptual Analysis of Disinformation (2009). URL: https://hdl.handle.net/2142/15205.
[11] L. Chen, Y. Zhang, R. Young, X. Wu, G. Zhu, Efects of Vaccine-Related Conspiracy Theories on
Chinese Young Adults’ Perceptions of the HPV Vaccine: An Experimental Study, Health
Communication 36 (2021) 1343–1353. URL: https://www.tandfonline.com/doi/full/10.1080/10410236.2020.
1751384. doi:10.1080/10410236.2020.1751384.
[12] A. Yagi, Y. Ueda, T. Kimura, HPV Vaccine Issues in Japan: A review of our attempts to promote
the HPV vaccine and to provide efective evaluation of the problem through social-medical and
behavioral-economic perspectives, Vaccine 42 (2024) 125859. URL: https://www.sciencedirect.com/
science/article/pii/S0264410X24004079. doi:10.1016/j.vaccine.2024.03.080.
[13] I. O. Quintana, R. Reimann, M. Cheong, M. Alfano, C. Klein, Polarization and trust in the evolution
of vaccine discourse on Twitter during COVID-19, PLOS ONE 17 (2022) e0277292. URL: https:
//journals.plos.org/plosone/article?id=10.1371/journal.pone.0277292. doi:10.1371/journal.pone.
0277292, publisher: Public Library of Science.
[14] K. Gunaratne, E. A. Coomes, H. Haghbayan, Temporal trends in anti-vaccine discourse on
Twitter, Vaccine 37 (2019) 4867–4871. URL: https://www.sciencedirect.com/science/article/pii/
S0264410X1930876X. doi:10.1016/j.vaccine.2019.06.086.
[15] M. Reddi, R. Kuo, D. Kreiss, Identity propaganda: Racial narratives and disinformation, New
Media &amp; Society 25 (2023) 2201–2218. URL: https://doi.org/10.1177/14614448211029293. doi:10.
1177/14614448211029293, publisher: SAGE Publications.
[16] Z. Bastick, Would you notice if fake news changed your behavior? An experiment on the
unconscious efects of disinformation, Computers in Human Behavior 116 (2021) 106633. URL:
https://www.sciencedirect.com/science/article/pii/S0747563220303800. doi:10.1016/j.chb.2020.
106633.
[17] M. Butter, P. Knight (Eds.), Routledge handbook of conspiracy theories, Routledge, Abingdon,</p>
      <p>Oxon ; New York, NY, 2020.
[18] L. Viola, ‘Barren lesbians plotting sterilization’: gender stereotypes and prejudices in health
disinformation narratives, a cross-cultural analysis of social media of the HPV vaccine, in:
C. Tebaldi, A. Plum, C. Purschke (Eds.), Conspiracy as Genre: Narrative, Power and Circulation,
Bloomsbury Academic, London, 2025.
[19] C. Birchall, P. Knight, Conspiracy Theories in the Time of Covid-19, 1 ed., Routledge, London,
2022. URL: https://www.taylorfrancis.com/books/9781003315438. doi:10.4324/9781003315438.
[20] J. Kirchner, C. Reuter, Countering Fake News: A Comparison of Possible Solutions Regarding User
Acceptance and Efectiveness, Proceedings of the ACM on Human-Computer Interaction 4 (2020)
1–27. URL: https://dl.acm.org/doi/10.1145/3415211. doi:10.1145/3415211.
[21] S. G. Bajaj, Digital Disinformation Threats and Ethnocultural Diasporas, in: G. Adlakha-Hutcheon,
C. Kelshall (Eds.), (In)Security: Identifying the Invisible Disruptors of Security, Springer Nature
Switzerland, Cham, 2024, pp. 53–65. URL: https://doi.org/10.1007/978-3-031-67608-6_3. doi:10.
1007/978- 3- 031- 67608- 6_3.
[22] H. Seo, R. Faris, Comparative Approaches to Mis/Disinformation| Introduction, International
Journal of Communication 15 (2021) 8. URL: https://ijoc.org/index.php/ijoc/article/view/14799,
number: 0.
[23] J. Gursky, M. Riedl, S. Woolley, The disinformation threat to diaspora
communities in encrypted chat apps, Brookings (2021). URL: https://www.brookings.edu/articles/
the-disinformation-threat-to-diaspora-communities-in-encrypted-chat-apps/.
[24] D. Freelon, T. Lokot, Russian Twitter disinformation campaigns reach across the American
political spectrum, Harvard Kennedy School Misinformation Review 1 (2020). URL: https:
//misinforeview.hks.harvard.edu/article/russian-disinformation-campaigns-on-twitter/. doi:10.
37016/mr- 2020- 003.
[25] K. Mimizuka, I. Trauthig, WhatsApp, Misinformation, and Latino Political Discourse
in the U.S. | TechPolicy.Press, Tech Policy Press (2022). URL: https://techpolicy.press/
whatsapp-misinformation-and-latino-political-discourse-in-the-u-s.
[26] R. J. Brodie, A. Ilic, B. Juric, L. Hollebeek, Consumer engagement in a virtual brand community:</p>
      <p>Feature
Multimedia Content
Verified Users
Hashtags
Mentions</p>
      <p>Statistic
agenda
climatescam
greatreset</p>
      <p>todos
newworldorder
agenda21
billgates
people
their
global
video
piped
contra
sobre
pfizer
información
nothing
naciones
humanidad
nicht
plandemia
world
mundo
mundial
wefpuppets</p>
      <p>están
repentinitis
globalista
ahora
about
gardasil
chile
merck
papillomavirus</p>
      <p>libertad
nuevoordenmundial
wefpuppet
globalismo</p>
      <p>entre
dictadurasanitaria
covid19
noalaagenda2030
watch
ukraine
russia
vacunas</p>
      <p>being
giorgiameloni
malditaagenda2030
goodnews</p>
      <p>Count
2035
2000
524
136
118
85
66
55
39
24
17
17
12
11
8
6
5
3
3
3
2
2
2
2
2
1
1
1
1
1
1</p>
    </sec>
  </body>
  <back>
    <ref-list>
      <ref id="ref1">
        <mixed-citation>
          [1]
          <string-name>
            <given-names>J. A.</given-names>
            <surname>Tucker</surname>
          </string-name>
          ,
          <string-name>
            <given-names>A.</given-names>
            <surname>Guess</surname>
          </string-name>
          ,
          <string-name>
            <given-names>P.</given-names>
            <surname>Barbera</surname>
          </string-name>
          ,
          <string-name>
            <given-names>C.</given-names>
            <surname>Vaccari</surname>
          </string-name>
          ,
          <string-name>
            <given-names>A.</given-names>
            <surname>Siegel</surname>
          </string-name>
          ,
          <string-name>
            <given-names>S.</given-names>
            <surname>Sanovich</surname>
          </string-name>
          ,
          <string-name>
            <given-names>D.</given-names>
            <surname>Stukal</surname>
          </string-name>
          ,
          <string-name>
            <given-names>B.</given-names>
            <surname>Nyhan</surname>
          </string-name>
          , Social Media, Political Polarization, and
          <article-title>Political Disinformation: A Review of the Scientific Literature</article-title>
          ,
          <year>2018</year>
          . URL: https://papers.ssrn.com/abstract=3144139. doi:
          <volume>10</volume>
          .2139/ssrn.3144139.
        </mixed-citation>
      </ref>
      <ref id="ref2">
        <mixed-citation>
          [2]
          <string-name>
            <given-names>J. Van</given-names>
            <surname>Prooijen</surname>
          </string-name>
          ,
          <string-name>
            <surname>K. M. Douglas</surname>
          </string-name>
          ,
          <article-title>Belief in conspiracy theories: Basic principles of an emerging research domain</article-title>
          ,
          <source>European Journal of Social Psychology</source>
          <volume>48</volume>
          (
          <year>2018</year>
          )
          <fpage>897</fpage>
          -
          <lpage>908</lpage>
          . URL: https: //onlinelibrary.wiley.com/doi/10.1002/ejsp.2530. doi:
          <volume>10</volume>
          .1002/ejsp.2530.
        </mixed-citation>
      </ref>
      <ref id="ref3">
        <mixed-citation>
          [3]
          <string-name>
            <surname>J.-W. Van Prooijen</surname>
            ,
            <given-names>M. Van Vugt</given-names>
          </string-name>
          ,
          <source>Conspiracy Theories: Evolved Functions and Psychological Mechanisms, Perspectives on Psychological Science</source>
          <volume>13</volume>
          (
          <year>2018</year>
          )
          <fpage>770</fpage>
          -
          <lpage>788</lpage>
          . URL: http://journals. sagepub.com/doi/10.1177/1745691618774270. doi:
          <volume>10</volume>
          .1177/1745691618774270.
        </mixed-citation>
      </ref>
      <ref id="ref4">
        <mixed-citation>
          [4]
          <string-name>
            <given-names>E.</given-names>
            <surname>Bonnevie</surname>
          </string-name>
          ,
          <string-name>
            <given-names>A.</given-names>
            <surname>Gallegos-Jefrey</surname>
          </string-name>
          ,
          <string-name>
            <given-names>J.</given-names>
            <surname>Goldbarg</surname>
          </string-name>
          ,
          <string-name>
            <given-names>B.</given-names>
            <surname>Byrd</surname>
          </string-name>
          ,
          <string-name>
            <given-names>J.</given-names>
            <surname>Smyser</surname>
          </string-name>
          ,
          <article-title>Quantifying the rise of vaccine opposition on Twitter during the COVID-19 pandemic</article-title>
          ,
          <source>Journal of communication in healthcare 14</source>
          (
          <year>2021</year>
          )
          <fpage>12</fpage>
          -
          <lpage>19</lpage>
          . ISBN:
          <fpage>1753</fpage>
          -
          <lpage>8068</lpage>
          Publisher: Taylor &amp; Francis.
        </mixed-citation>
      </ref>
      <ref id="ref5">
        <mixed-citation>
          [5]
          <string-name>
            <given-names>K. T.</given-names>
            <surname>Simms</surname>
          </string-name>
          ,
          <string-name>
            <given-names>S. J. B.</given-names>
            <surname>Hanley</surname>
          </string-name>
          ,
          <string-name>
            <given-names>M. A.</given-names>
            <surname>Smith</surname>
          </string-name>
          ,
          <string-name>
            <given-names>A.</given-names>
            <surname>Keane</surname>
          </string-name>
          ,
          <string-name>
            <given-names>K.</given-names>
            <surname>Canfell</surname>
          </string-name>
          ,
          <article-title>Impact of HPV vaccine hesitancy on cervical cancer in Japan: a modelling study, The Lancet</article-title>
          .
          <source>Public Health</source>
          <volume>5</volume>
          (
          <year>2020</year>
          )
          <fpage>e223</fpage>
          -
          <lpage>e234</lpage>
          . doi:
          <volume>10</volume>
          .1016/S2468-
          <volume>2667</volume>
          (
          <issue>20</issue>
          )
          <fpage>30010</fpage>
          -
          <lpage>4</lpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref6">
        <mixed-citation>
          [6]
          <string-name>
            <given-names>B.</given-names>
            <surname>Stabile</surname>
          </string-name>
          ,
          <string-name>
            <given-names>A.</given-names>
            <surname>Grant</surname>
          </string-name>
          ,
          <string-name>
            <given-names>H.</given-names>
            <surname>Purohit</surname>
          </string-name>
          ,
          <string-name>
            <given-names>K.</given-names>
            <surname>Harris</surname>
          </string-name>
          , Sex, Lies, and
          <article-title>Stereotypes: Gendered Implications of Fake News for Women in Politics</article-title>
          ,
          <source>Public Integrity</source>
          <volume>21</volume>
          (
          <year>2019</year>
          )
          <fpage>491</fpage>
          -
          <lpage>502</lpage>
          . URL: https: //doi.org/10.1080/10999922.
          <year>2019</year>
          .
          <volume>1626695</volume>
          . doi:
          <volume>10</volume>
          .1080/10999922.
          <year>2019</year>
          .
          <volume>1626695</volume>
          , publisher: Routledge _eprint: https://doi.org/10.1080/10999922.
          <year>2019</year>
          .
          <volume>1626695</volume>
          .
        </mixed-citation>
      </ref>
      <ref id="ref7">
        <mixed-citation>
          [7]
          <string-name>
            <given-names>L.</given-names>
            <surname>Chen</surname>
          </string-name>
          ,
          <string-name>
            <given-names>Q.</given-names>
            <surname>Ling</surname>
          </string-name>
          ,
          <string-name>
            <given-names>T.</given-names>
            <surname>Cao</surname>
          </string-name>
          , K. Han, Mislabeled, fragmented, and
          <article-title>conspiracy-driven: a content analysis of the social media discourse about the HPV vaccine in China</article-title>
          ,
          <source>Asian Journal of Communication</source>
          <volume>30</volume>
          (
          <year>2020</year>
          )
          <fpage>450</fpage>
          -
          <lpage>469</lpage>
          . URL: https://www.tandfonline.com/doi/full/10.1080/01292986.
          <year>2020</year>
          .
          <volume>1817113</volume>
          . doi:
          <volume>10</volume>
          .1080/01292986.
          <year>2020</year>
          .
          <volume>1817113</volume>
          .
        </mixed-citation>
      </ref>
      <ref id="ref8">
        <mixed-citation>
          [8]
          <string-name>
            <given-names>B.</given-names>
            <surname>Wiggins</surname>
          </string-name>
          , '
          <article-title>Nothing Can Stop What's Coming': An analysis of the conspiracy theory discourse on 4chan's /Pol board</article-title>
          ,
          <source>Discourse &amp; Society</source>
          <volume>34</volume>
          (
          <year>2023</year>
          )
          <fpage>381</fpage>
          -
          <lpage>398</lpage>
          . URL: https://doi.org/10.1177/ 09579265221136731. doi:
          <volume>10</volume>
          .1177/09579265221136731, publisher: SAGE Publications Ltd.
        </mixed-citation>
      </ref>
      <ref id="ref9">
        <mixed-citation>
          [9]
          <string-name>
            <given-names>M.</given-names>
            <surname>Demata</surname>
          </string-name>
          ,
          <string-name>
            <given-names>V.</given-names>
            <surname>Zorzi</surname>
          </string-name>
          ,
          <string-name>
            <surname>A</surname>
          </string-name>
          . Zottola (Eds.),
          <article-title>Conspiracy theory discourses, number volume 98 in Discourse approaches to politics, society and culture</article-title>
          , John Benjamins Publishing Company, Amsterdam ; Philadelphia,
          <year>2022</year>
          .
        </mixed-citation>
      </ref>
    </ref-list>
  </back>
</article>