In the face of world-changing events, narratives on the messaging platform Telegram, including instances of disinformation, tend to arise and evolve at high speeds. However, key signals of this process, including newly emerging or idiosyncratic concepts, often elude traditional, top-down analyses. Addressing the need for inductive approaches to narrative evolution on Telegram, this paper operationalizes quantitative methods from the field of corpus-based discourse analysis. On a technical and methodological level, the paper discusses how data from Telegram's messages and images can be collected and preprocessed for the purposes of a 'keyness' (Log Ratio) analysis that surfaces salient nouns and verbs for further investigation. On an empirical level, this method is then applied to a case study of 225 predominantly Dutch-speaking Telegram channels (spanning the period March 2017- March 2022), revealing some of the dynamics that govern their recent shift from propagating narratives about the coronavirus pandemic to narratives concerning the war in Ukraine. This case study is accompanied by an interactive demonstrator that enables readers to further explore the processed dataset. The paper concludes with a reflection on the status of and future avenues for this 'distant reading' approach in relation to established interpretative practices.
overload on Telegram and beyond, the development In political science, the concept of ‘narrative’ has of inductive, machine-guided methods for mining broadly been defined as a form of discourse in which narratives from (social media) texts at ‘big data’ humans “construct disparate facts in [their] own scale has become an active area of research. First worlds and weave them together cognitively in or- examples of such computational analyses of narrader to make sense of [their] reality” [1, p.135]. At a tives can be traced back to work on scripts, story time when world-changing events such as pandemics grammars, and planning formalisms from the field and wars happen in rapid succession, this process of of artificial intelligence [ 4]. More recent continnarrative sense-making is intensified on social me- uations of this line of research have mapped the dia. There, spanning countless posts and channels, underlying structures and dynamics of narratives eclectic facts are continuously (re)combined into by representing them as (evolving) networks of relanew stories, including instances of disinformation tions between ‘actants’ figuring in texts, the latter and conspiracy theory. A prototypical example of concerning people, places, or organizations that are this are the narratives that circulate on Telegram, a detected through techniques such as Named Entity messaging platform that through a lack of central- Recognition (NER) [5]. Following a similar logic, ized content moderation tends to harbor conspiracy some texts have explored the possibilities aforded theories and other misleading or antagonistic dis- by co-occurence networks of inductively-sourced course usually not tolerated on social media such as hashtags to trace dynamics of converging narratives Twitter [2, 3]. In this prolific environment, newly- [6]. These empirically-informed approaches have coined and often idiosyncratic concepts (such as thus yielded first insights into the structural ties the provocative ‘denazification’ used by the Rus- that allow online conspiracy theories and other narsian government to legitimize the war in Ukraine) ratives to form from seemingly disparate concepts can emerge and propagate freely, which renders and information. keyword-based query designs and other top-down As this paper aims to elaborate, the study of methods for identifying narratives on the platform online narratives, including the aforementioned rather inefective. network-based approaches, can benefit from bottomup methods for identifying the idiosyncratic and evolving concepts that constitute those narratives.
Previous literature in media studies has for inword embeddings to demonstrate how platforms content travels from one channel to its many folsuch as 4chan form incubators for “robust vernacu- lowers, who can receive and forward the content lar innovations” [7]. These conceptual innovations to other channels, but not respond to it. As such, include neologisms such as ‘redpill’ (referring to an Telegram channels can efectively be considered “deawakening from ignorance), whose emerging and positories” and “amplifiers” of narratives [11, p.3]. shifting meanings can be interpreted as traces of Identifying relevant Telegram channels from the evolving narratives that define antagonistic sub- which to mine these narratives is a non-trivial matcultural communities. Comparable assumptions ter, as channels can be scattered and dificult to underpin quantitative, corpus-based analyses that identify by channel name alone. Therefore, channels trace the propagation of specific ‘vernacular’ con- were retrieved by means of an established ‘snowcepts between platforms as means of identifying the balling’ method for Telegram research described ‘mainstreaming’ of fringe narratives [8, 9, 10]. Here, in Peeters and Willaert [12]. This method repurinnovative or marked words that appear outside of poses Telegram’s afordance of message-forwarding the fringe environments in which they were first ob- between channels as a means of identifying related served can be considered traces of a wider adoption channels. It assumes that if one channel forwards a of certain narratives. message from another channel, a meaningful connec
Addressing the need for inductive approaches to tion or shared interest exists between both. Starting narrative evolution on Telegram, the present paper from a seed list of channels defined based on expert operationalizes quantitative methods from the field knowledge, the researcher can thus retrace these of corpus-based discourse analysis. On a technical links to other channels, bringing into view a netand methodological level, the paper oefrs a discus- work of interconnected channels in a bottom-up sion of how data from messages and images from way. the platform can be collected and preprocessed for For the purposes of this paper, a tailor-made the purposes of a ‘keyness’ (Log Ratio) analysis that scraper based on Python’s Selenium library was surfaces salient nouns and verbs for further investi- used to automate and scale-up this process.1 The gation. On an empirical level, this method is then network of channels under investigation in this arapplied to a case study of 225 predominantly Dutch- ticle was first mapped in the summer of 2021. At speaking Telegram channels (spanning the period that time, these channels were mainly preoccupied March 2017- March 2022). This case study tests the with the coronavirus pandemic and associated nardouble hypothesis that 1) around the time of the ratives, making this a suitable sample for exploroutbreak of the war in Ukraine, Telegram channels ing further narrative evolutions. The contents of that previously spread disinformation narratives these channels (both texts and images) were subseon the coronavirus pandemic embraced narratives quently scraped again in March 2022. This results about the war, and that 2) this shift might reveal in a dataset of 821,020 messages from 225 public aspects of the underlying mechanisms governing the Telegram channels pertaining to Dutch-speaking evolution of disinformation on the platform. To far-right and conspiracy-theory communities, spanfoster further exploration, this case study is accom- ning a period between 18 March 2017 and 11 March panied by an interactive demonstrator that allows 2022. users to search and plot words from the dataset by An initial inspection of this dataset revealed their keyness scores. The paper concludes with a that narratives were constructed in both messages wider reflection on the status of and future avenues and images, with some images containing relefor this ‘distant reading’ approach to narratives in vant patches of text. Working towards a ‘multirelation to established interpretative practices. modal’ analysis that considers these aspects of the data, the channel contents were processed further along two tracks. Firstly, the texts embedded in 2. Data Collection the images were programmatically extracted using Google’s Tesseract-OCR engine, by means of the Python-tesseract wrapper.2 Secondly, the languages of the retrieved texts (from both posts and images) were detected using the Python ‘langdetect’ library.3 This created opportunities for working
evolution in message texts and images from public Telegram channels pertaining to Dutch-speaking far-right and conspiratorial communities. Following the taxonomy of platform afordances proposed in Van Raemdonck and Pierson [11], Telegram channels can be considered to aofrd “directed and isowith linguistically-homogeneous subcorpora in the subsequent analysis.
After preprocessing, it was found that of the retrieved messages, ca. 85% (697,364 messages) contained a non-empty message text field, and ca. 33% (267,956 messages) contained an image file. 4 After cleaning the outputs of the OCR for images, such as removing ‘texts’ that only contained newline characters, texts could be extracted from ca. 67% (179,904) of the images. Automated language detection revealed that the corpus was multilingual (in part due to the forwarding of messages from international channels), with English and Dutch being the most prominent languages. Of the message texts, ca. 21% (143,120) were classified as written in English, and ca. 57% (399,842) in Dutch. For the images from which texts were extracted, we found ca. 53% (94,803) contained text in English, and ca. 28% (51,138) contained text in Dutch. The prominence of English texts in the images again points towards an international dynamic of message and content forwarding between channels.
In order to inductively detect signals of narrative evolution in the collected data, this paper applies the method of ‘keyness’ analysis. This approach from the fields of corpus linguistics and corpusbased discourse analysis is directed at identifying ‘key’ items (e.g. words) in a target corpus in relation to a reference corpus based on the frequencies of items in both corpora. As such, a keyness analysis can support an exploratory approach to texts that gives an indication of their “aboutness” [13, p.227]. Arguably, this makes the method well suited for our purposes of identifying emerging narrative signals in texts. The keyness metric chosen for this paper is that of Log Ratio, which is defined as the “binary log of the ratio of relative frequencies” [14]. This gives a measure of the actual observed diference between two corpora for a key item (rather than a measure of statistical significance). The advantage of this is that it allows for the sorting of items by the size of the actual frequency diference between the corpora, enabling us to find the top N most key items. In order to calculate the Log Ratio for an item in target corpus C1 and a reference corpus C2, we take the binary logarithm of the ratio of the normalised frequencies of the term in C1 and C2 (which are each multiplied by a factor of 1,000,000 for readability purposes).5
As illustrated in Figure 1, our overall approach to narrative detection on Telegram, then, is to detect these key items from a reference corpus of texts grouped by week in relation to all remaining data. We then consider the items with the highest keyness scores for each timestamp, thus opening them up for further interpretation. Concretely, this technical pipeline comprises the following steps: 1. We filter the data by content type (message texts, image texts, or combinations of both) and language (Dutch or English). 2. We group the texts by timestamps (viz. per week of data). 3. We clean the texts at each timestamp by removing hyperlinks and emojis. 4. We perform part of speech tagging and retain only nouns and verbs (as we consider these to express core concepts). 5. We calculate the frequencies for these items per timestamp (week). 6. We calculate the Log Ratio of the target corpus (normalized frequencies) in relation to all other weeks.
7. Finally, we rank words by keyness score.
On a conceptual level, this approach returns keyness scores for items in relation to the combined
multiple images, the scraper only stored the first image attached to the message.
//kristopherkyle.github.io/corpus-analysis-python/ Python_Tutorial_7.html https: data that precede and follow it – ofering a distant Ukraine and the Russia-Ukraine crisis in general” inperspective on distinctive (key) narrative signals deed become more frequent in the discourse of these for each week’s worth of data in relation to the full communities. The actual (pro-Russian) narratives period minus that week. The keyness scores for themselves were then analysed on the basis of closethe final timestamp have a special status in this readings of articles from the most frequently shared regard, as they reveal key items in relation to all of domains in the dataset [idem.]. As the dataset the preceding data, illustrating what is key at the investigated in the aforementioned study closely relast moment of observation. It should be acknowl- sembles the one introduced in the present article, edged upfront that this keyness analysis does not we can hypothesize that a similar transition from yet integrate semantics, apart from the significance coronavirus-related narratives to narratives about attributed to nouns and verbs as key indicators the war in Ukraine should be observable in our corof narratives. As will be expanded upon in the pus. Moreover, we can also hypothesize that our inconclusion, this approach thus requires further in- ductive approach can reveal more detailed traces of terpretation and contextualization of the detected the actual narratives that thus emerge, thus opening key items. up perspectives on the more fundamental dynamics
In order to illustrate this method and make an underlying this narrative evolution.
empirical contribution to the study of narrative In order to interpret the results of the keyness
dynamics on Telegram, the following section zooms analysis in light of these hypotheses, they have been
in on a case study that investigates the relation integrated into an interactive demonstrator or
‘obbetween narratives about the coronavirus pandemic servatory’ [17] that allows for interactive exploration
and the war in Ukraine as expressed in our corpus. and plotting of terms based on their keyness scores.
This ‘observatory’ covers the full dataset (only
snapshots of which are discussed in the present paper)
4. Case Study and Findings and is openly available online.6
A first observation that can be made on the basis
Recent and on-going events such as the coronavirus of our keyness analysis, is that we can indeed see
pandemic and the war in Ukraine have kindled an emerging traces of narratives concerning the war
interest in the evolutionary dynamics of (disinfor- in Ukraine. The table in Figure 2 shows the top
mation) narratives among researchers, civil society 20 nouns and verbs (by keyness score) retrieved
actors, and journalists. One comparative analysis of for the last four weeks of English message texts in
international fact-checks has for instance revealed the dataset. From this overview, it follows that
some striking, high-level parallels between disinfor- discourse in these messages distinguishes itself from
mation surrounding both events in terms of style previous weeks through references to the war in
and contents [15]. Examples from this study in- Ukraine. Possible first traces are already observed
clude references to Nazism (e.g. the coronapass as in the week of February 20 in the form of a
refa Nazi ‘health passport’ or Ukraine as a region that erence to “mobilisation”. Further, more explicit
should be ‘denazified’), and recurring conspiracies references can be found in ensuing weeks, which
about secret laboratories (e.g. false claims that the feature high-keyness words such as “demilitarize”
coronavirus was created in a lab and references to and “bombards”
A recent study by the Institute of Strategic Di- gitimizing the war such as “denazify” (27/02/2022) alogue confirms that this can indeed be the case clearly evoke Nazism. The analysis likewise fore[16]. Based on the analysis of a dataset of 229 grounds references to the biolaboratories conspiracy German-language Telegram channels (spanning the mentioned earlier (e.g. “biolaboratories”, “biosciperiod between 1 November 2021 and 27 February entist” (13/03/2022)). Results of a wider search 2022) pertaining to far-right and conspiracy the- for terms referring to biology laboratories shown ory communities, this study has shown that terms in Figure 3 reveal that an earlier segment of the from a preconstructed list of 80 keywords related to “Russia, Ukraine, the breakaway regions in Eastern 6https://jvansoest.github.io/ channels continuously adapt narratives to match ongoing events.
above indeed reveals traces of narratives related to the war in Ukraine in communities that were previously mainly concerned with the pandemic.
Furthermore, our inductive approach brings into
view three more general dynamics governing this
transition. Firstly, it was possible to observe both
emerging narratives as well as more ‘stable’
undercurrents. Secondly, our case study suggests that
certain narratives recur over time. Thirdly,
expanding the scope of the investigation indicates that the
recent shifts between narratives are part of a longer
Figure 2: Top 20 nouns and verbs with highest key- process of narrative evolution.
ness scores for message texts in English for the last four Given the specific nature of the corpus under
conweeks of the dataset
Finally, the repeated occurrence of ‘biolaborato- narrative innovations and narrative continuity in a ries’ as a high-keyness item suggests that within the bottom-up way by operationalizing keyness analysis Dutch-speaking disinformation communities under (Log Ratio). For another, the paper applied this investigation, narratives contextualizing the war in method to the case study of narrative evolution Ukraine are but a salient pivot point in an ongo- on Dutch-speaking Telegram channels (pertaining ing process of narrative evolution. Ofering a more to far-right and conspiracy theory communities). ‘zoomed out’ perspective, Figure 4 shows the re- It has thus been shown how keyness analysis can sults of the keyness analysis for texts in English be applied to Telegram data to inductively idenfrom both messages and images combined, covering tify traces of emerging or persistent narratives that a more extended period of time. A closer inspec- might warrant further investigation. tion of key items predating the Russian invasion of It should be acknowledged that the exploratory Ukraine hint at a range of other events that have scope of the present paper has its limitations. Buildbeen appropriated to match the then-predominant ing on these initial results, at least two pathways agenda of the communities. Our method for in- for future research can be envisaged. On a methodstance picks up traces of references to the freedom ological and technical level, more work is needed convoys in Canada (e.g. “Winnipeg”, “blockading” to reduce noise and introduce additional granular(06/02/2022)), which demonstrates how Telegram ity in the analysis. As has been illustrated in this paper, transferring methods from corpus-based dis- detection and optical character recognition (text course analysis to Telegram requires intensive data- extraction from images) suitable for Telegram’s idpreprocessing. Future investigations might for in- iosyncratic (visual) discourse. Along the same lines, stance explore more refined methods for language future research might complement the aggregated perspective on ofer and explore the distributions of key items over channels, thus bringing into perspective more intricate relations between channel dynamics and discourse. Finally, additional methodological work is needed to situate the retrieved items in their wider semantic networks, for instance through statistically-informed co-occurrence analyses. Introducing further granularity, one promising avenue here would be to contextualize key items through graph-like representations of narratives inferred from the sentences’ argument structure [18].
On a more conceptual level, our analysis raises bigger questions of meaning and interpretation. As indicated, the keyness analysis itself does not capture the semantics of the messages and image texts under investigation. Meaning has to be assigned to key items by the human interpreter, for instance by considering and comparing combinations of key items, by looking up the retrieved key words in the corpus and reading the messages or image texts in which they figure, or through broader cultural or media-theoretical contextualization. This foregrounds the question of how critical frameworks might be developed that streamline and formalize the integration of inductive methods from data science and interpretative approaches from the humanities. Proposals for such frameworks have been made under the denominator of ‘data hermeneutics’ [19, 20], opening up the field for future work on actionable implementations.