This paper presents a natural language processing technique for studying memory on the far-right political discussion foru m/pol/ on 4chan.org. Memory and the use of history play a pivotal role on the far-right for temporally structuring beliefs about social life and order. However, due in part to methodological limitations, there is a lack of knowledge regarding the speci昀椀c historical entities that make up the far-right memory culture and wider historiography. To better grasp the structure of farright memory, this paper opts for a data-intensive methodology, using machine learning on a data set of approximately 66 million posts fro/mpol/ from 2020. 19,821 random posts were manually annotated, according to the presence of historical entities. A昀琀er evaluating interrater reliability, data were used to train a naïve Bayes text classi昀椀er to learn the lexical features of so-called “posts of memorPyO”M( s). A昀琀er parameter tuning, the model extracted from the dataset a total of 1.083.47P1OMs with a precision score of98.43%. It is argued that this technique provides a novel way to automate the identi昀椀cation of historical entities within the far-right authored text, of bene昀椀t for the 昀椀elds of memory studies and farright studies, two 昀椀elds that have traditionally relied on more qualitative close-reading approaches. By investigating the mnemonic features of the/pol/ posts during steps in the methodological pipeline, the paper contributes important insights into the challenges of identifying and classifying lexical features in hyper-vernacular digital spaces like 4chan, where communication is highly de昀椀ned by intertextuality, semantic ambiguity, and cacography.
Memory plays a pivotal role for far-right groups as it does in most processes of collective
identity formation. Previous work on the memory practices of far-right groups and actors has
emphasized, how elements from the past are strategically energized within this ideological
environment as symbolic and cultural building blocks for forming identity, strengthening group
a昀케liations, systematizing ideological strands, and directing contemporary political objectives
[
No study has to our knowledge mapped the far-right memory-scape in its totality. This is
due in part to the limited ability of qualitative methods to make generalizable claims about
the wider far-right milieu. Instead, such methodological approaches are more o昀琀en employed
for the purpose of ethnographic studies of select subsections or case studies of the far-right,
such as the memory practices of single actors or group2s5,[
Large-scale data-driven approaches o昀er a partial solution to this issue of generalizability. More speci昀椀cally, statistical techniques for supervised and unsupervised learning make it possible to analyze the lexical qualities of memory on the far-right in ways normally unfeasible by traditional close-observational studies.
To reconstruct the memory-scape of the far-right, this paper presents an observational study
that combines systematic annotation and text classi昀椀cation to the ‘politically incorrect’-board
(colloquially known a/spol/) of 4chan.org, a social media site and chat forum well known for
harboring far-right views and content6,[
Posts from 4chan’s/pol/-board were randomly extracted from a dataset of threads from 2020 using 4chan’s API. In total19, 821 random posts were manually annotated by one of the authors inDoccano [27] for the presence of memory, sorting all posts into two non-overlapping groups: posts of memoryP(OM) and posts not of memory n(on-POM). Conceptually, memory on /pol/ is de昀椀ned as reference to historical entities in posts targeting those narrative categories in which a disorganized past becomes meaningful in discourse. Without assuming a priori the precise contents of such entities, they are lexically and semantically speaking related to historical signi昀椀ers such as the names of past individuals, events, periods, etc. In order to delineate between entities conceptually understood by users as belonging to either the past or present, the concept only considers entities related to before the year 2000.
With this approach out of the19, 821 posts, 1, 236 meet the criteria as beingPOMs, with the other18, 585 posts consequently being classi昀椀ed as non-POMs. In other words,6.24% of posts contained references to a historical entity before the year 2000.
To test annotator reliability, three independent raters were recruited and tasked with
annotating a subset of the19, 821 posts. A昀琀er being instructed in the coding procedure, they were each
provided with a dataset of500 posts consisting of 50/50 randomly selected posts from each
category (POM/non-POM). Cohen’s was calculated for the original annotations and each rater
independently: Rater 1 = .94, rater 2 = .92, and rater 3 = 0.82. Landis and Koch’s
benchmark for interpreting kappa values suggests that aof more than 0.81 is considered
‘almost perfect’ [24]. Fleiss’ – recommended for multiple raters determining among nominal
categories [
Before model training, a series of text normalization transformations were applied to the posts, speci昀椀cally lemmatization, case-folding, and removal of punctuation. The posts were subsequently structured using a vector space model of lexical features, speci昀椀cally the empirical probability of words and sequential combinations (i.e., n-grams). The n-gram range was treated as a parameter value together with minimum and maximum document feature frequency.
For model training, data were balanced with under-sampling, that is, by randomly sampling
1, 236 non-POMs without replacement, corresponding to the total numberPoOfMs, and
removing the rest of the majority category, resulting in a balanced dataset of 50% posts from each
category. As opposed to over-sampling (i.e., multiplying data from the minority category to
match the level of the majority category), research suggests that under-sampling provides
better sensitivity at the cost of speci昀椀city [
The multinomial Naive Bayes algorithm was chosen as a classi昀椀cation technique due to its
computational e昀케ciency and level of explainability 3[
=1 = ∈{ 1, 2} ( ∣ ) (1) (2)
Model parameters were optimized using a train/test split ratio of 75/25 w1i4th9, 760 candidate parameter values for nine parameters using a 昀椀ve-fold cross-validation method, totaling 748, 800 昀椀ts. When comparing an unoptimized model without default parameters to the optimized model, precision increased fro6m3.49% to 76.19%. By shi昀琀ing the decision threshold value, making the model more discriminant in only classifying posts with a predicted probability over0.9 as a POM, precision increased to98.43%.
A note on the interpretation of parameter optimization, the optimal n-gram range w[1a,s5], suggesting that historical entities can be expressed through a multitude of complex word sequences. The minimum document frequency of a feature was determined as two documents (i.e., the model 昀椀lters out words that appear in less than two posts). The fact that the optimal minimum document frequency is not higher, suggests that even very rare words contribute to model learning, seeing as they might represent obscure historical entities. Conversely, optimal maximum document frequency was determined to b3e0%, a somewhat extensive upper boundary, suggesting that the corpus is riddled with many common words that have little signi昀椀cance in terms of learning.
To understand the classi昀椀er’s behavior, we looked at the speci昀椀c n-grams that were most predictive of the two categories. By counting the number of times each word occurs in all posts divided by the number of posts in each category, we calculated the percentage of times a word has appeared in one or the other category. In other words, by dividing the number of times a word appears in thePOM category by the number of times it appears in thenon-POM category, we are able to calculate a ‘memory ratio’ for each word and rank them according to this scale. To avoid division by zero in the instances where a word only occurs in one category, a pseudo-count of 1 was added to all word counts, similar to the Laplace smoothing technique used to regularize the naive Bayes algorithm to avoid a probability estimate of zero when a feature value does not occur in a given category. Looking at the top and bottom of this sorted list of n-grams in table1, we see what words most strongly correlate with each category i.e., what features are most and least ‘memory-like’.
The results indicate that the model correctly picks up on key historical entities that to a human interpreter is understandable as having mnemonic signi昀椀cance, for example, distinct historical persons, events, or places like ‘Hitler’, ‘Rome’, ‘Holocaust’, ‘Weimar’ and ‘WW2’, but also more general historical or temporal terms such as ‘history’, ‘century’, ‘ancient’, ‘civilization’, and ‘ancestor’. There are also less historically signifying terms like ‘catholic’, ‘Germany’, or ‘Europe’, implying that these have been used in mnemonic context across multiple posts for the model to start associating these terms with thPeOM category. Such re-contextualization of generally nondescript terms is also interesting in the cases of ‘build’ and ‘steal’, which indicate that /pol/ users ascribe speci昀椀c sentiments to their mnemonic discussions, expressed through not only historical entities themselves, but also with descriptive modi昀椀ers such as verbs.
Conversely, the features in table1 with the lowest memory ratio expectedly signify little mnemonic substance or contain less contextual meaning (such as words like ‘she’, ‘her’, ‘anyway’, ‘no no’, and ‘ah’). The low scores of words like ‘Biden’, ‘virus’, ‘test’, and ‘death’ also indicate that discussions about memory do not overlap with discussions about contemporary events in 2020, such as the election of Joe Biden and the COVID-19 pandemic. Terminology speci昀椀c to 4chan like ‘tfw’ (that face when), ‘kek’ (synonymous to LOL; laughing out loud), ‘image’, ‘post’, as well as ‘rare’ (referring to the ‘rare 昀氀ag’ meme about users from small or unfamiliar countries), are also unrelated to tPhOeM category, suggesting that discussions about memory are a distinct subtheme on/pol/, branched o昀 from more general 4chan topics.
Looking at the predicted probability oPfOMs provided by the model, many posts with high predicted probably are also generally lengthier and mention several historical entities. For example, one post with a word length o1f33 (decidedly higher than the corpus average o4f4 number of words per post), and with 9a9.23% probability of being aPOM, repeats the word ‘USSR’ four times as well as consisting of other signi昀椀cant historical entities, such as ‘Soviet Union’ and the names of former Russian leaders. In contrast, scanning through posts with lowPOM probability, these are much shorter (posts with on1l0y% POM probability contain on average 23 words), and use many unspeci昀椀c words.
Examples from the data set can provide further insights into the model’s learning behavior. Consider a post from the dataset that reads: ‘enjoy your bat meat, you fucking barbarians.’ While the term ‘barbarian’ was originally used in ancient Greece to refer to non-Greek peoples based on cultural-linguistic di昀erences, and would therefore adhere to the target class criteria, the context within which it is used here – involving the zoonotic origin of the coronavirus – is detracted from its original historical context. This is despite the remnant of its historically rhetorical function still undergirding its usage in the posts (i.e., to ‘uncivilize’ someone by stereotyping them as barbarian). The semantic ambiguity – that there is not a direct correlation between the use of a term and that term always represents a historical entity – necessarily complicates the machine learning procedure, because the model relies on the assumption that there exists an unambiguous lexical distinction between thPeOM and non-POM category’s textual content. Semantic ambiguity at a lexical level tends to be the rule rather than the exception in natural language, and hence an inevitable condition of any machine learning process dealing with unstructured text from a real-life environment. Some terms will inherently 昀氀uctuate in their potential for expressing memory.
Depending on the circumstance, the composition of entities can be more or less metonymically representative of that particular conceptual category to which they are assumed to belong. By exploring the lexical features of an abstract category such as memory, we are not only taking the necessary, precautionary steps of transparently revealing the data that goes into the machine learning model but are also shedding light on the complex ways that memory is expressed through language in decidedly ambiguous ways. This can be demonstrated from an example in the dataset: ‘we wuz vikangz.’ Brie昀氀y put, the post is a satirical rehashing of the meme colloquially known as ‘we wuz kangz’, which was originally directed towards a type of Afrocentric memory concerning the disputed and anachronistic claim that ancient Egypt was a black civilization. Consequently, the rehashing is now being used to satirize the pretense of people claiming to have Viking ancestry. While certainly interesting on its own as a case for how memory can be embedded in multilayered intertextual contexts, the post also directs our attention towards a speci昀椀c characteristic of the data. That is, historical entities can be represented in text by lexical symbols that may be synonymous in their meaning but which are orthographically heterogeneous in their written format. The words ‘Vikings’, and ‘Vikangz’ both connote the same historical entity, but since they are spelled di昀erently it blocks the machine learning model from disambiguating their semiotic synonymity.
The presence of homonyms also needs to be taken into consideration as these also distort the informational quality of the data. As an example, the word ‘Rome’ was o昀琀en used to refer to the historical Mediterranean civilization but is also used to denote the capital of present-day Italy. Likewise, ‘Romans’ would generally refer to the people of ancient Rome, but would also 椀昀gure in discussions of biblical scripture, referring to the Letter to the Romans, part of the Pauline epistles. While rare, these homonymic cases exemplify some of the linguistic features of the data that need to be considered when constructing the machine learning model.
The subtle variations of meaning and semantics explored in the dataset present di昀케culty for
accurately extracting memory on/pol/. Nevertheless, by highlighting these cases, the article
has identi昀椀ed some of the preliminary linguistic characteristics that undergird discourses about
memory on/pol/, which ultimately become important to consider when building and
evaluating machine learning models for 4chan-speci昀椀c and/or history-related classi昀椀cation problems.
Consequently, a major contribution of this study lies in its impelling of a still rather nascent
and sparse 昀椀eld of research that employs various natural language processing techniques in
the study of memory in the digital realm speci昀椀cally, e.g.42[
It also speaks directly to the di昀케culty of operationalizing a socially situated memory
concept in contrast to more individual-oriented studies attempted by neurobiologists and cognitive
psychologists, most recently pointed out by29[] and [28]. Similar critiques of the supposed
conceptual unclarity of “memory”, regarding the concept’s supposed over-extension and
semantic overloading (for example in the metaphorical misuse of psychological terms such as
“trauma” in supra-individual social contexts), resulting in redundant and unsophisticated uses
of the concept as a rhetorical signal, rather than as a clearly de昀椀ned analytical tool, have also
been pointed out by [
However, we do not argue to have presented anything approaching a sui generis methodology, completely separate and distinct from previous endeavors. If anything, the detailing of many of the elements of note involved in applying machine learning, from evaluating statistical biases to the exploration of linguistic variations, shows how a project like this requires an inclusive, interdisciplinary outlook capable of combining and balancing the potential from multiple perspectives: including both theory and method, the close reading spotlight and the distant reading bird’s eye view, human awareness and computer industriousness.
There are also certain limitations involved with this method. As has been pointed out previously, there is a general disconnect on both a practical and theoretical level between the lexical data that the models were trained on and those conceptual de昀椀nitions that structured the qualitative interpretation of that data. While ‘historical entities” as a concept, allegorically symbolizing objects of some abstract and intangible past, may be comprehensible on a theoretical level as the basic constituents of mnemonic communication, there is no guarantee that such a concept, once operationalized, is bijectional translated into exactly replicable lexical items making up a conversation on/pol/. In other words, the word tokens that are the fundamental components of the machine learning models are dimensionally, linguistically, and conceptually speaking di昀erent from the theoretical de昀椀nition of memory, even when such inferential parallelism is assumed in the study’s design. The polysemantic and cacography of natural language thus impedes machine learning, for example when it has to determine the mnemonic signi昀椀cance of ambivalent features, or when it needs to learn from quirky spelling and terminology.
Moreover, given the multifarious representations that conceptually fall under the category of historical entities, there is also a certain bias related to the initial, manually coded dataset. Even though the dataset was constructed from a random sampling o/fpol/ posts, there are quite likely numerous historical entities that did not make it into this relatively small subset of the general/pol/ conversation, even if they might have appeared relatively frequently and would have been important to the board’s collective memory. Such biases would then be replicated in the model’s algorithm, resulting in favoritism of historical entities that statistically might describe very well the core memory o/npol/, represented by historical entities and other “memory lingo” so common as to appear distinctly in the random sampling. However, it would be blind to the “edges” of this collective memory, to the speckling of historical entities that the model never got a chance to see, and which were subsequently not included in the extraction process and therefore most likely wouldn’t feature as some of the top words in the 昀椀nal topic modeling. This is, of course, but an unavoidable condition of necessarily decomplexifying the exceedingly amorphous cultural concept of memory into quanti昀椀able entities suitable for study. With any perspective striving for the macroscopic, there is bound to be a corresponding loss in detail. [18] [19] [20]
A. Jatowt, D. Kawai, and K. Tanaka.Digital History Meets Wikipedia: Analyzing Historical Persons in Wikipedia. Conference Paper. 2016. doi:10.1145/2910896.2910911. url: https: //doi.org/10.1145/2910896.291091 1.
N. Kanhabua, T. N. Nguyen, and C. Niederée. “What triggers human remembering of events? A large-scale analysis of catalysts for collective memory in Wikipedia”IE.IEnE:/ACM Joint Conference on Digital Libraries (2014), pp. 341–350.
W. Kansteiner. “Finding Meaning in Memory: A Methodological Critique of Collective Memory Studies”. In:History and Theory 41.2 (2002), pp. 179–197. [21] K. L. Klein. “On the Emergence of Memory in Historical Discourse”. RIne:presentations 69 (2000), pp. 127–150. [22] C. Kølvraa. “Embodying ‘the Nordic race’: imaginaries of Viking heritage in the online communications of the Nordic Resistance Movement”. InP:atterns of Prejudice 53.3 (2019), pp. 270–284. doi: 10.1080/0031322x.2019.1592304. url: https://doi.org/10.1080/0031322 X.2019.1592304. [23] K. Krippendor昀. Content analysis: An introduction to its methodology. 3rd edition. Thousand Oaks, CA: Sage, 2013. [24] J. R. Landis and G. G. Koch. “The Measurement of Observer Agreement for Categorical Data”. In: Biometrics 33.1 (1977), pp. 159–174. doi: 10.2307/2529310. url: http://www.jst or.org/stable/252931.0 [25] C. Miller-IdrisT.he extreme gone mainstream: commercialization and far right youth culture in Germany. Princeton: Princeton University Press, 2017. [26] [27]
A. Nagle.Kill All Normies: Online Culture Wars from 4chan and Tumblr to Trump and the Alt-Right. Zero Books, 2017.
H. Nakayama, T. Kubo, J. Kamura, Y. Taniguchi, and X. Liangd.occano: Text Annotation Tool for Human. 2018. url: https://github.com/doccano/doccano. [28] J. Olick.The Politics of Regret: On Collective Memory and Historical Responsibility. Routledge, 2007. [29] J. K. Olick, Vinitzsky-Seroussi, V., and D. Levy. “Introduction”. ITnh:e Collective Memory Reader. Ed. by D. L. Je昀rey K. Olick Vered Vinitzky-Seroussi. Oxford University Press, 2011, pp. 3–62. [30] G. J. R. “Memory and Identity: The History of a Relationship.” InC:ommemorations. Ed. by J. R. Gillis. Princeton: Princeton University Press, 1994, pp. 3–24.