Catching Feelings: Aspect-Based Sentiment Analysis for Fanfiction Comments about Greek Myth

Abstract

The application of sentiment analysis in literary studies has been limited and often criticized, yet aspectbased sentiment analysis (ABSA) ofers interesting applications in this domain because it addresses some limitations of traditional SA tools and provides a more detailed and context-sensitive analysis of sentiment. To investigate its usage in literary reception studies, we apply ABSA to a corpus of ±25,000 comments written by readers in response to fanfiction about Greek mythology on fanfiction website Archive of Our Own (AO3), one of the largest platforms for fanfiction in English. Our ABSA pipeline detects sentiment (positive/negative) associated with eight aspects of fanfiction stories (general evaluation, Greek mythology, character, character emotion, reading experience, writing style, events and storyworld, and non-specific sentiment). We explain the process of data collection and annotation and present a small inter-annotator agreement study (Pairwise Cohen'0s.86 for aspects and 0.88 for sentiments). We develop, evaluate, and fine-tune a machine-learning pipeline for ABSA, tackling the aspect extraction and sentiment analysis tasks respectively. We obtain the best results using NuNER for aspect extraction (0.5 macro F1) and Twitter-roBERTa-sentiment for sentiment analysis (0.75 macro F1). Finally, we outline some avenues for future research and reflect on the generalizability of our method to other domains, especially to fanfiction from other fandoms and platforms but also other social media.

Keywords

• eol>aspect-based sentiment analysis (absa)
• online discourse
• fanfiction
• reader response
• classical reception
• web data

1. Introduction

Fanfiction – stories inspired by existing works, written by and for an audience of fans, and published for free online – is a burgeoning domain of online cultural actiAvritcyh.ive of Our Own (AO3), one of the largest English-language websites for publishing and reading fanfiction, recently reported hosting over 13 million wor3k1s].[ Fan studies scholars emphasize the need to understand fanfiction in relation to its community contexts, because fanfiction is usually written ”within and to the standards of a particular fannish communit9y]”[and ”is often so deeply embedded within a specific community that it is practically incomprehensible to those who don’t share exactly the same set of references3[ 5 ].” As such, ”a fan fiction story cannot be viewed as a wholly self-contained object, a text delimited by the first and last words that appear on the screen, the way that readers of novels and other genres of print culture conventionally read books as bounded by their covers11[].” Due to the deeply intertwined nature of fanfiction texts and the shared practices and contexts of their online community – which fans cfaalnldom – fanfiction texts must be approached ”as both literary and digital objects4[].” Scholars need methods to study the ways fanfiction is received, embedded and evaluated in online spaces like AO3 that account for its hybrid literary/digital nature.

Additionally, fanfiction often revolves around emotional trajectories. Love, heartbreak, mental illness and other emotional highs and lows are frequently central to fanfiction, which has been described as ”emotional landscapes of readin3g3[]” both because of its intradiegetic emphasis on emotionality and because of its capacity to evoke strong emotions in readers.

The comment-section on AO3 lets readers share their afective responses to fanfiction reading, which we conceptualize as sentiments in the current paper. Sentiments can develop over the course of the reading and may extend or endure into the time after. Through comments, readers express these sentiments in the social context of their fan community. Sentiments are important for understanding which aspects of fanfiction are loved, enjoyed, and appreciated. The hybrid digital/textual nature of fanfiction comment, the enormous scale of fanfiction and comment production and exchange as born-digital objects, and the fanfiction community’s emphasis on both intradiegetic and extradiegetic emotion make fanfiction comments suitable for computational approaches such as sentiment analysis (SA).

The application of Natural Language Processing (NLP) methodologies to date in literary studies has been limited 5[ , 23 ]. Specifically, SA, which categorizes texts as positive, neutral, or negative, is often criticized and considered inadequate for the detail-oriented research needs of literary scholars20[ , 34 ]. The sentiments present in literary or narrative text often cannot be reduced to a negative/positive binary. To address some of the limitations of existing SA tools, aspect-based sentiment analysis (ABSA) has gained attention. Unlike conventional SA that labels the sentiment of entire documents, paragraphs, or sentences, ABSA operates at the aspect level, extracting specific aspects and their corresponding sentiment polaritie3s, [ 36]. Although ABSA ofers a more granular approach to sentiment mining, its application has largely been confined to commercial domains such as customer review3s6[] and applications in (computational) literary studies still need to be explored.

To fill these research gaps – the application of ABSA to literary studies data and the use of NLP to analyze textual/digital fanfiction comments – we employ ABSA to detect the aspects of fanfiction that readers refer to when writing comments and the sentiments commenters attach to these aspects, conceptualized in a binary fashion (positive/negative). We address a methodologically-oriented research question about fanfiction reception on AO3: • What are the afordances and limitations of aspect-based sentiment analysis (ABSA) for analyzing fanfiction comments?

In other words, we investigate whether ABSA is an efective approach to analyzing the aspect-sentiment combinations in a dataset of fanfiction comments. We limit this research to one platform (AO3) for reasons of scope. We focus on the Greek mythology fandom because Catching Feelings is part of the larger projecAtnchoring and Innovating Greek Myth in Contemporary Online Fanfiction (2022-2026), which focuses on the way contemporary online fanfiction transforms cultural material about Greek mythology and the way this cultural material resonates with online fan communitie1s.

Section 2 outlines the processes of data collection and annotation. Secti3ondescribes our ABSA pipeline. Section4 reports on the inter-annotator-agreement of the annotated dataset, and presents the results of various ABSA pipelines. Sectio5nexamines these results. Finally, Sections 6 reflects on the reproducibility, generalizability and limitations of our approach to ABSA for fanfiction comments and outlines some avenues for future research, and in Section 7 we answer our research question.

2. Data

2.1. Data Collection

We based our comment dataset onMythFic Metadata [ 29 ], an existing collection of metadata for 5.155 works of fanfiction about Greek mythology that were published on AO3 between 2008 and 2022.2 We used the list of unique identifiers for the stories inMythFic Metadata as input for the AO3-api [ 14 ] to collect comments on those stories. The resulting dataset contains 25.970 comments. Table1 provides descriptive statistics of the comments in terms of character- and word count.

We must address three limitations of this dataset. First, fanfiction comments, like any type of textual review or comment, do not fully capture the complex mental and emotional processes of reading experience and evaluation. Instead, these comments reflect only the elements of the reading experience that commenters are consciously aware of and chose to comment on, in the way they chose or were led by discursive norms to represent these elements textually. Second, commenters are only part of the users of AO3, and may not be representative of the reader reception happening all over the platform, let alone the fanfiction reception happening 1To find out more about this project, visit theAnchoring Innovation website. 22008 is the year that AO3 went into open beta. on other platforms. In a 2013 census of AO3, 43.6% of respondents reported commenting on stories they had read (n = 4,358)7[]. When MythFic Metadata was collected, 1,584 stories in the fandom Ancient Greek Religion and Lore, or 30.7%, of the fandom’s works, had not been commented on at all. Additionally, commenters can be characterized as above-averagely engaged or committed readers, since they invest the time and efort to comment. Finally, the content of fanfiction comments is co-shaped by the culture and afordances of AO3. Like with online book reviews, social factors like ”the interactive nature of the reviewing platform or reviewers’ desire to cultivate a persona or gain follower22s][” influence these texts. Additionally, fan communities difer between platforms because ”the relationship that fans have to technology and online platforms is integral to the culture of these communit1ie3s].”[ The exact dynamics of such relationships remain largely unknown and so the generalizability of our findings to other platforms may be limited.

2.2. Annotation

Of the 25,970 comments, 25,240 contain text and 730 contain only emojis. The emoji-only comments were discarded in our subsequent analysis. From the textual comments, we created an annotated subset using the following criteria: • Length: 100 - 4,000 characters3. • Language: English as predicted by the Python package langid25[] with high confidence (>0.9).

From the remaining 12,866 comments, we randomly sampled 1,000 to create the annotations.

Following existing research, we define sentiments as ”social constructs of emotions that develop over time and are enduring10[].” Since we conceptualize aspects here as particular features of a fanfiction story that elicit sentiments, it becomes possible to identify and computationally extract positive and negative sentiments as they relate to particular aspects of fanfiction reception.

To develop an annotation guide we conducted exploratory analysis of the comment data, such as an analysis of the most-frequent noun chunks as detected by SpaCy1[ 7 ] and a topic model of longer comments (>300 characters) created with Top2Ve1c].[Based on these explorations, we formulated a scheme of 8 aspects of fanfiction that were frequently commented on. Table2 provides examples of each aspect-category. Following the methodological principle of ethical fabrication 1[ 9 ] these examples were fabricated to protect the privacy of commenters while still mirroring as closely as possible the content of the data. Some aspects were chosen bottom-up, i.e. based on the exploratory analysis, and others were chosen top-down, i.e. based on aspects of comment-data that interest us.

The eight resulting aspect-categories are: • Canon: reference to how fanfiction transforms, critiques, or engages with canonical material, including references to Greek mythology but also source materials from popular culture. 3Shorter comments convey little to no evaluation, longer comments tend to go of-topic or quote extensively from the fanfiction. The length-limitation for writing comments on AO3 is 10,000 characters. • Character: reference to and assessment of character, characterization, character appearance and relationships between characters from the story. • Emotion: references to the emotions experienced by characters intradiegetically. • Events and Storyworld: reference to plot events and settings, specific scenes, world building, story content like tropes, and general plot elements like twists or endings. • General: reference to the story as a whole or in general terms. • Reading Experience: reference to reading experience, such as emotional engagement (of the reader, not the characters), absorption or narrative tension. • Style: reference to how the story was written down or rendered, including writing style, word choices, metaphors, turns of phrase, voice, perspective.

• NULL: expressions of sentiment that do not refer to specific words or aspects.

One central aspect of fanfiction comments we exclude is expressions that foster a bond with the author. Because fanfiction exchange relies on a gift economy [ 16 ], expressions of gratitude, appreciation and encouragement are an important part of commenting, and phrases like ”Thank you for writing this” or ”Please write more!” occur very frequently. An existing LDA topic model of fanfiction comments showed that ”author encouragements” and ”requests for story” were among the most prominent topics in comments on another fanfiction website, Fanifction.net [ 26 ]. However, because these types of expressions do not explicitly evaluate aspects of stories themselves, but rather relate to the status and position of authors and readers in the community, we disregard them here.

We used INCepTION [ 21 ] to annotate (see Figure1 for an example). The first author annotated all 1,000 comments in the annotation-set. One co-author annotated an overlapping 100 comments to determine inter-annotator agreement (Sectio4n.1).

3. Aspect-Based Sentiment Analysis

Aspect-based sentiment analysis (ABSA) is a type of information extraction (IE) task consisting of two subtasks: aspect extraction (AE) and sentiment analysis (SA) of the sentiments associated with these aspects. Approaches to these tasks in and beyond computational humanities include rule-based models22[], machine learning models3[ , 10, 36, 30 ] and, more recently, generative large language models12[ , 18, 8 ]. We selected a machine learning approach because (a) formulating domain-specific rules is time-consuming and would not generalize well over other datasets, and (b) while approaches using generative Large Language Models (LLMs) are currently in full development, the efects of their opaque nature, tendency to propagate bias and hallucinate are not yet fully understood in IE-applications for the literary domain [ 12 ]. However, exploring the use of LLM’s for ABSA presents an interesting avenue for future research.

Given the evaluative nature of the comment section on AO3, ABSA could be a good fit for assessing the sentiments of fanfiction readers on specific aspects of these texts. At the same time, the strong influence of subcultural community norms on the description of aspects and the expression of sentiments in these comments may make it challenging to apply an out-ofthe-box method.

4. Results

4.1. Evaluating the Annotations

Before training a model on our annotations, we conducted a small-scale inter-annotator agreement study of the 100 comments annotated by both annotators. Our inter-annotator agreement was exceptionally high: pairwise Cohen’ s of 0.86 (aspects) and 0.88 (sentiments). This is likely due to the intensive collaboration and discussion between the annotators on the annotation guidelines and because of our shared familiarity with the discursive and social norms of fanfiction communities. The only categories showing notable confusion werEevents and Storyworld and Character, with 9 instances of confusion between annotators. This is somewhat to be expected since characters often participate in story-events and always exist in the storyworld, so the distinction between these aspects can be blurry. Both annotators found a clear prevalence of positive evaluations (n=206 agreed occurrences in the annotation-set) of aspects over negative evaluations (n=4). This aligned with our expectations based on the exploratory data analysis and on a review of the existing literature, where research has shown that fanfiction comments are often written in a ”positive style [ 28 ].” model

F-score (micro) F-score (macro) accuracy

4.2. Results of the Aspect Extraction

We tested three models for joint aspect extraction and categorization: roBERTa-ba2se4][ (Macro F1 0.25), Twitter-roBERT-sentiment2[] (Macro F1 0.31), and NuNER [ 6 ] (Macro F1 0.45). Evaluation metrics for each of these models are listed in Ta5blaend per aspect in Table7. Since NuNER performed best, we tried to improve its results further by splitting the tasks of aspect extraction and categorization, to minimal efect (from 0.45 to 0.50 Macro F1).

Note that the numbers for support for split- and joint task approaches are diferent because in the split task, aspect extraction is a separate first step where some aspects may not have been detected. Additionally, the support-numbers in Tab7ledo not perfectly match the number of annotations reported in Tabl3e, because in a handful of instances the aspect-term in an annotated sentence proved difÏcult to locate due to variations in spelling, punctuation, capitalization or even spaces. model

4.3. Results of the Sentiment Analysis

We used the Twitter-roBERTa-sentiment model since it has already been fine-tuned for sentiment detection, so we were expecting it to perform well on a polarity detection task. These results were decent (0.75 macro F1) (Tabl6e). This may be because sentiment polarity does not deal with as many categories as the aspect-task (two instead of eight), and because the model has been trained on Twitter-data, which is in some ways similar to our dataset of fanfictioncomments. Both types of texts are relatively brief, use internet jargon and often convey strong sentiments or opinions.

5. Analysis

Based on these results, the afordances and limitations of ABSA for analyzing fanfiction comments seem strongly diferentiated by aspect category. As expected, the models perform best for those aspect categories that have high support in the annotations, especialGlyeneral and Style. Emotions are particularly problematic, perhaps due to their low support but perhaps also because emotions are not commonly considered entities or aspects in NER-tasks.

Figure 2 is a confusion matrix between the best-performing aspect-extraction model (NuNER) and our gold standard annotations. As you can see, the only significant source of confusion was betweenCharacter and Event & Storyworld aspects, which makes sense as characters usually participate in events and interact with story worlds. Entities that the model failed to detect were left out of Figure2 for the sake of interpretability. Manual inspection of these left-out results reveal that manyEvents were not detected.Emotion was also difÏcult for all models to detect, perhaps because words that refer to Ethme otion-aspect also often occur in descriptions ofReading Experience. Detection of Canon was often partial, but detecting these partial matches (often revolving around words like ‘retelling,’ ‘original’ ‘allusion’ or ‘portrayal’) is still useful in mapping on a large scale Choawnon is discussed in the comments. Characters that were very frequent in the data (such as Achilles) were more often accurately detected than less prominent characters (such as DemeterG).eneral aspects were captured well in most models, with the exception of some relatively long and unique figures of speech, such as ‘I take my hat of to you.’ Non-specific expressions of afect were also detected relatively well, including the kinds of borderline-incoherent expressions that are specific to fanfiction communities. Examples of these include ‘WAHHH’ and ‘HNNG’. NuNER even identified ‘skdjahskdjhsd’ as a NULL-aspect; correctly, in our opinion, as keyboardsmashes often convey strong but non-specific sentiments.

Figure3 is a confusion matrix between the two annotators, for whom thCeharacter and General aspects were easiest to agree on. With theGeneral aspect, then, the skills of hu

precision recall f1-score support Twitter-roBERTa-sentiment precision recall f1-score support roBERTa-base character general event NULL reading style canon emotion micro avg macro avg weighted avg character general event NULL reading style canon emotion micro avg macro avg weighted avg

NuNER character general event NULL reading style canon emotion micro avg macro avg weighted avg NuNER (split-task) micro avg macro avg weighted avg 0.33 0.72 0.08 0.31 0.16 0.48 0.21 0.00 0.36 0.28 0.36 0.35 0.72 0.08 0.41 0.19 0.46 0.26 0.09 0.39 0.32 0.39 0.45 0.78 0.42 0.58 0.32 0.74 0.33 0.27 0.53 0.49 0.53 0.55 0.55 0.55 precision recall f1-score support precision recall f1-score support man annotators overlap with those of NuNER. FoSrtyle, it seems that NuNER performed well because Style is often described in the comments using a relatively specific and limited vocabulary. In our test set, theStyle tokens contained only 30 unique words, compared to 53 for Emotion, 93 forCanon and 287 forEvents & Storyworld. The most common words forStyle are also quite consistent and content-related, such as ’written’, ’writing’, ’describe’, ’language’, and ’words. In comparison, the most common words detected in other aspect-categories are often stop-words. This may make Style relatively easy to detect computationally.

With NuNER’s help, we also found some small errors in our annotations, such as characternames that were overlooked in the part of the annotation-set that was only annotated by one author. In future work, an annotation efort where each text is annotated by multiple people may provide better results.

6. Discussion

The machine-learning approach to ABSA is more generalizable to other domains, especially other fandoms, than a rule-based model would be. Particularly in the relatively successful categories ofGeneral and Style, we expect no significant diferences between fandoms in how commenters refer to these aspects of fanfiction. We do expect, however, that our model will be difÏcult to generalize across diferent platforms for fanfiction exchange, as research has shown that ”our lack of knowledge about platform influence on norms of commenting makes it difÏcult to generalize about the dynamics of fanfiction commenting beyond AO3 2[ 8 ].” The importance of subcultural norms in expressing the sentiments of members of fanfiction communities like AO3 may also make it difÏcult to apply our ABSA pipeline to other types of comments, such as book reviews on Goodreads or comments on Youtube.

We want to highlight five potential directions for future research. Firstly, it may improve results to annotate more data in a targeted way, supplementing those aspects for which support was low. Secondly, it may be of interest to analyze the 730 comments containing only emojis, using, for example, the Multidimensional Lexicon of Emoj1is5][. Third, it may be interesting to tackle the task of ABSA with a generative LLM, since these models are gaining ground as information extractors. GoLLie3[ 2 ] looks especially promising, as this prompting framework would allow us to feed our existing detailed guidelines for annotation directly to a generative model. Fourth, a hybrid approach to ABSA, particularly adding some rule-based elements to our existing ML pipeline, may be a productive way to improve outcomes for some of the aspects that were difÏcult to detect using our current setup. For example, we could use an emotion lexicon like EmoLex2[ 7 ] to detect the Emotion aspect and create lists of characters and story settings based on existing metadata 2[ 9 ] to detect the Character aspect and the storyworld-dimension of thEevents and Storyworld aspect. Finally, a fruitful next step in the field of fandom studies would be to examine the outcomes of our current best model in more detail to generate domain-specific insight into the ways sentiment is expressed and attached to particular aspects of fanfiction in the comment data.

7. Conclusion

NuNER delivered the best performance at our aspect-extraction task. Although results were poor for some aspects (especiallEymotion but alsoEvents & Storyworld), other aspects (especiallyGeneral and Style) were more accurately detected. Furthermore, since the model detected many partial matches in the aspect extraction for aspects such aCsanon, results are still very useful to researchers wanting to know which aspects are mentioned in connection to positive and negative sentiments in the fanfiction comment data. Our approach of using Twitter-roBERTa-sentiment proved sufÏciently suitable to detecting positive and negative sentiment. Overall, fanfiction comments seem suited to ABSA because many of the texts explicitly express sentiments regarding specifically named aspects of fanfiction stories.

8. Data Access Statement

To protect the privacy of the fanfiction community, the dataset of fanfiction comments will not be made available for reuse. Derived data and code aarveailable on Githu.b

9. Acknowledgments

This research was conducted as part of a Transnational Access Fellowship at GhentCDH & Lt3 at Ghent University in the project Computational Literary Studies Infrastructure (CLSINFRA), which has received funding from the European Union’s Horizon 2020 research and innovation programme under grant agreement No. 101004984.

The projectAnchoring and Innovating Greek Myth in Contemporary Online Fanfiction is supported by the Dutch ministry of Education, Culture and Science (OCW) through the Dutch Research Council (NWO), as part of the Anchoring Innovation Gravitation Grant research agenda of OIKOS, the National Research School in Classical Studies, the Netherlands (project number 024.003.012).

References

[1] D. Angelov . Top2Vec: Distributed Representations of Topics . arXiv preprint. 2020 . doi: 10 .48550/arXiv. 2008 . 09470 .

[2] F. Barbieri , J. Camacho-Collados , L. Neves , and L. E. Anke . “ TweetEval: Unified Benchmark and Comparative Evaluation for Tweet Classification” . InC:oRR abs/ 2010 .12421 ( 2020 ). url: https://arxiv.org/abs/ 2010 .12421.

[3] M. Birjali , M. Kasri , and A. Beni-Hssane . “A comprehensive survey on sentiment analysis: Approaches, challenges and trends” . InK: nowledge-Based Systems 226 ( 2021 ), p. 107134 . doi: 10 .1016/j.knosys. 2021 . 107134 .

[4] S. R. Black . “ Adding a digital dimension to fan studies methodologies” . TInra:nsformative Works and Cultures 33 ( 2020 ). doi: 10 .3983/twc. 2020 . 1725 .

[5] C. Blevins and A. Robichaud2 . : A Brief History » Tooling Up for Digital Humanities . 2011 .

[6] S. Bogdanov , A. Constantin , T. Bernard , B. Crabbé , and E. BernarNdu.NER: Entity Recognition Encoder Pre-training via LLM-Annotated Data . arXiv preprint, arXiv:2402.15343 . 2024 .

[7] centreoftheselights. AO3 Census: Site Use . https://archiveofourown.org/works/16988355 /chapters/39932727. 2013 .

[8] J. Chun and K. Elkins . “eXplainable AI with GPT4 for story analysis and generation: A novel framework for diachronic sentiment analysis” . Int:ernational Journal of Digital Humanities 5.2 ( 2023 ), pp. 507 - 532 . doi: 10 .1007/s42803-023-00069-8.

[9] F. Coppa , ed. The Fanfiction Reader: Folk Tales for the Digital Age . Ann Arbor: University of Michigan Press, 2017 .

[10] G. D'Aniello , M. Gaeta , and I. La Rocca . “ KnowMIS-ABSA: an overview and a reference model for applications of sentiment analysis and aspect-based sentiment analysis” . In: Artificial Intelligence Review 55.7 ( 2022 ), pp. 5543 - 5574 . doi: 10 .1007/s10462-021-10134- 9.

[11] A. De Kosnik. Rogue Archives : digital cultural memory and media fandom . Cambridge: MIT Press, 2016 .

[12] T. Dejaeghere , E. Lefever, P. Singh , and J. Birkholz . “ Evaluating generative LLMs for named entity recognition in literary-historical texts”. PInro:ceedings of the 34th Meeting of Computational Linguistics in The Netherlands (CLIN 34) . Clin. Leiden , the Netherlands, forthcoming, forthcoming.

[13] C. Fiesler and B. Dym . “Moving Across Lands: Online Platform Migration in Fandom Communities” . In: Proceedings of the ACM on Human-Computer Interaction 4.Cscw1 ( 2020 ), pp. 1 - 25 . doi: 10 .1145/3392847.

[14] A. Flores .ao3_api. https://github.com/ArmindoFlores/ao3_ a . p2i022 .

[15] R. Godard and S. Holtzman . “ The Multidimensional Lexicon of Emojis: A New Tool to Assess the Emotional Content of Emojis” . InF:rontiers in Psychology 13 ( 2022 ). doi: 10.3 389/fpsyg. 2022 . 921388 .

[16] K. Hellekson . “A Fannish Field of Value: Online Fan Gift Culture” . InC:inema Journal 48.4 ( 2009 ), pp. 113 - 118 . url: https://www.jstor.org/stable/2561973.3 [17] M. Honnibal and I. Montani. spaCy 2: Natural language understanding with Bloom embeddings, convolutional neural networks and incremental parsing . url/https://zenodo.org/doi/10.5281/zenodo.1212303. 2023 .

[18] E. Hosseini-Asl , W. Liu, and C. XiongA. Generative Language Model for Few-shot AspectBased Sentiment Analysis . arXiv preprint . 2022 . doi1: 0 .48550/arXiv.2204.05356.

[19] B. Kelley . Loving fanfiction: exploring the role of emotion in online fandoms . London ; New York: Routledge, Taylor & Francis Group, 2021 .

[20] E. Kim and R. Klinger . “ Who Feels What and Why? Annotation of a Literature Corpus with Semantic Roles of Emotions” . InP:roceedings of the 27th International Conference on Computational Linguistics. Santa Fe , New Mexico, USA: Association for Computational Linguistics, 2018 , pp. 1345 - 1359 . url:https://aclanthology.org/C18-111. 4

[21] J.-C. Klie , M. Bugert , B. Boullosa , R. E. de Castilho, and I. Gurevych. “ The INCEpTION platform: Machine-assisted and knowledge-oriented interactive annotation”.PIrno:ceedings of the 27th international conference on computational linguistics: System demonstrations . Santa Fe , USA: Association for Computational Linguistics, 2018 , pp. 5 - 9 . urhl:ttp ://tubiblio.ulb.tu-darmstadt. de/10627 .0/ M. Koolen, J. Neugarten , and P. Boot . “' This book makes me happy and sad and I love it' A Rule-based Model for Extracting Reading Impact from English Book Reviews” . In: Journal of Computational Literary Studies 1 .1 ( 2022 ), pp. 1 - 22 . doi: 10 .48694/jcls.104.

[23] J. Kuhn and N. Reiter . A Plea for a Method-Driven Agenda in the Digital Humanities . Sydney, Australia: Adho, 2015 .

[24] Y. Liu , M. Ott , N. Goyal , J. Du , M. Joshi , D. Chen , O. Levy , M. Lewis , L. Zettlemoyer , and V. Stoyanov . Roberta: A robustly optimized bert pretraining approach . arXiv preprint, arXiv: 1907 .11692. 2019 .

M. Lui and M. Baldwin . “langid.py: An Of-the-shelf Language Identification Tool” . In: Proceedings of the ACL 2012 System Demonstrations. Jeju Island , Korea, 2012 , pp. 25 - 30 .

url: https://aclanthology.org/P12-300.5 [22] [25]

[26] S. Milli and D. Bamman . “Beyond Canonical Texts: A Computational Analysis of Fanfiction” . In:Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing . Austin, Texas: Association for Computational Linguistics, 2016 , pp. 2048 - 2053 . doi: 10 .18653/v1/ D16 -1218.

[27] S. M. Mohammad and P. D. Turney . “ Crowdsourcing a Word-Emotion Association Lexicon” . In: Computational Intelligence 29 .3 ( 2013 ), pp. 436 - 465 .

[28] J. Neugarten . “ Delicious Angst and tooth-rotting fluf: Distant reading community discourses of emotion in Harry Potter fanfiction comments” . InJ:ournal of Fandom Studies 11 . 2 & 3 ( 2023 ), pp. 205 - 28 . doi: https://doi.org/10.1386/jfs\_ 00082 \_ 1.

[29] J. Neugarten and R. Smeets .MythFic Metadata: Exploring Gendered Violence in Fanfiction about Greek Mythology . Radboud Data Repository . 2023 . doi1: 0 .34973/2mye- 8468 .

[30] D. Nguyen , M. Liakata , S. DeDeo, J. Eisenstein , D. Mimno , R. Tromble , and J. Winters . “ How We Do Things With Words: Analyzing Text as Social and Cultural Data” . FIrno:ntiers in Artificial Intelligence 3 ( 2020 ), p. 62 . doi: 10 .3389/frai. 2020 . 00062 .

[31] C. Rebaza . AO3 Celebrates 13 Million Fanworks . webpage. 2024 . url: https://www.transf ormativeworks.org/ao3-celebrates - 13 - million-fanwor.ks/ [32] [33] O. Sainz , I. Garcıá-Ferrero, R. Agerri , O. L. de Lacalle, G. Rigau, and E. Agirre. “ GoLLIE: Annotation Guidelines improve Zero-Shot Information-Extraction” . TInh:e Twelfth International Conference on Learning Representations. Vienna, Austria, 2024 , pp. 1 - 25 . url: https://openreview.net/forum?id= Y3wpuxd7u .9 N. Samutina . “ Emotional landscapes of reading: fan fiction in the context of contemporary reading practices” . InI:nternational Journal of Cultural Studies 20.3 ( 2017 ), pp. 253 - 269 . doi: 10 .1177/1367877916628238.

[34] T. Schmidt and M. Burghardt . “ An Evaluation of Lexicon-based Sentiment Analysis Techniques for the Plays of Gotthold Ephraim Lessing” . InPr:oceedings of the Second Joint SIGHUM Workshop on Computational Linguistics for Cultural Heritage , Social Sciences, Humanities and Literature. Ed. by B. Alex , S. Degaetano-Ortlieb , A. Feldman , A. Kazantseva , N. Reiter , and S. Szpakowicz. Santa Fe , New Mexico: Association for Computational Linguistics, 2018 , pp. 139 - 149 . url:https://aclanthology.org/W18-451. 6

[35] C. Tosenberger . “Mature Poets Steal: Children's Literature and the Unpublishability of Fanfiction” . In: Children's Literature Association Quarterly 39.1 ( 2014 ), pp. 4 - 27 . doi: 10 .1353/chq. 2014 . 0010 . url: http://muse.jhu.edu/content/crossref/journals/childrens%5 C%5Fliterature%5C%5Fassociation%5C%5Fquarterly/v039/39 .1.tosenberger.h.tml W. Zhang , X. Li , Y. Deng , L. Bing , and W. Lam . A Survey on Aspect-Based Sentiment Analysis: Tasks, Methods, and Challenges . arXiv preprint. 2022 . url:http://arxiv.org/abs /2203.01054.