=Paper=
{{Paper
|id=Vol-3878/57_main_long
|storemode=property
|title=Hits or Misses? A Linguistically Explainable Formula for Fanfiction Success
|pdfUrl=https://ceur-ws.org/Vol-3878/57_main_long.pdf
|volume=Vol-3878
|authors=Giulio Leonardi,Dominique Brunato,Felice Dell'orletta
|dblpUrl=https://dblp.org/rec/conf/clic-it/LeonardiBD24
}}
==Hits or Misses? A Linguistically Explainable Formula for Fanfiction Success==
https://ceur-ws.org/Vol-3878/57_main_long.pdf
Hits or Misses? A Linguistically Explainable Formula for
Fanfiction Success
Giulio Leonardi1,*,† , Dominique Brunato2,† and Felice Dell’Orletta2,†
1
University of Pisa
2
Istituto di Linguistica Computazionale “Antonio Zampolli”, ItaliaNLP Lab, Pisa
Abstract
This study presents a computational analysis of Italian fanfiction, aiming to construct an interpretable model of successful
writing within this emerging literary domain. Leveraging explicit features that capture both linguistic style and semantic
content, we demonstrate the feasibility of automatically predicting successful writing in fanfiction and we identify a set of
robust linguistic predictors that maintain their predictive power across diverse topics and time periods, offering insights into
the universal aspects of engaging storytelling. This approach not only enhances our understanding of fanfiction as a genre
but also offers potential applications in broader literary analysis and content creation.
Keywords
fanfiction, Italian corpus, success prediction, linguistic features, Explainable Boosting Machine
1. Introduction and Motivation machine learning offer a powerful lens for making ex-
plicit patterns that may explain the complex interplay
The growing proliferation of online literary content has between reader engagement and content success.
led to the emergence of new genres and storytelling This paper moves in this field and presents a computa-
forms, with fanfiction being particularly popular among tional analysis focused on Italian fanfiction, addressing
teens and young adults. Fanfiction consists of stories the following research questions: i.) Can the success of
created by fans (mostly hobby authors) that extend or Italian fanfiction be automatically predicted using stylis-
alter the narrative of existing popular media like books, tic and lexical features of the texts?; ii.) Which types of
movies, comics or games, and represents a significant features demonstrate the highest predictive capability,
portion of user-generated content on the web [1]. In re- and how consistent are these features across different
cent years, the widespread popularity that this genre has time periods and thematic domains?; iii.) To what ex-
assumed has prompted research into the linguistic and tent can these features be explained in terms of their
stylistic elements that contribute to its success, mirror- contribution to predicting success?
ing studies conducted on more traditional literary genres Our contributions. i.) We collected a corpus of Ital-
[2, 3, 4], among others. ian fanfiction stories enriched with metadata considered
Understanding the elements that contribute to narra- as proxies of their success; ii.) We investigate the relation-
tive success is a fascinating area of research with implica- ship between stylistic and lexical features of stories and
tions across various fields, from literary analysis to digital their success from a modeling perspective; iii.) We iden-
humanities. From a socio-linguistic perspective, it can tified the most influential features in success prediction,
offer deeper insights into people and culture. It also has showing the key role played by form and stylistic related
significant applications in areas such as personalized con- features across time and thematic domains of fanfictions.
tent recommendation and educational technology [5, 6]. The paper is structured as follows: Section 2 briefly
While personal interests undoubtedly play a crucial role contextualizes our study among relevant literature; Sec-
in predicting a reader’s engagement with a literary con- tion 3 presents the reference corpus of Italian fanfiction
tent, the way information is presented can also evoke stories that we collected; in Section 4 we provide an
different reactions and levels of interaction, ultimately overview of the approach we devised including the de-
influencing the narrative’s success. In this regards, recent scription of features used for classification and the classi-
advancements in Natural Language Processing (NLP) and fiers employed. Section 5 discusses the main findings and
offers a fine-grained analysis of the classification results
CLiC-it 2024: Tenth Italian Conference on Computational Linguistics, in terms of feature explainability. In Section 6 we sum-
Dec 04 — 06, 2024, Pisa, Italy
*
Corresponding author.
marize key findings and outlining promising directions
†
These authors contributed equally.
for future research in this field.
$ g.leonardi5@studenti.unipi.it (G. Leonardi);
dominique.brunato@ilc.cnr.it (D. Brunato);
felice.dellorletta@ilc.cnr.it (F. Dell’Orletta)
© 2024 Copyright for this paper by its authors. Use permitted under Creative Commons License
Attribution 4.0 International (CC BY 4.0).
CEUR
ceur-ws.org
Workshop ISSN 1613-0073
Proceedings
2. Related Work techniques is still limited. Mattei et al. [14] employ lin-
guistic profiling to analyze a corpus of Italian fanfiction
The exploration of online content and its engagement inspired by the Harry Potter series, with the purpose of
levels has increasingly benefited from advancements in identifying linguistic patterns associated with success.
NLP and machine learning. Different perspectives have Inspired by this previous study, our research aims to ex-
been touched upon considering different textual domains, tend these findings through a computational modeling
typology of linguistic features and quantitative metrics approach, investigating the power of linguistic features
to operationalize a very subjective concept like success. for predicting fanfiction success and their generalization
The study by Toubia and colleagues [7] explores how across different experimental settings.
the structure of narratives, particularly the internal se-
mantic progression measured by features derived from
dense word representations, affects the success of stories 3. Corpus Construction
across different text typologies (movies, TV shows, and
academic papers). Berger and colleagues [8] examine As a first step, we compiled a reference corpus of Ital-
how the linguistic structure of online content affects user ian fanfiction. To this end, we searched available texts
engagement, specifically by modeling sustainable atten- on efpfanfic.net, one of the largest Italian websites dedi-
tion. This concept goes beyond just attracting a reader cated to publishing and reading amateur stories, focusing
with a catchy headline or advertisement; it also encom- specifically on stories labeled in the fanfiction genre.
passes the likelihood that a reader will continue viewing Using a web scraping system, we extracted fanfictions
or reading the content. In their analysis of more than based on the Harry Potter series, a highly popular fandom
35,000 online contents from heterogeneous sources, they on the site, boasting 57,196 stories published between
emphasize the role of features related to processing ease 2003 and 2023. Figure 1 presents the temporal distribution
and emotional language. of these fanfictions up to 2020.
In the realm of literary works, Ashok et al. [2] first Additionally, we gathered a secondary corpus consist-
leverage stylometric analysis and machine learning tech- ing of 2,441 stories based on The Lord of the Rings series.
niques to predict the success of popular English novels This secondary corpus served as a test set to assess the
from the Gutenberg Project. Their approach demon- influence of thematic domains on the analysis of story
strated the potential of these techniques for assessing success.
literary success. Extending these findings, Maharajan For this study, we focused on the first chapter of each
et al. [9] proposed a multi-task approach to simultane- fanfiction to ensure a consistent analysis. While it is
ously evaluating success and genre prediction. Using widely recognized that thematic units within stories —
deep learning representations, in addition to hand-craft particularly the beginnings and endings — often differ
features related to topic, sentiment, writing style, and from the middle sections due to their distinct narrative
readability of books, they obtained better performance roles, we observed that the majority of stories (69%) con-
than the single success prediction task approach. Focus- sist of only a single chapter, making them effectively self-
ing on contemporary English-language literature, the contained. The efpfanfic portal allows users to review
study by Bizzoni and colleagues [10] investigate how per- each chapter with ratings marked as negative, neutral, or
ceived novel quality is influenced by a broad spectrum of positive. Consistent with prior research such as [9] we
textual features — such as those related to readability and used the absolute number of reviews to define the success
sentiment — and how these perceptions vary depending of a story, which we consider broadly as popularity. This
on the reader’s level of expertise. approach is based on the assumption that a high num-
The growing volume of online fanfiction has also been ber of interactions, regardless of their sentiment, reflects
the subject of numerous studies, either from the perspec- strong reader’s engagement. This is especially confirmed
tive of text mining by using NLP or through a qualita- since in our dataset negative reviews represent less than
tive lens via a manual examination. A comprehensive 1% of the total.
survey of analyses in this direction has been recently To formulate our success prediction task, we estab-
provided by [11]. For example, Milli and Bamman [12] lished a review threshold to classify each story as either
explore the relationship between fanfiction and its orig- a success or a failure. After analyzing the distribution of
inal canon, offering one of the first empirical analyses reviews for Harry Potter texts (Figure 2), we decided to
of this genre. Similarly, Sourati et al. [13] find that the exclude stories that fell in the middle of the distribution –
similarity between fanfictions and their original stories those that could not be clearly defined as successes or fail-
— particularly in terms of emotional arcs and character ures. Consequently, stories with fewer than two reviews
dynamics—correlates significantly with fanfiction’s pop- (25th percentile) were classified as failures, and those
ularity. with more than six reviews (75th percentile) as successes.
In the context of Italian fanfiction, research using NLP Stories within the interquartile range were excluded from
Table 1 success grounded on interpretable factors, we decided
Descriptive Statistics for the Harry Potter (HP) and Lord of to leverage explicit features modelling both style-related
The Rings (LOTR) Corpora and lexical aspects of text as input for the classification
system. To evaluate the effectiveness and robustness of
Corpus #texts #negatives #positives avg. #tok
these features, we conducted experiments across three
HP 26,032 13,058 12,974 1911 conceptually distinct scenarios to evaluate the ability to
LOTR 932 526 406 1946 discriminate success in different contexts. Specifically,
the first scenario is in-domain: the classifier is evaluated
on texts within the same thematic domain as the training
set, using 10-fold cross-validation on the HP corpus. The
second scenario is out-domain: the classifier is evalu-
ated on texts from a different thematic domain than the
training set. In this case, the HP corpus is used as the
training set, while the LOTR corpus serves as the test set.
Finally, in the cross-time scenario, the temporal im-
pact on classification is considered. The classifier is
trained solely on texts from the HP corpus published
in 2011 and sequentially tested on texts from each other
year from 2003 to 2020. The 2011 texts were chosen for
training because this year has the largest amount of data
Figure 1: Distribution of all fanfictions from the Harry Potter (3,755 texts), is approximately central within the tempo-
corpus by year of publication (up to 2020).
ral range [2003, 2020], and is particularly significant for
fanfiction production due to the release of the final film
in the Harry Potter saga.
The main components of our approach are detailed in
the following sections.
4.1. Success Predictors
A comprehensive set of features was extracted for each
story in the corpus. These features were categorized into
two primary groups: linguistic features, reflecting the
text’s linguistic style and structure and lexical features,
representing the semantic content of the text.
Figure 2: Distribution of published fanfiction from the Harry
Potter corpus by number of reviews in the first chapter. 4.1.1. Linguistic Features
To model text’s linguistic style and structure, we drew in-
spiration from the linguistic profiling framework, a NLP-
the analysis. We also excluded texts published after 2020, based methodology in which a large set of linguistically-
considering them too recent for meaningful comparison. motivated features automatically extracted from anno-
As summarized in Table 1, the final corpora, hereafter tated texts is used to obtain a vector-based representa-
abbreviated as HP (Harry Potter) and LOTR (The Lord of tion of it. Such representations can be then compared
the Rings), consist of 26,032 and 932 texts, respectively. across texts representative of different textual genres
and varieties to identify the peculiarities of each [15].
For our study, we relied on Profiling-UD1 , a multilin-
4. Methodology gual tool inspired by this framework, which extracts over
130 linguistic features from texts using the Universal De-
Based on the newly collected dataset and its internal
pendencies (UD) annotation formalism. As described in
distinction, we formulated the task of success prediction
Brunato et al. [16], these features encompass a range of
as a binary classification problem, that is: given a story,
linguistic phenomena that can be classified into distinct
the model is asked to predict whether it belongs to the
groups covering e.g. shallow text features (e.g. document
successful or unsuccessful class, where the two classes
and sentence length, average word length), distribution
were defined according to the metric based on the number
of grammatical categories, inflectional morphology and
of reviews received by readers.
In line with our main purpose to construct a model of 1 http://linguistic-profiling.italianlp.it/
syntactic properties related to local and global parse tree Table 2
depth structure. Classification Accuracy(%) of the Models. ‘Ling.’ and ‘Lex.’
These features have proven effective in tasks related refer respectively to models trained on linguistic and lexical
to modeling text form, such as assessing text complex- features. The baseline corresponds to the majority class label.
ity, and identifying stylistic traits of authors or author Scenario SVM Ling. EBM Ling. SVM Lex. Baseline
groups. Building on previous research on a similar cor-
pus of fanfiction [14], we hypothesize that these features in-domain 65.03 66.15 69.95 50.16
can also distinguish between successful and unsuccessful out-domain 59.22 64.70 43.45 56.43
avg. cross-time 62.02 62.81 49.31 49.20
fanfictions from a modeling perspective.
average 62.09 64.55 54.24 51.93
4.1.2. Lexical Features
The second representation employed is based on lexi- better accuracy compared to linear models. Additionally,
cal information and leverages the relative frequency of with a reasonable number of features, the model remains
n-grams in each document. The choice of n-grams, in explainable. Each shape function can be visualized as
contrast to more powerful semantic representation de- a two-dimensional plot, with the feature value on the
rived from embeddings, is deliberately motivated by the x-axis and the score assigned by the shape function on
desire to use lexical features that remain completely ex- the y-axis. A score greater than 0 indicates a contribution
plicit. The model, henceforth referred to as the Lexical towards the positive class, whereas a score less than 0
Model, consists of the following features: indicates a contribution towards the negative class. The
final prediction value for a record is simply the sum of
• Forms: unigrams, bigrams, and trigrams of to- the scores obtained from each shape function, potentially
kens. transformed by the link function. Beyond analyzing in-
• Lemmas: unigrams, bigrams, and trigrams of lem- dividual shape functions, the average contribution of
mas. each feature can be evaluated by taking the mean of the
• Characters: sequences of characters at the be- absolute values of the assigned scores.
ginning or end of words, ranging from 1 to 4 There are various algorithms within the family of
characters in length. GAMs, primarily distinguished by the method used to
fit the shape functions. In the case of the EBM, stan-
4.2. Classifiers dard gradient boosting is used. However, in each boost-
ing iteration, the algorithm sequentially cycles through
In line with our research questions, the explainability each feature, constructing each univariate shape function
of the classification is crucial to evaluate the impact of through bagged boosted trees. This method has proven
linguistic and lexical features on the prediction of suc- to be one of the most effective for training a GAM.
cess. Therefore, two classification algorithms that allow For our study, the EBM was employed exclusively for
for a precise global explanation of the predictions were experiments based on linguistic features due to the ex-
selected. cessive dimensionality of the lexical model. This high
The first classifier employed is a linear Support Vector dimensionality would have rendered the GAM too com-
Machine. By fitting a decision hyperplane in the feature plex to interpret and too time-expensive to train.
space, this method enables the examination of the hy-
perplane’s coefficients to assess the importance of the
features. 5. Results and Discussion
The second algorithm employed is the Explainable
Boosting Machine (EBM), which belongs to the family of The classification results are summarized in Table 2, for
Generalized Additive Models (GAMs). As explained in each model and scenario under evaluation.
[17] a GAM is a model of the form: For models using linguistic features, in the in-domain
scenario both the SVM and the EBM outperform the ma-
jority class baseline, with accuracies of 65.03% and 66.15%
∑︁
𝑔(𝑦) = 𝛽0 + 𝑓𝑛 (𝑥𝑛 ) (1)
respectively, compared to 50.16% for the baseline. This
where 𝑔(.) is called the link function, used to model indicates that both classifiers are effectively capturing
the output (e.g., the logistic function for classification). the linguistic patterns associated with success within the
Each 𝑓𝑛 (.) is referred to as a shape function, which is a same thematic domain.
univariate function modeling the relationship between For linguistic models, in the out-domain scenario the
the feature 𝑛 and the target. performance of the SVM drops significantly, with an ac-
The prediction is thus a sum of 𝑛 non-linear and arbi- curacy of 59.22%, whereas the EBM experiences a less
trarily complex shape functions, generally resulting in
tures. We provide an in-depth analysis of this model in
the following section.
5.1. The Model of Success
To gain a better understanding of the classification results
and identify the most influential features for predicting
success, we ranked the features according to the absolute
value of their weight in the EBM classifier model trained
on the entire training set. Table 3 presents an extract of
Figure 3: Classification Accuracy in the Cross-Time Setting
the top 15 features. The analysis reveals that, in addi-
tion to basic text features such as the average document
length (measured in tokens [1]) and the average word
drastic decline, achieving an accuracy of 64.70%. How- length (in characters [2]), more complex linguistic prop-
ever, both classifiers still perform better than the baseline, erties play a crucial role. Among these, features related
suggesting some degree of ability to generalize of the lin- to verbal predicates and verbal morphology emerge as
guistic features across different thematic domains. particularly influential. This suggests that the syntac-
The lexical model, in the in-domain scenario, achieves tic and morphological characteristics of verbs, such as
an accuracy of 69.56%, outperforming all models with lin- tense, mood and person, provide valuable information
guistic features, suggesting that lexical features provide for the classifier prediction, highlighting the importance
a more powerful representation for in-domain success of deeper linguistic structures in building a model of
prediction. Nevertheless, in the out-domain scenario, the successful writing.
lexical model does not surpass the baseline, indicating While this ranking highlights the ‘global’ importance
a complete lack of predictive ability. This suggests that of features, it does not explain their effect on classifica-
lexical features, which are primarily based on the content tion. For a more detailed analysis, Figure 4 in Appendix
of the specific fanfiction’s narrative universe, perform A highlights the threshold values for each of the top
well within the same thematic domain but lose all sig- 15 ranked features, indicating the point at which the
nificance outside of it. Conversely, linguistic features, expected classification shifts from one class to another.
which focus on the form of the text, appear to be more Additionally, it provides the number of instances in the
adaptable regardless of the theme. training set for each feature value. Interestingly, there
Figure 3 presents the performance over time for classi- are some features which split almost exactly the amount
fiers trained with linguistic features. Additionally, two of data into two subsets. For example, the features rep-
baselines are shown: "Random Choice", which randomly resenting word length (char_per_tok) has a discriminant
selects between the two classes, and "Maj. Class", which threshold of 4.55 characters which distinguishes success-
always assigns the majority class from the correspond- ful stories – typically with longer words – from unsuc-
ing training set (2011 stories), i.e. the positive one. The cessful ones – usually with shorter words. Similarly, fea-
results of the lexical model in the cross-time scenario tures related to the (morpho-)syntactic profile of the text
were insignificant, as they were very similar to the "Maj. such as the percentage of conjunctions (dep_dist_conj)
Class" baseline. The classifier, therefore, defaults to as- and non-finite verb forms (verbs_form_dist_Fin) show a
signing the negative class, demonstrating no predictive similar pattern. For these features, values lower than the
capability. To avoid confusion, the lexical model results discriminant threshold contribute to predicting the nega-
are not included in this Figure. In contrast, the cross- tive class, effectively splitting the data into two groups
time results for models using linguistic features are more with comparable densities. Regarding verb presence (ver-
meaningful: the results remain stable around an average bal_head_per_sentence), an increased use of verbs corre-
of 62%, regardless of the dominant class in the tested lates with the unsuccessful class. This finding contradicts
year and the classifier used (avg. cross-time in Table 2). the idea that higher readability, typically conveyed by a
The cross-time scenario further suggests that linguistic predominantly verbal prose rather than a nominal one,
features possess greater adaptability beyond their own is a good indicator of writing quality. However, it aligns
domain, maintaining a considerable degree of general- with observations by Ashok et al. [2], who identified
ization over time. Conversely, lexical features seem func- similar patterns in canonical literary novels.
tional only within the specific domain of the training set, Features related to verbal morphology also show a
losing all predictive power for texts from different do- peculiar trend. For instance, a complementary perspec-
mains. Overall the model that performed best on average tive emerges concerning the use of person morphology.
across the three scenarios, and with the least variance Increasing the use of second person plural beyond a rela-
in performance, is the EBM trained with linguistic fea- tively low threshold (0.4) positively affects the prediction
of success, which may indicate an alignment with the impact of the author’s popularity and productivity on
Reader-Insert2 format, a specific type of fanfiction where the success of their fanfiction.
the reader assumes the role of the protagonist, heavily
relying on second-person narration. In contrast, an ex-
cessive use of the first person plural is associated with References
the negative class.
[1] K. Hellekson, K. Busse, Fan fiction and fan com-
munities in the age of the internet: new essays,
Table 3 McFarland, 2014.
Top 15 Scores of the EBM Trained with Linguistic Features [2] V. G. Ashok, S. Feng, Y. Choi, Success with style:
# feature score Using writing style to predict the success of novels,
in: Proceedings of the 2013 conference on empirical
#1 n_tokens 0.121
methods in natural language processing, 2013, pp.
#2 char_per_tok 0.098
#3 verbal_root_perc 0.095
1753–1764.
#4 verbs_num_pers_dist_Plur+2 0.090 [3] J. Brottrager, A. Stahl, A. Arslan, U. Brandes,
#5 verbs_num_pers_dist_Plur+1 0.088 T. Weitin, Modeling and predicting literary recep-
#6 upos_dist_SYM 0.080 tion. a data-rich approach to literary historical re-
#7 n_sentences 0.077 ception, Journal of Computational Literary Studies
#8 aux_tense_dist_Imp 0.077 1 (2022). URL: https://doi.org/10.48694/jcls.95.
#9 verbs_tense_dist_Imp 0.072 [4] M. Algee-Hewitt, S. Allison, M. Gemma, R. Heuser,
#10 aux_tense_dist_Pres 0.067 F. Moretti, H. Walser, Canon/archive : large-scale
#11 verbal_head_per_sent 0.066 dynamics in the literary field, 2018. URL: https://
#12 dep_dist_conj 0.065
litlab.stanford.edu/LiteraryLabPamphlet11.pdf.
#13 tokens_per_sent 0.064
[5] Reviews matter: How distributed mentoring
#14 verbs_form_dist_Fin 0.053
#15 n_prepositional_chains 0.052 predicts lexical diversity on fanfiction.net, 2018.
URL: https://api.semanticscholar.org/CorpusID:
265096028.
[6] S. Sauro, Fan fiction and informal language learning,
6. Conclusion The handbook of informal language learning (2019)
139–151.
Understanding success factors in literary writing is an [7] O. Toubia, J. A. Berger, J. Eliashberg, How quan-
evolving area of cross-disciplinary research. This study tifying the shape of stories predicts their success,
on Italian fanfiction demonstrated the feasibility of pre- Proceedings of the National Academy of Sciences of
dicting success using computational methods and ex- the United States of America 118 (2021). URL: https:
plainability techniques. Notably, we found that features //api.semanticscholar.org/CorpusID:235648521.
related to style and structure of texts show greater ro- [8] J. A. Berger, W. W. Moe, D. A. Schweidel, What
bustness than lexical ones across different domains and holds attention? linguistic drivers of engagement,
time periods. This suggests that the way a story is crafted Journal of Marketing 87 (2023) 793 – 809. URL: https:
may be more universally appealing than specific word //api.semanticscholar.org/CorpusID:255250393.
choices or thematic elements. [9] S. Maharjan, J. Arevalo, M. Montes, F. A. González,
We believe that the implications of this study extend T. Solorio, A multi-task approach to predict lika-
far beyond fanfiction research. On the one hand, it pro- bility of books, in: Proceedings of the 15th Confer-
vides new methodologies for analyzing online literary ence of the European Chapter of the Association for
phenomena offering potential contributions to digital hu- Computational Linguistics: Volume 1, Long Papers,
manities. From the NLP perspective, it could inform text 2017, pp. 1217–1227.
generation models, potentially guiding the creation of [10] Y. Bizzoni, P. F. Moreira, I. M. S. Lassen, M. R. Thom-
content that resonates more effectively with readers. sen, K. Nielbo, A matter of perspective: Build-
Future research could explore the generalizability of ing a multi-perspective annotated dataset for the
these findings to other languages and genres, as well study of literary quality, in: N. Calzolari, M.-Y.
as the investigation on the dynamics of evolving reader Kan, V. Hoste, A. Lenci, S. Sakti, N. Xue (Eds.),
preferences over time by also considering alternative Proceedings of the 2024 Joint International Con-
measures to gauge success. Additionally, this study does ference on Computational Linguistics, Language
not take into account the importance of the author; a Resources and Evaluation (LREC-COLING 2024),
potential future development would be to consider the ELRA and ICCL, Torino, Italia, 2024, pp. 789–800.
2
URL: https://aclanthology.org/2024.lrec-main.71.
https://fanlore.org/wiki/Reader-Insert
[11] D. Nguyen, S. Zigmond, S. Glassco, B. Tran, P. J. A. Top 15 Features of the EBM
Giabbanelli, Big data meets storytelling: using ma-
chine learning to predict popular fanfiction, Social
Network Analysis and Mining 14 (2024) 58.
[12] S. Milli, D. Bamman, Beyond canonical texts: A com-
putational analysis of fanfiction, in: J. Su, K. Duh,
X. Carreras (Eds.), Proceedings of the 2016 Confer-
ence on Empirical Methods in Natural Language
Processing, Association for Computational Lin-
guistics, Austin, Texas, 2016, pp. 2048–2053. URL:
https://aclanthology.org/D16-1218. doi:10.18653/
v1/D16-1218.
[13] Z. Sourati Hassan Zadeh, N. Sabri, H. Chamani,
B. Bahrak, Quantitative analysis of fanfictions’ pop-
ularity, Social Network Analysis and Mining 12
(2022) 42.
[14] A. Mattei, D. Brunato, F. Dell’Orletta, The style of
a successful story: a computational study on the
fanfiction genre, in: J. Monti, F. Dell’Orletta, F. Tam-
burini (Eds.), Proceedings of the Seventh Italian
Conference on Computational Linguistics, CLiC-it
2020, Bologna, Italy, March 1-3, 2021, volume 2769
of CEUR Workshop Proceedings, CEUR-WS.org, 2020.
URL: https://ceur-ws.org/Vol-2769/paper_52.pdf.
[15] H. van Halteren, Linguistic profiling for authorship
recognition and verification, in: Proceedings of the
42nd Annual Meeting of the Association for Com-
putational Linguistics (ACL-04), Barcelona, Spain,
2004, pp. 199–206. URL: https://aclanthology.org/
P04-1026. doi:10.3115/1218955.1218981.
[16] D. Brunato, A. Cimino, F. Dell’Orletta, G. Venturi,
S. Montemagni, Profiling-UD: a tool for linguis-
tic profiling of texts, in: N. Calzolari, F. Béchet,
P. Blache, K. Choukri, C. Cieri, T. Declerck, S. Goggi,
H. Isahara, B. Maegaard, J. Mariani, H. Mazo,
A. Moreno, J. Odijk, S. Piperidis (Eds.), Proceedings
of the Twelfth Language Resources and Evaluation
Conference, European Language Resources Associ-
ation, Marseille, France, 2020, pp. 7145–7151. URL:
https://aclanthology.org/2020.lrec-1.883.
[17] Y. Lou, R. Caruana, J. Gehrke, Intelligible mod-
els for classification and regression, Proceedings
of the ACM SIGKDD International Conference
on Knowledge Discovery and Data Mining (2012).
doi:10.1145/2339530.2339556.
Figure 4: Visualization of the Shape Functions of the Top 15 Linguistic Features of the EBM. In each graph pair, the x-axis
represents the feature value, the y-axis of the line plot indicates the score assigned by the shape function, and the marked
threshold value denotes the feature value at the zero score point. For the features represented by absolute numbers (i.e.
n_tokens, char_per_tok, n_sentences, and n_prepositional_chains), the values are displayed as raw counts. For the remaining
features, which are expressed as percentage distributions, the values are shown accordingly. More details about how these
features are calculated are reported in [16].