This paper presents the outcomes of a study that leverages emotion annotation to investigate the narrative dynamics in novels. We use two lexicon-based models, VADER sentiment annotation and a novel annotation of 8 primary NRC emotions, comparing them in terms of overlaps and assessing the dynamics of the sentiment and emotional arcs resulting from these two approaches. Our results indicate that whereas the simple valence annotation does not capture the intricate nature of narrative emotions, the two types of narrative pro昀椀ling exhibit evident correlations. Additionally, we manually annotate selected emotion arcs to comprehensively analyse the resource.
approaches – the 昀椀rst one-dimensional, rule- and sentence-based, and the second multidimensional, embedding- and paragraph-based – a convergence of their results on a large-scale literary corpus would help in gaining an understanding of both methods’ reliability as more or less diverging methods for studying literary texts. A radical divergence, instead, could indicate that the complexity of literary language is too high for relatively simplistic analyses and that more sophisticated methods are needed.
To capture meaningful aspects of the reading experience, previous work has tested the potential
of sentiment analysis [
Approaches toemotion annotation are predominantly based on the theory of universal
emotions by Ekman [
We use the “Chicago Corpus”, which spans 9,089 novels published in the US (1880-2000), and is
a unique corpus both in terms of siz1e,and diversity. It was compiled based on the number of
libraries holding each novel with preference for very circulated works. It is not homogeneous
1Studies on literary quality o昀琀en rely on corpora <of1,000 books [
For annotation of emotion intensities, we use the NRC A昀ect Intensity Lexicon [
To annotate for valence, we chose a simple lexicon-based approach, usiVnAgDER at the
sentence level [
We compare the emotion-based annotation of the novels with their simple annotation of
VADER valence, using two central representations of the texts: their overall average emotional
and sentimental intensity and the dynamics of emotion and sentiment arcs as estimated by the
Hurst exponent (indicating arc persistence), as going beyond the sentiment or emotion
intensities of novels to look at arc dynamics allows us to compare the similarities and strengths of
each approach.
2Other studies have used the corpus5[
The initial inquiry focuses on the distribution of emotion intensities within the corpus and its relationship with the overall VADER valence distribution. By examining the average emotional intensity of the novels, we examine predominant tendencies of the corpus. Moreover, by comparing the distributions of the emotions as well as the correlation between mean emotion intensities and mean valence of novels, we see whether valence subsumes emotions. If there was no correlation between novels’ mean values in emotions and mean valence, it may mean that emotion-based annotation and sentiment annotation capture di昀erent text facets. Similarly, a strong correlation between all emotion values and the valence of texts may indicate that emotion-based annotation does not contribute much beyond that which a valence-based approach captures.
In addition to assessing the average emotion, we assess the dynamics of emotion and sentiment
arcs in the narratives, which have recently appeared promising in modeling reader
appreciation [
We observe the distribution of the level of persistence in emotion and sentiment arcs using both emotion-based and sentiment-based annotations. If the two resources returned radically di昀erent Hurst exponents, it would mean that the patterns elicited by a simple analysis for valence are very di昀erent from those elicited by emotion analysis. In other words, the “composition” of all the emotions in one single dimension gives way to dynamics that are di昀erent from the patterns of any individual emotion. In contrast, overlap between the Hurst distribution of the sentiment arcs and that of the emotion arcs would indicate comparability of the two annotations. It could give insight into which emotions are drawing the overall sentiment Hurst of the corpus towards higher versus lower exponents. Emotion Valence Joy Trust Anger Fear Sadness Disgust Surprise Anticipation
Mean 0.031 6.126 7.352 3.536 4.737 4.336 2.278 2.927 5.124 0.039 1.598 1.302 0.921 1.321 0.907 0.559 0.474 0.824
As ranges of VADER’s valences and emotion intensities are not the same (-1 to 1 vs. 0 to 20) we cannot directly compare the two sets of distributions, but observe their behavior considering their own means.
First, applying VADER uncovers a subtle positivity-bias in our corpus, into which the
distribution of speci昀椀c emotions provides further insight. The emotiontsrust and anticipation might
contribute to the right-skewed distribution, indicating a prevalence of positive emotional
expressions, while joy appears to have a long right tail. Positive emotions may pull the skew
towards higher values, which aligns with the prominence of positive valence in our corpus.
Moreover, negative emotions tend to cluster towards moderately lower intensity levels, and
the mean of emotions like disgust and anger have much lower mean values than more positive
emotions like joy and trust (Tabl1e).The positive skew may be related to literary texts
having high positive emotional content. Yet, it is essential to acknowledge that linguistic factors
may in昀氀uence this bias, as languages can exhibit inherent positivity [
When directly correlating emotions with valence, we 昀椀nd the most signi昀椀cant negative correlation between valence and fear. This correlation suggests that VADER assigns lower sentiment scores to novels with higher intensities of fear, aligning with the expectation that fear, a negative emotion, would exhibit a stronger negative correlation with the overall sentiment analysis (Fig. 3). The opposite is true for joy, and most other emotions correlate with the corpus’ overall valence in an expected way: trust and anticipation are positively correlated with valence, while anger, disgust, and sadness are negatively correlated (F4ig).. These 昀椀ndings indicate a convergence between the sentiment analysis and emotion analysis. However, certain complex emotions, such as anticipation, trust and surprise, exhibit lower correlations with valence (Table1), suggesting divergence from what is captured the VADER annotation. These emotions are less intuitively and clear-cut positive or negative, which may contribute to the weaker correlation, why they may not be adequately captured via valences.
The analysis of the Hurst exponent of sentiment and emotion arcs reveals clusters in similar areas, suggesting a degree of interrelation between the Hurst exponent of sentiment and emotion arcs (Fig. 5). The overlap indicates that using emotion pro昀椀ling enables a nuanced understanding of the development of di昀erent emotional tones within the narratives, where VADER provides a more generalized and less transparent representation of the novels’ “prismatic” internal dynamics. Looking at the distribution of Hurst exponent of titles in our corpus, we 昀椀nd that, tendentiously, surprise, anticipation and disgust cluster slightly below or at the distribution of the Hurst based on valence (“Hurst” in Fi5g)., while the remaining emotions cluster slightly above it. All distributions of Hurst based on emotion arcs are signi昀椀cantly smoother with longer tails than that based on valence.
Speci昀椀cally, we see a tendency of the Hurst based on surprise being slightly lower and exhibiting a slightly di昀erent distribution than, e.g., fear and joy, at an average 0.57 (Tab1le,Fig. 5). Note that the standard deviation is high (0.07), so that this di昀erence should be regarded a tendency only, but may align with our intuition that it is easier to envision more progressive and linear increases or decreases in, e.g., joy than in surprise values – an emotion that may peak “surprisingly”.
In sum, the dynamics of each individual emotion arc may o昀er a complex picture of a novel’s progression, while the Hurst exponent of valence-based arcs o昀ers a general outline that is more correlated to the Hurst of more clear-cut emotions like fear and joy 6(F).ig.
To further assess the reliability of our emotion-based annotation, we inspected arcs of novels that showed strong values of HurstT:he Old Man and the Sea by Ernest Hemingway, which has one of the lowest Hurst exponents for joy in our corpus, as wellAaPsortrait of the Artist as a Young Man by James Joyce, which has one of the highest Hurst exponents for fear in our corpus. Fig. 7 shows our manual annotation of the correspondence of narrative events with emotion arcs in Hemingway’sThe Old Man and the Sea. Note that peaks in fear and joy seem to co-occur in this novel. While this may appear puzzling, our inspection con昀椀rms that this co-occurence of positive and negative emotions actually illustrates a central characteristic of Hemingway’s prose style and the story overall: even in moments of crisis, Hemingway’s protagonist continues to re昀氀ect on his love for the sea and on natural beauty, leading to complex feelings and contradictory emotional intensities in key scenes – love and hatred, fear and admiration (see, i.a., box 7 in Fig.7). Such complexity is also re昀氀ected in the protagonist’s character: his hardships and endurance, but essentially optimistic outlook on life.
Emotion arcs in James Joyce’sA Portrait of the Artist as a Young Man parallel Jockers22[] valence-based arc of the same novel through the Syuzet package and human annotation, which has been called a “man in the hole” shape, with one central cris3is.
Here, only predominantly negative emotions are elevated in the main rise, and intensities
of joy and fear do not co-occur as in Hemingway’s prose (cf. Fi7g).. The more independently
developing arcs are re昀氀ected in more varied Hurst exponents in thPeortrait, which is among
the top 50 books in our corpus with the highest standard deviation between their emotion
arcs’ Hurst exponents. In thePortrait, the Hurst of the negative emotions anger, fear, and
sadness is > than 0.9, while the Hurst of anticipation, joy, and trust hovers around 0.8. The
three negative emotions exhibit a clear and steady rise and decline around the central crisis.
As the Hurst exponent measures persistence, i.e., whether increases are followed by increases
or decreases by decreases, a very high Hurst exponent here adequately indicates the slow rise
and decline of negative emotions. Arcs of positive emotions are less persistent but still exhibit
more persistence than, for example, the arc of surprise, which here has a Hurst exponent of
0.58 and appears mean-reverting (cf. pink line in Fig8.). Overall, the Hurst based on valence
for the Portrait is 0.71, a value that may represent the average dynamics of various emotion
arcs, and which does not capture the subtle but distinct di昀erence between trends in positive
and negative emotion arcs in the novel.
3Cf. Note that emotion arcs do not appear to show the ringing artifacts, the arti昀椀cial positive trend in the beginning
in Jockers [20], connected to Syuzet’s low-pass 昀椀lter (cf. Swa昀ord [
Our analysis of the distribution of emotion intensities and Hurst exponents based on emotionand sentiment arcs suggested that more valences subsume diverse emotional interactions. Some emotions are expectedly correlated to valence, while less clear-cut emotions like surprise seem to be less captured by valence annotation, and the Hurst of their arcs less correlated to that of the valence-based arc. Moreover, some emotion arcs, like surprise, are on average less persistent, and some, like fear, more persistent than valence-based arcs, which suggests that we may get a more nuanced understand the internal dynamics of novels, including progressions of emotions less clearly positive or negative, by analysing the Hurst of various emotion arcs.
Our inspection of individual titles suggested that the emotions and sentiments expressed in the text are not such that are explicitly felt by characters, nor transparently transmitted to readers. Rather, they are emotional textures narratives, from which the readers may derive complex (reading) experiences. The the co-occurrence of peaks in emotion arcs as seen in the case of Hemingway, as well as the di昀erence between the Hurst exponents of emotion arcs as seen in the case of Joyce is not trivial, as it tells us something important about plotting narrative arcs in general: namely, thastentiment valence does not stand in direct relation to plot and narrative events, but rather subsumes trends in emotion evocation that pertain to both style and events. In other words, when plotting arcs based on emotions we are observing trends in a narrative event-style continuum, that is less well captured by valence annotation. In the future, we suggest studying the dynamics of emotion- and sentiment-based arcs closely, seeking to assess arc dynamics at both a local and global level, as well as linking represented emotion to the actual reader experience and appreciation. [15] [16]