A core aspect of human storytelling is the element of narrative time. In this paper, we propose a model of narrative revelation using the information-theoretic concept of relative entropy, which has been used in a variety of settings to understand textual similarity, along with methods in time-series analysis to model the properties of revelation over narrative time. Given a beginning state of no knowledge about a story (beyond paratextual clues) and an end state of full knowledge about a story's contents, what are the rhythms of dissemination through which we arrive at this 昀椀nal state? Using a dataset of over 2,700 books of contemporary English prose, we test for various time-dependent characteristics of narrative revelation against four stylistic categories of interest: audience age level, prestige, point-of-view, and 椀昀ctionality.
Italo Calvino was fond of quoting a Sicilian expression that “time takes no time in a stor8y]”. [
A narrator can tell a story that traverses centuries in a few sentences or can slow time down
to the point where a few seconds takes minutes to describe. Such manipulations of time – one
of the great loves of narrative theory28[
The fact that stories take time means that the dissemination of information – the ordering
and divulging of facts about the storyworld – plays an important role in the meaning of the
story. Independent ofwhat is told,how it is told is a key aspect of a story’s meaning.
Narrative theorists refer to this discrepancy as “discourse structur1e3”, [
Less attention has been paid to the more elementary question of the amount of novel
information imparted at any given moment in a story. Given a beginning state of no
knowledge about a story (beyond paratextual clues14[]) and an end state of full knowledge about
a story’s contents, what are the rhythms of dissemination through which we arrive at this
昀椀nal state? Does a narrator introduce more information early and then spend time going over
more familiar terrain or, conversely, withhold key pieces of information that we only learn
about towards the close of the story? Are there periods of local exploration, where narrators
spend more time introducing novel information, and periods of exploitation (to borrow a classic
framework from computer science), where narrators immerse audiences in already established
characters, themes, and situations? Do these practices exhibit predictable, periodic behavior
or are they more akin to random walks? Finally, how are such practices impacted by the social
situatedness of narrative 1[
In this paper, we draw on the 昀椀elds of information theory and statistics (including time
series analysis, in the latter case) to develop a model of narrative revelation to capture the
relative amount of new information communicated by authors over narrative time. We use the
information-theoretic concept of relative entropy or Kullback-Leibler divergence to quantify
how much new information is introduced in a given book during a window of text at a given
time , relative to the prior window at tim e− 1 . We then employ various techniques from
statistics and time series analysis to characterize the temporal dynamics of the resulting traces,
at both the aggregate level (across our corpus) and at the level of individual books. Relative
entropy has been applied to the study of textual di昀erence in numerous settings9][, including
parliamentary discourse2][ and the evolution of scienti昀椀c English [
We apply our measure of narrative revelation to the CONLIT datas2e4t][, which includes approximately2700 books from12 genres drawn from contemporary English prose published since 2001. We use available partitions in the data to test the relationship between patterns of narrative revelation and di昀erent social categories. In particular, we concentrate on the following categories in our analysis, including the relevant classes from the CONLIT data: 昀椀ctionality (昀椀ction / non-昀椀ction), prestige (prizewinning novels / bestsellers), age level (YA + Middle School / Adult Fiction), and point-of-view (昀椀rst person / third person). Note that all but the 椀昀rst condition on 昀椀ctional narratives.
Understanding the dynamics of narrative revelation can provide an important window into
the nature of human storytelling using computational methods. First, it can provide an
objective measure of informational novelty within texts, which can then be associated with reader
judgments. While beyond the scope of the present work, future work will want to explore
this relationship between the rate of novel information and readers’ a昀ective states. Such a
measure can also provide insights into the e昀ects that social settings have on the revelation of
narrative information, such as audience type or the narrator’s goals regarding the
instrumentality of information being communicated (facticity/昀椀ctionality), as well as potentially reveal
audience preferences for story structure when it comes to the distribution of new information.
In particular, it can give us the means to model what is known as the explore/exploit trade-o昀
when it comes to narrative communication3[
A number of approaches to the computational modeling of discourse structure have been
proposed. Schmidt [
Our work builds on this prior work in at least two important ways. First, we utilize a large
and diverse collection of publicly successful long narrative for2m4]s. [This overcomes
limitations surrounding prior work’s use of arti昀椀cially constructed corpo2r9a] o[r small
historical literary collection2s1[
Our reliance on Kullback-Leibler divergence as our principal measure of “information
revelation” also brings analytical a昀ordances. Prior work has shown its relevance for understanding
a variety of cultural domains (see Chang and DeDeo9][for an overview), including the study
of the novelty of parliamentary discours2e][,the evolution of scienti昀椀c English [
Our work is perhaps closest in spirit to that of Thompson, Wojtowicz, and DeDe3o3][and their conceptualization of the explore/exploit paradigm in a narrative setting, although it di昀ers in three key ways. First, their data derive primarily from time-ordered acts of speech (including from parliamentary and social media sources), rather than long narrative forms. Second, they use distributions derived from topic modeling within adjacent windows, while we use wordcount distributions when computing Kullback-Leibler divergence for adjacent time windows. Third, whereas they use random-walks based on Levy Flights to model their resulting timeindexed sequences of Kullback-Leibler divergences, with an eye speci昀椀cally towards capturing narrative (dis)continuity, our focus is on more fundamental properties of time-indexed data like average, trend, and, in particular, dependency structure, for which we use statistical regression and time series analysis.
We de昀椀ne narrative revelation as the practice of disseminating novel information over narrative time with respect to a local prior window of text. Given what has come immediately before, how surprising is any new passage? To capture this concept of surprise, we use KullbackLeibler divergence (KLD), which calculates the relative entropy (or divergence) between two probability distributions: (, ) = ∈ ∑ () log () () , (1) equal to zero, with equality holding if and onl y iafnd are equal. where is a (discrete) state space and and are probability mass functions de昀椀ned on , such that () = 0
implies() = 0 . Note that the quantity in (1) is always greater than or
For our purposes, the state space is time-varying, with de昀椀ned as the union of all words in the -th and ( −1) -st adjacent (non-overlapping) windows o1f000 words each. The function s and are estimated from the word frequencies in these two respective windows, using Laplace smoothing to avoid0 values. While conditioning on word frequencies limits the amount of semantic context that can be inferred from a given window of text, it has the advantage of observing the literal distribution of information over narrative time. The end result of this approach is a time series of KLD values, say{ } = 1 , capturing the extent to which information disseminated at time within a given narrative is novel compared to that disseminated just previously at tim e− 1 , with larger KLD corresponding to greater novelty. As su ch, is intended to capture the amount of new information disseminated over narrative time. We refer to these representations as the non-normalized time series. Figure 1 provides examples of this approach and the resulting values and behavior.
In order to test for an association between revelation and narrative time in aggregate, we also create normalized representations of for each book to control for di昀ering book lengths. To do so, we 昀椀rst subset all books into 50 equal parts, then subsamp1le000 words for each part, and then compute for the = 49 resulting pairs of adjacent windows. We refer to these representations as the normalized time series. As we discuss in Section 4.2 these serve as the basis of our regression analysis to better understand the linear tren ds of at the aggregate level.
Using these approaches, we formulate the following hypotheses:
H1. Average Rate of Revelation. We expect average revelation at the book level to vary by all of our measured social categories. Speci昀椀cally, prior wo2r9k] [has indicated that 昀椀ctional narratives are more predictable at the sentence level and thus we expect to see lower levels of average revelation with respect to 昀椀ctionality at the document level. We also expect average revelation to be negatively associated with reading level and positively associated with prestige (more information being more “di昀케cult” for readers to process and thus potentially more valued by elite readerships).
H2. The Slope of Revelation. We expect there to be an association between revelation
and narrative time. Prior theoretical work has suggested that narratives exhibit structural
patterns [
H3. Dependency Patterns of Revelation. Given assumptions about the value of narrative structure to narrative meaning, we expect there to be discernible dependency patterns to the rise and fall of revelation, with the present extent of revelation driven by that of the past in non-trivial ways (e.g., lagged dependency). While no prior work has suggested that narrative revelation should follow predictable dependency patterns, it could be the case that this is a latent structural feature to narrative plotting and potentially drives reader enjoyment.
We quantify the average revelation by calculating, for each book, the average of the values over times for our non-normalized time series, standardized to account for the considerable di昀erences in book length in the CONLIT data. To evaluate the support in our data for the speci昀椀c hypotheses with respect to our various two-level factors of social categories, we use two-sample -tests and report the results in the form of Cohen ’sas a measure of e昀ect size (see Table 1). We 昀椀nd that average revelation is associated with all social variables in our data set with the exception of point-of-view. The largest e昀ect size is reserved for the factor of instrumentality: non-昀椀ction books engage in higher rates of average information revelation over narrative time (see also Figure 2). Surprisingly, prestige as captured by prize-winning novels exhibit e昀ects almost as large as instrumentality and greater than those associated with reading level. This supports prior work that has shown signi昀椀cant stylistic di昀erences between prizewinning and bestselling novels25[] and adds a further dimension to understand the ways in which prestige-driven selection e昀ects prioritize distinctive stylistic traits.
A regression analysis was conducted to test the association of our narrative revelation variable with our narrative time variab le and 昀椀ctionality, as well as their interaction, here using the normalized time series. All e昀ects were found to be statistically signi昀椀cant (regression coe昀케cient -values <10−16). The association between the amount of narrative revelation and narrative time was small but negative (slope coe昀케cient−0.0038, in comparison to an intercept of 3.947), suggesting that as narratives progress, narrative revelation decreases slightly on average in our entire corpus.
Figure 2 provides the average KLD value for each narrative section by category along with the standard error. For the purposes of visualization, we show our 50-part model as described above as well as a 3-part model, where we divide each book’s KLD values for all windows into three equal-sized sections and take the average. As we can see for our 50-part model, 昀椀ction books had on average a lower intercep0t.(343 lower) and a steeper slope0.(002 steeper) than non-昀椀ction, indicating that 昀椀ctional books have lower overall levels of narrative revelation (as shown above in 4.1) and also a more pronounced decay in narrative revelation. We also note that for both categories we observe increases in average KLD in the 昀椀nal 1-2 sections of the 50-part model, suggesting that a common approach to narrative closure involves introducing increased levels of novel information toward the end (something we miss in the more generalized 3-part model). Such distinctive structure towards the close of narratives is considerably less pronounced however than the severity of decline of information revelation in the opening sections of a book. Finally, we found that youth 昀椀ction was similarly associated with greater decreases of revelation over narrative time, but that prestige and point-of-view were not.
The results in the previous two sections pertain to the behavior of the average and slope of narrative revelation in aggregate across books in the CONLIT data. Understanding the behavior of revleation at the level of individual books is also of substantial interest but requires a more nuanced analysis. The sequences{
} = 2 are time series, not only of varying lengths but also, as it turns out, of varying complexity.
Exploratory analysis of the non-normalized KLD time series reveals that, while they in general oscillate, they nevertheless do not typically have a dominant frequency (as determined using the findfrequency function of the R Forecast package), suggesting the absence of strictly periodic (and hence easily predicted) behavior of narrative revelation over narrative time. Further exploration of the autocorrelation behavior of the KLD time series suggests the use of ARIMA models. Such models are the workhorse of modern time series analysis and consist of three components: autoregressive (AR), integrated (I), and moving average (MA). The autoregressive component refers to behavior where the va l ue at time can be predicted by earlier value s −1 , … ,
− , for some lags, suggesting a regression-based relationship with itself. The moving average component allows for this regression-based relationship to have dependent errors, say over time scales of leng t.h And the integrated component allows for such combined AR-MA behavior to ride on top of a polynomial trend of o r,daekrin to the way a line with slope underlies a cloud of points within classical linear regression.
We used the auto.arima function in theR Forecast package to 昀椀t a separate ARIMA model to each KLD time series in the CONLIT data. This function includes data-driven selection of the triple(, , )
, which we take as the unit of primary interest in our analysis. Of th2e754 books analysed,59% exhibited a trend ( > 0 ). Of those, 80% exhibited downward trends (i.e. negative slopes). Non-昀椀ction books were 2x more likely to be in the positive slope class. For our second variable3,9% of all books exhibited autoregressive behavio r>(0 ), meaning that in a strong minority of books the successive values of narrative revelation are correlated. Within this group we see that 75% have 昀椀rst order dependencies (p=1) and another 20% have second-order (p=2), accounting for almost all books with auto-regressive behavior. Where there is a correlation between successive windows, it tends to reach only 1-2 windows back. Finally, = 0 was selected for all books in the data set, indicating that these characteristics of autoregression and/or trend can be viewed as occurring with a backdrop of white noise.
As in Section 4.1, we test the distribution of our two variables of interest, trend (d) and dependence (p) across our four social categories (Table 2). We report the percentage of books associated with each kind of time-dependent behavior for each category. As we can see from the breakdowns, there are only two scenarios where we observe meaningful di昀erences between categories (> 5% di昀erence among books). The 昀椀rst is at the level of dependence for Youth books, where we see 7% fewer books exhibiting auto-regressive behavior. This suggests that books targeting younger audiences skew in favor of less patterning and more consistency when it comes to narrative revelation. This is underscored by the fact that this e昀ect is even stronger for Middle School books compared to Young Adult books. The second notable difference is similar to what we observed in Figure 2. Fiction books are more likely to exhibit a detectable trend in the levels of revelation over narrative time, a trend which is overwhelmingly negative (downward).
Our work has aimed to continue prior e昀orts in modeling the temporal dimensions of narrative communication. Narratives have a fundamental temporal dimension that impacts their meaning. Accordingly, we have provided a novel method for capturing the dissemination of new information over narrative time as well as highlighted the utility of well-established statistical methods for capturing temporal relationships in time-series data. Our hope is that these frameworks can be applied towards the further study of computational narrative understanding to deepen our knowledge about the typicalities and particularities of human storytelling.
Our models support prior wor2k9,[ 26] in showing how 昀椀ctional narratives exhibit signi昀椀cantly more investment in patterns of narrative exploitation than narrative exploration. Fiction tends to engage in lower levels of narrative revelation overall and those levels decline more precipitously over narrative time. Fictional narratives invest more heavily in immersing readers in well-known information rather than continuously introducing novel information, an e昀ect that grows stronger over the course of a book’s narrative. Both 昀椀ction and non-昀椀ction exhibit a tendency to increase narrative revelation in the 昀椀nal closing sections of a book, suggesting a more universal narrative tendency with regards to narrative structu5]r.e [
When it comes to books targeting di昀erent reading audiences, we see that the intended age level of audiences and the selection preferences of elite audiences do appear to e昀ect levels of narrative revelation (and for younger audiences lower levels of temporal dependence). Authors engage in lower overall levels of revelation when writing for younger audiences, and higher levels when attempting to appeal to elite audiences.
One major open question for this line of research is the degree to which KLD covers the diverse ways that “narrative revelation” may instantiate itself. Changes in vocabulary distribution over narrative time that our models capture is one way of thinking about novel information in a narrative. But we can also imagine how new or surprising information could be encoded in very similar language but provides a key as yet unknown insight. The revelation of a murderer in a mystery is the most obvious example where a single name would provide very high levels of “revelation,” but low levels of KLD. The fact that our measures are inversely associated with audience reading levels suggest that narrative revelation as we are modeling it may be capturing the informationload and/or narrative complexity as much as the potential cognitive disposition of “surprise” on the part of readers. On the other hand, the trend towards increases in late-section rises of KLD that we are seeing suggests that our models may be capturing this idea of narrative revelation as a function of novel information as it relates to key plot points.
Similarly, because our models condition on local revelation – where the amount of novel
information is measured with respect to an immediate prior window – we cannot know if such
late-stage increases in revelation are absolutely novel or a return to information that references
earlier parts of the narrative (performing a sense of narrative “closure”). Future models could
explore the extent of revelation with respect to larger windows of text or even the entire text,
in essence capturing the absolute novelty of any new passage with respect to what a book has
divulged up to that point. Such work would also open the door to questions of non-linearity,
as when a passage refers back to a distant prior passage and continues the narrative a昀琀er some
interlude 2[
This research was generously supported by the Social Sciences and Humanities Research Council of Canada (895-2013-1011).