A narrative of relative complexity will often include events beyond simple facts that happen in the story world-such as plans, orders, wishes of the characters- or references to events that happen in a di erent story world-stories or anecdotes being told by the characters. These correspond to di erent modes of reporting events. Identifying spans of text corresponding to di erent modes of reporting is a signi cant challenge. The present paper proposes a mechanism for segmenting the text of synopses of narrative plots into spans that correspond to di erent views of the storyworld in terms of temporal, spatial or modal coordinates. This is achieved by considering syntactic structure to identify cues for the start of embedded discourses and continuity over features such as tense, voice or mode to identify the points where the embedded discourse ends. This process can handle embedded discourses that span over several sentences and recursive nesting of discourses. The solution is tested against a corpus of synopses hand-annotated for start and end of embedded discourses. Acceptable precision and recall metrics are reported.
The events included in a narrative do not always constitute a sequence of events that are equally true in the storworld in a steady succession. Sometimes the narrative refers to events that are not held to be true–but rather inform about wishes or plans of characters–or that were true at an earlier point in time. The start of the spans of text a ected by such instances are often indicated linguistically by means of speci c syntactic constructions, such as in nitive sentences, modal verbs or reported speech constructions. The end of the span of text that is a ected may be indicated by changes in tense or voice. These phenomena constitute an important challenge for the interpretation of narrative text, because even assuming a certain ability to construct a conceptual representation of each event from the given text, the interpreter is still faced with the need to decide in what mode–as a truth or as a wish–and with what temporal and spatial coordinates to add it to the representation being constructed for the storyworld.
Examples of this type of sentence are: “the girl orders the gendarmes to destroy the garden” or “The forecaster warns the weather may be cold.” These sentences describe a principal event–an order or a warning–which involve a secondary event–destroying the garden or the coming of cold weather. In each case, the meaning of the verb includes connotations about the distance between the secondary event and the principal event. Sometimes a sentence introduces an embedded discourse and then the reporting mode introduced by that embedded discourse continues over a number of sentences. Consider the following example:
A tsar has his queen and their son Ivan. The groom predicts [a sister will be born who shall be a terrible witch. The sister shall devour her father and mother and all people under their command.] Ivan asks the tsar for permission to go out for a walk.
The second sentence introduces a prophecy. The third sentence continues describing parts of the prophecy. The fourth sentence returns to the telling of the main story, and no longer refers to the prophecy. In terms of changes in reporting mode, the span of text corresponding to the prophecy should be identi ed as the one marked between square brackets. This is the type of task we want to address. We work on the hypothesis that such changes in mode of reporting may be agged by changes in the features of the clauses that determine the time/space coordinates for the event: tense, voice, mode. In the example, the events corresponding to the prophecy are all presented in future tense.
The present paper explores the viability of developing such a computational solution for
identifying spans of text corresponding to di erent modes of reporting events based on the
Stanford Core NLP tool for English [
The distinction between modes of reporting for a narrative that is considered in this paper is based on pragmatic criteria: it is intended to inform a process of representation of the information contained in the narrative, and it is concerned with clustering the events reported into sets that are attributed a similar degree of certainty in the same possible world.
From the point of view of linguistics, there are a wide range of phenomena that might
in uence this kind of classi cation/clustering task. The most relevant ismodality, de ned as
the way in which statements in a language may be marked in terms of their relation with reality
or truth [
Another relevant phenomenon is reported discourse, which appears when an agent reports
discourse originally contributed by a di erent agent, and which may itself be an utterance or a
belief [
Instances of reported discourse may be further characterised by their attribution relations
[
The work of [
A di erent approach addresses the task of segmenting plot synopses identifying turning
points in the narrative [
When characters in a story report events by themselves telling a story, each such telling
is considered to introduce a new narrative level. The task of annotating narrative levels over
long texts has been addressed by [
The procedure described in this paper relies on the constituency parse provided by the
Stanford Core NLP toolkit [
A faithful reconstruction of the meaning of a given story requires that the di erent modes of reporting be identi ed and correctly assigned to each and every one of the events mentioned. One possible way of achieving this is by identifying the spans of text corresponding to each reporting mode and the set of events mentioned in each span.
Because the aim of this research initiative is to explore the application of text processing tools to the task of identifying the meaning of a narrative in terms of the storyworld it describes, it was considered more practical to operate on synopses of larger narratives rather than on the narratives themselves. This is based on the assumption that synopses attempt to condense the structure of the plot for a given narrative, leaving out the details that add value to the work but not necessarily to the general structure.
To inform this process, a small corpus of plot synopses has been considered. This corpus
draws upon two di erent sources: the synopses of the set of plots for Russian folk tales originally
analysed by Vladimir Propp [
The statistics of the spans of reporting mode that appear in the manual annotation are presented in Table 1 for the Propp tales and in Table 2 for the D’Aulnoy tales.
The process of text segmentation needs to be informed by the following factors: the set of events reported by the text (often a single sentence in the text reports more than one event, each represented by a particular subclause), the syntactic structure of sentences whose verbs indicate a speci c reporting mode (such sentences are likely to imply a change in reporting mode between the event reported by the main clause and any reported in clauses subordinate to the main verb) and punctuation marks used to indicate relative groupings of subclauses (commas, brackets, or quotation marks).
An extraction procedure is proposed that builds a sequence of subclauses for each sentence– each corresponding to a subtree describing an event. We develop heuristics for identifying points in the narrative where the events being told di er signi cantly in the mode in which they are reported, and consider these points as candidates for breaks between spans for di erent reporting modes.
The di erent reporting modes that we have identi ed as being marked in this way are: events that occur in signi cantly di erentemporal coordinates ( ashbacks, ashforwards), events that are narrated in a di erentmodality (wishes, plans, prophecies, curses, orders), events that are speci cally marked as beingnarrated by particular characters (indirect speech, reported speech). 3.2.1. Sub-Clause Extraction A sentence with subordinate clauses will need to be broken down into its constituent parts. The relations between these parts are captured by assigning labels to identify subordinate clauses and adding these labels in argument positions in the main clause to represent the subordination relation.
The result of this process is a basic tree that represents the main clause as a simple structure with either noun phrases or subclause identi cation nodes as arguments. An example of this process is shown in Table 3.
A recursive search for subclauses is applied. Recursion allows consideration of subclauses that appear as adjuncts to noun phrases in other clauses. The type of subclauses that may appear nested within a noun phrase in the constituency trees produced by the Stanford parser are: relative clauses (“[The boy who was hungry] asked for a snack”, past participle sentences (“[The horses bought at the market] proved worthless”) and gerund sentences (“He found [a ring belonging to his mother]”). In each example, the part of the sentence corresponding to the subclause attached to the noun phrase has been marked in brackets. When extracting this type of subclauses, duplication of the subject may be required. An example of this process is shown in Table 4.
For each sentence, the set of subclauses extracted from it are ordered so that relations of embedding are presented in the same order as in the discourse: main clause followed by embedded clause. The procedure is applied in succession to the resulting sequence of subclauses.
For each clause identi ed in the manner described above, the system extracts the following information: voice, tense and main verb. The system also considers the following additional aspects that play a role on boundary identi cation: whether the clause includes a subordinated clause, whether the main verb of the clause has connotations of change in reporting mode, whether the clause includes a clause-grouping punctuation sign (opening or closing). Finally, given the recursive nature of embedded discourse, the system maintains stack data structures that allow it to keep track at any given point of: the depth of levels of quotation and the depth of levels of embedding.
At each point, a look ahead stage has been added to check whether: the following clause starts with quotation marks, or the following clause di ers in tense/voice/mode from the current one.
The heuristic that decides at each point of the discourse whether to consider the start of a span with a di erent reporting mode for events relies on the following conditions: whether the syntactic structure of the given clause includes embedded subclauses, whether the main verb of the clause is considered to change the mode of reporting–as de ned by a resource le that lists verbs of this type–and whether the clause start with opening quotation marks. The decision on the end of a span of discourse relies on the following conditions: whether the clause ahead involves changes in tense, voice and mood; whether the current clause already opened a span of reported discourse–which would make it a main clause and necessarily followed by at least one subordinate subclause– and whether closing quotation marks appear at the end of the subclause.
The described solution can be discussed from two points of view: in terms of recall and precision metrics of the outcomes of the solution with respect to the manually annotated reference corpus and in terms of how it compares with existing solutions for similar tasks.
To provide a quantitative evaluation of the quality of the results, we consider: precision and recall over positions of embedded story starts, precision and recall over positions of embedded story ends, and precision and recall over bags of words identi ed for speci c spans as annotated in the reference corpus. The current outputs for the Propp tales are shown in Table 5 and current outputs for the D’Aulnoy tales are shown in Table 6.
Results for the two di erent data sets are shown separately due to the signi cant di erences
that exist between them. The synopses for Propp tales are telegraphic transcriptions of the
formal analysis described in shorthand in Propp’s book. The synopses for D’Aulnoy’s tales
are more elaborate descriptions provided by Williams in her thesis [
Reasonably high (0.84 and 0.90) precision on starts means that most of the starts of embedded stories found by the parse are indeed starts that appear marked in the reference. The lower values (0.62 and 0.40) for recall on starts means that a certain proportion of the starts marked are not being located correctly. This suggests that there are further features that identify the start of these spans, beyond verbs that indicate changes in reporting mode, that would need to be considered. Reasonably high (0.84 and 0.85) precision on ends means that most of the ends of embedded stories found by the parse are indeed ends that appear marked in the reference. Low (0.62 and 0.37) recall on ends means that a large proportion of the ends marked are not being located correctly. However, it is important to note that, because the procedure treats the location of starts and ends separately, the values for location of ends will be considerably a ected by any errors in the identi cation of the corresponding starts. Relatively high values on precision (0.95 and 0.80) and recall (0.92 and 0.89) for the identi cation of words belonging to speci c spans suggest that, once the values for locating starts and ends are improved, the procedure would work well.
It is also important to consider that some of the incorrect diagnoses leading to low values in precision and recall may be tracked back to error rates in the parser. The results of the parser chosen for the task very often misrepresent the relations between the constituents of the sentence. For instance, the parses of sentences such as “the girl orders the gendarmes to destroy the garden” consider that “the gendarmes” as subject of the clause “to destroy” and not as direct object of the clause “orders”. These may explain why precision and recall for retrieving words for a speci c span do not reach maximal values. A solution enhanced with the results of a dependency parse will be considered as further work and should help improve results.
The fundamental challenge for the development of adequate machine learning solutions to tasks
of segmenting/classifying reported discourse in text has been identi ed as the lack of training
resources of appropriate size and coverage, both in the context of attribution relations [
As long as there are no data sets large enough to train machine learning solutions to the task, the proposed solution may provide a simple baseline that obtains results with acceptable values of precision and recall over basic features. Whereas the segmentations generated are clearly far from perfect, they can provide a valuable starting point for any e orts on developing corpora with the assistance of human annotators. At worst, they can be considered as a way to help create the seed data sets required for automatic augmentation via data synthesis.
A procedure for detecting spans of text that correspond to di erent modes of reporting events has been proposed. The proposed procedure operates directly on a syntactic parse of a story, without requiring any special depths of semantic analysis. It is also designed to operate on inputs based on their syntactic structure and a lexical resource that identi es verbs which indicate a change in reporting mode. These characteristics allow it to operate as a baseline solution for obtaining an initial segmentation in the absence of su cient volumes of annotated data to train machine learning solutions.
This paper has been partially supported by the CANTOR project (PID2019-108927RB-I00) funded by the Spanish Ministry of Science and Innovation.