<!DOCTYPE article PUBLIC "-//NLM//DTD JATS (Z39.96) Journal Archiving and Interchange DTD v1.0 20120330//EN" "JATS-archivearticle1.dtd">
<article xmlns:xlink="http://www.w3.org/1999/xlink">
  <front>
    <journal-meta>
      <journal-title-group>
        <journal-title>J. L. L. Calle)
http://nil.fdi.ucm.es// (P. Gervás); http://nil.fdi.ucm.es// (J. L. L. Calle)</journal-title>
      </journal-title-group>
    </journal-meta>
    <article-meta>
      <title-group>
        <article-title>Representing Complex Relative Chronology Across Narrative Levels in Movie Plots</article-title>
      </title-group>
      <contrib-group>
        <contrib contrib-type="author">
          <string-name>Pablo Gervás</string-name>
          <xref ref-type="aff" rid="aff0">0</xref>
        </contrib>
        <contrib contrib-type="author">
          <string-name>José Luis López Calle</string-name>
          <xref ref-type="aff" rid="aff0">0</xref>
        </contrib>
        <aff id="aff0">
          <label>0</label>
          <institution>Facultad de Informática, Universidad Complutense de Madrid</institution>
          ,
          <addr-line>Madrid, 28040</addr-line>
          <country country="ES">Spain</country>
        </aff>
      </contrib-group>
      <pub-date>
        <year>2024</year>
      </pub-date>
      <volume>000</volume>
      <fpage>0</fpage>
      <lpage>0003</lpage>
      <abstract>
        <p>Movie narratives over the last few decades have increasingly featured very complex chronologies. These include a number of phenomena ranging from relatively simple use of ashbacks to complete alteration of the chronological order of events (Pulp Fiction) that sometimes force radical revision of the interpretation of the narrative (Memento). Computational treatment of this type of tropes requires a detailed understanding that will require temporal representation of the chronology of the narratives for these movies. The present paper explores the set of challenges that arise when the discourse for a given narrative departs from traditional ordered chronology, and proposes a model for representing the complexities involved. Narratives often convey at di erent points of its span a story -or more - inside the story that is unfolding. That embedded story might account for events already happened in the story being told, or be a di erent story, whether ctional or not, and belong to the world in which the story takes place, or to another world or time. In the same fashion that the embedded story could account for a past or present event, they can narrate a future event, that may or not actually happen. But that is precisely what occurs when we describe a plan, a wish, a promise, something likely to happen, or things that are or were conditioned to past, present or future events. . . All in all, the events mentioned in them are also told, and despite some may be expressed as a chance past and lost, it might still become true; others have as yet not happened, and may still come to be true, or not take place at all. The present paper proposes a representation of events in a story that captures the relevant features of these aspects.</p>
      </abstract>
      <kwd-group>
        <kwd>eol&gt;movie corpus</kwd>
        <kwd>narrative structure</kwd>
        <kwd>embedded stories</kwd>
        <kwd>modal discourses</kwd>
        <kwd>chronology</kwd>
      </kwd-group>
    </article-meta>
  </front>
  <body>
    <sec id="sec-1">
      <title>1. Introduction</title>
      <p>The plot lines of modern ctional movies often involve complex transitions in time that results
from presenting the narrative material of the plot in an order radically di erent from that in
which events took place. Examples such as Pulp Fiction or Memento have accustomed viewers
to dealing with plot storylines presented with very severe disruptions relative to the original
chronology of the story being told. Such breaks in chronology are sometimes just presented
seamlessly and left for the viewer to identify and sometimes presented as ashbacks initiated by
a character in the story telling other characters about events in the past. This is an occurrence
of embedded stories, which introduces di erent narrative levels: theframe story in which the
telling character appears, and the embedded story that the telling character tells. The situation
becomes even more di cult when such mechanisms are combined with further complexity
arising from stories that occur in more than one world, i.e, stories that combine characters in
the real world either travelling to or telling about a ctional world that is di erent from the
real one.</p>
      <p>The challenge of correctly representing the plot lines of this type of movie has to address a
number of aspects. First, a mechanism needs to be provided to represent relative chronology of
events along a particular timeline. Second, it needs to consider explicitly two separate timelines:
the one for when events are described in the discourse presented to the viewer and the one for
when events happen in the storyworld being described. Third, it needs to allow the explicit
representation of di erent worlds. Each di erent world will have its own timeline independent
of other worlds. Fourth, it needs to consider that di erent narrative levels are likely to involve
di erent timelines. These will either be timelines for a di erent world–if the embedded story
refers to a di erent world than the frame story–or for the same world but at a di erent time
relative to the point of the frame story in which the embedded story appears–if the embedded
story is for instance a ashback. Finally, if embedded discourses introduced by modal verbs
are involved–subclauses of a sentence describing possibilities, wishes, intentions, plans . . . –the
representation needs to distinguish between fragments of a particular world that are told to the
viewer as being true in the world and those that are not. When events are not being told as
true, the modality involved (wish, plan, intention. . . ) needs to be identi ed.</p>
      <p>The present paper describes a proposal for a representation that addresses these requirements.
This proposal arose in response to a need of a representation expressive enough to capture
the complexities of movie plots in annotations intended for a corpus of movie plots annotated
with elements relevant to narrative structure. The idea of building such a corpus came about
to inform a process of automated processing of narrative to build explicit representations of
narrative structure.</p>
    </sec>
    <sec id="sec-2">
      <title>2. Previous Work</title>
      <p>Four di erent topics need to be covered to provide background for the paper: the di erent
layers in narrative that may result in di erent chronologies for the same story, and existing
e orts to represent time, narrative levels and narrative structure.</p>
      <sec id="sec-2-1">
        <title>2.1. Layers within Narrative with Potentially Distinct Chronologies</title>
        <p>
          An important distinction to be made is the one between a sequence of events and a discourse
that orders and presents events, as suggested by [
          <xref ref-type="bibr" rid="ref1">1</xref>
          ]. Even then the actual assignment of speci c
terms to each these two concepts varies depending on the particular theorist. The sequence of
events told more or less alligns with what classical narratologists refer to as fabula or histoire, the
discourse that presents the events in a particular order similarly alligns with what is traditionally
referred to as sjuzhet or discours. To reduce the risk of confusion, we will refer in this paper to
the actual sequence of events as fabula and to the order of presentation of events in a given
telling as discourse. An important challenge to address will be to capture the di erence in
relative order of appearance of events between fabula and discourse.
        </p>
        <p>
          The challenge of identifying di erences in relative chronology for the events of the story
between the order in which they appear in the fabula and the order in which they appear in
the discourse has been identi ed by del Monte and Marchesini [
          <xref ref-type="bibr" rid="ref2">2</xref>
          ]. They propose a manual
method for identifying in the text elementary units of narrative structure, “which are processes
or event- sequences of clearly demarcated duration”, called narremes after Collier [
          <xref ref-type="bibr" rid="ref3">3</xref>
          ], and then
reorganizing these narremes chronologically into a representation of the fabula of a novel.
        </p>
        <p>
          Even though the basic mode of communication using language is to assume that statements
are intended to be true descriptions of the world, speakers may also refer to events by assigning
to them a di erent attitude with respect to their truth value. This di erent attitude towards
the relation between the given statement and reality or truth is usually marked explicitly by
modality [
          <xref ref-type="bibr" rid="ref4">4</xref>
          ]. Di erent languages address this task in di erent ways. In English, the most usual
means for expressing modality are either auxiliaries–such as may or can – or speci c verbs
that carry the corresponding meaning lexically – such as want or need. When attempting to
establish relative chronology between events in a discourse, events within the scope of a modal
statement must be considered speci cally.
        </p>
        <p>
          A similar situation arises with respect to reported discourse, where a speaker reports an
utterance or belief originally contributed by someone else [
          <xref ref-type="bibr" rid="ref5">5</xref>
          ]. This can take the form of direct
speech–the exact words of the reported agent are given verbatim–or as indirect speech–what
the other person said is paraphrased. These di erent forms are usually marked typographically.
In English, direct speech is marked by enclosing it between quotes, and indirect speech is
usually introduced by a verb of speech followed by complementizer that. Instances of reported
speech involve two di erent times: the time when the speech is reported and the time when
the reported speech took place.
        </p>
        <p>
          A related phenomenon is that of embedded stories. These occur when a speaker within a given
discourse tells a story as part of her discourse. In narratological terms, the original discourse
of the speaker acts is considered the frame story, and the story that she tells is known as the
embedded story [
          <xref ref-type="bibr" rid="ref6">6</xref>
          ]. Because similar situations may also take place within the story being told,
the phenomenon is recursive. Each time an embedded story is told, a new narrative level is
being introduced. Each di erent narrative level is likely to have a di erent chronology and may
even have a di erent modality, depending on whether the story is factual or ctional 7[].
        </p>
      </sec>
      <sec id="sec-2-2">
        <title>2.2. Annotating Narrative Levels</title>
        <p>
          The annotation of narrative levels was the subject of a workshop that established a set of
guidelines and an annotated corpus [
          <xref ref-type="bibr" rid="ref8">8</xref>
          ]. The set of guidelines was further re ned in [
          <xref ref-type="bibr" rid="ref9">9</xref>
          ]. These
guidelines opt for a system of identify narrative levels in terms of relative depth, with number
1 assigned to “a narrative that is not embedded in any other narrative”, and then numbering
successive embeddings with increasing numbers. In each case, the number only indicates a
similar depth of embedding, and it does not imply any type of relation between stories embedded
at a similar depth. The guidelines outline the di erence betweenstory time–actual series of
events that occur– and narrative time–series of events as presented. They rely on Genette
[
          <xref ref-type="bibr" rid="ref6">6</xref>
          ] for their de nition ofanachrony – “all forms of discordance between the two temporal
orders of story and narrative”. The guidelines are designed to annotate spans of text that
correspond to di erent narrative levels. The annotation begins “at the point in the text where
the narrative transitions either to a higher or lower level” and ends “when the phenomenon
ends and the narration returns to the level it had previously been at”. The guidelines indicate
that such changes may ocurr in the middle of a sentence and that the new narrative level may
carry on into another sentence, paragraph or even chapter. Changes in relative chronology
are only marked either as prolepsis–evoking in advance an event that will take place later–or
analepsis–evocation after the fact of an event that took place earlier.
        </p>
        <p>
          Further re nements of the annotation guidelines [
          <xref ref-type="bibr" rid="ref10">10</xref>
          ] argue that when a character’s thoughts
are presented vividly this may suggest an additional change in narrative level, in the sense that
a di erent world is being considered [
          <xref ref-type="bibr" rid="ref11">11</xref>
          ]. They argue that the same narrative level pertains,
and only the world being referred to changes.
        </p>
        <p>
          The need to represent separately di erent narrative levels when news articles include reported
speech is defended by Silvano et al. [
          <xref ref-type="bibr" rid="ref12">12</xref>
          ]. This proposal includes means for annotating the
instances of reported speech, and clustering the set of events so reported as connected by a
temporal link of type Identity. The authors argue that this solution addresses the problem that
news articles do not usually allow the establishment of an accurate relative temporal order
between the events reported.
        </p>
      </sec>
      <sec id="sec-2-3">
        <title>2.3. Representing Time</title>
        <p>
          It has long been accepted that narrative involves telling about events that happened in time
and that telling itself takes time [
          <xref ref-type="bibr" rid="ref13">13</xref>
          ]. In computational terms, the fundamental reference as
to how to represent time is Allen’s temporal algebra [
          <xref ref-type="bibr" rid="ref14">14</xref>
          ], which outlines the set of possible
temporal relations between events that themselves take a nite amount of time (duration) to
happen. Allen outlines that events are not just set at particular points on a timeline, but that
they should be considered as intervals that can also overlap (or not), be contiguous (or not),
or be contained within one another (or not). He proposes a set of mathematical relations to
describe all possible relations between such events.
        </p>
        <p>
          The speci c challenges of representing time in narrative are addressed by Mani [
          <xref ref-type="bibr" rid="ref15">15</xref>
          ]. Mani
argues that narrative relies on temporal abstractions that may involve considering time either
in vague terms, or with imprecise information or at di erent granularities. To deal with
these di culties, he proposes the concept of chronoscope, an underspeci ed representation
of ambiguity based on factoring out common elements. By relying on temporal abstractions
chronoscopes allow annotators to collapse temporal relations, or to zoom the representation to
di erent time granularities.
        </p>
        <p>
          An e ort to annotate texts with temporal information is TimeML [
          <xref ref-type="bibr" rid="ref16">16</xref>
          ]. The annotation
guidelines for TimeML provide de nitions for how to identify events–which may be introduced
by verbs, nominalizations, adjectives, predicative clauses or prepositional phrases–and how to
mark the corresponding spans of text with XML-type tags. Such types allow the introduction
of attributes, including an event ID. It also allows the introduction of link tags that serve to
annotate relations between events. Events are represented in a link tag by its ID. TLINKS are
speci c link tag intended to annotate temporal relations between events.
        </p>
        <p>
          The importance of annotating relative temporal information in narratives is highlighted by
Macovei [
          <xref ref-type="bibr" rid="ref17">17</xref>
          ]. Macovei postulates the concept of time tracks built out of time segments, which
are groups of events presented in order. The temporal relations between time tracks and time
segments can be used to analyse the chronology of stories. Macovei marks here time tracks and
time segments using TimeML. The goal of having time tracks is to allow the visualization of the
adventures of a particular character, separate from the rest of the story.
        </p>
        <p>
          The annotation of temporal relations over narrative texts is taken to the extreme in
NarrativeTime, the rst timeline-based framework for full temporal annotation [
          <xref ref-type="bibr" rid="ref18">18</xref>
          ]. NarrativeTime
builds a dynamic timeline which compiles and visualizes all the relative temporal information
contained in the full set of all possible TLINKS in the text. It also allows for branching of the
timeline to represent uncertainties in temporal information as reported in text.
        </p>
      </sec>
      <sec id="sec-2-4">
        <title>2.4. Representing Narrative Structure</title>
        <p>
          A related e ort is SceneML [
          <xref ref-type="bibr" rid="ref19">19</xref>
          ], a framework for annotating scenes in narratives. SceneML relies
on the concept of scene–a span of text that exhibit continuity of time, location and character–
as basic unit of annotation for narratives. SceneML allows for the explicit representation
of entities–scenes, scene descriptions segments (SDSs), characters, times and locations–and
relations–scene-scene narrative progression links, relational links connecting times, characters,
locations to the scenes in which they are participant elements and connecting SDSs to the
scenes they comprise. Narrative progression links are used to indicate relations between scenes
that are adjacent. Relations considered are: sequence–change of scene involves only character
or location–, analepsis– there is a ashback in one scene to a scene in the past–,prolepsis–
ashforward to a scene yet to come– andconcurrent – the transition takes us to di erent
characters and a di erent place but at the same time.
        </p>
      </sec>
    </sec>
    <sec id="sec-3">
      <title>3. A Proposal for Representing Complex Chronology in Movies</title>
      <p>The main contribution of this paper is a proposal for a method to represent the set of events
narrated in a story in such a way that it provides an informative representation of their relative
chronology. This representation is intended to allow the explicit capture the in uence on the
relative chronology of events of the aspects outlined in the introduction, namely, transitions
between di erent universes, di erent narrative levels, and di erent types of modality of reported
speech.</p>
      <p>In order to achieve this, the events narrated in the story are represented explicitly as action
units. An action unit is a representation of an event that includes the following attributes:
• Universe: the world in which the event takes place
• Narrative Level: the depth of narrative level at which the event is narrated (with respect
to the primary frame story, which is numbered as 0)
• Action Unit ID: a univocal identi er for the action unit
• Mode: identi es events that di er in the degree of truth assigned to them by the speaker
(captures issues such as modality)
• Relative Order: describes temporal relation of the action unit with other action units,
using Allen’s temporal algebra (number refers to each of the types of relations proposed
by Allen)</p>
      <p>• Content transcription: a number of elds that capture the informational content of the
action unit, including: type of action, subject of the action, direct object of the action,
indirect object of the action (if present), location of the action, and a set of action units
that may be referred to when the action involves a verb of speech</p>
      <p>Whenever verbs of speech are involved, the clause corresponding to the reporting of the
speech and the clause corresponding to the reported speech are encoded as separate action
units, and the reported clause is referred by its Action Unit ID as direct object of the reporting
clause. This allows both for the explicit notation of di erences between the two clauses in terms
of world, narrative level and chronology, and for the relation of embedding between them.</p>
      <p>The proposed scheme for representation is illustrated with examples from the lm “The
Princess Bride” (Rob Reiner, 1987). An example of this situation is shown in Figure 1. The scene
describes how the mother enters the room while the boy is playing video games and tells the boy
that his grandfather has arrived. The telling is divided into action unit 2 (the telling, an event
that actually happens) and action unit 4 (the information told, reference to grandfather’s arrival,
an event that has already happened). The representation for action unit 2 states explicitly that
action unit 4 is told as direct object of action 2 (encoded by referring to the Action Unit ID in
the Direct Object column). Because the event referred to by action unit 4 is mentioned rather
than presented as actually happening, a di erent narrative level is assigned to it (1 instead of 0).
The type of reference is indicated in the Mode column, to show that this event is mentioned
as something that has happened in the past. The temporal relation between action unit 2 and
action unit 4 is described (in terms of Allen’s relations) in the column for Relative Order, stating
that 4 precedes 2. This implies that 2 happens after 4 but not necessarily contiguously. This
type of relation allows us to capture a certain vagueness in the inferred temporal relations. The
entrance of grandfather into the room, which happens a moment later, is represented as action
5. The scene captures two separate streams of events: the grandfather’s arrival at the house
(action unit 4), and the stream of actions we actually see on the screen (1, 105, 106 and 5). Action
4 is assigned to narrative level 1 (event being told), whereas all the other actions are assigned
to narrative level 0. The relative temporal order within the stream of action units 1, 105, 106
and 5 is de ned asX meets Y, because each action unit does takes place immediately after the
preceding one in that stream. In this example, all the action units take place in the same story
world, so the value for the column Universe remains constant (A, referring to contemporary
USA).</p>
      <p>The representation of narrative levels involving a change of world can be shown in the
example in Figure 2. In this case the discourse alternates between events happening in the real
world (grandfather reading the story to the child, action units 7, 7a, 10 and 10a) and events
happening in the ctional world (the story of Wesley and Buttercup, action units 8, 9, 11 and
12). In this case, the event of reading particular segments of the book has been encoded as two
separate action units, one describing the physical act of reading from the book (action unit 7)
and one that describes the act of reading out aloud the events that the book tells (action unit
7a). Allen’s equals relation has been used to indicate the fact that these acts overlap in time.
The same considerations apply to action units 10 and 10a. It is important to note that in this
case both the world in which the action units happen and the narrative level change between
these two streams of events: action units 7 and 10 take place in universe A and narrative level
0, and action units 8, 9 , 11 and 12 take place in universe B and narrative level 1. The actual
nesting between reporting and reported events is represented explicitly to show that action
unit 7 involves the telling of action units 8 and 9 (action unit IDs 8,9 indicated as Direct Object
for action unit 7), and action unit 10 involves the telling of action units 11 and 12 (action unit
IDs 11,12 indicated as Direct Object for action unit 10).</p>
      <p>The temporal relations in this example are particularly interesting because they illustrate
the challenges that the task faces. The temporal relations shown between events that take
place in universe A show a continuous timeline: 7 is followed by 10. However, the temporal
relations shown shown between events that take place in universe B obey a separate timeline: 8
is followed by 9 which is followed by 11, which starts 12. Whenever the narrative involves a
change in universe, a separate timeline needs to be considered. Action unit 12 di ers from the
preceding ones in that it describes not an event of speci c duration but the setting in motion of
a process that will develop over time. For this reason its temporal relation is de ned as starting
after event 11. This use of Allen’s relation allows us to capture some of the underde ned nature
of relative temporal relations in narrative discourse.</p>
      <p>The representation of narrative levels that involve a longer story but not involving a change
of world can be shown in the example in Figure 3 In this case, Iñigo Montoya tells (action unit
43) the Man in Black about his father’s death (action units 45 to 51) before challenging the Man
in Black to a duel (action unit 52). All the action units happen in the same world (universe B),
but at di erent narrative levels. The story being told by the grandfather to the boy corresponds
to narrative level 1, and the story told by Iñigo Montoya to the Man in Black corresponds to
narrative level 2. The change of location between the two narrative levels is also represented:
narrative level 1 takes place in the ruins atop the cli , narrative level 2 takes place in the smithy
belonging to Iñigo’s father. In terms of chronology, the transcription indicates that the death of
Domingo Montoya precedes (in the ctional world) the rst action that we have been told of the
story (the rst event of the story, action unit 45 precedes action unit 8, Wesley’s rst interaction
with Buttercup). The relative limits of Iñígo’s telling of the story are represented by indicating
that the event immediately before the reporting starts the telling (action unit 43). and the event
immediately after the telling (the challenge that is action unit 52) nishes it. The set of action
unit IDs for the events in Iñigo’s story is indicated as direct object of the telling (action unit 43).</p>
      <p>A further example in Figure 4 shows how modal events are represented. In this scene,
following the kidnapping of Buttercup (action unit 26) Vizzini tells (action unit 27) his confederates
about this plan (action units 28-35). In this case a similar structure as in the prior example is
used to represent the telling. The kidnapping of Buttercup (action unit 26) starts the telling and
the departure on a ship (action 36) nishes it.</p>
      <p>However, in this case, the set of events told is presented as a plan, which implies a speci c
modality: this is not something that has happened but something that Vizzini intends to happen
in the future. This is indicated in the Mode column by assigning the plan value to these events.</p>
      <p>The play of narrative levels is more interesting here than in the preceding examples. Again,
the main reference here is narrative level 1 (the story the grandfather is telling the boy), and
Vizzini’s telling of this plan becomes narrative level 2. However, the plan itself involves two
further instances of nesting. These further nestings involve a certain modality. First, Vizzini
explains (as part of his telling in action unit 27) that Humperdinck will suspect (action unit 29)
that the Guilderians have kidnapped Buttercup (action unit 30). The relations between these
events are complex. Humperdinck’s suspicions (action unit 29) are only part of Vizzini’s plan.
The event that Humperdinck (may) suspect (action unit 30) also carries a certain modality: even
if Vizzini’s plan succeeds, this is only a suspicion (so it is marked as such using the suspicion
label). The fact that these events are nested within one another implies that action unit 30 has a
layered modality: a suspicion that is part of a plan. This fact must be taken into account when
making inferences related to it or based on it. Second, Vizzini explains (as part of his telling
in action unit 27) that–according to his plan–Humperdinck will, on nding Buttercup’s body
(action unit 33) believe (action unit 34) that the Guilderians have killed Buttercup (action unit
35). The relations between these events are similarly complex. Humperdinck’s belief (action
unit 34) is part of Vizzini’s plan. The event that Humperdinck (may) believe (action unit 35) also
carries a certain modality: even if Vizzini’s plan succeeds, this is only a belief (so it is marked as
such using the belief label). The fact that these events are nested within one another implies
that action unit 35 has a layered modality: a belief that is part of a plan. Again, this must be
taken into account when making inferences related to it or based on it.</p>
      <p>The modal nature of some of these statements implies that, even though they are predicated
on the same world as the events that they are nested in, they may not be considered on the
same relative timeline. The temporal relations that can be inferred are relatively clear: both
Humperdinck’s suspicion and his belief refer to events that have happened in the past. However,
their modality implies that, while we can establish a relative temporal order between the moment
when Humperdinck holds the corresponding thoughts and the suspected or believed events, we
can not actually place them in a speci c relative order with the non-modal events. This is true
not only with respect to the events that have already happened (because the events in the plan
have not themselves happened) but also with respect to the events in Vizzini’s plan (both the
suspicion and the belief are in fact in contradiction with the events that Vizzini plans). This
suggests that more complex representation mechanisms may be required to deal with these
cases.</p>
    </sec>
    <sec id="sec-4">
      <title>4. Discussion</title>
      <p>It is important to note, that, in contrast with many of the approaches reviewed in Section 2, the
proposal presented in this paper does not involve directly annotating over the text that conveys
the narrative, but rather it builds a parallel representation that explicitly captures the relevant
information gleaned from the text. This representation is intended to allow the explicit capture
the in uence on the relative chronology of the issues outlined in the introduction. Nevertheless,
as similar phenomena need to be captured, the proposed representation can be compared to the
methods for representing time used in existing annotation processes.</p>
      <p>
        The approach followed in the proposed schema with respect to the representation of narrative
levels di ers from the one proposed by Kearns [
        <xref ref-type="bibr" rid="ref9">9</xref>
        ] in that rather than marking that start and end
of the span of text corresponding to a narrative level it represents individual events as separate
representational units, and it assigns a narrative level number to each event. However, the
spans of events for a given narrative level may be reconstructed by clustering the events with
the same narrative level that are contiguous in the representation. When annotating, Kearn’s
approach only captures changes in narrative level, whereas our approach also captures changes
in world that is being described, whenever a change in narrative level also implies a change to
narrative events that take place in a di erent world. This is frequent when the embedded story
occurs in a ctional world. This richer representation allows a ner grain of analysis of ctional
texts, where changes in actual world are frequent, and of modal discourses, which involve a
change in possible world even when the actual world is taken as reference. The representation
of temporal information associated with narrative levels in Kearns’ approach is restricted to
whether the change in level involves a change forward or backward relative to story time. Our
proposal allows for more detailed representation of relative temporal relations based on Allen’s
temporal algebra.
      </p>
      <p>
        With respect to the proposal in Silvano et al. [
        <xref ref-type="bibr" rid="ref12">12</xref>
        ], the solution proposed here attempts to
explore in more detail what relative temporal relations might be inferred between the events.
The set of relative temporal relations that may be inferred is limited: in some cases the relations
are vague and in some cases there is insu cient information to establish speci c relations.
      </p>
      <p>
        The detailed notation for temporal abstractions proposed by Mani [
        <xref ref-type="bibr" rid="ref15">15</xref>
        ] for would allow a
more ne-grained representation of temporal relations than is a orded by the current solution
based on Allen’s temporal algebra [
        <xref ref-type="bibr" rid="ref14">14</xref>
        ], as it does not require precise information about temporal
relation. An extension of the proposal in this paper to allow the use of chronoscopes will be
considered as further work.
      </p>
      <p>
        The notation for clauses introduced in this paper alligns with the elements of text that are
associated to events in TimeML [
        <xref ref-type="bibr" rid="ref16">16</xref>
        ]. They are also assigned an ID, and relative temporal
relations with other events are explicitly in the X column in a way similar to the use of TLINKS.
      </p>
      <p>
        The scheme we propose allows explicit representation of characters as in SceneML [
        <xref ref-type="bibr" rid="ref19">19</xref>
        ] but
it presents an advantage of more detailed representation of temporal relations between events
using Allen’s temporal algebra, as discussed in the comparison with Kearns’ approach.
      </p>
      <p>
        The scheme proposed in this paper allows, by ltering the set of events based on the values for
particular columns, for the visualization of the adventures of particular characters as intended
by Macovei with the use of time tracks [
        <xref ref-type="bibr" rid="ref17">17</xref>
        ]. The importance of being able to consult a full
picture of the relative temporal timeline for the various events in a story is also advocated by
NarrativeTime [
        <xref ref-type="bibr" rid="ref18">18</xref>
        ]. With respect to these approaches, our proposal adds explicit representation
of di erent worlds that might be involved in di erent segments of the text, as in texts that
combine ctional and non- ctional worlds, or texts that refer to di erent historical periods.
      </p>
    </sec>
    <sec id="sec-5">
      <title>5. Conclusions</title>
      <p>The paper proposes a representation for the events in a ctional movie plot that captures the
mention of di erent worlds, transitions between narrative levels arising from the telling of
stories embedded in other stories, modal statements and relative chronologies between the
various timelines involved (for di erent worlds, di erent modalities, or di erent temporal
segments within the same world). The examples presented suggest that existing annotation
schemas may need to be extended with additional representational mechanisms to account for
the set of phenomena discussed in the paper. As future work we intend to explore how the
proposed representation can be applied to the plots of a number of movies.</p>
    </sec>
    <sec id="sec-6">
      <title>Acknowledgments</title>
      <p>This paper has been partially supported by the CANTOR project (PID2019-108927RB-I00) funded
by the Spanish Ministry of Science and Innovation.</p>
    </sec>
  </body>
  <back>
    <ref-list>
      <ref id="ref1">
        <mixed-citation>
          [1]
          <string-name>
            <given-names>J.</given-names>
            <surname>Culler</surname>
          </string-name>
          ,
          <article-title>Story and discourse in the analysis of narrative, Narrative theory: Critical concepts in literary and cultural studies 1 (</article-title>
          <year>2004</year>
          )
          <fpage>117</fpage>
          -
          <lpage>31</lpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref2">
        <mixed-citation>
          [2]
          <string-name>
            <given-names>R.</given-names>
            <surname>Delmonte</surname>
          </string-name>
          ,
          <string-name>
            <given-names>G.</given-names>
            <surname>Marchesini</surname>
          </string-name>
          ,
          <article-title>A semantically-based computational approach to narrative structure</article-title>
          ,
          <source>in: IWCS 2017-12th International Conference on Computational Semantics-Short papers</source>
          ,
          <year>2017</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref3">
        <mixed-citation>
          [3]
          <string-name>
            <given-names>G.</given-names>
            <surname>Collier</surname>
          </string-name>
          ,
          <article-title>The Rocks and Sticks of Words: Style, Discourse and Narrative Structure in the Fiction of Patrick White</article-title>
          , volume
          <volume>5</volume>
          ,
          <string-name>
            <surname>Brill</surname>
          </string-name>
          ,
          <year>2021</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref4">
        <mixed-citation>
          [4]
          <string-name>
            <given-names>P.</given-names>
            <surname>Portner</surname>
          </string-name>
          , Modality, Modality, OUP Oxford,
          <year>2009</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref5">
        <mixed-citation>
          [5]
          <string-name>
            <given-names>D. Y.</given-names>
            <surname>Oshima</surname>
          </string-name>
          , Perspectives in reported discourse,
          <source>Ph.D. thesis</source>
          , Stanford University Stanford,
          <year>2006</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref6">
        <mixed-citation>
          [6]
          <string-name>
            <given-names>G.</given-names>
            <surname>Genette</surname>
          </string-name>
          ,
          <string-name>
            <given-names>J.</given-names>
            <surname>Lewin</surname>
          </string-name>
          ,
          <string-name>
            <given-names>J.</given-names>
            <surname>Culler</surname>
          </string-name>
          , Narrative Discourse:
          <article-title>An Essay in Method, Cornell paperbacks</article-title>
          , Cornell University Press,
          <year>1983</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref7">
        <mixed-citation>
          [7]
          <string-name>
            <given-names>P.</given-names>
            <surname>Gervás</surname>
          </string-name>
          ,
          <article-title>A discourse interpretation engine sensitive to truth revisions in a story</article-title>
          ,
          <source>in: Tenth Annual Conference on Advances in Cognitive Systems</source>
          , Arlington, Virginia,
          <year>2022</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref8">
        <mixed-citation>
          [8]
          <string-name>
            <given-names>E.</given-names>
            <surname>Gius</surname>
          </string-name>
          ,
          <string-name>
            <given-names>N.</given-names>
            <surname>Reiter</surname>
          </string-name>
          ,
          <string-name>
            <given-names>M.</given-names>
            <surname>Willand</surname>
          </string-name>
          ,
          <article-title>A shared task for the digital humanities: annotating narrative levels (</article-title>
          <year>2019</year>
          ).
        </mixed-citation>
      </ref>
      <ref id="ref9">
        <mixed-citation>
          [9]
          <string-name>
            <given-names>E.</given-names>
            <surname>Kearns</surname>
          </string-name>
          ,
          <article-title>Annotating and quantifying narrative time disruptions in modernist and hypertext ction</article-title>
          ,
          <source>in: Proceedings of the First Joint Workshop on Narrative Understanding, Storylines, and Events</source>
          ,
          <year>2020</year>
          , pp.
          <fpage>72</fpage>
          -
          <lpage>77</lpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref10">
        <mixed-citation>
          [10]
          <string-name>
            <given-names>N.</given-names>
            <surname>Ketschik</surname>
          </string-name>
          ,
          <string-name>
            <given-names>B.</given-names>
            <surname>Krautter</surname>
          </string-name>
          ,
          <string-name>
            <given-names>S.</given-names>
            <surname>Murr</surname>
          </string-name>
          ,
          <string-name>
            <given-names>Y.</given-names>
            <surname>Zimmermann</surname>
          </string-name>
          ,
          <article-title>On the theory of narrative levels and their annotation in the digital context</article-title>
          ,
          <source>Journal of Cultural Analytics</source>
          <volume>6</volume>
          (
          <year>2021</year>
          ).
        </mixed-citation>
      </ref>
      <ref id="ref11">
        <mixed-citation>
          [11]
          <string-name>
            <surname>M.-L. Ryan</surname>
          </string-name>
          ,
          <article-title>Possible worlds, arti cial intelligence, and narrative theory</article-title>
          , Indiana University Press,
          <year>1991</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref12">
        <mixed-citation>
          [12]
          <string-name>
            <given-names>M. d. P.</given-names>
            <surname>Silvano</surname>
          </string-name>
          ,
          <string-name>
            <given-names>E.</given-names>
            <surname>Amorim</surname>
          </string-name>
          ,
          <string-name>
            <given-names>A.</given-names>
            <surname>Leal</surname>
          </string-name>
          , I. Cantante,
          <string-name>
            <given-names>M. d. F. H.</given-names>
            <surname>d. Silva</surname>
          </string-name>
          ,
          <string-name>
            <given-names>A.</given-names>
            <surname>Jorge</surname>
          </string-name>
          ,
          <string-name>
            <given-names>R.</given-names>
            <surname>Campos</surname>
          </string-name>
          ,
          <string-name>
            <given-names>S. S.</given-names>
            <surname>Nunes</surname>
          </string-name>
          ,
          <article-title>Annotation and visualisation of reporting events in textual narratives</article-title>
          ,
          <source>in: Proceedings of Text2Story 2023: Sixth Workshop on Narrative Extraction From Texts</source>
          ,
          <year>2023</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref13">
        <mixed-citation>
          [13]
          <string-name>
            <given-names>M.</given-names>
            <surname>Sternberg</surname>
          </string-name>
          ,
          <article-title>Telling in time (i): Chronology and narrative theory</article-title>
          ,
          <source>Poetics today 11</source>
          (
          <year>1990</year>
          )
          <fpage>901</fpage>
          -
          <lpage>948</lpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref14">
        <mixed-citation>
          [14]
          <string-name>
            <given-names>J. F.</given-names>
            <surname>Allen</surname>
          </string-name>
          ,
          <article-title>Time and time again: The many ways to represent time</article-title>
          ,
          <source>International Journal of Intelligent Systems</source>
          <volume>6</volume>
          (
          <year>1991</year>
          )
          <fpage>341</fpage>
          -
          <lpage>355</lpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref15">
        <mixed-citation>
          [15]
          <string-name>
            <surname>I. Mani</surname>
          </string-name>
          ,
          <article-title>Chronoscopes: a theory of underspeci ed temporal representations, in: Annotating, Extracting and Reasoning about Time</article-title>
          and Events: International Seminar, Dagstuhl Castle, Germany, April 10-
          <issue>15</issue>
          ,
          <year>2005</year>
          . Revised Papers, Springer,
          <year>2007</year>
          , pp.
          <fpage>127</fpage>
          -
          <lpage>139</lpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref16">
        <mixed-citation>
          [16]
          <string-name>
            <given-names>R.</given-names>
            <surname>Saurı</surname>
          </string-name>
          ,
          <string-name>
            <given-names>J.</given-names>
            <surname>Littman</surname>
          </string-name>
          ,
          <string-name>
            <given-names>B.</given-names>
            <surname>Knippen</surname>
          </string-name>
          ,
          <string-name>
            <given-names>R.</given-names>
            <surname>Gaizauskas</surname>
          </string-name>
          ,
          <string-name>
            <given-names>A.</given-names>
            <surname>Setzer</surname>
          </string-name>
          ,
          <string-name>
            <given-names>J.</given-names>
            <surname>Pustejovsky</surname>
          </string-name>
          ,
          <source>Timeml annotation guidelines version 1.2. 1</source>
          ,
          <year>2006</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref17">
        <mixed-citation>
          [17]
          <string-name>
            <given-names>A. MACOVEI</given-names>
            , An introduction to time tracks, Editors: Maria Mitrofan Daniela Gîfu Dan Tu ,sDan
            <surname>Cristea</surname>
          </string-name>
          (
          <year>2016</year>
          )
          <fpage>19</fpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref18">
        <mixed-citation>
          [18]
          <string-name>
            <given-names>A.</given-names>
            <surname>Rogers</surname>
          </string-name>
          ,
          <string-name>
            <given-names>M.</given-names>
            <surname>Karpinska</surname>
          </string-name>
          ,
          <string-name>
            <given-names>A.</given-names>
            <surname>Gupta</surname>
          </string-name>
          ,
          <string-name>
            <given-names>V.</given-names>
            <surname>Lialin</surname>
          </string-name>
          ,
          <string-name>
            <given-names>G.</given-names>
            <surname>Smelkov</surname>
          </string-name>
          ,
          <string-name>
            <given-names>A.</given-names>
            <surname>Rumshisky</surname>
          </string-name>
          , Narrativetime:
          <article-title>Dense temporal annotation on a timeline, arXiv preprint</article-title>
          arXiv:
          <year>1908</year>
          .
          <volume>11443</volume>
          (
          <year>2019</year>
          ).
        </mixed-citation>
      </ref>
      <ref id="ref19">
        <mixed-citation>
          [19]
          <string-name>
            <given-names>T.</given-names>
            <surname>Alrashid</surname>
          </string-name>
          ,
          <string-name>
            <given-names>R. J.</given-names>
            <surname>Gaizauskas</surname>
          </string-name>
          ,
          <article-title>A pilot study on annotating scenes in narrative text using sceneml</article-title>
          .,
          <source>in: Text2Story@ ECIR</source>
          ,
          <year>2021</year>
          , pp.
          <fpage>7</fpage>
          -
          <lpage>14</lpage>
          .
        </mixed-citation>
      </ref>
    </ref-list>
  </back>
</article>