The activities of people as well as of artificial agents in reality, virtual reality or simulation can be recorded as data that discretize trajectories of body parts and the ensuing force events. While these data provide vast amounts of information they are, by themselves, meaningless. Only when we put them into context we assign a specific meaning to these data. Increasingly, the notion of narratives is being used to describe the result of this semiotic process, i.e. we observe events and fit them into a story that makes sense to us. In this work a formal model is presented and discussed that can be employed to represent a narrative using the subset of FOL that is expressible in OWL DL.
"Not we sing the songs, the songs sing us"
The concept of a narrative has migrated from its original domain in the literary sciences to a multitude of diverse and increasingly distant research fields. It has become an important element in research on computer games [1] or in history [2] to name a few of these domains. At long last it has also arrived in the cognitive sciences where narratives are regarded to be a central means of sense making [3]. From there it was merely a short jump over to the field of cognitive robotics where the concept is employed to describe semantically annotated episodes of recorded activities [4].
When we observe people or artificial agents performing everyday activities in reality, virtual
reality or simulation, we collect episodic data that represent trajectories of body parts and
force events. While these data contain large quantities of information they are, by themselves,
meaningless. Only when we put them into a pragmatic context we assign specific meanings
to these data. For example, we can interpret the same observed episode as either throwing
something or dropping something. This diference in the narrativization, consequently, yields
two distinct narratives:
(
Please note that the situation spawning these two minimal narratives can be identical. We will, therefore in our model need to diferentiate between a situation, which has not been narrativized and a description of a situation, which pairs a situation with a selected conceptualization, i.e. interpretation, thereof. In addition to becoming meaningful, this pairing will, in turn, evoke a pragmatic stance that ascribes, for example, a specific perspective and intention to the agent(s) acting in the narrative. In the following, we will present a formal theory of narratives using the logical calculus of OWL-DL. Before introducing the theory, we will present the ontological commitments and underlying foundational framework together with the ontological design patterns, relevant to this work. 2. Foundational Framework and Prior Art The model proposed and outlined herein neither stands alone nor is it the first attempt to cast narratives in a formal framework. The following will therefore, briefly describe previous work on formalizing narratives before discussing the foundational commitments and ontological framework employed in this approach.
As it should be, each individual approach for representing narratives formally is driven by the specific requirements of the given application at hand. For example, the model of Meghini et al. is seeking to organize information provided in digital libraries and, therefore, equates a narrative with an event and allows for events to feature dependent events [5]. The model uses RDF based on OWL and narrative events can have spatial or temporal relations between them, but the main purpose is to connect digitally represented entities to pertinent events, e.g. the Divine Comedy as a book and the person Dante Algheri can be connected by a narrative Dante writes the Divine Comedy. In this approach events are also not formally specified as no foundational framework is employed.
A diferent approach by Kroll et al makes a useful distinction between factual relations, as expresses by knowledge graphs, and narrative relations that constitute hypothetical relation connecting factual ones [6]. For this the approach needs to employ RDF* to express relations that range over relations. This approach can, therefore, be employed to postulate, for example, causation relation as a narrative that connects hitherto isolated knowledge graphs.
A little closer to the task at hand in this work is the work described by Evans et al. that seeks to classify (partial) sensory data as a narrative that can be framed as an inductive logic programming task [7]. While their focus lies on reducing the search space by finding hypotheses – which equal narratives in their approach – that provide as simple an explanations of the observed data as possible. This approach can, therefore, be deemed compatible yet orthogonal to the one presented herein, as it focuses not on representing narratives in an ontology, but on a classification approach that employs such a model as a target representation.
It has become a sensible and important part of ontology engineering to make the underlying foundational commitments of a given model explicit. This formal theory of narratives cum model is based on the DOLCE+DnS Ultralite (DUL) foundational framework [8, 9]. This decision is greatly motivated by the underlying ontological commitments of DUL, its axiomatization as well as the incorporation of the Descriptions and Situations module. Firstly, DUL is not a revisionary model, but seeks to express stands that shape human cognition. Furthermore, it assumes a multiplicative approach – however rather than capturing the flexibility of our usage of objects via multiple inheritance it is also possible to combine a reduced ground classification with a descriptive approach for handling this flexibility. For this, a primary branch of the ontology represents the ground physical model, e.g. objects and events, while a secondary branch represents the social model, e.g. roles and tasks. All entities in the social branch would not exist without a cognitive agent, i.e. they constitute social objects that represent concepts about or descriptions of entities, as, for example, the construal of an observable event where an object moves from an agent’s hand to the floor into the interpretations given in Examples 1 and 2.
Every axiomatization in the physical branch can, therefore, be regarded as expressing some physical context whereas axiomatizations in the descriptive social branch are used to express social contexts. A set of dedicated relations is provided that connect both branches. For example, the relation classifies connects ground objects, e.g. a hammer, with the roles they can play, i.e. potential classifications. Thus, we can state that a hammer can in some context be conceptualized as a murder weapon, a paper weight or a door stopper. Nevertheless, neither its ground ontological classification as a tool will change nor will hammers be subsumed as kinds of door stoppers, paper weights or weapons via multiple inheritance.
The approach presented herein extends the Socio-physical Model of Acitivites (SOMA) by including a new module for representing narratives. Naturally, SOMA is also based on the DUL foundational framework and its plugin IOLite [10]. Consequently, SOMA has two knowledge branches; one physical and a social, which leads to a distinction between objects and events in the physical branch on the one hand, as well as roles and tasks in the social branch on the other. Beßler et al. explain that axiomatizations in the physical branch express physical contexts which can be classified by axiomatization in the social context. For example, a glass and its properties of being a designed physical artifact would be described using parts of the physical branch, but its potential usage or afordances would be axiomatized using the social branch [11]. SOMA is built out of multiple modules for diferent aspects. For example, the SAY module which defines the theories required by linguistic processing of instructions [ 12]. An overview of some of the modules is given in Figure 1 including the module for representing narratives (NARR) described in this work.
First, we look at the ground partition of the theory, that represents episodes that occur in the world and can be recorded, e.g. as visual or force dynamic data.1 These numeric data can be seen as the type of sensory data used in the approach of Kroll et al (2021).
→ → → → → →
∀(INCE(, ) → Event())
∃(INCE(, ))
Event()
∀(HASP(, ) → PhysicalObject())
∃(HASP(, ))
(
We model an episode as a dul:situation (
Episode() → DESCR(, ) → ∃(DESCR(, ) ∧ Narrative())
The describes relation (DESCR) holds between a dul:description (e.g., an narrative) and entities
that are conceptualized by the description (
Narratives define construals of dul:tasks and dul:roles within the episode they describe.
Specifically, we distinguish between dul:tasks that are conceptualizations of dul:actions, and the
dul:roles that are narrative-specific conceptualizations of the given function of dul:physicalObjects.
Consequently, we introduce two relations definesTask (DEFT) and definesRole (DEFR) that link a
narrative to the respective entities.
dul:actions are types of dul:events (
Finally, we formalize the narrative concept by axiomatizing its relationship to the episodic content described, and the concepts defined by it.
Narrative() → Narrative() → Narrative() → Narrative() →
∀(DESCR(, ) → Episode()) ∃(DEFT(, ) ∧ Task()) ∃(DEFR(, ) ∧ Role())
Our theory views narratives as dul:descriptions (
NAR(, ) → NAR(, ) → NARR(, ) → NARR(, ) → NART(, ) → NART(, ) →
Episode() ∧ Concept() ∃(DESCR(, ) ∧ DEF(, )) NAR(, ) ∧ Role() ∃(DESCR(, ) ∧ DEFR(, )) NAR(, ) ∧ Task() ∃(DESCR(, ) ∧ DEFT(, ))
The narrativizes relation links episodes and dul:concepts (16). Any dul:concept that is linked to a ground entity in an episode, e.g. via the dul:classifies relation, is defined in a narrative that describes the episode (17). Specifications of this relation further constrain the type of the construed dul:concept (18), and how the dul:concept is related to the narrative that defines it (19). Another specifications of this relation further constrain the type of the narrativized dul:task (20), and how the dul:concept is related to the narrative that defines it (21). The formalization of other sub-relations, i.e., narrativizesRole and narrativizesTask, is done analogously. The manifestation of an narrative (MNAR) is, in our view, a dul:situation that satisfies (SAT ) the narratives describing the episodic entities that are included in the dul:situation (22). More concretely, it is a dul:situation where an dul:agent executes the dul:task defined by the episode by following a dul:plan (which is another type of dul:description) involving dul:physicalObjects playing certain dul:roles and dul:regions setting specific dul:parameters for that execution. Hence, dul:situations in which narratives are manifested also satisfy the dul:plan that the dul:agent executes (23).
MNAR() → MNAR() → ∃!(SAT(, ) → Narrative()) ∃!(SAT(, ) → Plan()) A formalization of dul:plans that describe dul:tasks evoked by episodes is part of the existing DUL ontology. What makes a narrative a special type of description is, among other things, that it assumes a specific perspective. As this need to be included an a corresponding model, we propose to extend the work from spatial cognition to the narrative domain as follows: Perspective() →
HASO(, ) →
Origo() → Egocentric() →
Abstract() Perspective() ∧ Origo() Role() Perspective() ∧ Origo() (22) (23)
Lastly, we can connect narratives with their assumed perspective by postulation the relation of having a point of view (PoV) that holds between narratives and perspectives (29). (29)
→ ∀(HASPoV(, ) → Perspective())
This work is by no means done and, therefore, still in progress. Along with assuming a specific perspective on an episode, narratives also feature a teleological stance and in many cases also a normative valence. This needs to be included to arrive at a comprehensive model that allows for reasoning about narratives, e.g. what the specific diferences between two distinct narrativizations of an identical episode are and even what they mean. This type of reasoning would extend the semantics employed, for example, in opinion mining and sentiment analysis, as narratives could then be grouped and compared in terms of perspective, stance or valence.
The contribution of the work presented herein is to provide a representational framework that can readily be employed in cognitive robotics to counterpart the notion of a task that is given to a robotic agent and can be executed by finding an appropriate action with the notion of a narrative that looks at an action and seeks to makes sense of it. Ideally, one could match the task leading to an action and the narrative describing the action to express if that task has been successfully executed by the agent from the point of view of the narrativizer. As most readers will know there can be vast diferences in these judgments, for example, between parents and children concerning the question if a room has been properly cleaned.
Again, it is important to note that this approach explicitly rejects an objective notion of a narrative, i.e. to equate a narrative with what has objectively happened and can be recorded an stored as data. As a descriptive notion a narrative assumes a specific point of view on the episodic event. It, therefore, provides a spin on the event and is subjective. Nevertheless, these views can be shared by collectives and become well established frames in which larger historical or everyday episodic events can be seen by societies or groups.
Next to finishing the model proposed herein and populating it for the domains of everyday activities and social policies as it is planned in the EASE and MUHAI projects2. In the latter the implementation of a social observatory is planned where narrative networks as personal dynamic memories (PDMs) [13] can be detected in large collections of knowledge graphs concerning social equality and policy. In the former project in cognitive robotics narrative enabled episodic memories (NEEMS) [14] are constructed and stored for analysis that describe executions of everyday activities.
2The research reported in this paper has been (partially) supported by the FET-Open Project 951846 “MUHAI – Meaning and Understanding for Human-centric AI” (http://www.muhai.org/) funded by the EU Program Horizon 2020 as well as the German Research Foundation DFG, as part of Collaborative Research Center (Sonderforschungsbereich) 1320 “EASE – Everyday Activity Science and Engineering”, University of Bremen (http://www.ease-crc. org/). The research was conducted in subproject P01