Making reference, mainly, to the “non-fictional narratives” domain, we will suggest, firstly, an operational definition for highly ambiguous terms like “elementary events” and “complex events”. We will then raise the problem of how to represent these events/complex events in a computer-suitable form. We will introduce therefore a language, NKRL (Narrative Knowledge Representation Language), expressly specified and implemented for dealing with narratives and temporal information. Afterwards, we will show briefly how this language can be used for questioning and inferencing operations on knowledge bases of “events” formalised according to the NKRL approach.
1 Introduction
“Event-based” tools and systems seem to be particularly popular today. They range
from highly formalised systems like event algebras [
In the framework of the NKRL project (NKRL = Narrative Knowledge
Representation Language), see [
In the following, we will first explain informally, Section 2, what is denoted by terms like “elementary/complex events” in a narrative/NKRL context. We will then show, Section 3, how these informal notions can be translated into precise formal representation structures that constitute the gist of the NKRL language. Section 4 will supply some details about NKRL and will include two sub-sections, the first devoted to an outlook of the NKRL’s knowledge representation techniques and the second to the querying/inferencing procedures. Section 5 will consists in a short “Conclusion”. 2
Narratives, elementary and complex events ‘Narrative’ information concerns the account of some real-life or fictional story (a “narrative”) involving concrete or imaginary ‘characters’. In a NKRL context we are mainly concerned with “non-fictional narratives”, like those typically embodied into corporate memory documents (memos, policy statements, reports, minutes etc.), news stories, normative and legal texts, medical records, many intelligence messages, surveillance videos, actuality photos for newspapers and magazines, material (text, image, video, sound…) for eLearning, Cultural Heritage material, etc. We can note that this choice is only due to very practical constraints – to profit, e.g., from the financial support of the European Commission – and, as it will appear clearly in the following, nothing (at least in principle) could prevent us from dealing with the whole “Gone with the wind” fictional-narrative novel according to an NKRL approach.
More precisely, we assume that (fictional or non-fictional) narratives correspond
to the basic layer, the “fabula (a Latin word: fable, story, tale, play) layer”, introduced
by Mieke Bal [
Defining a narrative/complex event as a ‘stream of elementary events’ would correspond, once again, to some sort of ‘dull’ definition without being able to specify what an “elementary event” can be. In an NKRL context, this point is also particularly important from a practical point of view given that, as we see later, each elementary event is separately encoded making use of the NKRL knowledge representation tools. According then to a well-known Jaegwon Kim’s definition, see [14, 15], a “monadic” event – which can be considered as equivalent to an “elementary event” – is identified by a triple [x, P, t] where x is an object that exemplifies the n-ary property or relation P at time t (where t can also be an interval of time); “monadic” means then that the nary property P is exemplified by a single object x at a time. To make reference to one of the recurrent examples in the theoretical discussions about events, “Brutus stabs Caesar”, the Kimian interpretation of this event corresponds then to the representation of an individual x, Brutus who, at time t, is characterised by the property P exemplified by his stabbing of Caesar. Without entering now in the theoretical controversies raised by this sort of definition, see [9: 8-13] for some information in this context, we can note that, from an NKRL point of view, a more ‘practically useful’ – more complete and structured – definition of elementary event is that introduced by Donald Davidson [16, 17], particularly popular in the linguistic domain. This last focuses the representation of an elementary event on the “action verb” characterising the global conceptual category of the event more than – as in the Kimian approach – on the “generalised properties” of this event. In this way, the Davidsonian representation of “Brutus stabs Caesar” becomes: ∃e.stab(e, b, c), where e is an event variable. The global meaning of this formalism corresponds to: “There is an event e such that e was a stabbing of Caesar (c) by Brutus (b)”. Moreover, as emphasised above when we have listed some important characteristics of narratives/complex events, “roles” have a particular importance in a narrative. Our preferred formalism for the representation of elementary events is then the so-called “neo-Davidsonian” approach: the neo-Davidsonians, see [18, 19, 20], assume in fact that the event argument e above must be the only argument of the (verbal) predicate: this implies then, necessarily, the introduction of thematic (functional) “roles” for expressing the relations between events and their participants. The formalization of “Brutus stabs Caesar” becomes then now: ∃e[stab(e) & agent(e) = b & object(e) = c] . (1)
Apart from the theoretical implications, what expounded above is particularly
important because it supplies us with a ‘pragmatically useful’ and ‘operational’
criterion for recognizing and isolating – in some way, for ‘defining’ – “elementary
events”. The criterion consists then in the identification, within the description in
natural language (NL) terms of the global stream representing a narrative/complex
event, of a specific ‘generalized natural language predicate’: this represents then the
‘core’ of a new elementary event. The predicate corresponds usually to a verb – to
stick to the previous example, recognizing “stabs” as a verb in the NL chain “Brutus
stabs Caesar” should be sufficient for signalling the presence of an elementary event –
but, according to the neo-Davidsonian approach, this predicate can also, in case,
correspond to a noun (…Jane’s amble along the park…) or an adjective (“… worth
several dollars…”) when these last have a predicative function. Of course, a drawback
of this criterion concerns the fact that its utility is limited to the recognition of the
elementary components of narratives/complex events expressed in NL terms, while
narratives are “multimedia” in essence – a photo representing President Obama
addressing the Congress, or a short video showing three nice girls on a beach are
surely narrative documents (the first including only an elementary event) but they are
not, of course, NL documents. A classical way of getting around this problem consists
in annotating multimedia narratives in natural language see, e.g., [21]. In [
Eq. 1 above – an n-ary form of representation – shows clearly that the now so popular W3C proposals like RDF(S), OWL or OWL 2 – see [22, 23, 24] – are, at least in their standard format, unable to supply a basis for representing elementary events on a computer. All the W3C representations are, in fact, of the binary type, based on the classical ‘attribute – value’ model, where a property/attribute can only be a binary relationship linking two individuals or an individual and a value. The inadequateness of this approach to take into account complex representational problems like those linked with narratives, spatio-temporal information, and any sort of events and complex events is now widely recognized see, e.g., [25, 26, 27, 28, 29].
Note that the argument often raised in a W3C context and stating that any representation making use of n-ary relations can be always converted to one making only use of binary relations without any loss of expressiveness is incorrect with respect to the last part of the sentence. It is true in fact that, from a pure formal point of view, any n-ary relationship with n > 2 can always be reduced, in a very simple way, to a set of binary relationships. This possibility is well illustrated, among other things, by the successful representation of the NKRL’s core in the terms of a (W3C) binary language like RDF, see [30]. However, this fact does not change at all the intrinsic, ‘semantic’ n-ary nature of a simple statement like “Bill gives a book to Mary” that, to be understood, requires to be taken in its entirety. This means – see also Eq. 1 above – to make use of a semantic predicate of the GIVE type that introduces its three arguments, “Bill”, “Mary” and “book” through three functional relationships (roles) like SUBJECT (or AGENT), BENEFICIARY and OBJECT, the whole n-ary construction being – this is the central point – reified and necessarily managed as a coherent block at the same time. Only in this way it will be possible to infer, in a querying/inferencing context that, e.g., the above elementary event is linked, in the framework of a wider narrative, to another elementary event relating Mary’s birthday; for the formal details see, e.g., [25].
Efforts done, in a strict W3C context, to extend their binary languages using some n-ary features have not been very successful until now, see, for more details, [9: 1721]. Another way of trying to adapt/extend the traditional ‘binary’ tools to take into account complex, dynamic situations consists in reducing the notion of “role” from its normal status of ‘functional relationship’ to that of ‘static’ concept or class, see [9: 138-149] for a discussion about this topic. These role-concepts can then be used within all sorts of complex binary schemata or patterns to represent causality, mereology, participation etc. A well-known example of this approach is the “Descriptions and Situations (DS)” model, see [31], implemented as a plug-in extension to the DOLCE system [32], an (OWL-based) ‘upper ontology’. A recent variation on this approach is represented by Event-Model-F, see [33], based in turn on a reduced version of DOLCE, DOLCE+DnS Ultralite (DUL), see http://www.loacnr.it/ontologies/DUL.owl.
Several actual n-ary models that, among other things, can be used to represent in computer-usable ways elementary events have been described in the literature, see [9: 22-33] for a review. The n-ary model used in NKRL can be denoted as: (Li (Pj (R1 a1) (R2 a2) … (Rn an))) , (2) where Li is the symbolic label identifying (‘reifying’) the particular n-ary structure (e.g., the global structure corresponding to the representation of previous “John gives a book to Mary” example), Pj is a conceptual predicate, Rk is a generic “functional role” and ak the corresponding predicate argument (e.g., the individuals JOHN_, MARY_ etc.). Note that each of the (Ri ai) cells of Eq. 2, taken individually, represents a binary relationship in the W3C (OWL, RDF…) languages style. The main point is here that, as already stated, the whole conceptual structure represented by (2) must be considered globally.
Similarities between neo-Davidsonian expressions for elementary events like that of Eq. 1 and the formal structure of Eq. 2 are evident. However, some important differences exist. To avoid both the typical ambiguities of natural language and possible ‘combinatorial explosion’ problems – see the discussion in [9: 56-61] – both the (unique) conceptual predicate of Eq. 2 and the associated functional roles are ‘primitives’. Predicates Pj pertain in fact to the set {BEHAVE, EXIST, EXPERIENCE, MOVE, OWN, PRODUCE, RECEIVE}, and the functional roles Rk to the set {SUBJ(ect), OBJ(ect), SOURCE, BEN(e)F(iciary), MODAL(ity), TOPIC, CONTEXT}. Two special operators, date-1 and date-2 – that can be assimilated to functional roles – are used to introduce the temporal information associated with an elementary event: see, e.g., [9: 76-86, 194-201] for a detailed description of the formal system used in NKRL for the representation and management of temporal information. The NKRL representation of specific elementary events – that corresponds to the concrete instantiations (called “predicative occurrences”) of general structures in the style of Eq. 2 – is then a sort of canonical representation.
Several predicative occurrences – denoted by their symbolic labels Li and representing formally a (possibly structured) set of elementary events – can be associated within the scope of second order structures called “binding occurrences”. These are, in practice, labelled lists formed of a “binding operator Bn” with its arguments. The operators are those used in NKRL to represent the “connectivity phenomena” that guarantee the global coherence of narrative/complex events, see Section 2 above. They are: ALTERN(ative), COORD(ination), ENUM(eration), CAUSE, REFER(ence) – the ‘weak causality operator’, introducing two arguments where the second is necessary but not sufficient to explain the first – GOAL, MOTIV(ation) – the ‘weak intentionality operator’, where the first argument is not necessary to realise the second, which is however sufficient to explain the first – COND(ition), see [9: 91-98]. The general expression of a binding occurrence is then: (Bnk arg1 arg2 … argn) , (3)
Eq. 3 is particularly important in an NKRL context because it supplies also the formal expression – once again, a ‘pragmatic/operational’ form of definition – of the notion of “narrative/complex event”. The arguments argi of Eq. 3 can, in fact, i) correspond directly to Li labels – i.e., they can denote simply the presence of particular elementary events represented formally as predicative occurrences – or ii) correspond recursively to new labelled lists in Eq. 3 format. In the first case, the global narrative/complex event represents merely a chronological stream of elementary events, temporally characterized, where all these events have the same logical/semantic weight and the operator Bn corresponds to COORD (or ENUM/ALTERN). In the second case, we can suppose, e.g., that a given sequence of events – an Eq. 3 list of the COORD... type – represents the CAUSE of another sequence of events. The global representation of this narrative/complex event will then correspond to an Eq. 3 list labelled as CAUSE, having as arguments arg1 the COORD... list including the elementary events at the origin of the complex event and as arg2 the COORD... list including the elementary events that represent together the consequence, see also the simple example at the end of Section 4.1 below. What expounded above is in agreement with the remarks expressed by several authors – see [34] for example – about the possibility of visualizing under tree form the global, formal expression of a narrative/complex event made up of several elementary events.
After having introduced, in the previous Sections, the general theoretical framework
underpinning the NKRL approach to the narrative/complex events problem, we will
now illustrate briefly some points concerning its concrete implementation – see [
NKRL innovates with respect to the current ontological paradigms by adding to the usual ‘ontologies of concepts’ an ‘ontology of (elementary) events’, i.e., a new sort of hierarchical organization where the nodes correspond to n-ary structures in the style of Eq. 2 above. In the NKRL’s jargon, these n-ary structures are called “templates” and the corresponding hierarchy – i.e., the ontology of elementary events – is called HTemp (hierarchy of templates). Templates can be conceived as the canonical, formal representation of generic classes of elementary events like “move a physical object”, “be present in a place”, “produce a service”, “send/receive a message”, etc.
Note that, in the NKRL environment, an ‘ontology of concepts’ (according to the traditional meaning of these terms) not only exists, but it represents an essential component of this environment. The ‘standard’ ontology is called HClass (hierarchy of classes): structurally and functionally, HClass is not fundamentally different from one of the ontologies that can be built up by using tools in a ‘traditional’ Protégé style, see [35]. An (extremely reduced) representation of HClass is given in Figure 1 – HClass includes presently (April 2010) more than 7,000 concepts.
When a specific elementary event pertaining to one of the ‘general classes’ represented by templates must be encoded, the corresponding template is instantiated giving rise to a “predicative occurrence”. To represent then a simple elementary event (corresponding to the identification of the surface predicate “offer”) like: “British Telecom will offer its customers a pay-as-you-go (payg) Internet service in autumn 1998”, we must select firstly in the HTemp hierarchy the template corresponding to “supply a service to someone”, represented in the upper part of Table 1. This template is a specialization of the particular MOVE template corresponding to ‘transfer of resources to someone’ – Figure 2 below reproduces a fragment of the ‘external’ organization of HTemp. In a template, the arguments of the predicate (the ak terms in Eq. 2) are concretely represented by variables with associated constraints: these are expressed as HClass concepts or combinations of concepts, i.e., the two ontologies, HTemp and HClass, are then strictly intermingled.
SUBJ OBJ [SOURCE BENF [MODAL [TOPIC [CONTEXT {[modulators]}
When creating a predicative occurrence (an instance of a template) like c1 in the lower part of Table 1, the role fillers in this occurrence must conform to the constraints of the father-template. For example, in occurrence c1, BRITISH_TELECOM is an individual, instance of the HClass concept company_: this last is, in turn, a specialization of human_being_or_social_body. payg_internet_service is a specialization of service_, a specific term of social_activity, etc. The meaning of the expression “BENF (SPECIF customer_ BRITISH_TELECOM)” in c1 is self-evident: the beneficiaries (role BENF) of the service are the customers of – SPECIF(ication) – British Telecom. The ‘attributive operator’, SPECIF(ication), is one of the four operators used for the set up of the structured arguments (expansions) of conceptual predicates like MOVE, see [9: 6870]. In the occurrences, the two operators date-1 and date-2 materialize the temporal interval normally associated with an elementary event, see again [9: 76-86, 194-201].
More than 150 templates are permanently inserted into HTemp; HTemp, the NKRL ontology of events, corresponds then to a sort of ‘catalogue’ of narrative formal structures, which are very easy to extend and customize.
To supply now an at least intuitive idea of how a complete narrative/complex event is represented in NKRL, and returning to the Table 1 example, let us suppose we would now state that: “We can note that, on March 2008, British Telecom plans to offer to its customers, in autumn 1998, a pay-as-you-go (payg) Internet service…”, where the specific elementary event corresponding to the offer is still represented by occurrence c1 in Table 1.
To encode correctly the new information, we must introduce first an additional predicative occurrence labelled as c2, see Table 2, meaning that: “at the specific date associated with c2 (March 1998), it can be noticed, modulator obs(erve), that British Telecom is planning to act in some way” – the presence of a second surface predicate in the NL expression of the complex event denotes the presence of a second elementary event. obs(erve) is a ‘temporal modulator’, see [9: 71-72], used to identify a particular timestamp within the temporal interval of validity of an elementary event. We will then add a binding occurrence c3 labelled with a GOAL Bn operator, see the previous Section, to link together the conceptual labels c2 (the planning activity) and c1 (the intended result). The global meaning of c3 can be verbalized as: “The activity described in c2 is focalised towards (GOAL) the realization of c1”. In agreement with the remarks at the end of the last Section, c3 – the representation of the global narrative/complex event – can also represented under tree form, having GOAL as top node, and two branches where the leaves are L1 = c2 and L2 = c1.
Reasoning in NKRL ranges from the direct questioning of a knowledge base of
narratives represented in NKRL format – by means of search patterns pi (formal
queries) that unify information in the base thanks to the use of a Filtering Unification
Module (Fum), see [9: 183-201] – to high-level inference procedures. These last make
use of the richness of the representation to establish ‘interesting’ relationships among
the narrative items stored within the base; a detailed paper on this topic is [
The NKRL rules are characterised by the following general properties: • All the NKRL high-level inference rules can be conceived as implications of the type:
X iff Y1 and Y2 … and Yn . (4) • In Eq. 4, X corresponds either to a predicative occurrence cj or to a search pattern pi and Y1 … Yn – the NKRL translation of the ‘reasoning steps’ that make up the rule – correspond to partially instantiated templates. They include then, see the upper part of Table 1 above, explicit variables of the form vari. • According to the usual conventions of logic/rule programming, see [33: 105-170] – InferenceEngine understands each implication as a procedure. This reduces ‘problems’ of the form X to a succession of ‘sub-problems’ Y1 and … Yn. • Each Yi is interpreted in turn as a procedure call that tries to convert – using, in case, backtracking procedures – Yi into (at least) a successful search pattern pi. These last must then be able to unify (using the standard Fum module) one or several of the occurrences cj of the NKRL knowledge base. • The success of the unification operations of the pattern pi derived from Yi means that the ‘reasoning step’ represented by Yi has been validated. InferenceEngine continues trying then to validate the reasoning step corresponding to Yi+1. • In line with the presence of the operator ‘and’ in Eq. 4, the implication represented by Eq. 4 is fully validated iff all the reasoning steps Y1, Y2 … Yn are validated.
All the unification operations pi/cj required from the inference procedures make use only of the unification functions supplied by the Filtering Unification Module (Fum) introduced above. Apart from being used for the direct questioning operations, Fum constitutes as well, therefore, the ‘inner core’ of the InferenceEngine modules.
From a practical point of view, the NKRL high-level inference procedures concern
mainly two classes of rules, ‘transformations’ and ‘hypotheses’ see, e.g., [
Let us consider, e.g., the ‘transformations’. These rules try to ‘adapt’, from a semantic point of view, a search pattern pi that ‘failed’ (that was unable to find a unification within the knowledge base) to the real contents of this base making use of a sort of ‘analogical reasoning’. They attempt then to automatically ‘transform’ pi into one or more different p1, p2 … pn that are not strictly ‘equivalent’ but only ‘semantically close’ (analogical reasoning) to the original one. In a transformation context, the ‘head’ X of Eq. 4 is then represented by a search pattern, pi.
Operationally, a transformation rule can be conceived as made up of a left-hand side, the ‘antecedent’ – i.e. the formulation, in search pattern format, of the ‘query’ to be transformed – and of one or more right-hand sides, the ‘consequent(s)’ – the NKRL representation(s) of one or more queries (search patterns) to be substituted for the given one. Denoting with A the antecedent and with Cs all the possible consequents, the transformation rules can then be expressed as:
A(vari) ⇒ Cs(varj), vari ⊆ varj (5)
With respect to Eq. 4 above, X coincides now with A – a search pattern – while the reasoning steps Y1, Y2 … Yn are used to produce the search pattern(s) Cs to be used in place of A. The restriction vari ⊆ varj – all the variables declared in the antecedent A must also appear in Cs – assures the logical congruence of the rules. More formal details are given, e.g., in [9: 212-216].
Let us consider a concrete example, which concerns a recent NKRL application about the ‘intelligent’ management of ‘storyboards’ in the oil/gas industry, see also [37]. We want then ask whether, in a knowledge base where are stored all the possible elementary and complex events related to the activation of a gas turbine, we can retrieve the information that a given oil extractor is running. In the absence of a direct answer we can reply by supplying, thanks to a transformation rule like that (t11) of Table 3, other related events stored in the knowledge base, e.g., an information stating that the site leader has heard the working noise of the oil extractor, see Figure 3. Being unable to demonstrate directly that an industrial apparatus is running, the fact that an operator can hear its working noise or note its operational aspect can represent a proof of its running status.
Expressed in natural language, this result can be paraphrased as: “The system cannot assert that the oil extractor is running, but it can certify that the site leader has heard the working noise of this extractor”.
With respect now to the hypothesis rules, these allow us to build up automatically a sort of ‘causal explanation’ for an elementary event (a predicative occurrence cj) retrieved within a NKRL knowledge base. In a hypothesis context, the ‘head’ X of Eq. 4 then represented by a predicative occurrence, cj. Accordingly, the ‘reasoning steps’ Yi of Eq. 4 – called ‘condition schemata’ in a hypothesis context – must all be satisfied (for each of them, at least one of the corresponding search patterns pi must find a successful unification with the predicative occurrences of the base) in order that the set of c1, c2 … cn predicative occurrences retrieved in this way can be interpreted as a context/causal explanation of the original occurrence cj.
For example, to mention a ‘classic’ NKRL example, see [
A recent development of NKRL concerns the possibility of using the two above modalities of inference in an ‘integrated’ way, see [9: 216-234]. More exactly, it is possible to make use of ‘transformations’ when InferenceEngine is working in the ‘hypothesis’ environment. This means that, whenever a search pattern pi is derived from the ‘condition schema’ of a hypothesis to implement a step of the reasoning process, we can use it ‘as it is’ – i.e., in conformity with its ‘father’ condition schema – but also in a ‘transformed’ form if the appropriate transformation rules exist. The advantages are essentially that i) a hypothesis deemed to fail can now continue if a transformed pi is able to find an unification within the knowledge base, getting then new values for the hypothesis variables; ii) this strategy allows us to explore in a systematic ways all the possible implicit relationships among the data in the base.
NKRL deals with the representation and management of ‘elementary’ and ‘complex’ events by making use of n-ary and second order knowledge representation structures. One of its main characteristics concerns the addition of an ontology of events to the usual ontology of concepts. Its inference solutions employ advanced causal- and analogical-based reasoning techniques to deal with the events and their relationships.
NKRL is also a fully operational environment, implemented in two versions (fileoriented and Oracle-based) and developed thanks to several European projects. Many successful applications in many different domains (from ‘terrorism’ to the ‘corporate’, ‘cultural heritage’ and ‘legal’ domains, to the management of ‘storyboards/historians’ for the gas/oil industry…) have proved the practical utility of this tool.