Towards Modeling Conceptual Dependency Primitives with Image Schema Logic Jamie C. MACBETH a and Dagmar GROMANN b,1 a Smith College, Northampton, MA, 01063, USA b University of Vienna, Gymnasiumstraße 50, A-1190 Vienna, Austria Abstract. Conceptual Dependency (CD) primitives and Image Schemas (IS) share a common goal of grounding symbols of natural language in a representation that allows for automated semantic interpretation. Both seek to establish a connection between high-level conceptualizations in natural language and abstract cognitive building blocks. Some previous approaches have established a CD-IS correspon- dence. In this paper, we build on this correspondence in order to apply a logic de- signed for image schemas to selected CD primitives with the goal of formally tak- ing account of the CD inventory. The logic draws from Region Connection Calcu- lus (RCC-8), Qualitative Trajectory Calculus (QTC), Cardinal Directions and Lin- ear Temporal Logic (LTL). One of the primary premises of CD is a minimalist ap- proach to its inventory of primitives, that is, it seeks to express natural language contents in an abstract manner with as few primitives as possible. In a formal anal- ysis of physical primitives of CD we found a potential reduction since some primi- tives can be expressed as special cases of others. Keywords. Conceptual Dependency, Formal Modeling, Image Schemas 1. Introduction Natural language understanding remains to be one of the major challenges of modern Ar- tificial Intelligence (AI) and cognitive systems. One approach to tackling this challenge is to map the potentially infinite compositional variety of natural language sequences onto abstract, unambiguous base forms in a (semi-)formal representation. Conceptual Dependency (CD) comprises one such framework that performs this function by decom- posing language into complex combinations of language-independent conceptual primi- tives [14,15]. CD evolved to reduce the number of conceptual primitives in the system, generalizing them, increasing levels of decomposition, and reducing chances of multiple representations of the same concept. A second major such system is that of Lakoff [7] and Johnson [5] called image schemas (IS), building on embodied cognition [16]. Sensori-motor experiences with the 1 Corresponding Author: University of Vienna, Gymnasiumstraße 50, A-1190 Vienna, Austria; E-mail: dagmar.gromann@gmail.com Copyright c 2019 for this paper by its authors. Use permitted under Creative Commons License Attribution 4.0 International (CC BY 4.0). external world form patterns that are believed to shape abstract conceptualizations, such as reasoning and language, and they are generally described as spatio-temporal relation- ships. A first important step towards image-schematic computational models is their un- ambiguous, formal representation. A formalization approach to image schemas, the so- called Image Schema Logic ISLM , builds on various calculi for modeling their spatial and temporal dimension, but also movement and dynamic dimension. A correspondence between CD and IS has been established before [10] and backed with empirical evidence [2]. Building on this previous correspondence, we evaluate the utilization of ISLM for modeling CD primitives. Since CD representations lack a level of formality, utilizing a well-defined logic towards this end can be highly beneficial to check on the CD inventory for potential redundancies. As a very first experiment, this paper models a selection of physical primitives that are mapped to a comparable set of image schemas, as established in previous work [10], with a view to evaluating their correspondence in ISLM . We found that certain primitives could be removed without loss of CD expressiveness. 2. Conceptual Dependency This section first introduces Conceptual Dependency (CD) and a selected number of primitives focusing on physical aspects. It then continues to elaborate on previously es- tablished correspondences between CD and image schemas as a basis for our assumption that ISLM can be applied to formalizing selected CD primitives. 2.1. Conceptual Dependency Primitives As a theory of meaning representation, CD was developed as an alternative natural lan- guage understanding system to formal, linguistic theories at that time [14,15,9]. It intends to equip computational systems with human-like understanding of language that mirrors human cognition. It decomposes meaning of natural language into language-agnostic structures, called conceptual primitives. 2.2. CD Constructs and Syntax The main well-formed expression in the Conceptual Dependency representation system is the conceptualization. CD conceptualizations have basic elements that are described and depicted in Table 1. CD has numerous primitives used to represent thought, perception, social interac- tion, and communication (see [15]). However, in this paper, we narrow our focus to its physical, spatial, and object-defining primitives, whose names and descriptions are given in Table 2. Example sentences in Table 2 are taken from a previous study on crowd- sourcing the annotation of natural language sentences with CD primitives [11]. CON- TAIN is utilized to specify physical objects in conceptualizations, which is why it is also provided. Denominations of CD primitives resonate English words, however, the concepts they identify differ from the lexical definitions of those words. For instance, “ingest” relates to events of animate beings consuming food or drinks. The CD primitive INGEST is broader in meaning as it relates to a variety of acts where a substance or object enters Table 1. Descriptions of the components and constructs of CD conceptualization diagrams as described in [15]. Examples of full conceptualization diagrams are given in Table 4. CD Construct Description PPs Picture producers (PPs) denote physical objects serving in various roles in a conceptualization. In representing the sentence “Amy took a breath,”, “Amy” and “breath” (or “air”) are PPs. ACTs ACTs are conceptual primitives representing things that can be done by an actor to an object, or events that can happen to an object. PTRANS and INGEST, described in Table 2, are examples of ACTs. PP ks +3 ACT One kind of CD conceptualization is an ACT performed by a PP as the actor. For example, for “Amy took a breath,” could be (partially) represented by Amy ks +3 INGEST. The double arrow represents the two-way dependency relationship between the actor and the ACT. o ACT o PP A conceptualization can have a PP as an object. The arrow points from the object PP to the primitive ACT having that PP as an object. For example, the PP “air” could serve as the object of an INGEST for “Amy took a breath.” D / PP1 A conceptualization can have a direction case (also called the ACT o DIRECTION primitive). Directions are specified as the locations of o PP2 PPs, with the two PPs indicating the “from” and “to” for primitive acts involving movement. For example, for “Amy took a breath”, PP1 , the destination of the movement of air, would be Amy’s lungs. PP jt *4 PA Picture producers can be described by picture aiders (PAs), “state” predicates that take the form STATE(VALUE). CONTAIN() is an example of such a state predicate. / PA1 PP jt Picture producers can go through state changes described by a pair of o PA2 PAs. A state change of a PP is also a type of ACT. ks +3 JT The result causation connective (indicated by the triple arrow labeled “r”), which connects two conceptualizations, indicates that one event or r act resulted in another. ks +3 the body of an animate being, such as air, injections, transdermal absorption into the skin or even single-cell organisms absorbing a molecule through its cell wall. Some of the primitives have abbreviated names; for example PTRANS is short for “Physical TRANSfer”. 2.3. Conceptual Dependency and Image Schemas Image schemas were introduced as abstract spatio-temporal relationships that aim to bridge the gap between sensorimotor, embodied experiences and high-level conceptual- izations, such as natural language and reasoning. Image Schemas generalize sensorimo- tor experiences, by which they abstract away from lexical manifestations of language, similar to CD. For instance, repeated experiences of objects or people in concave objects, Table 2. Six physical primitives acts of CD, and the CONTAIN primitive used to specify properties of objects in conceptualizations Primitive Description Example/“Target” Sentences PTRANS A person, object, or thing changes Can be used to represent part of the physical position or location. conceptualization of “Matthew flew home from Los Angeles,” as Matthew ks +3 PTRANS. MOVE A person, object, or thing moves a part Used to represent “Kevin crossed his of its body or part of itself. arms,” as Kevin ks +3 MOVE with “arms” as the object. EXPEL A change in spatial relationship between Can be used to represent part of the two picture producers (PP1 and PP2) conceptualization of a sentence such as beginning with PP1 being on the inside “Michelle threw up her lunch,” as of PP2, and ending with PP1 being on Michelle ks +3 EXPEL. the outside of PP2. INGEST A change in spatial relationship between Can be used to represent part of the two picture producers (PP1 and PP2) conceptualization of a sentence such as beginning with PP1 being on the outside “Amy took a deep breath,” as of PP2, and ending with PP1 being on Amy ks +3 INGEST. the inside of PP2. CONTAIN Denotes containment relations for Can be used to represent part of the objects in CD conceptualizations as a conceptualization of a phrase like “a frog Picture Aider (PA). in a box” as frog jt *4 CONTAIN(box). such as water in a glass, a tissue in a box, a person in a room, reinforce our basic pattern of the image schema C ONTAINMENT, that is, something with an inside, an outside, a boundary, and a container where becoming contained at some moment in time requires motion. First correspondences between CD primitives and image schemas were established on a theoretical basis with annotations of natural language examples by three experts [10] and are depicted in Table 3. This mapping has been experimentally reinforced by replicating a crowdsourcing-based annotation project of CD primitives [11] for image schemas, uitilizing the same dataset of linguistic sequences for both experiments [2]. It has to be noted that several distinct physical primitives can be expressed by identical (combinations of) image schemas. This repetition in image-schematic mapping moti- vated our assumption that more complex physical primitives, such as INGEST and EX- PEL that map to three image schemas, might be expressible as a combination of less complex primitives, such as PTRANS and CONTAIN. It seemed only natural to utilize an existing, closely related logic to formally analyze this assumption. A second major motivator for our choice of logic was the existing modeling of dy- namic aspects of C ONTAINMENT [3], one of the central image schemas in our map- ping to CD primitives, which generally occurs in combination with movement along a S OURCE PATH G OAL. As such it provides an excellent basis for formally analyzing complex physical primitives and the feasibility of expressing more complex primitives with simpler ones, thereby reducing the number of required primitives to model meaning underlying natural language sequences. Table 3. Mappings between the physical primitive acts, the PP, and CONTAIN primitives of CD, and the related image schemas and spatial primitives. CD Primitive Related Image Schema(s) Related Spatial Primitives PTRANS S OURCE PATH G OAL S OURCE, G OAL, PATH, M OVE, D IRECTION MOVE S OURCE PATH G OAL, S OURCE, G OAL, PATH, M OVE, PART-W HOLE D IRECTION, PARTS, W HOLE, INGEST S OURCE PATH G OAL, I N, B OUNDARY, C ONTAINER, C ONTAINMENT, F ORCE S OURCE, G OAL, PATH, M OVE, D IRECTION EXPEL S OURCE PATH G OAL, O UT, B OUNDARY, C ONTAINER, C ONTAINMENT, F ORCE S OURCE, G OAL, PATH, M OVE, D IRECTION PP O BJECT CONTAIN C ONTAINMENT C ONTAINER, I N, O UT, B OUNDARY 3. Formalization Language We rely on a previously introduced formal language for image schema modeling called ISLM [4] and refer the interested reader to this reference for a full account. In this paper, we will limit our description to an overall summary of specific crucial elements of the logic that draws from existing calculi, required to model the selected CD primitives. The logic is a combined one of RCC-8, cardinal directions, QTC, and LTL with 3d Euclidean space for the spatial domain briefly touched upon in this chapter. 3.1. Spatial Dimension Building on previous work, such as [1], ISLM utilizes Region Connection Calculus (RCC-8) [12] for basic topological relations, in which two objects can be disconnected (DC) or partially overlapping (PO). Also, one object can be a proper part of another ob- ject (PP), a tangential proper part of another object (TPP), or a non-tangential proper part of another object (NTPP). Thereby, it is possible to denote the (lack of) contact between two objects. To model directionality, Ligozat’s [8] cardinal directions are applied in the mode of a fixed observer outside the model, which results in six binary predicates: Left, Right, FrontOf, Behind, Above, and Below. 3.2. Movement Dimension To deal with the dynamic aspects of movement, the logic relies on Qualitative Trajec- tory Calculus (QTC) [17] and selects three possible movements of objects in relation to each other from its variant QTCB1D : object O1 moves towards O2 (O1 O2 ), object O1 moves away from O2 (O1 ←- O2 ), and object O1 is at rest with respect to O2 ’s position (O1 |◦ O2 ). 3.3. Temporal Dimension To simplify the complexity of modeling time in cognitive theories, ISLM relies on a linear temporal logic (LTL) over the reals [6,13]. The syntax is as follows: ϕ ::= p | > | ¬ϕ | ϕ ∧ ϕ | ϕUϕ This allows us to express Fϕ (at some time in the future, ϕ) defined as >Uϕ, and Gϕ (at all times in the future, ϕ) defined as ¬F¬ϕ. 4. A Comparison of Requirements One of the most crucial aspects of CD is that it requires a PP, a picture producer, to be a physical object. Image schemas, in contrast, have no such requirement, even though it might be counterintuitive to model image schemas without any relation to objects [3]. A dependency between a PP and some primitive ACT is established, which might be a mental operation [15]. For instance, to “eat” means to take something inside, to INGEST it. This ACT requires a clear direction to the inside of the object. CD originally was highly diagrammatic as depicted with the visual representation of the DIRECTION case in Table 1, which states that some object on the left hand side is moved from a previous location on the lower right hand side of the diagram to a new location on the upper right hand side of the diagram. Requirements specific to INGEST and EXPEL ACTs are the movement of a PP in a specific DIRECTION with the entailed change of location. In contrast to other types of movement, this CD primitive involves a relation to CONTAIN, either as leaving or entering a container. However, this relation is not explicitly established in CD. In the example diagrams in Table 2, the PP “frog” becomes CONTAINed in a PP “box”. Please keep in mind that PP here refers to picture-producer. Depending on which ACT applies, the DIRECTION is to the inside or outside. This is highly similar to the image schema C ONTAINMENT, which subsumes both directions. For DIRECTION, there is the addi- tional requirement that it might only connect locations, whereas CONTAIN would be considered a state, which cannot be mixed. There is no required relation between CON- TAIN and DIRECTION, and it is not necessary that the latter coincide with a container in either location, start or end. CD also explicitly distinguishes objects (animate or inanimate) and persons (per definition animate) which is not the case for image schemas. Both of these are mapped to the image schema O BJECT, which could be equalled to PP. As such, some requirements for INGEST and EXPEL are similar to those of the image schema C ONTAINMENT, as the PP that ACTs as CONTAINer can have one opening (putting food into your mouth), two openings (breathing through the nose), or several openings (transdermal absorption into the skin) through which objects or persons can go. In the original CD version, MOVE is restricted to the movement of body parts, which over time has been broadened to denote also the movement of parts of PPs. In contrast, the general motion entailing a change of location is modeled as PTRANS. For instance, “John placed his hand over his mouth” falls into the former category of primi- tives, whereas “John went home” requires the latter. The change of location for INGEST and EXPEL ACTs is from the inside to the outside or vice versa. This type of movement requires a PP, a DIRECTION, and an instrument, which is not further specified by CD. In the interpretation of this paper we consider the instrument either as a second PP utilized to cause the movement (e.g. a vehicle) or a PATH serving as the basis for the movement. General requirements in CD foresee modifications of primitives to account for tenses in language. This corresponds to past, future, negation, start of transition, end of tran- sition, conditional, continuous, interrogative, timeless, and present [15]. For the sake of simplicity, we will limit those cases to the ones introduced in Section 3.3. 5. Modeling CD Primitives in ISLM As the most central element of CD primitives, we need to first establish an equivalence between picture-producers and O BJECTs based on previous findings presented in Sec- tion 2.3, which we will refer to as O BJECTs in this section since PP will here refer to RCC proper part from now on. Such O BJECTs can change their locations, which in line with ISLM we model using QTC (see Section 3.2) M OVEMENT A LONG PATH and which corresponds to PTRANS. On PATH Toward(O1 , O2 ) := (O1 O2 ∧ DC(O1 , O2 )) In order to model the MOVE CD primitive, which represents animate actors moving parts of their bodies (e.g. arms or legs, or diaphragm muscles to represent a sentence like “Amy took a breath”), we need to take into consideration that a body part of an animate actor is a proper part of their body. For the sake of simplicity, we utilize PP(O1 , O2 ), where object O1 is a proper part of object O2 and can be a tangential proper part (T PP) or a non-tangential proper part (NT PP). This allows us to model MOVE as a special case of M OVEMENT A LONG PATH. It requires three objects, since it concerns the body part (O1 ), a body (O2 ), and an object that the body part moves toward (O3 ). In special cases, it is possible that O3 coincides with O2 when the body part is moved towards the body or represents another PP of the body, such as “John placed his hand over his mouth”, where PP(O1 , O2 ) ∧ PP(O3 , O2 ). Move Toward(O1 , O2 , O3 ) := PP(O1 , O2 ) ∧ On PATH Toward(O1 , O3 ) In the cases of PTRANS and MOVE, the inverse movement of one object away from another is also possible, which would be modeled by replacing with ←- . On PATH From(O1 , O2 ) := (O1 ←- O2 ∧ DC(O1 , O2 )) This implicitly allows us to include the CD primitive DIRECTION, which is a re- quirement for both PTRANS and MOVE. Again here it is possible that the third object is a proper part of the body as well. Move From(O1 , O2 , O3 ) := PP(O1 , O2 ) ∧ On PATH From(O1 , O3 ) One central basic primitive is CONTAIN, which as a state in CD can be modeled utilizing the static representation of C ONTAINMENT in ISLM as suggested by Hedblom et al. [3]. Like Hedblom et al, we augment ISLM with predicates opening of(op, O) to represent op is an opening of O, inside of(in, O) representing that in is the inside of O, and outside of(out, O) representing that out is on the outside of O. Contained Inside(O1 , O2 ) := inside of(in, O2 ) ∧ PP(O1 , in) In order to become contained, an object needs to cross the opening of the con- tainer. For instance, when breathing the air might pass into the body through the open- ing “mouth”. The definition below shows a close relation between several primitives. It utilizes On PATH Toward, utilized to model PTRANS above, and Contained Inside, utilized to model CONTAIN, above. Crossing Opening(O1 , O2 , opening) := opening of(opening, O2 ) ∧ (DC(O1 , O2 ) ∧ On PATH Toward(O1 , opening)) ∧ F(PO(O1 , opening)) Crossing Opening is one important modeling component for INGEST, which is sim- ilar to the inward directed movement of a C ONTAINMENT image schema and equivalent to the Going I N in the ISLM implementation of dynamic C ONTAINMENT [3]. The fact that this modeling reuses the modeling of PTRANS and CONTAIN establishes a direct connection to INGEST. To make this relation between primitives more explicit, we show the CD diagram on the left and the ISLM definition on the right in Table 4. As can be seen in the ISLM modeling and the CD diagram, INGEST can be treated as a composition of PTRANS and CONTAIN, if the DIRECTION is modeled as in ISLM . It could be argued that the explicit Crossing Opening is missing in this case, however, in the CD diagram, this is also missing for INGEST. Thus, ISLM not only facilitates the detection of CD primitive interrelations but also fosters a higher precision in their definitions. In Table 5, we perform the same modeling exercise for the CD primitive EXPEL, which equally can be viewed as a composition of PTRANS, CONTAIN, and DIREC- Table 4. Correspondence of INGEST with PTRANS, CONTAIN and DIRECTION through ISLM on the sen- tence “Amy took a deep breath.” CD Diagram ISLM air o  / inside(Amy) Going I N(air, Amy, mouth) := D Crossing Opening(air, Amy, mouth) ∧ Amy ks +3 INGEST o F(Contained Inside(air, Amy)) o outside(Amy) air o  / inside(Amy) D Amy ks JT +3 PTRANS o o outside(Amy) Going I N(air, Amy, mouth) := Crossing Opening(air, Amy, mouth) ∧ r F(Contained Inside(air, Amy)) / CONTAIN(Amy) air jt o TION with the only difference of a change of direction in CD. As can be seen in Ta- ble 5, the same ISLM elements are being used for Going O UT as for Going I N with the addition of the final state being outside. In the previously established correspondences between CD primitives and image- schematic constructs, INGEST and EXPEL were mapped to C ONTAINMENT and S OURCE PATH G OAL. With ISLM we could now show that in fact these two are spe- cific cases of C ONTAINMENT, which in order to be dynamic requires movement along a PATH, a CONTAIN relation and a DIRECTION. Since these are other CD primitives, INGEST and EXPEL can be modelled utilizing compositions of PTRANS, CONTAIN, and DIRECTION, which further reduces the CD inventory. 6. Conclusion In this paper, we evaluated the formal modeling of Conceptual Dependency primitives, with the objective of allowing for a more fine-grained comparison and detection of poten- tial redundancies. Since a previous correspondence to image schemas was established, we decided to utilize the well-defined Image Schema Logic ISLM , which turned out to be well applicable to the modeling of CD primitives. This modeling exercise allowed us to establish an equivalence between INGEST and EXPEL with the only difference of their DIRECTION, and also show that both could be modeled as a composition of PTRANS, CONTAIN, and DIRECTION, which means they could be considered redundant. Since this was only a first experiment on a limited set of primitives, in the future we want to extend the formalization the full CD repository, which might bring further equivalences among primitives but also to image schemas to the light. Table 5. Correspondency of EXPEL with PTRANS, CONTAIN, and DIRECTION through ISLM on the sen- tence “Michelle threw up her lunch.” CD Diagram ISLM lunch Going O UT(lunch, Michelle, mouth) := Contained Inside(lunch, Michelle)∧ o  / outside(Michelle) F(Crossing Opening D Michelle ks +3 EXPEL o (lunch, Michelle, mouth)∧ o inside(Michelle) F(outside of(lunch, Michelle))) lunch Going O UT(lunch, Michelle, mouth) := o  / outside(Michelle) Contained Inside(lunch, Michelle)∧ D ks +3 PTRANS o Michelle JT F(Crossing Opening o inside(Michelle) (lunch, Michelle, mouth)∧ r / F(outside of(lunch, Michelle))) lunch jt o CONTAIN(Michelle) References [1] B. Bennett and C. Cialone. Corpus guided sense cluster analysis: a methodology for ontology devel- opment (with examples from the spatial domain). In 8th Int. Conf. on Formal Ontology in Information Systems (FOIS), volume 267, pages 213–226. IOS Press, 2014. [2] D. Gromann and J. C. Macbeth. Crowdsourcing image schemas. In TriCoLore (C3GI/ISD/SCORE), 2018. [3] M. M. Hedblom, D. Gromann, and O. Kutz. In, out and through: formalising some dynamic aspects of the image schema c ontainment. In Proceedings of the 33rd Annual ACM Symposium on Applied Computing, pages 918–925. ACM, 2018. [4] M. M. Hedblom, O. Kutz, T. Mossakowski, and F. Neuhaus. Between c ontact and s upport: Introducing a logic for image schemas and directed movement. In Conference of the Italian Association for Artificial Intelligence, pages 256–268. Springer, 2017. [5] M. Johnson. The Body in the Mind: The Bodily Basis of Meaning, Imagination, and Reason. The University of Chicago Press, Chicago and London, 1987. [6] F. Kröger and S. Merz. Temporal Logic and State Systems (Texts in Theoretical Computer Science. An EATCS Series). Springer Publishing Company, Incorporated, 1 edition, 2008. [7] G. Lakoff. Women, Fire, and Dangerous Things. What Categories Reveal about the Mind. The University of Chicago Press, 1987. [8] G. Ligozat. Reasoning about cardinal directions. J. Vis. Lang. Comput., 9(1):23–44, 1998. [9] S. L. Lytinen. Conceptual dependency and its descendants. Computers & Mathematics with Applica- tions, 23(2):51–73, 1992. [10] J. Macbeth, D. Gromann, and M. M. Hedblom. Image schemas and conceptual dependency primitives: A comparison. In Proceedings of the Joint Ontology Workshop (JOWO). CEUR, 2017. [11] J. C. Macbeth and M. Barionnette. The coherence of conceptual primitives. In Proceedings of the Fourth Annual Conference on Advances in Cognitive Systems. The Cognitive Systems Foundation, June 2016. [12] D. A. Randell, Z. Cui, and A. G. Cohn. A spatial logic based on regions and connection. In Proc. of the 3rd International Conference on Knowledge Representation and Reasoning (KR-92), 1992. [13] M. Reynolds. The complexity of temporal logic over the reals. Annals of Pure and Applied Logic, 161(8):1063 – 1096, 2010. [14] R. C. Schank. Conceptual dependency: A theory of natural language understanding. Cognitive Psychol- ogy, 3(4):552–631, 1972. [15] R. C. Schank. Conceptual Information Processing. Elsevier, New York, NY, 1975. [16] L. Shapiro. Embodied cognition. New problems of philosophy. Routledge, London and New York, 2011. [17] N. V. D. Weghe, A. G. Cohn, G. D. Tré, and P. D. Maeyer. A qualitative trajectory calculus as a basis for representing moving objects in geographical information systems. Control and cybernetics, 35(1):97– 119, 2006.