Introducing ISAAC: the Image Schema Abstraction And Cognition modular ontology

Abstract

Embodied cognition and the theory of cognitive metaphors ground our commonsense reasoning ability in language, linking subjective perception of the external world with cognitive inferential patterns. Furthermore, commonsense reasoning is linked to human sense-making, pattern recognition and knowledge framing abilities. This work presents ISAAC, the Image Schema Abstraction And Cognition modular ontology, a new resource that formalizes the cognitive theory of Image Schemas. Image Schemas are conceptual dynamic building blocks originated by recurring sensorimotor interactions with the physical world. These experiential patterns provide coherence and structure to entities, sequences of events and situations we experience everyday. ISAAC ontology provides a formalization of theoretical state of the art literature background and integration of diferent theories regarding linguistic and factual entities already operationalised in ontological modules like ImageSchemaNet, the image-schematic layer built on top of FrameNet.

1. Introduction

Image schemas (IS), initially introduced by Johnson [ 1 ] and Lakof [ 2 ] within the tradition of embodied cognition, are considered early (pre)conceptual building blocks that derive from recurring sensorimotor experiences and shape abstract cognition, including commonsense reasoning and natural language (see e.g. [ 3, 4 ]). In fact, image schemas are described as internally structured gestalts [ 1 ] consisting of basic elements that compose image schemas to a unified whole [ 5, 6 ]. Even though the theoretical tenets of these initial publications are largely adopted, several further specifications and extensions have been proposed, including extensions in order to allow for their formalization [7]. Mandler and Pagán Cánovas [ 3 ], for instance, analyse image schemas from the perspective of developmental psychology and propose in (D. Gromann) (A. Gangemi); https://dagmargromann.com/ (D. Gromann) https://www.unibo.it/sitoweb/stefano.degiorgis2/en (S. De Giorgis); https://www.unibo.it/sitoweb/aldo.gangemi © 2022 Copyright for this paper by its authors. Use permitted under Creative Commons License Attribution 4.0 International (CC BY 4.0). increasing order of complexity: (i) spatial primitives (SP), namely parts / roles composing IS gestaltic entities; (ii) image schemas, taken from Johnson; and (iii) schematic integrations, namely image-schematic structures enriched with other type of knowledge or stimula e.g. emotions, forces etc. (see Section 2.1). Hedblom et al. [6, 8, 7] propose to taxonomically organize image schemas and analyse their combinatorial aspects. The last two theories difer, for instance, in their understanding of IS, and how they can combine or can be grouped. To formalize and unify these diferent perspectives on IS, we propose to utlize ontology modularization.

The Image Schema Abstraction And Cognition (ISAAC) ontology aims at being a formal harmonization of the above fundamental theories on image schemas. As an example of image schemas, let’s consider the expression I can’t get out of this bad situation: the “situation” is conceptualized as a CONTAINER, whose boundaries are blocking the free movement of some agent; this example activates the well established CONTAINMENT image schema, with the addition of a BLOCKAGE aspect. These IS are (gestaltically) composed at least by the CONTAINER and INSIDE and by the BLOCKED and BLOCKER spatial primitives. Based on this image schematic gestaltic nature, as described in detail in Section 3, ISAAC ontology uses a frame semantics approach to represent IS as frames, and its SP as necessary roles.

While their existence in natural language has been studied by means of corpus-based (e.g. [9, 10]) and machine learning methods (e.g. [11, 12, 13]), few approaches try to formalize image schemas (e.g. Image Schema Logic [7]), and to connect them to existing lexical resources. ISAAC constitutes an attempt to contribute towards this objective and at the same time provide an approach for harmonising varying theoretical perspectives on the same topic.

The paper is organized as follows: Section 2.1 outlines the state of the art about IS conceptualization and formalization; Section 2.2 describes the frame semantics approach adopted for ISAAC; Section 3 introduces the structure of ISAAC modular ontology, while more details can be found in Section 3.1, Section 3.2 and Section 3.3, focusing respectively on Johnson’s “Metaphors we Live By”[ 1 ], Mandler & Pagan-Cànovas [ 3 ] developmental theory and Hedblom’s et al. work on image-schematic taxonomy and combinatorial aspects [6, 8, 7]. In section 4 the ISAAC ontology is tested, showing possible cross-module inferences about the image-schematic structure of complex situations. Finally, Section 5 draws conclusions and discusses possible future developments.

2. Preliminaries

In the following sections we provide more details about Image Schemas and their formal structure according to diferent theories, as well as the introduction to a frame semantics approach, used here to formalise Image Schemas as frames and Spatial Primitives as their roles.

2.1. Defining image schemas

Perspectives on embodied cognition, the hypothesis that cognitive concepts are rooted in sensorimotor experiences with the world, range from symbolic (e.g. [14]) to fully embodied theories [15]. In the latter category, Lakof [ 2 ] and Johnson [ 1 ] propose image schemas. “Schema” refers to our capability to “conceptualize situations at varying levels of schematicity” [16] and “image” resonates the imagistic capability in the sense of “mental representation”, schematic gestalts that integrate information from various modalities [ 4 ], rather than purely visual images [ 2 ].

According to Johnson’s [ 1 ] famous definition, “an image schema is a recurring, dynamic pattern of our perceptual interactions and motor programs that gives coherence and structure to our experience” [ 1 ]. For instance, playing with shape puzzles1 as an infant, represents early experiences of spatial boundedness and CONTAINMENT. They are directly meaningful experiential gestalts, that is, they are internally structured compositions of parts to form coherent, uniform wholes. These repeatedly experienced structures are thought to shape higher-level abstract cognition, such as language. For instance, They are at a crossroad in their relationship [17] depicts the relationship as a PATH, on which participants travel eventually encountering some form of BLOCKAGE or capability to move towards diferent directions. By way of metaphorical projection, as in the famous case of the LOVE IS A JOURNEY conceptual metaphor, structures of the physical domain as source can be mapped onto some abstract target domain, e.g. the START (starting a relationship), PATH (being in love), and GOAL (as eventual ending point of the “relationship’s path”). Johnson [ 1 ] and Lakof [ 2, 18 ] provided numerous linguistic and sensorimotor examples as well as related high-level entailments without, however, fully formalising their theory.

To provide a more formal account, Hedblom et al. [6] propose to use the unified metalanguage Distributed Ontology, Modeling and Specification Language ( DOL) to represent shared gestalt structures of image schemas as a family, that is, a set of interlinked theories. Such a gestalt grouping of experiential structures implies a distinction between primitive and complex types. To this end, Mandler and Pagán Cánovas’ approach [ 3 ], rooted in developmental psychology, was adopted to distinguish spatial primitives, image schemas, and schematic integrations. Spatial primitives are said to be the very first preverbal building blocks infants form that quickly compose to more complex structures, the elements (parts) that compose the coherent unified ifnal structures (wholes). These wholes, or spatial events, built from spatial primitives, are image schemas. Finally, schematic integration refers to the combination of IS and SP with non-spatial elements, such as emotions. Hedblom et al. [6] take up this initial definition and depict spatial primitives as roles participating in the image schematic event. Thereby, imageschematic structures, either primitive or not, can be grouped based on experiential gestalt family resemblances. This initial formalisation is later extended as Image Schema Logic (ISL ) [7]).

2.2. Frame semantics approach

Frames have been widely applied in cognitive theories, and are defined as cognitive representations of prototypical and recurrent features of events or situations. Frame structures rely on frame semantics [19], where lexical units evoke frames, representing specific situations. For example, representing an apparently simple situation like being blocked in a trafic jam as a frame structure would require some necessary roles such as a moving agent (car), a path towards some direction (street), and some entity blocking the agent preventing its movement (another car); but some further external roles should be expressed, to make explicit the complex notion of “trafic jam”, like the duration of the “being-stuck-state”, the fact that the blocked agent is in turn the blocker of some other agent, and some optional roles could be added, like 1A game where objects of specific shapes have to be inserted into openings of the same shape the non-linearity of the path, the car crash that caused the trafic jam, the increasing bad mood of the drivers, the pollution caused by engines, etc.

Our approach relies on a representation of the three chosen image schema theories formalized as frames and roles, since Fillmore himself explicitly compares frames to other notions, such as experiential gestalt [20], stating that frames can refer to a unified framework of knowledge or a coherent schematization of experience. Thus, widely acknowledged FrameNet frames [21] as formalised [22] in Framester [23] provide a theoretically well founded, and lexically grounded validated basis, for commonsense knowledge patterns. Framester, in fact, other than providing a formal semantics for frames in a curated linked data version of multiple linguistic resources (e.g. besides FrameNet, WordNet [24], VerbNet [25], BabelNet [26], etc.), already hosts a cognitive layer which includes MetaNet [27], an ontology for cognitive metaphors, and ImageSchemaNet [28], which connects image schematic sensorimotor patterns to the above-mentioned linguistic resources. Framester can be used to jointly query (via a SPARQL endpoint2) the resources aligned to its formal frame ontology3.

3. ISAAC ontological modules

To introduce the ISAAC ontology4, we first present its individual modules, which include: (i) Johnson87 abbreviated to J87, modeled from [ 1 ]; (ii) a module on the work of Mandler and Pagán Cánovas [ 3 ] abbreviated to MPC; and (iii) Hedblom et al., abbreviated to HED, modeled from several works of Hedblom et al. [7, 29, 6]. Johnson’s [ 1 ] work ofers an embodied approach to a philosophy of cognition, where formality is often sacrificed in favor of a broader topic coverage. Mandler and Pagán Cánovas [ 3 ] use a cognitive approach of developmental psychology, and explicitly specify some theoretically essential elements, especially spatial primitives. Hedblom et al. [7, 29, 6] formalise the dynamics of compositionality among image schemas and spatial primitives in DOL, which we refactored into OWL 2 axioms and SWRL rules as presented below. All ontology components are annotated with a e x : b i b R e f annotation property, taken from [30], which is used to insert quotes of the original definitions from authors’ works, and keeps track of the origin, including author, text, year of publication of the resource, and page of the quote. The J87 module has a twofold purpose: a) provide philosophical assumptions used as theoretical grounding in the other ISAAC modules; b) keep track (also via e x : b i b R e f property) of the original implications of main concepts introduced by Johnson in a Semantic Web ontological structure. The frame semantics approach becomes particularly relevant in the MPC and HED modules, with the introduction of spatial primitives as components (roles) of the image-schematic structures (frames) and thanks to HED’s compositionality dynamics, IS family and IS profile formalization. 2Framester SPARQL endpoint is available at: http://etna.istc.cnr.it/framester2/sparql 3Framester Schema is available at: https://w3id.org/framester/schema/ 4The ISAAC ontology and all its modules are available here: https://github.com/StenDoipanni/ISAAC/tree/main/ISAAC_ontology_network 3.1. Johnson 1987 (J87 ) module J87 is the ontological module representing Chapters 1-5 of [ 1 ], leaving apart chapters on imagination and a general theory of meaning. The ontological representation follows a topdown approach and the main notable classes are the following: • j 8 7 : G e s t a l t S t r u c t u r e : “Gestalts” are described as “not unanalyzable givens or atomistic structures. They can be “analyzed”, since they have parts and dimensions, but any such attempted reduction will destroy the unity (the meaningful organization) that made the structure significant in the first place” [ 1 ]. • j 8 7 : I m a g e S c h e m a : annotated with all the diferent definitions in the source publication.

Image schemas are explicitly defined as “Image Schematic Gestalt Structures”, which is why this class is modeled as subclass of j 8 7 : G e s t a l t S t r u c t u r e . • j 8 7 : E n t a i l m e n t : this class represents a cluster of possible, probable, necessary or prototypical implications that an image schema might have. Johnson does not formalize or operationalize the following assumptions, therefore they have to be intended as top-down theoretical assertions without any ambition of logical formalism. For instance in [ 1 ] CONTAINER j 8 7 : e n t a i l s three entailments, which are modeled as j 8 7 : C O N T A I N E R _ E n t a i l m e n t instances: j 8 7 : L a w _ o f _ e x c l u d e d _ m i d d l e , j 8 7 : T r a n s i t i v i t y , and j 8 7 : N a t u r e _ o f _ n e g a t i o n . It is clear that even if, for the sake of coherence with the original source, they are represented at the same ontological level, these individuals are very diferent entities. In fact, j 8 7 : L a w _ o f _ e x c l u d e d _ m i d d l e states that everything is either P (in the container) or not-P (outside the container). j 8 7 : T r a n s i t i v i t y is listed as entailed by CONTAINER for the prototyping of the syllogism (, ) ∧ (, ) ⟹ (, ) based on the assumption that the containment relation is usually transitive, thus referring to transitivity of containment. j 8 7 : N a t u r e _ o f _ n e g a t i o n finally is instead listed because Johnson states that our ability to negate derives directly from our conceptualisation of categories as containers, so negating something is stating that it is not in some category and therefore, it is not containedIn some (metaphorical) container. • j 8 7 : G e s t a l t C r i t e r i o n : this class takes as instances those that are said to be the necessary criteria for a gestalt structure to be “emergent and salient” in our experience, and these criteria are j 8 7 : P e r v a s i v e , j 8 7 : S i m p l y - s t r u c t u r e d , j 8 7 : W e l l - s t r u c t u r e d and j 8 7 : W e l l u n d e r s t o o d . However, these criteria to recognise an image schema are neither formalised nor defined consistently.

Image schemas are modeled as subclass of gestalt structures, with the peculiarity of having “parts and dimensions” [ 1 ]. The relation of subsumption was utilised when an image schema was portrayed to be more specific, e.g. BALANCE subsumes AXIS_BALANCE, EQUILIBRIUM, POINT_BALANCE and TWIN_PAN_BALANCE. Considering the relational structure of this module a form of compositionality among image schemas is expressed by Johnson about PATH, for which the property d u l : h a s C o m p o n e n t is used to state PATH d u l : h a s C o m p o n e n t SOURCE, force_vector, GOAL and vector_tracing_a_path.5 5No graphical notation is used for some of these image-schematic structures since no precise type or exact nature is provided by Johnson, so using the graphical notation would mean committing to an idea without theoretical nor empirical grounding.

3.2. Mandler and Pagán Cánovas (MPC) module

Mandler and Pagán Cánovas [ 3 ] introduce crucial terminological and methodological distinctions taken up by many other works, especially that of spatial primitive and image schema. So MPC introduces the m p c : S p a t i a l P r i m i t i v e class that represents image-schematic conceptual building blocks.

MPC specifies Johnson’s vague “components” as spatial primitives and shifts the angle from a linguistic and philosophical to a psychological m p c : D e v e l o p m e n t a l P e r s p e c t i v e , which is an instance of the m p c : I S _ A p p r o a c h class.

MPC Classes In particular the MPC module includes the following main classes: • m p c : S p a t i a l P r i m i t i v e : “first conceptual building blocks formed in infancy” [ 3 ]. • m p c : I m a g e S c h e m a : “simple spatial stories build from spatial primitives” [ 3 ]. • m p c : S c h e m a t i c I n t e g r a t i o n : blending of spatial primitives or image schemas with nonspatial elements, e.g. emotion. • m p c : I S _ C o m p l e x i t y C r i t e r i o n : first preverbal conceptual understanding of infants and its development provide information on the most fundamental image schemas, i.e., the time of development of conceptual understanding specifies the complexity of the associated image schema. • m p c : I S _ C o m b i n a t i o n : has instance m p c : : A N I M A T E _ T H I N G , which combines m p c : A N I M A T E _ M O V E and m p c : T H I N G , however, which parts/qualities/roles are combined or whether it represents a coactivation of distinct image schemas remains unclear. • m p c : I S _ g r o u p i n g : clustering image schemas in groups (better structured in the HED module).

MPC Object Properties To declare relations among its classes the MPC module introduces important object properties, which we exemplify rather than completely list. For instance, m p c : h a s S p a t i a l I n p u t with domain m p c : S c h e m a t i c I n t e g r a t i o n and range m p c : S p a t i a l P r i m i t i v e or m p c : I m a g e S c h e m a is defined as a “process similar to what is called simplex network in Conceptual Integration Theory” [ 3 ] referring to the role that the unstructured element takes from the organizing frame structure imported from the input space. The inverse relation to this is m p c : i s T o p o l o g y P r o v i d e r F o r that, even without being a direct commitment, implies a similarity between image schema components and roles of a frame, operationalized in the HED module. The main commitment of this module is the m p c : m a k e s U s e O f object property, that has range m p c : S p a t i a l P r i m i t i v e and explicitly declares the nature of Johnson’s gestalt structure.

3.3. Hedblom (HED ) module

The HED module inherits the theoretical approach from the MPC one, and focuses on representing all the image schemas and spatial primitives specified in the listed publications. It provides a sound system for compositionality represented via object properties and SWRL rules, originally represented in the DOL language [6].

HED Classes One of the most central theoretical contributions in the HED module is h e d : I S _ F a m i l y , which attributes a specific name to the clustering of image schemas, in contrast to the generic m p c : I S _ g r o u p i n g . It explicitly declares an additional dimension: complexity. In contrast to the complexity criterion in MPC, it focuses on the complexity of compositionality, rather than the complexity of individual image schemas. Here the main classes used to formalize the introduction of new formal constraints: • h e d : I S _ C o m p l e x i t y : IS Complexity increases proportionally to the addition of spatial primitives, as shown in [6], and it is expressed in OWL 2 via the object property h e d : f i n e T u n i n g F o r . • h e d : I S _ F a m i l y : Image schemas consist of diferent “parts”; “These ‘parts’ can either be removed or added while still capturing the same basic image schema, generating what can be described as an image schema ‘family’ [6]. • h e d : I S _ F o r m : IS are here presented in two possible forms: static or dynamic. The “static” form denotes the notion of the possibility of things being in some way, i.e., the spatial co-location of entities and its configuration. The “dynamic” form equally denotes the co-location of entities with the addition of some form of movement of one or both entities [29]. • h e d : I S _ p r o f i l e : The IS profiles are defined as “groupings of image schemas that capture the spatiotemporal relationships related to particular events.”, namely e.g. the set of IS activated by some concept or event, or even sentence, description, situation etc. • h e d : I S _ C o m b i n a t i o n T y p e : this class is used to express the possible combination types, which is subclass of h e d : I S _ T r a n s f o r m a t i o n and takes as instances three types of combinations: (i) h e d : C o l l e c t i o n : a set of IS which do not alter the gestaltic properties of a particular spatio-temporal relationship, but together are able to represent a particular experiential structure; (ii) h e d : S t r u c t u r e d C o m b i n a t i o n : (previously h e d : S e q u e n c e ) it is similar to the collection but with the important addition of a sequential cause-efect relation; (iii) h e d : M e r g e : probably the most interesting one, since it is the combination of IS in such a way that gestaltic properties are altered [31].

To the above list, it is important to add the h e d : S p a t i a l P r i m i t i v e and h e d : S p a t i a l S c h e m a (IS) classes. Since the combinatorial nature of IS is a central contribution to the HED module, its object properties are presented in more detail.

HED Object Properties The relations between IS and SP are formalized via object properties; here we list the most relevant: • h e d : c o m b i n a t i o n T y p e : some IS has some combination type. • h e d : c o m b i n e s W i t h : some IS can combine with some other IS or with some SP to compose more complex structures. • h e d : f i n e T u n i n g F o r : This property expresses the increasing complexity of IS through the use of one or more SP. Some more basic IS use just one or few SP, while more complex ones are “fine tuning” the more basic ones, since they are more specialised and articulated in their compositional structure and meaning (e.g. linear path movements vs circular path ones) [6]. • h e d : g r o u p e d I n F a m i l y : some IS or SP is grouped in some IS family. Its inverse is h e d : g r o u p s . • h e d : h a s C o m b i n a t i o n E l e m e n t : some IS has combination element some other IS. More speciifcity, according to the combination type, can be expressed with its three subproperties: h e d : h a s C o l l e c t i o n E l e m e n t , h e d : h a s S e q u e n c e E l e m e n t and h e d : h a s M e r g i n g E l e m e n t . • h e d : h a s F o r m : some IS has form static or dynamic. • h e d : I S _ f a m i l y I n t e r s e c t i o n W i t h : two IS families can have an intersection when some IS is grouped in (at least) two families e.g. h e d : G O I N G _ I N is the intersection result of h e d : S O U R C E _ P A T H _ G O A L and h e d : C O N T A I N M E N T . • h e d : I S _ p r o f i l e : this property links some d u l : C o n c e p t or d u l : E v e n t to some IS profile, in turn the IS profile has some h e d : p r o f i l e P a r t i c i p a n t some IS.

• h e d : m a k e s U s e O f : some IS makes use of some SP or some other IS.

Furthermore, the HED module includes several SWRL rules to operate (i) property value assignment - to avoid so called “diamonds” [32] - and (ii) named individual inference rules, not possible with OWL expressivity.6

We exemplify the refactoring of the HED module from DOL to OWL 2 with SOURCE_PATH and SOURCE_PATH_GOAL. The object property h e d : f i n e T u n i n g F o r derives from a footnote in [6], and is represented in the HED module as a SWRL rule: h e d : m a k e s U s e O f ( ? x , ? y ) ∧ h e d : f i n e T u n i n g F o r ( ? z , ? x ) → h e d : m a k e s U s e O f ( ? z , ? y ) (1) asserting that if some entity ? x , (an image schema) h e d : m a k e s U s e O f some ? y , (a spatial primitive), and some entity ? z is declared as being more complex than ? x via the property h e d : f i n e T u n i n g F o r , then the knowledge that we infer by declaring that ? z is more complex than ? x is that it h e d : m a k e s U s e O f the same spatial primitives of ? x , while the knowledge that is declared is the unique spatial primitives that it h e d : m a k e s U s e O f . Since the distinction on the linguistic level is not always clear, an additional quality is added to image schemas: h e d : I S _ F o r m diferentiating h e d : S t a t i c from h e d : D y n a m i c . SWRL rules 2 and 3 are about h e d : S t a t i c and h e d : D y n a m i c IS form: ( h e d : h a s F o r m ( ? x , h e d : S t a t i c ) ∧h e d : i s S t a t i c F o r m F o r ( ? x , ? y ) ) → h a s F o r m ( ? y , h e d : D y n a m i c ) (2) ( h e d : h a s F o r m ( ? x , h e d : D y n a m i c ) ∧ h e d : i s D y n a m i c F o r m F o r ( ? x , ? y ) ) → h a s F o r m ( ? y , h e d : S t a t i c ) (3) In detail Axiom 2 states that if some ? x h e d : h a s F o r m h e d : S t a t i c and h e d : i s S t a t i c F o r m F o r some ? y , than ? y h e d : h a s F o r m h e d : D y n a m i c . Axiom 3 is the opposite of Axiom 2.

This module also provides a class for the grouping of image schemas activated when conceptualizing a complex event, action, etc., that is, h e d : I S _ p r o f i l e . One example taken from [31] is the “turducken”7. The h e d : T u r d u c k e n _ p r o f i l e is an instance of h e d : I S _ p r o f i l e , taking 6All the SWRL rules inferences are developed with the SWRLTab 2.0.11 Protégé-OWL development environment and tested with both HermiT 1.4.3.456 and Pellet reasoners in Protégé, version 5.5.0 7A dish with a chicken stufed inside a duck that in turn is stufed inside a turkey. as h e d : p r o f i l e P a r t i c i p a n t the image schema GOING_IN, SCALE, ITERATION and CONTAINMENT. A further novel element in the HED module is the class h e d : I S _ C o m b i n a t i o n T y p e , subclass of h e d : I S _ T r a n s f o r m a t i o n and taking as instances three types of image schema combinations [31], namely, h e d : C o l l e c t i o n , h e d : M e r g e and h e d : S t r u c t u r e d C o m b i n a t i o n . Axioms formalizing these three type of combinations and allowing further useful inferences are:

h e d : h a s M e r g i n g E l e m e n t ( ? x , ? y ) → h e d : c o m b i n a t i o n T y p e ( ? x , h e d : M e r g e ) h e d : h a s C o l l e c t i o n E l e m e n t ( ? x , ? y ) → h e d : c o m b i n a t i o n T y p e ( ? x , h e d : C o l l e c t i o n ) h e d : h a s S e q u e n c e E l e m e n t ( ? x , ? y ) → h e d : c o m b i n a t i o n T y p e ( ? x , h e d : S e q u e n c e ) h e d : h a s C o m b i n a t i o n E l e m e n t ( ? x , ? y ) ∧ h e d : h a s C o m b i n a t i o n E l e m e n t ( ? x , ? z ) ∧ o w l : d i f f e r e n t F r o m ( ? y , ? z ) → h e d : c o m b i n e s W i t h ( ? y , ? z ) h e d : c o m b i n e s W i t h ( ? x , ? z ) ∧ h e d : g r o u p e d I n F a m i l y ( ? x , ? y ) ∧ h e d : g r o u p e d I n F a m i l y ( ? z , ? k ) ∧ h e d : h a s M e r g i n g E l e m e n t ( ? h , ? x ) ∧ h e d : c o m b i n a t i o n T y p e ( ? h , h e d : M e r g e ) → h e d : I S _ f a m i l y I n t e r s e c t i o n W i t h ( ? y , ? k ) Axiom 4 states that if some ? x h e d : h a s M e r g i n g E l e m e n t some ? y , then the combination type of ? x is h e d : M e r g e . Axioms 5 and 6 state the same about the other two types of combination: h e d : C o l l e c i o n and h e d : S e q u e n c e . Axiom 7 allows the inference that, given an entity ? x and a number N of elements ? y , if ? x h e d : h a s C o m b i n i n g E l e m e n t more than one element, then each of them h e d : c o m b i n e s W i t h all the others. Finally, Axiom 8 formalizes IS families intersection: if ? x h e d : c o m b i n e s W i t h ? z and ? x is h e d : g r o u p e d I n F a m i l y ? y , while ? z is g r o u p e d I n F a m i l y ? k , and an entity ? h , being the result of a merging process, h e d : h a s M e r g i n g E l e m e n t ? x , then we can say that there is an occurrence of a h e d : I S _ f a m i l y I n t e r s e c t i o n W i t h ? y (? x ’s family) and ? k (? y ’s family).

The Hedblom module, building on the conceptualization of moving objects [ 3 ], addresses spatial primitives’ compositionality, whose addition or subtraction determines the structure of a specific image schema (e.g. MOVEMENT_ALONG_A_PATH + START_PATH = SOURCE_PATH).

3.4. ISAAC integration module

Finally the ISAAC module imports all the previous ones importing therefore all the abovementioned concepts and inferences, and introduces, to extend inference capabilities, three more axioms based on, and derived directly from, previous ones, allowing some inferences described in Section 4.

h e d : p r o f i l e P a r t i c i p a n t ( ? x , ? y ) ∧ h e d : h a s C o m b i n a t i o n E l e m e n t ( ? y , ? z ) →

h e d : p r o f i l e P a r t i c i p a n t ( ? x , ? z ) h e d : p r o f i l e P a r t i c i p a n t ( ? x , ? y ) ∧ h e d : c o m b i n a t i o n T y p e ( ? y , ? k ) → i s a a c : s t r u c t u r e d A c c o r d i n g T o ( ? x , ? k ) (4) (5) (6) (7) (8) (9)

Axiom 9 states that, given some ? y being a profile participant to some IS profile ? x , and having some combination element ? z , then the IS profile ? x will have as profile participant also ? z . Axiom 10 states that, given some ? x having as profile participant some ? y , if ? y has some combination type ? k , then the IS profile ? x will be structured according to ? k . To conclude, Axiom 11 states that, given some ? x , if it includes in profile some ? y which is of type m p c : S c h e m a t i c I n t e g r a t i o n (in particular here is shown the axiom for m p c : E M O T I O N , but there is a parallel one for m p c : F O R C E ), then the entity ? x inherits its topology from that specific schematic integration.

4. Evaluation

To test possible inferences, we revisit the example scenario mentioned before: a complex event like “being blocked in a trafic jam”. We want to be able to infer the IS playing a role in the event, and we want to be able to express information about the structure of the event as mentioned in Section 2.2. Therefore, we create an individual: “being_blocked_in_a_trafifc_jam_profile” ( i s a a c : b t j _ p r o f i l e ) which we could assert, by commonsense reasoning that will involve as participant some h e d : B L O C K A G E , and we want to additionally express the above mentioned idea that there is some emotion involved, therefore we introduce another individual: i s a a c : A n g e r r d f s : s u b C l a s s O f m p c : E M O T I O N (thanks to the expressivity in OWL 2 it is possible to have m p c : E m o t i o n as intensional individual in the MPC module and at the same time as extensional class of situations in the ISAAC module).x With these assertions, performing the reasoning automatically, we get the following inferences: 1. i s a a c : b t j _ p r o f i l e involves as participant some h e d : C O N T A C T , h e d : S O U R C E _ P A T H _ G O A L and h e d : O B J E C T ; 2. i s a a c : b t j _ p r o f i l e is structured according to some h e d : S t r u c t u r e d C o m b i n a t i o n ; 3. i s a a c : b t j _ p r o f i l e inherits its topology from m p c : E M O T I O N .

About inference number 1: In this naive example, it could be asserted that there is no physical “contact” among entities involved in a trafic jam, which is true, but the blockage happens exactly due to the desire to avoid the physical contact. In some sense the spatial area of cars involved in the event is larger than the mere physical shape, and this can be compliant to the idea of having a “contact” between what we could call the safe space of the entity involved. Furthermore, the inclusion of h e d : O B J E C T is almost always the case, confirming some perplexities in including a dedicated IS, since its ontological status would be more or less equivalent to o w l : T h i n g . The inference including h e d : S O U R C E _ P A T H _ G O A L instead is pretty accurate, since a trafic jam is a h e d : B L O C K A G E situation happening on the path between some source point and a target goal.

About inference number 2: since h e d : B L O C K A G E has combination type h e d : S t r u c t u r e d C o m b i n a t i o n the i s a a c : b t j _ p r o f i l e inherits this structure. Note that this does not restrict the amount of combination types which can structure a h e d : I S _ p r o f i l e , since this detail depends on the granularity of detail considered. We can consider the inference plausible since the gestaltic properties of each and any of the IS involved are not altered.

Considering finally inference number 3: the import of diferent modules coming from diferent theories in an integration module, namely, the ISAAC module, allows to reuse classes and concepts introduced in one module while making it possible and harmonizing available inferences. This is realizable since the m p c : S c h e m a t i c I n t e g r a t i o n is defined as the class of situations in which some IS co-occurr with some entity diferent from sensorimotor patterns, but still relevant for inner investigation, according to the embodied cognition approach, in particular Emotions and Forces (cf. Section 3.2).

5. Conclusions

This work presented ISAAC, the Image Schema Abstraction And Cognition ontology, which is a module aiming at integrating and broadening possible inferences coming from the formal transposition of existing theoretical systems regarding image schemas and embodied cognition. Future developments include the introduction of more theories and elements, as well as the formal refinement of some dynamics which are still left partially obscure, namely further details about dynamics of compositionality, I S _ P r o f i l e internal organization and taxonomical relations among IS. Furthermore, thanks to the integration of these modules it is possible to envision some (i) theoretical clarification, such as a more formal investigation of situations satisfying pure top-down classes descriptions such as j 8 7 : E n t a i l m e n t , demonstrating which assumptions are logically sound and in which world use-cases scenarios; and (ii) interesting investigation of situations satisfying the m p c : S c h e m a t i c I n t e g r a t i o n class, namely a formal analysis of the frame and role structure of e.g. some complex event or action, including elements rooted in the inner self, like emotions, values, and embodied experiences of external physical laws, like forces and sensorimotor cognitive patterns.

Acknowledgments

This work is supported by the H2020 project TAILOR: Foundations of Trustworthy AI – Integrating Reasoning, Learning and Optimization – EC Grant Agreement number 952215. [6] M. M. Hedblom, O. Kutz, F. Neuhaus, Choosing the right path: image schema theory as a foundation for concept invention, Journal of Artificial General Intelligence 6 (2015) 21–54. [7] M. M. Hedblom, O. Kutz, T. Mossakowski, F. Neuhaus, Between contact and support: Introducing a logic for image schemas and directed movement, in: Conference of the Italian Association for Artificial Intelligence, Springer, 2017, pp. 256–268. [8] M. M. Hedblom, M. Pomarlan, R. Porzel, R. Malaka, M. Beetz, Dynamic action selection using image schema-based reasoning for robots (2021). [9] A. Papafragou, C. Massey, L. Gleitman, When English proposes what Greek presupposes:

The cross-linguistic encoding of motion events, Cognition 98 (2006) B75–B87. [10] J. A. Prieto Velasco, M. Tercedor Sánchez, The embodied nature of medical concepts: image schemas and language for pain., Cognitive processing (2014). doi:Path10.1007/s10339-0130594-9. [11] D. Gromann, M. M. Hedblom, Body-mind-language: Multilingual knowledge extraction based on embodied cognition., in: AIC, 2017, pp. 20–33. [12] D. Gromann, M. M. Hedblom, Kinesthetic mind reader: A method to identify image schemas in natural language, in: Proceedings of Advancements in Cogntivie Systems, 2017. [13] L. Wachowiak, D. Gromann, Systematic analysis of image schemas in natural language through explainable multilingual neural language processing, in: Proceedings of the 56th Linguistic Colloquium, Peter Lang Group, Forthcoming. [14] A. Clark, Being there: Putting brain, body, and world together again, MIT press, 1998. [15] L. Meteyard, S. R. Cuadrado, B. Bahrami, G. Vigliocco, Coming of age: A review of embodiment and the neuroscience of semantics, Cortex 48 (2012) 788–804. [16] R. W. Langacker, Foundations of cognitive grammar: Theoretical prerequisites, volume 1,

Stanford university press, 1987. [17] G. Lakof, Metaphor and war: The metaphor system used to justify war in the gulf, Peace

Research (1991) 25–32. [18] G. Lakof, M. Johnson, et al., Philosophy in the flesh: The embodied mind and its challenge to western thought, volume 640, Basic books New York, 1999. [19] C. J. Fillmore, Frame semantics, in: Linguistics in the Morning Calm, Seoul: Hanshin, 1982, pp. 111–138. [20] G. Lakof, M. Johnson, Metaphors we live by, University of Chicago press, 1980. [21] C. J. Fillmore, Frames and the semantics of understanding, Quaderni di semantica 6 (1985) 222–254. [22] A. G. Nuzzolese, A. Gangemi, V. Presutti, Gathering lexical linked data and knowledge patterns from framenet, in: Proceedings of the sixth international conference on Knowledge capture, ACM, 2011, pp. 41–48. [23] A. Gangemi, M. Alam, L. Asprino, V. Presutti, D. R. Recupero, Framester: a wide coverage linguistic linked data hub, in: European Knowledge Acquisition Workshop, Springer, 2016, pp. 239–254. [24] G. A. Miller, WordNet: An electronic lexical database, MIT press, 1998. [25] K. K. Schuler, VerbNet: A broad-coverage, comprehensive verb lexicon, University of

Pennsylvania, 2005. [26] R. Navigli, S. P. Ponzetto, Babelnet: Building a very large multilingual semantic network, in: Proceedings of the 48th annual meeting of the association for computational linguistics, 2010, pp. 216–225. [27] A. Gangemi, M. Alam, V. Presutti, Amnestic forgery: An ontology of conceptual metaphors, in: S. Borgo, P. Hitzler, O. Kutz (Eds.), Proceedings of the 10th International Conference, FOIS 2018, volume 306 of Frontiers in Artificial Intelligence and Applications , IOS Press, 2018, pp. 159–172. URL: https://doi.org/10.3233/978-1-61499-910-2-159. [28] S. De Giorgis, A. Gangemi, D. Gromann, Imageschemanet: Formalizing embodied commonsense knowledge providing an imageschematic layer to framester, Semantic Web Journal, forthcoming (2022). [29] M. M. Hedblom, D. Gromann, O. Kutz, In, out and through: Formalising some dynamic aspects of the image schema containment, in: S. Bistarelli, M. Ceberio, F. Santini, E. Monfroy (Eds.), Proceedings of the Knowledge Representation and Reasoning Track(KRR) at the Symposium of Applied Computing (SAC), 2018, pp. 918–925. [30] S. De Giorgis, A. Gangemi, Exuviae: an ontology for conceptual epistemic comparison, in: Graphs and Networks in the Humanities 2022 - Proceedings of the 6th International Conference, Amsterdam, Netherlands, 2022. Accepted for publication. [31] M. M. Hedblom, O. Kutz, R. Peñaloza, G. Guizzardi, Image schema combinations and complex events, KI-Künstliche Intelligenz 33 (2019) 279–291. [32] R. Hoekstra, J. Breuker, Polishing diamonds in owl 2, in: International Conference on Knowledge Engineering and Knowledge Management, Springer, 2008, pp. 64–73.

References

[1] M. Johnson , The Body in the Mind: The Bodily Basis of Meaning, Imagination, and Reason , The University of Chicago Press, Chicago and London, 1987 .

[2] G. Lakof , Women, Fire, and Dangerous Things . What Categories Reveal about the Mind , The University of Chicago Press, 1987 .

[3] J. M. Mandler , C. Pagán Cánovas , On defining image schemas , Language and cognition 6 ( 2014 ) 510 - 532 .

[4] L. Talmy , The fundamental system of spatial schemas in language , in: B. Hampe , J. E. Grady (Eds.), From perception to meaning: Image schemas in cognitive linguistics , volume 29 of Cognitive Linguistics Research, Walter de Gruyter, 2005 , pp. 199 - 234 .

[5] B. Hampe , Image schemas in cognitive linguistics: Introduction, From perception to meaning: Image schemas in cognitive linguistics 29 ( 2005 ) 1 - 14 .