In this extended abstract, we present the Diagrammatic Image Schema LanDgIuSLa)g.eD(ISL is a structured visual language to represent image-schematic relationships in diferent situations and linguistic expressions. Building on previous visualisation research, as well as how image schemas can be ontologically organisDeIdS,L ofers a more systematic representation language for abstract concepts than previously possible. In particular, we discuss some of the underlying motivations and design decisions forDISL and introduce the conceptual primitives that are covDerISeLd. bTyheir use is illustrated with the classic example oOfbject_Into_Container.
Image schemas are mental patterns learned from perceptual experiences that map embodied, spatiotemporal relationships onto conceptual constru1c].tiDoensspi[te their name, image schemas are not images, but cognitive gestalts whose cognitive structure can be visualised through mental imagery and, thus, can be illustrated diagrammatically. They capture the conceptual information of relationships between objects and their environments with commonly mentioned examples being concepts suchCaosntainment, Support andSource_Path_Goal.
In cognitive linguistics, the underlying information in image schemas is occasionally explained with the help of illustrations, which show the salient features of a given image schema. Notable mentions are the work of Johnson’s introductory illustr1a],ttihoensim[age-schematic drawings by Mandler2[], Hurtienne et al3.][and Talmy’s visual representations of force-dynam4]i,cs [ and the rather comprehensive visual grammar used to describe linguistic relationships by Langacker 5[]. Work like these, demonstrate the desire for a visual language to illustrate the semantic content of the image schemas. However, it also showcases the complexity of such endeavours as diferent representation methods more often than not are visually disjoint. Additionally, these representations of image schemas in the literature are typically ad hoc in the sense that there appear to be no general principles that guide these illustrations.
To address this issue, we develop the Diagrammatic Image Schema LangDuIaSgLe).(Used for the diagrammatic representation of image schemas, it focuses on systematising the visual combinations rulesD. ISL takes inspiration from previous visualisations of image schemas and combines it with the research on how image schemas can be divided into compositional primitives 6[] and the research that showcase how based on the primitives the image schemas can be structured into hierarchical family gr7,a8p]h. sT[ogether this foundation ofers a more systematic and compositional methodology to represent both individual image schemas as well as the events that take place with them.
In order to represent the image schemDaISs,L defines a set of symbols for the representation
of some conceptual primitives, the recurring building blocks of image schemas, and further
defines principles how to compose an image schema from these components into spatial stories.
This means thaDtISL can be used to analyse complex events and diagrammatically show how
diferent image schemas interact with one another in real-life scenarios. We believe this to be
beneficial for many areas, not the least in research on cognitive interpretations of visual
imageschematic metaphors (e.g9][), research on spatial expressions and configurations (1e0.g]). a[nd
research in cognitive robotics on events and action segmentation based on image-schematic
reasoning (e.g.1[
The purpose of this paper is to provide a preliminary view of the approach we have taken, show a selection of conceptual primitives that we have chosen, show their diagrammatic representation, and illustrate their use with an example.
Human conceptualisations rely partially on recurring and embodied patterns of
spatiotemporal relationships, which are often referred tiomagse schemas [
Image schemas are representations of simple spatial events using the primitivOesb,-e.g., ject_Into_Container.
Schematic integrations are conceptual representations to include non-spatial elements, by projecting feelings or non-spatial perceptions to blends structured by image schemas, e.g., Blocked_Motion.
This distinction allows to represent image schemas as temporal stories with diferent ‘parts.’ However, many traditional image schemas do not represent temporal stories in that they contain an (obvious) change over timSeu. pport is one such example, that while a temporal dimension is needed to determine the physical stability of the supporting object, it is still possible to identifySupport in an instance. To account for these ‘static manifestations of image schemas’ we extend the classification i1n3[] as follows: Primitive amalgamation The integration of several conceptual primitives into a whole representing a static combination of entities, their relationships and attrSiubupptoers,te..g., The diference between image schemas and conceptual primitives is of particular interest to the work in this paper, as it relies on how spatial primitives provide the elements that are used to create thDeISL diagrams. However, as the three examples for spatial primitives mentioned above (namelyC,ontainer, Contact, Open) illustrate, conceptual primitives are a divers group of structures, which play diferent roles within image schemas. If we consider image schemas as kind of ‘spatial stories’, then some conceptual primitives play the role of the ‘characters’, or predicates in linguistics, (e.g.,Oabnject and aContainer), some conceptual primitives describe the relationships between the ‘characters’ (eO.gb.j,etchteis Contained inside the Container), and some attributes of the ‘characters’ (e.gC.,otnhteainer is Open). The image schema unfolds as ‘characters’ enter or leave the stage, and their relationships and attributes change over time. Since the diferent roles of the conceptual primitives determine their roles in the diagrammatic representation, we suggest to further classify them as follows: Independent primitives may occur in an image schema independently of other primitives, e.g., Object, Container, Path, Spatial Region.
Relational primitives define the relationship between two (or more) independent primitives, e.g., Contact, Link, Contained. They are dependent on their relata in the sense that they cannot be present without the independent primitives they relate.
Attributive primitives define attributes of another conceptual primitiveO,ep.egn.,/Closed are attributes oCfaontainer’s openings andMoving andAt_Rest are attributes of Objects.
Independent, relational and attributive primitives provide the basic building blocks for composing primitive amalgamations, image schemas and schematic integrations.
Our diagrammatic representation of image schemas and schematic integrations follows
inspiration from work1[
In this section, we present an overview of the conceptual primitives that we currently selected forDISL and their symbolic representations.
Independent conceptual primitives are the cognitive primitives that can be added to a panel without requiring the existence of another primitive.
Figure1 contains an overview of the symbolic representations of these independent cognitive
primitives. Following convention (e1.g]).,[we represent objects as shapes with solid matter by
adding colours and borders, Ptahteh is a line, following Johnson’s and Mandler’s depictions
[
Relational conceptual primitives express the relationship between two independent conceptual primitives. A basic spatial primitiveLoiscation, which is the relationship between some object and the region of space it occupies within the context of a scene (including other objects). Location is represented within a panel by the way the object is located within the frame of the panel in the context of other independent conceptual primitives (e.g., objects and paths). In Figure2, some relational primitives are depicted.
In Figure3, the attributive primitives are depicted as ‘crowns’ for independent primitives. The focus lies on discussing attributes that are related tMooetiitohne,rthe embodied experience of force acting on the body, the force-dynamic primUitmipvhe, or attributesCoofntainers.
Motion is the attributional conceptual primitive of motion of an object in some directed form. (Undirected motion, e.g., a trembling animal, would not instManottiiaotne.) Motion is represented by an arrow, which indicates the trajectory.
Figure4 illustrates howDinISL the representation of conceptual primitives are combined to represent image schemas diagrammatically. Individual scenes are represented by panels, which represent amalgamations of several conceptual primitives. Following Western conventions, the order of the scenes, from left to right, represent temporal succession.
The first scene contains onOebject, oneContainer and onePath with an end (an attributive primitive). TheContainer has an open border. The spatial relations between the primitives are represented by their representation in the scene, in particuOlbajre:ctthaend theContainer are disconnected, tOhebject is on thePath, and the end of thPeath is contained within the Container. Further, thOebject has the image schemaMotion_Along_Path, in the direction of the end of the path. Thus, the first panel contains symbols for three independent primitives and three attributional primitives. The next two panels represent other amalgamations that difer from the previous one by significant qualitative changes of attributes of the independent primitives or their spatial relations. In the second panOeblj,etchtecrosses the boundary of theContainer. In the third panel, the object is no longer moving oPnatthhbeut is at rest contained within tChoentainer. Together, the whole diagram represents the image schema Object_Into_Container.
Note thaOtbject_Into_Container leave much information unspecified. E.g., it is possible that thCeontainer is in motion or at rest, thatOtbhject is moving on its own or that it is caused to move, that tChoentainer is occupied or that it is full at the end of the process. By adding additional information in the form of additional primitives, one can refine the information present in the image schema. For instance, one could rOebfinjeect_Into_Container by specifying the movement of thCeontainer asCaused_Motion in panels 1 and 2 and specifying the container aFusll in the third panel. This would yield an image-schematic representation of something likeContainer_is_Filled.
This extended abstract introduced some of the visual components of the Diagrammatic Image Schemas LanguageD,ISL. We illustrated its use with a help of an example. In the full paper on DISL, we are going to provide rules that guide the combination of conceptual primitives into primitive amalgamations, image schemas and schematic integrations. Further, we will discuss the intended semantics of the symbolic representations of the primitives, and extend our approach to include some important force primitives. Another important aspect that we did not address in this abstract is diagrammatic inferencDesISwL.itNhamely, how it follows in Figure4 that thCeontainer in the third panel is not empty. This too will be addressed in more extended work. [8] M. M. Hedblom, Image Schemas and Concept Invention: Cognitive, Logical, and Linguistic
Investigations, Cognitive Technologies, Springer Computer Science, 2020. [9] J. M. Cunha, P. Martins, P. Machado, Using image schemas in the visual representation of concepts., in: TriCoLore (C3GI/ISD/SCORE), 2018. [10] U.-J. Rüetschi, S. Timpf, Using image schemata to represent meaningful spatial configurations, in: OTM Confederated International Conferences” On the Move to Meaningful Internet Systems”, Springer, 2005, pp. 1047–1055. [11] K. Dhanabalachandran, V. Hassouna, M. M. Hedblom, M. Küempel, N. Leusmann, M. Beetz, Cutting events: Towards autonomous plan adaption by robotic agents through imageschematic event segmentation, in: Proceedings of the 11th on Knowledge Capture Conference, 2021, pp. 25–32. [12] M. Pomarlan, J. A. Bateman, Embodied functional relations: A formal account combining abstract logical theory with grounding in simulation, in: Formal Ontology in Information Systems: Proceedings of the 11th International Conference (FOIS 2020), volume 330, IOS Press, 2020, p. 155. [13] J. M. Mandler, C. P. Cánovas, On defining image schemas, Language and Cognition 6 (2014) 510–532. [14] M. M. Hedblom, O. Kutz, R. Peñaloza, G. Guizzardi, Image schema combinations and complex events, KI-Künstliche Intelligenz 33 (2019) 279–291. [15] T. Akimoto, Cogmic space for narrative-based world representation, Cognitive Systems Research 65 (2021) 167–183.