Embodied Cognition and cognitive metaphors theory take their origin from our use of language: sensorimotor triggers are disseminated in our daily communication, expression and commonsense knowledge. In this work, we extend the existing ImageSchemaNet ontology, an ontology representing knowledge about image schemas, focusing on a specification of the notion of “object”: Substance. The investigation is intended as an extension of ImageSchemaNet, extending the Image Schematic layer developed on top of FrameNet and integrated in the Framester resource. Finally, this methodology allows the extraction of sensorimotor triggers for Substance from WordNet, VerbNet, MetaNet, BabelNet and many more.
Image schemas (IS) are conceptual structures proposed within the framework of embodied
cognition. They capture sensorimotor experiences and influence abstract cognition, including
commonsense reasoning and the semantic structures of natural language (see e.g. Mandler
and Hampe [
For instance, in the expression “They’re filled with jealousy,” or “She’s overflowing with love,” the emotions are conceptualized as amorphous entities occupying a physical space, restricted in movement by aContainer, which is the body. This is a well-established conceptual metaphor, ARE SUBSTANCES, MONEY IS A SUBSTANCE, QUALITIES ARE SUBSTANCES, https://stendoipanni.github.(iSo./De Giorgis);https://www.unibo.it/sitoweb/aldo.gang(eAm.iGangemi)
© 2023 Copyright for this paper by its authors. Use permited under Creative Commons License Atribution 4.0 International (CC BY 4.0). TAXATION IS FORCEFUL REMOVAL OF A SUBSTANCE, TIME IS A SUBSTANCE, andWEALTH IS A SUBSTANCE.
While the presence of image schemas in natural language has been investigated using corpusbased [8, 9] and machine learning method1s0,[ 11, 12], there are few formal approaches to operationalize image schemas (e.g., Image Schema Log6ic]); [and connect them to existing resources for semantic representation.
Here, the term ”image schema profile” is employed as defined in1[
hTe approach adopted in this work is derived directly from Fillmore’s Frame Semantics.
Image Schemas are in fact explicitly defined by Johnso3n]a[s internally structured gestalts,
that are, composed of spatial primitives (SP) that make up more complex image schemas as
unified wholes of meaning 4[
Frames are schematizations of recurrent situations, requiring a certain number of necessary
elements (namely semantic roles) participating to the situation to occur. Frame Semantics
is described more in detail in Secti2o.1n. ImageSchemaNet is the ontology (formal and
explicit representatin of knowledge of a certain domain) for Image Schemas. In the context of
ImageSchemaNet each image schema is modeled in OWL2 synstax as a frame with a certain
number of roles. Currently ImageSchemaNet covers the following IS:
• Containment: an experience of boundedness, entailing an interior, exterior and a
boundary [
hTe formal representation oSfubstance is important from a theoretical perspective since it is in some way related to the notion of “Mass-Count” introduced in Lakof & Jo1h8n]s,oannd[ it is documented by Hurtienn1e9][ in the Image Schema Catalogue (ISCAT) repository.
Formalizing knowledge about the concepStuobfstance would improve commonsense knowledge extraction, allowing possible automatic inferences such as: if there is an entity which triggerSsubstance, it needs to be spatially located in some soCrotnotfainer or at least boundedSurface. Furthermore, automatic detection of substance could provide ways to detect conceptual metaphors mapping multiple domains (such as Emotions, Money, Time, etc.) on the Substance one. Finally, automatic detecStuibosntance could find useful integration in works such as [20], exploring afordances and behavior of liquid entities.
hTis work preludes some research questions such as: • Is Substance an image schemaper se? Or is it a property omfater which determines the typology of aOnbject as being of type “Mass-Count”, and consequently determining its afordances? • What are the defining properties Soufbstance? (For example its mereology, including homeomery, homogeneity, arbitrary divisibility, etc.) hTis work is a preliminary investigation to provide an answer about the ontological nature ofSubstance, therefore, in this perspective, it is not interesting nor useful trying to provide a top-down formalization which would not reflect real world occurrences. For this reason this work focuses on populating a knowledge graph of semantic trigSguebrsstoafnce, namely, real world occurrences, to foster future meaningful clustering and investigations of subtypes and properties restrictions. In the next paragraph we provide details about the adopted approach.
In this section we provide details about current state of the art about the domain of image
schemas. Firstly we mention relevant works in the image schema domain, while in 2S.e1ction
we describe previous works modeling image schemas from an ontological perspective.
hTere is no clear agreement on “Substance” as Image Schema or Spatial Primitive.
Studies have been developed aboOutbject by Peña [21], and Langacker2[
Cienki [23] describes theObject image schema as:
An object is a material thing which we can see and feel. We may think of an object as a discrete item.
In particular thOebject image schema has been the focus of Santibanez w2o4r]k,a[nd it is described as: hTe OBJECT image-schema is experientially grounded in our physical and social interaction with our own bodies and with other discrete entities in the world. More recently, Szwedek25[] defines an Object as: […] An OBJECT schema can be defined as mater, with density as a fundamental property, in some bounded form.
hTe Substance IS that we examine in this work seems to difer from Cienki and Santibanez’ Object for “being a discrete item”. Consider as example the famous Sorite’s paradox, for which a pile of sand remains a pile of sand if one single grain of sand is taken of, but if we continue grain by grain, there is no clear threshold determining which is the last grain of sand that can be taken away, for that entity to remain a pile of sand. FurthSeurbmstoarne,ce seems to difer also from Szwedek’Osbject definition, for its specification of being “in some bounded form.” hTe Substance IS has been documented by Hurtienn1e9][, and it is reported in the Image Schema Catalogue (ISCA1T.) 1An updated and upgraded version is available as Image Schema Repositoryhthteprse:/:/github.com/dgromann/ ImageSchemaRepository
hTerefore, in this work our focus is on the notion of “Substance” as specificatiOobnjeocft lacking the restriction of some bounded form. We propose to modSeulbtsthaence IS as a semantic frame (described in Secti2o.1n) subclass of thOebject class in ImageSchemaNet. 2.1. Framester, ImageSchemaNet and the Frame Semantics Approach Frames in a most general notion are (cognitive) representations of typical features of a situation. Fillmore’s frame semantic2s6[] has been most influential in conjugating linguistic descriptions and characterisation of related knowledge structures to describe cognitive phenomena. Words, multi-words expressions, and phrases are associated with frames based on the common scene they evoke. A resource formalizing this theory is Fram2e7N],eitn[which frames are explained as situation types.
Fillmore explicitly compares frames to other notions, such as experiential2g8e],stalts [ stating that frames can refer to a unified framework of knowledge or a coherent schematization of experience. Thus, widely acknowledged frames provide a theoretically well-founded and practically validated basis for commonsense knowledge paterns.
Framester2[9, 30] is a linked data hub that provides a formal semantics for f3r0a],mbeass[ed
on Fillmore’s frame semantic2s6][. In Framester semantic3s0[]
observed/recalled/anticipated/imagined situations are occurrences of frames. It creates/reengineers linked data versions
of linguistic resources, such as WordN31e]t, O[ntoWordNet3[
ImageSchemaNet is a formal and re-usable representation of image schemas as Semantic Web technology in form of an ontological layer. It presents a formal representation of image schemas as a new layer of the Framester hub. Since a major flaw in current image schema theory was the lack of agreement about the lexical coverage of image schemas, ImageSchemaNet introduces an image-schematic layer linking IS and SP to FrameNet frames and frame elements, WordNet synsets (sets of contextual synonyms) and word supersenses, VerbNet verbs, etc., thereby creating a formal, lexicalized integration of cognitive semantics, enactive theories, and frame semantics. Currently, ImageSchemaNet provides lexical coverage and formalization for the following six image schemaSso:urce_Path_Goal, Containment, Center_Periphery, Part_Whole, Support, andBlockage.
Framester can be used to jointly query (via a SPARQL end3p)oainlltthe resources aligned to its formal frame ontol4o.gFyramester has been used37[] to formalize the MetaNet resource of conceptual metapho5,rbsased on FrameNet frames as metaphor sources and targets (framebased), as well as to uncover semantic puzzles emerging from a logical treatment of frame-based metaphors. 2hTe MetaNet schema in Framester’s OWL is athttps://w3id.org/framester/metanet/sch.ema/ 3http://etna.istc.cnr.it/framester2/sparql 4hTe Framester Schema is available ath: ttps://w3id.org/framester/schema/ 5hTe MetaNet schema in Framester’s OWL is athttps://w3id.org/framester/metanet/sch.ema/
ProposingSubstance as a specification ofObject in ImageSchemaNet requires some steps to populate the knowledge graph of semantic triggers, as described in the following. Adopting the frame semantics approach as mentioned above means that every lexical unit referring to a portion of reality that can only be understood having in mind some notion of “substance” will be a trigger for tShuebstance image schema. We proceed describing the knowledge graph population step by step, as shown in Figu1.rAell the yellow rectangles are classes of entities, coming from multiple semantic web resources, while blue ovals are SPARQL queries, provided as additional mater6ial.
Manual Lexical Units Selection The initial step in the process involves a manual selection of a limited set of lexical units that are directly related to the conceptual frame being constructed, in this case: Substance. We call this set “Starting Lexical Material” (SLM). The SLM set is 6hTe SPARQL queries are available here:https://github.com/StenDoipanni/QUOKKA selected manually considering tSuhbestance examples in the ISCAT repository, and then further expanded using the WordNet resource, which can be accessed through its online user interfac7e. This expansion aims to leverage the relationships of hyponymy and synonymy among terms. By manually selecting an initial set of lexical units and expanding it using WordNet, we can lay the foundation for building a robust conceptual frame for the desired domain.
All the following steps are performed via SPARQL queries to the Framester e8ndpoint. ConceptNet-driven triggering Some ConceptNet relations are used to gather knowlegde, as shown in Figur1e. By leveraging these ConceptNet relations, we can explore the interconnectedness between lexical units and concepts, expanding the initial set of terms and enriching the conceptual frame construction process. In F1igiutries shown as a lexical unit like “substance” leads to a more specific entity in ConceptNet, suchcna:snanosubstance.
WikiData lexical triggering The WikiData lexical triggering step relies on the entities obtained from the ConceptNet-driven triggering process. The Starting Lexical Material (SLM) set serves as the input variable to retrieve all corresponding WikiData entries1. tInheFigure individual shown iwsiki:mixture.
DBpedia factual triggering In parallel with the WikiData lexical triggering step, it is possible to retrieve entities aligned with the DBpedia resource, which provides factual grounding for the domain knowledge base.
Frame-driven triggering In a separate branch, as depicted in Fig1u, rtehe selected lexical units from the Starting Lexical Material (SLM) set are examined to determine if they serve as lexical triggers for existing FrameNet frames. This step aims to leverage pre-existing frames that may partially overlap with the desired domain or provide more specific or general situational schematization. In this case are shown and declared as triSgugbesrtsafnocre the fs:Substance andfs:Amalgamation frames.
Frame element-driven triggering This step pertains to a scenario for which certain aspects of Substance may already be addressed by existing frames, enabling the adoption of the established structure of formalized semantic roles, where frame element activation revolves around the activation of semantic roles associated with the occurrence of a certain situation, i.e., a Frame occurrence, within the specified domain.
FrameNet Lexical Units triggering This step naturally follows from the preceding paragraphs and the adoption of a frame structure. By accessing the online user interface of the FrameNet resourc9,eit becomes apparent that certain lexical units are identified as evoking 7WordNet online resource is available hhetrtep:s://wordnet-rdf.princeton.edu/ 8hTe Framester endpoint is available hereh:ttp://etna.istc.cnr.it/framester2/sparql 9hTe FrameNet online user interface can be foundhatttps://framenet.icsi.berkeley.edu/fndrupal/about specific frames. To fully comprehend the semantics of these lexical units, a common understanding of the system of semantic relations and the contexts in which they convey their meanings is required. The purpose of this SPARQL query is to incorporate the FrameNet lexical units as triggers for the frame being constructed. These lexical units are declared in FrameNet as triggers for frames that exhibit either total or partial overlap with the intended domain. WordNet lexical triggering The activation of lexical material plays a significant role in semantic detection, and it is accomplished by automatically reintroducing the results of the Frame activation query into the workflow. This refers to the frames that were previously manually selected. The rationale behind this step is that if an entity evokes a FrameNet frame that is related to the frame being modeled, then that entity should also be activated in relation to the frame itself.
Close Match triggering In addition to WordNet synsets, entities from various semantic web resources are aligned with frames in the Framester hub. This alignment is established at the meta-level using tshkeos:closeMatch object property. It declares that a concept identified by a specific URI in one resource has a close match with another concept identified by a diferent URI in another resource. Although the two entities remain distinct, they point to the same or similar aspect of reality.
Here, we specify the entities aligned withstkhoes:closeMatch relation from several resources, how they interlink with each other, and the specific SPARQL query for each resource: • WordNet synsets: These are sets of contextual synonyms. As explained in the previous paragraph, if two lexical units can be used as synonyms in the same context, it can be inferred that the considered context is possibly a subframe of the frame being modeled. hTerefore, declaring the entire synset as a trigger for a frame results in a significant increase in coverage, including all the senses of the terms that can be used in similar situations. Some frames that schematize events or actions may hsakvoesa:closeMatch relation to verbs or nouns that point to those events or actions. The query to retrieve WordNet synsets subsumed by a frame is the one mentioned in the previous paragraph, while the general query to retrieve those alignsekdovsia:closeMatch is mentioned at the end of this paragraph. • VerbNet verbs: Verbs from the VerbNet resource can be retrieved through the alignment between WordNet “word senses” and VerbNet “key senses”, as well as through the close match alignment with frames. The verbCoalesce_22022100 is shown as example in Figure1. • PropBank frames: Frames from the PropBank resource are aligned with FrameNet frames through thsekos relation. By providing the URIs of the FrameNet frames as input for the “PropBank triggering” SPARQL query, entities from the PropBank resource can be collected. The example shown is thpeb:fuse.01 frame. • BabelNet entities: A further multilingual coverage is provided through the alignment of BabelNet with Framester frames. The updated online version of BabelNet (5.2) may difer in size and coverage compared to the version in the Framester resource (3.7). Nonetheless, it is possible to retrieve entities from over 270 languages throsukgohst:hceloseMatch alignment. • Premon entries: Premon entries are an extension of the lemon model by the W3C
Ontology - Lexica Community Group.
YAGO Ontology triggering The WordNet lexical grounding is utilized once again to retrieve entities from the YAGO (Yet Another Great Ontology) resource. In this case, the alignment is achieved through thoewl:sameAs property towards WordNet synsets.
Semantic role-driven triggering While potential roles participating in a specific frame are extracted from FrameNet frame elements (as described in the “Frame element-driven triggering” paragraph), according to Framester semantics, they are not the only sources for structural elements that can serve as roles in a frame occurrence.
In Framester, triggering assertions from FrameNet frame elements are extended to include multiple sources of semantic roles: VerbNet arguments, PropBank roles, and WordNet tropes. Semantic type-driven triggering A dimension that complements the previously mentioned aspects is the semantic type of an entity.
hTe final knowledge graph, which consists of several thousands of semantic triggers, from the above mentioned semantic web resources, is available on the ImageSchemaNet GitHub repository. A summary table with details about semantic web resources reused and the exact number of entities per resource triggerinSgutbhsteance IS is available as additional material on the ImageSchemaNet GitHu10b..
In the next section we show how, thanks to the knowledge retrieval and frame building workflow shown in Figure1, it is possible to perform automatic detection from natural language of theSubstance image schema.
Following the methodology described3i8n],[we are able to perform automatic image schema detection from a sentence via reusing the FRED tool.
FRED [39] could be defined as a “situation analyzer”. It is a hybrid knowledge extraction system to generate knowledge graphs directly from natural language. It is built with a pipeline that includes both statistical and rule-based components, and produces as output RDF and OWL knowledge graphs, with embedded entity linking, word-sense disambiguation, and frame/semantic role detection, aligning entities in the produced graph directly to entities from Framester resource.
Consider the sentence “I feel a mixture of anger and emotion” taken from Stefanowitsch’s example [40]. The example is taken from the ISCAT repository, and refers to the MetaNet cognitive metaphoErMOTIONS ARE SUBSTANCES. We give the sentence as input to the FRED tool which generates the graph shown in Fi1g. ure 10The ImageSchemaNet repository is available here: https://github.com/StenDoipanni/ISAAC/tree/main/
ImageSchemaNet
As shown in figure, there is a main “feeling” event, which takes as VerbNet role Theme the “mixture” lexical unit. This unit evokes thfes:Substance and fs:Amalgamation frames, which in turn triggerStuhbestance image schema.
Furthermore, the lexical unit “mixture” is disambiguatwedn:omnixture-noun-1, and the FRED tool disambiguates the lexical units referring to the “Emotions” domain to thewn:anger-noun-1 andwn:emotion-noun-1 WordNet synset as well as on the DBpedia enititiedsb:Anger anddb:Emotion.
We started from sparse definition for a less investigated image schema, in particular adopting a botom-up approach, considering the examples in the Image Schema Catalogue repository, to generate a formal representation of the conScuebpsttaonfce, introducing it in the already existing ImageSchemaNet ontology.
Substance is modeled as subclass oOfbject, and we populated the knowledge graph of semantic triggers reusing multiple semantic web resources from the Framester hub.
We finally performed a automatic image schema detectionSufobrstance generating a knowledge graph from natural language with the FRED tool.
Future works include a refinement of the detection process, in order to consider the domains involved, and possibly detect specific cognitive metaphors.
hTis work is supported by the H2020 projects TAILOR: Foundations of Trustworthy AI - Integrating Reasoning, Learning and Optimization – EC Grant Agreement number 952215 – and the Italian PNRR MUR project PE0000013-FAIR: Future of Artificial Intelligence Research. [7] S. De Giorgis, A. Gangemi, D. Gromann, Imageschemanet: Formalizing embodied commonsense knowledge providing an image-schematic layer to framester, Semantic Web Journal forthcoming (2022). [8] A. Papafragou, C. Massey, L. Gleitman, When English proposes what Greek presupposes: hTe cross-linguistic encoding of motion events, Cognition 98 (2006) B75–B87. [9] J. A. Prieto Velasco, M. Tercedor Sánchez, The embodied nature of medical concepts: image schemas and language for pain., Cognitive processing (20141).0d.o1i:007/ s10339-013-0594-9. [10] D. Gromann, M. M. Hedblom, Body-mind-language: Multilingual knowledge extraction based on embodied cognition, in: AIC, 2017, pp. 20–33. [11] D. Gromann, M. M. Hedblom, Kinesthetic mind reader: A method to identify image schemas in natural language, in: Proceedings of Advancements in Cogntivie Systems, 2017. [12] L. Wachowiak, D. Gromann, Systematic analysis of image schemas in naturallanguage through explainable multilingualneural language processing, in: Proceedings of the 56th Linguistic Colloquium, Peter Lang Group, Forthcoming. [13] M. M. Hedblom, O. Kutz, R. Peñaloza, G. Guizzardi, Image schema combinations and complex events, KI-Künstliche Intelligenz 33 (2019) 279–291. [14] T. Oakley, Image schemas, The Oxford handbook of cognitive linguistics (2007) 214–235. [15] R. W. Gibbs Jr, D. A. Beitel, M. Harrington, P. E. Sanders, Taking a stand on the meanings of stand: Bodily experience as motivation for polysemy, Journal of Semantics 11 (1994) 231–251. [16] G. Lakof, Women, Fire, and Dangerous Things. What Categories Reveal about the Mind, hTe University of Chicago Press, 1987. [17] J. M. Mandler, How to build a baby: Ii. conceptual primitives., Psychological review 99 (1992) 587. [18] G. Lakof, M. Johnson, et al., Philosophy in the flesh: The embodied mind and its challenge to western thought, volume 640, Basic books New York, 1999. [19] J. Hurtienne, J. H. Israel, Image schemas and their metaphorical extensions: intuitive paterns for tangible interaction, in: Proceedings of the 1st international conference on Tangible and embedded interaction, 2007, pp. 127–134. [20] M. Pomarlan, M. M. Hedblom, R. Porzel, Panta rhei: Curiosity-driven exploration to learn the image-schematic afordances of pouring liquids., in: AICS, 2021, pp. 106–117. [21] M. S. P. Cervel, Subsidiarity relationships between image-schemas: an approach to the force schema, Journal of English Studies (1999) 187–208. [22] R. W. Langacker, Cognitive grammar, Basic Readings 29 (2008). [23] A. Cienki, Image schemas and gesture, From perception to meaning: Image schemas in cognitive linguistics 29 (2005) 421–442. [24] F. Santibáñez, The object image-schema and other dependent schemas, Atlantis (2002) 183–201. [25] A. Szwedek, Objectification: a new theory of metaphor, in: English Now: Selected papers from the 20th IAUPE conference in Lund, volume 112, 2007, pp. 308–317. [26] C. J. Fillmore, Frame semantics, in: Linguistics in the Morning Calm, Seoul: Hanshin, 1982, pp. 111–138. [27] C. Baker, Framenet, present and future, in: The First International Conference on Global
Interoperability for Language Resources, 2008, pp. 12–17. [28] G. Lakof, M. Johnson, Metaphors we live by, University of Chicago press, 1980. [29] A. Gangemi, M. Alam, L. Asprino, V. Presuti, D. R. Recupero, Framester: a wide coverage linguistic linked data hub, in: European Knowledge Acquisition Workshop, Springer, 2016, pp. 239–254. [30] A. Gangemi, Closing the loop between knowledge paterns in cognition and the semantic web, Semantic Web 11 (2020) 139–151. [31] G. A. Miller, Wordnet: A lexical database for english, Communications of the ACM 38 (1995) 39–41. [32] A. Gangemi, R. Navigli, P. Velardi, The ontowordnet project: extension and axiomatization of conceptual relations in wordnet, in: OTM Confederated International Conferences” On the Move to Meaningful Internet Systems”, Springer, 2003, pp. 820–838. [33] K. K. Schuler, VerbNet: A broad-coverage, comprehensive verb lexicon, University of
Pennsylvania, 2005. [34] R. Navigli, S. P. Ponzeto, Babelnet: Building a very large multilingual semantic network, in: Proceedings of the 48th annual meeting of the association for computational linguistics, 2010, pp. 216–225. [35] S. Auer, C. Bizer, G. Kobilarov, J. Lehmann, R. Cyganiak, Z. Ives, Dbpedia: A nucleus for a web of open data, in: The semantic web, Springer, 2007, pp. 722–735. [36] F. M. Suchanek, G. Kasneci, G. Weikum, Yago: a core of semantic knowledge, in: Proceedings of the 16th international conference on World Wide Web, 2007, pp. 697–706. [37] A. Gangemi, M. Alam, V. Presuti, Amnestic forgery: An ontology of conceptual metaphors, in: S. Borgo, P. Hitzler, O. Kutz (Eds.), Proceedings of the 10th International Conference, FOIS 2018, volume 306 ofFrontiers in Artificial Intelligence and Applications , IOS Press, 2018, pp. 159–172. URL: https://doi.org/10.3233/978-1-61499-910-2-15. 9doi:10.3233/ 978-1-61499-910-2-159. [38] S. De Giorgis, A. Gangemi, D. Gromann, The racing mind and the path of love: Automatic extraction of image schematic triggers in knowledge graphs generated from natural (2022). [39] A. Gangemi, V. Presuti, D. R. Recupero, A. G. Nuzzolese, F. Draicchio, M. Mongiovì, Semantic web machine reading with FRED, Semantic Web 8 (2017) 873–893. UhRtLt:ps: //doi.org/10.3233/SW-160240. doi:10.3233/SW-160240, 10.3233/SW-160240. [40] A. Stefanowitsch, S. T. Gries, Corpus-based approaches to metaphor and metonymy, volume 171, Walter de Gruyter, 2007.