Definitions of image schemas have generally been found to be underspecified in image-schematic literature, which is particularly true for the image schema VERTICALITY. A frequently adopted approach to address this issue is to provide formal definitions. The use of image schemas in natural language is one central source of evidence for their existence, but spatial language is highly flexible and rarely contains explicit disambiguated spatial information. Thus, applying formal definitions that require non-linguistic spatial properties for the annotation of linguistic manifestations of image schemas is problematic. In this paper, we propose an interdisciplinary review of definitions and suggest annotation guidelines for the image schema VERTICALITY in natural language, including an annotation experiment with three experts.
As basic sensorimotor-informed cognitive structures [
While spatial relations vary widely across languages and cultures, the complexity and diversity
can be analyzed and explained with an underlying set of universal image schemas. In other
†These authors contributed equally.
words, VERTICALITY might be used diferently in a specific language but with the same basic
configuration, also in its metaphoric use. Accounts to formalize image schemas attempted to
counteract the vagueness of image schema definitions [
Thus, this paper establishes annotation rules and configurations that can easily be reused by
other annotation projects to identify image schemas in a transparent and reproducible manner,
even in other languages apart from the word class-specific rules. Our guidelines can increase
trust in text-based image schema analyses, whose methodology might otherwise be labeled
as “alchemy” or “a matter of interpretive ingenuity” [
In terms of manual evaluation, subjective experience when encountering a word has been
investigated. Gibbs et al. [
In terms of automating the annotation process, Gromann and Hedblom [16] used an unsupervised clustering approach to group verb-preposition pairs with related nouns and co-occurrence frequencies as feature vectors by image schema. Wachowiak and Gromann [17] used a supervised approach utilizing large pre-trained language models to classify sentences into eight diferent image schemas, including VERTICALITY, for which the model reached an F1-score of 0.81. However, the examples used to train the model are annotated at sentence-level and consist of short sentences collected from image schema literature. To get information about the individual words indicating an image schema, the researchers used the explainability method LIME [18] that identifies the words the model relied on to make its prediction.
In comparison to these existing ways of annotating image-schematic language, we rely on real-world language use and not examples gathered by introspection, expert annotators rather than laypeople or the crowd, and annotation on text- rather than sentence-level to provide annotators with suficient context to interpret the meaning of lexical units.
In initial annotation experiments, we found definitions of image schemas, in particular VERTICALITY, to be insuficiently specific to provide clear guidelines on which lexical units to annotate. To start, we select VERTICALITY and seek to answer the question whether specific instances of linguistic sequences, such as This book crowned his career as a writer or She sits at the head of the table, where in the latter case the vertical axis is tipped over to the horizontal one, should be annotated or not.
One important notion for image schemas in general and for VERTICALITY in particular are
afordances, i.e., properties that the environment provides to living beings [ 19]. For instance,
stairs aford climbing, which is an important notion for linguistic annotations, since not all
linguistic mentions of stairs do indeed focus on this property, e.g. These are interesting stairs
does not. Johnson defines VERTICALITY as emerging from our tendency to employ an UP-DOWN
orientation in picking out meaningful structures of our experiences [
Brugman and Lakof [ 21] ofer a more detailed analysis by discussing VERTICALITY within the context of the distinct senses of the preposition over. Deane [22] nicely summarizes this analysis by identifying the following predominant characteristics that co-occur with the preposition over as: Verticality, Boundary-Traversal, Surmounting, Covering, Potential Interaction, and Contact. While several of these potentially co-occur, the one characteristic that interests us the most is VERTICALITY, for which the following examples are provided: a. The lamp hangs over the table. b. The plane flew over the city. c. He jumped over the wall. d. He walked over the hill.
While Example a relates purely to VERTICALITY, Example b and d are also boundary-traversal, and Example c represents a case of surmounting. For us, this means collocation with other image schemas, that is, every occurrence of path-traversal and path relate to the image schema SOURCE-PATH-GOAL. However, collocation is not considered in this experiment.
Ekberg [23] analyzes linguistic manifestations of VERTICALITY in English and Swedish and proposes five principles of transformations of the prototypical VERTICALITY image schema. First, a vertical axis can be transformed to a horizontal one by ‘tipping’ it over, e.g. the Swedish upp (up) and ner (down) can be transformed to the horizontal plane in Han gick upp och ner i korridoren (He walked up and down the corridor). Second, an end-point focus can be introduced to index the location along a vertical trajectory, e.g. Hon bodde en trappa upp (She lived one lfoor up). Third, the metaphorical projection of time into space and as a mover along a vertical path in the Western world can be considered related to this image schema, e.g. tankar som når upp i vår egen tid (thoughts that reach up into our own time). Fourth, a zero-dimensional entity can be transformed to a path towards a one-dimensional extended entity, e.g. Klänningen nådde ner till anklarna (The dress reached down to the ankles). Zero-dimensional here refers to the construed trajectory for an entity that is in fact one-dimensional, i.e., a point in space rather than a line. A dress in the above example is considered a point in space rather than a line and thus, the trajectory of having a starting point and an end point is construed. Finally, inanimate objects can acquire characteristics of the human body, such as Han satt längst upp vid bordet (He sat at the head of the table).
Conceptual metaphors represent a cognitive mechanism that enables the projection of conceptual
structures from a source to a target domain [24], where the former can be recurring physical
experiences and the latter an abstract domain [
Borneto [27] looks at how the German verbs stehen (to stand) and liegen (to lie) are used to describe physical and abstract situations. He finds that in the physical world, stehen usually requires the subject to be in a vertical position, whereas liegen requires the subject to be in a horizontal position or one lacking dimensionality. For example, to say that a book is (upright) on a shelf, one can say Das Buch steht im Regal. On the other hand, when a book is in a horizontal position on a table, one can say Das Buch liegt auf dem Tisch. Moreover, he identifies patterns in the metaphoric usage of stehen. For instance, it can be used to describe extract entities as if they were standing like a human to describe their level of stability, e.g. Die Bank steht fest auf eigenen Beinen (The bank is firmly on its own two feet). Overall, Borneto comes to distinguish four main semantic types of VERTICALITY in regards to the verb stehen: (1) human verticality, i.e., the human body being in an upright position as in the example before; (2) geometric verticality, i.e., there being a vertical axis in geometrical space; (3) saliency verticality, i.e., something rises and, thereby, stands out ;(4) gravity verticality, an object falling downwards. Here, geometric verticality is the most static, and gravity verticality is the most dynamic. We specify our own definition based on these details in Section 4.2.
We derive annotation rules to be considered when annotating linguistic sequences with the image schema VERTICALITY from: (1) the above discussion on diferent perspectives of VERTICALITY, and (2) annotation experiments in which we discussed and compared our annotations of various natural language examples. Before introducing the VERTICALITY-specific rules, we present some general annotation guidelines that are also applicable to other image schemas.
Step-wise general instructions for the annotation procedure closely relate to that proposed by
the Metaphor Identification Procedure (MIP) [
A. if yes, annotate as “metaphoric” image schema
When annotating a word or phrase as image schematic, we decided to annotate the units that led to the classification as well as all units contributing to VERTICALITY. For instance, from our annotation experiment and corpus, the lexical unit trouser-rousing describing men getting excited over nude pictures in a magazine alone describes the image schema. However, crawled alone might not necessarily describe a vertical axis, but only in the combination crawled under rocks. Since consciously construing all potential interpretations of a lexical unit is a challenging task, we recommend consulting dictionary definitions in order not to accidentally overlook more basic meanings of abstract units and additionally consider potential transfers across word classes (see Section 4.2).
In this section, we present specific annotation rules for (not) annotating VERTICALITY in linguistic sequences that derive from the above definitions and our experiences trying to systematically annotate this image schema in natural language. In fact, starting from existing definitions and descriptions of VERTICALITY configurations, we iteratively refined our annotation rules in several annotation experiments on diferent datasets, describing the last experiment after finalizing the annotation rules in this paper.
General: The main notion is an UP-DOWN orientation that the meaning of lexical units in a sequence afords in a specific context, where, however, the vertical axis can potentially be tipped to a horizontal plane, e.g. walking up and down a corridor. The main dimensions for this image schema are a vertical orientation or positioning of an entity or viewer, a vertical association between entities as well as instances of imagined or construed VERTICALITY. The main configurations of this image schema are depicted in Fig. 1, which distinguishes between static and dynamic descriptions. Positioning is frequently static, e.g. She is higher up the hierarchical ladder than him, including instances of being upright, e.g. standing tall, and adjectives, such as high or deep. Vertical associations are generally dynamic, e.g. She is climbing the career ladder. For each sequence, the meaning of lexical units should be analysed to evaluate whether their meaning in context represents one of the above static or dynamic configurations and ultimately afords VERTICALITY.
Fictive Motion: The question whether to annotate fictive or construed motion is a very central one in image schema annotation. Egorova et al. [28] describe fictive paths as figurative rather than metaphoric language that describe geospatial location rather than motion. Ekberg [23], however, argues that relating a static entity or “meaning of a lexeme” to a dynamic one shows our ability to mentally construe a path and thus, dynamic VERTICALITY, e.g. The dress reached down to the ankles. We decided to annotate such instances of fictive or construed motion. Verbs: Some verbs are more prototypical for the image schema VERTICALITY than others, e.g. climbing or falling. Other verbs, such as nodding or shrugging, caused long discussions on whether they evoke VERTICALITY or not. If the meaning of the verb in context relates to one of the vertical configurations depicted in Fig. 1, it should be annotated. Vertical head and shoulder movement in nod and shrug would be annotated if used in this meaning, physically or metaphorically. Verbs can also be static as in being in an upright position, e.g. standing. In contrast, stand back does not relate to an upright orientation and would not be annotated. Verbs combined with prepositions, i.e., phrasal verbs, are not automatically annotated when containing up or over, since their connection to a vertical axis is not evident, e.g. make up. Nouns: Nouns indicating that an entity is at a specific point on a vertical axis are annotated, for example, peak or level. Nouns can also indicate dynamic movement along a vertical axis, e.g. descent or ascension. Body parts, especially head and feet, are frequently related to VERTICALITY, however, not always. For instance, head of the company implies someone is at the highest point of the vertical axis describing the company’s hierarchy, whereas best foot forward focuses on forward motion rather than on a vertical motion or position.
Transfer Across Word Classes: At times, a word in a specific word class strongly resonates
or directly derives from another word class with a very specific physical meaning. The authors
of the MIP method [
Examples For Orientation: Lastly, we want to introduce some prototypical literal and metaphorical examples, counterexamples, and boundary cases, i.e., borderline examples that caused discussions but in the end were annotated, covering a substantial amount of our rules. This overview allows annotators to judge new samples more eficiently and reliably by allowing for direct comparisons.
a. Prototypical Literal: (i) He is climbing the mountain. (vertical movement) (ii) The lamp hangs over the table. (static vertical configuration) (iii) The train goes under the bridge. (movement (non-vertical) + static vertical configuration) b. Prototypical Metaphorical: (i) I feel down/low today (static vertical configuration without explicit reference point) (ii) It was like a guardian angel had come down. (vertical movement) (iii) The comedian escalated the jokes. (vertical movement) (iv) The joke went over my head. (movement (non-vertical) + vertical configuration) c. Counterexample:
(i) Put your best foot forward (dynamic, but no vertical configuration) d. Boundary cases: (i) He is suppressing the urge to kill.
(ii) The time is standing still.
One of the main arguments for establishing annotation rules for image schemas is to make
image schema text analyses more transparent, comparable, and reproducible. To ensure this
goal is met, we urge to report all important decisions taken when analyzing — similar to the
suggestions by the authors of the MIP [
To test the reliability and usability of our annotation rules, the three authors of this paper annotated natural language texts. This annotation experiment was conducted after finalizing the guidelines. Thus, it is entirely distinct from the annotations done to derive any rules. We chose the Open American National Corpus (OANC)1 as it is freely available, has a permissive license allowing transparent modification, and contains coherent texts of diferent genres. These coherent texts allow us to understand each word’s meaning in its full context, which is crucial when identifying the metaphoricity of a word and its potential connection to more basic physical usages. Furthermore, the texts contain complex sentences that use domain-specific language. In previous work on metaphor annotation [29], we found that such complex and naturally occurring language is much more dificult to annotate for humans and machine learning models than the thought-up, unambiguous examples in datasets such as the Image Schema Database [30] or Lakof’s Metaphor List 2.
From the OANC, we annotated 19 texts belonging to the journal category. Each annotator read the texts once and used Merriam-Webster and the Cambridge Dictionary for word definitions. Overall, this resulted in 451 annotated sentences. In 88 of those sentences, at least one annotator identified the image schema VERTICALITY. We compared annotations after the first nine texts to resolve disagreements and misunderstandings regarding the guidelines. The initial interannotator agreement as measured by Fleiss’ Kappa for these first nine texts is 0.40. After this round of discussion, the agreement increased to 0.49 for the following 10 texts. One reason for the rather low score is that identifying VERTICALITY needs very thorough reading, especially when the basic meaning is found only in a sub-component of a compound word. Thus, the annotators sometimes missed instances of VERTICALITY even though they did agree to the annotation when brought up during discussion. Other times it is dificult to come to an agreement, for instance with phrasal verbs like wind up, where the relation of up to its VERTICALITY-related meaning is debatable. 1https://anc.org/data/oanc/ 2http://www.lang.osaka-u.ac.jp/sugimoto/MasterMetaphorList/metaphors/index.html
In this paper, we presented the first steps towards guidelines for annotating image schemas in natural language in a systematic, transparent, and reproducible fashion. While some rules apply to image schemas in general, as of now, we primarily focused on the image schema VERTICALITY, as it especially sufered from underspecification. We provided a thorough discussion and definition of VERTICALITY based on literature and annotation experiences, highlighting the schema’s main dimensions and configurations. The resulting guidelines were tested with three annotators applying them to 451 sentences.
Checking Basic Meanings. In the annotation process, it is important to evaluate all meanings of a lexical unit. Direct interpretation and understanding of meaning might lead to overlooking more basic, physical underlying meanings. Thus, we recommend consulting definitions for most lexical units to ensure that no metaphoric projections are accidentally omitted. For instance, the word approach might directly be interpreted as purely abstract. However, when considering the physical meaning of the (noun-related) verb to come near, there is an underlying physical experience. This example also relates to the transfer across word classes, where a word class with a purely abstract meaning derives from another word class with a metaphoric meaning. For instance, high-rise as a noun clearly relates to the corresponding adjective high and verb rise, two prototypical examples of VERTICALITY. In this annotation procedure, we included and recommend considering transfer across word classes. However, as this rule requires examining the metaphorical projections among diferent meanings of a lexical unit, this can be particularly challenging for non-native speakers. In general, identifying more basic meanings can be dificult for non-native speakers as it requires a firm grasp of the usage and semantics of the word in question. This dificulty can be also seen as a limitation of our annotation experiment in Section 5, with all three annotators being non-native English speakers.
Image Schema Collocation. Collocation of image schemas, even within the same lexical unit, is a common phenomenon, which represents future work to extend the proposed procedure to all commonly proposed image schemas. Within this annotation experiment, we noticed that, besides collocation, clearly delimiting image schemas is at times challenging. In particular, SUPPORT and CONTACT frequently occur with similar prepositions as VERTICALITY, especially on and over. If an entity A is on another entity B, A is vertically higher than B but represents a prototypical example of SUPPORT. Thus, in our view, these instances aford SUPPORT rather than VERTICALITY. To pour the juice over ice implies a vertically higher juice being poured over a vertically lower ice for the liquid to flow. However, spread the mustard over the chicken refers to covering the surface of the chicken with mustard without an evident relation to VERTICALITY. It is open for discussion whether annotating the former example with this image schema already represents an instance of overcommitting to interpreting a presumed underlying scenario. Languages Beyond English. All our discussions and example annotations are based on English language samples. While we assume that the basics of our annotation rules are transferable to other languages, any word class-related rules and some of the more fine-grained discussions will not apply to other languages. For instance, our discussion of how to annotate body parts, such as head, was based on how it is used in English. Looking at the Mexican language Mixtec, Regier [31] and Brugman [32] found that body parts are used as a standard way to describe spatial configurations. To describe the configuration of a stone under a table one says yuù w ̃ híyaà čìì-mesá (stone the be-located belly-table). Similarly, being above something can be described as being at something’s head if the lower object is vertically extended or animal’s back if it is horizontally extended. Interestingly, Mixtec makes no diference between being on something or above something, thus, not allowing the clear separation into the image schemas VERTICALITY and SUPPORT we described in the paragraph before. Another example of this sometimes blurred line between vertical configurations with and without contact is Japanese, where ue ni can be used for describing configurations where something is on or above an object [31]. evidence for image schemas of concrete and abstract verbs, in: Proceedings of the Annual Meeting of the Cognitive Science Society, volume 23, 2001. [15] D. Gromann, J. C. Macbeth, Crowdsourcing image schemas., in: TriCoLore (C3GI/ISD/S
CORE), 2018. [16] D. Gromann, M. M. Hedblom, Kinesthetic mind reader: A method to identify image schemas in natural language, in: Advancements in Cogntivie Systems, 2017. [17] L. Wachowiak, D. Gromann, Systematic analysis of image schemas in natural language through explainable multilingual neural language processing, in: Proceedings of the 29th International Conference on Computational Linguistics, 2022, pp. 5571–5581. [18] M. T. Ribeiro, S. Singh, C. Guestrin, ” Why should I trust you?” Explaining the predictions of any classifier, in: Proceedings of the 22nd ACM SIGKDD, 2016. [19] W. Kuhn, An Image-Schematic Account of Spatial Categories, in: Spatial Information
Theory, Springer, 2007. [20] I. C. Martínez, S. A. Español, D. I. Pérez, The interactive origin and the aesthetic modelling of image-schemas and primary metaphors, Integrative Psychological and Behavioral Science 52 (2018) 646–671. [21] C. Brugman, G. Lakof, Cognitive topology and lexical networks, in: Lexical ambiguity resolution, Elsevier, 1988, pp. 477–508. [22] P. Deane, Multimodal spatial representation: On the semantic unity of over, in: From
Perception to Meaning, De Gruyter Mouton, 2008, pp. 235–284. [23] L. Ekberg, The mental manipulation of the vertical axis: How to go from ’up’ to ’out’, or from ’above’ to ’behind’, Lexical and syntactical constructions and the construction of meaning (1995) 69–88. [24] G. Lakof, M. Johnson, Metaphors we live by, University of Chicago press, 1980. [25] L. Cian, Verticality and conceptual metaphors: A systematic review, Journal of the
Association for Consumer Research 2 (2017) 444–459. [26] E. K. Dodge, J. Hong, E. Stickles, Metanet: Deep semantic automatic metaphor analysis, in: Proceedings of the Third Workshop on Metaphor in NLP, 2015, pp. 40–49. [27] C. S. Borneto, Liegen and stehen in German: A study in horizontality and verticality,
Cognitive linguistics in the redwoods (1996). [28] E. Egorova, L. Moncla, M. Gaio, C. Claramunt, R. S. Purves, Fictive motion extraction and classification, International Journal of Geographical Information Science 32 (2018) 2247–2271. [29] L. Wachowiak, D. Gromann, Does GPT-3 Grasp Metaphors? Identifying Metaphor Mappings with Generative Language Models, in: Association for Computational Linguistics: ACL, 2023. [30] J. Hurtienne, Image Schema Database (ISCAT), 2017. URL: http://zope.psyergo.
uni-wuerzburg.de/iscat. [31] T. Regier, The human semantic potential: Spatial language and constrained connectionism,
MIT Press, 1996. [32] C. Brugman, The use of body-part terms as locatives in Chalcatongo Mixtec, in: Survey of California and Other Indian Languages, 1983.