Abstract

Using image schema violations to expose hidden semantical structures in metaphors 1

6. Acknowledgements

We thank the reviewers for their thoughtful and interesting comments which we tried to ignore as far as possible.

5. Conclusion

In this paper, we investigated the hypothesis that meaning is not exclusively a linguistic phenomenon, but rather strongly embedded in embodied experiences.

We explored infringes of the invariance principle for analogies in a few domains by violating expectations stemming from such embodied experiences. To explore this, we used four diferent attributes (orientation, size, shape and colour) and presented examples in which the image-schematic skeleton had been manipulated. Our examples support the idea that non-linguistic meaning is by necessity present in the conceptualisation of objects and concepts as the violation of expectation is registered.

It is a first step towards a larger goal of better understanding what the structure of analogies actually looks like. However, more research is required to better understand the impact and generality of the arguments presented via the examples. We make no claims that by altering the image-schematic structure conventional metaphors will result in meaningful, yet surprising outcomes. It is likely that there is more at work here. We do, however, argue that such manipulations do play a role and should be taken into consideration in future investigations.

4. Discussion

It is a well-established phenomenon that analogies, metaphors and conceptual metaphors require that meaning is drawn from domains other than those that are directly exposed or presented. We have argued that part of this hidden information is based on conceptual skeletons that, in turn, are based on generalisations from embodied experiences. In particular, when looking at conventional metaphors and tropes in popular culture, and by manipulating their image-schematic core, it is possible to identify the underlying analogical information skeleton at work.

This violation of expectations has been demonstrated to increase cognitive lfexibility [ 26 ], to improve memory [24] and is suggested to be a fundamental principle in humour [ 2, 36 ]. As such, identifying the components that are crucial for the analogical transfer to be considered conventional is a key to successfully creating situations in which the expectations are broken without destroying the analogical shape.

Additionally, the field of artificial intelligence could greatly benefit from a better understanding of the underlying principles of analogical reasoning. Syntactic approaches to machine made analogies exist [ 11, 27 ], but they often perform poorly as the semantic content is disregarded. Methods that aim to include semantics into this automatic approach [ 15, 35 ] could benefit from learning the rules of what it means to violate the expectations.

3.4. Orientation

One of the earliest image schemas that we acquire is understanding Verticality [16]. It is believed that this concept is learned based on the body’s vertical axis, and based on the vertical efects gravity has on moving (falling) objects (Up_Down movement), without involving a deeper understanding of the underlying physics [19].

A conventional conceptual metaphor is ‘SUCCESS is UP.’ We see this in concepts such as a social hierarchy, ‘to look up to someone’ and ‘climbing the career ladder.’ All of these expressions are built on the skeleton of Verticality. Obviously, we can look down on someone and we can ‘fall from grace.’ But imagine a scenario in which the more power you have the lower you are placed. One real life example of this is the prime minister of the Czech Republic. As a result of a bloody, political history through defenestration3, their ofice is located in the ground floor of Prague’s castle. In more abstract domains the notion of ‘GOOD is UP’ can be seen in how Heaven is above and Hell is below us. It is rare to see this mapping inverted.

Another way to change semantic content through changes in orientation is to play with the concept of Verticality and ‘horizontality.’ Both time and journeys are often conceptualised (in the West) as Paths along a horizontal axis [ 1 ]. This translates to the classic view of a narrative, moving from one scene to the next in a horizontal fashion, or in the scientific literature, from top to bottom. Imagine instead of time being conceptualised from left to right it is seen as going from right to left, or as seen in some non-western languages, as a narrative going ‘backwards.’ There are a significant amount of movies that play with exactly this phenomenon. For instance, Pulp Fiction [25] (1994) and Memento [ 7 ] (2000) where the audience is presented with a reversed narrative, slowly learning more and more of how the story began. This paper aimed to expose the reader to a similar cognitive process 3The act of throwing someone out of a window. by forcing the reader to either violate the classic scientific narrative, from top to bottom, and instead follow the red thread by reading backwards, or to violate the intuitive order of the content by sticking to the traditional vertical reading pattern. 3.3. Size For the attributes regarding size, the two most important spatiotemporal image schemas are Scale and Containment. These are some of the most fundamental image schemas and children learn their rules early in life. For instance, how larger objects cannot (in normal scenarios) be Contained in smaller objects. However, this principle is sometimes violated. For instance, in the House of Leaves [8], the reader is invited into a house which grows larger on the inside but remains indistinguishable on the outside (similar to, but clearly outpacing Dr Who’s Blue Box Paradox). This violation of physical laws sends an ominous message fitting to the increasingly thrilling plot. The Scale of the inside of the house has a superior status to the outside contradictory to common sense. Another, perhaps more commonly accepted violation of the image-schematic principles of size in Containment is that while the universe is infinitely large, God still holds it in his hand.

Despite being set in a fictive or abstract setting, these examples are still based on physical alterations through described changes in spatial structure. In comparison, there are other examples playing with contradictory expectations of size. One example is the conceptual metaphor ‘BIG is STRONG’ [18]. From experience children learn early on that larger people are stronger than them, that larger objects are (often) more rigid and stable and so on. In opposition to this expectation, it is a common trope to let the smallest person or object to be the most powerful. For instance, in Men in Black Will Smith’s character is initially disappointed by the minuscule size of the alien hunting gun ‘The Noisy Cricket,’ only to realise that it might very well be the most powerful one available [ 4 ]. This contradiction in expectation is confusing for Will Smith’s character Agent J, yet highly entertaining to the audience. Likewise, in Star Wars the young Luke Skywalker is disappointed that Yoda, the most powerful Jedi Master, has a height of approximately two feet [12]. 3.2. Shape Shape is another attribute that also is relevant for afordances. Learning the afordances of (simple) objects is something that comes early on in childhood alongside the formation of image schemas. For instance, a classic toy asks toddlers to match blocks of diferent shapes to their respective holes on a box. This teaches them that there are constraints to an object’s shape in order for them to, for instance, Contain things. A classic example is to think of a container without an inside. According to the theory of recognition-by-parts [ 5 ] an object can be recognised by breaking it into smaller shapes, called geons. A container is by definition a hollow object. However, Grande Arche de La Defense west of Paris, is a house built as a tesseract, basically a hollow hypercube. This manipulation of a house as a hollow shell violates the idea of a house being a container as it is the border between the outside and the hollow inside which constructs not the walls, but the conventional house itself.

These kinds of designs are common as designers consistently work on changing conventional shapes into more innovative compositions. Yet they are limited by the objects’ original functionality. One can argue whether a chair that does not aford ‘sitting’ ( Support) is a chair at all. However, as long as this afordance is taken into account, the shape of the chair doesn’t matter much as stools, beach chairs and armchairs could be considered diferent forms of the same object.

3.1. Colour

While colour gradients most commonly are not considered part of a repository of image schemas, many attributes of colours are associated with image-schematic notions, for instance, the Scale from bright to dark. Hedblom [14] presents an abstract painting that plays with the image schema Verticality based conceptual metaphor ‘UP is GOOD/DOWN is BAD’ by, in violation, combining it with the conceptual metaphors for ‘LIGHT is GOOD/DARK is BAD’. In consequence, UP becomes BAD and DARK becomes GOOD, and DOWN becomes GOOD and LIGHT becomes BAD. This form of breaking expectations in visual art is a common way to send conflicting messages.

For instance, it is largely accepted that light and/or bright colours are associated with happiness and energy. Yet there are plenty of domains that match lit atmospheres with horrifying subjects to generate unsettling emotional reactions. The most obvious examples, perhaps, are hospital environments, and the bright light emanating from a dentist lamp or an interrogation room. Another prototypical example of how bright, colourful settings are violated is the domain of CLOWNS. Traditionally seen as happy and ridiculous characters clad in bright colours, they are often depicted as tragic and sad or even as the murderous villains in horror movies despite their colourful outfits and make-up (e.g. in Steven King’s It [17]).

Additionally, colours are often associated with particular emotions [ 30 ]. While this is partly grounded in cultural experiences, there is also support for the position suggesting that there is an innate relationship between colours and particular emotions [ 37 ]. Photography, filmmakers and artists play with these colour expectations in order to either emphasise a particular emotion or to set them in direct opposition.

3. A Series of Attribute Manipulations

By twisting the foundation of the invariance principle of metaphors, it is possible to reveal and identify unexpressed semantic expectations. This non-verbal information is a form of hidden semantics based on associations, and it is hidden in the sense that we are typically unaware of its workings. Revealing and exploring some of this hidden semantics is the topic of this paper. Regarding the relevant associations studied, we here restrict ourselves to looking at the use of the imageschematic skeleton found in metaphors, and we look at how a violation of the default interpretation impacts the expected transfer of information. We look at a few diferent attributes commonly associated with metaphors: orientation, size, shape and colour, and present a number of scenarios in which the image-schematic skeleton has been turned around.

2. Background

2.3. Conceptual Metaphors

Conceptual metaphor theory was introduced to explain how information can be analogically transferred from one domain to another [18, 21]. In this respect, (conceptual) metaphors are a vital part of natural language [ 29 ], but also constitute a fundamental aspect of more abstract means for communication.

A classic conceptual metaphor is ‘LOVE is WAR.’ The structure transfers all kinds of domain specific notions from WAR into the concept of LOVE: e.g. love ‘kills,’ to ‘hit’ on someone and fight for a relationship, etc. This is also possible in metaphors such as how Cupid shoots arrows not to kill, but to make people fall in love. Observe that the conceptual metaphor ‘WAR is LOVE’ is not quite as successful, and frankly rather disturbing. Interestingly enough, in some situations this is also used. For instance, art pieces such as Banksy’s Rage, Flower Thrower, depicts a man who instead of throwing a bomb/rock, is about to launch a bouquet. By doing so, the metaphor uses attributes from the domain of LOVE to promote PEACE, arguably the opposite of WAR. The domains in conceptual metaphors are, therefore, not interchangeable.

Despite the domains not being interchangeable, the underlying structure of the source domain needs to be the structure also in the target domain, a phenomenon known as the invariance principle [ 33 ]. Following the embodied mind hypothesis and how image schemas construct these embodied experiences into meaningful patterns, it can be argued that image schemas construct the underlying structure found in the invariance principle [ 34 ]. However, it is noteworthy to mention, that while the spatiotemporal relationships of image schemas construct deeper meaning of objects, many metaphors are also built on more shallow attributes such as colours or shapes. In the next section, we play with the underlying theories behind expectancy violations theory [ 6 ] by looking at some examples in which the expected conclusions of particular conceptual metaphors and image schemas are turned around.

2.2. Image schemas

Image schemas are the conceptual building blocks that are learned from embodied experience in early infancy [16, 20, 23]. They are abstract, mental patterns capturing spatiotemporal relationships between objects and their surroundings. Some classic examples are Verticality, Containment and Source_Path_Goal. They often construct the conceptual skeleton for analogical reasoning and conceptual metaphors [ 15, 35 ]. Additionally, they are directly related to the physical afordances that objects realise [13]. For instance, a table ofers the afordance of Support and a cup the afordance to Contain liquids.

As conceptual gestalts, image schemas impose plenty of embedded ‘logical rules,’ or common sense dogmas, related to real life objects. For instance, it is not possible to Contain a larger object in a smaller one and objects that are coming closer (Near_Far) appear to grow in size (Scaling) rather than shrink. Rules such as these are important prerequisites in order to be able to make meaningful conclusions in analogies and metaphors both in everyday and in more abstract domains such as seen with conceptual metaphors.

2.1. Embodied Cognition

There is increasing support and empirical evidence for the claim that a large part of human cognition arises in a direct connection to and dependence on the body’s sensorimotor experiences [ 22, 28 ]. A particular strength of this theory is its ability to explain the large degree of spatial language in metaphors and when describing abstract concepts as for instance: ‘to be stuck in life,’ ‘the rise to power’ or ‘falling in love.’ However, this is a phenomenon that can be found in analogical reasoning and concept generation as well [15] and it has substantial support from neuroscience and psychology [ 3, 9, 10, 31, 32 ].

Based on this embodied cognition hypothesis, one may argue that all of our expectations and predictions are based, to a certain extent, on a combination of embodied experiences rooted in the sensorimotor programs and more ‘high-level’ principles resulting from e.g. repeated linguistic, cultural and social exposure. One theory for how embodiment is represented in the mind on a more neurological level is through generalised patterns derived from these experiences, often called image schemas.

1. Introduction

Understanding natural language, appreciating the arts and poetry, and finding meaningful patterns in chaos, are fundamental aspects of human cognition. In fact, it can be argued that any form of non-verbal communication relies on the receiver drawing upon hidden information to ‘fill in the blanks.’ For instance, an abstract painting with blotches of colour in seemingly random patterns will not in itself have a clear message. Due to its abstract nature, it lacks semantics by default. Yet, art critics and museum visitors often have similar interpretations of particular art pieces despite diferent expertise and background. This form of uniform intuition requires that there is a hidden meaning not expressed with linguistic symbols. Instead, attributes such as particular shapes, patterns and colours allow the viewer to make associations and analogies to other situations and concepts, and from them build an interpretation. The nature of the fundamental principles underlying this ability has been a philosophical debate at least since Aristotle’s introduction of Associationism in De Memoria Et Reminiscentia.

The same kind of principle can be found across all forms and modalities of expression. Embodied cognition suggests that all cognition stems from the body’s interaction with its environment [ 28 ]. This means that for all forms of communication there is an underlying layer of semantics based on patterns learned from the sensorimotor experiences. One framework for this is the work on image schemas [16, 20]. Image schemas are spatiotemporal relationships constructing a conceptual skeleton in metaphors in literature, arts and other domains.

In this short paper, we explore this underlying semantics by looking at image schemas and manipulating the shape of conceptual metaphors. Further, by violating the embodied default interpretations of these constructs, we invite the reader to acknowledge that despite the lack of a direct linguistic or semiotic expression of their semantics, meaning is present.

Abstract. Even in the absence of verbal language, meaning and nonverbal communication still remain. For abstract domains such as music, poetry and visual arts, perceivers often have a strong intuition about the particular message of individual pieces. One theory for this is the embodied nature of semantics as well as repeated exposure to cultural conventions. It is suggested that embodiment takes the form of mental patterns called image schemas. They are spatiotemporal relationships between objects and their environment. Intrinsically meaningful, they often feature as a structural skeleton for conceptual metaphors, where information from one domain is transferred onto another. These information skeletons are never directly exposed, and instead hidden in between the lines. With the purpose of exposing their presence, we present a series of examples of conventional metaphors and popular cultural references where the image-schematic skeleton is violated. Our brief analysis demonstrates that by manipulating this embodied skeleton, previously non-expressed meaning is revealed. [8] Mark Z. Danielewski. House of Leaves. Pantheon Books, 2000. [9] Jerome Feldman and Srinivas Narayanan. Embodied meaning in a neural theory of language. Brain and Language, 89(2):385 – 392, 2004. Language and MotorIntegration. [10] Vittorio Gallese and George Lakof. The brain’s concepts: the role of the sensory-motor system in conceptual knowledge. Cognitive neuropsychology, 22(3):455–79, 2005. [11] Dedre Gentner. Structure-mapping: A theoretical framework for analogy.

Cognitive Science, 7:155–170, 1983. [12] George Lucas (Director). Star Wars: Episode IV – A New Hope. Lucasfilm, 1977. [13] James J. Gibson. The theory of afordances, in perceiving, acting, and knowing. towards an ecological psychology. In Robert Shaw and John Bransford, editors, Perceiving, Acting, and Knowing: Toward an Ecological Psychology, pages 67–82. NJ: Lawrence Erlbaum, Hillsdale, 1977. [14] Maria M. Hedblom. Beneath the paint: A visual journey through conceptual metaphor violation. In Proceedings of the Third Joint Ontology Workshops (JOWO), volume 2050, Bozen-Bolzano, Italy, September 21–23, 2017, 2017.

CEUR. [15] Maria M. Hedblom, Oliver Kutz, and Fabian Neuhaus. Image schemas in computational conceptual blending. Cognitive Systems Research, 39:42–57, 2016. [16] Mark Johnson. The Body in the Mind: The Bodily Basis of Meaning, Imagination, and Reason. The University of Chicago Press, Chicago and London, 1987. [17] Stephen King. It. Viking, 1986. [18] Zoltán Kövecses. Metaphor: A Practical Introduction. Oxford University

Press, USA, 2010. [19] Oliver Kutz, Nicolas Troquard, Maria M. Hedblom, and Daniele Porello.

The Mouse and the Ball: Towards a Cognitively-Based and Ontologically

Grounded Logic of Agency. In Proceedings of the 10th International Con

ference on Formal Ontology in Information Systems (FOIS 2018), pages 141–148. IOS Press, 2018. [20] George Lakof. Women, Fire, and Dangerous Things. What Categories Reveal about the Mind. The University of Chicago Press, Chicago and London, 1987. [21] George Lakof and Mark Johnson. Metaphors We Live By. University of

Chicago Press, 1980. [22] George Lakof and Mark Johnson. Philosophy in the Flesh. Basic Books, 1999. [23] Jean M. Mandler. The Foundations of Mind : Origins of Conceptual Thought:

Origins of Conceptual Though. Oxford University Press, New York, 2004. [24] Michaela Porubanova, Daniel Joel Shaw, Ryan McKay, and Dimitris Xygalatas. Memory for expectation-violating concepts: the efects of agents and cultural familiarity. PloS one, 9(4):e90684, 2014. [25] Quentin Tarantino (Director). Pulp Fiction. A Band Apart Jersey Films, 1994.

References

[1] Kathleen Ahrens and Alicia L.T. Say . Mapping image schemas and translating metaphors . In Proceedings of Pacific Asia Conference on Language, Information and Computation , pages 1 - 8 , February 1999 .

[2] Gabriella Airenti . Playing with expectations: A contextual view of humor development . frontiers in Psychology , 7 : 1392 , 2016 .

[3] Lisa Aziz-Zadeh and Antonio Damasio. Embodied semantics for actions: Findings from functional brain imaging . Journal of Physiology Paris , 102 ( 1- 3 ): 35 - 39 , 2008 .

[4] Barry Sonnenfeld (Director). Men in Black. Columbia Pictures , Amblin Entertainment, Parkes/MacDonald Productions, 1997 .

[5] Irving Biederman . Recognition by components: A theory of human image understanding . Psychological Review , 94 ( 2 ): 115 - 117 , 1987 .

[6] Judee K Burgoon. Expectancy violations theory . The international encyclopedia of interpersonal communication , pages 1 - 9 , 2015 .

[7] Christopher Nolan (Director). Memento. Summit Entertainment Team Todd , 2000 .

[26] Simone M. Ritter , Simone Kühn, Barbara CN Müller, Rick B. Van Baaren , Marcel Brass , and Ap Dijksterhuis . The creative brain: Corepresenting schema violations enhances TPJ activity and boosts cognitive flexibility . Creativity Research Journal , 26 ( 2 ): 144 - 150 , 2014 .

[27] Martin Schmidt , Ulf Krumnack, Helmar Gust, and Kai-Uwe Kühnberger . Heuristic-Driven Theory Projection: An Overview . In H. Prade and G. Richard, editors, Computational Approaches to Analogical Reasoning: Current Trends , volume 548 of Computational Intelligence. Springer-Verlag, 2014 .

[28] Lawrence Shapiro. Embodied Cognition . New problems of philosophy . Routledge, London and New York, 2011 .

[29] Ekaterina Shutova , Simone Teufel, and Anna Korhonen . Statistical metaphor processing . Computational Linguistics , 39 ( 2 ): 301 - 353 , 2013 .

[30] David R. Simmons . Colour and emotion. New directions in colour studies , pages 395 - 414 , 2011 .

[31] Marco Tettamanti , Giovanni Buccino, Maria Cristina Saccuman, Vittorio Gallese, Massimo Danna, Paola Scifo, Ferruccio Fazio, Giacomo Rizzolatti, and Daniela Perani . Listening to action-related sentences activates frontoparietal motor circuits . Journal of Cognitive Neuroscience , pages 273 - 281 , 2005 .

[32] Barbara Tomasino , Franco Fabbro, and Paolo Brambilla . How do conceptual representations interact with processing demands: An fMRI study on actionand abstract-related words . Brain research , 1591C: 38 - 52 , 2014 .

[33] Mark Turner . Language is a Virus . Poetics Today , 13 ( 4 ): 725 - 736 , 1992 .

[34] Mark Turner . An image-schematic constraint on metaphor . Conceptualizations and mental processing in language , pages 291 - 306 , 1993 .

[35] Tony Veale . The analogical thesaurus . In J. Riedl and R . Hill, editors, Proceedings of the Fifteenth Innovative Applications of Artificial Intelligence Conference , pages 137 - 142 . AAAI Press, 2003 .

[36] Tony Veale . Incongruity in humor: Root cause or epiphenomenon? Humor - International Journal of Humor Research , 17 ( 4 ), 2004 .

[37] Marcel R. Zentner . Preferences for colours and colour-emotion combinations in early childhood . Developmental Science , 4 ( 4 ): 389 - 398 , 2001 .