Introduction

This contribution is a position paper on some possible directions for making advances towards accurate and usable characterisations of image schemas. It is a slightly extended and media-adapted version of the presentation given at the Eighth Image Schema Day. The essential point pursued here is that we can draw useful lessons from parallels in the treatment of image schemas over the past 20 years and rather similar discussions that have arisen in attempts to characterise film, particularly narrative film. Both see an increasing orientation, and often re-orientation, to questions of embodiment. Engaging with this facet of the phenomena at hand will be suggested to be an essential precondition for progress. Considering some of the developments being pursued for treating film may then also offer some ideas for broadening accounts of image schemas.

Some basic theoretical building blocks

To begin the discussion of the relationships between the definition of image schemas and their occurrence in film and film analyses, it is useful to set out three areas of discussion related closely to metaphor: first, metaphor considered as a literary device; second, cognitive metaphor theory; and third, metaphorical mappings involving blending.

First, metaphor as a literary device has naturally been used in interpretations and analyses of literature for a very long time, but the question of whether this rhetorical trope is possible visually has led to some more critical discussion. Some authors just do not like what are seen as essentially linguistic achievements being applied to other expressive forms. Film-makers have of course never been limited in this way and there are many examples of quite explicit metaphors in film from film's earliest days; notable examples are the well-known association drawn between the violent suppression of striking workers by police and the slaughtering of a cow in Sergei Eisenstein's Strike (1924) or the construction of commuters heading to work as a flock of sheep in Charlie Chaplin's Modern Times (1936). Whereas these early examples were typically constructed filmically by directly intercutting visuals showing source and target domains, this method is nowadays seen as rather too direct, and so metaphors are constructed more subtly, building the invited associations into the storyworld in more or less motivated fashions. Several films by Alfred Hitchcock make extensive use of such constructions, including, for example, the metaphors of a psychological trauma appearing throughout Spellbound (1945) as discussed by Gibbs, Jr. [1] or the much commented closing scene from North by Northwest (1959). Several further examples are discussed in Whittock [2] and somewhat formalised in Bateman [3].

Second, with the advent of cognitive metaphor theory by George Lakoff and Mark Johnson [4], it became far more accepted that metaphor be treated as a cognitively anchored operation of structuring mappings across domains, rather than simply a 'style of speech', or rhetorical trope. This has been taken up by a broad range of scholars; early discussions can be found in Kennedy [5], while many analyses of pictorial metaphor in advertisements are offered by Forceville [6]. Multimodal metaphors, where source and target domains are expressed in different modalities, have now received considerable attention in many media, including film (Forceville and Urios-Aparisi [7]).

Third, blending theory, as set out by Mark Turner and Gilles Fauconnier [8], provides further refinements on the specific formal properties of the kinds of structured mappings that one also sees in metaphors [9]. Blending has been taken up by many scholars, including many concerned with literary interpretation [e.g., 10,11,12,13]. In film, blends are found in many places, ranging over particular aspects of narrative, such as perspective-taking, flashbacks (Gordejuela [14]) or similar, genres (Allen [15]), pastiche (Veale [16]), and media (Bateman [3]). Although blending, particularly when construed formally along the lines of morphisms between theories as proposed by Goguen [17] and articulated in detail by Kutz et al. [18], offers substantial insights into the nature of creativity and emergence of new concepts, such formalisations are generally independent of time. However, when actually used, the deployment of blends and metaphors is always a dynamic process of interpretation that unfolds by accruing clues and indications of precisely what is to be blended. This aspect of their working will be of considerable concern below.

With these three constructs on the table, we can turn to our main concern: image schemas, as introduced by Mark Johnson [19] and Lakoff and Johnson [20], along with substantial refinements and further discussion from Oakley [21], Grady [22], Gibbs, Jr. [23,24], and many more. Image schemas are most commonly introduced as dynamic embodied patterns acquired in early child development in interaction with the world. They are seen as the bridge between embodiment/perception and higherlevel, more abstract, conceptualisations. Various catalogues or lists of image schemas appear in the literature, with examples such as containment, contact, source-path-goal, blockage, removalof-constraint, centre-periphery, near-far, and many more. However, despite their appeal, image schemas have proved difficult to formalise. This is precisely because of their dual role in relating conceptualisations to embodiment since these realms constitute two very different kinds of phenomena ontologically.

To help towards formalisation, image schemas are often broken down into finer categories. Examples here include the view from a psychological developmental perspective proposed by Mandler and Pagán Cánovas [25] involving spatial primitives, image schemas, and schematic integration. There is also the common proposal building on Grady [22] concerning primary metaphors, i.e., organisations of experience that must be in place before more abstract conceptual representations can be built up by blending. These primary metaphors do not rely on sharing features, as is assumed to be the case with standard metaphorical mappings, but rather experiential correlations between 'fundamental dimensions of experience'. Primary metaphors are then thought to be directly embodied. Again, as usually the case in all areas of metaphor, although originally primarily related to verbal examples, since Ortiz [26] primary metaphors have also been extended multimodally -first to include the visual, and then to broader ranges of sensory modalities as well; this development will also be central below. Finally, quite a different kind of decomposition is pursued by Hedblom et al. [27], who develop a logical treatment where image schemas are organised into a collection of hierarchically related formal theories of varying complexity. More complex image schemas are then formed by the logical combination of the formal theories describing their simpler components.

Regardless of their precise treatment, however, image schemas are generally assumed to operate at a fundamental level in our dealings with the world and communication about that world. They are suggested as fundamental stepping stones from bodily experience to abstract thought. But, despite commonly being used in analyses of various kinds, it remains less than clear just what such analyses need to be saying in order to be effective and revealing about the phenomena being addressed. This problem is illustrated well by the concern raised with respect to purely verbal analysis by Kimmel; as he remarks:

"Although embodiment has been rightly identified as a highly interesting issue for analysis, the undifferentiated way that cognitive literary studies have equated image schemas with embodiment makes it hard to pinpoint particularly embodied text passages or deduce anything relevant about a text at large (given the near-ubiquity of image schemas in language). " (Kimmel [11,

p. 161])

There are now accordingly moves to add more empirical support for such statements, primarily in the form of experiments where depictions that are supposed to invoke particular image schemas are placed in combination with embodied tasks of various kinds, to see if there is indeed any interaction.

As one such method, interesting results have been found concerning how told stories can have directly embodied responses for participants performing selected tasks. Kimmel [28], for example, shows interactions between stories that are hypothesised to involve force image schemas and the performance of various grasping actions. Similarly, Gibbs reports work where participants are presented with two contrasting stories, one concerning a smooth relationship and one that runs less smoothly, followed by physical actions or the imagination of physical actions. As Gibbs explains: "My basic hypothesis was that people understand these two stories not by merely activating a relationships are journeys conceptual metaphor, in which the source domain is structured by the source-path-goal image schema. Instead people imaginatively simulate themselves in the journey and actually experience some embodied sense of the sourcepath-goal image schema as part of their understanding of the stories." (Gibbs, Jr. [23, p. 128])

Even though the stories in such experiments are typically constructed to ensure that there is no explicit indication of the particular image schemas assumed to be at work, the subsequent tests nevertheless regularly show effects that are difficult to characterise without the image schema being present. This kind of embodied simulation therefore appears to find quite substantial support empirically and so clearly needs to be taken further. How precisely this is to be done raises substantial challenges, however.

Image schemas, metaphors, and films

There are already many detailed considerations of image schemas in film [29,30]; Quendler [31] and Forceville [32] offer overviews, while Fahlenbrach [33] and Coëgnarts and Kravanja [34] contain many examples. A particularly useful overview table showing the distribution of image schemas across studies can be found in Coëgnarts and Kravanja [35, p. 119]. One of the reasons for this increasing engagement with film has been the growing realisation that treatments of film simply demand accounts of embodiment in order to explain the effectiveness of the medium. Support for this also received a substantial boost from the investigations of the role of simulation in film reception developed by Vittorio Gallese and Michele Guerra [36], now extended considerably in a series of publications [37,38].

Nevertheless, many discussions of image schemas in film still limit themselves to broadly descriptive accounts in which particular segments in films are labelled exhibiting one or the other image schema (Dancygier and Vandelanotte [39]). For example, some portion of film might be labelled as showing a source-path-goal organisation when some protagonist is on a journey, or as exhibiting verticality when the protagonist falls, etc. It is not immediately clear how the particularly embodied aspect of image schemas contributes to analyses of this kind, although Forceville [40] and Forceville and Jeulink [29] suggest that it is due to the engagement of image schemas that particular emotional and affective responses to film might be best explained.

A rather more explicit account of the role of embodiment is proposed by Müller and Kappelhoff [41], who argue that applying image-schema 'labels' as a post-hoc treatment of what has already happened in engagement with a film is insufficient. Instead, one needs to address that engagement in its own right and, for this, they propose the central construct of the cinematic expressive movement. This is a generalisation of already existing embodied accounts of gesture and, in particular, the use of gesture to dynamically construct extended metaphors in interaction (Cienki and Müller [42]).

One example of this approach offered by Müller and Kappelhoff is an extended sequence near the beginning of William Wyler's film Jezebel (1938), where the main character, played by Bette Davies, arrives at a party. The scene opens with the character arriving before the house where the party is taking place on a galloping and rearing horse, which she jumps off of, leaving a servant boy to take care of the horse and exchanging a few lively words with him before running into the house, past several waiting servants, into the main room of the party where she is shown moving through largely static groups of party guests. Both the sound and the camera are highly dynamic; the sound emphasises ongoing action, while the camera follows both her movement and its own movement around groups and servants providing drinks for the party-goers. As Müller and Kappelhoff then observe: "For the viewers, this figuration of cinematic expressive movements becomes an obstacle course of affective mood alternations, which they literally realize as a sequence of perceptual sensations taken as bodily experience and from which metaphorical meaning emerges successively. " (Müller and Kappelhoff [41, p. 169])

As Müller and Kappelhoff note in several further detailed examples, this mode of engaging with audiovisual materials is by no means limited to narrative film and can play an equally strong role in non-fictional genres such as news reporting. The critical point, echoing and extending the position of Gallese and Guerra above, is that engagement with film and similar audiovisual media is already a thoroughly embodied experience.

Commonalities between image schema definitions and treatments of film

At this point it is interesting to pay closer attention to a striking parallel between the development of discussions of image schemas and that of film analysis. On the one hand, we are frequently reminded when considering image schemas that:

"when we describe the image-schematic structure alone, we never capture fully the qualities that are the flesh and blood of our experience. " (Johnson [43,p. 28])

On the other hand, we have the development of 'neo-phenomenology' in film [44,45], which also emphasises that without consideration of the embodied nature of perception, analysis of film is in considerable danger of simply missing the point. Vivien Sobchack makes this particularly clear:

"Nearly every time I read a movie review in a newspaper or popular magazine, I am struck once again by the gap that exists between our actual experience of the cinema and the theory that we academic film scholars write to explain it-or, perhaps more aptly, to explain it away. " (Sobchack [44,p. 53])

Authors such as Sobchack, Müller and Kappelhoff, and others all emphasise that film needs to be seen as a 'specific mode of experience', where filmic features such as cinematic movement expressions function as affective temporal gestalts (Müller and Kappelhoff [41,p. 21]). Extended metaphors are then seen as emergent in performance, rather than being fixed blocks of meaning that are simply applied to data to label interpretations. This dynamic generation of meaning through embodied perception in film is precisely the position argued to hold for image schemas as well by, for example, Gibbs:

"image schemas may be described as emergent properties that arise from different 'cycles of operation' constituting a person's life and represent a kind of 'structural coupling' between brain, body, and world. Image schemas reflect a form of stability within cognitive systems. " (Gibbs, Jr. [23, p. 131]) "my argument has been that image schemas are created on-the-fly as part of people's ongoing simulations . . . Image schemas are not divorced from their bodily origins . . . This perspective helps restore image schemas to their rightful status as 'experiential gestalts'... " (Gibbs, Jr. [23, p. 132]) Thus, we have, on the one hand, image schemas as 'experiential gestalts' and, on the other hand, film as driving 'temporal gestalts' of orchestrated affective responses. As Forceville [40] suggests in his analyses of the force image schema in several animation films, the essential embodiment of image schemas is consequently crucial to their use in analysis. The idea is that embodied viewing itself involves intrinsic responses that are directly associated with, or even caused by, what is perceived. Such responses can then ground more complex, and abstract, conceptual metaphors that nevertheless always maintain contact with their embodied foundation.

A very appealing aspect of Gibbs' approach for both film analysis and for considerations of image schemas in general is then to consider image schemas not only in terms of 'simulations' but also as simulators. As he explains: "a simulator provides something close to what it actually feels like in a full-bodied manner" (Gibbs, Jr. [23, pp. 118-119]). Drawing an analogy with flight simulators, then: "As a simulator, image schemas provide a kinesethetic feel that is not simply the output of some abstract computational machine, but the results of full-bodied experiences that have textures and a felt-sense of three-dimensional depth. " (Gibbs, Jr. [23, p. 119

])

Construing image schemas in this way has several important consequences both for defining image schemas further and for their use in analysis, particularly of films. Moreover, as we shall briefly now discuss, it also raises the possibility of more direct 'depictions' of image schemas making use of the expressive resources of film.

Embodiment in representation and the role of multimodality

Many authors have proposed visual representations for image schemas over the years as a contribution to their formalisation and in order to support interpretations of their intended import (e.g., Langacker [46]; Talmy [47], Hedblom and Kutz [48]). The most recent such proposal is the image schema language set out in Hedblom et al. [49], which seeks to overcome the tendency of earlier visual representations to become quite complex by being more thoroughly 'diagrammatic' in the sense of supporting compositionality. All such proposals are, however, static, which stands in opposition to the understanding of image schemas as dynamic processes set out above. Indeed, the dynamic view of image schemas clearly suggests that there may be no such thing as 'static' image schemas at all, thereby raising even more challenges for static representations. As Gibbs argues: "People continually simulate 'static' schemas in a more dynamic manner than is mostly assumed in cognitive linguistics" [23, p. 120]. This is also supported developmentally: "The acts of going in and out of containers are what matter to infants, more than the containers themselves; these are not static conceptions" (Mandler and Pagán Cánovas [25, p. 515]); an argument also made by Dewell [50].

Nevertheless, it is of course by no means the case that only dynamic representations can represent dynamic situations and so the issue of representation needs to be addressed more carefully. Many static media have developed techniques for expressing movement while remaining within the limits of their materiality. The techniques employed in representations of image schemas have until now been rather limited, however. For example, some representations employ arrows to show directions of movement, others employ sequential panels as found in comics, and still others employ further resources common to comics such as blurring and motion lines (cf., e.g., McCloud [51]; Cohn [52]; Bateman [53]).

For the purposes of capturing image schemas, however, all representations, whether those are filmic, static pictures, diagrams, verbal expressions, comics, tangible interfaces, or even logics, can be evaluated by asking how well they function as embodied 'simulators' for the phenomena they are attempting to capture. Similar considerations can in fact be applied to all representations. This is an essential result of current work on multimodality and multimodal semiotics of the kind set out in Bateman et al. [54] and provides a clear method for asking (and evaluating) what effective representations or depictions of image schemas could be.

One can also, for example, even consider alternative verbal representations. Cameron [55] argues against the classic A is B representation of metaphors in favour of a form emphasizing the dynamics of metaphor construction, where the relation between source (vehicle) and target is characterized as 𝑉 ∼ 𝑇 → 𝑀 ing accompanied by "short descriptive summaries of metaphorizing trajectories; the metaphorizing narrative to describe multiple interwoven trajectories." (Cameron [55,p. 33]). An example would be: "𝑀 ing : a dry desert of unending blankness ∼ how the sleepless nights feel" (Cameron [55,p. 27])

We can immediately see how this formulation may indeed be more effective in offering a 'simulator' for the intended experience, simply because it draws on an enriched verbalisation capable of evoking embodied experience. This is not directly present in a sleepless night is unending desert version, although with suitable additional filling-in of intentions this difference could be reduced. This is actually the situation holding for diagrammatic representations in general. The bare iconic depictions in such representations place constraints on interpretation -in the form of 'operations' that are compatible with the iconics -but these need to be filled in indexically to ground them in experience, i.e., in simulation. The appropriate kind of 'filling in' is then usually cued conventionally by particular expressive techniques -for example, and as noted above, those developed for comics. Any static representations of image schemas should therefore take particular care to specify how this enrichment of the bare iconics is intended to proceed.

An exception to the need to fill in additional details may, however, be offered by media which already match the embodied experience of image schemas sufficiently; this appears to be satisfied by suitably designed films. Now, such depictions do not need to strive for maximal realism -it appears that certain depictions of movement, force, resistance can be evoked by animation as well (Forceville [40]). It would be interesting, therefore, to explore the boundaries of this: considering, for example, whether some kinds of anthropomorphic characters are required or whether abstract (deformable) shapes might be sufficient as well.

Although following this line of development might help achieve better, i.e., more directly interpretable, representations of image schemas, we are still left facing the key problem of how to get at, and even model, image schemas' essential experiential aspect. For this, simulation appears inescapable. Earlier proposals for relating accounts to simulation, such as those found in embodied construction grammar, are summarised in Bateman [56] and explored further in, for example, Bateman et al. [57] and Pomarlan and Bateman [58]. Here the essential idea was to employ hybrid reasoning, where symbolic representations are placed in correspondence with abstractions linked directly to simulation conditions. Simulations then run and their final states are fed back to symbolic representations via image-schematic constructs. This would then match with Gibbs' conclusion that: "Image-schematic reasoning does not simply mean doing something with one's mind, but constructing a simulation of experience using one's body" [23, p. 115]. As we saw above, this is also the conclusion reached in studies of film, where it is the direct experience and engagement of viewers with the unfolding filmic experience that appears prior, not a labelling in terms of metaphors or image schemas. The link down to actual first-person simulation seems crucial. But how can one get to such an internal view?

In cognitive robot architectures such as those developed within the interdisciplinary research centre on 'Everyday Activity Science and Engineering' (EASE), an explicit logging of internal states is a central component of the model (Beetz et al. [59]). These 'narrative enabled episodic memories' (NEEMs) are intended primarily for learning ways of performing tasks on the basis both of observing human behaviour and of simulated trial-and-error, and for performing meta-cognition on how to solve problems that arise regularly in everyday activities. One might, therefore, also see this as a potential source of 'internal' data of the form intrinsic to the simulation view of image schemas. That is, internal states of actuators, forces applied, resistances met, and so on may offer usable data for revealing image-schematic generalisations, albeit not human image schemas. Attempting to learn patterns from such datasets might nevertheless be indicative of how such processing might proceed in humans and it would be an interesting range of experiments to see just what image-schema-like generations might be acquired. There is certainly still very much to explore here.

Finally, one might then complete the loop back to representations and visualisations of situations characterised in terms of such 'image schemas'. Within the EASE framework, there are already close hook-ups between robot descriptions, robot perception, and visualisations of agents and situations in the form of 'digital twins'. This means that, ideally, all internally represented states and actions can be re-played in 3D virtual reality. This might then offer a useful interface both for exploring acquired image schemas further and for investigating distinct visualisation options directly. Crucial will be the appropriate maintenance of the dynamics of the phenomena explored throughout as only then do we remain within the domain of phenomena relevant for image schemas.

Closing remarks

This position paper has proposed that it might be beneficial to explore several connections between the areas of simulation, visualisation, and robotics for moving closer to an embodied understanding of image schemas. Similarities between shortcomings observed for formalising image schemas and for analysing film were used to suggest gaps that need filling. Although there is a long history of discussions concerning the 'metaphorical' nature of thought and experience which may be true in an abstract kind of way, the ready import of metaphorical notions into embodied engagement with the world has not so far been successful. Operating with image schemes largely in terms of abstract labels may not then be enough. The treatment of both image schemas and metaphorical readings of experience as essentially emerging from dynamic embodiment appears unavoidable.

This suggests a couple of areas where further research might be profitably explored for revealing more about the workings of image schemas:

• First, it would perhaps be worthwhile designing (and evaluating) distinct medial variants of image schema depictions. Different media have different affordances and so might support different points of access to image schemas and for making the gaps in existing proposals clear. Here it would even be interesting to explore possibilities beyond the visual, with other sensory modalities perhaps making additional contributions to how image schemas are to be conceived. Developing these would certainly constitute a range of attractive student projects, for example. • And second, performing learning on robotically acquired internal states while robots are performing various tasks might be investigated for the potential emergence of image-schemal-like generalisations. The data used in such studies should naturally range across all modes available to the robot so as to provide holistic views of the robots' placement in their environments, and be linked to goals and notions of success and difficulty. This is evidently considerably more challenging and so might provide a range of interesting projects for various stakeholders in the area.

Both directions, individually or combined, mark out paths by which image schemas might be brought successively under tighter theoretical control.