Image Schematic Metaphors in Air Traffic Controllers’ Language Stephan Huber1 , Patrick Schulz1 , Eric Hauke1 and Jörn Hurtienne1 1 Julius-Maximilians-Universität Würzburg, Sanderring 2, Würzburg, 97074, Germany Abstract Our objective is to identify image schematic metaphors (ISM) in the application domain of air traffic control and thereby learn about controllers’ mental models. Through tagging ISM in controllers’ spoken language, we identified metaphors that indicate three perspectives on air traffic, namely (1) a geographical perspective of an aircraft’s physical position and flight path, (2) a metaphorically instantiated distribution of responsibility for air traffic, and (3) metaphors referring to the organizational level of air traffic management. We discuss target domains that may pose particular challenges for the design of coherent interfaces due to their mapping to multiple source domains, sometimes competing with physical mappings. Our main preliminary contribution is a list of metaphorical instantiations serving as foundation for innovative, yet intuitive-to-use interfaces in future prototypes for the context of air traffic control. Keywords image schema, air traffic control, safety critical, intuitive use 1. Introduction Image schema are learned in early childhood through experiences with environmental condi- tions, processes and constraints [7]. Connecting abstract target domains and physical source domains, image-schematic metaphors (ISM) widen the range of how we make sense of our world. Acquired through shared experience, the image schemas not only manifest in our mental models but also our language [7]. Conveniently for user interface designers, ISM can be extracted from the users’ language to understand how they think about their domain of application. Coding of language can be done manual [8, 17] or with the support of linguistic pattern recognition [2], machine learning or rule based extraction [3, 18]. Once identified for a context, ISM can be repeatedly used for design processes. Designs inspired by ISM support intuitive [7] and age-inclusive [16] interaction, which has been shown in a variety of application domains so far, including architecture, banking, social robotics, infotainment, and many others [6]. The Sixth Image Schema Day (ISD6), March 24–25, 2022, Jönköping University, Sweden Envelope-Open stephan.huber@uni-wuerzburg.de (S. Huber); patrick.schulz@stud-mail.uni-wuerzburg.de (P. Schulz); eric.hauke@stud-mail.uni-wuerzburg.de (E. Hauke); joern.hurtienne@uni-wuerzburg.de (J. Hurtienne) GLOBE https://psyergo.uni-wuerzburg.de (J. Hurtienne) Orcid 0000-0002-5619-3222 (S. Huber); 0000-0001-5295-9772 (J. Hurtienne) © 2022 Copyright for this paper by its authors. Use permitted under Creative Commons License Attribution 4.0 International (CC BY 4.0). CEUR Workshop Proceedings http://ceur-ws.org ISSN 1613-0073 CEUR Workshop Proceedings (CEUR-WS.org) 1.1. Context of air traffic control In safety-critical domains, such as air traffic control, a fast performance that is not impeded by complex interfaces is crucial. Air traffic controllers monitor and coordinate traffic within their sector through communication with pilots and controllers of neighboring sectors. By law controllers are required to document any clearance they give to aircraft. Developments of their workstation have so far been mainly technology-driven. Past ethnographic studies revealed that existing interfaces enforced a dualistic perspective on air traffic separating the perspective on physical air traffic (i.e. the radar image on a vertical display) from the management of responsibilities regarding the same traffic (i.e. documentation and communication with paper flight strips on a commonly horizontal interface) [14]. For the sake of increased efficiency while maintaining safety, recent attempts of redesigning air traffic control workstations strive towards one coherent interface incorporating documentation and communication of clearances with the existing radar information [1, 4, 9, 15]. In order to make the next innovative generation of interfaces more intuitive-to-use and thus efficient, as a first step, controllers’ current mental models need to be determined. One approach towards understanding controllers’ mental model of their work environment is tagging Image Schematic Metaphors (ISM) in their language [13]. Hence, in this work, we contribute the first collection of ISM for the domain of air traffic control. Whereas prior work followed a needs based approach focusing on the why of controllers’ actions [5], the paper at hand explores the how of their views. Within the many departments and roles that are responsible for a smooth procession of air traffic our initial focus lies on approach controllers. The main task of approach controllers entails picking up incoming air traffic, channeling it, and feeding it in a structured manner towards the airport. Oftentimes they additionally take responsibility for outgoing traffic crossing their sector. In sum, our research question is “What cognitional structures of air traffic controllers on the approach position can be derived from image schematic metaphors occurring in their work place related language?”. 2. Method 2.1. Data source Air traffic controllers’ language on the job and particularly during radio communication is highly standardized for minimizing the risk of misunderstandings. This includes linguistic reductions such as the omission of prepositions, which often help to identify ISM [11]. For instance, the clearance “descend to 5000ft” is shortened as “descend 5000ft” in order to prevent any misconceptions of the preposition “to” as the numeral “two”. Due to this highly standardized and reduced vocabulary, common radio communication does not qualify for a promising data source. Instead, we used audio recordings of interviews that took place within the context of a user centered design process at two larger German airports. Eight air traffic controllers talked about their preceding shift on the approach position for about one hour each. Rather than following a fixed guide, researchers followed up on their observations of the preceding shift and asked controllers to explain events and actions of specific situations in more detail. Since no shift with live air traffic is exactly the same, events and topics varied between participants. 8 Interviews Transcribed Coded Prioritized Validated ____________________ ____________________ ____________________ 1a 1a ____________________ 1b 1b IIIIII ____________________ ____________________ 2a 2a ____________________ 2b 2b ____________________ ____________________ 2c 2c ____________________ … … Σ 8 hours 69 000 words >2000 instances 133 metaphors 35 metaphors Figure 1: Illustration of our research process from interviews to validated metaphors 2.2. Data processing and metaphor extraction First, we transcribed the recorded interviews. Then three of the authors mutually coded ISM in the first interview. For the remaining seven interviews, two of the authors mutually coded ISM in the transcript and discussed their findings with a third author until achieving consent on all instances. Coding was guided by the list of 47 image schemas from the categories BASIC, SPACE, CONTAINMENT, MULTIPLICITY, PROCESS, FORCE, and ATTRIBUTE reported by Hurtienne et al. [8]. From the transcribed interviews we extracted primary metaphors as well as complex metaphors. For example, when a controller spoke of a controller responsible for an adjacent sector of “the colleague in the south”, we coded the primary metaphor CARDINAL DIRECTION is A CONTAINER as well as the complex metaphor COLLEAGUE IN THE CARDINAL DIRECTION is SUBJECT IN A CONTAINER. Subsequently, we listed all identified ISM and prioritized metaphors, which were instantiated in more than one interview. To further condensate the data, we consolidated metaphors with identical target domain and similar source domain. Finally, in participative sessions with three controllers, we validated statements that led to ambiguous ISM where one target domain mapped to multiple source domains (Figure 1). During those validation sessions, we prompted the participating controller with a source domain (e.g., SECTOR) and asked them to form a work related sentences. We coded the ISM usage in that sentence(s) live and then presented other instantiations, which we had found. From the controllers’ explanations, we learned whether the instantiations differed based on context or between individuals. Controllers also sorted out some source domains of which they thought to be exclusively used in conversations with visitors (i.e. here, user researchers) but which are not part of their usual language. This affected mostly Germanized terms or sentences controllers would directly use without translation from English on-the-job as well as during casual communications with other professionals. 3. Results Our main result is a list of 35 domain-specific ISM instantiated in controllers’ spoken language (see Table 1). From a participative analysis with controllers, we learned that some labels for target domains are linked to two or three source domains (i.e. image schemas) and sometimes are additionally used as labels for the physical world. Table 1 Selection of 35 validated air traffic control specific ISM in alphabetical order. Target domains that map to multiple source domains are ranked descending by controllers’ preference or estimated frequency. Instantiations are translations from German. Most statements regarding individual working positions are valid and applicable for other sectors as well. TARGET DOMAIN is SOURCE DOMAIN Exemplary instantiation AIRCRAFT is an OBJECT ”Would you accept the aircraft higher?” AIRCRAFT is a CONTAINER ”Can you take out the speed of that aircraft?” AIR SPACE is a CONTAINER ”Allow them to fly into Airspace Charlie.” AIR SPACE is the start/goal of a PATH ”They seem to go from this airspace to that airspace.” APPROACH is an OBJECT ”I give him an ILSaapproach.” CAPACITY is UP-DOWN ”We should open the feeder to raise capacities.” CAPACITY is a SUBSTANCE ”I have no capacities for additional aircraft.” CARDINAL DIRECTION is a CONTAINER ”She is working in the north.” [license, position] DOWNWIND is a SURFACE ”Bring them all on the downwind for me.” DOWNWIND is a LOCATION ”That aircraft is south of the downwind.” [geographic] FINAL is a LOCATION ”This aircraft is at 5 mile final.” [fixed landmark] FINAL is a SURFACE ”This aircraft is on the final.” [final part of approach] FREQUENCY is a SURFACE ”They are on the emergency frequency.” FREQUENCY is a CONTAINER ”I am out, you can go in.” FREQUENCY is an OBJECT ”She passed me that frequency.” FEEDER is the goal of a PATH ”Go ahead, send them to the feeder.” FEEDERbis a SURFACE ”You sit on feeder today.” FEEDERbis a CONTAINER ”I will open up the second feeder.” [position] FLIGHT LEVEL is the goal of a PATH ”They climbed up to flight level 280.” FLIGHT LEVEL is a CONTAINER ”They are in flight level 280.” (DIGITAL) FLIGHT STRIP is a SURFACE ”That is not written on the flight strip.” HANDOVER is a LOCATION ”I forgot to tell you at the handover that […].” LABEL is a LOCATION ”The flight level is written at my label.” SPACING [mile] is a COUNT ”Behind a heavy I need 5, else not less than 3 miles.” SPACING [mile] is an OBJECT ”This aircraft has 6 miles.” SPACING [mile] is a CONTAINER ”They come in 4 mile intervals.” PILOT is the goal of a PATH ”I sent it to the pilot.” RUNWAY is an OBJECT ”I can give you the southern runway.” RUNWAY is a CONTAINER ”The runway is closed.” SECTOR is an OBJECT ”Do you have the -sector?” [license] SECTOR is an OBJECT ”I need an extra sector.” [division of responsibility] SPEED is an OBJECT ”Can you take out the speed of that one?” SQUAWK BOX is a CONTAINER ”To talk to the tower I need to open the squawk box.” SQUAWK BOX is a SURFACE ”Please communicate that over the squawk box.” TOWERbis a SUBJECT ”The tower is nice.” [operator in the tower position] a ILS = Instrument Landing System b Generalizes to other working positions [Tower, Feeder, Pick Up, Radar Controller] 3.1. Multiple source domains One prevalent example in approach controllers’ language is final, which refers to the final approach slope (SURFACE) or a fixed point on it (LOCATION) – depending on the preposition (see Table 1). Another example is frequency, which is associated with different source domains according to context. After receiving a frequency from a colleague (OBJECT), controllers go into or out of a frequency (CONTAINER), while they perceive other actors as being always on the frequency (SURFACE). Controllers have validated all of these linguistic instances, albeit they claim imagining the frequency mostly as a CONTAINER. Similarly, the communication interface squawk box changes its role from entering the channel (CONTAINER) to communicating over it (SURFACE). Representative for a third category of contradictory ISM is feeder, which mostly refers to the position on a task level in approach control that previous sectors send aircraft to (goal of a PATH), and who feeds the stream of approaching aircraft towards the airport. Additionally, controllers refer to the associated workspace on which they sit (SURFACE) as feeder and speak about the administrative act of opening the feeder position (CONTAINER), that is, assigning controllers to work there. Within the same category are miles as the typical unit for horizontal distance measurement in air traffic control. Controllers mostly use them when referring to spacing between aircraft: aircraft can possess a number (COUNT) of miles (OBJECT). Yet miles also describe the separation intervals in which aircraft arrive (CONTAINER, see Table 1) and label certain virtual landmarks in the physical domain (“I drive it in via 8 miles” ). Finally, there is the target domain sector. Similar to feeder described above, it can be used like a CONTAINER when opening a position or sector. Mostly, controllers treat sectors as OBJECTs. Interestingly, however, the meaning changes with the semantic context. Possession of a sector is synonym for the license to work this sector. Requiring an additional sector, on the other hand, points towards the need of splitting the responsibility for a currently merged air space. 3.2. Terminological overlaps between physical and metaphorical worlds In the context of air traffic control, some source domains are relevant both physically and metaphorically. Cardinal directions, for example, are geographical labels for points or regions (in the physical world or on a map) [12], but controllers used them as well metaphorically to describe the responsibility for a certain region. When referring to the physical position of aircraft, cardinal directions are mostly instantiated as LOCATION, (“sending an aircraft to the north” ) and less often as SURFACE or CONTAINER. The administrative responsibility is limited to metaphoric instantiations as CONTAINER. Our first association with the term runway is the physical surface. On the organizational level, controllers use runway with respect to the clearance for landing on a certain runway. Here, clearances are treated as OBJECTs that can be handed around. A further usage of runways is with respect to their availability – instantiated in language as CONTAINERs that can be closed, opened and contain a certain capacity. While aircraft are indisputably physical objects and containers, controllers use the term also on a metaphorical level depicting responsibility. For instance, controllers speak about passing the aircraft around (like an OBJECT), which is physically neither happening nor possible, or reducing its velocity by “taking out the speed” of this CONTAINER. Finally, tower and aircraft are sometimes instantiated as personifications representing the people operating them. Controllers would talk about personality traits or states of the tower [instead of the tower operator] or explain what they told the aircraft [instead of the pilots]. 3.3. Metaphors inspired by the current interface Most users see digital forms as containers they can fill in/out. Despite the digitalization and context menus popping up, controllers refer to the document of a [digital] flight strip still as SURFACE on which they can write. In contrast, the labels on the radar providing information such as current speed or altitude of each individual aircraft are instantiated as LOCATIONs. A reason may be that each label is attached to the visual representation (head symbol) of the physical position detected by radar. This ISM contradiction may be particularly interesting when attempting to merge the two tasks of monitoring and documenting into one interface. 3.4. Metaphorical models contradicting the physical world Clearances pose an interesting special case. Controllers generally refer to clearances as OBJECTs given to pilots or aircraft (as a personification of pilots). For instance, they give the clearance for climbing or descending to a certain flight level, approaching the airport or landing on a runway. However, when giving the clearance to reduce speed or altitude, controllers also speak of “taking speed out of” an aircraft, which still requires giving a permission but on a metaphorical level opposes the direction of the gesture. In detail, the controllers can only give permissions but not physically take out speed of aircraft themselves, as this is the pilots’ responsibility. 4. Discussion As expected, controllers’ view on air traffic entails the division into a physical and an administra- tive perspective – both manifested in language. In detail, rooted in traditional interface design for air traffic control aircraft are not only localized geographically in three dimensional space but also allocated to a sector, which involves a responsibility dimension. To ensure innovative, intuitive interaction, designers should consider insights on controllers’ mental models through the ISM occurring in their language as inspiration when merging the two – or even three – views on air traffic into one coherent interface. Interestingly, we found the meaning of the target domains to vary with the syntactic (prepo- sitions) or semantic context of instantiations. Depending on the context, target domains may refer to physical or administrative aspects of controllers’ work – sometimes with contradicting metaphorical patterns. Designers of future interfaces need to treat these terms and symbolic representations in their interface drafts with great care. As a foundation for novel workstations that unite the tasks of air traffic controllers, we encourage the usage of ISM listed and described in our results. 4.1. Limitations A first caveat is that our data are exclusively based on interviews with approach controllers whom we observed and interviewed after a shift on the feeder position. Looking at the data, some metaphors had been actively used by only few or even single controllers during the interviews, but were validated in the final step. Following the thoughts of Kövecses [10], these linguistic variations within this subculture of approach controllers may occur on the individual dimension. Whereas participants emphasized during validations of some ISM that they are generalizable for other positions, we strongly encourage cross checking the ISM in cooperation with respective positions before implementing them in prototypes. If ISM in language are not a product of controllers’ cognitive perspective on the domain but merely induced by the interface – as is likely the case for digital flight strips and labels – they might well be variable between different control centers or even different positions with varying workstation setups. Additionally, nuances of meaning in the ISM presented here may have changed through translation. Therefore, future work should test the generalizability of the ISM we identified and complement the list with diverging findings from other control centers, languages and national air traffic control systems and cultures. Low numbers of annotators are a common limitation in manual ratings and may cause subjective bias to the identified ISM [2]. As raters mutually coded the language samples in this study the reliability among them could not be measured. However, in a methodologically similar approach mutually trained annotators achieved substantial agreements [16]. 5. Future Work We tagged ISM in transcribed interviews with air traffic controllers and validated the most relevant and prevalent instantiations with experienced controllers. Our findings may fuel theoretical discussions about the intersections of physical and metaphorical worlds in persons’ mental models. For practitioners, the list of identified metaphors may serve as inspiration for innovation in air traffic controllers’ workstations. In future work, we aim to create interfaces based on the identified ISM that challenge the current layout. There are two research questions to be addressed in future design processes and long-term evaluations: Is it even feasible to create an interface that unites the different perspectives on air traffic control, which are currently so clearly distinct in the interface as well as the controllers’ mental models? If this is not the case, but following political decisions, the tasks are still consolidated into one interface – how long does it take for controllers’ language to adapt? Acknowledgments We thank all air traffic controllers for their precious time in this explorative approach and members of the DFS Deutsche Flugsicherung for their support at various stages. We appreciate the anonymous reviewers’ feedback that led to improvements of this paper. Part of this work was funded by the German Federal Ministry for Economic Affairs and Energy. References [1] Johanna Gramlich, Selina Pauli, Stephan Huber, Cordula Baur, and Jörn Hurtienne. Fin, whale, coin and flatterer: Exploring tangibles for air traffic control. In Sixteenth Interna- tional Conference on Tangible, Embedded, and Embodied Interaction, pages 1–13, New York, NY, USA, 2022. Association for Computing Machinery. [2] Dagmar Gromann and Maria M Hedblom. Breaking down finance: A method for concept simplification by identifying movement structures from the image schema path-following. pages 1–16. CEUR Workshop Proceedings, 2016. [3] Dagmar Gromann and Maria M Hedblom. Kinesthetic mind reader: A method to identify image schemas in natural language. Advances in Cognitive Systems, 5, 2017. [4] Hejar Gürlük, Malte-Levin Jauer, and Maria Uebbing-Rumke. Design and evaluation of a multi-touch interaction language for approach controllers. In International Conference on Human-Computer Interaction in Aerospace, pages 1–4, New York, NY, 2014. Association for Computing Machinery. [5] Stephan Huber, Johanna Gramlich, and Tobias Grundgeiger. From paper flight strips to digital strip systems: Changes and similarities in air traffic control work practices. Proceedings of ACM Human-Computer Interaction, 4(CSCW1):Article 028, 2020. [6] Joern Hurtienne, Diana Löffler, Clara Hüsch, Daniel Reinhardt, Robert Tscharn, and Stephan Huber. Happy is Up, Sad is Down - 65 Metaphors for Design. BIS Publishers, 2020. [7] Jörn Hurtienne and Johann Habakuk Israel. Image schemas and their metaphorical ex- tensions: intuitive patterns for tangible interaction. In Proceedings of the 1st international conference on Tangible and embedded interaction, pages 127–134, 2007. [8] Jörn Hurtienne, Kerstin Klöckner, Sarah Diefenbach, Claudia Nass, and Andreas Maier. Designing with image schemas: resolving the tension between innovation, inclusion and intuitive use. Interacting with Computers, 27(3):235–255, 2015. [9] Satoru Inoue, Kazuhiko Yamazaki, Hajime Hirako, and Toshiya Sasaki. Applying hu- man centered design process for designing air traffic control interfaces. In International Conference of Design, User Experience, and Usability, pages 307–316. Springer, 2015. [10] Zoltán Kövecses. Metaphor and culture. Acta Universitatis Sapientiae, Philologica, 2(2):197–220, 2010. [11] George Lakoff and Rafael Núñez. Where mathematics comes from, volume 6. New York: Basic Books, 2000. [12] Gérard Ligozat. Reasoning about cardinal directions. Journal of Visual Languages & Computing, 9(1):23–44, 1998. [13] Diana Löffler, Anne Hess, Andreas Maier, Jörn Hurtienne, and Hartmut Schmitt. Devel- oping intuitive user interfaces by integrating users’ mental models into requirements engineering. In 27th International BCS Human Computer Interaction Conference (HCI 2013) 27, pages 1–10, 2013. [14] Wendy E Mackay. Is paper safer? the role of paper flight strips in air traffic control. ACM Transactions of Computer-Human Interaction, 6(4):311–340, 1999. [15] Michael Traoré and Christophe Hurter. Exploratory study with eye tracking devices to build interactive systems for air traffic controllers. In International Conference on Human-Computer Interaction in Aerospace, pages 1–9. ACM, 2016. [16] Robert Tscharn. Innovative and age-inclusive interaction design with image-schematic metaphors. Thesis, University of Wuerzburg, 2019. [17] Armin Winkler, Kristian Baumann, Stephan Huber, Robert Tscharn, and Jörn Hurtienne. Evaluation of an application based on conceptual metaphors for social interaction between vehicles. In Proceedings of the 2016 ACM Conference on Designing Interactive Systems, page 1148–1159. Association for Computing Machinery, 2016. [18] Eloi Zablocki, Patrick Bordes, Laure Soulier, Benjamin Piwowarski, and Patrick Gallinari. Lip6@ clef2017: Multi-modal spatial role labeling using word embeddings working notes. In CLEF 2017, pages 1–17, 2017.