Efective interaction design for automated vehicles is a challenging matter. Be it interacting driver/operator position or being interacted with by the vehicle as a part of its road environment, the challenge lies with how to integrate the new contents or address the novel challenges brought about by the vehicle being (partially) automated. This typically takes the form of additional indicators (e.g., external displays) or exploration into diferent interaction modalities. In this paper, I want to share my view on how many automation challenges might be better addressed using minimal additional indicators or re-using already existing indicators and techniques as much as possible. Based on experiences from recent studies(specifically, one study on external HMI designs and one on an in-vehicle HMI), I argue for a more frugal approach to automation interaction design.
The transition from fully manual to automated transport is a continuous process and will persist as such for quite some time to come. Within this transitional period, humans are confronted with a wide range of automation functions or automated subsystems across an equally wide range of means of transportation. Even when limited to only the road environment, there are passenger which all look diferent depending on who manufactured them and can difer in their degree of automation and other related constraints (e.g., maximum velocity or braking behaviour). Taking this into consideration, there is agents.
It is this complex interaction space that appears to call for a need for additional information and clarification on how this space is to be navigated by the human user
an automated vehicle and might have to perform partial tasks, (de-)activate systems, or intervene or it might be someone who is simply a part of the vehicle’s driving nicate the vehicle in order to navigate trafic safely and efectively - just as with non-automated vehicles. The is then to pursue other modalities, such as haptics [6] or environment and needs to have an avenue to commu- [5]. Thus, with the adding of additional information flow © 2023 Copyright for this paper by its authors. Use permitted under Creative Commons License auditory [7] in order to distribute the load across sensory Attribution 4.0 International (CC BY 4.0). channels. Another strategy is to ask the question: “Is it Still, the participants from this study had all mentioned really necessary?” and potentially reduce the amount of a strong preference for additional information and indiadditional indicators used, reduce the complexity of the cators, despite their reliance on “traditional” indicators communicated content, re-use already existing interfaces, during the interactions. Thus, our take-away from this cues, or indicators, or similar strategies. study was that we needed to reduce the complexity, use
Over several studies and instances, I have now had ex- only very simple animations (e.g., blinking) and to not periences from which I went away with a strong tendency use images or verbal content that needs additional mental towards the latter, i.e., trying to reduce the communica- processing. In a field study later in the project [ 9], [10], tion down to what is strictly necessary and steering away we did a performance assessment of the eHMIs, which from additional indicators, modalities as the exploratory had finally been realized as one-dimensional LED-strips. default. Here, I would like to share the experiences from The lessons we had learned previously were well-learned three such studies across two cases to exemplify the idea. as it appeared, as we finally did see interaction success when the indicators were active as opposed to the control 2.1. Case 1: External on-vehicle condition. While the efects were noticeable, they were indicators not major and it was clear that the shuttle was able to navigate trafic also in the control condition – just not quite as well. In this study in particular, we noticed that vehicle position (e.g., stopped further into an intersection vs. with greater distance as well as the fact whether the shuttle was visibly accelerating or decelerating were by far the most used indicators by the other trafic participants.
which we investigated eHMI (external Human-Machine Interface) designs for automated shuttles to communicate with manual vehicles and other road users [8]. It was a comparison study with several diferent visualisations on a front-mounted display as well as two side-mounted displays towards the front. The designs themselves were based on an online-questionnaire, from which we had 2.2. Case 2: In-cabin indicators already extracted what we thought were the most suitable colours, icons, and animation patterns. In the study, In another study from a diferent project, we made simiwe had the shuttle circulate around the track and then lar experiences, this time regarding in-vehicle indicators interact with manually driven vehicles and pedestrians [11]: For the purpose of exploring the suitability of (amcrossing the road across several interaction scenarios and bient) light displays to signal driving modes and control diferent visualization conditions. transitions in highly automated vehicles, we had installed
As far as the eHMI comparison went, we stopped the lights below the windshield, on the steering wheel, and in study before we got to the double digits in terms of par- the footwell. These would then emit light signals in two ticipant numbers, as we had learned what we needed by conditions (static and dynamic (e.g., pulsing, blinking), then: There would be little to no usable data for com- with the colours having been informed by a previous parison regardless of the number of participants due to online questionnaire. Among the results from this study a clearly visible lack of willingness to engage with the were the findings that the steering wheel light was too eHMIs. The reasons for this ranged from the participants distracting and needed to be used more sparingly. The not even noticing the indicators to them noticing but light in the footwell, despite having been the largest in deciding to not make use of them as the interaction went terms of illuminated surface, was not found to have any along. In the post-interaction interviews, we learned that noticeable efect or even be noticed by most participants the participants were very quick to revert back to what while driving. The other lights were found to have an they knew when interacting with the shuttle and its un- efect, though it was not very large overall when comknown or hard to expect driving behavior. This means pared to the control condition without lights. By itself, that they would look at the standard lights and turn in- this is about what one should expect when running a dicators and, when in doubt, simply wait and see what study: In practice, some things work other than intended happens, as they were never in any real danger. When and some not at all. the participants did look at the indicators, we learned What was interesting, however, was an observation that many of the designs that seemed to work on paper that was rather minor in the grander scheme of things (and according to the questionnaire results) did not work and, thus, not included in the original publication but as well in context, especially in light of how drivers ap- is something which I would like to share here: The reportioned their attention while driving: Anything that sults from the post-interaction interviews yielded more required sustained attention (short animations in partic- positive comments towards the light interfaces and also ular) was dificult to process, leading the participants to a strongly expressed need for additional indicators in again go with their usual gaze and action patterns for automated vehicles, which was not reflected in the quaninteraction with manually driven vehicles. titative data. When looking at the data more closely, we found this discrepancy to occur within-participants as well, i.e., a single participant would speak highly of the lights and express a need for additional HMIs or indicators, yet that same participant would rate light and baseline conditions similarly in the questionnaires. Just like with the first eHMI study, it seemed that the indicators the participants expressed to require, were not actually that necessary in the actual interaction and quickly too much. This can then result in participants interacting with what they would be used to normally in such cases, i.e., interacting just as they would in the baseline, thus explaining the lack of diferences in actual performance.
So what do we take away from this? On the one hand, there is a component of uncertainty when interacting with automated vehicles and that component needs to be addressed. On the other hand, users are quick to fall back to what they are used to from interacting with manually driven vehicles and the success rate in this “interaction baseline” is already rather good. A part of this is certainly related to the fact that controlled field study conditions cannot be high risk conditions, so the measurable efects of a safety-critical technology should be expected to be more limited in field trial condition. Another aspect is certainly that just because an interface was developed, does not mean that it was the ideal interface for the purpose, so a smaller than expected success over the control condition may just as well be a hint towards improvability of the interface.
Another take-away, however, is the observation of “what works” and how users are easily able to reapply that which worked before to the novel context, despite explicitly expressing to require additional information or indicators (as it was the case in eHMI study 1 or the invehicle HMI). There is a diference between the transition phase, in which we are right now, where the automation technologies are novel, manifold, and to many unknown in detail, to a phase in which the technology has been in use for some time and is largely known to most in terms of capabilities and limitations. In such a context, explanatory indications are not necessary, as users already know what to expect. Users need to know if a machine or device is on or of (status indicator), need rudimentary information regarding the planned actions (see: turn indicators, braking light) and take the rest from the information present in the context.
In all the provided examples but especially the mixed trafic experiments in eHMI study 2, we have had a glimpse of users falling back into these for them safe and established interaction behaviours, despite interacting with a novel technology – and for the most part, it worked out just fine. From this, I believe that the communication of additional, automation-specific information, is still important but perhaps not as vital as we might have additionally assumed. Furthermore, I would like to use this as a motivation to explore even simpler indicators to communicate automation-relevant information, to even re-use as many of the existing indicators as we can. If, to only provide an illustrative example, turn indicators sufice, why use something else? The resulting HMIs and interfaces will certainly be less fancy or futuristic, perhaps a bit boring even, and less attractive for the often brilliant and creative minds that explore interface and interaction design. This is why I would like to put explicitly put this direction forward, discuss with and encourage others to explore frugal interaction design, employing existing designs and interaction channels to solve novel challenges. It might not look that exciting in practice, though the long-term interaction results might outdo the short-term excitement gained by a nice looking HMI that nobody else had done before.
which has received funding from the European Union’s Horizon 2020 research and innovation program under grant agreement number 875597. The Innovation and Networks Executive Agency (INEA) is not responsible for any use that may be made of the information this document contains. Proceedings of the 31st Australian Conference on projects, Frontiers in Robotics and AI 9 (2022). Human-Computer-Interaction, OZCHI’19, Associ- URL: https://www.frontiersin.org/articles/10.3389/ ation for Computing Machinery, New York, NY, frobt.2022.949135. doi:1 0 . 3 3 8 9 / f r o b t . 2 0 2 2 . 9 4 9 1 3 5 . USA, 2020, p. 437–441. URL: https://doi.org/10.1145/ [11] A. G. Mirnig, M. Gärtner, V. Wallner, C. Demir, 3369457.3369509. doi:1 0 . 1 1 4 5 / 3 3 6 9 4 5 7 . 3 3 6 9 5 0 9 . Y. Dönmez Özkan, J. Sypniewski, A. Meschtscher[5] T. Tanaka, K. Fujikake, Y. Yoshihara, N. Karatas, jakov, Enlightening mode awareness: Guiding H. Aoki, H. Kanamori, Study on acceptability of and drivers in the transition between manual and autodistraction by driving support agent in actual car en- mated driving modes via ambient light, Personal vironment, in: Proceedings of the 7th International and Ubiquitous Computing - Theme Issue on EngagConference on Human-Agent Interaction, HAI ing with Automation: understanding and designing ’19, Association for Computing Machinery, New for operation, appropriation, and behavior change. York, NY, USA, 2019, p. 202–204. URL: https://doi. Springer (2023). Under review. org/10.1145/3349537.3352765. doi:1 0 . 1 1 4 5 / 3 3 4 9 5 3 7 .
3 3 5 2 7 6 5 . [6] G. D. Gomes, R. Flynn, N. Murray, Hedgehog:
A steering wheel prototype to continuously enhance driver’s situation awareness during conditional vehicle automation, in: Adjunct Proceedings of the 14th International Conference on Automotive User Interfaces and Interactive Vehicular Applications, AutomotiveUI ’22, Association for Computing Machinery, New York, NY, USA, 2022, p. 222–225. URL: https://doi.org/10.1145/3544999.
3554973. doi:1 0 . 1 1 4 5 / 3 5 4 4 9 9 9 . 3 5 5 4 9 7 3 . [7] P. Larsson, Speech feedback reduces driver distraction caused by in-vehicle visual interfaces, in: Proceedings of the Audio Mostly 2016, AM ’16, Association for Computing Machinery, New York, NY, USA, 2016, p. 7–11. URL: https://doi.org/10.1145/ 2986416.2986435. doi:1 0 . 1 1 4 5 / 2 9 8 6 4 1 6 . 2 9 8 6 4 3 5 . [8] A. G. Mirnig, M. Gärtner, A. Meschtscherjakov,
M. Tscheligi, Blinded by novelty: a reflection on participant curiosity and novelty in automated vehicle studies based on experiences from the field, in: F. Alt, S. Schneegass, E. Hornecker (Eds.), Mensch und Computer 2020 - Tagungsband, ACM, New York, 2020, p. 373–381. doi:1 0 . 1 1 4 5 / 3 4 0 4 9 8 3 .
3 4 0 5 5 9 3 . [9] A. G. Mirnig, M. Gärtner, V. Wallner, M. Gafert,
H. Braun, P. Fröhlich, S. Suette, J. Sypniewski, A. Meschtscherjakov, M. Tscheligi, Stop or go? let me know! a field study on visual external communication for automated shuttles, in: 13th International Conference on Automotive User Interfaces and Interactive Vehicular Applications, AutomotiveUI ’21, Association for Computing Machinery, New York, NY, USA, 2021, p. 287–295. URL: https://doi. org/10.1145/3409118.3475131. doi:1 0 . 1 1 4 5 / 3 4 0 9 1 1 8 .
3 4 7 5 1 3 1 . [10] A. G. Mirnig, M. Gärtner, P. Fröhlich, V. Wallner, A. S. Dahlman, A. Anund, P. Pokorny, M. Hagenzieker, T. Bjørnskau, O. Aasvik, C. Demir, J. Sypniewski, External communication of automated shuttles: Results, experiences, and lessons learned from three european long-term research