Developments in the field of semi-autonomous driving will increasingly free the driver from the main driving task. Conditional autonomous driving allows the driver not to constantly monitor her/his environment, but s/he must still be able to regain control of the vehicle at any time if the situation requires it. That's why it is important to maintain driver's Situation Awareness while is performing a secondary task. This paper presents a model of an adaptive, full-body and multimodal Human Vehicle Interaction (HVI) for supervision.
If the driver is engaged in a secondary task, s/he will be out of the control loop and a return to
manual driving will be more dificult because it will require a higher cognitive load [
Overlay application
Visual (and auditory) Personal device Interaction
Location Haptic (vibrations)
Backrest and pan Visual and vocal
Vehicle’s interior
Conveyed information
Obstacle around the car Lane marking degradation
Information & alerts nearby environment Environment condition
Car status
The objective of our work is to design a full-body and multisensory experiences by considering
the whole car interior and peripheral interaction in order to support driver supervision task
and increase situational awareness. The objective is to find a combination of modalities of
interaction that should adapt to the environment and the automation state. The choice of
combination may also depend on the driver’s state in a second step. The second objective of this
concept is to convey information about the vehicle information to the driver in order to increase
her/his trust to the system. Thus, vehicle’s actions may not match driver’s expectations in some
situations. That is why transparency should be taken into account [
The model of adaptive, full-body and multimodal Human Vehicle Interaction (HVI) for supervision presented in Figure 1 combines haptic interaction (vibration in seat), visual interactions (ambient lights and split-screen application) and vocal interaction (conversational agent). This HVI conveyed information to the driver about her/his environment through diferent modalities and part of the car. The various reflections on the design of these concepts are described below and in Table 1: • Haptic seat: Because driver may no longer have her/his hands on steering wheel and will look away from the road the seat seems promising for transmitting information through haptic interactions [5, 6].
Hypothesis: it may provide the driver with a more complete peripheral and extended vision of her/his environment [7]. • Overlay application: Personal devices used by the driver while performing secondary task represents a novel and promising opportunity to communicate SA-related information to the driver in a peripheral way [8].
Hypothesis: it may provide the driver with a more complete peripheral and extended vision and increase her/his trust to the automation [9]. • Conversational agent and ambient lights: Because ambient lights are often used to transmit information into peripheral vision of the user, it should represent a promising visual interaction to guide driver’s attention [10]. Moreover, to make the user perceive the vehicle as a companion, it will be interesting to communicate with it like with any other passenger. The driver should be able to ask the vehicle about her/his surrounding environment conditions, car status...
Hypothesis: it may develop transparency of the automation and increase driver’s situation awareness and trust to the system.
The introduction of tangible interactions within my project is currently under consideration. In the case of the haptic seat, it is a question of physically represent an extension of the vehicle environment through vibrations in the seat. Particularly, vibrations will map the proximity sensors and sensors bound to the identification of lane markings into the driver’s seat. The reasoning is quite similar regarding the use of ambient lights in the vehicle interior. Indeed, they should spatially represent the vehicle’s environment by directing the driver’s attention. The driver could then choose to obtain more information about this state by questioning the conversational agent. It might also be interesting to make the conversational agent more tangible by using a physical representation to begin with. This may comfort the driver when using the autonomous system and might increase the trust in the system. I would like to take advantage of this conference to discuss the tangibility of the proposed concepts. I also would like to get expert advice on how to use tangible interactions to increase driver’s situation awareness and trust such as the project Carvatar [11] that uses a robot head placed on the car dashboard to inform the driver about the car’s state and situation using social cues and anthropomorphism.
This work has been supported by Hasler Foundation in the framework of Ad-Vitam project. The authors would like to thank all the persons who contributed to this paper. [5] T. Grah, F. Epp, M. Wuchse, A. Meschtscherjakov, F. Gabler, A. Steinmetz, M. Tscheligi, Dorsal haptic display: A shape-changing car seat for sensory augmentation of rear obstacles, in: Proceedings of the 7th International Conference on Automotive User Interfaces and Interactive Vehicular Applications, AutomotiveUI ’15, Association for Computing Machinery, New York, NY, USA, 2015, p. 305–312. URL: https://doi.org/10.1145/2799250. 2799281. doi:10.1145/2799250.2799281. [6] A. Riener, A. Ferscha, P. Frech, M. Hackl, M. Kaltenberger, Subliminal vibro-tactile based notification of co<sub>2</sub> economy while driving, in: Proceedings of the 2nd International Conference on Automotive User Interfaces and Interactive Vehicular Applications, AutomotiveUI ’10, Association for Computing Machinery, New York, NY, USA, 2010, p. 92–101. URL: https://doi.org/10.1145/1969773.1969790. doi:10.1145/1969773. 1969790. [7] M. Capallera, P. Barbé-Labarthe, L. Angelini, O. A. Khaled, E. Mugellini, Convey situation awareness in conditionally automated driving with a haptic seat, in: Proceedings of the 11th International Conference on Automotive User Interfaces and Interactive Vehicular Applications: Adjunct Proceedings, AutomotiveUI ’19, Association for Computing Machinery, 2019, p. 161–165. URL: https://doi.org/10.1145/3349263.3351309. doi:10.1145/3349263.3351309. [8] D. Miller, A. Sun, M. Johns, H. Ive, D. Sirkin, S. Aich, W. Ju, Distraction becomes engagement in automated driving, Proceedings of the Human Factors and Ergonomics Society Annual Meeting 59 (2015) 1676–1680. URL: https://doi.org/10.1177/1541931215591362. doi:10.1177/1541931215591362. arXiv:https://doi.org/10.1177/1541931215591362. [9] M. Capallera, Q. Meteier, E. d. Salis, L. Angelini, S. Carrino, O. Abou Khaled, E. Mugellini, Tâche secondaire et conscience de l’environnement, une application mobile pour véhicule semi-autonome, in: Actes de la 31e conférence francophone sur l’Interaction HommeMachine (IHM 2019), ACM, 2019, p. 12:1–10. URL: https://hal.archives-ouvertes.fr/ hal-02381596. doi:10.1145/3366550.3372258. [10] A. Löcken, W. Heuten, S. Boll, Autoambicar: Using ambient light to inform drivers about intentions of their automated cars, in: Adjunct Proceedings of the 8th International Conference on Automotive User Interfaces and Interactive Vehicular Applications, AutomotiveUI ’16 Adjunct, Association for Computing Machinery, New York, NY, USA, 2016, p. 57–62. URL: https://doi.org/10.1145/3004323.3004329. doi:10.1145/3004323.3004329. [11] J. Zihsler, P. Hock, M. Walch, K. Dzuba, D. Schwager, P. Szauer, E. Rukzio, Carvatar: Increasing trust in highly-automated driving through social cues, in: Adjunct Proceedings of the 8th International Conference on Automotive User Interfaces and Interactive Vehicular Applications, AutomotiveUI ’16 Adjunct, Association for Computing Machinery, New York, NY, USA, 2016, p. 9–14. URL: https://doi.org/10.1145/3004323.3004354. doi:10.1145/3004323.3004354.