=Paper=
{{Paper
|id=Vol-2183/position3
|storemode=property
|title=Safe, Productive, and Socially Accepted Text Input in Highly Automated Driving
|pdfUrl=https://ceur-ws.org/Vol-2183/position3.pdf
|volume=Vol-2183
|authors=Clemens Schartmüller,Andreas Riener
}}
==Safe, Productive, and Socially Accepted Text Input in Highly Automated Driving==
https://ceur-ws.org/Vol-2183/position3.pdf
Barcelona, Spain | September 3, 2018 MobileHCI 2018 Workshop on Socio-Technical Aspects of Text Entry
Safe, Productive, and Socially
Accepted Text Input in Highly
Automated Driving
Clemens Schartmüller Abstract
Technische Hochschule As of today, in-vehicle text-based interfaces are used to en-
Ingolstadt (THI), Germany ter route information, select contacts in the phone book,
Johannes Kepler University, Linz
or search for pieces of music. They are optimized to re-
clemens.schartmueller@thi.de
quire low cognitive load, visual attention, and motor skills.
With the advent of automated driving, however, the driver-
passenger will require new ways and means to enter long
and more sophisticated texts, such as in typical office work.
To be able to design interfaces supporting this in a safe but
Andreas Riener also attractive fashion, we explore requirements for produc-
Technische Hochschule tive text input in highly automated vehicles and illustrate a
Ingolstadt (THI), Germany potential solution – the DAMOW assistant – with a fictional
Johannes Kepler University, Linz user story. In comparison with static office environments,
andreas.riener@thi.de we identify new issues to be tackled, as well as a need to
discuss several socio-technical concerns. Maybe even a
shift away from the classical desktop metaphor (i.e., WIMP
paradigm) as a whole is required, back to command-based
interfaces?
Author Keywords
Permission to make digital or hard copies of part or all of this work for personal or
classroom use is granted without fee provided that copies are not made or distributed
Automated Driving; Text-based Input; Office Work; Desktop;
for profit or commercial advantage and that copies bear this notice and the full citation Take-Overs; WIMP.
on the first page. Copyrights for third-party components of this work must be honored.
For all other uses, contact the owner/author(s).
Copyright held by the owner/author(s).
CCS Concepts
MobileHCI, 2018 Barcelona, Spain. •Human-centered computing → Ubiquitous and mobile
computing systems and tools; Text input.
18
Barcelona, Spain | September 3, 2018 MobileHCI 2018 Workshop on Socio-Technical Aspects of Text Entry
Introduction once with the “laptop on the lap” (Option A), and once with
Text-entry interfaces for drivers currently mainly target com- the “mobile office package” (Option B).
posing informal messages, defining navigational goals and
infotainment instructions – all required to occupy relatively Situation scenario: 7 a.m. in a suburb of Munich. After a
low amounts of cognitive, visual, and motor-ressources (cf. long night in which Professor Libelle finished her research
“The 15-second rule”, [3]). Highly automated driving (SAE paper just in time before the deadline, she wakes up real-
level 3, [6]), however, will allow drivers to engage more fre- izing that she forgot to prepare the HCI course’s exam for
quently in Non-Driving Related Tasks (NDRTs), effectively today. Fifteen minutes later she starts her car and sets the
opening up the possibility to execute more challenging text L3 Automated Driving System to drive her to the univer-
entry tasks, like in office work. Existing research shows sity “quickest possible” . However, soon after, her journey
that engaging in business-related tasks during driving is is quickly halted by a traffic jam due to construction sites
desired, especially by commuters [13]. However, they are ahead.
currently hardly supported by specialized interfaces besides
Option A: “Laptop on the Lap”
in first responder vehicles [10], although statistics show
Mrs. Libelle realizes that she cannot hold the exam with-
that commercial traffic is prevalent (60% of the new vehicle
out starting to prepare it now. She therefore grabs her lap-
registrations in Germany in 2016 are company cars, [8]).
top and starts writing. Frustrating fiddling around with the
We assume that people will engage in productive activities
touchpad makes her completely forget to keep an eye on
during highly automated driving, whether or not suitable
the road, while her hands start to cramp due to the uncom-
interfaces exist. A lack thereof presents a safety risk due
fortable typing posture. Suddenly, her car starts beeping
to extended off-road glances, etc., similar to illegal smart-
with a “Please Take-Over!” warning-sign blinking in the
phone usage in the car. This is underpinned by study re-
dashboard. Startled by the alarm, she tries to grab the
sults (n=1600) from insurance company Allianz, reporting
steering wheel, which is blocked by the notebook on her
that 46% of German smartphone owners admitted using
lap. She throws the laptop onto the passenger seat and
their devices manually (i.e., without specialized in-car inter-
rashly begins to steer without actually assessing the situa-
faces) during conventional driving (L1-2), with 24% reading
tion. The result is a barely avoided crash with construction
and 15% writing text messages [9], implying a substantial
workers and a broken laptop screen.
impact on traffic accidents. In highly automated driving (L3),
the driver-passenger will still need to take-over control of Option B: “Mobile Office Package”
the vehicle upon appropriate notification in case of emer- She realizes that she bought the ”mobile office” add-on for
gency and/or functional limitations. her car, just for cases like this one. “Hey car, open up a
new text document” she says and her car’s digital assistant
Prof. Libelle and her Highly Automated Journey overlays the windshield with a new document. She fills the
To illustrate issues but also capabilities of (the lack of) pro- document using the speech-to-text functionality, adds the
ductive textual interfaces for highly automated driving (L3), half-complete Fitt’s law formula using a reduced form-factor
we narrate the situation scenario of Mrs. L. Libelle, profes- keyboard integrated into the steering wheel and selects text
sor for Human-Computer Interaction in a hurry, in two ways: for formatting using a rotary knob co-located with the key-
19
Barcelona, Spain | September 3, 2018 MobileHCI 2018 Workshop on Socio-Technical Aspects of Text Entry
board, when after a while the digital assistant interrupts her: the Windows-Icons-Menus-Pointers (WIMP) paradigm
"Construction site ahead, please take-over driving control!". and takes “a (conceptual) step back” to command-based
Luckily, with the road being in her peripheral vision (due to interfaces. Our Digital Assistant for Mobile Office Work
the windshield display), it is easy for her to assess the situa- (DAMOW) includes a smart, context-aware, voice-interaction
tion and, as her hands were already on the steering wheel, based command interface for mode changes (initiating
also to quickly maneuver around the construction site. Min- workflows, changing text-editing modes, . . . ), utilizing the
utes later she arrives at the university – just in time for the current trend to “smart voice assistants” (cf. Google Home
exam and relieved that she still made it. and Amazon Alexa) and their safety benefits due to non-
existing visual attention requirements. Secondly, visual
Issues and Vision feedback with the currently written text, current interac-
The hypothetical example of Professor Libelle highlights the tion modus and reality augmentations, are given as semi-
most critical added requirement to text input interfaces in transparent overlay on the windshield. Windshield displays
driving: safety. In L3 automated driving driver-passengers were proven to mitigate take-over performance drawbacks
still need to occasionally perform a driving task. They need caused by NDRTs [15] (even though possibly causing text-
to respond to Take-Over Requests (TORs) in a safe man- legibility issues), and also to increase system trust [18].
ner, which makes it essential for (non-driving related) user Thirdly, a haptic interface (e.g., reduced form keyboard or
interfaces to support involved cognitive, visual and motor haptic touch display combined with a rotary knob) is used to
processes, and thereby counteracting the NDRT’s distract- provide an intuitive opportunity for high precision character-
ing nature. On the other hand, they further need to be at- to-character tasks, such as text formatting or entering for-
tractive in order to justify additional costs (cf. the “mobile mulas. Lastly, important notifications, such as TORs, are
office package” ), lower the entry barrier and actually get always given at least bimodally (visual and auditory) and
used. We hypothesize that staying productive while being will interrupt ongoing NDRTs to reduce stress and improve
mobile is a major attractiveness factor and will be even safety (cf. [14, 17]). However, while conceptualizing the
more impacting in the near future’s socio-economical con- idea, several potential problems emerged. Increasingly
text. Effectively integrating textual interfaces in not-yet fully complex systems could potentially increase perception time
automated vehicles adds several points of consideration or cognitive load. Further, socio-technical issues like pri-
differing from the typical static office workplace, such as: vacy arise which we want to discuss in the workshop.
Ergonomics [1] that ensure comfortable and efficient typing
but also take-over motor readiness [16], cognitive workload Workshop Discussion
calibration [19], motion sickness [2], and (visual) attention to Considering professor Libelle’s user story, several scenarios
the driving situation [11]. highlighting socio-technical issues emerged:
No more WIMP, no more Desktop
To account for the issues, we envision a truly multimodal • Acceptance: What if she had a passenger who is
interface, which we roughly illustrated in the user story’s getting annoyed by constant voice-commands, unre-
Option B. It opposes the typical desktop metaphor with lated to him/her or she herself generally doesn’t like
talking to a digital assistant?
20
Barcelona, Spain | September 3, 2018 MobileHCI 2018 Workshop on Socio-Technical Aspects of Text Entry
• Privacy: What if one of professor Libelle’s students analysis with the workshop’s experts’ discussion findings in
was also in the traffic jam, able to read parts of the order to build a solid foundation for future research.
upcoming exam on his/her teacher’s windshield dis-
play? Conclusion
• Learnability: Can we just invent new multimodal text To realize productive and safe text input interfaces for highly
input interfaces and hope that people will be willing to automated driving will be a challenging task considering
learn how to use them, or should we focus on existing the added requirements compared to conventional office
techniques? environments. We conceptualize the DAMOW, a smart and
context-aware multimodal Digital Assistant for Mobile Office
• Social intra-/inter-/extra-vehicle collaboration: Work and identify several socio-technical issues we hope to
What are essential collaboration scenarios that need discuss and explore in the workshop.
to be implemented in an office-oriented automotive
text input interface and how ? Authors’ Biographies
• User groups: How to make sure that non tech-savvy Dipl.-Ing. Clemens Schartmüller is a PhD student and re-
people accept and use the technology? search assistant in the Human Computer Interaction Group
• Changing paradigms: Do we need to reconsider the at Technische Hochschule Ingolstadt (THI, Germany). His
desktop metaphor? Is WIMP still suitable for a highly early scientific career consists of prototype-driven automo-
dynamic mobility-context? What are alternatives? tive user interface research with a special interest in explor-
What are the advantages / drawbacks of different ing opportunities and tackling challenges for using auto-
interaction paradigms? mated vehicles as office workplace, emphasizing objective
evaluation.
We realize that this (incomplete) list matches, at least partly, Prof. Dr. Andreas Riener is a professor for Human-Machine
well-established design principles and heuristics, such Interaction and Virtual Reality at THI and leading the human-
as defined by Grice [4], Nielsen [12] or the ISO standard computer interaction group. His research interests include
9241-110 [7], reconfirming the need to repeatedly discuss driving ergonomics, driver state estimation from physiolog-
them especially for novel interfaces. Although our research ical measures, human factors in driver-vehicle interfaces,
considering text input in (highly) automated vehicles is in and trust/acceptance/ethics in automated driving.
early stages, we believe to be able to contribute to a diverse
range of specialized application areas (and thus opinions) Acknowledgements
for text input, but also profit from the gathered experienced This work is supported under the "Innovative Hochschule"
researchers at the workshop. Besides that, the authors program of the German Federal Ministry of Education and
hope to be able to introduce themselves in the research Research (BMBF) under Grant No. 03IHS109A (MenschIN-
community and open up channels for future collaborations. Bewegung).
As a follow-up to the workshop, we plan to conduct a Con-
textual Inquiry study [5] with professionals who might bene-
fit from in-vehicle interfaces for office work, and combine its
21
Barcelona, Spain | September 3, 2018 MobileHCI 2018 Workshop on Socio-Technical Aspects of Text Entry
REFERENCES Autofahrern. Technical Report. Allianz AG.
1. Heiner Bubb, Klaus Bengler, Rainer E. Grünen, and https://www.allianzdeutschland.de/
Mark Vollrath. 2015. Automobilergonomie. Springer die-allianz-ablenkungsstudie-2016/id
International Publishing. 663–684 pages. DOI:
10. Andrew L Kun, Jerry Wachtel, W Thomas Miller, Patrick
http://dx.doi.org/10.1007/978-3-642-13505-7
Son, and Martin Lavallière. 2015. User Interfaces for
2. Cyriel Diels and Jelte E. Bos. 2016. Self-driving First Responder Vehicles: Views from Practitioners,
carsickness. Applied Ergonomics 53, October (mar Industry, and Academia. In Proceedings of the 7th
2016), 374–382. DOI: International Conference on Automotive User
http://dx.doi.org/10.1016/J.APERGO.2015.09.009 Interfaces and Interactive Vehicular Applications
(AutomotiveUI ’15). ACM, New York, NY, USA,
3. Paul Green. 1999. The 15-Second Rule for Driver
163–170. DOI:
Information Systems The 15-Second Rule for Driver
http://dx.doi.org/10.1145/2799250.2799289
Information Systems. Human Factors Cd (1999), 1–9.
11. Robert E. Llaneras, Jeremy Salinger, and Charles A.
4. H. Paul Grice. 1975. Logic and Conversation. Syntax
Green. 2013. Human factors issues associated with
and Semantics 3 (1975), 45–47.
limited ability autonomous driving systems: Drivers’
5. Karen Holtzblatt, Jessamyn Burns Wendell, and allocation of visual attention to the forward roadway.
Shelley Wood. 2004. Rapid contextual design: a how-to Proceedings of the Seventh International Driving
guide to key techniques for user-centered design. Symposium on Human Factors in Driver Assessment,
Elsevier. Training, and Vehicle Design (oct 2013), 92–98. DOI:
http:
6. International SAE and On-Road Automated
//dx.doi.org/10.17077/drivingassessment.1472
Driving (ORAD) Committee. 2014. Taxonomy and
Definitions for Terms Related to On-Road Motor 12. Jakob Nielsen and Rolf Molich. 1990. Heuristic
Vehicle Automated Driving Systems (J3016 Ground evaluation of user interfaces. In Proceedings of the
Vehicle Standard). SAE International 2014-01 (2014). SIGCHI conference on Human factors in computing
https://saemobilus.sae.org/content/J3016 systems Empowering people - CHI ’90. ACM Press,
New York, New York, USA, 249–256. DOI:
7. ISO. 2008. ISO 9241-110 Ergonomics of
http://dx.doi.org/10.1145/97243.97281
human-systems interaction - Part 110: Dialogue
principles. September (2008). 13. Nicole Perterer, Christiane Moser, Alexander
Meschtscherjakov, Alina Krischkowsky, and Manfred
8. Kraftfahrt Bundesamt. 2017. Jahresbilanz der
Tscheligi. 2016. Activities and Technology Usage while
Neuzulassungen 2016. (2017).
Driving. In Proceedings of the 9th Nordic Conference
9. Jörg Kubitzki and Wolfgang Fastenmeier. 2016. on Human-Computer Interaction - NordiCHI ’16. ACM
Ablenkung durch moderne Informations-und Press, New York, New York, USA, 1–10. DOI:
Kommunikations- techniken und soziale Interaktion bei http://dx.doi.org/10.1145/2971485.2971556
22
Barcelona, Spain | September 3, 2018 MobileHCI 2018 Workshop on Socio-Technical Aspects of Text Entry
14. Sebastiaan Petermeijer, Pavlo Bazilinskyy, Klaus 17. Philipp Wintersberger, Andreas Riener, Clemens
Bengler, and Joost De Winter. 2017. Take-over again: Schartmüller, Anna-Katharina Frison, and Klemens
Investigating multimodal and directional TORs to get Weigl. 2018. Let Me Finish before I Take Over: Context
the driver back into the loop. Applied Ergonomics 62, and Attention Aware Device Integration in Highly
July 2017 (2017), 204–215. DOI: Automated Vehicles. In Proceedings of the 10th
http://dx.doi.org/10.1016/j.apergo.2017.02.023 International Conference on Automotive User
Interfaces and Interactive Vehicular Applications
15. Clemens Schartmüller, Andreas Riener, Philipp
(Accepted). ACM, Toronto, Canada.
Wintersberger, and Anna-Katharina Frison. 2018.
Workaholistic : On Balancing Typing- and 18. Philipp Wintersberger, Tamara von Sawitzky, and
Handover-Performance in Automated Driving. In 20th Andreas Riener. 2017. Traffic Augmentation as Means
International Conference on Human-Computer to Increase Trust in Automated Driving Systems. In
Interaction with Mobile Devices and Services, Proceedings of CHItaly’17. ACM Press, Cagliari, 7.
MobileHCI 2018 (In Press). ACM, Barcelona, Spain. DOI:http://dx.doi.org/10.1145/3125571.3125600
16. Philipp Wintersberger, Paul Green, and Andreas 19. Mark S. Young and Neville A. Stanton. 2002. Attention
Riener. 2016. Am I Driving or Are You are Or We Both? and automation: New perspectives on mental
A Taxonomy For Handover and Handback in underload and performance. Theoretical Issues in
Automated Driving. Power (2016), 1–7. Ergonomics Science 3, 2 (2002), 178–194. DOI:
http://dx.doi.org/10.1080/14639220210123789
23