The growing and risky industry of nomadic apps for drivers Carlos Carvajal1, Andrés Rodríguez2 and Alejandro Fernández3 1,2 LIFIA Research Center, National University of La Plata, La Plata, Argentina 3 CICPBA, F.I., LIFIA Research Center, National University of La Plata, La Plata, Argentina carlos.carvajall@info.unlp.edu.ar, andres.rodriguez@lifia.info.unlp.edu.ar, alejandro.fernandez@lifia.info.unlp.edu.ar Abstract. HCI researchers have worked for decades defining methods and techniques to assess the attention demands of in-vehicle information systems (IVIS). Acceptance test methods have been proposed that must be passed for the safe use of IVIS. Most of these methods require expensive test environ- ments and highly trained personnel for its implementation. This article makes a review of those strategies with focus in the cost and development process phase. In the realm of mobile application ecosystems (aka "apps"), guidelines and cer- tification programs exist. Apps must pass them to be considered as automotive- ready systems or to integrate with OEM infotainment devices. However, getting into the category of certified applications does not guarantee full compliance with the criteria established by formal methods accepted by the automotive in- dustry and international standards. Moreover, many studies show the high risk of using IVIS while driving, which lead to consider that the current predomi- nant approaches to assess attention demands of automotive apps and to guide IVIS design are not enough. Efficient cost-benefit methods applicable in early phases of application development, as well as context-adaptive interfaces have the potential to contribute to the improvement of safe driving environments. Keywords: IVIS, driver attention, empirical methods, visual demand, cognitive demand, analytical techniques. 1 Introduction A Vehicle Information System is a software application that processes vehicle data and/or other data from different sources to finally provide valuable and action- relevant information to the vehicle driver and/or to other stakeholders [1]. When the Information System is used inside a car it is called: “In-Vehicle Information System”, or IVIS. It can run as a mobile application installed in smartphones and other “mobile or nomadic devices”. Information systems in the vehicle can be either introduced in portable devices or run as OEM systems that are permanently installed and are part of the original vehicle. The latter are designed by companies that understand driving and have a group of professionals who permanently conduct studies of their applications, Copyright c 2020 for this paper by its authors. Use permitted under Creative Commons License Attribution 4.0 International (CC BY 4.0). 1 2 seeking their continuous improvement, keeping in mind to maintain safe driving con- ditions. Driving distraction is understood as any activity that distracts the focus of the pri- mary activity that in this case is driving, including talking or writing on the phone, talking with people in the car, manipulating the controls of devices such as stereo or navigation system. In short, a driving distraction is any activity that moves attention away from safe driving practices [2]. Many studies, both in the United States of America and in Europe report that accidents due to driver distractions have reached annual costs of around 40 billion dollars and 5 thousand deaths [3]. There’s consider- able contribution of driver distraction caused using cell phones to traffic accidents. Nowadays, the accident rate is inversely proportional to the age of the involved driv- ers. However, it is worth asking whether decades in the future, when the current age group of adolescents grow older, will this fact still hold? This article analyzes relevant strategies that support the development of vehicular information systems. Section 2 compares related work with this article review. Section 3 shows formal evaluation strategies for assessing driver distraction in the automotive context. Section 4 de- scribes the work done by the software industry to safeguard the security of its imple- mentations in the automotive environment. Section 5 highlights relevant discussion topics for this research. Finally, conclusions section aims to summarize this study and to describe our potential future work. 2 Comparison to related work Many authors warn about the risks introduced by software applications embedded in the vehicle. This opinion is not fully shared by Heinrich who conducted two review articles about automotive telematic applications between 2013 and 2015. He stated that “Despite of the concerns in the past there is no increase of accidents due to the use of integrated devices” [4]. In the introductory section of this article, it is high- lighted among several facts, that using cell phone is an important cause for traffic accidents, with a high proportion in young population. Heinrich suggested that automotive applications can run on a smartphone, but use the OEM installed screen, and by this way apply all the industry guidelines [4]. He agrees with the MirrorLink standard strategy (reviewed in detail later in the “Automo- tive Mobile Apps” section) in which nomadic devices can only work paired with the certified infotainment system of the car where the screen conversions are carried out to comply with the guidelines and standards. Android Auto, since its 2019 update, no longer allows applications to run directly from the smartphone; it forces applications to pair with the large display of the car. However, avoiding the use of applications directly from mobile devices is almost impossible. The development of secure mobile applications for the automotive context constitutes an important research challenge for Human Computer Interaction. “Speech recognition technologies may reduce the crash risk” [5]. Strayer et al. presented, between 2013 and 2015, three researches[6] developing a cognitive distrac- tion scale for tasks in the automotive cockpit. Starting from the single activity of 2 3 operating a motor vehicle with a base quantification of 1.0, then listening radio: 1.21, conversing with passenger: 2.33, using a hands-free cell phone: 2.27, interacting with a speech to text system: 3.06 and finally doing mental operations with the top rating of 5.0. They demonstrated that interacting with voice-based systems in the vehicle may have consequences that negatively affect traffic safety [6]. Just listening to voice messages (not considering a response) has a cognitive workload rating like conversing on a cell phone. Next, they tested a personal assistant system, Apple Siri, which re- quired interaction with the driver and got a higher value of 4.0. in the “Strayer work- load rating” [7]. Therefore, Strayer and Heinrich conclusions differ greatly related with distraction impact of speech technologies. Heinrich pointed out that more restrictive and complex standards for OEM devices could incentivize the use of nomadic devices without controls, which potentially af- fect the overall safety” [5]. Strayer et al. [8] observed that, when nomadic devices are used in conjunction with the built-in infotainment systems (and the large screens they offer), lower workload levels are obtained. However, Ramnath et al. [9] highlight the potential dangers of these dominant ecosystems. On the one hand, they conclude that touch interaction is so dangerous that it leads to not complying with the NHTSA guideline on eye behavior, while voice command interaction does comply. They also find dangerous increases in driver reaction times through voice interaction and worse negative results through touch. 3 Evaluation Strategies Strategies for the evaluation of IVIS can be classify into four families. Visual demand strategies apply empirical methods to assess the impact of the use of an application on the visual attention of the driver. Cognitive demand strategies assess the impact of the use of an application in the cognitive load of the driver. Analytical strategies use pre- dictive models to assess the potential for distraction without the need for experimental tests or functional prototypes. Finally, subjective methods rely on the user’s opinion. In the following sections we discuss common aspects of each of these categories and present representative strategies. We pay special attention to the element of cost, and applicability of each strategy in the IVIS life cycle. A table summarizes relevant pub- lications in each category and provides cost indicators (the research is understood as costly when for its complexity, it could be carried out with the support of the automo- tive industry or government entities), the existence of a financial sponsor, and the stage in the IVIS development process. 3.1 Visual Demand, Empirical Techniques Visual Demand research is carried out using techniques that evaluate driver gaze be- havior to asses driver workload to perform a given task. Two methods are the most used in this domain: “glance time” and “occlusion test”. Glance time testing involves measuring eye glance away from the road in two dimensions for a specific IVIS task: total glance time and mean glances time, with an eye tracker device. Occlusion Test- 3 4 ing require a see-through device (such as lenses or googles with crystal liquid shut- ters) and is used to restrict the time that the driver can see the tool under test. Goggles are configured with the vision and non-vision times, and this is used to quantify the time required to complete an objective. In terms of eye trackers costs, there are pro- fessional solutions at 10,000 US Dollars [10]. Eye gaze measurements Occlusion techniques are supported by an ISO international standard, specifically ISO 16673: 2007. Formally, Occlusion techniques require specialized goggles to achieve the shut- ter and open effect, that can have a significant cost above 4.000 US Dollars [11]. There are also studies that have simulated the effect of occlusion glasses by obscuring the application’s interface with theoretically similar effects [12]. In terms of the life cycle of the IVIS application, empirical visual demand assessments are typically per- formed in last stages of products development. This is logical, because a useful and realistic application visual performance test is more effective as you get closer to the product’s final version. Table 1 mentions relevant studies related to visual demand assessments. Table 2 presents a summary of main tests used in visual demand assess- ments, their metrics and whether they are defined in international standards or guide- lines. 3.2 Cognitive and Mixed Demand, Empirical Techniques The analysis of the cognitive demand provides information to designers and software developers to gauge the usability of the portable application, for example when pre- senting information in different ways and selecting alternatives less cognitively de- manding. Detection Response Task (DRT) is one of the most popular methods to evaluate the cognitive load of a task. The method is based on the thesis that suggests that increased cognitive load of a task would reduce the driver’s attention to other visual, tactile or auditory information. While performing the task under test, drivers are presented with a sensory stimulus every 3–5 seconds and are asked to respond to it by pressing a button attached to their finger. More demanding tasks result in the driv- er more frequently missing and not answering the presented DRT stimuli. “Response times and hit rates are interpreted as indicators of the attentional effect of cognitive load.” [13]. DRT is mainly used in final development stages. Table 3 shows some articles that report DRT using. Table 4 describes the main metrics to evaluate and the international standard that support the method. 3.3 Analytical Techniques Analytical methods are based on predictive models that can assess the potential for distraction without the need for experimental tests or functional prototypes. Early stages of development can benefit by having approximate measurements of the per- formance of a task and thus optimize its iterative development. This kind of tech- niques aim to model the human behavior in the automotive context and requires an important knowledge level in order to create, learn and understand the model, hence is a challenge for Human Computer Interaction researchers [14]. Table 5 summarizes articles that report the use of analytical methods. A summary of tests commonly used 4 5 in analytical techniques, their main metrics and if they are defined in an international standard or guideline is found in table 6. 3.4 Subjective Methods Based on the ISO 9241-11 standard, the usability of a system refers to its ability to be used in each context of use to achieve goals of effectiveness, efficiency and satisfac- tion. Regarding the satisfaction condition, subjective user assessments are required by asking them to rate his experience with the IVIS interaction. In table 7 is listed repre- sentative researches of the main subjective evaluation methods. The Nasa TLX (Task Load Index) is a widely assessment tool for perceived workload, used in wide variety of research domains. An optimized version for the automotive context is known as: Driving Activity Load Index – DALI. Both SUS (System Usability Scale) and DALI methods are questionnaire type evaluations, they are simple tests to implement and allow a quantification of user perceptions. 4 Automotive Mobile Apps Android Auto manages an ecosystem of “auto ready” mobile apps that have approved a certification program. The Android Auto interface is optimized for the automotive context and is manageable by touch or voice commands. In relation to Android Auto design guidelines, it is striking that there is no specific mention of any of the formal evaluation methods, nor to any international standards or driver distraction interna- tional guidelines. However, many of the defined principles can be considered in- spired by good practices defined in the automotive industry. Android Auto since July 2019 introduces an important User Interface optimization and a paradigm change, because it begins to get rid of the smartphone UI and moves towards the exclusive use of in-car displays [15]. On the other hand, Apple Car Play try to provide a safe envi- ronment in the automotive context with iPhone ecosystem. Apple defines Human Interface Guidelines to develop adequate apps for the driving environment. Apple Car Play validates its guidelines compliance to adopt third party compatible apps to its ecosystem. Nevertheless, the Apple CarPlay development API is closed and not everyone has the possibility to build apps. It is necessary to get an Apple Mfi Manu- facturing License that is normally available by companies with their own industrial facilities [16]. Nowadays Apple Car Play has only a few third-party compatible apps. MirrorLink is a car-based technology that claims to be designed to allow a car driver to safely access information, entertainment and communication features from a mo- bile device while driving [17]. Like Android Auto and Apple CarPlay, MirrorLink considers development guidelines for its compatible apps, based on the general prin- ciples of industry control entities. MirrorLink development tools, examples and tuto- rials are only available for Android operating system. MirrorLink requires IVIS appli- cations to certify at his Authorized Test Lab [18]. To the date, checking the Mir- rorLink website, one can see that there’s no updates for many years. In several online 5 6 forums, it is argued that the platform has lost its relevance due to Apple and Android implementations that have taken their place [19]. 5 Discussions There are situations where paying the right attention and being well focused can be the difference between life and death. The automotive context is a very particular scenario for the study of human computer interfaces, since there is a clear primary activity set in the real and physical world, which is to drive safely. Any additional interaction while driving constitutes a potentially dangerous competition for driver’s attention. In line with this need, the automotive industry and government control agencies have sponsored various investigations that have produced a series of regional regulations and guidelines to align secondary activities related to the use of IVIS in the driver’s cab. Wiese et al. [20] have categorized these efforts into two groups: In- terference Mitigation and Workload Management. IVIS safety standards are related mainly with Interference Mitigation, with strategies that minimize the number and duration of IVIS glances required. Typical design considerations for conventional mobile applications include maximizing user attention, but this is not consistent with the automotive context. The software industry, in its concern about this peculiarity of IVIS, has prepared a series of reference guides for software developers and has pre- pared qualification plans for third-party applications prior to presenting them on its car product portals. However, these validation criteria reflect a lack of rigor far from the formality of the complex standards demanded by the automotive industry and regional control agencies. MirrorLink makes a considerable effort to try to align itself with the rigorous regional automotive controls, so much so that it even demands vali- dation of the applications in its certified laboratories. Perhaps that is precisely one of the motivators for their loss of relevance as ecosystem for IVIS against the duopoly of its competitors. Ramnath et al. [9] studied the reaction time of a driver under various scenarios and found that this response was surprisingly better under the influence of alcohol or cannabis consumption, than during the interaction with an IVIS, either in the various implementations of Android Auto or Apple Car Play. This research was conducted in a simulated environment and among their main conclusions find that an undistracted driver typically reacts in 1 second to stimuli, and these times increase percentage-wise in the following order (starting with the best results and ending with the worst): with alcohol use, cannabis use, hands-on phone free, Android Auto by voice, Apple CarPlay by voice, manual use of the phone, Android Auto touch, Apple CarPlay touch [9]. In short, they demonstrate that using an IVIS while driving can be more dangerous than doing it under the influence of alcohol or cannabis which sup- ports the hypothesis that the current approaches that guide IVIS design are not enough for reach adequate levels of secure drive. There is a clear condition to be solved, which is to adapt the guidelines for the development of mobile applications that must be optimized in their condition and treated as secondary activities in the context of safe driving. On the other hand, effective cost-benefit strategies are required since most formal IVIS distraction assessment methodologies are demanding in terms of 6 7 equipment and specialists to process the results, as previously highlighted in Evalua- tion Strategies section. This implies that they are commonly outside the scope and budget of typical software development and maintenance projects. The footprint of attention of the activities required by an IVIS must be managed with a holistic ap- proach that does not depend solely on the magnitude of attention that the application demands (commonly evaluated in simulated environments). Evaluation should also acknowledge the variable attention demands of the road conditions, and of other tools/devices/situations in the cockpit. Several approaches can be proposed at this point, such as the idea of a collaborative "attention grounding" by Wiese et al. [20] or the "attention account" for pervasive attentive user interfaces by Bulling [21] (draw- ing an analogy with bank account). 6 Conclusions Formal methodologies for IVIS attention assessment are usually complex and expen- sive to implement, focused on scientific research. That is why much of the research reviewed in this article had financial support from car manufacturers or government entities. Lamm et al., in their analysis of the research literature on evaluation of In- Vehicle Information Systems, find that methods applied at early stages of develop- ment such as those based on predictive models of behavior are not popular in Auto- motive HCI research [22], which can be verified with a simple search in Google Scholar and realize low number of references for articles related with predictive methods in the IVIS context [23]. Nowadays Android and iOS have defined acceptance environments for IVIS de- veloped by third parties. Like any good practice, it coincides in its spirit with much of what is defined by different international traffic regulatory agencies as well as world standards like ISO and SAE. However, passing these certifications does not guarantee compliance with the strictest acceptance levels developed by the scientific community and used by the automotive industry for decades. We believe that there is a lack of affordable formal methods applicable mainly in early stages of In-Vehicle systems development, that could benefit software developers without financial sup- port from large corporate research projects, but who want to (and should) adhere to formal methods for attention management in automotive context. The methods known as “mixed” imply the combination of quantitative and qualita- tive evaluations [24]. These have had important acceptance in other science disci- plines, but in the IVIS development niche are still considered infrequent in their study. There’s a research opportunity when considering this approach for the study of methods and tools that support the software developers work. The focus for this future work will be related with techniques attached to scientific rigor and economic feasi- bility in its implementation. Mixed methods like DRT variants (quantitative), predic- tive techniques or software tools (economic) and usability evaluations (qualitative), promise to be material for a framework that define what we might know as the In- Vehicle Information System attention footprint. 7 8 Another field to explore, is the IVIS-driver-roadway dynamic, which potentially offers better answers to real world environments where it is not enough to consider the resources demanded by each situation but also when and where drivers should adapt their attention. Supporting adaptive and user focused interfaces for secondary tasks in the demand for user attention, has a significant development potential in the automotive context and thus contribute with the improvement of safe driving envi- ronments. Table 1. Visual Demand Assessments Title Author How Costly Sponsor Stage Assessing In-Vehicle Sec- Ljung Occlusion Glasses Yes Volvo Final ondary Tasks with the NHTSA Guidelines [25] Using occlusion to measure Domeyer Occlusion Glasses, Yes Toyota Final the effects of the NHTSA J. 44 participants in a participant criteria on driv- real vehicle. er distraction testing [26] Table 2. Visual Demand Assessments in International Guidelines and Standards Method Main Metrics International Guideline International Standard 15-seconds A task could be acceptable None SAE J2364 Rule [27] while driving if it can be completed in 15 seconds. Occlusion Total Task Time, AAM, JAMA, NHTSA ISO 16673 Total Shutter Open Time Eye Gaze Total Glance Time AAM, EsoP, JAMA, ISO 15007 Measurement Single Glance Duration NHTSA R Ratio, TSOT/TTT Lane Change Mean Deviation, MDEV None ISO 26022 Task Table 3. Cognitive and Mixed Demand Assessments Title Author How Costly Sponsor Stage Detection-Response Task— Stojme- Visual, Tactile and No Slovenian Beta, Uses and Limitations. [13] nova Auditory DRT Research Final Assessing the visual and Strayer 120 participants Yes Traffic Final cognitive demands of IVIS comparing DRT Safety [28] results. Foundation Table 4. Cognitive and Mixed Demand Assessments in International Standards Method Main Metrics International Guideline International Standard Detection Mean response time. Type None ISO 17488 Response of DRT variants: visual, Task tactile and auditory Table 5. Analytical Techniques Title Author How Costly Sponsor Stage An extended keystroke Pettitt Extended KLM Yes UK De- Early level model (KLM) for method for model partment 8 9 predicting the visual de- human behavior. for mand of IVIS [29] Transport Evaluating distraction of Purucker KLM extended Yes Hyundai Early in-vehicle information model to predict: systems while driving by Total eyes-off-road predicting total eyes-off- times (TEORT). road times with KLM [30] Table 6. Analytical Techniques in International Standards Method Main Metrics International Guideline International Standard KLM or QN-MHP meth- Total Completion None SAE J2365 ods variants, ACT-R Time of IVIS Tasks, cognitive architecture TEORT, eye glance theories. behavior Table 7. Subjective Methods Title Author How Costly Sponsor Stage SUS – A quick and dirty Brooke Subjective evalua- No Digital Last usability scale (not specified J. tion method Equipment Stages for automotive domain) Co. Evaluating driver mental Pauzié Subjective Evalua- No French Last workload using (DALI) [31] A. tion of drivers’ Institute Stages mental workload References 1. C. Kaiser, A. Stocker, A. Festl, G. Lechner, and M. Fellmann, “A research agenda for vehicle information systems,” 26th Eur. Conf. Inf. Syst., 2018. 2. NHTSA, “Distracted Driving.” [Online]. Available: https://www.nhtsa.gov/risky- driving/distracted-driving. [Accessed: 21-Apr-2019]. 3. F. Tango and M. Botta, “Real-time detection system of driver distraction using machine learning,” IEEE Trans. ITS, vol. 14, no. 2, pp. 894–905, 2013. 4. C. Heinrich, “Fighting Driver Distraction - Worldwide Approaches,” no. 13, 2013. 5. C. Heinrich, “FIGHTING DRIVER DISTRACTION – RECENT DEVELOPMENTS 2013 - 2015,” no. 15, pp. 1–8, 2015. 6. D. L. Strayer et al., “Measuring Cognitive Distraction in the Automobile,” AAAFoundation.org, no. June, pp. 202–638, 2013. 7. D. L. Strayer, J. Turrill, J. R. Coleman, E. V Ortiz, and J. M. Cooper, “Measuring Cognitive Distraction in the Automobile II: Assessing In-Vehicle Voice-Based Interactive Technologies,” AAAFoundation.org, no. October, pp. 202–638, 2014. 8. D. L. Strayer et al., “Visual and Cognitive Demands of Using Apple’s CarPlay, Google’s Android Auto and Five Different OEM Infotainment Systems,” 2018. 9. R. Ramnath, N. Kinnear, S. Chowdhury, and T. Hyatt, “Interacting with Android Auto and Apple CarPlay when driving : The effect on driver performance A simulator study,” IAM RoadSmart, p. 55, 2020. 10. “Eye Tracking Research In The Field.” [Online]. Available: https://www.tomshardware.com/news/tobii-pro-glasses-2-eye-tracking,33575.html. 9 10 [Accessed: 04-Nov-2019]. 11. “Visual Occlusion Goggles.” [Online]. Available: https://redscientific.com/visual- occlusion-goggles.html. [Accessed: 04-Nov-2019]. 12. M. Krause, N. Donant, and K. Bengler, “Comparing Occlusion Method by Display Blanking to Occlusion Goggles,” Procedia Manuf., 2015. 13. K. Stojmenova and J. Sodnik, “Detection-Response Task Uses and Limitations,” 2018. 14. C. Harvey and N. A. Stanton, Usability Evaluation for In-Vehicle Systems. 2013. 15. T. Kerns, “Google is getting rid of Android Auto’s smartphone UI — here’s why,” 2019. [Online]. Available: https://www.androidpolice.com/2019/07/30/android-auto-app-going- away-assistant-driving-mode/. [Accessed: 17-Nov-2019]. 16. “Discover Apple CarPlay Apps List from Third-Party Developers.” [Online]. Available: https://www.cleveroad.com/blog/discover-apple-carplay-apps-list-from-third-party- developers. [Accessed: 03-Mar-2020]. 17. C. Rosamond, “What is MirrorLink?,” Auto Express Website, 2017. [Online]. Available: https://www.autoexpress.co.uk/car-news/99194/what-is-mirrorlink-guide-to-the-car- smartphone-hook-up-system. [Accessed: 03-Mar-2020]. 18. “MirrorLink certification for Smartphones.” [Online]. Available: https://www.7layers.com/certification/mirrorlink-certification. 19. “Can we give Mirrorlink some attention?” [Online]. Available: https://www.reddit.com/r/windowsphone/comments/5bvxm6/can_we_give_mirrorlink_so me_attention_we_need_it/. [Accessed: 03-Mar-2020]. 20. E. E. Wiese and J. D. Lee, “Attention grounding: A new approach to in-vehicle information system implementation,” Theor. Issues Ergon. Sci., 2007. 21. A. Bulling, “Pervasive Attentive User Interfaces,” Computer (Long. Beach. Calif)., vol. 49, no. 1, pp. 94–98, 2016. 22. L. Lamm and C. Wolff, “Exploratory Analysis of the Research Literature on Evaluation of In-Vehicle Systems,” pp. 60–69, 2019. 23. C. Harvey and N. A. Stanton, “Trade-off between context and objectivity in an analytic approach to the evaluation of in-vehicle interfaces,” 2012. 24. G. Burnett, “How do you assess the distraction of in-vehicle information systems ? A comparison of occlusion, lane change task and medium- fidelity driving simulator methods,” 3rd Int. Conf. Driv. Distraction Ina., 2013. 25. R. B. Mikael Ljung, “Assessing In-Vehicle Secondary Tasks with the NHTSA Visual- Manual Guidelines Occlusion Method Mikael,” Adv. Hum. Asp. Transp., 2018. 26. J. E. Domeyer, “Using occlusion to measure the effects of the NHTSA participant criteria on driver distraction testing,” Hum. Factors Ergon. Soc., 2014. 27. P. Green, “The 15-second rule for driver information systems,” Proc ITS Am., 1999. 28. D. L. Strayer and J. M. Cooper, “Assessing the visual and cognitive demands of in-vehicle information systems,” Cogn. Res., 2019. 29. M. Pettitt, “An extended keystroke level model (KLM) for predicting the visual demand of in-vehicle information systems,” Hum. Factors Comput. Syst., 2007. 30. C. Purucker, “Evaluating distraction of in-vehicle information systems while driving by predicting total eyes-off-road times with keystroke level modeling,” 2017. 31. A. Pauzie, “Evaluating driver mental workload using the driving activity load index(DALI),” Eur. Conf. Hum. centred Des. Intell. Transp. Syst., pp. 67–77, 2008. 10