=Paper=
{{Paper
|id=Vol-2747/paper19
|storemode=property
|title=The growing and risky industry of nomadic apps for drivers
|pdfUrl=https://ceur-ws.org/Vol-2747/paper19.pdf
|volume=Vol-2747
|authors=Carlos Carvajal,Andrés Rodríguez,Alejandro Fernández
}}
==The growing and risky industry of nomadic apps for drivers==
https://ceur-ws.org/Vol-2747/paper19.pdf
The growing and risky industry of nomadic apps for
drivers
Carlos Carvajal1, Andrés Rodríguez2 and Alejandro Fernández3
1,2 LIFIA Research Center, National University of La Plata, La Plata, Argentina
3
CICPBA, F.I., LIFIA Research Center, National University of La Plata, La Plata, Argentina
carlos.carvajall@info.unlp.edu.ar,
andres.rodriguez@lifia.info.unlp.edu.ar,
alejandro.fernandez@lifia.info.unlp.edu.ar
Abstract. HCI researchers have worked for decades defining methods and
techniques to assess the attention demands of in-vehicle information systems
(IVIS). Acceptance test methods have been proposed that must be passed for
the safe use of IVIS. Most of these methods require expensive test environ-
ments and highly trained personnel for its implementation. This article makes a
review of those strategies with focus in the cost and development process phase.
In the realm of mobile application ecosystems (aka "apps"), guidelines and cer-
tification programs exist. Apps must pass them to be considered as automotive-
ready systems or to integrate with OEM infotainment devices. However, getting
into the category of certified applications does not guarantee full compliance
with the criteria established by formal methods accepted by the automotive in-
dustry and international standards. Moreover, many studies show the high risk
of using IVIS while driving, which lead to consider that the current predomi-
nant approaches to assess attention demands of automotive apps and to guide
IVIS design are not enough. Efficient cost-benefit methods applicable in early
phases of application development, as well as context-adaptive interfaces have
the potential to contribute to the improvement of safe driving environments.
Keywords: IVIS, driver attention, empirical methods, visual demand, cognitive
demand, analytical techniques.
1 Introduction
A Vehicle Information System is a software application that processes vehicle data
and/or other data from different sources to finally provide valuable and action-
relevant information to the vehicle driver and/or to other stakeholders [1]. When the
Information System is used inside a car it is called: “In-Vehicle Information System”,
or IVIS. It can run as a mobile application installed in smartphones and other “mobile
or nomadic devices”. Information systems in the vehicle can be either introduced in
portable devices or run as OEM systems that are permanently installed and are part of
the original vehicle. The latter are designed by companies that understand driving and
have a group of professionals who permanently conduct studies of their applications,
Copyright c 2020 for this paper by its authors. Use permitted under Creative Commons
License Attribution 4.0 International (CC BY 4.0).
1
2
seeking their continuous improvement, keeping in mind to maintain safe driving con-
ditions.
Driving distraction is understood as any activity that distracts the focus of the pri-
mary activity that in this case is driving, including talking or writing on the phone,
talking with people in the car, manipulating the controls of devices such as stereo or
navigation system. In short, a driving distraction is any activity that moves attention
away from safe driving practices [2]. Many studies, both in the United States of
America and in Europe report that accidents due to driver distractions have reached
annual costs of around 40 billion dollars and 5 thousand deaths [3]. There’s consider-
able contribution of driver distraction caused using cell phones to traffic accidents.
Nowadays, the accident rate is inversely proportional to the age of the involved driv-
ers. However, it is worth asking whether decades in the future, when the current age
group of adolescents grow older, will this fact still hold? This article analyzes relevant
strategies that support the development of vehicular information systems. Section 2
compares related work with this article review. Section 3 shows formal evaluation
strategies for assessing driver distraction in the automotive context. Section 4 de-
scribes the work done by the software industry to safeguard the security of its imple-
mentations in the automotive environment. Section 5 highlights relevant discussion
topics for this research. Finally, conclusions section aims to summarize this study and
to describe our potential future work.
2 Comparison to related work
Many authors warn about the risks introduced by software applications embedded in
the vehicle. This opinion is not fully shared by Heinrich who conducted two review
articles about automotive telematic applications between 2013 and 2015. He stated
that “Despite of the concerns in the past there is no increase of accidents due to the
use of integrated devices” [4]. In the introductory section of this article, it is high-
lighted among several facts, that using cell phone is an important cause for traffic
accidents, with a high proportion in young population.
Heinrich suggested that automotive applications can run on a smartphone, but use
the OEM installed screen, and by this way apply all the industry guidelines [4]. He
agrees with the MirrorLink standard strategy (reviewed in detail later in the “Automo-
tive Mobile Apps” section) in which nomadic devices can only work paired with the
certified infotainment system of the car where the screen conversions are carried out
to comply with the guidelines and standards. Android Auto, since its 2019 update, no
longer allows applications to run directly from the smartphone; it forces applications
to pair with the large display of the car. However, avoiding the use of applications
directly from mobile devices is almost impossible. The development of secure mobile
applications for the automotive context constitutes an important research challenge
for Human Computer Interaction.
“Speech recognition technologies may reduce the crash risk” [5]. Strayer et al.
presented, between 2013 and 2015, three researches[6] developing a cognitive distrac-
tion scale for tasks in the automotive cockpit. Starting from the single activity of
2
3
operating a motor vehicle with a base quantification of 1.0, then listening radio: 1.21,
conversing with passenger: 2.33, using a hands-free cell phone: 2.27, interacting with
a speech to text system: 3.06 and finally doing mental operations with the top rating
of 5.0. They demonstrated that interacting with voice-based systems in the vehicle
may have consequences that negatively affect traffic safety [6]. Just listening to voice
messages (not considering a response) has a cognitive workload rating like conversing
on a cell phone. Next, they tested a personal assistant system, Apple Siri, which re-
quired interaction with the driver and got a higher value of 4.0. in the “Strayer work-
load rating” [7]. Therefore, Strayer and Heinrich conclusions differ greatly related
with distraction impact of speech technologies.
Heinrich pointed out that more restrictive and complex standards for OEM devices
could incentivize the use of nomadic devices without controls, which potentially af-
fect the overall safety” [5]. Strayer et al. [8] observed that, when nomadic devices are
used in conjunction with the built-in infotainment systems (and the large screens they
offer), lower workload levels are obtained. However, Ramnath et al. [9] highlight the
potential dangers of these dominant ecosystems. On the one hand, they conclude that
touch interaction is so dangerous that it leads to not complying with the NHTSA
guideline on eye behavior, while voice command interaction does comply. They also
find dangerous increases in driver reaction times through voice interaction and worse
negative results through touch.
3 Evaluation Strategies
Strategies for the evaluation of IVIS can be classify into four families. Visual demand
strategies apply empirical methods to assess the impact of the use of an application on
the visual attention of the driver. Cognitive demand strategies assess the impact of the
use of an application in the cognitive load of the driver. Analytical strategies use pre-
dictive models to assess the potential for distraction without the need for experimental
tests or functional prototypes. Finally, subjective methods rely on the user’s opinion.
In the following sections we discuss common aspects of each of these categories and
present representative strategies. We pay special attention to the element of cost, and
applicability of each strategy in the IVIS life cycle. A table summarizes relevant pub-
lications in each category and provides cost indicators (the research is understood as
costly when for its complexity, it could be carried out with the support of the automo-
tive industry or government entities), the existence of a financial sponsor, and the
stage in the IVIS development process.
3.1 Visual Demand, Empirical Techniques
Visual Demand research is carried out using techniques that evaluate driver gaze be-
havior to asses driver workload to perform a given task. Two methods are the most
used in this domain: “glance time” and “occlusion test”. Glance time testing involves
measuring eye glance away from the road in two dimensions for a specific IVIS task:
total glance time and mean glances time, with an eye tracker device. Occlusion Test-
3
4
ing require a see-through device (such as lenses or googles with crystal liquid shut-
ters) and is used to restrict the time that the driver can see the tool under test. Goggles
are configured with the vision and non-vision times, and this is used to quantify the
time required to complete an objective. In terms of eye trackers costs, there are pro-
fessional solutions at 10,000 US Dollars [10]. Eye gaze measurements Occlusion
techniques are supported by an ISO international standard, specifically ISO 16673:
2007. Formally, Occlusion techniques require specialized goggles to achieve the shut-
ter and open effect, that can have a significant cost above 4.000 US Dollars [11].
There are also studies that have simulated the effect of occlusion glasses by obscuring
the application’s interface with theoretically similar effects [12]. In terms of the life
cycle of the IVIS application, empirical visual demand assessments are typically per-
formed in last stages of products development. This is logical, because a useful and
realistic application visual performance test is more effective as you get closer to the
product’s final version. Table 1 mentions relevant studies related to visual demand
assessments. Table 2 presents a summary of main tests used in visual demand assess-
ments, their metrics and whether they are defined in international standards or guide-
lines.
3.2 Cognitive and Mixed Demand, Empirical Techniques
The analysis of the cognitive demand provides information to designers and software
developers to gauge the usability of the portable application, for example when pre-
senting information in different ways and selecting alternatives less cognitively de-
manding. Detection Response Task (DRT) is one of the most popular methods to
evaluate the cognitive load of a task. The method is based on the thesis that suggests
that increased cognitive load of a task would reduce the driver’s attention to other
visual, tactile or auditory information. While performing the task under test, drivers
are presented with a sensory stimulus every 3–5 seconds and are asked to respond to it
by pressing a button attached to their finger. More demanding tasks result in the driv-
er more frequently missing and not answering the presented DRT stimuli. “Response
times and hit rates are interpreted as indicators of the attentional effect of cognitive
load.” [13]. DRT is mainly used in final development stages. Table 3 shows some
articles that report DRT using. Table 4 describes the main metrics to evaluate and the
international standard that support the method.
3.3 Analytical Techniques
Analytical methods are based on predictive models that can assess the potential for
distraction without the need for experimental tests or functional prototypes. Early
stages of development can benefit by having approximate measurements of the per-
formance of a task and thus optimize its iterative development. This kind of tech-
niques aim to model the human behavior in the automotive context and requires an
important knowledge level in order to create, learn and understand the model, hence is
a challenge for Human Computer Interaction researchers [14]. Table 5 summarizes
articles that report the use of analytical methods. A summary of tests commonly used
4
5
in analytical techniques, their main metrics and if they are defined in an international
standard or guideline is found in table 6.
3.4 Subjective Methods
Based on the ISO 9241-11 standard, the usability of a system refers to its ability to be
used in each context of use to achieve goals of effectiveness, efficiency and satisfac-
tion. Regarding the satisfaction condition, subjective user assessments are required by
asking them to rate his experience with the IVIS interaction. In table 7 is listed repre-
sentative researches of the main subjective evaluation methods. The Nasa TLX (Task
Load Index) is a widely assessment tool for perceived workload, used in wide variety
of research domains. An optimized version for the automotive context is known as:
Driving Activity Load Index – DALI. Both SUS (System Usability Scale) and DALI
methods are questionnaire type evaluations, they are simple tests to implement and
allow a quantification of user perceptions.
4 Automotive Mobile Apps
Android Auto manages an ecosystem of “auto ready” mobile apps that have approved
a certification program. The Android Auto interface is optimized for the automotive
context and is manageable by touch or voice commands. In relation to Android Auto
design guidelines, it is striking that there is no specific mention of any of the formal
evaluation methods, nor to any international standards or driver distraction interna-
tional guidelines. However, many of the defined principles can be considered in-
spired by good practices defined in the automotive industry. Android Auto since July
2019 introduces an important User Interface optimization and a paradigm change,
because it begins to get rid of the smartphone UI and moves towards the exclusive use
of in-car displays [15]. On the other hand, Apple Car Play try to provide a safe envi-
ronment in the automotive context with iPhone ecosystem. Apple defines Human
Interface Guidelines to develop adequate apps for the driving environment. Apple
Car Play validates its guidelines compliance to adopt third party compatible apps to
its ecosystem. Nevertheless, the Apple CarPlay development API is closed and not
everyone has the possibility to build apps. It is necessary to get an Apple Mfi Manu-
facturing License that is normally available by companies with their own industrial
facilities [16]. Nowadays Apple Car Play has only a few third-party compatible apps.
MirrorLink is a car-based technology that claims to be designed to allow a car driver
to safely access information, entertainment and communication features from a mo-
bile device while driving [17]. Like Android Auto and Apple CarPlay, MirrorLink
considers development guidelines for its compatible apps, based on the general prin-
ciples of industry control entities. MirrorLink development tools, examples and tuto-
rials are only available for Android operating system. MirrorLink requires IVIS appli-
cations to certify at his Authorized Test Lab [18]. To the date, checking the Mir-
rorLink website, one can see that there’s no updates for many years. In several online
5
6
forums, it is argued that the platform has lost its relevance due to Apple and Android
implementations that have taken their place [19].
5 Discussions
There are situations where paying the right attention and being well focused can be
the difference between life and death. The automotive context is a very particular
scenario for the study of human computer interfaces, since there is a clear primary
activity set in the real and physical world, which is to drive safely. Any additional
interaction while driving constitutes a potentially dangerous competition for driver’s
attention. In line with this need, the automotive industry and government control
agencies have sponsored various investigations that have produced a series of regional
regulations and guidelines to align secondary activities related to the use of IVIS in
the driver’s cab. Wiese et al. [20] have categorized these efforts into two groups: In-
terference Mitigation and Workload Management. IVIS safety standards are related
mainly with Interference Mitigation, with strategies that minimize the number and
duration of IVIS glances required. Typical design considerations for conventional
mobile applications include maximizing user attention, but this is not consistent with
the automotive context. The software industry, in its concern about this peculiarity of
IVIS, has prepared a series of reference guides for software developers and has pre-
pared qualification plans for third-party applications prior to presenting them on its
car product portals. However, these validation criteria reflect a lack of rigor far from
the formality of the complex standards demanded by the automotive industry and
regional control agencies. MirrorLink makes a considerable effort to try to align itself
with the rigorous regional automotive controls, so much so that it even demands vali-
dation of the applications in its certified laboratories. Perhaps that is precisely one of
the motivators for their loss of relevance as ecosystem for IVIS against the duopoly of
its competitors. Ramnath et al. [9] studied the reaction time of a driver under various
scenarios and found that this response was surprisingly better under the influence of
alcohol or cannabis consumption, than during the interaction with an IVIS, either in
the various implementations of Android Auto or Apple Car Play. This research was
conducted in a simulated environment and among their main conclusions find that an
undistracted driver typically reacts in 1 second to stimuli, and these times increase
percentage-wise in the following order (starting with the best results and ending with
the worst): with alcohol use, cannabis use, hands-on phone free, Android Auto by
voice, Apple CarPlay by voice, manual use of the phone, Android Auto touch, Apple
CarPlay touch [9]. In short, they demonstrate that using an IVIS while driving can be
more dangerous than doing it under the influence of alcohol or cannabis which sup-
ports the hypothesis that the current approaches that guide IVIS design are not enough
for reach adequate levels of secure drive. There is a clear condition to be solved,
which is to adapt the guidelines for the development of mobile applications that must
be optimized in their condition and treated as secondary activities in the context of
safe driving. On the other hand, effective cost-benefit strategies are required since
most formal IVIS distraction assessment methodologies are demanding in terms of
6
7
equipment and specialists to process the results, as previously highlighted in Evalua-
tion Strategies section. This implies that they are commonly outside the scope and
budget of typical software development and maintenance projects. The footprint of
attention of the activities required by an IVIS must be managed with a holistic ap-
proach that does not depend solely on the magnitude of attention that the application
demands (commonly evaluated in simulated environments). Evaluation should also
acknowledge the variable attention demands of the road conditions, and of other
tools/devices/situations in the cockpit. Several approaches can be proposed at this
point, such as the idea of a collaborative "attention grounding" by Wiese et al. [20] or
the "attention account" for pervasive attentive user interfaces by Bulling [21] (draw-
ing an analogy with bank account).
6 Conclusions
Formal methodologies for IVIS attention assessment are usually complex and expen-
sive to implement, focused on scientific research. That is why much of the research
reviewed in this article had financial support from car manufacturers or government
entities. Lamm et al., in their analysis of the research literature on evaluation of In-
Vehicle Information Systems, find that methods applied at early stages of develop-
ment such as those based on predictive models of behavior are not popular in Auto-
motive HCI research [22], which can be verified with a simple search in Google
Scholar and realize low number of references for articles related with predictive
methods in the IVIS context [23].
Nowadays Android and iOS have defined acceptance environments for IVIS de-
veloped by third parties. Like any good practice, it coincides in its spirit with much
of what is defined by different international traffic regulatory agencies as well as
world standards like ISO and SAE. However, passing these certifications does not
guarantee compliance with the strictest acceptance levels developed by the scientific
community and used by the automotive industry for decades. We believe that there is
a lack of affordable formal methods applicable mainly in early stages of In-Vehicle
systems development, that could benefit software developers without financial sup-
port from large corporate research projects, but who want to (and should) adhere to
formal methods for attention management in automotive context.
The methods known as “mixed” imply the combination of quantitative and qualita-
tive evaluations [24]. These have had important acceptance in other science disci-
plines, but in the IVIS development niche are still considered infrequent in their
study. There’s a research opportunity when considering this approach for the study of
methods and tools that support the software developers work. The focus for this future
work will be related with techniques attached to scientific rigor and economic feasi-
bility in its implementation. Mixed methods like DRT variants (quantitative), predic-
tive techniques or software tools (economic) and usability evaluations (qualitative),
promise to be material for a framework that define what we might know as the In-
Vehicle Information System attention footprint.
7
8
Another field to explore, is the IVIS-driver-roadway dynamic, which potentially
offers better answers to real world environments where it is not enough to consider
the resources demanded by each situation but also when and where drivers should
adapt their attention. Supporting adaptive and user focused interfaces for secondary
tasks in the demand for user attention, has a significant development potential in the
automotive context and thus contribute with the improvement of safe driving envi-
ronments.
Table 1. Visual Demand Assessments
Title Author How Costly Sponsor Stage
Assessing In-Vehicle Sec- Ljung Occlusion Glasses Yes Volvo Final
ondary Tasks with the
NHTSA Guidelines [25]
Using occlusion to measure Domeyer Occlusion Glasses, Yes Toyota Final
the effects of the NHTSA J. 44 participants in a
participant criteria on driv- real vehicle.
er distraction testing [26]
Table 2. Visual Demand Assessments in International Guidelines and Standards
Method Main Metrics International Guideline International Standard
15-seconds A task could be acceptable None SAE J2364
Rule [27] while driving if it can be
completed in 15 seconds.
Occlusion Total Task Time, AAM, JAMA, NHTSA ISO 16673
Total Shutter Open Time
Eye Gaze Total Glance Time AAM, EsoP, JAMA, ISO 15007
Measurement Single Glance Duration NHTSA
R Ratio, TSOT/TTT
Lane Change Mean Deviation, MDEV None ISO 26022
Task
Table 3. Cognitive and Mixed Demand Assessments
Title Author How Costly Sponsor Stage
Detection-Response Task— Stojme- Visual, Tactile and No Slovenian Beta,
Uses and Limitations. [13] nova Auditory DRT Research Final
Assessing the visual and Strayer 120 participants Yes Traffic Final
cognitive demands of IVIS comparing DRT Safety
[28] results. Foundation
Table 4. Cognitive and Mixed Demand Assessments in International Standards
Method Main Metrics International Guideline International Standard
Detection Mean response time. Type None ISO 17488
Response of DRT variants: visual,
Task tactile and auditory
Table 5. Analytical Techniques
Title Author How Costly Sponsor Stage
An extended keystroke Pettitt Extended KLM Yes UK De- Early
level model (KLM) for method for model partment
8
9
predicting the visual de- human behavior. for
mand of IVIS [29] Transport
Evaluating distraction of Purucker KLM extended Yes Hyundai Early
in-vehicle information model to predict:
systems while driving by Total eyes-off-road
predicting total eyes-off- times (TEORT).
road times with KLM [30]
Table 6. Analytical Techniques in International Standards
Method Main Metrics International Guideline International Standard
KLM or QN-MHP meth- Total Completion None SAE J2365
ods variants, ACT-R Time of IVIS Tasks,
cognitive architecture TEORT, eye glance
theories. behavior
Table 7. Subjective Methods
Title Author How Costly Sponsor Stage
SUS – A quick and dirty Brooke Subjective evalua- No Digital Last
usability scale (not specified J. tion method Equipment Stages
for automotive domain) Co.
Evaluating driver mental Pauzié Subjective Evalua- No French Last
workload using (DALI) [31] A. tion of drivers’ Institute Stages
mental workload
References
1. C. Kaiser, A. Stocker, A. Festl, G. Lechner, and M. Fellmann, “A research agenda for
vehicle information systems,” 26th Eur. Conf. Inf. Syst., 2018.
2. NHTSA, “Distracted Driving.” [Online]. Available: https://www.nhtsa.gov/risky-
driving/distracted-driving. [Accessed: 21-Apr-2019].
3. F. Tango and M. Botta, “Real-time detection system of driver distraction using machine
learning,” IEEE Trans. ITS, vol. 14, no. 2, pp. 894–905, 2013.
4. C. Heinrich, “Fighting Driver Distraction - Worldwide Approaches,” no. 13, 2013.
5. C. Heinrich, “FIGHTING DRIVER DISTRACTION – RECENT DEVELOPMENTS
2013 - 2015,” no. 15, pp. 1–8, 2015.
6. D. L. Strayer et al., “Measuring Cognitive Distraction in the Automobile,”
AAAFoundation.org, no. June, pp. 202–638, 2013.
7. D. L. Strayer, J. Turrill, J. R. Coleman, E. V Ortiz, and J. M. Cooper, “Measuring
Cognitive Distraction in the Automobile II: Assessing In-Vehicle Voice-Based Interactive
Technologies,” AAAFoundation.org, no. October, pp. 202–638, 2014.
8. D. L. Strayer et al., “Visual and Cognitive Demands of Using Apple’s CarPlay, Google’s
Android Auto and Five Different OEM Infotainment Systems,” 2018.
9. R. Ramnath, N. Kinnear, S. Chowdhury, and T. Hyatt, “Interacting with Android Auto and
Apple CarPlay when driving : The effect on driver performance A simulator study,” IAM
RoadSmart, p. 55, 2020.
10. “Eye Tracking Research In The Field.” [Online]. Available:
https://www.tomshardware.com/news/tobii-pro-glasses-2-eye-tracking,33575.html.
9
10
[Accessed: 04-Nov-2019].
11. “Visual Occlusion Goggles.” [Online]. Available: https://redscientific.com/visual-
occlusion-goggles.html. [Accessed: 04-Nov-2019].
12. M. Krause, N. Donant, and K. Bengler, “Comparing Occlusion Method by Display
Blanking to Occlusion Goggles,” Procedia Manuf., 2015.
13. K. Stojmenova and J. Sodnik, “Detection-Response Task Uses and Limitations,” 2018.
14. C. Harvey and N. A. Stanton, Usability Evaluation for In-Vehicle Systems. 2013.
15. T. Kerns, “Google is getting rid of Android Auto’s smartphone UI — here’s why,” 2019.
[Online]. Available: https://www.androidpolice.com/2019/07/30/android-auto-app-going-
away-assistant-driving-mode/. [Accessed: 17-Nov-2019].
16. “Discover Apple CarPlay Apps List from Third-Party Developers.” [Online]. Available:
https://www.cleveroad.com/blog/discover-apple-carplay-apps-list-from-third-party-
developers. [Accessed: 03-Mar-2020].
17. C. Rosamond, “What is MirrorLink?,” Auto Express Website, 2017. [Online]. Available:
https://www.autoexpress.co.uk/car-news/99194/what-is-mirrorlink-guide-to-the-car-
smartphone-hook-up-system. [Accessed: 03-Mar-2020].
18. “MirrorLink certification for Smartphones.” [Online]. Available:
https://www.7layers.com/certification/mirrorlink-certification.
19. “Can we give Mirrorlink some attention?” [Online]. Available:
https://www.reddit.com/r/windowsphone/comments/5bvxm6/can_we_give_mirrorlink_so
me_attention_we_need_it/. [Accessed: 03-Mar-2020].
20. E. E. Wiese and J. D. Lee, “Attention grounding: A new approach to in-vehicle
information system implementation,” Theor. Issues Ergon. Sci., 2007.
21. A. Bulling, “Pervasive Attentive User Interfaces,” Computer (Long. Beach. Calif)., vol.
49, no. 1, pp. 94–98, 2016.
22. L. Lamm and C. Wolff, “Exploratory Analysis of the Research Literature on Evaluation of
In-Vehicle Systems,” pp. 60–69, 2019.
23. C. Harvey and N. A. Stanton, “Trade-off between context and objectivity in an analytic
approach to the evaluation of in-vehicle interfaces,” 2012.
24. G. Burnett, “How do you assess the distraction of in-vehicle information systems ? A
comparison of occlusion, lane change task and medium- fidelity driving simulator
methods,” 3rd Int. Conf. Driv. Distraction Ina., 2013.
25. R. B. Mikael Ljung, “Assessing In-Vehicle Secondary Tasks with the NHTSA Visual-
Manual Guidelines Occlusion Method Mikael,” Adv. Hum. Asp. Transp., 2018.
26. J. E. Domeyer, “Using occlusion to measure the effects of the NHTSA participant criteria
on driver distraction testing,” Hum. Factors Ergon. Soc., 2014.
27. P. Green, “The 15-second rule for driver information systems,” Proc ITS Am., 1999.
28. D. L. Strayer and J. M. Cooper, “Assessing the visual and cognitive demands of in-vehicle
information systems,” Cogn. Res., 2019.
29. M. Pettitt, “An extended keystroke level model (KLM) for predicting the visual demand of
in-vehicle information systems,” Hum. Factors Comput. Syst., 2007.
30. C. Purucker, “Evaluating distraction of in-vehicle information systems while driving by
predicting total eyes-off-road times with keystroke level modeling,” 2017.
31. A. Pauzie, “Evaluating driver mental workload using the driving activity load
index(DALI),” Eur. Conf. Hum. centred Des. Intell. Transp. Syst., pp. 67–77, 2008.
10