=Paper=
{{Paper
|id=Vol-2629/9_poster_neti.pdf
|storemode=property
|title=Exploring Different Feedback Styles of Performance in a Self-Assessment Application for Older Adult Drivers
|pdfUrl=https://ceur-ws.org/Vol-2629/9_poster_neti.pdf
|volume=Vol-2629
|authors=Surya S Neti,James Ren Hou Lee,Jennifer Boger
|dblpUrl=https://dblp.org/rec/conf/persuasive/NetiLB20
}}
==Exploring Different Feedback Styles of Performance in a Self-Assessment Application for Older Adult Drivers==
https://ceur-ws.org/Vol-2629/9_poster_neti.pdf
Exploring different feedback styles of performance in a
self-assessment application for older adult drivers1
Surya S Neti1, James Ren Hou Lee1 and Jennifer Boger2
1
Systems Design Engineering, University of Waterloo, Waterloo, ON, Canada
2
Research Institute of Aging, Waterloo, ON, Canada
Abstract. As we age our ability to perform day-to-day activities can change. One
such activity that is complex and cognitively challenging is driving. Research
has shown that older adult drivers have the high- est crashes per mile driven and
are more vulnerable to injuries due to frailty. Driving cessation however has been
known to cause social isola- tion and depression among older adults. This study
aims to better engage the older adult in the discussion and decision about when
to stop driv- ing. SmartDrive, a self-assessment application intended to promote
safe driving decisions by providing feedback and recommendations based on user
performance in driving-related cognitive tasks, has been evaluated in this study.
The researchers specifically explore the reactions elicited by different styles of
feedback presented to the user (text-only, score map and visuals) to identify the
most appropriate style that would persuade them to consider their driving ability
and the recommendation shown. Thematic analysis of interviews and cognitive
walkthroughs conducted with six actively driving older adults has been per-
formed and emergent themes are discussed in the context of feedback design.
Keywords: Olde adults, Driving, Self-assessment, Feedback.
1 Introduction
Cognitive abilities can change as we age and such changes can affect our performance
in day-to-day activities. Driving, often regarded as synonymous to autonomy, is one
such daily activity with significant cognitive demand. Studies have found that older
drivers have higher crashes per kilometre driven and are more vulnerable to injuries
due to frailty [1]. For an older adult, the decision to stop driving is a difficult one, with
research showing that driving cessation can cause social isolation, depression, and mor-
bidity [2][3][4], enforcing the idea that this decision requires considerable thought and
discussion with family and physicians [5][6]. This research aims to empower the older
adult in that discussion by providing them with information about their driving ability.
We propose SmartDrive, a digital application that implements clinically validated cog-
nitive tasks that have been correlated with on-road driving performance to help seniors
explore their driving health at home using a tablet, without assistance. Studies suggest
that different feedback presentation styles can cause varied perceptions amongst users
1
Supported by University of Waterloo
Persuasive 2020, Adjunct proceedings of the 15th International conference on Persuasive
Technology. Copyright © 2020 for this paper by its authors. Use permitted under Creative Com-
mons License Attribution 4.0 International (CC BY 4.0)
2
even with the same core information [10], therefore this study aimed to identify an
appropriate style with the intent to persuade older adult drivers to explore their driving
with the objective of promoting safe driving decisions to potentially reduce accident
occurrence in this demographic.
2 Design & Methods
For our pilot study, we employed a digitised version of Trail Making Test Part B
(TMTb), a cognitive task that has shown to have correlations with on-road driving per-
formance [7][8][9]. Feedback and recommendations based on user performance in this
task were designed. Table 1 shows the three performance labels used in the study, de-
vised using the completion times recorded in previous studies. Each label was presented
in three different summary styles (see Table. 2).
Table 1. Feedback presented to the user based on their task completion time.
TMTb time threshold Performance label Recommendation
< 57 seconds Average Repeat the task in two months
> 57 & < 255 seconds Below Average Visit physician for further assessment
> 255 seconds Deficient Visit physician for further assessment
Table 2. Feedback presented to the user based on their task completion time.
Summary Style Design Element (DE) 1 DE 2 DE 3
Text-only Text Labels Recommendation
Visual score map User score & Score map Labels Recommendation
Text & Images User & Average scores Driving images Recommendation
2.1 Study Design
Six senior drivers with a mean age of 73.83 years, all of whom possessed valid full G
driver’s licenses and are actively driving were recruited. After informed consent was
obtained, a demographics questionnaire was administered. This was followed by a cog-
nitive walk-through of SmartDrive to allow the participants to express their views on
the summary style presented after completion of the task. The interviews were audio
recorded and transcribed verbatim. Participants’ reactions to the styles presented and
their responses in the interviews were thematically analysed by two researchers to iden-
tify factors that would persuade use of the application.
3
3 Results
Theme-based segregation of emergent codes after implementing an inductive and de-
ductive approach are presented in Table 3 in the chronological order of user-reactions
that the styles elicited. Firstly, participants found the design flow coherent and gener-
ally agreed with the language and layout used. After performing the task, they were
quick to compare their scores with the thresholds provided and discussed different as-
pects of the design that would cause or prevent building trust in the application. Finally,
several recommendations that users could po- tentially follow to increase safety while
driving were suggested. All these themes were identified as persuading factors for con-
tinued use of SmartDrive.
Table 3. Four overarching themes and sub-themes influencing persuasiveness of Smart- Drive,
numbers in brackets indicate # of participants who mentioned the sub-theme; n total = 6.
1.Comperensibility 2. Performance Comparison
Information density (2) Performance relative to others (6)
Feedback tone (6) Performance relative to self (over time) (2)
Numeric/Word format (3)
Feedback style & colours (6)
1. Trustworthiness 2. Action Plan
Applicability to driving (5) Changes in driving habits (5)
Reliability (6) Self-Improvement (6)
Score explainability (3) Seek medical assessment (3)
Appropriate suggestions (5) Alternative Transportation (3)
4 Discussion
All six participants favoured the scoring map that presented the distance of their scores
from the two thresholds on a horizontal axis. This quantification of rela- tive perfor-
mance was cited as informative and influential in intent to use. High colour contrast
and low text density were also stated as reasons for its positive reception. Two partici-
pants preferred both text-only and visual score-map styles and sug- gested displaying
them sequentially to avoid clutter and promote layered mes- saging to prevent the user
from feeling overwhelmed. The driving images selected elicited varied responses, as
four participants found that this brought forward relevance to driving while two con-
sidered them judgemental and unnecessary. Different images will be explored in the
next iteration. While the style of feed- back had some impact, analysis revealed that the
4
feedback tone used, task appli- cability, and reliability of the scoring method were sig-
nificantly more important in developing sufficient trust in the application to prompt the
user in following the suggestions presented. All six participants were willing to use
SmartDrive again to track changes in driving-related cognitive status, and five of the
six participants were keen on pursuing a follow-up plan that ranged from minor changes
in their driving habits to an appointment with the doctor for further assessment.
5 Conclusions & Future Work
The first two themes identified are consistent with usability frameworks previ- ously
proposed for designing digital applications for older adults [11]. Need for ‘Trustwor-
thiness’ and an ‘Action plan’ through the sub-themes mentioned are new and emergent
concepts that have been recognised in this study as signif- icant factors affecting the
willingness of application-use. The research team is presently translating the sub-
themes into practical design features to incorpo- rate in the next version, which will be
further tested by a larger participant cohort. Follow-up interviews will be conducted to
explore any changes in driving behaviour and data will be analysed using the Out-
come/Change design matrix using the Behaviour Change Support System framework
[12] to determine the persuasiveness of the application.
References
1. Qin, Weidi, Xiaoling Xiang, and Harry Taylor. ”Driving Cessation and Social Iso- lation in
Older Adults.” Journal of aging and health (2019): 0898264319870400.
2. Choi, Namkee G., and Diana M. DiNitto. ”Depressive symptoms among older adults who
do not drive: association with mobility resources and perceived transportation barriers.” The
Gerontologist 56.3 (2016): 432-443.
3. Taylor, Brian D., and Sophia Tripodes. ”The effects of driving cessation on the elderly with
dementia and their caregivers.” Accident Analysis & Prevention 33.4 (2001): 519-528.
4. AZAD, NAHID. ”A survey of the impact of driving cessation on older drivers.” Geriatrics
Today 5 (2002): 170-174.
5. Kendall-Wilson T Depelteau A Whalen K Culp J. Hamdy RC, Kinser A. Driving and pa-
tients with dementia. Gerontology and geriatric medicine.., 2018
6. Eberhard, John. ”Older drivers’“high per-mile crash involvement”: The implications for li-
censing authorities.” Traffic injury prevention 9.4 (2008): 284-290
7. Haley Duncanson, Ann M Hollis, and Margaret G O’Connor. Errors versus speed on the
trail making test: Relevance to driving performance. Accident Analysis Pre- vention,
113:125–130, 2018.
8. Jane C Stutts, J Richard Stewart, and Carol Martell. Cognitive test performance and crash
risk in an older driver population. Accident Analysis Prevention, 30(3):337–346, 1998.
9. Papandonatos, George D., et al., ”Clinical utility of the Trail-Making test as a predictor of
driving performance in older adults.” Journal of the American Geriatrics Society 63.11
(2015): 2358-2364.
5
10. Choe, Eun Kyoung, et al. ”Persuasive performance feedback: The effect of fram- ing on self-
efficacy.” AMIA annual symposium proceedings. Vol. 2013. American Medical Informatics
Association, 2013.(10)
11. Becker SA, Webbe FM. Designing for older adult users of handheld technology. In2006
International Conference of the IEEE Engineering in Medicine and Biology Society 2006
Aug (pp. 3297-3300). IEEE.
12. Oinas-Kukkonen, Harri. ”Behavior change support systems: A research model and agenda.”
International Conference on persuasive technology. Springer, Berlin, Hei- delberg, 2010.