How Persuasive is a Virtual Coach in Terms of Promoting Web-based Intervention User Engagement?

Abstract

A selection of web-based eHealth interventions have evolved from innovative prototypes to evidence-based and clinically applied solutions for mental diseases such as depression and anxiety. Open-access, self-guided types of those solutions hold the promise of reaching and treating a large population against reasonable costs. However, a considerable factor that currently hinders the effectiveness of these self-guided web-based interventions is the high level of attrition. The absence of a human caregiver apparently has a negative effect on user adherence. Although clear evidence on the rationale for non-adherence is currently missing, recent studies indicate that user support could an important causal factor. In this paper we describe the protocol of an exploratory user study which aims to identify the persuasive features of virtual coaches in terms of user support. Virtual coaches are anthropomorphized instances of software, designed to engage and motivate users. We will measure user experience, engagement, motivation and perceived persuasiveness during usage of the 'Living to the Full' intervention. We will compare a condition with and without empathic virtual coach condition to explore its supportive capabilities.

Keywords

• eHealth
• web-based interventions
• embodied conversational agents
• virtual coaches
• virtual humans
• affective computing
• behavior change support system
• BCSS
• Intelligent Tutoring System
• ITS

-

port or a therapeutic service results in significantly higher effect sizes [ 9 ]. In addition, human-supported interventions achieve higher rates of adherence, that is more participants use the intervention as intended, e.g. by completing all the lessons of an intervention. [ 1,8,15 ]. Non-adherence is an important issue in web-based interventions for mental health [ 21 ] and becomes an even bigger problem when evidence-based therapies are deployed as free to access self-guided web-based therapeutic interventions [ 20 ]. In these interventions, adherence, defined as the percentage of users who complete all lessons, falls to a level as low as 1% [ 21 ] or even 0.5% [ 22 ].

The higher rates of adherence in human-supported interventions can be explained in favor of therapists, who prove to do an effective job in motivating clients during their change process. Interestingly enough, Talbot [ 12 ] describes in her meta-study that a key converging finding is that the involvement of a professional support provider, a therapist, is not necessary. What is key is a minimal level of non-guiding human contact. Irrespective of whether this type of contact is provided by a layperson or a professional, it has equally large positive effects on intervention adherence. Moreover, scheduling support can already have an effect of itself on treatment effectiveness. A telephone contact scheduled at the start of the treatment to take place as soon as a self-help book has been read, yields surprisingly large completion rates and treatment outcomes [ 13 ].

This poses the question what this support is that is needed to achieve higher rates of adherence and effectiveness. A study of Cavanagh and Millings [ 34 ] provides evidence of built-in ‘common factors’ such an generating hope, empathy and warmth, collaboration and feedback, that increase the effectiveness of interventions. However, there is no common definition of the support or ‘common factors’ that should be included in each intervention to be effective.

Furthermore, this opens up the opportunity to look into different ways of providing this support. There have been studies into the role technology can play in increasing adherence. Studies have shown positive effects of e.g. reminders, tailored advice and updates of the interventions website on adherence [ 21,35,36 ]. The study of Kelders et al. [ 8 ] has shown that technology in the form of dialogue support (supporting the interaction between the user and the system) has an influence on adherence. In this way, technology can be seen as persuasive: it tries to persuade the user to keep using the intervention and to change their behavior according to what is learned within the intervention.

Computers can be equipped with capabilities that can potentially enhance their persuasiveness as is shown in the field of affective computing and by using embodied conversational agents.

Affective Computing as a term was coined in the seminal article of Picard [ 26 ] who defined it as “computing that relates to, arises from, or influences emotion”. Picard [ 26, 27 ] stated that computers should be equipped with at least the basic functionality to notice and respond to emotions expressed by their users, in order to cater for a more natural form of human-computer interaction.

Cassell [ 38 ] defined embodied conversational agents as human-like appearances that have the same properties as humans in face-to-face conversation (see Figure 1 for an example), including: • • • •

The ability to recognize and respond to verbal and non-verbal input.

The ability to generate verbal and non-verbal output.

The ability to deal with conversational functions such as turn taking, feedback, and repair mechanisms.

The ability to give signals that indicate the state of the conversation, as well as to contribute new propositions to the discourse.

Both developments create capabilities to enrich the dialog between computer and user. This enriched dialog encompasses both verbal and non-verbal information and aims to acknowledge the role of human emotion. From a coaching perspective, the computer side of the dialog steers user emotion into productive directions. Deploying an affective embodied conversational agent as an adjunctive virtual coach to self-guided web-based interventions can possibly contribute to the need for support as expressed by users. However, this poses questions such as: • • •

When should support be delivered by the virtual coach? Can and should recognition of non-productive user status trigger a supportive action from the virtual coach? If so, what non-productive user states can the computer reliably distinguish? What empirical evidence consists for effective recognition and repair of nonproductive user states?

Research of non-productive user states so far has focused on frustration and boredom. Concerning frustration, Klein et al. [ 39 ] conducted an experiment during which the participant was deliberately frustrated during game play, after which the user was affectively supported by a text-based agent. Their results showed that the users chose to interact significantly longer when the system provided textual affective support compared to the condition during which no affective support was given. Based on these findings Klein et al. suggested that ‘computers are capable of alleviating strong, negative emotions, even when they are the source of these emotions’. However, frustration may not need remediation as argued by Mentis [ 40 ]. According to Mentis’, frustration among users of information systems is only of concern if it is associated with events that are outside of the user’s locus of control, such as a program bug. A frustrating event of this nature interrupts the user’s cognitive flow and ambition to fulfill his computerized tasks and is therefore detrimental to user adherence..

A second non-productive user state concerns boredom. Based on user state measurements during online learning Baker et al. [ 41 ] conclude that boredom is both a frequent and detrimental state with regards to learning. They propose that next to frustration detection, boredom detection should trigger computerized motivational interventions. D’Mello et al [ 42 ] point out that bored students have a high risk of becoming frustrated. They describe predictors of poor learning and increased drop-out rates such as pupils going off-task, zoning out and intentionally misusing (i.e. gaming) the learning system. Therefore they state that Intelligent Tutoring Systems (ITS) should include affective loop capabilities. D’Mello et al. [ 42 ] developed an ITS on biology that aimed to promote engagement and learning by dynamically detecting and responding to student’s boredom and disengagement. The system made use of an eye tracker to monitor the student’s gaze patterns and identifies when the user is no longer looking at the screen. If the longitude of the gaze away period exceeds a threshold, an animated pedagogical agent displays a motivational intervention, such as “please pay attention” in order to reengage the user. The gaze-reactive tutor resulted in significantly more accurate responses to the sub-category of deep-reasoning questions. However, overall, there were no significant learning gains. It appeared that there were significant positive overall learning effects for high proficient learners and negative overall learning effects for less gifted students. This effect shows that computer support delivery to users remains a challenging task with potentially counter-productive effects. 2

Our Research Proposal

To study the opportunities for a virtual coach within a web-based intervention for mental health, we will study user experience, engagement, motivation and perceived persuasiveness during usage of an intervention with a virtual coach as an adjunct to the existing web based intervention ‘Living to the Full’ [ 43 ]. We will use a betweensubjects design with two experimental conditions (empathic virtual coach and neutral virtual coach) and a control condition (the existing intervention without a virtual coach).

For the creation of the virtual coach, we will make use of the Virtual Human environment, as kindly offered by the Institute for Creative Technologies, University of Southern California [ 23 ]. As we make use of separate platforms, the users in the experimental conditions will have two screens, one for ‘Living to the Full’ and on for the virtual coach. At this stage, we therefore envisage a researcher to start up the Virtual Coach utterances at the right time, i.e. a wizard of Oz solution will be chosen. As interaction moments between user and virtual coach, we envisage: • An introduction moment during which the virtual coach presents itself to the user and indicates that he is there to support the user and is capable of answering user questions, to some extent • A supportive moment at the middle of the ‘Living to the Full’ intervention during which the virtual coach expresses its interest in the user (empathic coach condition) or gives the user the opportunity to ask questions (neutral coach condition) • A supportive and evaluative moment at the end of the ‘Living to the Full’ intervention during which both the neutral and empathic virtual coach ask about the user’s experiences, with the empathic virtual coach doing it in a empathic manner

After the session, participants will fill-out a questionnaire on user experience, engagement, motivation and perceived persuasiveness. This will be supplemented with qualitative interview questions. Based on the lessons learned we will prepare a second study where the intervention will be used in a more real-life setting, e.g. using more sessions, which enables us to also focus on the effectiveness of adding virtual coachsupport. 3

Future Research

We foresee further Virtual Coach research projects during which we will measure user boredom and/or frustration and a virtual coach that acts upon these measurements.

References

1. Cuijpers , P. , Donker , T. , Johansson , R. , Mohr , D. C., van Straten , A. , Andersson , G. : Selfguided psychological treatment for depressive symptoms: A meta-analysis . PLoS ONE 6 , e21274 .( 2011 )

2. Androutsopoulos , J.: From variation to heteroglossia in the study of computer-mediated discourse In: Thurlow C ., Mroczek , K. R. (eds.) Digital Discourse: Language in the New Media . Oxford University Press, Oxford p. 281 ( 2011 )

3. Alley , S. , Jennings , C. , Persaud , N. , Plotnikoff , R.C. , Horsley , M. , Vandelanotte , C. : Do Personally Tailored Videos in a Web-Based Physical Activity Intervention Lead to Higher Attention and Recall? - An Eye-Tracking Study. Emerging Technologies to Promote and Evaluate Physical Activity ( 2014 ).

4. Tanenbaum , J.G. , Antle , A.N. , Robinson , J.: Three Perspectives on Behavior Change for Serious Games. A better paradigm could be: learning via the web: e-learning as provided by ITS Intelligent Tutoring Systems (ITS) . In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems pp. 3389 - 3392 ( 2013 )

5. Fogg , B. J. Persuasive Technology: Using Computers to Change What We Think and Do . Morgan Kaufmann, San Francisco ( 2002 )

6. Bickmore , T. , Gruber , A. , Picard , R.: Establishing the computer-patient working alliance in automated health behavior change interventions . Patient education and counseling , 59 ( 1 ), 21 - 30 ( 2005 )

7. Oinas-Kukkonen , H. , Harjumaa , M. : Persuasive Systems Design: Key Issues, Process Model, and System Features Communications of the Association for Information Systems 24, Article 28 ( 2009 )

8. Kelders , S.M. , Kok , R.N. , Ossebaard , H.C. , Van Gemert-Pijnen, J.E.W.C. : Persuasive System Design Does Matter: A Systematic Review of Adherence to Web-Based Interventions . Med Internet Res14(6) , e152 ( 2012 )

9. Andersson , G. , Carlbring , P. , Berger , T. , Almlo¨v, J., Cuijpers , P. : What makes internet therapy work? Cognitive Behaviour Therapy , 38 , 55 - 60 ( 2009 )

10. Eysenbach G. : The Law of Attrition . J Med Internet Res . 7 ( 1 ): e11 ( 2005 )

11. Helgadóttir , F.D. , Menzies , R.G. , Onslow , M. , Packman A. , O'Brian , S. : Online CBT I: Bridging the Gap Between Eliza and Modern Online CBT Treatment Packages . Behaviour Change 26 ( 04 ), 245 - 253 ( 2009 )

12. Talbot , F. : Client Contact in Self-Help Therapy for Anxiety and Depression: Necessary But Can Take a Variety of Forms Beside Therapist Contact . Behaviour Change 29 ( 02 ), 63 - 76 ( 2012 )

13. Newman , M.G. , Szkodny L.E. , Llera , S.J. , Przeworski , A. : A review of technologyassisted self-help and minimal contact therapies for anxiety and depression: is human contact necessary for therapeutic efficacy? Clinical psychology review 31(1 ), 89 - 103 ( 2011 )

14. Andrews , G. , Cuijpers , P. , Craske , M.G. , McEnvoy , P. , Ttitov , N.: Computer therapy for the anxiety and depressive disorders is effective, acceptable and practical health care: A meta-analysis . PLoS ONE , 5 , e13196 ( 2010 )

15. Richards , D. , Richardson , T. : Computer-based psychological interventions for depression treatment: A Systematic review and meta-analysis . Clinical Psychology Review 32 , 329 - 342 ( 2012 )

16. Farvolden , P. , Denisoff , E. , Selby , P. , Bagby , R.M. , Rudy , L. : Usage and longitudinal effectiveness of a Web-based self-help cognitive behavioral therapy program for panic disorder . J Med Internet Res . 7 ( 1 ), e7 ( 2005 )

17. Christensen , H. , Griffiths , K.M, Korten , A.E , Brittliffe , K. , Groves , C. : A comparison of changes in anxiety and depression symptoms of spontaneous users and trial participants of a cognitive behavior therapy website . J Med Internet Res . 6 ( 4 ), e46 ( 2004 )

18. Christensen , H. , Griffiths , K.M. , Jorm , A.F. : Delivering interventions for depression by using the internet: a randomized controlled trial . BMJ 328 ( 7434 ), 265 ( 2004 )

19. Barak , A. , Klein , B. , Proudfoot , J.G. : Defining Internet-Supported Therapeutic Interventions . Ann. Behav. Med . 38 , 4 - 17 ( 2009 )

20. Cugelman , B. , Thelwall , M. , Dawes , P. : Online Interventions for Social Marketing Health Behavior Change Campaigns: A Meta-Analysis of Psychological Architectures and Adherence Factors . J Med Internet Res 13 ( 1 ), e17 ( 2011 )

21. Schubart , J. R. , Stuckey , H. L. , Ganeshamoorthy , A. , Sciamanna , C. N. Chronic health conditions and internet behavioral interventions: A review of factors to enhance user engagement . Computers Informatics Nursing 29 ( 2 ), 81 - 92 ( 2011 )

22. Andersson , G. , Carlbring , P. , Bergera , T. , Almlöva , J. , Cuijpers , P. : What Makes Internet Therapy Work? Cognitive Behaviour Therapy 38 ( 1 ), 55 - 60 ( 2009 )

23. Rizzo , A. , Lange , B. , Buckwalter , J.G. , Forbell , E. , Kim , J. , Sagae , K. , Williams , J. , Difede , J. , Rothbaum , B.O. , Reger , G. , Parsons , T. , Kenny , P.: SimCoach: an intelligent virtual human system for providing healthcare information and support International Journal on Disability and Human Development 10 ( 4 ) ( 2011 )

24. Kennedy , C.M. , Powell , J. , Payne , T.H. , Ainsworth J .: Active Assistance Technology for Health-Related Behavior Change: An interdisciplinary Review . J Med Internet Res 14 ( 3 ), e80 ( 2012 )

25. Norman , G.J. , Zabinski , M.G. , Adam , M.A. , Rosenberg , D.E. , Yaroch , A.L. , Atienza , A.A.:. A review of eHealth interventions for physical activity and dietary behavior change . AM J Prev Med 33 ( 4 ), 336 - 345 ( 2007 )

26. Picard , R.W.: Affective Computing. M.I . T. Media Laboratory Perceptual Computing Section Technical Report No. 321 ( 1997 )

27. Picard , R.W.: Affective computing: challenges . International Journal of Human-Computer Studies 59 ( 1-2 ), 55 - 64 ( 2003 )

28. Damassio , A. : Descartes' Error: Emotion, Reason, and the Human Brain . ( 1994 )

29. Murray , E.: Web-Based Interventions for Behavior Change and Self-Management: Potential, Pitfalls, and Progress. Med 2.0 1 ( 2 ), e3 ( 2012 )

30. Kasap , Z. , Moussa , M.B. Chaudhuri , P. , Magnenat-Thalmann , N. : ( 2009 ). Making Them Remember Emotional Virtual Characters with Memory . Computer Graphics and Applications 29 ( 2 ), 20 - 29 ( 2009 )

31. Amini , R. , Lisetti , C. , Yasavur , U. , Rishe , N.: On-Demand Virtual Health Counselor for Delivering Behavior-Change Health Interventions . In: Proceedings of the IEEE International Conference on Healthcare Informatics (ICHI'13) . IEEE. pp. 46 - 55 ( 2013 )

32. Lisetti , C.L. : Embodied Conversational Agents for Psychotherapy . In: Proceedings of the CHI 2008 Conference Workshop on Technology in Mental Health. ACM , pp. 1 - 12 ( 2008 )

33. Dixon-Woods , M. , Cavers , D. , Agarwal , S. , Annandale , E. , Arthur , A. , Harvey , J. , Hsu , R. , Katbamna , S. , Olsen , R. , Smith , L. , Riley R. , Sutton , A.J. : Conducting a critical interpretive synthesis of the literature on access to healthcare by vulnerable groups . BMC Medical Research Methodology 6 ( 35 ) ( 2006 )

34. Cavanagh , K. , Millings , A. : (Inter)Personal Computing: The Role of the Therapeutic Relationship in E-mental Health . Journal of Contemporary Psychotherapy 43 ( 4 ), 197 - 206 ( 2013 )

35. Brouwer , W. , Kroeze , W. , Crutzen , R. , De Nooijer, J. , De Vries , N.K. , Brug , J. , Oenema , A. Which intervention characteristics are related to more exposure to internet delivered healthy lifestyle promotion interventions? A systematic review . Journal of Medical Internet Research , 13 ( 1 ), e2 ( 2011 )

36. Fry , J.P. , Neff , R.A. : Periodic Prompts and Reminders in Health Promotion and Health Behavior Interventions: Systematic Review . Journal of Medical Internet Research 11 ( 2 ) e16 ( 2009 )

37. Eysenbach , G. : The Law of Attrition . J Med Internet Res 7 ( 1 ):e11 ( 2006 )

38. Cassell , J.: More Than Just Another Pretty Face: Embodied Conversational Interface Agents . Communications of the ACM 43 ( 4 ), 70 - 78 ( 2000 )

39. Klein , J , Moon, Y , Picard, R.W.: This computer responds to user frustration: Theory, design, and results . Interacting with Computers 14 ( 2 ), 119 - 140 ( 2002 )

40. Mentis , H.M.: Memory of frustrating experiences . In: Nahl, D. , Bilal , D . (eds.) Information and Emotion - The Emergent Affective Paradigm in Information Behavior Research and Theory. Information Today , Medford ( 2007 )

41. Baker , R.S., D'Mello , S.K. , Rodrigo , M.M.T. , Graesser , A.C. : Better to Be Frustrated than Bored: The Incidence, Persistence, and Impact of Learners' Cognitive-Affective States during Interactions with Three Different Computer-Based Learning Environments . International Journal of Human-Computer Studies 68 ( 4 ), 223 - 241 ( 2010 )

42. D'Mello , S. , Olney , A. , Williams , C. , Hays , P. : Gaze tutor: A gaze-reactive intelligent tutoring system . International Journal of Human-Computer Studies 70 ( 5 ), 377 - 398 ( 2012 )

43. Nijhuis , C. : Gerandomiseerde, gecontroleerde studie naar het effect van Acceptance and Commitment Therapy & Mindfulness op depressieve klachten : preventieve zelfhulpcursus "Voluit leven" Master Thesis , University of Twente, Behavioural, Management and Social Sciences ( 2010 ) http://purl.utwente.nl/essays/60478