The International Workshop on Behavior Change Support Systems provides a place to discuss recent advances in BCSS research. The aim of this year's workshop was to discuss and share examples and experiences from research on BCSSs regarding both the evaluation and improvement of these systems, as the systems use more and more types of data over longer periods of time. Fitting the multidisciplinary character of the workshop, a wide variety of topics was presented in which the use of extensive data plays a role: occupational health, diet and healthy foods, persuasive dialogues related to food intake, entertainment for people with disabilities, and telemonitoring of heart failure patients. Group discussions focused on two key aspects related to the use of more extensive data: approaches to encourage transparency of the use of data to prevent `black-box thinking', and pointing out a strategy for data driven coaching over longer periods of time.
3,
Behavior change support systems (BCSSs), with its specific implementations of
technology, have the potential to persuade people to change behaviors or even lifestyles.
With its ever growing possibilities, it is impossible to imagine our everyday life without
smart technology. Technology is not only embedded into our homes or workplaces,
people are carrying more and more technology with them throughout the day.
Technology that truly blends in with our lives and habits, and adapts to measured preferences,
behaviors, and responses, may more effectively support our goals and steer our
behavior. The increased pervasiveness of technology provides the opportunity to use real
world data tracked in daily life over longer periods of time in persuasive systems [
The increased use over time of persuasive systems introduces a number of
challenges, as well as opportunities. For example, the data that is being collected via BCSSs
provide great opportunities for data-driven coaching to enhance and retain health and
wellbeing of users. Large amounts of valid, real-time, context-based, and
technologygenerated data (e.g., log data [
At the same time, this increase of valuable quantitative data requires reflecting on
the evaluation of BCSSs. Traditional evaluation approaches do not always provide
insight into the unique interaction process between the user and the technology. In fact, a
description of how the technology supports the user in performing certain behavior is
often missing in such evaluations [
The sixth International Workshop on Behavior Change Support Systems provided a place to discuss recent advances in BCSS research. The aim of this workshop was to discuss and share examples and experiences from research on BCSSs regarding the use of more and more types of data for evaluation and improvement of these systems. The following section lists the broader set of topics of interest of which some were represented in the submissions. 2
Topics for submissions included, but were not limited to: • Smart monitoring for persuasive coaching especially in (but not limited to) the area of health and well-being • Developing just-in-time persuasive prompts and feedback to support behavior and to create adherence and engagement to different technologies, using data generated by smart sensors, self-tracking devices, wearables, etc. • Engagement, integration, connectivity, personalization, and changes in Persuasive
Technology • Interactive visualizations (including virtual coaches and dialogues) for personalization and social support • High tech, human touch/humanizing technology • Connectivity designs for social support, e.g. for lifestyle change and improving wellbeing • Design guidelines for the design, implementation and evaluation of BCSSs • Persuasive strategies related to different outcomes (engagement; resilience; attitudes; compliance; behaviors) and levels (individual; community; society) of change This year, we especially welcomed papers regarding the evaluation of BCSS: • Methods for measuring the impact of BCSSs and smart persuasive environments on individuals, community, and society • Methods for measuring the effect of persuasive strategies on task adherence (e.g., via fractional factorial designs) • Methods (including mixed methods approaches) for measuring various aspects of BCSSs in the wild; considering context and including process and product measurements in a real-life setting • Methods or approaches to evaluate the persuasiveness of different technologies for BCSSs (mobile, ubiquitous, ambient technologies, virtual environments, sensorbased, etc.) • Advanced big data analytics for analyzing and interpreting usage data and self-tracking data from (multimodal) sensors • Translating the outcomes into multimodal feedback cues, and their effects on adherence and outcomes • Advanced analytics to predict adherence and to identify usage patterns and its effects on adherence • Implementation strategies to deal with proprietary closed algorithm layers to gather reliably gather data of daily use, using commercial sensor devices 3
The five papers presented during this year’s edition of the workshop all relate to the
topic of how to use extensive data for the design and evaluation of BCSSs. At the core
of any BCSS should lie a clear vision of how persuasion can occur. In the first paper of
these proceedings, Kekkonen et al. contribute to this vision with a systematic literature
review to gain insight into existing interventions in occupational health [
In the second paper of this proceedings, Manninen & Tikka describe the
development process of a mobile system that stimulates the vegetable and fruit consumption of
its users [
Using automatically collected data, such as log data or observational data for actively
improving a system or tailoring its response to specific user actions, requires reliable
data measurement. In the third paper of this year’s BCSS proceedings, Van Delden &
Reidsma describe issues with the reliability and validity of data that can be at play when
relying on automatic data collection and analysis [
Once a technology is in use, researchers and developers can use extensive usage and
monitoring data to evaluate the system and to better tailor interventions to the user. In
the fourth paper, De Franco et al. propose an evaluation matrix for measuring the
persuasiveness of BCSS by analyzing interview data [
An application of using monitoring data to evaluate BCSSs is found in the last paper
of this proceedings. In this paper Cruz-Martinez et al. describe how daily monitoring
of patient measurements (e.g., weight, blood pressure, and heart rate, via logfiles of the
use of a platform (IMediSense)) in combination with qualitative interviews and
usability tests can be used to guide heart failure rehabilitation to prevent complications and
to coach patients health and wellbeing at home in a personal and motivating fashion.
At the hearth of this persuasive data-driven coaching approach is the premises that
keeping self-monitoring as a simple process is important to promote adherence to the
system among the users. Consequently, it is envisioned that the data analysis will enable
the timely identification of vital symptoms to prevent complications (resubmission to a
hospital) [
After the work in the previously described papers was presented, the group of participants was split in two groups in order to discuss two main challenges during the second half of the workshop. We introduced nine different challenges to choose from, five were based on the papers that were presented and there were four more general challenges related to extensive data in the design and evaluation of BCSSs. For each challenge we also provided one example of how we interpreted the challenge.
The group of participants was split into two groups and invited to pick one or more challenges for a group discussion. During the group discussion, participants were invited to elaborate on how they interpreted the challenge from their point of view and background. As a second step they were ask to add their experiences and examples from their own research. Based on this, participants were invited to discuss potential approaches to deal with the related aspects of the challenges. To conclude the workshop, both groups prepared a pitch to present their preliminary conclusions regarding their challenges and the outcomes of the discussions. The discussed challenges and the outcomes are described below.
Challenge 1: How can we keep the BCSS transparent for people (both developers and users), considering black-box thinking; how to let the system tell why it does something? During the discussion it became apparent that transparency primarily is the ability to explain why things happen in a system. In BCSSs, transparency can lack in several stages of the research process and in different elements of a system or. It often remains unclear what (behavioral) theory the system is built upon, of what elements a system exists and how these elements were operationalized (both functional and in the design). For example, how does a BCSS that uses data for automatic coaching make decisions? What parts are based on adaptive/learning algorithms, and what decisions are actually made by the designer? The participants agreed that this lack of transparency can have negative consequences for the trust in and use of systems. Several approaches were suggested to tackle this negative consequence. First, it was stated that true transparency goes beyond merely a description of the BCSS from an expert (or technological) point of view. It also requires the effort to support the `average user’ in understanding the working mechanisms of a system. In that sense, the advice of the participants to researchers and developers would be to not“ celebrate the mystery of the algorithms”. Rather, it is essential to explain the working mechanisms of a BCSS on different levels, in such a way that it is also understandable for the general public. At this time, developers (including companies) do often not allow to see the actual implementation of the algorithms. From a societal point of view it seems good to encourage development parties to share the algorithms used, and for researchers by providing a good example by always sharing theirs. There is also a task for editors of journals, for example by encouraging the publication of “permanent” links to such information. The group discussed several examples to indicate that in the long run taking accountability, being transparent, being credible, and showing integrity, can actually make a BCSS more effective. For one, this shift away from black-box thinking could help to explain to users why certain recommendations are made. Challenge 2: Which persuasive design features are suitable for data-driven coaching? And why (not)? The group discussed which persuasive features are suitable for data-driven coaching. The main conclusion of the discussion was that ‘persuasion patterns’ differ according to the stage of a user. For example, the needs of a newly diagnosed diabetes patient regarding automated coaching probably differ from the needs of a more ‘experienced’ patient. This might require a move towards user-phase adaptive coaching based on the user profile and log data regarding the use of systems where the system provides ‘active coaching’ and primary task support for new users. Over time, users of a BCSS will probably become aware of their own behavior and some habits are formed. In this phase the coaching application should provide “maintenance coaching” using unobtrusive interaction and communication. When a user tends to fall back into old habits of behavior, the BCSS should go back to the “active coaching” phase. This time, the content and subjects in the active coaching phase should differ from the content and topics provided at the first use. After all, users probably know the basic skills and coaching could therefore focus on personal goals to prevent fall backs in the future. The persuasive design features are all suitable for data-drive coaching and can be selected based on the data collected during the use of a BCSS. 5
Based on the papers and discussion we can conclude that to benefit from the persuasive opportunities offered by data-driven BCSSs, researchers and developers need to deploy a good rationale or strategy for using extensive data for the development and evaluation of their BCSSs and report this clearly. Obviously, data quality and data analysis should be accurate for the used approach. Finally, interpretation of the data and feedback to the user requires to be provided timely and in a relevant matter. From the discussions it is apparent that there is still plenty of work to be done to address this and other points to create more effective BCSSs. This work includes the development of more transparent systems and information about the working mechanisms, as well as new opportunities from a user-phase adaptive coaching approach. Both challenges can improve our next generation of BCSSs and, provide an interesting direction for the next BCSS workshops to focus on.
Acknowledgements. We like to thank all participants for their contributions to the workshop, both their papers and the invaluable input during the discussions. Especially a big thanks go out to Markku Kekkonen, Ryan Wold, Dominic De Franco, Laurence Alpay, Piiastiina Tikka, and Aniek Lentferink. 6