We are demoing Battle Buddy, an mHealth application designed to support access to physical and mental wellness content as well as safety planning for U.S. military veterans. This virtual human interface will collect multimodal data through passive sensors native to popular wearables (e.g., Apple Watch) and deliver adaptive multimedia content specifically tailored to the user in the interdependent domains of physical, cognitive, and emotional health. Battle Buddy can deliver health interventions matched to the individual user via novel adaptive logic-based algorithms while employing various behavior change techniques (e.g., goal-setting, barrier identification, rewards, modeling, etc.). All interactions were specifically designed to engage and motivate by employing the persuasive strategies of (1) personalization, (2) self-monitoring, (3) tunneling, (4) suggestion, and (5) expertise.
This mHealth application has a primary focus on the iPhone and is an extension of a previous virtual human mHealth application [8,7,10,11]. Given the initial focus on iPhones and the tight iOS integration, the primary wearable target device is the Apple Watch, which offers a rich multimodal set of sensors. Data will be read directly from the iOS HealthKit API and is processed using our custom application software. This allows us to support any hardware device able to write data to HealthKit, including FitBit and Garmin devices. Battle Buddy is a Unity application developed using a custom version of the Virtual Human Toolkit [6, 3,4] and RIDE [5], a rapid prototyping middleware for AI-driven simulations. The VHToolkit incorporates and enables automatic audio-visual sensing, speech recognition, natural language processing, nonverbal behavior generation, nonverbal behavior realization, text-to-speech generation, and rendering, features which will be added to Battle Buddy through user-centered design processes. Data is leveraged to enable the decision-making algorithms and intervention manager.
A subset of the data will be analyzed and evaluated on the client devices to provide users with real-time and actionable feedback (e.g., daily progress towards personal fitness goals, questionnaire results, etc.); the remainder of the data (e.g., UI interaction data, recorded user voice audio) will be collected and post-processed on the server. A user-centered iterative design process was used in the development of this application, leveraging findings of empirical reviews of effective behavior change techniques employed in mobile health apps [6]. In particular, the persuasive strategies of (1) personalization (e.g., creating a customized safety plan, (2) selfmonitoring (e.g., mood tracking), (3) tunneling (e.g., guided breathing exercise), (4) suggestion (e.g., app initiated pushes/prompts), and (5) expertise (e.g., quick connection to suicide hotline).
This work was sponsored by the US Army under contract W911NF-14-D-0005 and the VA Mission Daybreak Challenge, in collaboration with SoldierStrong. The content of the information does not necessarily reflect the position or the policy of the Government, and no official endorsement should be inferred.
Mozgai, S., Hartholt, A., Rizzo, A.: The passive sensing agent: A multimodal adaptive mhealth application. In: 2020 IEEE International Conference on Pervasive Computing and Communications Workshops (PerCom Workshops). pp. 1–3. IEEE (2020) Mozgai, S., Hartholt, A., Rizzo, A.S.: An adaptive agent-based interface for personalized health interventions. In: Proceedings of the 25th International Conference on Intelligent User Interfaces Companion. pp. 118–119 (2020) Mozgai, S., Lucas, G., Gratch, J.: To tell the truth: Virtual agents and morningmorality. In: Intelligent Virtual Agents: 17th International Conference, IVA 2017, Stockholm, Sweden, August 27-30, 2017, Proceedings 17. pp. 283–286. Springer (2017) Mozgai, S.A., Femminella, B., Hartholt, A., Rizzo, A.: User-centered design modelfor mobile health (mhealth) applications: A military case study in rapid assessment process (rap). In: Extended Abstracts of the 2021 CHI Conference on Human Factors in Computing Systems. pp. 1–8 (2021) Rizzo, A. Hartholt, A., Mozgai, S.: From combat to covid-19–managing the impactof trauma using virtual reality. Journal of Technology in Human Services 39(3), 314–347 (2021) Tam-Seto, L., Wood, V.M., Linden, B., Stuart, H.: A scoping review of mentalhealth mobile apps for use by the military community. Mhealth 4 (2018)