Efective interventions for the prevention and treatment of child and adolescent obesity play an important role in reducing the global health and economic burden of non-communicable diseases. Although multi-component interventions targeting various health behaviors are deemed promising, evidence for their efectiveness is still limited. Self-regulation seems to be a relevant working mechanism in this regard. Therefore, we propose a playful, smartphone-based self-regulation training that also utilizes the health benefits of a slow-paced breathing exercise. The mobile app uses the microphone of the smartphone to detect breathing sounds (e.g. inhalation, exhalation) and translates these sounds into a visual biofeedback on the smartphone screen. The design and evaluation of a very first prototype is described in this interdisciplinary work of obesity experts, clinical psychologists, young patients, and computer scientists. The apps' breathing detection module uses a random forest tree for quasi real-time classification of the incoming audio samples and biofeedback generation. A study with 11 children and adolescents with obesity was conducted to assess the prototype. Results indicate overall positive evaluations and suggestions for improvement. Implications and limitations are discussed, and an outlook on future work is provided. human-computer interaction, self-regulation, digital health intervention, biofeedback, breathing training, breathing detection CEUR Workshop Proce dings
cular diseases or mental disorders, are the leading cause of death worldwide, contributing to 73% of deaths [1].
0000-0001-5939-4145 (T. Kowatsch); 0000-0002-2576-6569 (Y. X. Lukic); 0000-0001-8761-8214 (O. C. Keller); 0000-0001-6478-7269 (N. Farpour-Lambert); 0000-0003-3144-3907 (D. l’Allemand-Jander) of emotional, motivational, and cognitive arousal that are conducive to positive adjustment and adaptation, as reflected in positive social relationships, productivity, achievement, and a positive sense of self.” [12, p. 900] For instance, obese children, require self-regulation skills to resist the urge to eat unhealthy food. In this context, it was demonstrated that children who have experienced loss of control eating report a higher use of maladaptive strategies for the regulation of emotions than children without a history of loss of control eating [13]. Similarly, another study identifies emotional regulation as the moderator for the relationship between perceived stress and emotional eating [14].
Furthermore, a systematic review indicates that selfregulation skills are among the best predictors of outcomes in obesity interventions in adults [15]. Another study with a representative sample of U.S. children not only found a link between self-regulation and the risk of obesity, but further identified that this link is stronger between boys and girls [16]. Moreover, a recent systematic review found that interventions with self-regulation interventions can be efective in children and adolescents, with possible health benefits [ 17]. All in all, selfregulation skills represent a relevant target in multicomponent interventions for the prevention and treatment of child and adolescent obesity.
To this end, we propose a playful self-regulation training for children and adolescents. The training is delivered via a mobile app and focuses on a breathing exercise. The app uses the microphone of a smartphone to detect inhalation, exhalation, silence and noisy sounds to then visually guide the user to perform a slow-paced breathing training. With the help of the visual biofeedback, breathing can be adjusted with the overall goal to improve self-regulation skills. A conceptual overview of the training is depicted in Fig. 1.
After a brief overview of related work in the next section, we describe the design of a very first prototype and the evaluation procedure targeting obese children and adolescents. We then present and discuss the results and conclude with a summary and outlook on future work.
because it is not only a common self-regulation tool [e.g., 18, 19, 20, 21, 22] but also shows positive ”side” efects on cardiac functioning and mental well-being [23, 24, 25].
The work of Carlier et al. [26] is similar to our training as they implemented a mobile game that uses the microphone of a smartphone to detect an ”ommm”-sound to then visually guide children through a breathing exercise. They tested their prototype with three children sufering from autism spectrum disorder. However, results indicated no efects on stress reduction or technology adaptation of breathing based on biofeedback visualization
Goal
Improving self-regulation skills Slow-paced breathing
(inhalation, exhalation, silence, and noise)
acceptance.
Empirical results of another related work by Shih et al. [27] were more promising. They implemented a similar self-regulation training and found positive effects on physiological outcomes and technology acceptance. However, the authors tested their prototype with 19 healthy university students and thus, these findings may not translate to children and adolescents. Another limitation of this study is that the authors employed a breathing detection model based on an attention-based long short-term memory model in conjunction with a preceding convolutional neural network which may not run on older smartphones with limited computational power.
Another study by Hunter et al. [28] investigated whether a slow-paced breathing training with a mobile app that features heart rate variability biofeedback afects the recovery from an artificial stressor. They found that the app had a significant efect on salivary alpha amylase recovery while not showing a significant efect on cortisol recovery or self-reported stress recovery. Technically, the app does not detect breathing but the heartbeat from the smartphone’s rear camera in conjunction with the flashlight. Thus, the app can present the breathing exercise’s impact on the user’s heart rate variability. However, the heart rate variability is not consciously controlled and its measurement is time-delayed. Consequently, the breathing training does not allow responsive user interaction.
The design and evaluation of the smartphone-based selfregulation training was collaboratively carried out by an interdisciplinary team of computer scientists, obese children and adolescents as well as several obesity experts including physicians, psychotherapists, and diet and sport experts from a children’s hospital. The project described in this work was also approved by the local ethics board. The specifics of the design and evaluation phases are outlined in the following sections. 3.1. Design of the Mobile App 3.1.1. User Interface A focus group discussion with 11 young patients, moderated by obesity experts, was conducted as a first step to gather design requirements for the self-regulation training. In response to that discussion, a first conceptual draft of the user interface was developed. The overall goal of the self-regulation training was to ”move” a boat sailing on an ocean towards an isle, far far away, with the help of slow-paced breathing. Specifically, exhalation should imitate wind that blows the boat forward while inhalation should imitate the collection of wind energy for the next breathing cycle. A draft of this idea is depicted in Fig. 2 which also shows a distance-to-destination indicator on the bottom and speech bubbles with additional breathing instructions.
Based on feedback from the obesity experts and young patients, several elements were dropped and revised to further streamline the user interface. For example, the distance indicator at the bottom and the wind energy indicator on the right-hand side were removed so that users could better focus better on the sailboat and its movements toward the destination isle. Further, the speech bubbles were replaced by a digital coach at the top of the screen who had the role to ”guide” users through Figure 2: Draft of the App’s User Interface the training. Moreover, moving clouds were introduced to support the boat movements towards the destination isle and to provide also a visual feedback for inhalation To match the style of the user interface of the app, the sounds. In the latter ”inhalation” case, clouds gathered video clip employs a comic-like character and elements together at the center of the screen while they moved of the mobile user interface (e.g. ocean, sailboat and apart when inhaling. The high-level biofeedback logic clouds). The resulting instructional video clip was shown was also defined collaboratively among young patients, to four young patients who were then asked to perform obesity experts and computer scientists. It is outlined in the communicated slow-paced breathing. The breathing Algorithm 1. A prototype of the graphical user interface technique was assessed by the obesity experts according was then implemented for the Android operating system. to the guidelines communicated through the video clip Fig. 3 shows a screenshot of that interface. and deemed appropriate. 3.1.2. Instructional Video Clip 3.1.3. Breathing Detection To ensure consistent and evidence-based instructions on The overall goal of the breathing detection module of how to perform a slow-paced breathing training, an in- the self-regulation training is to process audio signals structional video clip was produced with the help of the in quasi real time to distinguish between inhalation, exinvolved obesity experts. This video clip would be pre- halation, silence, and noise captured by a smartphone’s sented to the user before she or he would perform the microphone. self-regulation training with the app for the very first In a first step, we aimed at assessing the technical featime. The clip explains in ca. 30s how a deep abdominal sibility of this approach and built a database of audio breathing is conducted and instructs the audience to in- samples. Due to limited access to young patients and to hale through the nose and to exhale through the mouth reduce the burden of them as patients, as well as due to while performing circa six breath cycles per minute [25]. the feasibility character of this investigation, we decided You are doing great.
Keep it up! 124s digital coach messages destination clouds sailboat sailing route to collect audio samples from four doctoral students (2 females; all between 25 and 27 years old). We asked the doctoral students to sit comfortably in a chair in their ofice and perform a slow-paced breathing exercise for three minutes according to the instructions of the video clip described in Section 3.1.2. The audio was recorded with a Samsung Galaxy S6 Edge through a customized app that uses the Android AudioRecord API (PCM, 16bit, 44.1 kHz). The distance to the smartphone for these recordings was about 20cm, a distance we found optimal for the breathing exercise, too. One co-author listened to the resulting recordings and manually cut them into separate audio files labeling them as inhalation, exhalation, or silence. To collect additional samples for silence and noise, the same smartphone was used to record sounds in an ofice. This resulted in audio samples of passing cars and streetcars or the speech of ofice workers, which were manually marked as noise. Silent audio samples were manually labeled as silence. The data collection resulted in around 28,000 audio samples of 80ms length.
This length was chosen as it represented the minimum bufer size that could be acquired through the Android audio engine at the time of implementation.
In a second step, we calculated for each sample the ifrst 13 Mel-frequency cepstral coeficients [ 29] with a window size of 25ms and a 50% overlap. The coeficients
Input: detection = {inhalation, exhalation, silence, noise} Output: biofeedback = {clouds, sailboat, and
digital coach message} 1 while destination not reached do 2 switch detection do 3 case inhalation do 4 clouds gather together 5 case exhalation do 6 clouds expand 7 sailboat moves towards destination 8 digital coach provides positive
feedback case silence do
digital coach motivates user to inhale case noise do digital coach recommends to reduce surrounding noise 13 14 end
end were supplemented with four descriptive statistical measures. From the time-domain we used the mean, variance, and maximum of the raw audio amplitude and from the frequency-domain the peak frequency amplitude.
Third, and consistent with prior work that was successful in detecting breathing patterns [30], a Random Forest model was used with 100 trees, which was empirically found to result in the best prediction performance for our data set.
Since our audio database was relatively small compared to related work [e.g., 27] and to prevent our model from over-fitting, we applied k-fold cross-validation for training and validation. We trained the random forest model using the WEKA library. First, we applied a 80/20 training to test split over all four participants. Second, we conducted 10-fold cross-validation on the training data. Third, we tested the best model on the test set. The performance of the model on the test set is reported in Table 1. The results indicate that the trained model is able to appropriately diferentiate between the four classes for the breathing of known individuals.
Finally, the trained random forest model was integrated into the mobile app using the WEKA Android API. The feature extraction in the app was reproduced using Java in conjunction with the audio processing library OpenIMAJ 1.3.9. The resulting predictions of the incoming data would then trigger the animations of the user interface outlined in Algorithm 1. 3.2. Evaluation Procedure
A study with obese children and adolescents was con- Overall, 8 female and 3 male young 9-16 year-old (M ducted in the children’s hospital of the participating obe- = 12.6, SD = 2.4) children and adolescents with obesity sity experts to assess the technical feasibility and percep- participated in the study. All patients were able to reach tions of the self-regulation training. The procedure was the goal set by the training app in 40 to 120 seconds. as follows. However, obesity experts observed that due to the playful
First, patients were instructed to watch the educational character of the app, three subjects started to perform breathing video clip and to perform the breathing exer- an adverse breathing pattern (e.g. hyperventilation or cise without the app. Obesity experts provided feedback extensive and long exhalation), motivated by the goal to on their breathing to assure a correct technique. bring the sailboat as quickly as possible to its destination,
In a second step, obesity experts handed over the An- despite being instructed otherwise. droid smartphone, the same model used for data collec- The descriptive statistics of the patients’ self-reported tion (i.e. the Samsung S6 Edge), with the self-regulation perceptions of the participating patients are listed in Tatraining app to the patients. The experts then explained ble 2 and corresponding boxplots with raw responses are the purpose of the app and its visual feedback logic. shown in Fig. 4. The high average mean values for all
Finally, obesity experts asked the patients to perform constructs indicate that the patients found the training the app’s slow-paced breathing exercise. The patients’ app easy to use and conducive to relaxation at home. Adgoal was to ”sail” the sailboat to the destination. Dur- ditionally, the patients reported enjoying its actual usage ing the exercise and for safety purposes, obesity experts and indicated that they could even imagine using the observed the patients and intervened in case of any ad- app-based training every day. Finally, patients shared verse breathing activity. Moreover, they noted down that they were able to relax using the app. One-sample whether the goal was achieved and the sailboat reached sign tests confirmed these results as the self-reported the destination, and how long it took the patients to get scores all lie significantly above the neutral scale value there. of zero.
Afterwards, the patients received a questionnaire that The qualitative feedback indicated that the digital coach allowed them to assess the app. Constructs of inter- moderating the self-regulation exercise should be cusest were adopted from technology acceptance research tomizable, for example, regarding their outfit. Moreover, [31, 32] and included perceived ease of use, perceived en- the training session was perceived as too short and thus, joyment, expected usefulness at home, intention to use it was suggested to extend the journey with the sailboat. and perceived relaxation after use. Consistent with prior It was also suggested to add further elements to the ocean work [33, 34], a single item per construct was used to scene, for example, additional milestones such as smaller reduce the burden of the young patients. All items were isles or surface marker buoys as sub-ordinate targets that anchored on 7-point Likert scales ranging from strongly would trigger points when passing by with the sailboat. disagree (-3) to strongly agree (3). All constructs and Interestingly, there were some enquiries into whether item wordings are listed in Table 2. Finally, patients were the training app was available on Apple’s iOS platform, asked to write down any suggestions they may have to which indicated further interest in the training app. improve the app. Overall, the qualitative feedback confirmed the positive quantitative results presented above. strongly agree 3 self-regulation training might be smartphone addiction [35, 36]. Limiting the number of exercises per day could 2 be a solution in this regard. Finally, adding additional 1 playful elements as suggested by the young patients raises the question to which degree the experiential efect neither 0 of the app can potentially cancel out the development of −1 self-regulation skills and other health benefits of slowpaced breathing such as calming down or strengthening −2 the cardiac system. Related work provides first evidence −3 that both experiential and instrumental efects can coexdsistraognrgeley ePaesreceoifvuesde ePnejrocyemiveendt uEsaxetpfhueolcnmteeesds eInvttoeernuytsdioeany Preelarcxeaitvioend lfiisnteT[2hp7ies,r3fwo7ro]m.rkanhcaes oalfsothseevdeertaelctliiomnitmatoiodnesl. isFibrsats,etdheonofa small sample of only four doctoral students and not Figure 4: Boxplots and Answers of Self-reported Perceptions on breathing data from the target population. Both asof 11 Young Children and Adolescents with Obesity pects, the small sample size resulting in low variance of breathing sounds and the mismatch of model development with population A and assessment by population 5. Discussion B, limit the generalizability of the findings. Second, the training and test sets were collected in the same context, The current work presented a playful, smartphone-based i.e. the same recording environment and smartphone self-regulation training that was collaboratively devel- model, and contain breathing sounds from the same inoped by an interdisciplinary team of obesity experts, clin- dividuals. Consequently, the detection performance for ical psychologists, children and adolescents with obesity, unknown individuals, other devices, or diferent envias well as computer scientists. The evaluation of the train- ronments remains unknown. Third, the data collection ing with 11 young obesity patients showed the technical with respect to noisy environments was limited to only feasibility, as all patients were able to bring the sailboat one specific environment (the ofice). Fourth, only one to its destination. Moreover, self-reports of the partici- specific biofeedback theme was evaluated, i.e. the ”oceanpating patients resulted in overall positive technology and-sailboat” theme, and thus, it is open to which degree assessments and various suggestions for improvement visual elements may have an impact on the efects of were provided. the self-regulation training. Third, the cross-sectional
However, the evaluation also resulted in relevant in- study setting in the children’s hospital does not allow to sights regarding potential side efects. First, the playful draw any conclusions on long-term engagement with the biofeedback visualization motivated patients to adopt app. Finally, the evaluation procedure did not contain a breathing technique that could lead to dizziness due any validated instrument to assess self-regulation and to hyperventilation or prolonged periods of exhalation. thus, no conclusions can be drawn in this regard, too. A biofeedback visualization that ofers time-restricted inhalation and exhalation windows may overcome this problem. That is, the sailboat could only be moved for- 6. Summary and Future Work ward during a pre-defined time window that promotes a ”healthy” slow-paced breathing [27]. Another potential side efect promoted by the playful nature of the
National Science Foundation for their support through grants 159289 and 162724. proposed, implemented and evaluated a breathing-based self-regulation training with young patients afected by obesity.
However, the current work also points towards opportunities for upcoming research. First and foremost, future work may focus on environment-agnostic breathing detection that generalizes among individuals, smartphones, headsets, and various ”distracting” soundscapes. Second, micro-randomized trials may be conducted to assess an optimal balance of experiential vs instrumental interface designs. Third, guided-biofeedback interfaces may limit cheating in breathing, which, in turn, could reduce any adverse breathing patterns. And finally, the impact of self-regulation training on self-regulation skills and relevant lifestyle behavior should be assessed in longitudinal ifeld studies that target the prevention and treatment of child and adolescents obesity. self-regulation with obesity in boys vs girls in a PervasiveHealth’19, Association for Computing us national sample, JAMA Pediatrics 172 (2018) Machinery, New York, NY, USA, 2019, p. 452–461. 842–850. doi:1 0 . 1 0 0 1 / j a m a p e d i a t r i c s . 2 0 1 8 . 1 4 1 3 . doi:1 0 . 1 1 4 5 / 3 3 2 9 1 8 9 . 3 3 2 9 2 3 7 . [17] A. Pandey, D. Hale, S. Das, A.-L. Goddings, S.-J. [27] C.-H. Shih, N. Tomita, Y. X. Lukic, A. H. Reguera, Blakemore, R. M. Viner, Efectiveness of univer- E. Fleisch, T. Kowatsch, Breeze: Smartphone-based sal self-regulation–based interventions in children acoustic real-time detection of breathing phases for and adolescents: A systematic review and meta- a gamified biofeedback breathing training, Proc. analysis, JAMA Pediatrics 172 (2018) 566–575. ACM Interact. Mob. Wearable Ubiquitous Technol. doi:1 0 . 1 0 0 1 / j a m a p e d i a t r i c s . 2 0 1 8 . 0 2 3 2 . 3 (2019) Article 152. doi:1 0 . 1 1 4 5 / 3 3 6 9 8 3 5 . [18] M. Martin, M. Seppa, P. Lehtinen, T. Toro, [28] J. F. Hunter, M. S. Olah, A. L. Williams, A. C. Parks, Breathing as a Tool for Self-Regulation and Self- S. D. Pressman, Efect of brief biofeedback via a Reflection, Routledge, London, UK, 2016. doi: 1 0 . smartphone app on stress recovery: Randomized 4 3 2 4 / 9 7 8 0 4 2 9 4 7 2 5 7 2 . experimental study, JMIR Serious Games 7 (2019) [19] Y.-Y. Tang, Y. Ma, J. Wang, Y. Fan, S. Feng, Q. Lu, e15974. doi:1 0 . 2 1 9 6 / 1 5 9 7 4 .
Q. Yu, D. Sui, M. K. Rothbart, M. Fan, M. I. Pos- [29] S. B. Davis, P. Mermelstein, Comparison of parametner, Short-term meditation training improves at- ric representations for monosyllabic word recogtention and self-regulation, Proceedings of the Na- nition in continuously spoken sentences, IEEE tional Academy of Sciences 104 (2007) 17152–17156. Transactions on Acoustics, Speech, and Signal Prodoi:1 0 . 1 0 7 3 / p n a s . 0 7 0 7 6 7 8 1 0 4 . cessing 28 (1980) 357–366. doi:1 0 . 1 1 0 9 / T A S S P . 1 9 8 0 . [20] N. Harvey, Mindful little yogis: self-regulation tools 1 1 6 3 4 2 0 .
to empower kids with special needs to breath and [30] T. Rosenwein, E. Dafna, A. Tarasiuk, Y. Zigel, Derelax, Singing Dragon, London, UK, 2018. tection of breathing sounds during sleep using non[21] Z. Wang, A. Parnandi, R. Gutierrez-Osuna, Biopad: contact audio recordings, in: 36th Annual InterLeveraging of-the-shelf video games for stress self- national Conference of the IEEE Engineering in regulation, IEEE Journal of Biomedical and Health Medicine and Biology Society, 2014, pp. 1489–1492. Informatics 22 (2018) 47–55. doi:1 0 . 1 1 0 9 / J B H I . 2 0 1 7 . doi:1 0 . 1 1 0 9 / E M B C . 2 0 1 4 . 6 9 4 3 8 8 3 .
2 6 7 1 7 8 8 . [31] A. Kamis, M. Koufaris, T. Stern, Using an attribute[22] A. L. Eva, N. M. Thayer, Learning to breathe: A based decision support system for user-customized pilot study of a mindfulness-based intervention to products online: An experimental investigation, support marginalized youth, Journal of Evidence- MIS Quarterly 32 (2008) 159–177. doi:1 0 . 2 3 0 7 / Based Complementary & Alternative Medicine 22 2 5 1 4 8 8 3 2 .
(2017) 580–591. doi:1 0 . 1 1 7 7 / 2 1 5 6 5 8 7 2 1 7 6 9 6 9 2 8 . [32] F. D. Davis, Perceived usefulness, perceived ease [23] M. C. Schumer, E. K. Lindsay, J. D. Creswell, Brief of use, and user acceptance of information technolmindfulness training for negative afectivity: A sys- ogy, MIS Quarterly 13 (1989) 319–339. doi:1 0 . 2 3 0 7 / tematic review and meta-analysis, Journal of Con- 2 4 9 0 0 8 . sulting and Clinical Psychology 86 (2018) 569–583. [33] T. Kowatsch, D. Volland, I. Shih, D. Rüegger, F. Kündoi:1 0 . 1 0 3 7 / c c p 0 0 0 0 3 2 4 . zler, F. Barata, A. Filler, D. Büchter, B. Brogle, [24] A. Zaccaro, A. Piarulli, M. Laurino, E. Garbella, K. Heldt, P. Gindrat, N. Farpour-Lambert, D. l’AlleD. Menicucci, B. Neri, A. Gemignani, How breath- mand, Design and Evaluation of a Mobile Chat App control can change your life: A systematic review for the Open Source Behavioral Health Intervention on psycho-physiological correlates of slow breath- Platform MobileCoach, Springer, Berlin; Germany, ing, Frontiers in Human Neuroscience 12 (2018). 2017, pp. 485–489. doi:1 0 . 1 0 0 7 / 9 7 8 - 3 - 3 1 9 - 5 9 1 4 4 - 5 _ doi:1 0 . 3 3 8 9 / f n h u m . 2 0 1 8 . 0 0 3 5 3 . 3 6 . [25] M. E. B. Russell, A. B. Scott, I. A. Boggero, C. R. [34] T. Kowatsch, W. Maass, I. P. Cvijikj, D. Büchter, Carlson, Inclusion of a rest period in diaphragmatic B. Brogle, A. Dintheer, D. Wiegand, D. Durrer, R. Xu, breathing increases high frequency heart rate vari- Y. Schutz, D. l. Dagmar, Design of a health informaability: Implications for behavioral therapy, Psy- tion system enhancing the performance of obesity chophysiology 54 (2017) 358–365. doi:1 0 . 1 1 1 1 / p s y p . expert and children teams, in: Proc 22nd Euro1 2 7 9 1 . pean Conference on Information Systems (ECIS), [26] S. Carlier, S. Van der Paelt, F. Ongenae, Tel Aviv, Israel, 2014.
F. De Backere, F. De Turck, Using a serious [35] S. Haug, R. P. Castro, M. Kwon, A. Filler, game to reduce stress and anxiety in children T. Kowatsch, M. P. Schaub, Smartphone use and with autism spectrum disorder, in: Proceedings smartphone addiction among young people in of the 13th EAI International Conference on switzerland, Journal of Behavioral Addictions 4 Pervasive Computing Technologies for Healthcare, (2015) 299–307. doi:1 0 . 1 5 5 6 / 2 0 0 6 . 4 . 2 0 1 5 . 0 3 7 . [36] M. Kwon, D.-J. Kim, H. Cho, S. Yang, The smartphone addiction scale: Development and validation of a short version for adolescents, PLOS ONE 8 (2014) e83558. doi:1 0 . 1 3 7 1 / j o u r n a l . p o n e . 0 0 8 3 5 5 8 . [37] Y. X. Lukic, C.-H. I. Shih, A. H. Reguera, A. Cotti,
E. Fleisch, T. Kowatsch, Physiological responses and user feedback on a gameful breathing training app: Within-subject experiment, JMIR Serious Games 9 (2021). doi:1 0 . 2 1 9 6 / 2 2 8 0 2 .