Especially since the start of the COVID-19 pandemic, the majority of humans spend most of their waking life using digital screen devices. The present paper explains how adaptive interfaces could improve digital well-being by customizing the output data and digital environments to each user. line applications have introduced elements directed to help users control breathing and aid The impact of natural environments on hu- in reducing stress and anxiety. man health to support subjective wellbeing In natural environments, a dynamic exchange has recently been studied [1, 2]. With an ex- takes place between one's physiological state panding variety of electronic media and dig- and one's response to and interaction with ital environments, screen time has inevitably the surrounding space, for instance with oxyincreased across all ages. Research suggests gen intake and carbon dioxide output. In curscreen time is negatively associated with the rent digital environments, conversely, users development of cognitive abilities, and posi- can only respond by choosing static elements tively associated with sleep problems, depres- on the screen. Detecting the user's physiosion, and anxiety [3]. logical state while using digital devices, and Studies have explored self-regulated med- understanding how diferent output data imitation techniques that relieve stress, reduce pact the user can inform algorithmic processes burnout, improve emotional awareness, and of adaptive interfaces to customize the outheighten attention [4]. In recent years, on- put data to the user's physiological state - and to adapt and personalize the digital experience to improve well-being during screen usage.
© 2021 Copyright for this paper by its authors. Use permitted under Creative Commons License Attribution 4.0 InterCPWrEooUrckReshdoinpgs IhStpN:/c1e6u1r3-w-0s.o7r3g (nCCatEiEoUnUaRlR(C-CWBYSW4..o0o).rrgk)shop Proceedings 1.1. The Use of VR and
The cross-disciplinary project “Mindful
Technologies” [
Augmented Reality (AR) devices were used for a project in collaboration with breathing expert Wim Hof in 2018 [8]. Hof’s breathing technique was clinically studied [9] to voluntarily influence the sympathetic nervous system and immune system response. Using AR and a breathing sensor, the users would follow a virtually guided breathing technique and experience a visualization of their breathing patterns in AR and as audio.
The use of VR and AR devices paired with wearables to detect and display the physiological states to the users requires additional hardware, and demands more time spent on screens.
Physiological computing (PG) [10] uses physiological data as system inputs in real-time so software can be changed based on the user’s physiological state, and Photoplethysmography (PPG) [11] is a measurement technique enabling remote vital sign monitoring by using only cameras integrated in laptops and phones.
The project “NEHAN” [12] in 2018 at MIT (Massachusetts Institute of Technology) used PG and PPG to detect the user’s heart rate using the laptop webcam and machine learning, and to display virtual and audio environments to the user. The interactions with the digital environments were analyzed and the visuals and audio with the lowest heart rate were displayed to the user. 2. 2020 Survey on
Our ongoing research found that 55.7% of screen users (n=210) report they sometimes or often adjust screen settings like color, and, 87.1% of screen users experience symptoms like mental exhaustion and eye strain while using screens for a long time, whereas 12.9% don’t experience any symptoms.
The majority (88.6%) of screen users adjust screen settings to improve symptoms during lengthy periods of screen use. Prototype testing of Adaptive Interfaces [13] using PPG and color displays in uncontrolled settings between 2018 and 2020 (n=80) indicated personalization of colors to each user can lead to an inimproved education and reduced anxiety [15, 16].
Color and its efects on psychological functioning have been long intertwined although research in this area is still at a nascent stage.
Theory of Color draws associations between Figure 2: Frequency of Screen Adjustments color categories and emotional response. Longerwavelength colors were found to instigate a feeling of warmth whereas shorter wavelength dividualized reduction in heart rate by up to colors feel relaxing and cool [17]. 20% while experiencing the color display. The right combination of colors can produce a higher level of contrast which in turn 2.1. Adaptive Interfaces for can influence memory retention [18]. Color Screen-based Interactions is also seen to play a vital role in consumer psychology which is characterized by emoRecent development of imaging photoplethys- tional attachment, attention, memory, attitudes, mography (iPPG) leverages subtle changes in and behaviors [19]. light reflected from the skin to capture car- Research has also indicated that diferent diac activity. These signals can be recovered screen polarities and background colors can from many types of commercially available influence reading and comprehension of graphand low-cost cameras (e.g., webcams and sm- ics on a screen [20] with results showing that artphones) [14]. participants performed well while looking at
Continuously detecting physiological data a light mode screen rather than a dark mode of the user during screen- or audio-based in- screen. teractions is now becoming possible using (i) PPG to detect neurological or cardiac activity such as heart rate, heart rate variability or 3. Conclusion breathing rate without additional hardware.
Continuously customizing the output data Recent improvements in webcam quality to to the users’ physiological state is becoming record for more accurate (i)PPG, faster propossible with the ongoing research and de- cessing and cloud computing, along with users’ velopment of Adaptive Interfaces. higher comfort levels with webcams during the COVID-19 pandemic and the increased 2.2. Efect of Virtual presence of stressful screen time, have enabled the potential for a more personalized
Environments human-computer interaction. We hypotheVirtual environments have been optimized to size the research and development of Adapaid neurobehavioral processes including at- tive Interfaces to customize output data to the tention and emotion regulation. Technology users’ physiological and neurological state can is gradually being adopted by organizations result in a stress-reducing interaction within to train employees to be more productive. A digital environments. With the personalizapilot study showed that virtual stimulation tion of the user experience, screen settings and designs can support digital wellbeing as screen time becomes similar to a more dynamic interaction as experienced with natural environments. tion recognition from brain and heartbeat dynamics using wearable sensors,
Scientific reports 8 (2018) 1–15. [8] H. Bend, Augmented reality (ar) wim hof + breath biofeedback and vr film (2018). URL: https: //www.youtube.com/playlist?list=
PLxkGuUxHzSsVdyJJ00gL_zawpQRJti. [9] M. Kox, L. T. van Eijk, J. Zwaag,
J. van den Wildenberg, F. C. Sweep, J. G. van der Hoeven, P. Pickkers, Voluntary activation of the sympathetic nervous system and attenuation of the innate immune response in humans, Proceedings of the National Academy of
Sciences 111 (2014) 7379–7384. [10] G. Jacucci, S. Fairclough, E. T. Solovey,
Physiological computing, Computer 48 (2015) 12–16. [11] V. Kessler, M. Kächele, S. Meudt,
F. Schwenker, G. Palm, Machine learning driven heart rate detection with camera photoplethysmography in time domain, in: IAPR Workshop on Artificial Neural Networks in Pattern Recognition, Springer, 2016, pp. 324–334. [12] N. Picard, H. Bend, Mit 2018. detecting heart rate via webcam, display of personalized visuals and audio (2018). URL: https://www.youtube.com/ watch?v=TCUX0mNya5Q. [13] H. Bend, breathing.ai prototype testing and interviews nov 9 and 10, 2018, 2018. URL: https://www.youtube.com/ watch?v=mpbWQbkl8_g#t=20m15s. [14] E. M. Nowara, D. McDuf, A. Veeraraghavan, Systematic analysis of video-based pulse measurement from compressed videos, Biomedical Optics
Express 12 (2020) 494–508. [15] M. Marquess, S. P. Johnston, N. L.
Williams, C. Giordano, B. E. Leiby, M. D.
Hurwitz, A. P. Dicker, R. B. Den, A pilot study to determine if the use of a virtual reality education module reduces anxiety and increases comprehension in patients receiving radiation therapy, Journal of Radiation Oncology 6 (2017) 317– 322. [16] J. L. Maples-Keller, B. E. Bunnell, S.-J.
Kim, B. O. Rothbaum, The use of virtual reality technology in the treatment of anxiety and other psychiatric disorders, Harvard review of psychiatry 25 (2017) 103. [17] L. Wilms, D. Oberfeld, Color and emotion: efects of hue, saturation, and brightness, Psychological research 82 (2018) 896–914. [18] E. Krahn, Decomposing the efect of color on memory: How red and blue affect memory diferently, Pridi Banomyong International College, Thammasat
University (2018). [19] M. A. Dzulkifli, M. F. Mustafar, The influence of colour on memory performance: A review, The Malaysian journal of medical sciences: MJMS 20 (2013) 3. [20] Y.-N. Shih, R.-H. Huang, S.-F. Lu, The influence of computer screen polarity and color on the accuracy of workers’ reading of graphics, National Library of Medicine 45 (2013) 335–42.