Virtual Reality (VR) has emerged as a mainstream opportunity to develop immersive virtual environments (VE) that enable people to be exposed to relevant stimuli systematically and safely. In addition to providing controlled exposure to stimuli, VR systems allow a more efficient and unbiased information gathering with practical applications in diagnosis, intervention, and health monitoring processes. Despite scientific literature demonstrating numerous benefits, interacting with VR technology and its peripherals can be challenging, particularly for populations less familiar with technology (e.g., older adults) or with clinical conditions that are not adequately accommodated by the technology, such as cognitive (e.g., dementia) or physical impairments (e.g., blindness). The Workshop on Virtual Reality for Health and Wellbeing1 was a one-day workshop that was part of the scientific program of the 21st International Conference on Mobile and Ubiquitous Multimedia (MUM) 2022, held at Faculdade de Ciências da Universidade de Lisboa, Lisbon, Portugal. The workshop was organised by a team of junior and senior researchers with academic and scientific expertise in different areas, including VR tools for cognitive and functional training and rehabilitation, games for health, serious games, human-computer interaction, accessibility, pervasive health, digital health technologies, physiological sensing, artificial intelligence, and machine learning. The workshop aimed to provide a platform for networking and exchanging experiences, knowledge, and know-how on the strategies for developing effective and accessible VR tools for diagnosis, intervention, rehabilitation, and monitoring of health and wellbeing. The discussion topics included the strengths and limitations of VR technology in healthcare, ethical, privacy, and access implications of VR technology in clinical populations, and future scenarios,
challenges, and opportunities for using VR technology in health and wellbeing.
Six papers were submitted to the workshop. After a peerreview process involving two to three reviewers per manuscript submitted, five papers addressing various aspects of the use of VR in health and wellbeing were accepted. One author of each accepted paper was invited to present the paper at the workshop, followed by a Q&A session. At the end of each presentation session, workshop participants, including authors of the papers, were invited to discuss the findings presented based on a set of seed discussion topics. We would like to express our gratitude to all authors, keynote speakers, session chairs, participants, and organisation members for making the workshop a success. We hope the Virtual Reality for Health and Wellbeing workshop has been a fruitful meeting and has stimulated the discussion on the strengths, potential, and shortcomings of VR technology applied to promote users' health and well-being. Furthermore, we hope this meeting has laid the foundations for future international partnerships and collaborations.
Lisbon, Portugal, May, 2022 Filipa Ferreira-Brito
João Guerreiro Tiago Guerreio
the Portuguese Association of Psychologists. She holds a Bachelor's degree in Psychology from the University of Évora and a Master's in Neuropsychology from the Catholic University of Lisbon. She is finishing her PhD in the EnviHealth & Co Doctoral Program at the Faculdade de Medicina da Universidade de Lisboa, Portugal, while also serving as a research assistant at LASIGE, Department of Informatics, Faculdade de Ciências da Universidade de Lisboa, Portugal. Her research focuses on designing digital health platforms to assess and promote health-related outcomes, which has resulted in several international journal publications as both author and co-author. She was recognised for her work through the ISAMB/AstraZeneca scientific incentive award in 2019 and two LASIGE Distinguished Papers Awards for works published as first author in 2019 and 2020.
finished his Ph.D at Jožef Stefan Institute at 2015 in Ljubljana, Slovenia. In 2017 was visiting researcher at the University of Sussex, United Kingdom. His R&D experience is in the domains of Computer Science, Artificial Intelligence and Sensor data-analysis. He participated in numerous international and industry projects. He has more than 70 conference publications, and 13 international journal articles, of which 10 with impact factor. He has more than 1047 citations and h-index of 19 based on Google Scholar. He is a co-inventor of one international and two Slovenian patents. He received 2 best paper awards and the award “Best Young Scientist” for 2016 from the President of the Republic of Macedonia. He has won 3 Activity Recognition challenges: EvAAL 2013, – Valencia, Spain; International EMTEQ AR Challenge – London, United Kingdom; International competition ChallengeUP Multimodal Fall Detection – Budapest, Hungary. Oscar Mayora is a senior researcher at FBK, the head of the Digital Health Lab Unit at FBK Center for Health and Wellbeing, and scientific coordinator for the Joint Research Unit between the Trentino Local Health Trust, the Province of Trento and FBK as part of Trentino Salute 4.0 Initiative. He is also an adjunct professor in the Faculty of Cognitive Sciences at the University of Trento (Italy) and FH-Burgenland (Austria). He has published over 100 papers in International Conferences and Journals and participated as Guest Editor of special issues of Journals such as IEEE Intelligent Systems, EURASIP Signal Processing, Springer MONET, and IMIA Journal on Methods of Information in Medicine on the topic of Pervasive Healthcare. Dr. Mayora has coordinated research projects at National and International level. He has been involved in projects on the topic of pervasive healthcare and assistive technologies sponsored by different funding instruments such as EC FP6, FP7, H2020, and ICT Labs, among others, with different roles.
at the Department of Intelligent Systems at Jožef Stefan Institute, Ljubljana, Slovenia. He received his PhD from the Faculty of Computer and Information Science of the University of Ljubljana. At that time, his work focused on heuristic search algorithms. Later, he worked on text mining, bioinformatics, and other topics. Most of his career, though, his main research interest has been the interpretation of sensor and other health-related data using machine learning. He has been the principal investigator in numerous international research projects developing personal health systems and similar. He teaches ambient intelligence at Jožef Stefan International Postgraduate School. He has also served as the chair of the Slovenian Artificial Intelligence Society for two terms and is a member of several programme committees of international conferences. Emilija Kizevska is a researcher and PhD candidate at the Jožef Stefan Institute in Ljubljana, Slovenia. She finished her bachelor’s and master’s studies at the Faculty of electrical engineering and information technologies at the Ss. Cyril and Methodius University in Skopje, N.Macedonia. During her master’s studies, she has a few study visits to Anhalt University of Applied Sciences, Koethen, Germany. She has been a teaching assistant at FEEIT, Skopje, North Macedonia, for 2 years. Her research experience is in the domains of applied Artificial Intelligence and Machine Learning in the areas of Renewable energy and Ambient Intelligence. She has participated in research projects and industrial projects and has been a team supervisor or project team lead in three international competitions in the field of ICT and Robotics. She has held consulting positions across several software companies.
Ciências, Universidade de Lisboa. Previously, he was a Postdoctoral researcher at Carnegie Mellon University. He received his PhD with distinction and honour in Information Systems and Computer Engineering from Instituto Superior Técnico, Universidade de Lisboa. His research in the fields of Human-Computer Interaction and Accessible Computing aims to improve blind people’s access to the physical and digital worlds using novel non-visual interaction techniques and systems. João Guerreiro publishes his research regularly at the premiere venues in Human-Computer Interaction and Accessible Computing (e.g., ACM CHI and ACM ASSETS). Kathrin Gerling is an Assistant Professor at KU Leuven, Belgium. Her work broadly falls into Human-Computer Interaction and Physical Computing. She is particularly interested in how interfaces can be made accessible for audiences with special needs, and how playful interactive technologies can be leveraged to support well-being. Kathrin previously held an appointment as a Senior Lecturer (Assistant Professor) at the University of Lincoln, UK. She received a PhD in Computer Science from the University of Saskatchewan, Canada, and completed an MSc in Cognitive Science at the University of Duisburg-Essen, Germany.
institutes in Europe and the USA, including the Laboratoire de Production Microtechnique at the EPFL (Lausanne), the Institute of Neuroinformatics at the ETHZ (Zurich), at the Institute of Audiovisual Studies at the Technology Department of the Universitat Pompeu Fabra (Barcelona), where I was a Juan de la Cierva research fellow and head of the Robotic Systems Laboratory at the laboratory for Synthetic Perceptive, Emotive and Cognitive Systems (SPECS), and the Quality of Life Technologies and Entertainment technology centers of the Carnegie Mellon University (Pittsburgh). I am currently a professor at the Universidade da Madeira and a researcher at the Madeira Interactive Technologies Institute, where I lead the NeurorehabLab research group. Tiago Guerreiro is an Associated Professor at Universidade de Lisboa (Faculdade de Ciências) and a researcher at LASIGE. His main areas of expertise are Pervasive Healthcare, Mobile, and Accessible Computing. In these areas, he published 100+ peer-reviewed papers and received awards for 10+ papers and an ACM Best of Computing award in 2016. He is Editor-in-Chief for ACM Transactions on Accessible Computing, was the Web for All General and Program Chair in 2016 and 2015, respectively, and was ASSETS 2020 General Chair, among many other service roles. He was an invited expert supporting the European Commission in implementing the Web Accessibility Directive. He is the President of his institution’s Ethics Review Board, ViceDirector and past coordinator of LASIGE’s research line on Accessibility and Aging, and Coordinator of the Data Science Master at FCUL. He leads the Tech&People Labs.
Micaela Fonseca
Micaela Fonseca holds a PhD in Physics (Universidade Nova de Lisboa [PT], 2011). Micaela is a Principal Researcher at HEI-Lab (Digital Human-Environment Interaction Lab) and assistant Professor at the School of Communication, Architecture, Arts and Information Technologies, Lusófona University. Since 2012, she has been an invited Assistant Professor at NOVA FCT for Biomedical Engineering, Physics Engineering and a researcher at LIBPhys-UNL Micaela has been engaged in several VR-based simulation projects; she is co-founder of VR4NeuroPain and Games for Good. She has published several scientific papers in ehealth and serious games. Micaela also studied Directing Fiction and Ceramics (her works have been exhibited in contemporary galleries).
Jun Nishida is an incoming Assistant Professor at the University of Maryland, College Park at the Department of Computer Science. Previously he was a postdoctoral fellow at the University of Chicago, advised by Prof. Pedro Lopes and received a PhD in Human Informatics at the University of Tsukuba, Japan, in 2019. He is interested in exploring interaction techniques and wearable interfaces where people can communicate their embodied experiences to support each other, with applications in the fields of rehabilitation, education, and design. He has received ACM UIST Best Paper Award, ACM CHI Best Paper Honorable Mention Award, Microsoft Research Asia Fellowship Award, and Forbes 30 Under 30 Award, among others.
Designing for Maximized Entertainment. How to contribute to greater ubiquity of immersive digital environments for therapy?
For this workshop, our first keynote, Micaela Fonseca, began her presentation by calling the audience's attention to the fact that a significant shortcoming of traditional neurorehabilitation approaches is the high rate of patient drop-out. This limitation is even more evident when clients are required to do their rehabilitation exercises at home. According to Micaela's perspective, one way to counter the high drop-out rate observed and promote clients' adherence to rehabilitation VR activities, even when they are performed at home, is by treating clients as game players. Instead of a collection of commonly available exercises, a new paradigm for serious games (nowadays applied games) is needed. According to Micaela, for this new approach to be successful, it is necessary a switch from perceiving clients as someone with cognitive or motor limitations to perceived clients as game players that need to enjoy, connect, and be motivated to play the game outside the clinical context (i.e., client's home). However, to achieve this, the games developed for rehabilitation purposes need to provide (achieve) a game-like quality experience as they were designed for the average player. In other words, they need to be engaging and fun (by themselves) independently of being a game for rehabilitation.
To illustrate her perspective, Micaela provides an example PlayersAll project (EXPL/COMOUT/0882/2021). The PlayersAll project explores a novel approach to games for physical and cognitive upper-limb rehabilitation therapy for motor dysfunction clients. A VR-based motor rehabilitation game was developed to embed clients into a Druid's house, where they must follow Druid's instructions to make different magic potions successfully. In this game, the movements that clients need to perform to complete the task in Druid's game were inspired and developed in co-creation with occupational therapists. The ultimate goal of the PlayersAll project is to contribute to greater ubiquity of immersive digital environments for therapy so that their therapeutic benefits are delivered more consistently.
Following this perspective, Micaela's work team follows a co-creation methodology, where a multidisciplinary team (e.g., occupational therapists, psychologists, game designers, and game developers) together with stakeholders (clients, families, and caregivers) gathered efforts to find the best and more engaging VR-based rehabilitation applied games. By using a workflow that includes light, sound, and difficulty levels and sensors, such as eye-tracking, it will be possible to develop increasingly engaging digital games that adapt to clients' performance and skills as they progress in the game.
Based on the perspective presented, Micaela shared with the audience other VR-applied games developed to train technical skills, instrumental activities of daily life in children with autism (e.g., shopping and use of public transports), as well as to rehabilitate clients with psychological disorders such as phobias (e.g., fear of cockroaches) and Post-Traumatic Stress Disorders in war veterans. Another stream of research was presented, for example, studying the impact of specific interactions and movements on emotional magnitude through a virtual reality video game.
At the closing session of our workshop, Jun Nishida delivered an insightful lecture on the importance of bodily cues in human communication. While modern technology allows effective communication via video and text, it often leaves out crucial communication channels, such as bodily cues. These cues are crucial not only for face-to-face communication but also for conveying forces (muscle tension, joint stiffness, etc.), feelings, and emotions. Unfortunately, current communication paradigms rely only on symbolic and graphical communication, neglecting the significance of this additional modality. In his lecture, Jun Nishida addressed how physical experiences can be communicated across people. Nishida discussed his engineering of wearable devices which allow for sharing physical experiences between people, such as between a physician and a patient, including people with neuromuscular diseases and even children. These custom-built on-body interfaces include exoskeletons, virtual reality systems, and interactive devices based on electrical muscle stimulation. The concept was extended to support interactive activities, such as product design, by communicating one's bodily cues.
The lecture ended with a discussion on exploring possibilities enabled by a user interface that communicates more than audio-visual cues. The roadmap for using this approach in new territories, such as allowing more empathic communication, was also discussed. Nishida emphasised the significance of bodily cues in communication and how current technology often fails to account for them. However, through wearable devices and innovative engineering, Nishida demonstrated that it is possible to communicate physical experiences across individuals. The applications of this technology are diverse and can be used to enhance communication between physicians and patients, as well as in product design and other fields. Furthermore, the possibilities of a user interface that communicates more than audio-visual cues are vast, and this technology has the potential to enable more empathic communication in various domains.
In conclusion, Jun Nishida's lecture shed light on the importance of bodily cues in human communication and demonstrated how VR technology could be used to convey physical experiences across individuals.