The usage of VR technology to allow instructors and therapists to ofer safe, repeatable and diversifiable learning environments for individuals with Autism Spectrum Condition has recently bloomed in scientific literature. However, despite the encouraging results regarding the clinical efectiveness of this approach to relieve their social anxiety and improve their social emotional skills, several challenges are still to be addressed from both technological and clinical perspectives to achieve its full potential as a digital therapy. In this paper a description of the most critical challenges together with a suggested set of intertwined research directions is provided. Moreover, a description of the main achievements obtained out of an Innovation project aimed at tackling a subset of these challenges is provided, with a short discussion on the preliminary results obtained on a recent experimental pilot.
BITs-SST and F2F-SST in improving the social skills of
children and adolescents. More importantly, no
signifiIndividuals with Autism Spectrum Condition (ASC) typi- cant diferences between the two intervention modalities,
cally present some deficits in social communication and thus paving the way to expand intervention resources
are often characterized by the presence of restricted in- and increase access to services that could reduce many of
terests and repetitive behaviors. The rate of kids diag- the costs of getting therapeutic services for individuals
nosed with autism is increasing worldwide together with with ASC and their families.
the demand for psycho-behavioral therapies provided Previous work shows the potential of VR technology to
by Specialized Centers (SC). Lack of therapies for youth allow instructors and therapists to ofer safe, repeatable
with ASC leads to higher rates of depressed and socially and diversifiable learning environments for individuals
isolated individuals in their adulthood, which may lead with ASC [
Even though the first research papers exploring the it remains to be demonstrated that such interventions
usage of Virtual Reality Environments (VRE) to facilitate are able to train or modify the complete behaviour of the
skills development of people with ASC can be dated back selected individual [
Workshop on Virtual Reality for Health and Wellbeing, 21st International Conference on Mobile and Ubiquitous Multimedia, November 27th-30th, 2022, Lisbon $ esalvadori@fbk.eu (E. Salvadori)
© 2023 Copyright for this paper by its authors. Use permitted under Creative CPWrEooUrckReshdoinpgs IhStpN:/c1e6u1r3-w-0s.o7r3g CCoEmmUoRns LWiceonsrekAstthribouptionP4r.0oIncteerenadtiionnagl s(CC(CBYE4U.0)R.-WS.org)
Opportunities in VR for Autism prove the interaction and engagement towards achieving the therapeutic goals.
Serious games and afective computing: Despite In order to achieve a maturity level of VR for supporting the large amount of recent scientific works on the apindividuals with ASC, a series of challenges are required plication of VR to improve social and emotional skills to be addressed. In this paper we outline these challenges of individuals with ASC, in most cases the individual and the opportunities that each of them could provide in is exposed to immersive scenarios duplicating real-life the implementation of VR-empowered digital therapies: settings. However, the development of VR scenarios
Scalability of the VR validation: Most previous leveraging design principles around the concept of seristudies incorporating VR for autism target a very low ous games has provided evidence about a stronger ennumber of subjects that render the preliminary findings gagement by the participants [7]. The recent rise of the not generalisable. In [8] authors identified the need for afective computing approach in developing VR games more extensive involvement of clinicians, caregivers and with therapeutical objectives is a promising direction topatients as informants. Indeed, the generalisability of ward the deployment of engaging platforms, especially their findings was hindered by the lack of standard re- when the user group is composed by children and youth search design and the possibility to scale-up from prelim- with ASC, a target composed in large part by passionate inary findings. This applies to the duration of treatments, videogamers [16]. In particular the implementation of number of sessions and participants required for guar- multi-player VR settings is a very promising direction anteeing the results’ reliability [6] and is confirmed also since it facilitates a stronger interactions among the playin other scientific works [ 9, 7]. In order to overcome ers involved and opens interesting research directions this issue, future studies should concentrate in organis- toward the involvement of mixed groups composed by ing longitudinal Randomized Controlled Trials (RCTs) normo-typical and ASC individuals. involving enough participants to achieve statistical rel- Conversational agents in VR: The adoption of chatevance required for reproductivity of the studies and bots and conversational agents to benefit individuals with generalisations at larger scale. mental health issues has been largely studied in scien
Predictive therapy: Through the acquisition and tific research [ 17]. Many authors have explored their analysis of subjects data in larger longitudinal trials, it application as a therapy tool for people with ASC, i.e. would be possible to augment the knowledge about how as one of the components of the so-called“social robots”, patients interact in controlled VR environments and how robots that can interact with humans and the environthese interactions could impact in their therapy. The data ment, thanks to the continuous AI-based analysis of audio acquired could be used to train AI- machine learning and video data collected in real-time [18] or as a convermodels to infer patients’ state and eventually to improve sational agent facilitating social skills development [19]. treatment by implementing personalised, predictive ther- The application of virtual agent technologies associated apy approaches. The unique attributes of each individual to a number of Non-Player Characters (NPCs) within with ASC suggests that a data-driven approach to ther- a VR setting will therefore be an important element to apy is of utmost importance in order to better understand both relieve the therapist intervention during the session how they interact responding to diferent interventions as well as a tool to measure and evaluate social comand services [10]. munication skills improvement of the individuals with
Advanced digital excipients for therapy: A major ASC.
opportunity for realising the potential of virtual reality- Uptake in clinical practice: As for other types of
enabled digital therapeutics for ASC is the use of richer therapies, the adoption of VR therapy and its uptake in
sensing and actuating tools [11]. Current research is aim- clinical practice could be realised only after a clear
demoning to design a new generation of headsets that could be stration of its eficacy through successful clinical trials.
used as both, sensing and delivering platforms for immer- In [20] a first RCT examining a combination of VR
envisive environments. A major functionality of these devices ronment and one cognitive behavioural therapy session
would be the detection of a wider range of parameters to address specific phobias in young people with ASC is
within the headsets such as movement of face muscles reported. Despite the encouraging results obtained in the
[12], detection of skin conductivity, high resolution eye- trial, the additional impact of VR over traditional
thertracking [13], temperature sensing, inertial sensors, heart apy remains unclear, and the cost-efectiveness needs
rate, electroencephalography (EEG) [14], respiration rate to be assessed. Moreover, the lack of standardisation
and speech recognition among others. The combination of interventions and diagnosis/ASC classification tools
of these signals could provide better under-standing of emerges clearly in literature reviews on the topic [
Ethics and safety: As for other digitally-empowered performed by collecting heart-rate signals generated by therapies management, the use of VR for individuals with an HRM bracelet.
ASC would require a thorough investigation in the ethi- Compared to most of the VR tools proposed in literacal and safety issues involved in the use of the proposed ture to reinforce social and emotional skills which levertechnologies. The envisioned future solutions should ages on the replication of real life settings (like school, consider privacy-by-design approaches in which patients canteen, etc), the VR experience developed within XR4A personal data would be also kept confidential and secure is built around a fantastic game adventure that happens through adequate technical and organisational provision- partially on Earth and partially on a foreign planet. The ings (i.e. such as anonymisation, pseudonymisation, data idea is to transport the player into a more engaging setencryption, access control, etc.). In the same way, future ting where the group of teens act as a team of space solutions and their implementation should consider spe- explorers. The objective of the adventure is to help sciencific mechanisms to avoid stigmatisation of users. More- tists to retrieve a very rare material available on a planet over, a minimum ethical, security and safety provisioning called “Zentastic” they can use to build an instrument in future developments should consider as a minimum needed to fix the Ozone layer on Earth, thus saving hurequirement the guarantees proposed by GDPR regula- man beings from extinction. tion to protect personal information and patients rights; Figure 1 provides a snapshot from the three main VR other more strict considerations could be considered for scenes where the players are involved to accomplish the the specific case of ASC in which the target patients de- Mission assigned by the scientists. Each of the VR scenes cisional and communication capabilities may be more were designed in collaboration with the therapists of compromised due to the specific characteristics of the Cooperativa Albero Blu, a Rehabilitation Center based disease and the fact that therapy is suggested already for in Italy, and are aimed at facilitating collaborative and underaged patients. problem-solving activities among the teammates as well as to improving their executive functions, e.g. selective attention. 3. XR4A: VR-based Tele-therapy The VR application has been developed with the Unity for Autism framework [23], using assets bought on its asset store, and deployed on Meta Oculus Quest 2 headsets. The In this section we present the XR4A system as a pre- gaming backend is managed by the Photon Cloud, a SaaS liminary development intending to address some of the platform that facilitates the synchronization of events challenges and opportunities mentioned before. XR4A is among multiple players on a Unity-based VR application an Innovation-driven project co-funded by EIT Digital [24]. The VR application interacts with Ticuro Reply [25], aimed at developing an XR-based tele-therapy solution a platform developed by project partner Reply to facilifor teens and young adults with neurodevelopmental dis- tate tele-medicine and tele-monitoring, with a web-based orders like autism [22]. Compared to the large majority front-end that allows to manage the users and schedof scientific publications on VR for autism, the developed ule the VR session, and a mobile application collecting solution focuses more on exploring its viability as a tool heart-rate data information from each player via a Pofor remote therapy. Moreover, the developed platform lar Sense Verity Heart Rate Monitor bracelet [11]. This within XR4A is built in the perspective of tackling several platform facilitates the collection and analysis of clinical of the opportunities identified in Sect. 2, in particular and environmental data, in full compliance with privacy scalability of the validation, detection of emotional status, law (GDPR). In the Ticuro Reply backend, a Machine predictive therapy and serious games. Learning engine elaborates the flow of biometric data
The implemented VR solution focuses on a multi- from each player to facilitate stress detection through a player immersive setting where a small group of teens number of messages that can be triggered to the therapist can play and interact with each other through their digi- involved in the VR scene. tal avatars. Moreover, the therapist is also participating A first experimental pilot has been performed in July within the immersive setting with her own avatar acting 2022 with more than 30 kids and teens from diferent as a facilitator very similarly to what happens in tradi- age groups. The aim of this preliminary assessment was 4. Conclusions mainly understanding their level of engagement with the game environment and challenges, as well as the therapist’s capability to handle diferent situations in which Even though many research papers have provided evthe teams were involved along the adventure. Analysis of idence about the capability of VR technology to allow the post session individual interviews with teens showed instructors and therapists to ofer safe, repeatable and that there was a high appreciation of the VR game along diversifiable learning environments for individuals with the dimensions of: engagement, clarity, interaction with ASC, a number of challenges are still to be addressed from objects, movement, user experience and willingness to both technological and clinical perspectives to prove its play again in the VR environment. Ratings on these di- full potential as a digital therapy and to address issues mensions on a 1-5 Likert scale (1=low, 5=high) measuring such as generalisation, and scalability beyond the proofappreciation, reached an average rating equal or above 4 of-concept studies. for all dimensions. As a matter of fact, all the feedbacks In this paper a description of these critical challenges and insights collected from the interviews with teens and is provided, together with the most promising research observations of their behaviors in the VR environment directions that could tackle them thanks to latest adduring the game have been used to improve the design vances in several fields, i.e. Artificial Intelligence and of the VR game even further. new generation VR headsets. Some of the identified chal
In XR4A, the data collected during the interactions lenges have been explored within the framework of an include a complete log of users movements and inter- EIT Digital project called XR4A whose aim is to develop actions in order to conduct a machine learning process a VR-based tele-therapy for individuals with ASC that for inferring preferences and behaviours for enabling a can easily scale while facilitating therapists to evaluate prediction of users intentions and emotions for a further its clinical efectiveness. design of more personalised experiences potentially increasing the user engagement and therapeutic outcomes. Acknowledgments XR4A is currently tested within a feasi-bility study and is intended to be validated for eficacy in a further clinical The authors wish to thank EIT Digital for the support trial. within XR4A, our project partners Reply (Italy), iED
The platform developed within the framework of (Greece), as well as “Cooperativa Albero Blu” (Italy) and XR4A project will be commercialized by a spinof of FBK their therapists for the collaboration in the design and (MEEVA Srl) whose aim is to provide Specialised Cen- evaluation phases. Finally, we would like to thank the ters with a tool ena-bling digital therapy toward pre- teens and their families who participated to the prelimiadolescents and teens with autism and other neurodevel- nary assessment. opmental disorders. tion technology? a comprehensive survey, Artificial [19] M. R. Ali, Z. Razavi, A. A. Mamun, R. Langevin, Intelligence Review 53 (2020) 1039–1069. B. Kane, R. Rawassizadeh, L. Schubert, M. E. Hoque, [6] J. F. Herrero, G. Lorenzo, An immersive virtual real- A virtual conversational agent for teens with ity educational intervention on people with autism autism: experimental results and design lessons, spectrum disorders (asd) for the development of arXiv preprint arXiv:1811.03046 (2018). communication skills and problem solving, Educa- [20] M. Maskey, J. Rodgers, V. Grahame, M. Glod, tion and Information Technologies 25 (2020) 1689– E. Honey, J. Kinnear, M. Labus, J. Milne, D. Minos, 1722. H. McConachie, et al., A randomised controlled fea[7] L. Malinverni, J. Mora-Guiard, V. Padillo, L. Valero, sibility trial of immersive virtual reality treatment A. Hervás, N. Pares, An inclusive design ap- with cognitive behaviour therapy for specific phoproach for developing video games for children bias in young people with autism spectrum disorder, with autism spectrum disorder, Computers in Hu- Journal of autism and developmental disorders 49 man Behavior 71 (2017) 535–549. (2019) 1912–1927. [8] L. Adams, E. Simonof, K. Tierney, M. J. Hollocks, [21] K.-F. Kollias, C. K. Syriopoulou-Delli, P. SarigianniA. Brewster, J. Watson, L. Valmaggia, Developing a dis, G. F. Fragulis, The contribution of machine user-informed intervention study of a virtual reality learning and eye-tracking technology in autism therapy for social anxiety in autistic adolescents, spectrum disorder research: A review study, in: Design for Health 6 (2022) 114–133. 2021 10th International Conference on Modern Cir[9] F. Ke, J. Moon, Z. Sokolikj, Virtual reality–based cuits and Systems Technologies (MOCAST), IEEE, social skills training for children with autism spec- 2021, pp. 1–4. trum disorder, Journal of Special Education Tech- [22] XR4A project website, https://xr4a.eu/, 2022. Onnology 37 (2022) 49–62. line; accessed August 2022. [10] F. Bhuyan, S. Lu, I. Ahmed, J. Zhang, Predicting [23] Unity website, https://unity.com/, 2023. Online; aceficacy of therapeutic services for autism spectrum cessed August 2022. disorder using scientific workflows, in: 2017 IEEE [24] Photon Engine website, https://www.photonengine. International Conference on Big Data (Big Data), com/en-US/Photon, ???? Online; accessed August IEEE, 2017, pp. 3847–3856. 2022. [11] C. Wee, K. M. Yap, W. N. Lim, Haptic interfaces for [25] Reply, Ticuro homepage, https://www.reply.com/ virtual reality: Challenges and research directions, ticuro-reply/en/, 2023. Online; accessed September IEEE Access 9 (2021) 112145–112162. 2022. [12] Meta Oculus, <eta Quest Pro, https://www.meta.
com/it/en/quest/quest-pro/, 2023. Online; accessed
November 2022. [13] Tobii website, Pico Neo 3 Pro Eye, https://www.
tobii.com/products/integration/xr-headsets/ device-integrations/pico-neo-3-pro-eye, 2022.
Online; accessed November 2022. [14] NeuroSpec AG, DSI-VR300 homepage, https://www.
neurospec.com/Products/Details/1077/dsi-vr300, 2023. Online; accessed November 2022. [15] A. Dzedzickis, A. Kaklauskas, V. Bucinskas, Human emotion recognition: Review of sensors and methods, Sensors 20 (2020) 592. [16] Y. Li, E. M. Sokhadze, H. Luo, A. S. El-Baz, A. S.
Elmaghraby, Afective virtual reality gaming for autism, Modern Approaches to Augmentation of
Brain Function (2021) 575–606. [17] A. A. Abd-Alrazaq, M. Alajlani, A. A. Alalwan, B. M.
Bewick, P. Gardner, M. Househ, An overview of the features of chatbots in mental health: A scoping review, International Journal of Medical Informatics 132 (2019) 103978. [18] E. Ferrari, Artificial intelligence for autism spectrum disorders, in: Artificial Intelligence in Medicine, Springer, 2022, pp. 1579–1593.