Drama therapy is a form of group psychotherapy that uses improvised drama to help people learn how to interact with others. However, drama therapy faces issues such as a shortage of therapists, feeling of embarrassment about acting, and restrictions on venues. For this reason, we aim to develop XR-based drama therapy (XRDT) using an AI therapist in XR spaces. First, we examined the content of XRDT sessions in a self-created world in VRChat. From the content analysis of the questionnaires and interviews, we observed an improvement in cognitive empathy, where the participants softened their own ideas by considering others, and the efectiveness of the XRDT sessions was demonstrated.
1. Introduction 1.1. Background methods, and to break away from the situation where problems are temporarily and partially solved by consulting with people close to them, a tendency that stems from the Japanese temperament [2]. We have therefore focused on drama therapy, which promotes psychological transformation.
In recent years, lifestyles and values have become more diverse. The way people interact with others has also changed significantly, along with shifts in how we perceive social identity and a sense of community. In such a society, people are making eforts to understand oth- 1.2. What is Drama Therapy ers. However, many people also feel anxiety, worry, and Drama therapy is a form of group psychotherapy and exstress about their social lives. According to a survey by pressive arts therapy that was derived from psychodrama, the Japanese Ministry of Health, Labour and Welfare, which was proposed by Jacob Moreno in the 1920s, and more than 50% of Japanese workers live with stress. Of its framework was established in the UK and US in the these, 70% consult their family, friends, superiors, or col- 1970s [3]. Participants become scriptwriters, actors, and leagues about their problems. In contrast, less than 10% spectators in drama therapy and achieve psychological of people choose to consult a specialist such as an occupa- transformation through the power of improvisation [4]. tional physician or a telephone counseling service [1]. In Psychological transformation cannot be achieved unless other words, there is a tendency in Japan to try to solve one’s thinking is flexible, and this requires understanding problems and stress through self-help eforts, without and empathy for others. In drama therapy, participants relying on specialists. Moreover, many of these eforts do cultivate the ability to imagine the perspectives and feelnot lead to a real solution to the problem. Given this back- ings of others (cognitive empathy) by repeatedly experiground, it is necessary to appeal to society at large for a encing role-taking (perspective-taking) and movement new approach that enables individuals to solve problems imitation (mirroring) using fairy tales and familiar topics in a fundamental way using concrete and quantitative as material (Figure 1). In our previous research, we analyzed the experiences of alcoholics undergoing drama therapy and obtained results that allowed us to qualitatively evaluate the efectiveness of drama therapy [5].
I really want to be gentle
Drama Therapy: One scene from Cinderella
Bullying is so painful...
I'm bored, so I'll bully someone
I... Maybe it's lonely? Others do not understand me I don't understand the feelings of others In Real Life: Before Drama Therapy
Experiencing the pain of bullying
Stand on the side of the bully and think about why they are being mean
Energized and cheerful
Be kind
to others
In Real Life: After Drama Therapy
explore a highly versatile new therapeutic format. Thus, Issue 1 Solution 1
we will identify the current challenges in drama therapy Only 5 qualified drama
and propose XR-based solutions. To popularize drama therapists in Japan
therapy in Japan, three main issues need to be addressed:
(
switching are used to bring about a sense of identificaFigure 3: Expected psychological transformation from XRDT tion and other efects that are particularly pronounced when performed by multiple people. Therefore, assuming the minimum configuration, the members would be: 2.2. The potential of XRDT one user, one AI therapist, and three other participants (NPCs). The XRDT consists of three sessions: Warm-up, XRDT is designed to be used in conjunction with VR Main Work, and Cool-down. Each session must be detechnology to enable home training in real life. There- signed to make the user feel the benefits of XRDT more fore, we can speculate on the advantages of XRDT, which strongly, as shown in Figure 3. Furthermore, XRDT is not cannot be obtained in physical space, as shown in Fig- just about being better than real-space therapy. It aims ure 3. There are many research cases of the Proteus efect, to be a system that integrates cognitive science, machine which states that the appearance of an avatar in virtual learning, wearable computing, and other technologies space afects the user’s behavioral characteristics and into psychotherapy. The progressive nature of XRDT cognition [11], and it is expected that XRDT will also lies in its aim to improve human-computer interaction improve self-eficacy through the Proteus efect. In addi- through the recognition and generation of non-verbal tion, Kinoshita discusses the theatricality of cyberspace, information. Paralinguistic information, such as fillers, arguing that what actors, game players, and spectators and physical behavior, such as pauses and hand gestures, have in common is that they experience a transforma- are dificult to express in text. However, these account tion of their personality through identification with “oth- for more than 50% of human communication, including ers” [12]. Based on these findings, it is predicted that emotional messages [14]. In addition, non-verbal comthe Proteus efect will be observed in XRDT participants, munication is the most dificult aspect for actors, and who can freely view their own appearance and perfor- is the core of expressive power, a skill that captivates mance while participating in the session, and that their audiences. In other words, it can be said that people’s behavior in physical space will be activated. In addition, emotions are swayed by non-verbal information, and psyin recent years, the number of educational sites operated chological transformation is promoted. In recent years, in virtual space has increased, and phenomena such as afective computing has focused on developing systems increased student activity compared to real schools have that work with emotional application technology and been observed [13]. This indicates that the Pygmalion social behavioral science [15]. In addition, mental health efect, in which performance improves due to the expec- applications that provide cognitive behavioral therapy tations of others, is likely to be more apparent in virtual through conversations with users and promote the allevispace. In XRDT, non-player characters (NPCs) positioned ation of depressive symptoms [16] and therapy robots for as spectators, providing reactions similar to live specta- the care of the elderly [17] have been put into practical tors are a positive presence, as they laugh and express use. In XRDT, we aim to introduce these technologies their agreement with the participants’ performance, so by focusing on pauses first among the many non-verbal it is expected that the Pygmalion efect will occur in the information. participants. Furthermore, in XRDT, participants can experience processes such as Unlearn (zeroing out learning) and Relearn (relearning), which encourage self-growth 3.2. Research Procedure through adaptation to change. We expect that these psy- In XRDT, it is necessary to create an environment in chological efects and processes inherent in XRDT will which conversations are held with an AI therapist. In cause psychological transformation in participants by this study, we divided the research into three stages and acquiring perspective-taking and improving cognitive proceeded incrementally. First, as the first stage, we empathy. examined the content of the sessions that could be held in a virtual space based on drama therapy theory (Phase 1). Next, we created the AI therapist and learned how to have conversations with it (Phase 2). Then, non-verbal information is obtained and given to the AI therapist (Phase 3). In this paper, we report on our examination of the content of XRDT sessions, which is Phase 1 of the three phases of this research.
downloading the HomeKit (VRChat SDKTemplate) materials from the Unity Asset Store and processing them in Unity. VRChat is a platform that allows users to freely interact with avatars online. The head-mounted display (HMD) used by the participants was the Meta Quest Pro from Oculus VR, while the research collaborators and the ifrst author used the Meta Quest2 from Oculus VR.
4.4.1. Warm-up
Three playful activities were conducted to help the
participants and research collaborators (hereafter referred
to as “participants”) relax and prepare for Main Work to
come. (
Four participants took part in the experiment, and one session was conducted for each participant. Prior to the experiment, each selected a friendly, informal name for use during the session to foster a relaxed, engaging atmo- 4.4.2. Main Work sphere. One session involved a total of five people: one participant, one facilitator (the first author), and three collaborators. There were four participants in total (two men and two women) in their 20s to 70s (Mean age = 40.5 years, SD = 15.69), all of whom were healthy and had experience receiving face-to-face drama therapy. The participants were also qualified as clinical psychologists or certified psychological counselors and had obtained a master’s degree in psychology. The three research collaborators participated in all sessions. The research collaborators had experience in stage acting and had also participated as research collaborators in the first author’s previous study [5]. Participants were informed of the research collaborators’ backgrounds beforehand. The experiment was conducted in a classroom at the university for the participants and the facilitator (first author), and at the homes of the research collaborators, who connected to VRChat from their respective locations.
The participants performed three drama works. (A) In a scene where they discussed the habit of watching SNS and impulsive online shopping, the participants experienced being caught between two research collaborators, who were divided into those who agreed and those who disagreed. (B) A scene of interpersonal trouble that could occur in everyday life was reenacted. First, two research collaborators acted as the person who could not follow the rules and the person who called them out. Next, the participant took on the role of the person who called them out, following the same scenario (Figure 4). In the same scene, the avatar of the person who could not follow the rules was changed to another avatar, and the scene was re-enacted. (C) The research collaborators improvised two scenes based on the role of the fairy tale character Cinderella. The participants did not speak, but instead overlaid their avatar on the role being played by the research collaborators, allowing them to experience various perspectives
4.4.3. Cool-down Cool-down allowed participants to refresh and reflect on their lives without being trapped in the roles they played in Main Work. First, they did the “mood color” exercise again, which they had done as Warm-up. Participants were able to become aware of changes in their mood by comparing the colors they chose before the session with the colors they chose afterward. The changes in the participants’ moods were recorded and used as a reference Participant Collaborator for future support systems. Reflection was a time for participants to express what they felt or wanted to convey, and the aim was to gain self-awareness by looking at oneself objectively. In this experiment, participants were encouraged to discuss their feelings, including reflections, in the semi-structured interview conducted after the session.
4.5.1. Questionnaire using the qualitative data analysis software MAXQDA. The coding process was repeated several times, with each coder working on the data on diferent days. The first author then compared the results of the three coders, classified the codes into themes, and conducted a content analysis. In addition, the research collaborators were interviewed using a semi-structured interview format on the day after the second session. The interview audio was recorded and transcribed to create a verbatim transcript.
such as responses to questionnaires by non-participants. Therefore, the experimental results are presented in the form of an analysis comparing responses to questionnaires by participants before and after the experiment, and a content analysis, which is a qualitative data analysis of semi-structured interviews.
rience in the VR environment, we conducted a pre- and Of the four participants, one showed symptoms of VR post-questionnaire survey using a questionnaire. The sickness, and one showed symptoms of eyestrain immequestionnaires used were the SSQ (Simulator Sickness diately after the experiment, but there were no problems Questionnaire) [18] for VR sickness, the VEQ (Virtual Em- with their health afterward. bodimentQuestionnaire) [19] for bodily sensation, and the IPQ (Igroup Presence Questionnaire) [20] for pres- 5.2. IOS-Empathy ence. In addition, two questionnaires were used to measure the degree of empathy for others: the Inclusion of We compared the responses to the questionnaires beOther in the Self (IOS) [21] and the Empathy scale. The fore and after the experiment for IOS and Empathy. As IOS scale evaluates the psychological distance between a result, there were clear changes in the responses of oneself and the target person, consisting of two ques- all four participants on several items in the Empathy tions. Empathy is a seven-point Likert scale that asks questionnaire. For this reason, we conducted a separate about the degree of empathy felt for the target person, interview to determine the reasons for the changes (Taand consists of seven questions for each of Main Work A, ble 1). For example, in response to the question “What B, and C, for a total of 21 questions. These scales were do you think of people who don’t follow the rules?” a translated into Japanese by Sawada et al. [22] based on participant who had answered “Not care” before the exthe English questionnaire used in a previous study [23], periment, answered “Concerned” after the experiment. and the items were modified for use in this experiment. The reason given for this change was that “I became less indiferent and started to worry about them”. Addition4.5.2. Semi-structured Interview ally, other participants who had previously responded with “Anxious, Restless” changed their answer to “Not The participants were interviewed in a semi-structured care” after the experiment. This participant explained, interview after the experiment. Based on questions about “I began to think more deeply about their situation, and the diferences between face-to-face drama therapy and my feelings shifted from wanting to interfere to simply XRDT, and the suitability of VR environments and drama watching over them.” and, “Since they have found their therapy, the interview was conducted flexibly, with ad- place, there may be new developments from there, so I ditional questions asked to further explore the partici- no longer feel sorry for them”. It is thought that these pants’ thoughts and feelings. A verbatim transcript of changes are due to improved cognitive empathy, which the recorded and transcribed interview audio was pre- leads to psychological transformation. pared as text data. A team of three individuals, consisting of two analysts and the first author, coded the text data 5.3. Semi-structured Interview evoked, and enjoyed acting out their roles with boldness. This is probably because, when the participants took From the content analysis of the semi-structured inter- part in face-to-face drama therapy, they had experienced views, we obtained the results that the XRDT session is refraining from acting as they tried to guess the other expected to improve perspective-taking and cognitive person’s facial expressions, or hesitated while searching empathy and suppress feelings of shame (Figure 5). for the right answer. The XRDT session showed that the participants’ resistance to acting was removed. It could 5.3.1. Theme Generation and Code Classification be said that “psychological liberation” is based on “psyFrom the interviews of the four participants, Three ana- chological protection”. By wearing the mask of an avatar lysts generated 111 codes during the analysis. The first and protecting their true selves, the participants could author derived 13 themes from the 111 codes, and classi- suppress their feelings of shame and act boldly. Drama ifed each code by theme (Table 2). Then, the percentage therapy in this kind of psychological state becomes a of text data for the narratives to which the codes were meaningful session where participants can experience assigned was calculated using the qualitative data anal- the release of various emotions and experience the drama ysis software MAXQDA, and the extent to which each of diverse interpersonal relationships. theme was discussed by each participant was quantified (Table 3). 5.3.3. Physical Advantages of XRDT
5.3.2. Psychological Benefits of XRDT point position” and “sense of presence”. These can be said The most common thing the participants talked about to be the characteristics of VR. The act of playing a role was “psychological liberation”. The participants identi- is, in a sense, an action that is far removed from everyday ifed with the character that their avatar’s appearance life, and VR, which immerses you in a diferent world by isolating you from reality with a head-mounted display, Can become one with the character from a first-person perspective VR sickness Cinderella is modern and realistic Wanted a more talk-focused session Can move around and control body movements Reflected voluntarily based on awareness Did not feel nervous No eye contact, gestures, breathing Become a child’s perspective when the character is a child World view Empathy, where an individual identifies with a character’s emotions Tension Whether the emotional warmth experienced this time can be felt again Theme Psychological liberation Anxiety about VR devices Subjects that are easy to imagine Examination of Conversation formats Interest in VR Empathy Psychological protection Dynamic constraints Change in viewpoint position Playful experience Mirroring efect Sense of presence Anxiety about AI therapist is a good match for this. And, because each role has a unique viewpoint (height) that can be easily adjusted by switching avatars, the VR environment is particularly suitable for drama therapy. In addition, since it was the ifrst VR experience for most of the participants, they talked a lot about their “interest in VR”, such as how they enjoyed the virtual space and the avatar controls. 5.3.4. Drama Therapy Characteristics Drama therapy is an expressive arts therapy that aims to cultivate the ability to imagine the perspectives and feelings of others (cognitive empathy) through repeated experiences of role-taking (perspective-taking) and movement imitation (mirroring). We would like to draw attention to the results of the participants who had experienced face-to-face drama therapy, who said that they felt the drama therapy characteristics of “empathy”, “playful experience” and “mirroring efect” in XRDT. 5.3.5. Subject of the Session
this experiment and the order in which it was presented. The subjects were based on recent internet trends and everyday situations that might occur in social interactions with neighbors, and it was demonstrated that they were accepted regardless of the age or gender of the subjects. 5.3.6. Issues for Consideration
they only accounted for a small proportion of the overall
interview, we have classified them into four themes that
should be considered or examined. (
There is a possibility that the problem occurred due to the low battery charge of the Head Mounted Display, and we will pay even closer attention in the future to avoid risks such as distortion of the virtual space and falls in the physical space. (4) “Anxiety about the AI therapist”: In the work that included self-reflection during Cool-down period, the participants said that they felt the emotional warmth of the participants, and they said that they could not imagine whether an AI therapist or NPC could create the future, we will proceed with Phase 2: creating an AI such conversations and actions. This aspect will be a therapist and learning conversation, and Phase 3: acquirfocus of detailed investigation in future research. ing non-verbal information and providing it to the AI therapist, while considering the issues identified in Phase 5.3.7. Feedback from Research Collaborators 1. First, we will select a ready-made avatar that meets the conditions for XRDT. Then, using natural language The research collaborators shared their views on the ex- processing technology (NLP) and machine learning algoperiment based on their own experience of stage drama. rithms from the OpenAI API, we will embed conversation The research collaborators said that while face-to-face functions into the avatar. These will be created in the drama therapy can be stressful and taxing, it can also metaverse cluster provided by Cluster, Inc., with whom lead to powerful realizations and significant healing. Still, we are conducting joint research, using the technology achieving the same level of realization and healing in a provided by Cluster. Then, we will evaluate the AI theravirtual space may be dificult. For example, if you par- pist’s movement and conversation ability (Phase 2). Next, ticipate from home, you can go back to reality and do we will extract the characteristics of non-verbal informachores or get ready to go out right after the session is tion. We are focusing on pauses in particular among nonover. This ease of access, however, means that there is verbal information. Using data sets such as the Corpus no lingering efect, and you miss the chance to reflect of Everyday Japanese Conversation (CEJC) [24] and the on what you have experienced. However, the research Basic Transcription System for Japanese (BTSJ) [25], we collaborators said that, because they were not participat- will analyze pauses between conversations from speech ing in the sessions to gain something from XRDT, their detection (Phase 3). Following the coronavirus pandemic, motivation after the experiment would difer from that there is a need to create rules for the rapid increase in of the participants. The diference in sensitivity between online counseling (media literacy, ensuring a quiet and the participants and the research collaborators was also appropriate place, prohibition of unauthorized recording, evident in the change in tone of voice towards the other etc.) [26]. Therefore, from now on, this research will be avatar in Main Work B. When the research collaborators conducted in parallel with empirical research and ethical performed Main Work B as a trial, the reaction of all verification from the initial stage of development. the participants to the avatar whose tone of voice had changed was one of derision and teasing. This suggests that when the operator of the avatar is known, the con- Acknowledgments versation impression is less influenced by the avatar’s appearance. However, when the operator is unknown, the impression formed is largely shaped by the avatar’s appearance, including its voice. 5.3.8. Conclusion The purpose of this experiment was to verify whether the content of the sessions for XRDT that we created had the efect of promoting psychological transformation, such as the acquisition of perspective-taking and the improvement of cognitive empathy. Based on the analysis of the questionnaires and interviews, we can say that these efects are highly promising. We also found that it is possible to generate elements in the virtual space that accelerate psychological transformation, such as the emotional warmth that the participants felt from the research collaborators. However, whether the AI therapist and NPC of the XRDT we aim for can create emotional warmth is an issue for Phase 2 and Phase 3 of this research.
We would like to thank the psychology experts who participated as subjects and coders.
In addition, Professor Sylvia Shueni Pang of the University of London and Assistant Professor Yuji Hatada of the University of Tokyo provided us with multifaceted advice. We are grateful to them. [3] R. Emunah, Acting for real: Drama therapy process, (2003) 55–64.
technique, and performance, Routledge, 2019. [16] A. Darcy, et al., Evidence of human-level bonds es[4] D. R. Johnson, The theory and technique of transfor- tablished with a digital conversational agent: crossmations in drama therapy., The arts in psychother- sectional, retrospective observational study, JMIR apy 18(4) (1991) 285–300. Formative Research 5 (2021) e27868. [5] Y. Matsuda, [art-based research on life transforma- [17] T. Shibata, K. Wada, [clinical and empirical study tion through drama therapy - what ethnotheatre on the efects of robot therapy using the seal robot weaves] dorama sera¯p¯ı ni yoru jinsei hencho¯ no a¯to "paro"] azarashigata robotto "paro" ni yoru robotto be¯su risa¯ch¯ı-esuno shiata¯ ga tsumugu mono- (in serap¯ı no ko¯ka no rinsho¯ jissho¯ jikken ni tsuite (in japanese), Ritsumeikan University Master’s Thesis japanese), Journal of the Robotics Society of Japan Collection (2022). 29 (2011) 246–249. [6] G. Riva, Virtual reality in psychotherapy, Cyberpsy- [18] R. S. Kennedy, et al., Simulator sickness questionchology & behavior 8 (2005) 220–230. naire: An enhanced method for quantifying simula[7] S. A. Nijman, et al., Dynamic Interactive Social tor sickness, The international journal of aviation Cognition Training in Virtual Reality (DiSCoVR) psychology 3 (1993) 203–220. for social cognition and social functioning in peo- [19] D. Roth, M. E. Latoschik, Construction of the virple with a psychotic disorder: study protocol for tual embodiment questionnaire (veq), IEEE Transa multicenter randomized controlled trial, BMC actions on Visualization and Computer Graphics Psychiatry 19 (2019) 1–11. 26 (2020) 3546–3556. [8] A. Atsmon, S. Pendzik, The clinical use of digital [20] J. Vasconcelos-Raposo, et al., Adaptation and validaresources in drama therapy: An exploratory study tion of the Igroup Presence Questionnaire (IPQ) in of well-established practitioners, Drama Therapy a Portuguese sample, Presence 25 (2016) 191–203.
Review 6 (2020) 7–26. [21] A. Aron, et al., Inclusion of other in the self scale [9] J. Park, et al., AEDLE: Designing Drama Therapy and the structure of interpersonal closeness., JourInterface for Improving Pragmatic Language Skills nal of personality and social psychology 63 (1992) of Children with Autism Spectrum Disorder Using 596.
AR, in: Extended Abstracts of the 2023 CHI Con- [22] R. Sawada, et al., [verification of the transferability ference on Human Factors in Computing Systems, of empathy formed by vrpt to specific groups to out2023, pp. 1–7. groups] vrpt ni yori keisei sareta tokutei shu¯dan [10] S. F. Jonaghani, VR Drama Therapy, Master thesis, e no kyo¯kan no shu¯dan-gai e no ten’i kanosei no
University of Toronto, 2023. kensho¯ (in japanese), Proceedings of the Japan [11] N. Yee, J. Bailenson, The Proteus efect: The ef- Virtual Reality Society Conference 27 (2022) 3C1–4. fect of transformed self-representation on behavior, [23] M. Mado, et al., Efect of virtual reality perspectiveHuman communication research 33 (2007) 271–290. taking on related and unrelated contexts, Cyberpsy[12] K. Kinoshita, [toward a theoretical construction chology, Behavior, and Social Networking 24 (2021) of "participant": A preliminary discussion on the 839–845. applicability of the proteus efect to theater and film [24] H. Koiso, et al., Corpus of Everyday Japanese Constudies] "sanka-sha" no riron ko¯chiku ni mukete – versation, CEJC, in: The 96st Meeting of Special puroteusu ko¯ka no engeki eiga kenkyu¯ e no o¯yo¯ Interest Group on Spoken Language Understandkanosei ni tsuite no shiron– (in japanese), Theater ing and Dialogue Processing (SLUD), The Japanese Studies Collection, Bulletin of the Japan Theater Society for Artificial Intelligence (2022.12), The Association 70 (2020) 71–90. Japanese Society for Artificial Intelligence, 2022, [13] H. Saito, [the new way of thinking about schools p. 31.
and the shape of education that i felt when i tried [25] M. Usami, Basic Transcription System for Japanese: to create a school in the metaverse] meta ba-su BTSJ, 2019. ni gakkou wo tuku xtu te mi te kan zi ta atara [26] E. Ishikawa, [current status and challenges of online sii gakkou no a ri kata to kyouiku no katati (in counseling] onrain kaunsenringu no genjo¯ to kadai japanese), https://japan.cnet.com/article/35207142/, (in japanese), Bulletin of Kodomo Education Hosen 2023. University 12 (2021) 53–59. [14] M. L. Knapp, J. A. Hall, T. G. Horgan, Nonverbal Communication in Human Interaction, Cengage Learning, 2013. Google-Books-ID: rWoWAAAAQBAJ. [15] R. W. Picard, Afective computing: challenges, International Journal of Human-Computer Studies 59