E-mpathy and the Phantom Limb Sensation: a multisensory experience for embodiment of amputation

Abstract

In the context of promoting empathy among people without disabilities, we propose an application to allow users to experience having an amputated arm. By providing both visual and haptic feedback, our application offers a multisensory experience to enhance the sense of embodiment. The user of our application should still feel their real limb attached to their bodies, and yet see their virtual avatar and interact with the virtual environment as an amputee. A simple task of handling and positioning objects in a table is proposed for users to experience the difficulties of having a missing arm. Additionally, experiment participants are asked to answer a self-presence questionnaire regarding their embodiment of the virtual avatar.

1 INTRODUCTION

Phantom Limb Sensation (PLS) is the sensation that an amputated or missing limb is still attached to one’s body after a traumatic injury [ 11 ]. It is a common experience among amputees [ 14 ], and it is often painful (Phantom Limb Pain). In this paper, we propose e-mpathy: an immersive experience aiming at providing to users without an amputation some of the sensations felt by individuals with Copyright © 2020 for this paper by its authors. Use permitted under Creative Commons License Attribution 4.0 International (CC BY 4.0). a missing limb, including PLS. Our goal is that such an experience will help to raise both empathy and awareness through embodiment.

Using a hand tracker and a VR headset, users will be able to see their arms in a VR environment. In their perspective, the users’ left hand will appear whole, while part of their right forearm will be virtually amputated. In this sense, the proprioception of the nonexisting hand will be maintained, as the user still feels their own hand, despite not seeing it. The touch feeling of the right hand will be omitted when interacting with objects, though. As the user tries to touch virtual objects, the right hand should pass through them, and a collision should only render touch on the region of the amputation (see Figure 3). This touch is rendered using a haptic device placed between the arm and the forearm, as in Figure 2. 2 To develop e-mpathy, we have considered how virtual environments can support the embodiment of difference, how amputation has been approached in virtual reality applications, and to enhance such an experience of embodiment through multisensory stimulation.

Embodiment of Difference: Immersive virtual environments can provide vivid sensorimotor cues that make the embodied experience more tangible than imagination [ 1 ]. By changing self-representation in Virtual Reality (VR), one can influence the user’s perception and behaviors without having to experience the situation in the real world. Therefore, similar applications has been built to allow the embodiment of different characteristics and conditions, such as a different sex/gender [ 16 ], nationality [ 7 ], physical stature [ 15 ], mental illness [ 12 ], and disabilities [ 3 ].

One of these works is a VR experience created by the expressionist artist Marcel Schreur [ 3 ]. Such an experience explores Schreur’s unique life experience and his approach to art practice as a thirtyyear oral cancer and seven-year vascular dementia survivor. His work generates a space that allows participants to consider aspects of their own mental process through a lens of difference and disability, to embody some of the physical characteristics of Schreur’s condition. In this paper, we propose an experience with the same purpose of allowing empathy with the difference. Besides, we explore a different condition through the amputation.

Amputation in VR: Previous studies have already explored how the use of VR can help reducing the Phantom Limb Pain in patients with upper and lower limb amputation [ 2, 9 ]. Such studies usually provide patients with tasks in which they need to move the lost limb. In general, the studies show improvement in pain relief for most of the patients during the sessions, demonstrating that VR can be used as one of the treatment alternatives [ 2 ], especially for not being as invasive as medications and medical procedures.

Multisensory Stimulation: Studies have also shown that increasing the number of senses stimulated in a VR simulator can dramatically enhance a user’s sense of presence and their memory for the experience passed in VR [ 5,6 ]. In addition, multisensory stimulation is fundamental for the Sense of Embodiment (SOE) [ 8, 10 ].

The well known rubber hand illusion [ 8 ] demonstrates that after just a few seconds of synchronous visual-tactile stimulation, a participant will probably experience a profound illusion in which a rubber hand feels as if it were their real hand. Similarly, our application will provide both visual and haptic feedback to enhance the sense of presence and embodiment. 3

USE CASES

A VR application that allows for the embodiment of difference (e.g. experiencing amputation), provides a setup that can be used in multiple use cases, such as: • Embodiment Game: A player can do a task while embodying a different persona with other sets of abilities and physical characteristics. Tasks could be solved despite the portrait disability and avatars should be diverse to better represent reallife players. As impaired players can use avatars that do not match their real-life bodies, non-impaired players could have the opposite experience, allowing to share experiences and to understand each other’s limitations; • Social Studies: Such an application can also be used for empirical studies in which a person can be observed while portraying a different condition, or a tester can experience a different situation from a new perspective. Rather than only imagining, a virtual embodiment can support them to better understand and discuss a given topic or interest group; • Momentary Experience: An application can also provide a quick experience with no clear task of research purpose. The momentary experience should allow for a quick and accessible way of seeing the world through a different perspective, even if it is just a glimpse. The solution presented in this paper aims at providing this momentary experience. Kilteni et al. [ 8 ] report that the SOE consists of three subcomponents: the sense of self-location, the sense of agency, and the sense of body ownership. The last one emerges from a combination of top-down and bottom-up influences, including afferent sensory information (for example, visual, tactile, and proprioceptive input).

In order to support the SOE, our application setup consists of two main parts: Vision and Haptics. As shown in Figure 4, a virtual table will be rendered in front of the user and objects will appear on top of it. The user will visualize the VR environment using a VR headset (HTC Vive1). The headset provides a more immersive experience and allows the user to interact with the objects from an egocentric perspective.

Moreover, an optical hand tracking module (Leap Motion by Ultraleap2) will be attached to the VR headset as shown in Figure 2a. That allows users to interact with the environment using their own hands rather than controllers. The visualization of their virtual hands and direct control of their limbs in addition to the egocentric visualization were especially thought to enhance the sense of selflocation and agency. 4.2

Haptics

The user is able to touch and grab the virtual objects with their left hand, as shown in Fig. 3-a. As their virtual right arm will 1https://www.vive.com/us/product/vive-virtual-reality-system/ 2https://www.ultraleap.com/product/leap-motion-controller/ be amputated, the user can touch them only by sticking out their forearm, knocking or pushing objects, without a proper means to handle them (Fig. 3-b).

A force feedback device (EXOS Wrist DK2 by exiii Inc.3) is then used in the right forearm as to render forces related to the collisions with the virtual table and objects whenever the user tries to touch one of them (see Figure 2-b).

Finally, a real table is also set in front of the user’s left hand providing passive haptic feedback to the left limb. Likewise, real tracked objects will be atop the real table. However, no real table or objects is available to provide passive haptics to the right upper limb. All touch feedback to the right arm will be given by the actuation of the Exos device whenever the user touches the virtual table and objects with their partial right forearm. By matching proprioceptive, tactile, and visual feedback, we expect to enhance the sense of body ownership. 5

USER TASK

Regarding the user demonstration of our proposal, a simple task was designed and developed. Using only one arm at a time, the user is requested to move a colored object into a marker of the same color appearing in the table. Objects in the left side of the table will be real and have a tracked virtual version of them being rendered in the VR environment. Users will be able to hold these with their real left hands, feeling their actual shapes as well as see them in VR.

In the right side, however, objects will be purely virtual. The user will only be able to interact with them by pushing them around with the amputated arm, while receiving a haptic feedback whenever the objects are touched by the virtual right arm. 5.1

Prototype Demonstration and User Assessment

The project described in this work was accepted in the 3DUI contest4 at IEEE VR 20205, a high profile academic conference in the field [ 4 ]. In this event, the developed prototype was openly evaluated both by the academic community and by an official expert jury. The evaluation comprised of three criteria: novelty, provoked empathy and aesthetics.

Besides the official conference judgment of the prototype, a user assessment is meant to be taken place in a future work. Subjects will be asked to participate in a simple assessment of the experience they will have in the aforementioned task. Based on the work of Ratan [ 13 ], the Self-Presence Questionnaire (SPQ) will be used, which has the intention of measuring proto self-presence (body-schema) through questions about the integration of the user’s self-representation into a virtual environment. The SPQ is simply 3https://exiii.jp/wrist-dk2/ 4http://ieeevr.org/2020/contribute/3dui-contest.html 5http://ieeevr.org/2020/ comprised of five questions, all answerable in a 5 point Likert scale, namely: 1. When playing the game, how much do you feel like your avatar is an extension of your body within the game? 2. When playing the game, how much do you feel your avatar is a part of your body? 3. When using your avatar, to what extent do you feel like you can reach into the game through your avatar? 4. When using your avatar, to what extent do you feel like your right arm is elongated into the game through your avatar? 5. When playing the game, to what extent do you feel like your left hand is inside of the game? 6

FINAL COMMENTS

In this work, we proposed a multisensory VR application to support the embodiment of amputation and experiencing the Phantom Limb Sensation. By using haptic and visual elements we expect to enhance the senses of self-location, agency, and body ownership which are components of the Sense of Embodiment.

The virtual environment, the physical objects tracking and the visual and haptic cues were implemented in our experiment, as well as a simple object handling task and a self-presence questionnaire. As future work, further experimentation is required. To do so, we intend on developing a more refined set of user tasks in order to validate our design according to the real sensations and challenges experienced by amputees. Objects with different geometries should be added to the experiment, as to create a more diverse environment. A more profound usability assessment is also suggested in order to fully analyse the impact of this virtual experience.

ACKNOWLEDGMENTS

This study was partly funded by the Coordenac¸a˜o de Aperfeic¸oamento de Pessoal de N´ıvel Superior - Brasil (CAPES) - Finance Code 001, partly by CNPq and partly by the Swedish Council for Higher Education (UHR). We also acknowledge FAPERGS (project 17/2551-0001192-9), CNPq-Brazil (project 311353/2017-7) and a Linnaeus-Palme grant within the partnership (3.3.1.34.11116-2019) between the School of Information Technology at Halmstad University and the Institute of Informatics at UFRGS for their financial support.

References

[1] S. J. Ahn . Embodied experiences in immersive virtual environments: Effects on pro-environmental attitude and behavior . PhD thesis , Citeseer, 2011 .

[2] E. Ambron , A. Miller , K. J. Kuchenbecker , L. J. Buxbaum , and H. Coslett . Immersive low-cost virtual reality treatment for phantom limb pain: Evidence from two cases . Frontiers in neurology, 9 : 67 , 2018 .

[3] D. Bacchus . Life is beautiful. always: Using virtual reality to share the experience of disability . 2019 .

[4] A. Calepso , R. Guarese , R. Valer , Y. J. Iquiapaza Ccama , V. A. Oliveira , L. Nedel , and A. Maciel . 3dui and the phantom limb: multisensory experience for embodiment of amputation . In 2020 IEEE Conference on Virtual Reality and 3D User Interfaces Abstracts and Workshops (VRW) , pp. 517 - 518 , 2020 . doi: 10 .1109/VRW50115. 2020 .00107

[5] H. Q. Dinh , N. Walker , L. F. Hodges , C. Song , and A. Kobayashi . Evaluating the importance of multi-sensory input on memory and the sense of presence in virtual environments . In Proceedings IEEE Virtual Reality (Cat. No. 99CB36316) , pp. 222 - 228 . IEEE, 1999 .

[6] A. Gallace , M. K. Ngo , J. Sulaitis , and C. Spence . Multisensory presence in virtual reality: possibilities & limitations. In Multiple sensorial media advances and applications: New developments in MulSeMedia , pp. 1 - 38 . IGI Global, 2012 .

[7] V. Groom , J. N. Bailenson , and C. Nass . The influence of racial embodiment on racial bias in immersive virtual environments . Social Influence , 4 ( 3 ): 231 - 248 , 2009 .

[8] K. Kilteni , R. Groten , and M. Slater . The sense of embodiment in virtual reality . Presence: Teleoperators and Virtual Environments , 21 ( 4 ): 373 - 387 , 2012 .

[9] J. Kulkarni , S. Pettifer , S. Turner , and C. Richardson . An investigation into the effects of a virtual reality system on phantom limb pain: a pilot study . British Journal of Pain , p. 2049463719859913 , 2019 .

[10] D. Leonardis , A. Frisoli , M. Barsotti , M. Carrozzino , and M. Bergamasco . Multisensory feedback can enhance embodiment within an enriched virtual walking scenario . Presence: Teleoperators and Virtual Environments , 23 ( 3 ): 253 - 266 , 2014 .

[11] M. P. Z. Nejatkermany , E. Modirian, M. Soroush , M. Masoumi , and M. Hosseini . Phantom limb sensation (pls) and phantom limb pain (plp) among young landmine amputees . Iranian journal of child neurology , 10 ( 3 ): 42 , 2016 .

[12] D. L. Penn , J. D. Ivory , A. Judge , et al. The virtual doppelganger: Effects of a virtual reality simulator on perceptions of schizophrenia . The Journal of nervous and mental disease , 198 ( 6 ): 437 - 443 , 2010 .

[13] R. Ratan . Self-presence, explicated: Body, emotion, and identity extension into the virtual self . Handbook of Research on Technoself. New York, NY: IGI Global , 01 2012 .

[14] A. Woodhouse . Phantom limb sensation. Clinical and experimental pharmacology and physiology, 32 ( 1-2 ): 132 - 134 , 2005 .

[15] N. Yee and J. N. Bailenson . The proteus effect: Self transformations in virtual reality . Human Communication Research , 33 ( 3 ): 271 - 90 , 2007 .

[16] N. Yee , N. Ducheneaut , M. Yao , and L. Nelson . Do men heal more when in drag? conflicting identity cues between user and avatar . In Proceedings of the SIGCHI conference on Human factors in computing systems , pp. 773 - 776 , 2011 .