=Paper=
{{Paper
|id=Vol-2618/paper5
|storemode=property
|title=E-mpathy and the Phantom Limb Sensation: A Multisensory Experience for Embodiment of Amputation
|pdfUrl=https://ceur-ws.org/Vol-2618/paper5.pdf
|volume=Vol-2618
|authors=Renan Luigi Martins Guarese,Victor Adriel de Jesus Oliveira,Aimee Calepso,Rafael Valer,Yhonatan Iquiapaza,Luciana Nedel,Anderson Maciel
}}
==E-mpathy and the Phantom Limb Sensation: A Multisensory Experience for Embodiment of Amputation==
https://ceur-ws.org/Vol-2618/paper5.pdf
E-mpathy and the Phantom Limb Sensation:
a multisensory experience for embodiment of amputation
Renan Guarese*12 Victor de Jesus Oliveira†3 Aimee Calepso‡1 Rafael Valer§1
Yhonatan Iquiapaza¶1 Luciana Nedel||1 Anderson Maciel**1
1 Institute of Informatics, Federal University of Rio Grande do Sul, Porto Alegre, Brazil
2 Halmstad University, Halmstad, Sweden
3 St. Poelten University of Applied Sciences, St. Poelten, Austria
Figure 1: Our application allows users to experience having a missing arm and the Phantom Limb sensation. It provides both visual
and haptic feedback to enhance the sense of embodiment.
A BSTRACT a missing limb, including PLS. Our goal is that such an experience
will help to raise both empathy and awareness through embodiment.
In the context of promoting empathy among people without disabili-
ties, we propose an application to allow users to experience having Using a hand tracker and a VR headset, users will be able to see
an amputated arm. By providing both visual and haptic feedback, their arms in a VR environment. In their perspective, the users’ left
our application offers a multisensory experience to enhance the sense hand will appear whole, while part of their right forearm will be
of embodiment. The user of our application should still feel their virtually amputated. In this sense, the proprioception of the non-
real limb attached to their bodies, and yet see their virtual avatar existing hand will be maintained, as the user still feels their own
and interact with the virtual environment as an amputee. A simple hand, despite not seeing it. The touch feeling of the right hand will
task of handling and positioning objects in a table is proposed for be omitted when interacting with objects, though. As the user tries to
users to experience the difficulties of having a missing arm. Addi- touch virtual objects, the right hand should pass through them, and a
tionally, experiment participants are asked to answer a self-presence collision should only render touch on the region of the amputation
questionnaire regarding their embodiment of the virtual avatar. (see Figure 3). This touch is rendered using a haptic device placed
between the arm and the forearm, as in Figure 2.
Index Terms: Human-centered computing—Virtual reality;
Human-centered computing—Haptic devices 2 R ELATED WORK
To develop e-mpathy, we have considered how virtual environments
1 I NTRODUCTION can support the embodiment of difference, how amputation has been
Phantom Limb Sensation (PLS) is the sensation that an amputated approached in virtual reality applications, and to enhance such an
or missing limb is still attached to one’s body after a traumatic experience of embodiment through multisensory stimulation.
injury [11]. It is a common experience among amputees [14], and Embodiment of Difference: Immersive virtual environments can
it is often painful (Phantom Limb Pain). In this paper, we propose provide vivid sensorimotor cues that make the embodied experience
e-mpathy: an immersive experience aiming at providing to users more tangible than imagination [1]. By changing self-representation
without an amputation some of the sensations felt by individuals with in Virtual Reality (VR), one can influence the user’s perception
and behaviors without having to experience the situation in the
* e-mail: rlmguarese@inf.ufrgs.br real world. Therefore, similar applications has been built to allow
† e-mail: victor.oliveira@fhstp.ac.at the embodiment of different characteristics and conditions, such as
‡ e-mail: ascalepso@inf.ufrgs.br a different sex/gender [16], nationality [7], physical stature [15],
§ e-mail: rvaler@inf.ufrgs.br mental illness [12], and disabilities [3].
¶ e-mail: yjiccama@inf.ufrgs.br One of these works is a VR experience created by the expression-
|| e-mail: nedel@inf.ufrgs.br ist artist Marcel Schreur [3]. Such an experience explores Schreur’s
** e-mail: amaciel@inf.ufrgs.br unique life experience and his approach to art practice as a thirty-
year oral cancer and seven-year vascular dementia survivor. His
work generates a space that allows participants to consider aspects
Copyright © 2020 for this paper by its authors. Use permitted under of their own mental process through a lens of difference and dis-
Creative Commons License Attribution 4.0 International (CC BY 4.0). ability, to embody some of the physical characteristics of Schreur’s
Figure 2: The application setup consists of two main parts: Vision and Haptics. (a) For the visual component, a motion tracker is attached to the
VR headset to allow both visualization and control; (b) For the haptic component, a force feedback device is mounted on the user’s arm to convey
information about collisions with the virtual objects.
condition. In this paper, we propose an experience with the same pur- 4 M ATERIALS AND M ETHODS
pose of allowing empathy with the difference. Besides, we explore Kilteni et al. [8] report that the SOE consists of three subcomponents:
a different condition through the amputation. the sense of self-location, the sense of agency, and the sense of body
Amputation in VR: Previous studies have already explored how ownership. The last one emerges from a combination of top-down
the use of VR can help reducing the Phantom Limb Pain in patients and bottom-up influences, including afferent sensory information
with upper and lower limb amputation [2, 9]. Such studies usually (for example, visual, tactile, and proprioceptive input).
provide patients with tasks in which they need to move the lost limb. In order to support the SOE, our application setup consists of two
In general, the studies show improvement in pain relief for most of main parts: Vision and Haptics.
the patients during the sessions, demonstrating that VR can be used
as one of the treatment alternatives [2], especially for not being as
invasive as medications and medical procedures.
Multisensory Stimulation: Studies have also shown that increas-
ing the number of senses stimulated in a VR simulator can dramati-
cally enhance a user’s sense of presence and their memory for the
experience passed in VR [5,6]. In addition, multisensory stimulation
is fundamental for the Sense of Embodiment (SOE) [8, 10].
The well known rubber hand illusion [8] demonstrates that after
just a few seconds of synchronous visual-tactile stimulation, a partic-
ipant will probably experience a profound illusion in which a rubber
hand feels as if it were their real hand. Similarly, our application
will provide both visual and haptic feedback to enhance the sense of
presence and embodiment.
3 U SE C ASES
A VR application that allows for the embodiment of difference
(e.g. experiencing amputation), provides a setup that can be used in Figure 3: Players are able to interact with the environment using
their upper limbs. When touching an object, players should feel the
multiple use cases, such as:
feedback (a) on their hand when using their left arm, and (b) on the
area of the amputation when using their right arm.
• Embodiment Game: A player can do a task while embodying
a different persona with other sets of abilities and physical
characteristics. Tasks could be solved despite the portrait 4.1 Vision
disability and avatars should be diverse to better represent real- As shown in Figure 4, a virtual table will be rendered in front of
life players. As impaired players can use avatars that do not the user and objects will appear on top of it. The user will visualize
match their real-life bodies, non-impaired players could have the VR environment using a VR headset (HTC Vive1 ). The headset
the opposite experience, allowing to share experiences and to provides a more immersive experience and allows the user to interact
understand each other’s limitations; with the objects from an egocentric perspective.
Moreover, an optical hand tracking module (Leap Motion by
• Social Studies: Such an application can also be used for empir- Ultraleap2 ) will be attached to the VR headset as shown in Figure 2-
ical studies in which a person can be observed while portraying a. That allows users to interact with the environment using their
a different condition, or a tester can experience a different situ- own hands rather than controllers. The visualization of their virtual
ation from a new perspective. Rather than only imagining, a hands and direct control of their limbs in addition to the egocentric
virtual embodiment can support them to better understand and visualization were especially thought to enhance the sense of self-
discuss a given topic or interest group; location and agency.
• Momentary Experience: An application can also provide a 4.2 Haptics
quick experience with no clear task of research purpose. The The user is able to touch and grab the virtual objects with their
momentary experience should allow for a quick and accessible left hand, as shown in Fig. 3-a. As their virtual right arm will
way of seeing the world through a different perspective, even
if it is just a glimpse. The solution presented in this paper aims 1 https://www.vive.com/us/product/vive-virtual-reality-system/
at providing this momentary experience. 2 https://www.ultraleap.com/product/leap-motion-controller/
Figure 4: User testing the application, with the task objects in the background. Left - Frame of the application, both virtual arms can be seen at
once. Right - User can be seen looking at their virtually-amputated right arm.
be amputated, the user can touch them only by sticking out their comprised of five questions, all answerable in a 5 point Likert scale,
forearm, knocking or pushing objects, without a proper means to namely:
handle them (Fig. 3-b).
A force feedback device (EXOS Wrist DK2 by exiii Inc.3 ) is then 1. When playing the game, how much do you feel like your avatar
used in the right forearm as to render forces related to the collisions is an extension of your body within the game?
with the virtual table and objects whenever the user tries to touch
one of them (see Figure 2-b). 2. When playing the game, how much do you feel your avatar is
Finally, a real table is also set in front of the user’s left hand a part of your body?
providing passive haptic feedback to the left limb. Likewise, real
tracked objects will be atop the real table. However, no real table or 3. When using your avatar, to what extent do you feel like you
objects is available to provide passive haptics to the right upper limb. can reach into the game through your avatar?
All touch feedback to the right arm will be given by the actuation
of the Exos device whenever the user touches the virtual table and 4. When using your avatar, to what extent do you feel like your
objects with their partial right forearm. By matching proprioceptive, right arm is elongated into the game through your avatar?
tactile, and visual feedback, we expect to enhance the sense of body
5. When playing the game, to what extent do you feel like your
ownership.
left hand is inside of the game?
5 U SER TASK 6 F INAL C OMMENTS
Regarding the user demonstration of our proposal, a simple task was In this work, we proposed a multisensory VR application to support
designed and developed. Using only one arm at a time, the user is the embodiment of amputation and experiencing the Phantom Limb
requested to move a colored object into a marker of the same color Sensation. By using haptic and visual elements we expect to enhance
appearing in the table. Objects in the left side of the table will be the senses of self-location, agency, and body ownership which are
real and have a tracked virtual version of them being rendered in the components of the Sense of Embodiment.
VR environment. Users will be able to hold these with their real left The virtual environment, the physical objects tracking and the
hands, feeling their actual shapes as well as see them in VR. visual and haptic cues were implemented in our experiment, as well
In the right side, however, objects will be purely virtual. The user as a simple object handling task and a self-presence questionnaire.
will only be able to interact with them by pushing them around with As future work, further experimentation is required. To do so, we
the amputated arm, while receiving a haptic feedback whenever the intend on developing a more refined set of user tasks in order to
objects are touched by the virtual right arm. validate our design according to the real sensations and challenges
experienced by amputees. Objects with different geometries should
5.1 Prototype Demonstration and User Assessment be added to the experiment, as to create a more diverse environment.
The project described in this work was accepted in the 3DUI con- A more profound usability assessment is also suggested in order to
test4 at IEEE VR 20205 , a high profile academic conference in the fully analyse the impact of this virtual experience.
field [4]. In this event, the developed prototype was openly evaluated
both by the academic community and by an official expert jury. The ACKNOWLEDGMENTS
evaluation comprised of three criteria: novelty, provoked empathy This study was partly funded by the Coordenação de
and aesthetics. Aperfeiçoamento de Pessoal de Nı́vel Superior - Brasil (CAPES)
Besides the official conference judgment of the prototype, a user - Finance Code 001, partly by CNPq and partly by the Swedish
assessment is meant to be taken place in a future work. Subjects Council for Higher Education (UHR). We also acknowledge
will be asked to participate in a simple assessment of the expe- FAPERGS (project 17/2551-0001192-9), CNPq-Brazil (project
rience they will have in the aforementioned task. Based on the 311353/2017-7) and a Linnaeus-Palme grant within the partnership
work of Ratan [13], the Self-Presence Questionnaire (SPQ) will (3.3.1.34.11116-2019) between the School of Information Tech-
be used, which has the intention of measuring proto self-presence nology at Halmstad University and the Institute of Informatics at
(body-schema) through questions about the integration of the user’s UFRGS for their financial support.
self-representation into a virtual environment. The SPQ is simply
R EFERENCES
3 https://exiii.jp/wrist-dk2/
[1] S. J. Ahn. Embodied experiences in immersive virtual environments: Ef-
4 http://ieeevr.org/2020/contribute/3dui-contest.html
fects on pro-environmental attitude and behavior. PhD thesis, Citeseer,
5 http://ieeevr.org/2020/ 2011.
[2] E. Ambron, A. Miller, K. J. Kuchenbecker, L. J. Buxbaum, and into the effects of a virtual reality system on phantom limb pain: a pilot
H. Coslett. Immersive low-cost virtual reality treatment for phan- study. British Journal of Pain, p. 2049463719859913, 2019.
tom limb pain: Evidence from two cases. Frontiers in neurology, 9:67, [10] D. Leonardis, A. Frisoli, M. Barsotti, M. Carrozzino, and M. Berga-
2018. masco. Multisensory feedback can enhance embodiment within an
[3] D. Bacchus. Life is beautiful. always: Using virtual reality to share the enriched virtual walking scenario. Presence: Teleoperators and Virtual
experience of disability. 2019. Environments, 23(3):253–266, 2014.
[4] A. Calepso, R. Guarese, R. Valer, Y. J. Iquiapaza Ccama, V. A. Oliveira, [11] M. P. Z. Nejatkermany, E. Modirian, M. Soroush, M. Masoumi, and
L. Nedel, and A. Maciel. 3dui and the phantom limb: multisensory M. Hosseini. Phantom limb sensation (pls) and phantom limb pain (plp)
experience for embodiment of amputation. In 2020 IEEE Conference among young landmine amputees. Iranian journal of child neurology,
on Virtual Reality and 3D User Interfaces Abstracts and Workshops 10(3):42, 2016.
(VRW), pp. 517–518, 2020. doi: 10.1109/VRW50115.2020.00107 [12] D. L. Penn, J. D. Ivory, A. Judge, et al. The virtual doppelganger:
[5] H. Q. Dinh, N. Walker, L. F. Hodges, C. Song, and A. Kobayashi. Effects of a virtual reality simulator on perceptions of schizophrenia.
Evaluating the importance of multi-sensory input on memory and the The Journal of nervous and mental disease, 198(6):437–443, 2010.
sense of presence in virtual environments. In Proceedings IEEE Virtual [13] R. Ratan. Self-presence, explicated: Body, emotion, and identity
Reality (Cat. No. 99CB36316), pp. 222–228. IEEE, 1999. extension into the virtual self. Handbook of Research on Technoself.
[6] A. Gallace, M. K. Ngo, J. Sulaitis, and C. Spence. Multisensory New York, NY: IGI Global, 01 2012.
presence in virtual reality: possibilities & limitations. In Multiple [14] A. Woodhouse. Phantom limb sensation. Clinical and experimental
sensorial media advances and applications: New developments in pharmacology and physiology, 32(1-2):132–134, 2005.
MulSeMedia, pp. 1–38. IGI Global, 2012. [15] N. Yee and J. N. Bailenson. The proteus effect: Self transformations in
[7] V. Groom, J. N. Bailenson, and C. Nass. The influence of racial virtual reality. Human Communication Research, 33(3):271–90, 2007.
embodiment on racial bias in immersive virtual environments. Social [16] N. Yee, N. Ducheneaut, M. Yao, and L. Nelson. Do men heal more
Influence, 4(3):231–248, 2009. when in drag? conflicting identity cues between user and avatar. In
[8] K. Kilteni, R. Groten, and M. Slater. The sense of embodiment in Proceedings of the SIGCHI conference on Human factors in computing
virtual reality. Presence: Teleoperators and Virtual Environments, systems, pp. 773–776, 2011.
21(4):373–387, 2012.
[9] J. Kulkarni, S. Pettifer, S. Turner, and C. Richardson. An investigation