Dancing provides a powerful way for one to feel connected with others. Virtual Reality (VR) holds the promise of enabling shared dancing experiences over a distance, but major challenges remain, such as network latency, afordable consumer VR devices lacking full-body tracking, and users having limited real-world space in their homes. In this paper, we present and evaluate Duet, a two-player VR contemporary dance game/experience that provides a new solution to these problems. Our solution combines 1) a novel partner movement manipulation approach where each player only needs a small physical space to move, but their perception of their partner's movement is manipulated to allow the choreography to utilize a large virtual space, 2) a latency-tolerant visual metaphor for connection, in the form of an elastic string connecting the players, and 3) abstract avatar design that doesn't require full-body tracking and also increases the plausibility of the movement manipulation. We conducted a mixed methods exploratory user study with 22 participants (11 pairs) comprising a thematic analysis of semi-structured interviews and a visualization and analysis of perceived latency. The results suggest that Duet can mitigate the problems caused by latency, limited real-world space, and the lack of full-body movement tracking, although our data does not allow conclusions about Duet's generalizability to other dance styles or choreographies.
Dance games are traditionally focused on honing individual skills or trying to beat a high score (e.g., Dance Dance Revolution [G3]) or learning dance moves and learning new choreographies (e.g., Dance Central [G5]). Many popular Virtual Reality (VR) games such as Beat Saber [G2] and Dance Central VR [G6] have adopted the same approach. Such skill-focused competitive design represents, however, just one aspect of dancing.
For the longest time, dancing has been an activity for feeling connected with others. Some dance games
like Just Dance [G8] have a strong focus on dancing with your friends and even the aforementioned
Dance Dance Revolution has cultivated a strong dancing community around it [
In many aspects of VR, there seems to be a drive to simulate and try to replicate the real world. However, not leveraging the unique aspects that make VR diferent from the physical world has the danger of keeping VR experiences mere shadows of the real thing. People often have the fantasy that when they use VR devices, they can live their dreams and do impossible things. But, not only the limitations of the technology but the player’s own body and their limited play area can make executing these fantasies dificult.
Solutions to the problems above often relate to the user’s control of their own virtual body, focusing
on how a VR user can move expressively despite the limitations of the physical space (e.g., [
Contribution We contribute Duet, a novel two-player VR dance game/experience shown in Fig. 1 and on the supplemental video1. A key innovation of Duet is the partner movement manipulation that allows both players to dance in a limited physical space while experiencing a choreography for two dancers that dynamically utilizes a large virtual space. We implement this symmetrically—for instance, both players can perceive the other player as circling around them, even though both of them are in reality spinning around in place (supplemental video at 2min 2s). Additionally, Duet features latency-tolerant visual elements for connecting two dancers and choreographing their movements. Finally, the avatars of Duet are designed to only need the hand and head tracking aforded by consumer VR devices, while also making the manipulated movement appear visually plausible.
To the best of our knowledge, there is no prior work specifically on manipulating one’s dance partner’s movements as a tool for creating better dance experiences in VR. More generally, ample literature exists on avatar design and movement as well as VR dance experiences. 1Duet supplemental video: https://youtu.be/tXxanwhyHwM
VR experiences are often limited by the user’s body and physical abilities. This has been addressed
by exaggerating aspects such as running speed, jump height, and flexibility [
The players’ physical movements can also be mapped to virtual movements in arbitrary ways beyond
realism. For example, in a study by Riecke et al. [
Many of these re-mappings of movement data also provide solutions for locomotion in a virtual
space larger than the real space, which is a wide and active area of VR research [
Another problem in VR, especially while dancing, is the lack of physicality and haptic feedback. There have been multiple diferent solutions on bringing the sense of touch into VR [ 19, 20, 21]. Venkatraj et al. [22] goes as far as to explore haptics in VR co-embodiment where two users inhabit one avatar. However, most of these require dedicated hardware beyond mainstream consumer VR and the lack of haptic detail makes them non-ideal for dancing with a virtual partner.
Much of dance research in VR focuses on either teaching and instructing dance [
Social dancing in VR is still largely an unexplored area even though many VR users are looking for
these kinds of experiences [
Unlike the previous studies, our goal is not to instruct or teach dance to the player. Instead, we focus on how to create a game-like shared cooperative experience for two dancers.
This section describes the design of Duet in detail. We start with a brief overview of how Duet works and the primary inspiration for it. We then proceed to the three key problems we identified and the corresponding solutions developed.
Duet is a co-operative dancing game/experience that does not need the players to inhabit the same physical space and can be played with afordable Meta Quest VR headsets. It is meant to be easy to approach for all able-bodied people, even without any dance experience.
The experience starts by having both players see the other player arrive, with a brief moment for greeting and interacting. After this, the ring in Figure 1 appears and the players are prompted to touch their hands together, which forms the string between them. The ring then guides the players through a choreographed experience. The players’ goal is to keep the string going through the ring. If they fail, the world’s colors fade out and the music gets distorted, but the players are not otherwise penalized to keep the experience approachable for novice users.
The motivation for having the ring guide players movement originates from outside VR research. We were inspired by Bounden, an innovative "mobile dancing game" for two players, developed by Game Oven in collaboration with the Dutch National Ballet [G4]. In Bounden, both players hold on to a shared mobile phone and try to follow the movement instructions provided on the phone’s screen. The physical link between the two bodies provided by the shared phone provides a movement challenge and makes the players’ bodies entangled in interesting ways. Our goal was to try creating something like Bounden but in VR, utilizing the strengths of the medium and working around the obvious limitation of the players not being able to hold on to a shared physical object.
The visuals of the experience are inspired by underwater kelps that move with ocean waves, a being that is alive but not either animal or plant.
During the design and development, we identified three key problems inherent to VR pair dance and developed specific Duet features to address the problems.
A key problem of all consumer VR is that people have limited space to move in their homes. Ideally, one should be able to play VR games in a small area, which is not ideal for expressive dance movement. As elaborated in Section 2.1, many solutions for this problem do exist, but they all come with their limitations such as motion sickness.
We address the space limitation with the partner movement manipulation. In pair dance, the user’s focus of attention is on their partner, which suggests that making the partner’s movements free of the limitations of the physical space contributes to the overall feel of the experience. It also expands the range of possible pair dance choreographies.
To avoid motion sickness, the movement manipulation is applied only to one’s partner, e.g., both users may see their partner gliding in the space while perceiving their own movement as fully natural and nonexaggerated. This way, each user’s 1st person viewpoint is not artificially altered and visual-vestibular conflict is avoided.
We implement the manipulation symmetrically. This means that both players are seeing the same manipulations at the same time. This allows both players to have a similar experience designed to align with the music.
We employ two manipulation techniques: • Position manipulation: The choreography has parts where the other player circles around and lfies above the other player, i.e., we modify the translation and rotation of the partner’s avatar in each player’s view. In reality, both players are spinning around in place. This enables players to have more elaborate and dramatic experiences together while having a limited amount of space to move in the physical world. • Enforced mirroring of movements: At specific parts of the choreography, we modify the partner’s movements fully or partially mirror the player’s own movements, in order to elicit a stronger feeling of connection and movement synchronicity. This is less obvious to the player than what one would think, due to the abstract avatar design.
Much of social dance includes moving while maintaining physical contact, or moving separately but in a
synchronized manner. These are challenging for social VR due to the lack of force feedback to maintain
physical contact, and latency that prevents synchronizing movements based on visual feedback. Due
to the combined latency of networking, motion tracking, rendering, and human visuomotor control,
one cannot perfectly synchronize one’s movements with another dancer in real-time [
A key feature of Duet is the elastic string that connects the dancers, which we settled on after testing various alternatives such as a floating sphere that the user tries to follow or push. The string stretches according to player movements, which prevents losing connection due to latency and the lack of force feedback.
Complementing the string, we also feature the ring that provides the choreography guidance and challenge common to dancing games. The ring is animated to move along with the music. Keeping the string inside the moving and rotating ring guides the players to move together in a choreographed manner, while still leaving room for individual expression and improvisation.
To provide variety, the connection points of the string change between the players’ hands and heads at specific parts of the choreography.
As we intended Duet to be as approachable as possible, we chose the popular Meta Quest consumer VR headsets as our VR platform. This meant that we only had access to 3 tracked points: The headset and the handheld controllers.
The movement tracking problem is primarily addressed by Duet’s avatar design (Figure 2). As we could not track the user’s legs, we designed legless avatars somewhat reminiscent of the robed figures of the game Journey [G7]. Consistent with the string that connects the avatars, the avatars themselves are made of elastic cloth stripes.
Crucially, the avatar design also makes the partner movement manipulation look visually plausible, e.g., when one sees one’s dance partner gliding around them and soaring in the air. Having abstract avatars without details like facial features also allows us to brush aside some practical issues such as making sure that players are looking at each other. The enforced mirroring of movements (Section 3.2.2) is also concealed by the abstract avatar design.
Duet does not have a story, but we hope it can allow players to reflect on what it means to be connected or experience something together. The experience moves through four parts: Part 1: Introduction The experience and the choreography are built around a short emotional music piece. The choreography starts with simple wave motions with no manipulation of movement. During this part, the players get used to moving the string and how it sometimes moves from one of their hands to another. Part 2: Movement manipulation After the familiarization phase players’ movements start being manipulated in diferent ways. The players do not know when and how much their movements are alternated. The intensity of the added movements reflects how the music evolves. The goal of these movements is to encourage the players to move more and give them a feeling that the other player is closely connected and responsive to their own movements and the music.
The players see the other player circle around them before rising above the ground. The movement becomes livelier when the music swells. To make players feel like they are truly in sync with the other player, during some movements their moves are copied straight to the other player’s avatar (the enforced mirroring of Section 3.2.2).
Part 3: Dark In the next part of the experience, intensity is elevated by changing the players’ surroundings into darker colors and the players see their partner sinking below them. Part 4: Light Music and movement build towards the finale and the darkness is pushed away by lights. the movements are simple and predictable, allowing the players to say goodbye to each other.
The design process comprised multiple rounds of iterative prototyping and testing. The testing was mostly done by the research team and by bringing functional prototypes to a professional contemporary dance teacher and choreographer for feedback. The teacher also provided us with initial choreography, although we ended up changing it to better utilize the interaction mechanics.
The initial research team comprised two members: A doctoral student with a VR and game development background and one summer intensive class in dance, and a supervisor with 20+ years of experience in practicing diferent movement skills and arts ranging from medieval swordsmanship to contemporary and street dance. A third researcher joined later to help in the user study.
We conducted a mixed methods exploratory user study with 22 participants (11 pairs) to probe the following research questions and identify possible future research directions: RQ1: How do players experience Duet, the virtual space and their own and their partner’s movements? RQ2: How does the partner movement manipulation afect the experience? RQ3: How does network latency afect the experience? Is the design successful in hiding latency?
Participants were recruited by using diferent communication channels inside Aalto University. Every participant was compensated with a gift card to a restaurant inside the campus area (20 euros). 11 pairs participated in the study. The mean age of the participants was 27.1 years (sd. 2.89, min. 23, max. 33). 8 identified as men and 12 as women. One pair of participants did not share their age or gender. All participants were able-bodied and all except two had at least some VR experience. All the pairs knew each other before the study and signed up together.
The study consisted of two main parts, with a total duration of approximately 45 minutes. In the first part, each pair of participants danced through a 3 min 47 s choreography, after which a semi-structured interview was conducted, with both participants in the same interview to promote discussion. The interview template had 1 warm-up question about participants’ familiarity with VR, 4 questions about the user experience (e.g., what were the most memorable parts), 2 questions about experiencing the virtual space, 2 questions about the feel of the movement, and 2 questions about the partner movement manipulation. Clarifying follow-up questions were asked when relevant. The interview template is provided in the appendix.
The enforced mirroring was used twice during the choreography, for a total of 1 min 10 s, i.e., approximately one-third of the choreography. For a total of 46 seconds, the mirrored and partner movement was blended to ease the players into and out of the efect. For a total of 24 seconds, the movements were completely mirrored.
In the second part of the experiment, we studied the efect of latency using a within-subjects design, by having the participants dance continuously while diferent amounts of additional network latency were artificially induced. After each latency change, the participants were queried for how noticeable and how disturbing the latency was, using 5-point scales with anchors "1: Not noticeable - 5: Very noticeable" and "1: Not disturbing - 5: Very disturbing". We tested additional latencies of 0ms, 100ms, 250ms, and 1000ms. Each latency was tested twice for each participant pair, with the order of latencies randomized. Since we organized the study so that both participants were in the same room and wireless network, our latency setting of 0ms corresponds to a "best case" real-world latency; naturally, there is still some latency caused by networking, graphics rendering, and movement tracking.
First, informed consent was obtained from both participants. The participants were then instructed on how Duet works, after which they were helped to put on the VR headsets. As a safety measure, participants were introduced to the boundaries of the play area. Next, they were told to stand on foot markings they could see on the ground in the virtual space. By standing there they were able to connect to the server. After seeing each other’s avatars arrive into the virtual space they were instructed to touch their hands together to start Duet and the 1st part of the study described above.
After dancing through the choreography, the interview was conducted without the VR headsets, and the participants were then asked to put on the VR headsets again for the second part of the study. During the 2nd part of the study, the participants indicated the 5-point latency ratings by showing the corresponding number of fingers with their free hand that was not holding a VR controller.
The interviews were recorded and the recordings were transcribed automatically using OpenAI Whisper [33]. The transcripts were verified manually. The transcripts were then analyzed using reflexive thematic analysis [34]. The first author, who was responsible for the design and implementation of Duet, coded all the data. To avoid myopic interpretations caused by the author’s close connection with the project, the codes and themes were reviewed and refined with another author who had not participated in the design process. The same two authors also facilitated the user study and were present during the interviews.
The quantitative latency data was visualized and analyzed using means, medians, and confidence intervals. Motivated by the visualizations, exploratory correlation analyses were conducted using Spearman’s rank correlation. Spearman’s rank correlation was used because the 5-point rating data cannot be assumed to be normally distributed and the visually observed relation was nonlinear.
The thematic analysis resulted in 157 codes in total, which were grouped into 10 themes. Table 1 provides a summary of the themes. The themes are described below in more detail. The results are reflected in relation to the research questions in Section 6.
One common remark about the whole experience was that it was immersive (“I would say that at a point I felt like I was more present in the virtual environment.” [P3], “It was immersive.” [P4]) Many participants also described the experience as otherworldly, ethereal or somehow magical (“But then afterwards it began to be kind of like a magical experience” [P1], “I wish if I could continue it further because I feel that it’s something like Disney feeling to myself...” [P8], “It went from a simple dance to being something like a bit more ethereal and kind of like, I don’t know, almost magical in that sense.” [P18]). Multiple participants also mentioned how they thought that experience was somehow emotional, meaningful, or felt like art (“Exploration of what art can do.” [P9], “Poetic” [P11], “I think the music, the movement, plus what you see all give you a sensation of emotional climax” [P1].). It was also commonly mentioned that Duet felt relaxing, calming, or meditative (“It was pleasant and meditative” [P21], “It was very calming” [P6], “The environment was relaxing and calming for me” [P19]).
Many participants mentioned that the choreography was simple to follow and the ring and string mechanic was well received. (“I think it’s a very interesting idea, this sort of dance connection” [P13], “It’s a circle and there’s a string and you follow the string and you make the moves. I really liked that part.” [P5]).
The interviews also surfaced some complaints. One of the most common complaints from the participants was that the string changing from one hand to another was confusing at first ( “I forgot that I had to change the hands” [P5], “It was confusing because probably just because of the controllers or something. I was confused when I switched and where to look...” [P2]). Some shorter participants also felt that at some points the ring floated a bit too high for them ( “I was like on my tiptoes. But I’m shorter than everyone else.” [P20]. Many participants also said that they did not focus that much on their partner because they had to keep the string at the correct place (“I felt like I was mostly concentrating on the ring” [P22]).
Many participants mentioned feeling connected with their dance partner (“...it was quite intimate”, “...I feel very connected...” [P1], “I felt connected from the very start” [P11]). One participant also mentioned how the string felt like holding hands “It kind of felt like... holding hands. Holding in air quotes. The string there, that made me feel like we were holding something together” [P13]. One player explained how moving at the same time made them feel connected (“I feel very connected with you in the way that you feel like the other person is doing almost the same as you constantly” [P1]). This might have been partly thanks to the mirroring feature of Duet’s partner movement manipulation. “...I think it was the position of the other actor and also the interaction with it. Often times I was flying away quite much and staying close to me, so I had the feeling, ooh, okay, the space is really big, although I could only stand in this confined space.” [P9] “But since the other player was moving farther away and then closer, it felt large.”[P13] “I think for me like the most memorable moment was like when, at least like when things started building up, like in the beginning, it was just like the two of us like standing in front of each other, but when the person in front of me started actually like flying...”[P18] “...we’re both doing the same thing, which in real life that would not happen. So kind of like, you know, like mind playing fool where like I’m seeing him flying, but he’s not flying.”[P12] “...it was quite intimate” [P14], “...I feel very connected...”[P1], “I felt connected from the very start”[P13] “It kind of felt like... holding hands. Holding in air quotes. The string there, that made me feel like we were holding something together.”[P17] “I think it’s a very interesting idea, this sort of dance connection”[P13] “...initially I would say that I had some sort of problem understanding how it works, but slowly I understood it quite well and it wasn’t an issue going through...”[P3] “...I felt that it’s very mind healing experience because with the music and the rounding and I feel that it gives a mental freedom and something that we call mind healing, mind loving feeling.”[P8] “The visuals, the choreography itself as well when the whole thing moves around and then you have to kind of like turn around. It’s minimal but it’s still good. It still feels like you know dance-like.”[P18] “But then afterwards it began to be kind of like a magical experience” [P1] “...So when it comes to end, really I wish if I could continue it further because I feel that it’s something like Disney feeling...Really, I forgot that I was on the ground.”[P8] “the visuals were nice, especially when the other characters were floating. And you could see some tail or a flowy, feathery ending to that body. It was nice.”[P10] “It’s funny because like again I felt like the other, the avatar of the other person kind of like relatable in a weird way.”[P18] “I loved at the end the sparkly things that appeared with the music”[P10] “...it was interesting to see like like our motions match quite well so it almost felt reflective is what I would say like there will be instances where like I know there’s a person in front of me but the moment would be so similar the moment it felt like I’m looking (at a mirror)...”[P10] “For me, it was quite smooth.”[P16] “...pretty synchronous and then it was quite reactive and responsive”[P4]
All but one participant agreed that the position manipulation feature (Section 3.2.2) enhanced the experience.
“I felt excited.” [P16] “I think for me like the most memorable moment was like when, at least like when things started building up, like in the beginning, it was just like the two of us like standing in front of each other, but when the person in front of me started actually like flying...” [P18] The feature also appeared to elicit curiosity and wonder about how it works. “I was thinking that (is) the other person also experiencing having the exact same experience that I am having, or is it something diferent for them?” [P4] “...I also wondered how it’s possible” [P3] “In the beginning, it was just a movement of up and down, and then it started to go all over the room. I think in the middle of the study I said, how can you fly? And then he said, you’re lfying, so it was kind of reversed. So we were both having the same experience, but then you don’t see the other person flying. So it was like a VR inside a VR. Even in virtual reality, your partner is in the VR towards you, where he’s in another reality, but he’s not doing these old trick things...” [P10]
One participant remarked how the manipulation allows the dance to make use of the whole space beyond only dancing on the floor.
“...because I mean this is a VR so we can make use of the whole space. I think if it was more on like an axis that would go more two-dimensional. Then it would defeat the purpose. So I really liked that we could use the whole space. Or then the other one was floating and then I don’t know. It just felt like it was more of a world within this world.” [P20]
It was also mentioned that some manipulation results in movements that resemble familiar dance moves: “flying around and going behind you it kind of mimics like in real life when you’re having a dance with a person and you have to do this motion with your hand when they’re kind of like circling around you so holding your hand and you’re dancing with someone they have to circle around you...” [P18]
Only one participant called the position manipulation “confusing” but could not articulate a reason for the feeling.
At the part of the interview where we revealed the occasional enforced movement mirroring (Section 3.2.2), only four participants indicated noticing it. None of the participants mentioned noticing the feature during the earlier parts of the interview. Two of the participants who had noticed the feature indicated that it made them feel lonely “it made me feel very lonely because I thought, Oh my god, it’s just me in this universe” [P9], “I was like "aww". I’m alone dancing” [P19]. After the reveal, one participant also stated that they felt a bit betrayed (“I’m not angry either but you feel betrayed sort of thing” )[P17]. This indicates that the feature might hurt rather than enhance the experience, and designers should be mindful of how these kinds of manipulations are utilized.
On the other hand, one participant did not mind the mirroring because they separated the avatar character from the human dancer. (“I’m like oh that’s interesting but like I think ... I knew that you were like making those motions and everything but then in the sense that this experience was like ... I was in my own bubble sort of thing that I was just interacting with something like not necessarily you. I am separating you and the thingy...” )[P17] Possibly, if players accept that their partner is some form of hybrid human-virtual character, it may open new ways for interacting in VR.
Unsurprisingly, the visual environment was the most commonly mentioned reason for the space feeling large (“I also think the space felt very large, especially, I think, because of aesthetics of the whole outer world’s scenario and everything.”, “...it was replicating the universe.” [P4]).
However, many participants also mentioned that the partner movement manipulation did make the space feel larger: “...But since the other player was moving farther away and then closer, it (the space) felt large.” [P13] “...I think it was the position of the other actor and also the interaction with it. Often it was lfying away quite much and staying close to me, so I had the feeling, ooh, okay, the space is really big, although I could only stand in this confined space.” [P9]
The most common reason for the space feeling small was the Meta Quest’s safety boundary. When the Quest is used in a new space, it requires the users to indicate the safe play area by drawing it on the floor. We performed this step in the testing space, and when participants moved the area, a virtual wall/grid appeared to guide them back (“...I was conscious about the grid and the environment.” [P21],“I was going out of the boundary and it was showing me the boundary. So it felt like I’m going to hit something.” [P5]).
Many participants mentioned how the experience felt like dancing (“It was kind of like an actual dancing where one person follows and the other leads” [P20], “ I felt like dancing a little bit even when I was just turning” [P15], “I felt like I’m dancing” [P6]). However, some mentioned that it felt like something else like watching a spectacle or working together (“I almost felt like as an audience member.” [P18], “...working together in virtual space” [P14]). One user also mentioned how the position manipulation made the experience feel more like dancing (“I wouldn’t call it a dance for me personally...We were just moving up and down or in a circle. Maybe when actually we were doing the circle or when the other character was lfoating down. That was nice...” [P10]).
Two participants mentioned how they would have wanted to move more (“...when the music climaxed, there was almost an incentive to move the controller in another way, besides just staying inside the circle.” [P9], “...it didn’t feel like it was promoting a lot of motion like asking me to move around...” [P18]). On the other hand, the pacing and challenge level is hard to balance for everyone, and some participants did find even the current prototype a bit too much ( “like everything happened so quickly that I apparently didn’t even manage to look around properly. Maybe that’s the only (negative) thing for me.” [P15])
Two participants mentioned how the experience felt like exercise (“It was kind of a nice exercise with the music and everything” [P5], “I also liked that I had to squat as well, so it was exercising, but that’s a good thing” [P13]).
The visuals were considered a major part of the player experience and the avatar design was received positively “For me, the visuals were nice, especially when the other characters were floating. And you could see some tail or a flowy, feathery ending to that body. It was nice.” [P10],“It’s funny because like again I felt like the other, the avatar of the other person kind of like relatable in a weird way. They were kind of like magical, fantastical and so on. While being simple as well they were like this kind of like abstract shape ... in the beginning I was like oh that looks kind of funky, but then the more the thing went on it kind of like felt more and more relatable.” [P18])
Only one participant indicated disliking the avatar design, possibly due to expecting a more humanlike partner (“Actually I don’t like the character. Because firstly it looks a little bit scary. Because I couldn’t ifgure out which part is which. Because I thought it would be a human.” [P11])
Multiple participants described the movement as smooth or responsive (“For me, it was quite smooth.” [P16], “...pretty synchronous and then it (movement with your partner) was quite reactive and responsive” [P4]). Only one participant mentioned that the movement with the string in Duet feels delayed because of how the strings moves(“...the string was always kind of lagging behind a little bit.” [P15]).
The latency ratings are visualized in Fig. 3. The three lowest latency settings—which are in the range that can be expected in social VR—resulted in low ratings for both how noticeable and disturbing the latency was experienced, suggesting that such latencies do not pose major problems. Although this may be attributed to Hawthorne efect to some degree, the highest added latency of 1000ms was consistently rated as both disturbing and noticeable, suggesting that participants were willing to be honest and indicate when the latency was a problem for them.
For the three lowest latencies (0...250ms), there is no apparent correlation between the latencies and the ratings (Noticeable: Spearman = 0.013, = 0.882, Disturbing: Spearman = 0.002, = 0.980), but when the 1000ms latency is included in the analysis, a statistically significant correlation can be detected (Noticeable: Spearman = 0.245, = 0.001, Disturbing: Spearman = 0.324, < 0.001).
Below, we revisit the research questions of Section 4 in light of the results. We also reflect on potential areas for improvement and directions for future work.
Based on the interview data, it is clear that the participants enjoyed the Duet as an experience. The experience was perceived as relaxing, calming, and artistic by many, and the combination of music, visuals, and movement was appreciated.
Many participants reported that they felt a connection and were in sync with their dance partner. However, it should be noted that all the participants knew their dance partner before the study, which might have contributed to the connection they experienced (“I mean, we are also connected in real life, kind of husband and wife. So I think it was really nice to play with you (them) and kind of try to keep that string in there.” [P15]) Furthermore, both participants were in the same physical space, which might elicit a stronger connection than social VR dance over a distance, although the VR headsets did block any real-world visual contact between the participants.
On the negative side, some participants did not consider Duet as dancing. Some also pointed out that they would have preferred to move more. While most of the participants had no complaints, the simple choreography appeared to result in some participants feeling that the experience was too easy. To a degree, this was deliberate; we wanted to make sure that the choreography was easy enough to follow for novice players of all skill levels. For instance, we initially created additional movement challenge and complexity by using obstacles that slowly fall from the sky and the players have to dodge, but this was removed after a pilot test. If Duet was developed into a full game instead of a research prototype, such additional challenges could be gradually added as the game progresses.
In light of the data, manipulating the position of one’s partner in space works as intended. At least for some participants, it managed to create a sense of moving in a large space and a sense of wonder.
Changing the dance partner’s distance or elevation appears to afect the perceived size of the virtual space. It also seems to positively afect the experience in general, creating variety and memorable highlights, and making the minimal movement feel more like dancing. The fact that both players see the other flying while they both stand still was considered enjoyable and intriguing.
The results suggest that we managed to implement the enforced movement mirroring feature subtly and covertly. However, those who notice such mirroring may perceive it negatively, e.g., feeling deceived or lonely. This suggests that the feature should be used with caution, if at all. On the other hand, it might be possible to utilize the feature as a narrative tool in experiences that attempt to intentionally provoke negative emotions to elicit appreciation and reflection in players (see, e.g., [ 35, 36]). For instance, one might first elicit an expectation of meeting a loved character, and then subvert the expectation by having the character parrot back the movements and voice of the player to emphasize that they are in reality alone. A further potential benefit of the mirroring is that it might contribute to the movement feeling smooth and responsive and the players feeling synchronized with each other, but our present data does not allow establishing such a causal relationship and more research is needed.
The comments of one participant also suggest that it might be possible to intentionally frame the avatars as being "co-piloted" by a person and a game or VR app, which might allow even more extensive movement manipulation without alienating the users.
The interview comments about movement feel and the latency ratings in Fig. 3 suggest that Duet is successful in hiding small and moderate latencies inherent in social VR dancing, at least to some degree. The key feature in this regard is the elastic string that maintains visual and simulated-physical connection between the dancers even when their movements are not synchronized. At least for slow and meditative choreography similar to our current prototype, Duet appears to provide a viable and latency-tolerant interaction approach to social VR dance. However, this might change depending on the choreography and music—for instance, in the context of gestural control of musical instruments, it has been found that latency tolerance depends on the melody played, with slow and continuous melodies afording higher latencies [37].
Duet is a contemporary dance experience with simple, fluid, and relatively slow movements. The choreography was deliberately designed such that participants of all skill levels could experience it. However, this means that it is not as interesting for people who are looking to challenge themselves. The dificulty of a choreography and a player’s skill definitely afects the experience and future work would be needed to study how the mechanics work in designing more varied choreographies for all skill levels. New mechanics, such as falling obstacles, could also be introduced to challenge and guide the players’ movements. Duet is also a relatively short experience and some of its appeal to the participants can probably be attributed to its novelty, longer-term engagement remaining unclear. A short experience might have also been one of the reasons that most participants did not notice any enforced mirroring.
The radius of the ring in Duet is quite large. This makes the experience more forgiving and allows players to improvise their own movements if they so choose. To change the challenge level, the size of the ring could be adjusted. The ring also only evaluates if the player moves their hand correctly on a horizontal and vertical axis while the player can move closer and further away from the ring freely. This means that dance styles and choreographies that require small and specific movements cannot be instructed using the ring.
Another problem that could be addressed in future work is that many participants indicated focusing on the ring during the experience instead of their dance partner. This should be addressed in the future if the goal is to keep players’ focus on their partner. To achieve this, one might add more interactions between the dancers or change the ring into something else altogether.
During our study, the Meta Quest safety boundary was often triggered, which may make the space feel smaller and more constrained. We should have organized the user study in a larger space, but some users might also not have enough space in their homes; spinning around with one’s hand extended does require a space with a diameter of approximately 2 meters. We consider this a reasonable minimum for meaningful full-body dance movements; hence, the boundary problem reflects a fundamental limitation of VR dancing. To address the limitation, VR dance research could perhaps explore forms of dance such as finger tutting 2, which are only performed using a part of one’s body. This might also provide an interesting challenge for VR hand and finger tracking technology development.
Currently, our partner movement manipulation consists of the position manipulation and the enforced mirroring. In the future, this could be expanded. For example, the partner’s movements could be blended with predefined animations or motion capture [ 38], which could be helpful as a narrative element or in minimizing perceived latency.
While our data suggests that there were little to no diferences in how participants perceived the smaller latencies, our limited sample size means that more research is needed to confirm the results. Furthermore, more work is needed to explore other possible latency-tolerant interaction mechanics for social VR.
We have described the design of Duet, a novel two-player collaborative VR dance game/experience. A key innovation of Duet is its partner movement manipulation, which expands the range of shared movements and choreographies possible when the players dance in a small physical space, which is the case for most home VR users. Other defining features include the elastic string that visually connects the dancers in a latency-tolerant way, the ring that guides the players through choreography, and the abstract avatar design that makes the movement manipulation visually plausible and hides the limited motion tracking capabilities of consumer VR.
We evaluated Duet in a qualitative user study with 11 pairs of participants. The results suggest that we largely reached our design goals, creating a meditative and enjoyable social VR experience, with the positional partner movement manipulation eliciting wonder and contributing to an illusion of a large dance space and freedom of movement. On the other hand, the enforced mirroring feature of the partner movement manipulation was less successful, eliciting mixed reactions and a feeling of loneliness in some participants. Compared to Bounden [G4]—our primary inspiration—the lack of real physical contact between dancers in VR might also make one focus more on the user interface and less on one’s partner.
Beyond dancing, we believe that partner movement manipulation can provide a new tool for crafting social VR games and experiences.
This work has been supported by the European Commission through the Horizon 2020 FET Proactive program (grant agreement 101017779). 2For examples of finger tutting, see, e.g., https://www.youtube.com/watch?v=yJwPydpJHyA,https://www.youtube.com/watch? v=TzmBmpr9y3o [19] A. Gallace, M. Girondini, Social touch in virtual reality, Current Opinion in Behavioral Sciences 43 (2022) 249–254. [20] M. A. Otaduy, A. Okamura, S. Subramanian, Haptic technologies for direct touch in virtual reality, in: ACM SIGGRAPH 2016 Courses, 2016, pp. 1–123. [21] J. Perret, E. Vander Poorten, Touching virtual reality: a review of haptic gloves, in: ACTUATOR 2018; 16th International Conference on New Actuators, VDE, 2018, pp. 1–5. [22] K. P. Venkatraj, W. Meijer, M. Perusquia-Hernandez, G. Huisman, A. El Ali, Shareyourreality: Investigating haptic feedback and agency in virtual avatar co-embodiment, in: Proceedings of the CHI Conference on Human Factors in Computing Systems, 2024, pp. 1–15. [23] Y. Xu, Y. Li, Design and research of vr system in latin dance teaching, in: 2019 International Conference on Advanced Education, Service and Management, volume 3, The Academy of Engineering and Education, 2019, pp. 59–62. [24] O. Kivelä, Exploring the potential of VR games as exercise, Master’s thesis, O. Kivelä, 2020. [25] B. Sarupuri, R. Kulpa, A. Aristidou, F. Multon, Dancing in virtual reality as an inclusive platform for social and physical fitness activities: a survey, The Visual Computer 40 (2024) 4055–4070. [26] Y. Usui, K. Sato, S. Watabe, E. Yanagida, Vr teaching materials for dance practice, in: 2019 8th
International Congress on Advanced Applied Informatics (IIAI-AAI), IEEE, 2019, pp. 178–183. [27] D. L. Eaves, G. Breslin, P. Van Schaik, E. Robinson, I. R. Spears, The short-term efects of realtime virtual reality feedback on motor learning in dance, Presence: Teleoperators and Virtual Environments 20 (2011) 62–77. [28] Y. Xu, Y. Li, Design and research of vr system in latin dance teaching, in: 2019 International Conference on Advanced Education, Service and Management, volume 3, The Academy of Engineering and Education, 2019, pp. 59–62. [29] I. Kico, F. Liarokapis, Comparison of trajectories and quaternions of folk dance movements using dynamic time warping, in: 2019 11th International Conference on Virtual Worlds and Games for Serious Applications (VS-Games), IEEE, 2019, pp. 1–4. [30] S. Kirakosian, E. Maravelakis, K. Mania, Immersive simulation and training of person-to-3d character dance in real-time, in: 2019 11th International Conference on Virtual Worlds and Games for Serious Applications (VS-Games), IEEE, 2019, pp. 1–4. [31] S. Senecal, N. A. Nijdam, A. Aristidou, N. Magnenat-Thalmann, Salsa dance learning evaluation and motion analysis in gamified virtual reality environment, Multimedia Tools and Applications 79 (2020) 24621–24643. [32] I. Kico, D. Zelníček, F. Liarokapis, Assessing the learning of folk dance movements using immersive virtual reality, in: 2020 24th International Conference Information Visualisation (IV), 2020, pp. 587–592. doi:10.1109/IV51561.2020.00100. [33] A. Radford, J. W. Kim, T. Xu, G. Brockman, C. McLeavey, I. Sutskever, Robust speech recognition via large-scale weak supervision, in: International conference on machine learning, PMLR, 2023, pp. 28492–28518. [34] V. Braun, V. Clarke, One size fits all? What counts as quality practice in (reflexive) thematic analysis?, Qualitative Research in Psychology 18 (2021) 328–352. URL: https://www.tandfonline. com/doi/full/10.1080/14780887.2020.1769238. doi:10.1080/14780887.2020.1769238. [35] J. A. Bopp, E. D. Mekler, K. Opwis, Negative emotion, positive experience? emotionally moving moments in digital games, in: Proceedings of the 2016 CHI conference on human factors in computing systems, 2016, pp. 2996–3006. [36] E. D. Mekler, I. Iacovides, J. A. Bopp, " a game that makes you question..." exploring the role of reflection for the player experience, in: Proceedings of the 2018 annual symposium on computerhuman interaction in play, 2018, pp. 315–327. [37] T. Mäki-Patola, P. Hämäläinen, Latency tolerance for gesture controlled continuous sound instrument without tactile feedback, in: ICMC, Citeseer, 2004. [38] A. Bleiweiss, D. Eshar, G. Kutlirof, A. Lerner, Y. Oshrat, Y. Yanai, Enhanced interactive gaming by blending full-body tracking and gesture animation, in: ACM SIGGRAPH ASIA 2010 Sketches, 2010, pp. 1–2. [G1] VRChat. 2017. VRChat. https://hello.vrchat.com/. [G2] Beat Games. 2019. Beat Saber. https://store.steampowered.com/app/620980/Beat_Saber/. [G3] Konami. 1998. Dance Dance Revolution. Arcade. [G4] Game Oven. 2014. Bounden. Game [iOS, Android]. [G5] Harmonix Music Systems. 2010. Dance Central. [G6] Harmonix Music Systems. 2019. Dance Central VR. Microsoft Windows, Meta Quest 2. [G7] Thatgamecompany, Inc. 2012. Journey. Game [PlayStation 3, PlayStation 4, Windows, iOS].
Sony Computer Entertainment, Annapurna Interactive. [G8] Ubisoft. 2009. Just Dance.
Section Ratings
Questions Thanks, that was all for this part. Now, we have another part about network latency, where you will keep dancing, we will adjust the latency, and ask you how noticeable and disturbing different latency settings feel like.