=Paper=
{{Paper
|id=Vol-1833/7_Coady
|storemode=property
|title=Perspective Shifts in Mixed Reality: Persuasion through Collaborative Gaming
|pdfUrl=https://ceur-ws.org/Vol-1833/7_Coady.pdf
|volume=Vol-1833
|authors=Derek Jacoby,Yvonne Coady
|dblpUrl=https://dblp.org/rec/conf/persuasive/JacobyC17
}}
==Perspective Shifts in Mixed Reality: Persuasion through Collaborative Gaming==
https://ceur-ws.org/Vol-1833/7_Coady.pdf
Perspective Shifts in Mixed Reality: Persuasion through
Collaborative Gaming
Derek Jacoby1 and Yvonne Coady2
1
QVirt.com
derekja@qvirt.com
2
University of Victoria
ycoady@uvic.ca
Abstract. Mixed reality environments allow collaborative interaction with
shared virtual objects to extend beyond participants wearing head mounted dis-
plays. Teams of collaborators can now share perspectives within a virtual space,
and share their experiences with others through the use of green screens and real-
time camera mixing. Our work proposes to explore commodity-based mixed re-
ality as a personalizable persuasive technology. Specifically, our case studies are
designed to study heightened connectedness to environmental and societal
changes in Western Canada. By allowing participants to share their personal ex-
perience of history and the environment interactively through a series of small
shifts in perspective, we hope to better empower individuals to act on global is-
sues.
Keywords: mixed reality, empowerment, environmental and societal changes
1 Introduction
Our virtual reality (VR), augmented reality (AR), and mixed reality (MR) projects1
leverage new technologies to increase engagement and collaboration to enhance edu-
cation, learning, and data analytics environments. A great deal of synergy exists with
Virtual Reality for Impact2 , which is a program sponsored by VR headset maker HTC
to use new technologies to immerse people into experiences that can create connections
and empathy by sharing perspectives. The goal is to motivate society to act on global
issues based on a shared sense of commonality within the United Nations Sustainable
Development Goals (SDGs). Specific goals that touch on connections to environment
and society include:
Goal 11. Make cities and human settlements inclusive, safe, resilient and sustainable
Goal 12. Ensure sustainable consumption and production patterns
1
Supported in part through an Innovation Grant from Ericsson.
2 http://vrforimpact.com/
Copyright © by the paper’s authors. Copying permitted for private and academic purposes.
In: R. Orji, M. Reisinger, M. Busch, A. Dijkstra, M. Kaptein, E. Mattheiss (eds.): Proceedings
of the Personalization in Persuasive Technology Workshop, Persuasive Technology 2017, Am-
sterdam, The Netherlands, 04-04-2017, published at http://ceur-ws.org
Perspective Shifts in Mixed Reality: Persuasion through Collaborative Gaming 85
Goal 13. Take urgent action to combat climate change and its impact
Goal 15. Protect, restore and promote sustainable use of terrestrial ecosystems, sus-
tainably manage forests, combat desertification, and halt and reverse land degrada-
tion and halt biodiversity loss
Goal 16. Promote peaceful and inclusive societies for sustainable development, pro-
vide access to justice for all and build effective, accountable institutions at all levels
Recently, HTC Vive started a large scale, persuasive computing initiative called VR
for Impact.
Our work-in-progress asks the question, can we heighten connectedness and empa-
thy by personalizing the persuasive technologies involved in mixed reality with modeled
user interactions which detect perception and attitude shifts? Specifically, this short
paper overviews both the technology involved and the case studies we are pursuing.
Section 2 overviews the hardware, software, and approximate costs for creating a pro-
posed mixed reality museum installation at the BC Royal Museum. Section 3 outlines
the historical contexts for the shared perspectives, involving over 140,000 images
showing changes in Canada’s mountain environments and more than 700 ethnographies
written about Coast Salish peoples.
2 Commodity-Based Mixed Reality Technology
Projects like Microsoft’s RoomAlive [6] and YouTube’s Mixed-Reality Lab (Figure
13) create immersive experiences that augment or mix the reality of individuals in a
room by enabling interaction with virtual objects. Most importantly, cameras and pro-
jectors create a shared experience. By that we mean, other people watching can see the
environment the individual with the head mounted display is seeing. This way, individ-
uals are not only impacted by their own interactions, but by the experiences of others
as well. Perspective shifts may result from a better understanding of how others are
impacted, and potentially form a more empathetic connection within that context.
In terms of software, we leverage a popular framework for development of VR ap-
plications, Unity4, which can be augmented with plugins enabling easy deployment of
mixed reality extensions. Open source projects capable of large scale execution across
heterogeneous environments are also increasingly easy to deploy.
Interactions with virtual characters in our learning and analytics environments are
created using the Microsoft Bot Framework, which allow us to imbue artificial person-
alities with realistic traits. With spoken input occurring through the Microsoft speech
recognition engine, this leads to constrained, but natural feeling, dialogues not only
with other participants but also with the system.
Modeling of user behaviors and attitudes is an area of particularly active research.
From explicit modeling techniques such as structured dialogue trees, to probabilistic
drama management techniques [7] there are a variety of ways of making the system
personalized to user desires. As has been shown in the recent election, explicit modeling
3 http://www.youtube.com/watch?v=8T7ux3DXP_w&ab_channel=TomScott
4 http://unity3d.com/unity
86 Perspective Shifts in Mixed Reality: Persuasion through Collaborative Gaming
of user attitudes can be accomplished from otherwise innocuous social media interac-
tions [1], so designing a system to elucidate attitudes and attitudinal shifts in a gaming
context is certainly possible.
In terms of hardware, Table 1 shows how a minimally equipped mixed-reality mu-
seum installation can be created less than $15,000 Canadian Dollars.
Fig. 1. YouTube’s Mixed Reality Lab showing camera, booth and greenscreen (left) and pro-
jected environment (right).
Table 1. Approximate costs for a mixed-reality museum installation using commodity hard-
ware.
Minimum Setup
Hardware Units Cost/Unit Total
Head Mounted Displays for individualized Virtual Reality 2 $1,500 $3,000
Computers 3 $2,500 $7,500
Camera 1 $1,000 $1,000
Display Monitors 2 $1,000 $2,000
Booth and Greenscreen 1 $1,000 $1,000
$14,500
3 Persuasion and Perspective Shifts: Case Studies
Our work-in-progress consists of two case studies underway, both currently developed
as interactive web-based mapping applications. First is the Mountain Legacy Project
[2], which explores changes in Canada’s mountain environments.
Perspective Shifts in Mixed Reality: Persuasion through Collaborative Gaming 87
Fig. 2. Images from the Kootenay and Columbia Valleys showing a photo from 2009 (left) as
an overlay with a slider (middle) over an image from 1922 (right).
The project started with upwards of 120,000 systematic glass plate negatives taken by
land surveyors 1861-1953. More recently re-photographed images have been carefully
added as a basis for comparison and change detection (Figure 2). The resulting image
pairs can be analyzed, and the environmental impact of aggressive resource harvesting,
such as clear cutting, can be made immediately evident to participants in this study.
The second case study involves Ethnographic Mapping in the Coast Salish World
[4] (Figure 3). This project has compiled over 700 ethnographies written about Coast
Salish peoples over the past 150 years. Participants can explore the paradox of carto-
graphic boundaries for an indigenous community whose core social relationships are
borderless kin networks. The nature of continuous and overlapping regions instead of
discrete bounded areas represented in contemporary land claims maps can be experi-
enced first-hand, in the context of broader relationships that were influential at that
time.
Ultimately, these case studies will be combined, along with artifacts and scenarios
provided by our collaborators at the Royal BC Museum, to create an interactive educa-
tional game which present situations drawn from these environmental and ethnographic
data sources, and presenting challenges in investigating historical and current patterns
of resource use and their effect on the world. The effective use of in-museum interactive
applications has been well- established [3], but our contention is that immersive envi-
ronments will be even more compelling. There is a rich history of game-based collab-
orative learning approaches which we are fully drawing upon [5].
By shifting of perspectives into the scenarios within this investigation, and modeling
of user behaviors, it is hoped that attitudes can be quantified and effected throughout
the gameplay. Our hypothesis is that even watching this gameplay can be of significant
educational value, but that maximum attitudinal benefit is derived by the additional
engagement and agency of being fully immersed within the context of a persuasive
educational game. For example, users could watch each other, in a mixed reality envi-
ronment, while they navigate through an historic scene. While exploring artifacts, the
88 Perspective Shifts in Mixed Reality: Persuasion through Collaborative Gaming
group could try to determine which people they are interacting with, and where they
are in terms of time and place.
Fig. 3. Two historic overlays showing both the distribution of the Coast Salish peoples in 1930
and Salish Dialects before 1908 with adjustable opacity to explore the permeability of the
boundaries involved.
4 Key Challenges in Personalization
In the context of our proposed methodology, personalized persuasion relies on feedback
implicit and explicit user modeling data captured during the exploration of a historical
context. Ultimately, we are attempting to detect changes in an individual that may cor-
relate to a perspective shift. Once detected, we can guide the individual within their
exploration of the environmental and societal conditions involved in the educational
gameplay scenario.
Considering each set of goals aligned with their respective environmental or societal
theme, we can highlight our proposed approach and key challenges as follows:
Goal 12. Ensure sustainable consumption and production patterns
Goal 13. Take urgent action to combat climate change and its impacts
Goal 15. Protect, restore and promote sustainable use of terrestrial ecosystems, sus-
tainably manage forests, combat desertification, and halt and reverse land degrada-
tion and halt biodiversity loss
Proposal. Immersive exploration of changes introduced by exploitative, unsustain-
able practices over time.
Challenge. Do small, shared perspective shifts heighten personal connectedness to en-
vironment?
Perspective Shifts in Mixed Reality: Persuasion through Collaborative Gaming 89
Goal 11. Make cities and human settlements inclusive, safe, resilient and sustainable
Goal 16. Promote peaceful and inclusive societies for sustainable development, pro-
vide access to justice for all and build effective, accountable institutions at all levels
Proposal. Immersive exploration of examples of people and place that defied discrete
boundaries.
Challenge. Do personalized identity shifts heighten empathy to other groups within
historic contexts?
5 Conclusion and Future Work
In order for the United Nations Sustainable Development Goals to be realized globally,
we need to start by motivating individuals locally. Personalizing persuasive technolo-
gies, such as mixed reality, may be an import way to increase the likelihood of desired
behavior changes. By leveraging feedback through interaction with virtual objects and
conversational systems, we plan to detect perspective shifts and capitalize on persua-
sive opportunities within a mixed reality setting.
The development of map-based analytics is well underway, and the gamification of
these immersive analytic experiences is a natural next step supported by well-known
techniques from the gaming industry. The integration of conversational systems and
rich attitudinal modeling is more challenging, but using powerful cloud-based cognitive
computing platforms in combination reduces much of this speech recognition and con-
versational dialogue burden. The analysis of attitudes and attitudinal shifts from re-
sponse behavior is perhaps the most forward-looking and challenging of the approaches
being worked towards in this paper, but also the challenge with the most implications
for the design of effective learning and persuasive gaming applications.
References
1. Lassen, N.B., la Cour, L., Vatrapu, R.: Predictive analytics with social media data. The
SAGE Handbook of Social Media Research Methods, p. 328 (2017).
2. Sanseverino, M.E., Whitney, M.J., Higgs, E.S.: Exploring landscape change in mountain
environments with the mountain legacy online image analysis toolkit. Mountain Research
and Development 36(4), 407–416 (2016), http://www.jstor.org/ stable/moun-
resedeve.36.4.407.
3. Schroyen, J., Gabriëls, K., Luyten, K., Teunkens, D., Robert, K., Coninx, K., Flerackers, E.,
Manshoven, E.: Training social learning skills by collaborative mobile gaming in museums.
In: Proceedings of the 2008 International Conference on Advances in Computer Entertain-
ment Technology. pp. 46–49. ACM (2008).
4. Thom, B.: The paradox of boundaries in coast salish territories. Cultural geographies. 16(2),
179–205 (2009), http://dx.doi.org/10.1177/1474474008101516.
5. Wendel, V., Gutjahr, M., Göbel, S., Steinmetz, R.: Designing collaborative multiplayer se-
rious games. Education and Information Technologies 18(2), 287–308 (2013).
90 Perspective Shifts in Mixed Reality: Persuasion through Collaborative Gaming
6. Wilson, A.D., Benko, H.: Projected augmented reality with the room-alive toolkit. In: Pro-
ceedings of the 2016 ACM on Interactive Surfaces and Spaces (ISS ’16). pp. 517–520.
ACM, New York (2016), http://doi.acm.org/10.1145/2992154.2996362.
7. Yu, H., Riedl, M.O.: A sequential recommendation approach for interactive personalized
story generation. In: Proceedings of the 11th International Conference on Autonomous
Agents and Multiagent Systems: vol. 1, pp. 71–78. International Foundation for Autono-
mous Agents and Multiagent Systems (2012).