=Paper=
{{Paper
|id=Vol-2618/paper2
|storemode=property
|title=Gamifying the Museological Experience
|pdfUrl=https://ceur-ws.org/Vol-2618/paper2.pdf
|volume=Vol-2618
|authors=Antonio Coelho,Maria van Zeller,Pedro Cardoso,Liliana Santos,Roberto Vaz,José Raimundo
}}
==Gamifying the Museological Experience==
https://ceur-ws.org/Vol-2618/paper2.pdf
Gamifying the museological experience
António Coelho*1,3, Pedro Cardoso1,2,3, Maria van Zeller1, Liliana Santos1, José Raimundo1,3, Roberto Vaz1
1 FEUP – Faculty of Engineering, University of Porto, Rua Dr. Roberto Frias, 4200-465 Porto, Portugal
2 FBAUP – Faculty of Fine Arts, University of Porto, Avenida Rodrigues de Freitas, 265, 4049-021 Porto, Portugal
3 INESC TEC – Institute for Systems and Computer Engineering, Technology and Science, Rua Dr. Roberto Frias, 4200-465 Porto,
Portugal
Several museums are already using virtual and augmented
ABSTRACT reality technologies to provide new experiences to their visitors.
Museums continue to exert fascination in their visitors. However, The Smithsonian museum’ AR Bone Hall Exhibit [1] and the
the new generation of visitors expects museological experiences project Modigliani VR: The Ochre Atelier [2] from the Tate
that promote their active participation. It is in this context that Modern Museum, are some examples. Nonetheless, one of the
games and the gamification of such experiences capitalize on digital challenges museums face today in this context is the
experiential learning by experimenting and enacting with in-game simultaneous navigation and exploration of both these virtual and
embedded artefact surrogates and know-how. physical spaces. Mixed reality technologies aim at bridging such
In this article, we present four distinct projects that aim to enhance gap. However, every context has its very particular characteristics
the visitors’ experience in museums and green spaces, and also that need to be addressed with specific tools and designs in order
their effectiveness in informal learning. In the first project, to promote participation and stimulate creative practices in
gamification is used in combination with Augmented Reality to museological spaces.
provide a more engaging experience in a boat museum. The drive In the broad field of the Ludification of Culture, we can focus
of this experience is the metaphor of the stickers album collection on the area of games or the area of playful interaction [3]. In this
to unleash the relevant information of the key-artefacts of the particular work, we focus on the more structured use of games,
museum collection. The second and third projects focus on the use both full-fledged games and gameful design.
of pervasive games, more specifically location-based games, to This document is divided into four main sections, presenting
enhance the visitors’ experience and informal learning in a natural four examples of applications in museums. They were developed
park and a botanical garden, respectively. The second project by members of the Graphics, Interaction and Games research
presents the concept of a mobile app for outdoor nature laboratory, under the same supervision. The projects took place
experiences. The drive for the experience in the third project is the between 2016 and 2018. The first example shows an application
narrative that intertwines specific locations in the botanic garden of gameful design, which is commonly named Gamification, that
and a story inspired by the same place. Finally, in the fourth is the use of game design elements or game design techniques in
project, we focus on the potential of technology to provide non-game contexts. For the second and third examples, we use
accessibility in museums for people with special needs or games that extend the players’ limits of time and space, i.e., the
disability, focusing more specifically on blind visitors. Huizinga’s “magic circle” [4], to the context of the user, on what
is called a pervasive game. In the fourth and last example, we
approach the need of the museums to be inclusive, that is,
Keywords: Gamification, Augmented Reality, Pervasive games, accessible to all visitors including those with particular needs or/
Location-based games, Accessibility. and disabilities.
2 EXAMPLE 1 - GAMIFICATION IN MUSEUMS
Index Terms:
•Human-centered computing~Ubiquitous and mobile Museums consolidate the heritage of a country or a given region,
computing~Empirical studies in ubiquitous and mobile computing interlinking history, art, science and the territory. As an
•Applied computing~Education~Interactive learning inalienable landmark in the history of Portugal, the first example
environments is about Maritime Museums, and most particularly, the
exploration of traditional boats.
We propose a gamified approach based on the concept of
1 INTRODUCTION
sticker album collection and its integration in an Augmented
The use of digital technology can undoubtedly assist and create Reality (AR) mobile application. The concept of sticker album
new experiences for visitors in museums while promoting their collection is quite familiar to most people, mainly from their
active participation. The use of mobile devices during museum youth, and is the central dynamic of the gamification design,
visits has great potential for informal learning, playing an engaging the learner to collect more stickers and progress in the
important role in finding, reading and interpreting museological exploration of the museum. As a pervasive solution, we do not use
artefacts while being non-invasive. However, the continually physical support, but instead, a mobile application to provide the
evolving hardware asks for a constant search for new and learning experiences by uncovering the stickers using AR over the
alternative ways to use such devices, which in turn asks for the museum collection, in order to enhance the knowledge transfer
design and development of new software. An example is the and rewarding. In 2018, we developed a prototype for a boat
museum guide being assisted and sometimes replaced by diverse museum where digital stickers are obtained by overcoming
smartphone applications. challenges in the context of the exploration of the boats in the
museum rooms [5].
One of the main principles of gamification is the engagement
* acoelho@fe.up.pt
cycle. Thus, we have developed a progressive activity that
Copyright © 2020 for this paper by its authors. Use permitted under engages the player into small iteration cycles in order to provide
Creative Commons License Attribution 4.0 International (CC BY 4.0).
mastery. Each boat has a predefined number of stickers that, being the epic meaning of the experience, and informal learning is
initially drawn as empty slots, show the number of challenges to fostered on the challenges associated with the locked stickers. At
solve (this number can vary) in order to collect the boat. Figure 1 the same time, progression is achieved by collecting the missing
shows the typical iteration cycles for a specific boat. stickers. The AR layer is the key element that provides the
connection between this heritage and the gamified learning
experience.
3 EXAMPLE 2 – LOCATION-BASED GAMES IN PARKS: THE CASE
OF GAIA’S BIOLOGICAL PARK
Digital mobile games can offer engaging experiences and increase
communication effectiveness concerning cultural and natural
heritage. Location-based serious games allow us to overcome the
barriers of space and enables their players to explore the real
world around them while playing a game with educational or
cultural purposes.
A prototype of a mobile app was developed for the Gaia’s
Biological Park [6], in Portugal, to help in making science
communication engaging for visitors and in improving the overall
outdoor visitor experiences. The concept is to have a mobile app
to promote and disseminate the natural heritage in a tourism
context, with a map, the user’s location in real-time, points of
interest with geolocalized information (GPS), and small serious
games about the green spaces visitors are experiencing. This
project started in 2016.
Figure 1: Engagement cycle – iteration on a boat. One of those games, the virtual animal detector [7] (figure 3) is
The first sticker provides a brief description of the boat and a “treasure hunt” and collection game about the animals that live
challenges the user to identify a specific component of the boat in freely in the park. Some of them are difficult to see in the wild
(figure 2). The user will need to find it using the AR tool. In the on account of their behavioural habits. With this game, visitors
case of difficulty, a “help button” can be pressed, and a text hint can learn more about these species. Players must explore the park
will be provided (about its location). If the user is still not able to to find all the virtual game animals and collect pictures of them all
find it, another more detailed hint (like a drawing of the boat until they complete the collection. An in-game radar detects the
component) will be additionally provided. This provides mastery supposed location of a virtual animal when the player is near that
to proceed. location and displays the information on that species. When
discovered, a species gets registered in a collectables board, and
some information about it can be consulted on-demand.
Another example is the game of the jay [6], that tries to
communicate the dynamics of an oak tree forest ecosystem
through its gameplay. The player controls a jay that collects and
plants acorns. To make the sprouts grow, the player must collapse
clouds to make it rain. That also puts down wildfires. To win the
player must completely reforest the scenery.
Figure 2: First sticker with brief description of the boat.
As the user identifies the boat component, additional
information is provided in the AR layer, showing specificities and
details on top of the original museum boat. This launches the
second challenge, a simple quiz that assesses some information
provided previously in AR or on the site. After collecting this
second sticker, the boat image is complete in the album, and a
new multimedia content is available as a reward: a three-
dimensional model of the boat and a small animation of how the Figure 3: The virtual animal detector [7].
boat is constructed. Thus, the user progresses through each level
by surpassing each challenge, gaining access to digital stickers These games’ goal is to amplify and enrich the visitors’
(and associated multimedia content) as a reward, making the experience, augmenting our perception of the natural world so we
exploration of the museum and the real apprehension of can better understand it and protect it. Although the prototype was
knowledge more attractive. The AR layer has a significant role in designed for the Gaia’s Biological Park, its concept can be
making the connection with the heritage of the museum. adapted and applied to other parks, zoos, botanical gardens and
In conclusion, we propose the concept of the sticker album other green spaces. Presently, the AR implementation into this
collection integrated with AR to improve the museological app concept is being studied, with some promising preliminary
experience. The museum heritage is the anchor point that provides results.
4 EXAMPLE 3 – LOCATION-BASED GAMES IN MUSEOLOGICAL interactor/ player from the locale itself, but one that promotes a
SPACES: THE CASE OF THE UNIVERSITY OF PORTO’S stronger bond between them. We found that one way to do that is
BOTANICAL GARDEN to use that mobile application to focus their attention on the
The University of Porto’s Botanical Garden is a place interspersed physical world and not on their smartphones (or other devices), be
by Literature, Botany and History. The garden was home of the it for navigation, information seeking activities or for moments of
celebrated Portuguese writer Sophia de Mello Breyner Andresen, contemplation.
hence a place with significant influence on her work. In our
5 EXAMPLE 4 – ACCESSIBILITY FOR THE BLIND VISITORS
discernment, a visit to the Botanical Garden is not complete
without acknowledging these different facets as complementary Despite the growing concern about increasing accessibility for
dimensions that characterise this site. visitors with visual impairments [8], [9], [10], [11], only 5,5% of
With this in mind, we developed an experience for visitors to be blind and visually impaired people visit museums in Europe [12].
acquainted with those facets in the form of a location-based game Patrons with total and partial visual impairments represent a
that articulates educational content in Botany with Sophia’s significant potential market for museums [13], and positive
famous story “O Rapaz de Bronze” (the bronze boy). In this game economic effects are expected with the implementation of
(that resorts to GPS), players progress as they encounter particular assistive technologies to promote access to exhibitions [14],
Points of Interest (POI), moving in a specific sequence between besides enhancing their experience of the visit.
them by solving challenges hinted from previous successful steps. Haptic interfaces can take advantages of the dynamic nature of
In each POI, players are confronted with challenges that can be the kinesthetic sense and generate forces, allowing the exploration
solved by inspecting their surroundings, ergo creating a of virtual copies of museum and gallery collections [15] [16]. The
relationship between Literature and Botany, all found in situ. We Museum of Pure Form project allowed visitors to touch and
found this relationship to be crucial for developing player experience the properties of sculptures belonging to the collection
engagement with the locale, and to instigate location-awareness in of multiple European Museums using an arm exoskeleton and a
players. haptic interface for fingers [17], [18]. Visitors are required to pick
Exploratory visits with the garden staff were very welcome and up a pen of the device in order to interact with the Geomagic
necessary to build a proper experience. During the day of testing,1 Touch System [19] and to allow haptic experiences of objects
we resorted to participant-observers for collecting information from the National Museum of Transylvanian History of Cluj-
from participants while they were playing, acting as facilitators Napoca. Another project, developed in the context of the
for solving technical issues and identifying unexpected problems AMBAVis project and, in collaboration with Manchester Museum
in situ. Through these methods and by analysing player surveys, and Gallery Belvedere, allows patrons to explore artefacts,
we concluded that players were engaged with the experience, that augmenting the experience by providing vocal information about
was well-accepted amongst families and groups of children and the object material and characteristics [14].
teenagers. For them, learning through a location-based game Other museums are using 3D printing technologies with
proved to be a fun and enriching. However, specific interaction embedded sensors to provide digitally augmented touch replicas.
design issues arose for our consideration in future work. They offer tactile and hearing multisensory experiences that help
the visually-impaired visitors formulate mental images of the
objects while providing crucial contextual information, such as
the Tooteko project [20], developed for the Correr Museum.
Those technologies are being used as well to reconstruct the
object at the date of its origin, so visitors can learn about how an
Egyptian Cat Sarcophagus was 2500 years ago [21].
Contrary to digitally augmented touch replicas, gesture-based
interactive tactile reliefs do not have physical sensors attached to
detect users’ activities. Instead, their interaction is tracked by
computer-vision or other non-intrusive systems, in order to enrich
tactile reliefs with audio or additional multisensory information.
Researchers developed a gesture-based interactive audio guide for
the interpretation of the painting The Kiss [22] and to assist the
tactile exploration of the painting Madonna with Child and Angels
[23].
Figure 4: Location-based game in the botanical garden. Besides helping to create new ways for disabled visitors to
know about museum collections interactively, technology is also
In conclusion, navigating in the Botanical Garden as a being used to provide them with better physical access, indoor
museological space requires visitors to be attentive to their navigation and wayfinding in such environments. The result is an
surroundings. It is in those surroundings that is the motive for enhancement in their ability to move independently through
their visit, for their presence. Operating a mobile application in different galleries, without the aid of guides or escort. As an
this context needs to be an experience that does not alienate the example, a mobile application was developed at the National
Science Center to assist sightless visitors’ navigation by
1
presenting step-by-step audio instructions [24]. A similar interface
This pilot was developed within the scope of the project was proposed for the Museum of the Lighthouse for Blind [25].
BEACONING: Breaking Educational Barriers with
Contextualised, Pervasive and Gameful Learning. This case study 6 CONCLUSION
was tested on the 10th of February, 2018. Beaconing was an
Museums maintain the same epic meaning that attracts visitors.
H2020 project involving 16 universities and partner companies,
New media and technology can complement and not replace the
INESC TEC included, led by Coventry University mission of these institutions. The presented projects follow
(http://beaconing.eu/). different lines of research for creating frameworks for the design
and the development of AR applications and games. These key
studies have contributed with insights on how to make better or turn. Disability Studies,” Disabil. Stud. Q., vol. 33, no. 3, pp. 1–
more enriching museological experiences. The visitors’ 20, 2013.
experience should be the main drive for technological
[11] European Commission, “Memo: 19 destinations across Europe
breakthroughs in museums, and games, gamification, and mixed
reality technologies can provide improvements in such win awards for accessible and sustainable tourism,” no.
experience. However, these technologies must be balanced, so November, 2013.
they will not distract the visitors from the museum collection but, [12] K. Dash and G. Grohall, “Economic impact of creating and
instead, enhance their experience with these artefacts. exhibiting 3D objects for blind and visually impaired people in
Furthermore, these new technologies promote the inclusive
museums,” pp. 1–81, 2016.
museum where all visitors can have a great experience regardless
of any disability or particular needs. [13] S. Mesquita and M. J. Carneiro, “Accessibility of European
museums to visitors with visual impairments,” Disabil. Soc.,
ACKNOWLEDGMENTS vol. 31, no. 3, pp. 373–388, 2016.
The BEACONING research project, has received funding from [14] Project AMBAVis, “Inclusive technologies in museums,” 2017.
the European Union's Horizon 2020 - The EU Framework [15] J. P. Fritz and K. E. Barner, “Fritz, Jason P.; Barner, Kenneth, E.
Programme for Research and Innovation 2014-2020, under grant
(1999). "Design of a haptic data visualization system for people
agreement No. 687676.
The researchers are supported by the Operation NORTE-08-5369- with visual impairments,” IEEE Trans. Rehabil. Eng., vol. 7, no.
FSE-000049 supported by Norte Portugal Regional Operational 3, pp. 372–384, 1999.
Programme (NORTE 2020), under the PORTUGAL 2020 [16] F. Biocca, K. Jin, and Y. Choi, “Visual touch in virtual
Partnership Agreement, through the European Social Fund (ESF). environments: An exploratory study of presence, multimodal
The work was also supported by FCT-Austin grant Ref.
interfaces, and cross-modal sensory illusions,” Presence
PD/BD/142893/2018 and by FCT, reference
SFRH/BD/137169/2018, financed by the Human Capital Teleoperators Virtual Environ., vol. 10, no. 3, pp. 247–265,
Operational Programme (HCOP), under the European Social Fund 2001.
and national funds of the MCTES. [17] G. Jansson, M. Bergamasco, and A. Frisoli, “A new option for
the visually impaired to experience 3D art at museums: Manual
REFERENCES
exploration of virtual copies,” Vis. Impair. Res., vol. 5, no. 1, pp.
[1] D. Marques, Realidade Aumentada em Exposições de Museus - 1–12, 2003.
Experiências dos Utiliadores. Caleidoscópio - Edição e Artes [18] M. Carrozzino and M. Bergamasco, “Beyond virtual museums:
Gráficas, SA, 2018. Experiencing immersive virtual reality in real museums,” J.
[2] Tate Modern, “Modigliani VR: The Ochre Atelier – Behind The Cult. Herit., vol. 11, no. 4, pp. 452–458, 2010.
Scenes | Tate.” [Online]. Available: [19] R. Comes, “Haptic Devices and Tactile Experiences in Museum
https://www.tate.org.uk/whats-on/tate- Exhibitions,” J. Anc. Hist. Archaeol., vol. 3, no. 4, 2016.
modern/exhibition/modigliani/modigliani-vr-ochre-atelier. [20] F. D’Agnano, C. Balletti, F. Guerra, and P. Vernier, “Tooteko:
[Accessed: 16-Apr-2020]. A case study of augmented reality for an accessible cultural
[3] S. Deterding, D. Dixon, R. Khaled, and L. Nacke, “From Game heritage. Digitization, 3D printing and sensors for an audio-
Design Elements to Gamefulness : Defining ‘ Gamification ,’” tactile experience,” Int. Arch. Photogramm. Remote Sens. Spat.
2011. Inf. Sci. - ISPRS Arch., vol. 40, no. 5W4, pp. 207–213, 2015.
[4] J. Huizinga, Huizinga, J. (1949) Homo Ludens: A study of the [21] A. Reichinger, S. Schröder, C. Löw, S. Sportun, P. Reichl, and
play-element in culture. London, Boston and Henley: Routledge W. Purgathofer, “Spaghetti, sink and sarcophagus: Design
& Kegan Paul Ltd. Original edition, 1944. explorations of tactile artworks for visually impaired people,”
[5] A. Coelho and L. M. Costa, “The integration of augmented ACM Int. Conf. Proceeding Ser., vol. 23-27-Octo, no. October,
reality and the concept of sticker album collection for informal 2016.
learning in museums,” Commun. Comput. Inf. Sci., vol. 725, pp. [22] A. Reichinger, A. Fuhrmann, S. Maierhofer, and W.
107–115, 2017. Purgathofer, “Gesture-based interactive audio guide on tactile
[6] L. Santos, D. Pereira, P. Beça, and A. Coelho, “Aplicação móvel reliefs,” ASSETS 2016 - Proc. 18th Int. ACM SIGACCESS Conf.
para divulgação do património natural no turismo,” Tur. Comput. Access., pp. 91–100, 2016.
Desenvolv., vol. 27/28, pp. 1461–1473, 2017. [23] Y. Buonamici, F., Furferi, R., Governi, L., & Volpe, “Designing
[7] L. Santos, D. Pereira, R. Nóbrega, P. Beça, and A. Coelho, the architecture of a preliminary system for assisting tactile
“Jogos sérios digitais para a promoção do património natural – o exploration of bas-reliefs,” J. Des. Res., vol. 15, no. 2, pp. 110–
caso de um jogo de exploração de um parque na busca dos seus 127, 2017.
animais [poster + demo] {BEST DEMO},” Vj 2016, vol. 33, no. [24] D. Jain, “Pilot evaluation of a path-guided indoor navigation
2015, p. 2991, 2016. system for visually impaired in a public museum,” ASSETS14 -
[8] F. Candlin, “Blindness, art and exclusion in museums and Proc. 16th Int. ACM SIGACCESS Conf. Comput. Access., pp.
galleries,” Int. J. Art Des. Educ., vol. 22, no. 1, pp. 100–110, 273–274, 2014.
2003. [25] A. Meliones and D. Sampson, “Indoor blind navigator: A use
[9] Instituto Português de Museus, Museus e Acessibilidade. 2004. case for self-guided tours in museums,” ACM Int. Conf.
[10] A. Cachia, “Talking blind: disability, access, and the discursive Proceeding Ser., vol. Part F1285, pp. 17–20, 2017.