Museums continue to exert fascination in their visitors. However, the new generation of visitors expects museological experiences that promote their active participation. It is in this context that games and the gamification of such experiences capitalize on experiential learning by experimenting and enacting with in-game embedded artefact surrogates and know-how. In this article, we present four distinct projects that aim to enhance the visitors' experience in museums and green spaces, and also their effectiveness in informal learning. In the first project, gamification is used in combination with Augmented Reality to provide a more engaging experience in a boat museum. The drive of this experience is the metaphor of the stickers album collection to unleash the relevant information of the key-artefacts of the museum collection. The second and third projects focus on the use of pervasive games, more specifically location-based games, to enhance the visitors' experience and informal learning in a natural park and a botanical garden, respectively. The second project presents the concept of a mobile app for outdoor nature experiences. The drive for the experience in the third project is the narrative that intertwines specific locations in the botanic garden and a story inspired by the same place. Finally, in the fourth project, we focus on the potential of technology to provide accessibility in museums for people with special needs or disability, focusing more specifically on blind visitors.
Index Terms: •Human-centered computing~Ubiquitous and mobile computing~Empirical studies in ubiquitous and mobile computing •Applied computing~Education~Interactive learning environments
The use of digital technology can undoubtedly assist and create new experiences for visitors in museums while promoting their active participation. The use of mobile devices during museum visits has great potential for informal learning, playing an important role in finding, reading and interpreting museological artefacts while being non-invasive. However, the continually evolving hardware asks for a constant search for new and alternative ways to use such devices, which in turn asks for the design and development of new software. An example is the museum guide being assisted and sometimes replaced by diverse smartphone applications.
Several museums are already using virtual and augmented reality technologies to provide new experiences to their visitors. The Smithsonian museum’ AR Bone Hall Exhibit [1] and the project Modigliani VR: The Ochre Atelier [2] from the Tate Modern Museum, are some examples. Nonetheless, one of the digital challenges museums face today in this context is the simultaneous navigation and exploration of both these virtual and physical spaces. Mixed reality technologies aim at bridging such gap. However, every context has its very particular characteristics that need to be addressed with specific tools and designs in order to promote participation and stimulate creative practices in museological spaces.
In the broad field of the Ludification of Culture, we can focus on the area of games or the area of playful interaction [3]. In this particular work, we focus on the more structured use of games, both full-fledged games and gameful design.
This document is divided into four main sections, presenting
four examples of applications in museums. They were developed
by members of the Graphics, Interaction and Games research
laboratory, under the same supervision. The projects took place
between 2016 an
Museums consolidate the heritage of a country or a given region, interlinking history, art, science and the territory. As an inalienable landmark in the history of Portugal, the first example is about Maritime Museums, and most particularly, the exploration of traditional boats.
We propose a gamified approach based on the concept of
sticker album collection and its integration in an Augmented
Reality (AR) mobile application. The concept of sticker album
collection is quite familiar to most people, mainly from their
youth, and is the central dynamic of the gamification design,
engaging the learner to collect more stickers and progress in the
exploration of the museum. As a pervasive solution, we do not use
physical support, but instead, a mobile application to provide the
learning experiences by uncovering the stickers using AR over the
museum collection, in order to enhance the knowle
One of the main principles of gamification is the engagement cycle. Thus, we have developed a progressive activity that engages the player into small iteration cycles in order to provide mastery. Each boat has a predefined number of stickers that, being initially drawn as empty slots, show the number of challenges to solve (this number can vary) in order to collect the boat. Figure 1 shows the typical iteration cycles for a specific boat.
The first sticker provides a brief description of the boat and challenges the user to identify a specific component of the boat (figure 2). The user will need to find it using the AR tool. In the case of difficulty, a “help button” can be pressed, and a text hint will be provided (about its location). If the user is still not able to find it, another more detailed hint (like a drawing of the boat component) will be additionally provided. This provides mastery to proceed.
As the user identifies the boat component, additional information is provided in the AR layer, showing specificities and details on top of the original museum boat. This launches the second challenge, a simple quiz that assesses some information provided previously in AR or on the site. After collecting this second sticker, the boat image is complete in the album, and a new multimedia content is available as a reward: a threedimensional model of the boat and a small animation of how the boat is constructed. Thus, the user progresses through each level by surpassing each challenge, gaining access to digital stickers (and associated multimedia content) as a reward, making the exploration of the museum and the real apprehension of knowledge more attractive. The AR layer has a significant role in making the connection with the heritage of the museum.
In conclusion, we propose the concept of the sticker album collection integrated with AR to improve the museological experience. The museum heritage is the anchor point that provides the epic meaning of the experience, and informal learning is fostered on the challenges associated with the locked stickers. At the same time, progression is achieved by collecting the missing stickers. The AR layer is the key element that provides the connection between this heritage and the gamified learning experience. 3
Digital mobile games can offer engaging experiences and increase communication effectiveness concerning cultural and natural heritage. Location-based serious games allow us to overcome the barriers of space and enables their players to explore the real world around them while playing a game with educational or cultural purposes.
A prototype of a mobile app was developed for the Gaia’s Biological Park [6], in Portugal, to help in making science communication engaging for visitors and in improving the overall outdoor visitor experiences. The concept is to have a mobile app to promote and disseminate the natural heritage in a tourism context, with a map, the user’s location in real-time, points of interest with geolocalized information (GPS), and small serious games about the green spaces visitors are experiencing. This project started in 2016.
One of those games, the virtual animal detector [7] (figure 3) is a “treasure hunt” and collection game about the animals that live in freely in the park. Some of them are difficult to see in the wild on account of their behavioural habits. With this game, visitors can learn more about these species. Players must explore the park to find all the virtual game animals and collect pictures of them all until they complete the collection. An in-game radar detects the supposed location of a virtual animal when the player is near that location and displays the information on that species. When discovered, a species gets registered in a collectables board, and some information about it can be consulted on-demand.
Another example is the game of the jay [6], that tries to communicate the dynamics of an oak tree forest ecosystem through its gameplay. The player controls a jay that collects and plants acorns. To make the sprouts grow, the player must collapse clouds to make it rain. That also puts down wildfires. To win the player must completely reforest the scenery.
These games’ goal is to amplify and enrich the visitors’ experience, augmenting our perception of the natural world so we can better understand it and protect it. Although the prototype was designed for the Gaia’s Biological Park, its concept can be adapted and applied to other parks, zoos, botanical gardens and other green spaces. Presently, the AR implementation into this app concept is being studied, with some promising preliminary results.
The University of Porto’s Botanical Garden is a place interspersed by Literature, Botany and History. The garden was home of the celebrated Portuguese writer Sophia de Mello Breyner Andresen, hence a place with significant influence on her work. In our discernment, a visit to the Botanical Garden is not complete without acknowledging these different facets as complementary dimensions that characterise this site.
With this in mind, we developed an experience for visitors to be acquainted with those facets in the form of a location-based game that articulates educational content in Botany with Sophia’s famous story “O Rapaz de Bronze” (the bronze boy). In this game (that resorts to GPS), players progress as they encounter particular Points of Interest (POI), moving in a specific sequence between them by solving challenges hinted from previous successful steps. In each POI, players are confronted with challenges that can be solved by inspecting their surroundings, ergo creating a relationship between Literature and Botany, all found in situ. We found this relationship to be crucial for developing player engagement with the locale, and to instigate location-awareness in players.
Exploratory visits with the garden staff were very welcome and necessary to build a proper experience. During the day of testing,1 we resorted to participant-observers for collecting information from participants while they were playing, acting as facilitators for solving technical issues and identifying unexpected problems in situ. Through these methods and by analysing player surveys, we concluded that players were engaged with the experience, that was well-accepted amongst families and groups of children and teenagers. For them, learning through a location-based game proved to be a fun and enriching. However, specific interaction design issues arose for our consideration in future work.
In conclusion, navigating in the Botanical Garden as a museological space requires visitors to be attentive to their surroundings. It is in those surroundings that is the motive for their visit, for their presence. Operating a mobile application in this context needs to be an experience that does not alienate the 1 This pilot was developed within the scope of the project BEACONING: Breaking Educational Barriers with Contextualised, Pervasive and Gameful Learning. This case study was tested on the 10th of February, 2018. Beaconing was an H2020 project involving 16 universities and partner companies, INESC TEC included, led by Coventry University (http://beaconing.eu/). interactor/ player from the locale itself, but one that promotes a stronger bond between them. We found that one way to do that is to use that mobile application to focus their attention on the physical world and not on their smartphones (or other devices), be it for navigation, information seeking activities or for moments of contemplation. 5
Despite the growing concern about increasing accessibility for visitors with visual impairments [8], [9], [10], [11], only 5,5% of blind and visually impaired people visit museums in Europe [12]. Patrons with total and partial visual impairments represent a significant potential market for museums [13], and positive economic effects are expected with the implementation of assistive technologies to promote access to exhibitions [14], besides enhancing their experience of the visit.
Haptic interfaces can take advantages of the dynamic nature of the kinesthetic sense and generate forces, allowing the exploration of virtual copies of museum and gallery collections [15] [16]. The Museum of Pure Form project allowed visitors to touch and experience the properties of sculptures belonging to the collection of multiple European Museums using an arm exoskeleton and a haptic interface for fingers [17], [18]. Visitors are required to pick up a pen of the device in order to interact with the Geomagic Touch System [19] and to allow haptic experiences of objects from the National Museum of Transylvanian History of ClujNapoca. Another project, developed in the context of the AMBAVis project and, in collaboration with Manchester Museum and Gallery Belvedere, allows patrons to explore artefacts, augmenting the experience by providing vocal information about the object material and characteristics [14].
Other museums are using 3D printing technologies with embedded sensors to provide digitally augmented touch replicas. They offer tactile and hearing multisensory experiences that help the visually-impaired visitors formulate mental images of the objects while providing crucial contextual information, such as the Tooteko project [20], developed for the Correr Museum. Those technologies are being used as well to reconstruct the object at the date of its origin, so visitors can learn about how an Egyptian Cat Sarcophagus was 2500 years ago [21].
Contrary to digitally augmented touch replicas, gesture-based interactive tactile reliefs do not have physical sensors attached to detect users’ activities. Instead, their interaction is tracked by computer-vision or other non-intrusive systems, in order to enrich tactile reliefs with audio or additional multisensory information. Researchers developed a gesture-based interactive audio guide for the interpretation of the painting The Kiss [22] and to assist the tactile exploration of the painting Madonna with Child and Angels [23].
Besides helping to create new ways for disabled visitors to know about museum collections interactively, technology is also being used to provide them with better physical access, indoor navigation and wayfinding in such environments. The result is an enhancement in their ability to move independently through different galleries, without the aid of guides or escort. As an example, a mobile application was developed at the National Science Center to assist sightless visitors’ navigation by presenting step-by-step audio instructions [24]. A similar interface was proposed for the Museum of the Lighthouse for Blind [25]. 6
Museums maintain the same epic meaning that attracts visitors. New media and technology can complement and not replace the mission of these institutions. The presented projects follow different lines of research for creating frameworks for the design and the development of AR applications and games. These key studies have contributed with insights on how to make better or more enriching museological experiences. The visitors’ experience should be the main drive for technological breakthroughs in museums, and games, gamification, and mixed reality technologies can provide improvements in such experience. However, these technologies must be balanced, so they will not distract the visitors from the museum collection but, instead, enhance their experience with these artefacts. Furthermore, these new technologies promote the inclusive museum where all visitors can have a great experience regardless of any disability or particular needs.
The BEACONING research project, has received funding from the European Union's Horizon 2020 - The EU Framework Programme for Research and Innovation 2014-2020, under grant agreement No. 687676.
The researchers are supported by the Operation NORTE-08-5369FSE-000049 supported by Norte Portugal Regional Operational Programme (NORTE 2020), under the PORTUGAL 2020 Partnership Agreement, through the European Social Fund (ESF). The work was also supported by FCT-Austin grant Ref. PD/BD/142893/2018 and by FCT, reference SFRH/BD/137169/2018, financed by the Human Capital Operational Programme (HCOP), under the European Social Fund and national funds of the MCTES. [Online].
Available: https://www.tate.org.uk/whats-on/tateDesign Elements to Gamefulness : Defining ‘ Gamification ,’” 2011.
J. Huizinga, Huizinga, J. (1949) Homo Ludens: A study of the play-element in culture. London, Boston and Henley: Routledge & Kegan Paul Ltd. Original edition, 1944.
A. Coelho and L. M. Costa, “The integration of augmented reality and the concept of sticker album collection for informal learning in museums,” Commun. Comput. Inf. Sci., vol. 725, pp. 107–115, 2017.
L. Santos, D. Pereira, P. Beça, and A. Coelho, “Aplicação móvel para divulgação do património natural no turismo,” Tur.
L. Santos, D. Pereira, R. Nóbrega, P. Beça, and A. Coelho, “Jogos sérios digitais para a promoção do património natural – o caso de um jogo de exploração de um parque na busca dos seus animais [poster + demo] {BEST DEMO},” Vj 2016, vol. 33, no. 2015, p. 2991, 2016.
F. Candlin, “Blindness, art and exclusion in museums and 2003.
Instituto Português de Museus, Museus e Acessibilidade. 2004. A. Cachia, “Talking blind: disability, access, and the discursive 20, 2013.
European Commission, “Memo: 19 destinations across Europe win awards for accessible and sustainable tourism,” no. K. Dash and G. Grohall, “Economic impact of creating and exhibiting 3D objects for blind and visually impaired people in
S. Mesquita and M. J. Carneiro, “Accessibility of European museums to visitors with visual impairments,” Disabil. Soc.,
Project AMBAVis, “Inclusive technologies in museums,” 2017. J. P. Fritz and K. E. Barner, “Fritz, Jason P.; Barner, Kenneth, E. (1999). "Design of a haptic data visualization system for people with visual impairments,” IEEE Trans. Rehabil. Eng., vol. 7, no. 3, pp. 372–384, 1999. environments: An exploratory study of presence, multimodal interfaces, and cross-modal sensory illusions,” Presence Teleoperators Virtual Environ., vol. 10, no. 3, pp. 247–265, 2001. 1–12, 2003. the visually impaired to experience 3D art at museums: Manual exploration of virtual copies,” Vis. Impair. Res., vol. 5, no. 1, pp. M. Carrozzino and M. Bergamasco, “Beyond virtual museums: Experiencing immersive virtual reality in real museums,” J.
R. Comes, “Haptic Devices and Tactile Experiences in Museum A case study of augmented reality for an accessible cultural heritage. Digitization, 3D printing and sensors for an audioW.
W. Purgathofer, “Spaghetti, sink and sarcophagus: Design ACM Int. Conf. Proceeding Ser., vol. 23-27-Octo, no. October, 2016. 273–274, 2014.
A. Meliones and D. Sampson, “Indoor blind navigator: A use case for self-guided tours in museums,” ACM Int. Conf.