Providing accessible ways to our cultural heritage is a key responsibility for modern societies. This paper outlines ideas for using new Augmented and Virtual Reality technologies at museums. First, we will give a short introduction on the latest developments of AR/VR technology and their inherent potentials for museums. Secondly, the ongoing R&D project “HoloMuse” will be introduced which aims at developing new forms of Augmented Learning for cultural heritage venues. HoloMuse is a collaborative effort of technology partners, academic institutions, and worldwide renowned museums to make the next steps towards real innovative and meaningful museum visitor experiences based on AR/VR technologies and cutting-edge learning theories. We will outline several deployment scenarios of AR/VR in museums giving a prospective on the future of cultural learning.
Cultural heritage institutions are not only responsible for
preserving but also for mediating the abundant treasure of
worldwide art, architecture, history, and technology. Visiting a
museum should be an inspiring experience. It should enable us
to go beyond our own horizon, to foster creativity and new ideas.
Thus, one main challenge is to provide ways for non-expert
visitors to gain a deeper understanding of museum exhibitions.
This is the field of „museum interpretation“, which for a long
time was narrowed down to wall labels and museum tours, later
on extended by „audio guides“. In recent past, mobile
technologies like smartphones, tablets, and mobile apps came up
offering richer content, explaining museum exhibits with
multimedia such as text, images, audios, etc. However, the
traditional concept of museum interpretation remained, i.e. to
conceptualize the visitor as a learner which has to be „fed“ with
information. This passive understanding of learning has
massively been refuted by learning theory and educational
sciences (see e.g. [
Augmented Reality (AR), Virtual Reality (VR) and Mixed
Reality (MR) offer unprecedented new possibilities for inspiring
experiences as well as learning from and with art and culture as
well as for curating and designing exhibitions [
With more affordable VR devices such as Oculus Rift, Samsung VR or Google Cardboard VR immersive experiences are on the rise as well as massively adapted in current media art2. Museums are starting to take up this technology in order to give their audience more immersive experiences and learning opportunities3. Where for VR dedicated devices are used for providing the experience, currently most AR applications are geared towards smartphones and tablets as end devices. But the computer animation works) and for editorial content (e.g. “Zero Days VR” by Scatter or “Out of Exile” by Nonny de la Peña / Emblematic Group) 3 see [9]; e.g. “We are like vapours” by Jeffrey Shaw, Sarah Kenderdine & Cédric Maridet or experiences designed by Woofbert VR really interesting jump in technology is the introduction of ARglasswear. Big manufacturers such as Vuzix or Epson already have been working on the development of market-ready AR devices that are currently mainly used in industry. The most advanced AR-glasswear at the moment is Microsoft Hololens. The breakthrough of more market-ready devices for mass market is expected in the upcoming years and will lead to broad usage of AR head mounted applications.
The following questions drive the R&D project “HoloMuse”: How can AR and VR be employed for individual guiding and mediation, for cultural learning, as well as for personal mediation? How can these modes enable co-creative knowledge generation of and with visitors? How can it support group activities, for example of researchers and students involving physical items, cultural data as well as research data? And how to keep the focus in the gallery on the exhibited items and their context and not on a technological device? What kind of novel experiences with and of cultural data can be created that foster sustainable cultural learning? These questions led to the proposal of several use cases and innovative mediation modes for co-creative knowledge generation and cultural learning.
The R&D project HoloMuse will be carried out by Fluxguide
(http://www.fluxguide.com/) and several partners4. Fluxguide
already put many efforts in developing inspiring learning
environments to activate visitors, to interact and reconstruct art,
science and knowledge worldwide [
HoloMuse expands the experience inside and outside of museums with alternative and other realities (in time and place), and with visions for the future or fictive realities. The solution will also be able to integrate research data from the Digital Humanities directly into the mediation process.
The following paragraphs will outline concepts of the main HoloMuse “Modes” and give an overview of possibilities provided by AR/VR based museum mediation. 4 Renowned museums like Albertina or Deutsches Technikmuseum, technology partners like Microsoft, and scientific partners like the University of Applied Arts Vienna.
The mode HoloMuse.immersive.spaces provides users with engaging, immersive experiences inside and outside of the museum. By using digital and immersive storytelling – so-called explicit information mediation [8] – not only a transfer of information takes place. The physical space is augmented with virtual elements and visitors are sent on a journey into “another world” – for example through a reconstruction of historic places, a living vision for the future, or via bringing alive the fictive world of paintings or literature.
Thus, this mode allows to transform every possible existing location into a learning space, augmenting the physical location with new and old knowledge, stories, possible histories, presences and futures. HoloMuse allows to immerse into the depicted fictive world of an artwork. For example, a character of a painting could step out into the gallery space and act as copresent gallery guide introducing the (hi)story of the artwork and context of the depicted scene. The fictive character becomes an „authentic“ authority to give interactive and contextual information and to represent the artistic idea (see Fig. 2). Moreover, one can take a virtual fieldtrip in the physical world and explore, for example, a virtual reconstruction of a monument that has become merely a ruin in present times (see Fig. 3). This allows immersive experiences of a possible past. Also, future scenarios can be experienced: one could for example visualize the transformation of an existing landscape due to climate change 100 years from now, and therewith illustrate the meaning and specific interpretations of research data. Therefore, this mode can also be useful for science communication and vividly mediating research data. The experience of the visitor’s physical environment is altered and formed.
Humans learn from humans. Therefore, HoloMuse will be
integrated with human mediation and guided tours. During a
guided tour through the physical museum space a guide may
pass around virtual objects to individuals or a group. This
enables visitor interaction e.g. with closer explanations of fragile
museum artefacts which otherwise could not be touched in a
museum environment (see Fig. 5; a similar feature was also
presented by [
This mode allows museum visitors to explore exhibitions with an end device like smartphone or glass wear. Users receive interactive, 3D-multimedia information about key objects and navigate within the exhibition space by gestures. An example could be enabling visitors to read hieroglyphs and provide translations to them in relation to old Egypt objects. Virtual content-pins on objects reveal detail information and therewith foster personal interpretation. This may prompt the visitor to fill mental gaps, to ask questions by engaging with the exhibited object or trigger associations and memories related to visual hooks or entry points (see Fig. 4). A learning mode enables to trigger interactive challenges and explorative task while directly engaging the visitor with the original physical or a digitized virtual object (see Fig. 6).
The inclusion of interactive exercises in interaction with the virtual or physical object fosters learning processes for groups and individuals (see Fig. 6). It also offers promising opportunities for the important area of education mediation, e.g. for museum school class visits. Visitors could join the tour in the museum but also in virtual co-presence.
Especially for science communication, science centers and
technology museums, a virtual laboratory enables to conduct
interactive experiments in VR as well as AR. Simulations of
complex processes could be observed and tested in physical or
virtual co-presence in an exhibition space, a museum laboratory,
a university or at home. Instructed or independently. This opens
up a whole range of possibilities for interlinking digital
humanities research directly with museums, their collections,
exhibitions as well as informal cultural learning. Latest research
becomes interactively available to life-long learners, involving
the visitors in co-creative knowledge generation processes
around the data on multiple devices (see also [
In HoloMuse.curating.room users design their own exhibitions on basis of digitized cultural data from the museum collection or favorites collected during the museum visit on a mobile (see Fig. 7).
One scenario could be preceded by a real museum visit where users mark their favorite objects on a mobile museum guide. At the end of the exhibition the visitor receives AR glasses, e.g. a Microsoft HoloLens, and places objects in a “curating room” – also in collaboration with others. For visitors, the design of such a personal exhibition increases the active engagement with cultural artifacts. It could also be taken home as memory of a museum visit which can be displayed on smart devices as well as AR and VR devices and shared with friends and family.
For museum professionals, this “curating.room” tool addresses the issue that it is very hard to design an exhibition at a specific venue, taking into account the specific spatial situation of a gallery space as well as relations between objects and conservatory prerequisites. Even if exhibition designers are onsite for the design process, it takes huge efforts to do iterative positioning of artworks and other exhibition items. Smart use of new technologies may help here as well.
At the same time a virtual exhibition can also enable a
reexperience of historic exhibition settings [
The scenarios above only outline a first glimpse of possibilities, opportunities, but also challenges, of utilizing AR/VR technology for cultural interpretation. The concepts are based on the assumption that AR devices will be widely adopted by the mass market in near future outside of purely industrial applications. This is necessary to go beyond mobile devices as end-devices for AR applications and therewith enable a more immersive and seamless experience. First results show that it will be crucial to use those technologies to enhance cultural exhibitions, not to substitute them. That is why especially AR technology seems very promising. This is intertwined with the challenge that interactive AR/VR content really should give added value and new learning possibilities, and therefore should be integrated and backed in sound didactical and learning concepts. Thus, the project HoloMuse will go on focusing on both technological and conceptual challenges, together with museum professionals and technology partners.