Immersive technologies are permeating our lives universally. Their applications can be observed in medicine, manufacturing, construction, entertainment, and various other human endeavours. They have also made their mark in the realm of culture and art, virtualizing, reconstructing, and modelling historical, architectural, or artistic content in virtual or augmented reality. This article discusses the concept of creating a mobile application with immersive technologies for the Kasteyev Museum in Almaty, Kazakhstan. It will include features for recognizing and classifying exhibits, as well as their visualization, 3D modelling, and presentation on a mobile device screen in augmented reality mode. The article provides an overview of similar applications, describes the concept of creating a mobile application, and examines the planned functionality.
Each new generation inevitably leaves behind a layer of heritage, which is then passed on to
descendants primarily in physical form: paintings, sculptures, literary works, architectural
objects, etc. Thus, the memory of culture and history is transmitted over hundreds of years.
However, new technologies allow for a fundamental change in the approach to transmitting
knowledge about culture and history, as the form of interaction between people changes.
Information and communication technologies (ICT) could not help but affect this sphere of our
lives. [
The cultural heritage of a country is a source of spiritual enrichment for society. The necessity
of its preservation for future generations is undeniable. However, in the present time, during the
period of globalization, the rapid development of ICT, acceleration of the overall pace of
development and life of society, questions of the spiritual development of individuals recede into
the background, and the problem of indifferent attitudes towards the history and culture of a
nation arises quite acutely, especially among the younger generation. However, only this
knowledge forms in the youth a respect for the past, awakens a desire to continue traditions and
encourages them to contribute to the history and culture of their country [
One of the sources of knowledge about the history and culture of a region is museums.
However, today museums exist in conditions of high competition. They are forced to compete for
their visitors in the leisure market. To increase the competitiveness of such cultural institutions,
and especially to attract the younger generation, it is necessary to use new platforms for
communication with the modern audience. In this area, approaches such as the development of
dynamic accentuated websites using advanced graphics, creating video tours of exhibitions,
mobile applications with embedded elements of augmented reality, and others, are actively being
implemented [
One of the new modern methods of interaction between museum visitors and exhibits is the
use of extended reality (XR) technologies – augmented and virtual reality. Augmented reality (AR)
allows virtual objects to be integrated into the real surrounding space. Virtual reality (VR), on the
other hand, completely immerses users in a virtual world. [
Currently, the research team at the International Information Technology University (Almaty, Kazakhstan) is working on developing a virtual museum in the form of a mobile application, which will feature exhibit recognition based on machine learning algorithms, as well as their visualization using augmented reality technology. The A. Kasteyev Museum of Arts in Almaty is a partner in this project. This article presents the concept of developing the virtual museum. Various aspects will be discussed in detail which needs to be considered when developing such an application.
A separate type of virtual museum systems is mobile applications using augmented reality
technology. They do not replace a physical visit to the museum, as is the case with virtual tours
using virtual reality devices such as headsets and goggles. On the contrary, such applications are
used directly within the museum itself. However, augmented reality technology provides users
with additional opportunities to explore exhibits, such as virtual reconstruction, volumetric
visualization, and others [
For example, during their research, Siang et al. [
The authors also emphasize that developers of mobile AR applications should focus on creating a more enjoyable and engaging user experience by adding gamification features based on rewards. Additionally, museums should provide clear and multilingual user guides so that museum visitors can easily learn how to use a mobile AR application.
In conclusion, the authors note that future research should focus on gender and age
differences and whether these differences affect user acceptance of AR applications. Literature
shows that the adoption of IT by individuals varies depending on gender and age, as evidenced
by studies on user acceptance of AR applications [
Chan and Ismail presented a mobile application called AR Labuan Museum to enhance the
experiences of visitors to the Labuan Museum (Malaysia). It features 11 panels or artefacts
related to the era of the British in Labuan, each accompanied by 11 types of augmented reality
content presented in the form of videos and 3D model structures. Pre and post-testing were
conducted involving 38 voluntary participants at the Labuan Museum. Participants were
required to download the augmented reality mobile application onto their mobile devices and
join the exhibition tour (Figure 2). In conclusion, the authors write that the application is capable
of enhancing visitors' learning outcomes, meeting their museum expectations, and improving
their overall experience [
Baker et al. presented a conceptual model of mobile augmented reality for interacting with
museum visitors with hearing impairments (MARHIME). Based on the results of expert
assessments, it is assumed that the conceptual model consists of two layers. The first layer
represents six elements, namely: Aesthetics, Usability, Interaction, Motivation, Satisfaction, and
Enjoyment. The combination of these six elements is shown in Figure 3 [
The second layer is more focused on the technology and architecture of MARHIME, consisting
of four main components, namely: Mobile Augmented Reality (MAR), Museum, Engagement, and
Hearing Impairment (HI). At this level, the technology is divided into two components. The first
component is the hardware required for developing mobile augmented reality, consisting of
mobile devices. The second component is the software, which includes: Vuforia, Unity3D, C++,
Android SDK/Java SDK, a target database, target tracking for AR markers, and multimedia objects.
Multimedia objects in this model consist of four elements: 3D models, text, video, and images [
In [
As newer generations are less interested in museum visiting having many digital entertainment tools that do not require personal presence traditional cultural institutions have to figure the ways to attract these people. One of the tools is the use of technologies such as XR, including AR. The general idea of a digital museum in the form of an AR application suggests increasing the following features: 1. Engagement; 2. Involvement; 3. Experience; 4. Interest; 5. Entertainment.
Personalization in the AR museum app tailors the experience to each visitor's unique interests and preferences. Through user profiles and preferences, the app can recommend specific exhibits, artworks, or historical artefacts that align with a visitor's taste. Additionally, it can provide indepth information, audio guides, or augmented reality experiences based on the user's previous interactions with the app. By creating a customized journey, visitors feel more connected to the museum's collections and are more likely to engage deeply with the content that resonates with them.
The AR in the museum app is not only about entertainment but also about education. It provides visitors with learning opportunities through interactive and immersive experiences. Users can access informative videos, 3D models, and detailed descriptions that provide context and background information about the exhibits.
Higher Interest
The functionality of Storytelling and Narratives transcends commonly accepted exhibition methodologies by filling up historical accounts with vivid life. While navigating the museum's offerings, visitors are introduced to comprehensive narratives that fill the displayed artefacts and exhibits with context, emotional resonance, and profound elucidation. It mimics the presence of a sagacious guide, augmenting each museum visit into an engaging and erudite encounter. This feature facilitates the revelation of concealed historical chronicles and undisclosed stories underlying each exhibit, all achieved through the transformative potential of AR.
Leveraging augmented reality technology, visitors are provided with a unique opportunity to interact intimately with museum exhibits. By virtually manipulating and reassembling ancient artefacts, scrutinizing historical documents through simulated dissections, or participating in meticulously rendered historical reenactments, the interactive learning feature effectively metamorphoses the museum into an arena for intellectual curiosity and dynamic exploration. This capability furnishes visitors with the means to undertake proactive roles in the unravelling of historical narratives, thereby fostering a profound and participatory comprehension of historical contexts.
The feature of Access to Hidden Information highlights the potential for ongoing scientific exploration and understanding of the items on display, enhancing the user's understanding of historical context, cultural significance and scientific significance. By making it easier to discover hidden information, the AR feature in the museum app encourages users to engage in a dynamic process of discovery, fostering a deeper understanding of the scientific and cultural aspects of each museum exhibit.
Entertainment
Within the Interactive Storytelling feature of this application, AR technology seamlessly interlaces immersive narratives into the exploration of museum exhibits, bringing the user into historical occurrences. The user assumes an active role in the unfolding narrative, effectively reanimating historical events. The interactive storytelling component reconfigures the museum visit into an engaging expedition, positioning the user at the epicentre of the unfolding narrative, thereby fostering enduring recollections and a heightened affinity with historical eras.
The Educational Games feature embedded within the AR museum application drastically changes the process of knowledge delivery. While conventional museum visits may occasionally appear as static and passive affairs, this feature introduces interactivity and depth, thereby inspiring visitors to new visits. Employing gamified encounters, participants are allowed to partake in cognitive puzzles, undertake challenges, and engage in pedagogical quests that combine enjoyment with new information.
Gamified Experiences capitalizes on augmented reality technology to introduce an element of entertainment to the educational journey. Every exhibit element corresponds to a distinct level to be unlocked, and each historical insight contributes to a point-based scoring system. The introduction of gamification elements such as leaderboards, achievements, and rewards, enhances the visitor's experience by making it more entertaining and fostering competitiveness. This approach ensures that the museum visit is memorable, and socially interactive, and encourages repeated visits.
Enrichment
Hands-On Exploration. For select artefacts, the app may offer highly detailed 3D models that can be manipulated in real-time: rotation of the model to view it from different angles, zooming in to see intricate details, and even tapping on specific elements to learn more about them. This hands-on approach provides a deeper appreciation for the exhibit.
The AR museum application underscores an emphasis on accessibility and inclusivity, with a primary goal of affording all individuals, irrespective of physical or sensory constraints. The application may offer a suite of features, including voice narration, adjustable text sizes, and the capacity to tailor colour contrast, which collectively serves to address diverse requirements. Furthermore, the application exhibits multilingual support, thereby extending the accessibility of the museum's content to a worldwide demographic.
Incorporating a "Personalized Experience" attribute, the AR museum application customizes the visit to the museum according to the distinct preferences of each user. Through the utilization of user profiles and preferences, the application can proffer recommendations for exhibits, guided tours, or content that goes along with the individualized interests of each user. Furthermore, users possess the capability to establish bookmarks and manage virtual collections of their own, thus providing an unforgettable museum visit experience.
Immersive exploration of three-dimensional representations can be achieved through the "Interactive 3D Models" feature. It allows observing museum artefacts using interactive 3D models. Whether visitors encounter intricate works of art or ancient archaeological relics, these interactive 3D models bring history to life, creating a richer and deeper educational experience.
Involvement
Through the implementation of Customized Tours, the AR application provides the highest degree of personalization, thereby augmenting the museum visitation experience. Users possess the capacity to craft an individualized route within the exhibition space, where they can selectively choose artworks and artefacts that align with their specific interests. The way it customizes the experience depends on what the visitor likes. It works for people who love history, and art or those who are curious about certain topics. This feature is designed to give attendees control to explore the museum at a personal speed, making visits more interesting and meaningful.
The use of AR in museum apps transforms the way people engage with exhibits through the innovative Interactive Scanning feature. Using a smartphone or tablet's camera, a person can scan QR codes, images, or real artefacts to access additional information. This technology turns static artworks into dynamic experiences. Visitors will hear historical stories from important figures and discover hidden secrets of ancient artefacts. This feature makes each exhibit more engaging, making museum visits not just educational but also incredibly captivating.
As a person explores the museum, he or she will come across a series of fun and interactive challenges and quizzes. These are designed to test thinking skills and how observant the person is. If a visitor answers the questions correctly, completes tasks successfully, or reaches specific goals, he or she will earn rewards, and badges, or get access to special content.
This feature allows challenging people and competing with fellow visitors, making the learning experience more enjoyable. It adds excitement and a sense of competition to museum visits, making it a fun and memorable adventure suitable for all ages.
Thus, the development of the proposed virtual museum will be based on this concept. The authors will consider all the features, discussed above and will integrate them into the future application.
The proposed project holds significant importance on a national scale. The digital museum application being developed as part of the project, utilizing augmented reality technology, offers new, contemporary ways for cultural institutions to interact with audiences. Moreover, it has the potential to elevate public interest in the historical and cultural heritage of the region, particularly among youth, through the use of augmented reality. The application's interface, which will include English alongside Kazakh and Russian, will facilitate the dissemination of historical information about Kazakhstan abroad. Furthermore, the future application can serve as a benchmark and example for the development of similar multimedia interactive systems for other museums in the country.
This research was funded by the Science Committee of the Ministry of Science and Higher Education of the Republic of Kazakhstan (Grant No. AP19676803).