Nowadays, the world is increasingly trying to exploit the advantages of XR technologies in everyday life. People have started to become familiar with XR technologies and companies, institutions and even governments are trying to exploit this popularity to provide personalized services to their audience. In the case of museums, incorporating XR technologies may lead to better engagement with their audience by providing distance access to their collections or facilitating a better experience in the museum. The potential of those technologies has been exploited within museum environments for preservation purposes, facilitation of everyday functions like ticketing, or even promotion of the democratization concerning accessibility. The focus of this work is how museum visits can be inclusive using XR technologies to promote wellbeing in modern cities.
A lot of work is targeting the use of technology in the everyday life of a museum [1][2][3]. There is also much work for online users who need a virtual tour for exhibitions [4][5][6][7]. However, there is a limited amount of research that focuses on people with impairments and in most cases, applications are focused on only one impairment type. In [8] they state that most mobile AR apps for museum visitors do not support hearing-impaired visitors even though this community accounts for over 5% of the world's populace. Furthermore, in [9] they state that only 20 of the 90 low-vision studies recruited participants with low vision. Most of the research in the field is based on simulation of low vision. Also, in [10] they state that according to the US Center for Disease Control and Prevention, 25% of US adults have a disability that "impacts major life activities".
In this work, we are dealing with the user needs of people with impairments concerning museum visits and how those needs can be facilitated by XR technology functionalities to enhance the visiting experience in the museum or provide distance access to their collections. In this context, there are many questions to be answered: What are the XR technological tools that museums use to facilitate their visitors (distanced or in situ)? Is this technology equally helpful for all their visitors? What about people with impairments?
Milgram and Kishino defined a lot of concepts related to computer-generated realities [11,12]. Augmented Reality (AR) is about providing a digital layer of information to the vision [13] or even to other senses [13,14]. Virtual reality (VR) "produces a digital environment in which visual perception, sense of hearing, and sense of touch are highly similar to those of actual environment within a certain range" [15]. Also, VR is there to engage several human senses [16]. Mixed reality (MR) is not a clearly defined term [17,18]. In MR, the digitally displayed information tends to be an integral part of the real world. The user has the option to treat that object like it is real. For a common understanding, we use the following definitions [19]: VR enhances our presence and interaction with a computer-generated environment hiding from the user the real world, and AR enhances our perception and understanding of the real world by superimposing virtual information on our view of the real world, and MR is when real and virtual environments blend. Extended Reality (XR) is used to describe all these realities. However, in [20] they state that X should stand for any reality format. Going beyond the XR technologies is the metaverse [21][22][23][24]. We will adapt the definition from [21], where the metaverse can be seen as a virtual environment blending physical and digital, facilitated by the convergence between the Internet and Web technologies and XR. XR offers all the technological tools that are necessary for someone to build new worlds. It also offers devices for entering those worlds. Technologies related to XR [14,16,19,[25][26][27] are constantly evolving [28]. We may have some categories related to the ability to use these technologies in an indoor or outdoor setting: AR/MR can be indoor or outdoor while VR is mainly set to be indoor. Mobile AR can be categorized depending on the technology used to trigger the superimposing [26]. Another category is related to the level of immersion which depends on the type of device and the type of reality. For example, we cannot expect a fully immersive experience with the use of a desktop device or with the use of an AR app. The devices used for XR are mobile devices, desktop displays [16], HMDs (like Google cardboard [29] and Meta Quest Pro [30], CAVE [31], holographic, or smart glasses. Also, new types of devices that could offer a deeper sense of impressiveness have started to become common (like Haptic Feedback). AR devices like HMDs or smartphones [25] are used because they are portable and can facilitate the display of information in the real environment. Caves or desktop computers can be used for VR, while HMDs can be used for all XR systems. The most common devices for MR are see-through HMDs [32]. XR applications are created using dedicated development kits (commercial or not). There are many developments in software for VR (like [33][34][35][36][37]). For AR we have software like [38][39][40][41][42]. For MR we cannot identify any exclusive list of toolkits or devices, but the abovementioned software can be used in some cases [43].
According to [44], "in the context of health experience, an impairment is any loss or abnormality of psychological, physiological or anatomical structure or function". This definition also includes losses that occurred during the person's lifetime. In [45], it is stated that disability "is any restriction or lack (resulting from an impairment) of ability to perform an activity in the manner or within the range considered normal for a human being" and a handicap is a "disadvantage for a given individual resulting from an impairment or a disability, that limits or prevents the fulfilment of a role that is normal (depending on age, sex, and social and cultural factors) for that individual". A brief list of impairments includes (according to [45]): 1. Intellectual 2. Other psychological 3. Language 4. Aural 5. Ocular 6. Visceral 7. Skeletal 8. Disfiguring 9. Generalized, sensory, and other impairments. In [46], they state that impairments can be related to vision, hearing, mobility, cognitive, or speech. In all cases, the impairment might be temporary, situational, or permanent.
In a study [47], participants highlighted that "the biggest issues in XR technology are overuse of motion tracking, lack of flexibility with other hardware, lack of customization ability, lack of compatibility with assistive technology, lack of visual, and audio, and haptic cues and an overall lack of consideration for most kinds of disability needs". In this work, we mainly focus on the requirements that are required to assist impaired people. To understand the challenges that should be confronted when an XR system is designed, [47] determines some directions to be aware of:
• Understanding of specific diverse user needs and how they relate to XR.
Identifying modality needs that are not obvious but required.
Suitable authoring tools for supporting accessibility requirements in XR.
Using languages, platforms, and engines that support accessibility semantics.
Providing accessible alternatives for content and interaction.
Providing specific commands within the VR environment, which assist with navigation to support different modalities.
Using virtual assistive technologies to provide non-visual feedback. It is obvious that each impairment type needs to be treated in a special way. In [48] they also list needs together with requirements that are needed to be served to create a system that is accessible for people without excluding a type of impairment. In Table 1, we list these needs along with the proposed system requirements that provide help for impaired persons. Research that is currently available does not deal with all the types of impairment together. In [8], they deal with hearing-impaired persons, and they conclude that major elements of engagement for a mobile AR application are Aesthetics, Curiosity, Usability, Interaction, Motivation, Satisfaction, Self-Efficacy, Perceived Control, Enjoyment, Focused Attention, and Interest. [
In Table 2, we present a list of indicative research related to applications facilitating impaired persons during museum visits, along with app features that exploit XR functionalities. In all cases, these applications try to invent ways to transmit the information that cannot be perceived due to the impairment, using another channel. For example, they provide more acoustic information to low-vision persons. AR-based digital storytelling system without the use of images, to improve the experience of visually impaired visitors in the heritage site. AR, tangible interface, binaural audio, smart glass.
In this work, we dealt with systems or applications that are specially created for impaired persons. Most of the apps target only one impairment, providing ways to help these people have a better experience when visiting a museum. Furthermore, we listed several users' needs and requirements for any XR application focusing on impaired people. These requirements should be followed by application designers to promote inclusion. However, this does not happen today. We could hardly see an implementation of one or two of these requirements in applications that are for the general population. We believe that the research on the use of technology that assists people with impairments should be done with the impaired people and not using simulations. In [47] they mention that "For AR and VR to become truly accessible to persons with disabilities, then those who are experiencing disabilities must be included in the development. In the words of disability activist James Charlton, "Nothing about us without us.". In the future, the systems that are created for enhancing the museum experience should adapt to the personalized needs of the persons, impaired or not.
This research has been co-financed by the European Regional Development Fund of the European Union and Greek national funds through the operational program Competitiveness, Entrepreneurship, and Innovation, under the call RESEARCH-CREATE-INNOVATE (project code: T1EDK-2-01392).