Use of Analytical Hierarchy Process in Scenarios Design for a Digital Museum with XR components Ihor Bodnarchuk, Yuriy Skorenkyy, Taras Kramar, Oleksii Duda and Vyacheslav Nykytyuk Ternopil Ivan Puluj National Technical University, 56 Ruska St, Ternopil, UA46001, Ukraine Abstract Problem of user-oriented scenarios design for a digital museum is addressed with use of Analytical Hierarchy Process. Visitor’s motivations as well as characteristic features and restrictions inherent to XR technologies are considered. A set of scenarios focused at different types of visitors is developed for the digital museum of Ivan Puluj. Keywords1 Interaction scenario, virtual environment, analytical hierarchy process 1. Introduction Nowadays, digital transformation is not only the general trend but a recipe to overcome the most pressing problems of humanity including climate change and sustainability challenges, food and resources scarcity, income inequality. All industries and spheres of life undergo digitalization, which may differ considerably for modern types of activities comparing to traditional ones. Museums, which may be considered the memory devices of humanity, transform as well. First of all, the volume and variety of information produced by civilization is enormous and shows a further exponential increase. The only form capable of storing this amount of information is the digital one. Therefore, archives’ and museums’ collections have to be digitised. Second, a phrase ‘visiting a venue’ has different meaning in our information age than it had a century ago. Availability of distant resources and reduction of barriers for visiting a distant place virtually creates new situations for museums. Thirdly, cultural sector suffered greatly from pandemic restrictions in the last few years which makes it an urgent need to go virtual even for museums that are designed for a local audience. The present study focuses on application of Analytical Hierarchy Process (AHP), as a method for decision making support [1], to design optimal scenarios of a visitor interaction with a museum exhibition. In the case of digital museum, alternatives are not just exhibition rooms or stands, but also the very forms of interaction, namely visual or auditory, active or passive, 2D or 3D, immersive or augmented, individual or collective, etc. A detailed analysis of scenarios to be used for environments with eXtended Reality (XR) elements, in different sectors of the Milgram continuum [2] are analysed in chapter 2 to define alternatives for AHP. The most important and obvious motivations for visiting exhibitions are curiosity and educational needs. Design of exhibition, choice of the items and sequence of their display are to be purposeful to support both goals. Secondary motivations are cooperation and coherence, but there are more of motivations to be included into scenarios design. Motivations of museum visitors are discussed in chapter 3 to identify criteria for alternatives evaluation. In chapter 4 the AHP is used to evaluate and chose scenarios for particular use cases and chapter 5 is devoted to the conclusions. 1 ITTAP’2022: 2nd International Workshop on Information Technologies: Theoretical and Applied Problems, November 22–24, 2022, Ternopil, Ukraine EMAIL: bodnarchuk.io@gmail.com (A. 1); skorenkyy.tntu@gmail.com (A. 2); kramartar18@gmail.com (A. 3); oleksij.duda@gmail.com (A. 4); slavikvv89@gmail.com (A. 5) ORCID: 0000-0003-1443-8102 (A. 1); 0000-0002-4809-9025 (A. 2); 0000-0001-8060-0169 (A. 3); 0000-0003-2007-1271 (A. 4); 0000-0003- 1547-8042 (A.5) ©️ 2022 Copyright for this paper by its authors. Use permitted under Creative Commons License Attribution 4.0 International (CC BY 4.0). CEUR Workshop Proceedings (CEUR-WS.org) 2. Scenarios for eXtended Reality environments Recently, there is a growing need to realize potential use cases (scenarios) and contexts in which mobile augmented reality can be used, as it is important to evaluate initial service concepts with potential end users. It was found that the level of technological orientation of the users has a strong influence on the overall perception of the scenarios. Particular attention should be paid to evaluating AR scenarios at an early stage of development before any efforts are made to implement the services, and to identify potential use cases for future mobile AR services and understand their overall value to the user. When comparing the scenarios with each other, it was found [3] that the most valuable mobile AR services were those that demonstrated pragmatic utility for the user, for example, by saving time and effort. This became evident both in the evaluations of individual scenarios and in the overall evaluation of AR. It turned out that AR content offers the user rich and contextually relevant information for special needs in everyday mobile life. Although augmented reality (AR) technology has entered many market and knowledge areas such as gaming and leisure, it remains quite limited in the digital heritage. The paper [4] emphasizes the conceptual importance of AR as a vital environment in museums, especially when AR is combined with gamification techniques in the same application. The combination of augmented reality and gamification serves as a means of supporting the engagement of visitors in the outside world. In particular, it is suggested to choose a certain location of the museum as the starting point of the educational game. The player has the option to publish their achievements in the game on websites or social networks, which allows the user to gain a positive experience from learning about cultural history. Similarly, work [5] suggests the use of additional game and educational elements in the basic AR program to improve the overall experience of the museum visitor when familiarizing with the exposition. Augmented reality experiences in a museum setting can fit well with the user's environment, physical abilities and perceptions, and the user's position and movement in 3D space. In addition, AR services can be provided with minimal distraction and physical effort. In conclusion, AR technologies are mature enough to be standardized for museum use, while audiences appear ready to take advantage of the associated enhanced museum experiences to maximize both user satisfaction and learning outcomes. Paper [6] presents results related to the definition of a methodology that combines augmented reality (AR) with semantic techniques to create digital stories related to museum exhibitions. Unlike traditional AR approaches, elements of the real world, namely the content of the museum exhibition, are supplemented with additional input data. The methodology is based on the theory of cultural mediation and is based on a set of ontologies aimed at modeling a cultural resource and its relationship with external multimedia objects and resources. To provide simple tools for creating museum narratives, the methodology uses a set of recognized practices widely accepted by museum curators that have been formalized through inference rules. A qualitative analysis of the difference in suggestions for story- based and game-based apps [7] shows that teenagers may value gamification over narratives. The results of co-design sessions with teenagers in tangential work [8] show that they value games and stories when thinking about interesting museum tours. According to the results of the work [9], teenagers have many ideas about how they would prefer to communicate with museums, and are happy to share them when they have the opportunity. Essentially, the research highlights how much teenagers value technology and interaction. The article [10] presents a new concept of Adaptive Augmented Reality (Adaptive Augmented Reality – A2R), which is used in the context of creating an AR guide for visiting a museum. The additions provided are not only visual but also acoustic, while the visitor's interest is also tracked using physiological sensors so that the multimedia content delivered to the visitor's transparent AR display, which can be interacted with through gesture interaction, can be adapted according to its involvement and interests. This contribution focuses on an interdisciplinary, collaborative and UC-informed methodology used to identify and explore the motivations and needs of heritage professionals regarding the potential of an A2R approach to museum visiting. A common problem in museums is limited visitor access to artifacts due to their fragility, uniqueness, or simply the lack of physical space to display the objects. Moreover, visitors are allowed to touch the exhibits only in extreme cases. The work [11] combines natural interaction and augmented reality applied in a cultural heritage museum to overcome the problems of inaccessibility and lack of interaction with museum artifacts. It used ready-made digital components to provide a new approach to interaction with 3D copies of museum objects. Users can easily interact with digital content in a virtual environment that mimics the physical environment. A natural AR interaction system with virtual cultural heritage objects was presented, analyzed and tested. The system was created using off-the-shelf components and low-cost technologies to provide users with a new kind of experience in cultural heritage spaces. Using a smartphone in an HMD and Leap Motion, as well as software created in Unity3D, we implemented an Augmented Reality application that facilitates natural interaction with individual 3D models from the museum's collection. 3. Design of interaction scenarios for digital museum Immersive environments have been shown [12, 13] to enhance pursuasive effect due to wider fields of view and use of stereoscopic visuals combined with auditory content. New visual tools improving user experiences are important enhancers in education [14]. Educational scenarios are built on testing outcomes, collected in course of learning. Meaningful personalised recommendations for visitor allow to avoid the information overflow and maximise the user satisfaction [15]. For a particular visitor, the user data are incomplete and biased, however personalisation may be reduced to the level of user groups and the appropriate context. The context, be it educational, recreational, location-based or time- dependent, is closely linked with the visitor motivations. The least formalised approach is collaborative filtering in which users are grouped according to their rating of exhibition items. Content-based approaches, on the contrary, group visitors by their interaction patterns. Here, there are aspects of temporal dynamics in user bechaviour, biases in content-based neighborhood, influences of accessibility obtrusions. Quantitive data can be collected from visit time and proximity to exhibition items as well as certain interactions. Interactions, invluding guide control and movement control, are to be designed [16] ccording to the visitor preferences and motivations, user enjoyment, learning and fatigue. Digital museum of the prominent Ukrainian scientist and cultural figure Ivan Puluj, available at puluj-museum.tntu.edu.ua, which operates at the Ternopil Ivan Puluj National Technical University website since 2020, is a characteristic example of a collection of diverse artifacts and use of various types of resources as well as multiple modes of interaction with a visitor. The museum offers distant access to digitized collection of documents and artifacts, organized in the traditional exhibition and also offers VR-space for immersive experience, AR-components to be visualized with help of visitor’s mobile hand-held devices, video- and audio-guides. While in collective visits the particular visitor’s trajectory and the set of used technologies are chosen by the guide, for individual visitor this choice is made independently on the basis of available means of interaction, acquaintance with information technology and motivation of the visitor. We consider motivations to be the factor of paramount importance, though available technologies are definitive factor, too. For the present study, two use cases were chosen. In the first use case, the visitor is interested in an immersive experience, so he choses to visit VR spaces of the digital museum and Ivan Puluj’s laboratory. In the second use case the visitor choses to acquaint himself closely with Puluj’s inventions, so he uses AR models of devices constructed by the famous scientist. In both cases, some introductory information is to be presented to the visitor to put the displayed artifacts and documents in context. Visit duration and time-related restrictions be excluded from analysis as individual visits are considered, so we shall focus on representation techniques and modes of interactions. Whatever will be the chosen scenario, a visitor journey begins from the home page of the museum, which is a panoramic tour in the museum room (Fig. 1). Figure 1: The home page of the museum The first of two studied scenarios may be loosely represented by the map on the Fig.2. Figure 2: First tour scenario VR space is created with use of unity WebGL technology and offers to the visitor a freedom of motion through the museum virtual space or the laboratory virtual space, in which 3D models of the Puluj’s inventions are placed (see Fig. 3). Figure 3: VR space of the tour Controls will be familiar to young people as these are common for computer games development (Fig. 4). Audio guides may be enacted either by interaction with artifacts or exhibition stands or by using the video-guide in another browser window. The feedback may be collected by the google form and the statistical information about virtual visits may be offered by google analytics services. Figure 4: Controls for the user’s tour Figure 5: Second tour scenario Figure 6: An example of an AR element implemented in the virtual space 4. Analysis of the museum audiences In the second scenario, a visitor may combine the virtual gallery of Puluj’s inventions, in which descriptions, photos, drawings and scetchfab 3D models of devices are placed, with a set of marker- based AR visualisation of the devices. Again, supplementary information may be obtained through audioguides, linked to the exhibition elements or combined into the complete concise story. The second of two studied scenarios is represented by the map on the Fig. 4. Target audiences may be classified into three categories. The first, the largest and the most diverse one, represents students of different age who visit the museum with the educational goal. They, mostly, do not pursue their own goal but follow an educational curriculum, visiting the virtual museum under the supervision of their teacher alongside their classmates or individually, to collect material for their research, etc. The second broad category are adult persons who which to learn more about particular person (in our case, the prominent Ukrainian scientist Ivan Puluj) or historical period. They represent the most disperse category in terms of their interests and background as well as their level of proficiency in information technologies. The third category is relatively narrow and consists of historians who professionally study a period, a region or a personality. They expect to find authentic documents, evidences and artifacts. The first category, namely the students, have limited time and some pre-defined task to fulfill, which put some restrictions on the modalities of interactions. At the same time, they (in the same age group) have similar prerequisite knowledge in scope of the school courses of history, physics and geography therefore the scenarios of interaction with students may be the most strictly defined. On the contrary, the second category will decide of their interest, time to spent and satisfaction level independently, their motivation may change during the visit, they may be easily distracted from the virtual exhibition by an event in their real environments, so their require variability and flexibility, which may be provided by modular type of interaction scenarios. For the third category our museum may serve rather as an aggregator than the collection of original sources. In distinction from archives and collections, all items in the museum exhibition are digitized and put in context. Historians require indexes and descriptions for structuring the facts and evidences. They also need high quality digital copies to conduct independent studies. An exception will be the case when historian prepares a lecture and need images as illustrations. The technical means which mediate visits to the digital museum may be classified by the same framework which is use for distant learning and e-commerce. These are smartphones and tablets – the first type, the mostly used by school students; monitors, projectors and e-boards – the most widespread category and, finally, the head mounted displays (HMDs) ang virtual reality googles, which offer the best inversive experience but are the least available to wide audiences. Elements of the digital museum exhibition to combine in a scenario are the visuals (compositions on stands and tables of the exposition), audio guides, linked to the visuals, video-excursion, elements of Augmented Reality, Virtual Reality spaces, digital copies of authentic documents with explanatory descriptions. Questionnaires and feedback forms are used to receive information about the visitor traits, interests, requests and satisfaction. 5. Evaluation of a scenario in AHP Approach to museum tours gamification offered in [4] is acceptable for cases when we want user to look though every part of exhibition. Some gamification can be in place here. In our case we cannot use same approach because of virtual exhibition restrictions. These are only fixed set of positions and corresponding possible transitions. So, gamification in our case may be implemented as dynamic scenario for the museum tour based on possible positioning with transitions and visitor’s interests. Dynamic scenario means here that user may interact with museum artefacts presented as elements of VR, AR, implemented game elements etc. Consequently, we will offer some dynamic scenarios for the visitors. And the problem is to automate scenario choosing. Now this problem is analyzed and may be implemented with utilization of a multicriteria optimization method. It is necessary to collect some input data regardless to the optimization method. The methods for data collection were mentioned above and this list may contain not only quizzes, questioning but cookies for registered users, scenario analyses with AI for registered users. These elements are not in the scope of our paper and may be as the research object in the further researches. Finally, the visitor will get his own dynamic scenario as result of automated choice. Authors decided to use AHP as good formalizes and widely used in different fields where multicriteria optimization is required. It is a hierarchy needed for AHP, shown in the figure below. There is the visitor’s satisfaction on the top of this hierarchy as result of choosing the most suitable scenario for him/her. Scenarios 𝑆1 , … , 𝑆𝑛 consist of dynamic element on the fixed skeleton of positions and possible transitions. These scenarios are on the base of VR or with AR elements and scenario with game elements. Of cause, this list may be extended in future. Scenario will be chosen on the base of characteristics. Actually, these characteristics are input data, obtained from users. Figure 7: The hierarchy for scenario choosing These characteristics may be as following: • setting information; • previous visits; • feelings/emotions; • temporal agendas. Actually, this list consists of general characteristics and may be detailed dipper to the level of subcharacteristics. But for this paper it will do because we collected some data for our research on the level of listed above characteristics. Thus, using questioning we obtained matrices of pairwise comparisons for each alternative and for the alternatives regarding each criterion. These matrices are represented in the following tables 1 – 5. Table 1 Matrix of pairwise comparisons for the criteria with respect to the goal Criterion name 1 2 3 4 1. Setting information 1 6 3 1 2. Previous visits 1/6 1 1/3 3 3. Feelings/emotions 1/3 3 1 1/2 4. Temporal agendas 1 1/3 2 1 Table 2 Matrix of pairwise comparisons for the alternatives with respect to the criterion “Setting information” 1 2 3 1. VR elements scenario 1 2 1/6 2. AR elements scenario 1/2 1 1/3 3. Scenario with game elements 6 3 1 Table 3 Matrix of pairwise comparisons for the alternatives with respect to the criterion “Previous visits” 1 2 3 1. VR elements scenario 1 5 3 2. AR elements scenario 1/5 1 2 3. Scenario with game elements 1/3 1/2 1 Table 4 Matrix of pairwise comparisons for the alternatives with respect to the criterion “Feelings/emotions” 1 2 3 1. VR elements scenario 1 1 3 2. AR elements scenario 1 1 3 3. Scenario with game elements 1/3 1/3 1 Table 5 Matrix of pairwise comparisons for the alternatives with respect to the criterion “Temporal agendas” 1 2 3 1. VR elements scenario 1 1 6 2. AR elements scenario 1 1 6 3. Scenario with game elements 1/6 1/6 1 According to the [17] we used the scale for pairwise comparisons with values from 1 to 9 where odd numbers are main and even ones are additional intermediate values for comparisons. It is necessary to use some improvements of AHP for cases when there were more than 9 alternatives and/or more than 9 criteria as it is shown in [17] for software architectures for instance. And we should explain all the steps with detailed math formal grounding. But we are at the beginning of our research and we try only to automate scenarios choice using standard APH. So, there is not any reason to represent standard AHP. We used free online tool for calculations [12] for processing. Particularly, the weights of criteria are {0.409; 0.190; 0.168; 0.232} for {setting information; previous visits; feelings/emotions; temporal agendas} correspondently (Fig. 8). Figure 8: The weights of the criteria (attributes) Finally, we got the scored options as priorities for our alternatives as it is shown on the Fig. 9. Figure 9: The scored options as priorities for relevant alternatives So, the scenario with VR elements got the weight 0.380 with almost uniform contribution of each criterion. Scenario with AR elements got the smallest weight 0.281and criterion “Temporal agenda” is most significant for this scenario. Scenario with game elements got weight 0.339 with significant “Settings information” superiority. Figure 10: The activity diagram for developing the scenario of a virtual visit Based on the obtained weights, the specific scenario is to be constructed following the activity diagram (Fig.10) for the scenario development. For reliable predictions of the visitor gains and satisfaction one has to provide the scenario design module with statistical data on the cohort of users with a set of motivations and restrictions as well as processed feedback and the interaction response. However, in first stage these data are to be replaced by model assumptions and feedback from analogous exhibitions. 6. Conclusion Thus, we illustrated in our research the possibility for implementation of automated scenario choice for the virtual Ivan Puluj Museum. Use case shown in the previous chapter illustrates applying of AHP for automated scenario choice for particular user. This method may be implemented in the software complex of the museum. But we need to examine and to solve some supplementary problems. Initially mentioned above software must collect information about criteria priorities mainly not on the base of questioning but on the base of registered users’ activity. The second is implementation of modified AHP (MAHP) [19] for cases when more than 9 alternative scenarios will be developed and/or more that 9 criteria will appear. So, applying of AHP will do for scenarios choice automation but requires some improvements in technical and science aspects. 7. References [1] T. L. Saaty, L. G. Vargas. Decision making with the analytic network process (2006). Berlin, Germany: Springer Science+ Business Media, LLC. [2] P. Milgram, F. Kishino. A Taxonomy of Mixed Reality Visual Displays. (1994) IEICE Transactions on Information and Systems, E77-D, 1321-1329. [3] T. Olsson, T. Kärkkäinen, E. Lagerstam, L. Ventä-Olkkonen, User evaluation of mobile augmented reality scenarios, Journal of Ambient Intelligence and Smart Environments, 4 (2012), pp. 29-47, DOI: 10.3233/AIS-2011-0127 [4] R. Hammady, M. Ma, N. Temple, Augmented Reality and Gamification in Heritage Museums, CSG 2016: Serious Games pp 181–187, DOI:10.1007/978-3-319-45841-0_17 [5] I. Paliokas, A. T. Patenidis, E. E. Mitsopoulou et al. A Gamified Augmented Reality Application for Digital Heritage and Tourism. Appl. Sci., 10 (2020), 7868. https://doi.org/10.3390/app10217868 [6] N. Capuano, A. Gaeta, G. Guarino, S. Miranda, S. Tomasiello, Enhancing augmented reality with cognitive and knowledge perspectives: a case study in museum exhibitions, Behaviour and Information Technology 35 (2016), pp. 1-12, DOI:10.1080/0144929X.2016.1208774 [7] V. Cesário V. Nisi, Designing with teenagers: A teenage perspective on enhancing mobile museum experiences, International Journal of Child-Computer Interaction, 33 (2022), 100454, https://doi.org/10.1016/j.ijcci.2022.100454 [8] V. Cesário, D. Petrelli, V. Nisi, Teenage Visitor Experience: Classification of Behavioral Dynamics in Museums, In Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems (CHI '20). Association for Computing Machinery, New York, NY, USA, 1– 13. https://doi.org/10.1145/3313831.3376334 [9] V. Cesário, A. Coelho, V. Nisi, Design Patterns to Enhance Teens’ Museum Experiences, Proceedings of the 32nd International BCS Human Computer Interaction Conference (HCI), 2018, pp. 1-5, DOI:10.14236/ewic/HCI2018.160 [10] A. Damala, N. Stojanovic, Tailoring the Adaptive Augmented Reality (A2R) museum visit: Identifying Cultural Heritage professionals' motivations and needs, 2012 IEEE International Symposium on Mixed and Augmented Reality - Arts, Media, and Humanities, 2012, pp. 71-80, DOI: 10.1109/ISMAR-AMH.2012.6483992 [11] P. Kyriakou, S. Hermon, Can I touch this? Using Natural Interaction in a Museum Augmented Reality System, Digital Applications in Archaeology and Cultural Heritage, 12 (2019), e00088, https://doi.org/10.1016/j.daach.2018.e00088 [12] James J. Cummings and Jeremy N. Bailenson. How immersive is enough? A meta-analysis of the effect of immersive technology on user presence. (2015) Media Psychology 19,2 272-309. [13] Priska Breves.Biased by being there: The persuasive impact of spatial presence on cognitive processing. Computers in Human Behavior, 119 ,106723 (2021). [14] Yu. Skorenkyy, R. Kozak, N. Zagorodna, O. Kramar, I. Baran, Use of augmented reality-enabled prototyping of cyber-physical systems for improving cyber-security education, J. Phys.: Conf. Ser. 1840 (2021) 012026. [15] G. Pavlidis. Towards a Novel User Satisfaction Modelling for Museum Visit Recommender Systems. In: Duguleană, M., Carrozzino, M., Gams, M., Tanea, I. (eds) VR Technologies in Cultural Heritage. VRTCH 2018. Communications in Computer and Information Science, vol 904. Springer, Cham (2019). [16] R. Galdieri, M. Carrozzino. . Natural Interaction in Virtual Reality for Cultural Heritage. In: Duguleană, M., Carrozzino, M., Gams, M., Tanea, I. (eds) VR Technologies in Cultural Heritage. VRTCH 2018. Communications in Computer and Information Science, vol 904. Springer, Cham. (2019) [17] Kharchenko, A., Bodnarchuk, I., & Yatcyshyn, V. (2014). The method for comparative evaluation of software architecture with accounting of trade-offs. American Journal of Information Systems, 2(1), 20-25. [18] Analytic Hierarchy Process (AHP) Tool. Available on https://comcastsamples.github.io/ahp-tool/ [19] Harchenko, A., Bodnarchuk, I., & Halay, I. (2012). Stability of the solutions of the optimization problem of software systems architecture. In Proceeding of VII International Scientific and Technical Conference CSIT 2012 (pp. 47-48).