Application of Virtual and Augmented Reality Technologies for Creation of a Digital Museum of Scientific and Cultural Heritage of Ivan Puluj Oleksandr Kramar, Yuriy Skorenkyy, Oleksandr Rokitskyi and Taras Kramar Ternopil Ivan Puluj National Technical University, 56 Ruska St, Ternopil, UA46001, Ukraine Abstract The transformation of exhibition spaces to make different segments of society acquainted with the historical, cultural and scientific heritage requires use of modern information technology, including virtual and augmented realities. In this regard, it is appropriate for museums to use immersive techniques in order to preserve and demonstrate digitized scientific and cultural artifacts. The paper presents the concept of a digital museum of scientific and cultural heritage of the famous Ukrainian scientist in fields of physics, electrical engineering and public figure Ivan Puluj, which may contribute, in particular, to the growth of tourist attractiveness of Ternopil region and popularization of Ukrainian science in general. Using computer-aided 3D technology, a panoramic tour, virtual space and augmented reality application have been created based on archival information content associated with the name of this prominent scientist. Keywords Virtual reality, augmented reality, virtual tourism, cultural heritage, virtual tours 1. Introduction The rapid development of modern methods of computer scanning and 3D modeling, as well as virtual and augmented reality technologies provide opportunities for creating realistic copies of a variety of objects in accordance with the concept of digital twins [1]. Special attention in this context is to be paid to objects of scientific and cultural heritage (see for review [2-4]) for which a reconstruction in digital 3D-format opens wide prospects for educational activities and popularization of scientific achievements, formation of scientific worldview, comprehensive introduction of the latest technologies, preservation of historical traditions of educational institutions [5,6]. Immersive technologies (with the possibility of deep immersion in the virtual world) include human interaction with the digital visual environment and are designed to create an effect of presence through perception and interaction with 3D content (see, for example, one of the first significant ARCO projects [7], which was aimed at developing a set of digital museums technologies for the creation, preservation and demonstration of digitized cultural objects in virtual exhibitions, available both in museums and remotely). The authors [8] note that the creation of a virtual museum based on the concept of multi-component mixed reality, which combines elements of virtual and augmented reality, as well as web-3D allows to achieve a new level of visitor interaction with the information content of the exhibition. Ivan Puluj, who originated from the Ternopil region, belongs to the cohort of those famous scientists [9,10] who left behind a huge amount of archival material. Acquaintance with technical features of inventions, scientific works, translation and journalistic activity of our outstanding compatriot still represents considerable cognitive value, and not only for the scientific community, but also for a citizen, especially a pupil or a student. ITTAP’2021: 1nd InternationalWorkshop on Information Technologies: Theoretical and Applied Problems,November 16–18, 2021, Terno- pil, Ukraine EMAIL: kramaroitntu@gmail.com (A.1); skorenkyy.tntu@gmail.com (A.2); rokitskyi.o@gmail.com (A.3); kramartar18@gmail.com (A.4) ORCID: 0000-0003-0805-3732 (A. 1); 0000-0002-4809-9025 (A. 2) © 2021 Copyright for this paper by its authors. Use permitted under Creative Commons License Attribution 4.0 International (CC BY 4.0). CEUR Workshop Proceedings (CEUR-WS.org) In the digital age, age of advanced web technologies, more and more people, especially youngsters, are actively using mobile communication devices or computers for fast search for information not only for educational purposes, but also for entertainment and work. That is why the adaptive potential of traditional exhibition grounds and museums, in particular their ability to transform into virtual spaces, acquires special importance. Modern computer 3D-technologies allow not only to improve the information component of the museum exposition, raising it to a qualitatively higher level, but also to preserve its interactivity. It should be noted that the digital translation of historical and cultural heritage sites, including works of art, archival materials, archaeological records and artifacts, as well as monuments, buildings and historical landscapes, helps to organize and preserve the most important information [11, 12], promotes the adaptation of visitors and their orientation in an unfamiliar environments. 2. Object of Study In the present work the exposition of the Ivan Puluj Museum which was founded in 2010 at Ternopil National Technical University, has been digitized. Scanned textured 3D models of monuments to the outstanding Ukrainian scientist and public and political figure Ivan Puluj, located in the Ternopil region, were also created. It should be noted that the figure of this talented scientist always attracts considerable interest [9,10,13], because Ivan Puluj due to the diversity of interests and high level of achievements in various fields of activity proved himself as a physicist and electrician, electrician, designer and inventor, outstanding organizer in the field of science and education, a translator of the Bible, a true Ukrainian patriot. The work of the physicist contributed to the establishment of the atomistic theory of the structure of matter, became the basis for the discovery of X-rays and electrons, contributed to the formation of X-ray as a science, its active use in medicine, and scientific hypotheses about the mechanism and nature of cathode and X-rays to be evaluated properly (I. Puluj is mentioned in the fundamental monograph [14]). The scientist's achievements in the field of electrical engineering and electric power generation were rewarded by the highest governmental awards, inventions were patented in many European countries, and theoretical works on alternating current electrodynamics formed the foundation of certain sections of electrical engineering. It should be noted that the materials of archival and library funds were used during the preparation of the exposition, in particular: the Central State Historical Archive of Ukraine in Lviv, the Central State Archive of Supreme Bodies of Power and Government of Ukraine, the National Museum in Lviv, the Puluj Family archive, Lviv V. Stefanyk Scientific Library of the National Academy of Sciences of Ukraine, Scientific Library of National University “Lviv Polytechnic”, V. Vernadsky National Library of Ukraine, archives of the University of Vienna. All these are mainly reprints of scientific German-language works of the scientist, their translations into Ukrainian and English, his journalistic and epistolary heritage, autobiographies, memoirs of his contemporaries (for a review of archival materials see, for example, [15,16]), as well as specialized monographs and books [9,17,18]). Following the work [19], we briefly describe the content of the exposition, which is presented in eight stands. Each of them, except for the first and last, covers a certain period of life or activity of the scientist. The first stand contains biographical information. The second characterizes the years of study at Ternopil Gymnasium and the University of Vienna (the most interesting photos and places are specially highlighted, including copies of baptismal certificates, a fragment of an autobiography mentioning the founding of the “Hromada” society, letters to friends, contemporary photos of Ternopil, etc.). The materials of the third stand cover the scientific and inventive activities of the scientist at the Universities of Vienna and Strasbourg (there are photos of some works in the field of molecular physics, a monograph on cathode rays, reconstruction of the Puluj device for measuring mechanical heat equivalent). The Prague period of the scientist's life, dedicated to electrical engineering and the position of professor at the German Higher Technical School, is described on the fourth stand (photos of the title pages of the most important scientific publications and inventions in the field of practical electrical engineering and telephony, power plant design, etc). The fifth stand contains information about the discovery of X-rays, chronological information about the publications of W. Roentgen and I. Puluj, the first X-ray images and gives reasons to emphasize the significant contribution of Ivan Puluj to the study of nature, mechanism, properties and prospects of cathode rays. Activities to promote the creation of a Uk Ukrainian university, socio-political political views of I. Puluj, in particular on the role of Ukraine in the geopolitical concept of Europe, are reflected on the sixth stand. The seventh stand is devoted to the activities related to the assertion of the Ukrainian la language in church literature, cooperation with P. Kulish and I. Nechuj Nechuj-Levytskyj Levytskyj in the translation of the Bible. On the last stand there are photos covering the personal and family life of the scientist. In front of the stands around the museum hall there are books and brochures, Holy Scripture, Prayer Book and Psalter, scientific works and popular science articles, documents (copies of Ivan Puluj's student book, minutes of the University of Vienna Habilitation Commission signed by Ludwig Boltzmann, Josef Stefan, Victor Lang), exhibits from the Prague Polytechnic (kindly provided by Prof. I. Kraus) letters, appeals and memoranda, memoirs of contemporaries, commemorative coin, stamps and postal envelopes. The busts of I. Puluj by the sculptor Emanuil Mysko ((located located in the museum of TNTU in Ternopil), the busts of I. Puluj (located in the lobby of the main building of TNTU in Ternopil) and the monument to I. Puluj in Hrymayliv (Ternopil region) by sculptor Mykola Obeziuk were selected for photogrammetric scanning. The basic hypothesis of this study is that the digitization of these monuments of Ukrainian scientific and cultural heritage with further promotion in the web space will improve the visibility of higher education establishment, will intensify the flow of visitors to the University museum, and thus create another attractive location for tourism in the region. It is worth noting that the scientific and educational goal of this project was to promote the achievements of Ukrainian science and the formation of the scientific worldview of the younger generation, the development of patriotism of Ukrainian society on the example of the best scientists of the Ukrainian state. 3. Materials and Methods The activities for creating a virtual space of the digital museum of scientific and cultural heritage of Ivan Puluj was based on the following step step-by-step step method (see Figure 1). Initially, collecting and pre-processing processing the information content has been done, with activities in the open spaces and indoors, aimed at obtaining ning the necessary photographic data. Figure 1:: Conceptual structural diagram of the digital museum of Ivan Puluj with reflection of the main steps of the methodology of information system design Simultaneously, the collected data were pre-processed processed to meet the stated objectives of the study. The creation of the information and software complex took place at the laboratory stage (using the equipment and computing capacity of the Cyber Physical Systems Laboratory of TNTU, Quad Core Intel Core i7-7700K 4400 MHz CPU, Kingston HyperX DDR4-2400/1200 MHz RAM, NVIDIA GeForce GTX 950 2 Gb 1024 MHz GPU, 500GB Samsung 870 EVO SSD), which involved the actual production of 3D and panoramic models for publication in the web space. Creating a promotional website and augmented reality application was the final stage of the project, which provided interactivity and online accessibility to all models created in the previous stages. The data collection process (see Figure 2) was based on the use of terrestrial close-range photogrammetry (TCRP), aerial close-range photogrammetry (ACRP) and 3D modeling using specialized software to obtain high-quality models suitable for integration into a website and interactive viewing in the Sketchfab web-service. Thus, both for the lower part of the monuments and for the museum interior, the data required was acquired by employing the TCRP technique, using a smartphone camera with 12+5 MP resolution, f/2.0. The overflight, using an unmanned aerial vehicle (UAV), of the upper parts of the churches concerned used a DJI Ryze Tello, with a 1/2.3 inch CMOS sensor and 5 MP. Figure 2: Method of obtaining photographs for creating three-dimensional textured models of monuments (by analogy with technology [3]) In Ternopil Ivan Puluj National Technical University, the collection of the Ivan Puluj’s Electronic Fund in the institutional repository [15] and Ivan Puluj's museum have been actively replenished with new materials for more than a decade. With the help of the action-camera with 360° module Insta360 ONE R Twin Edition, panoramic photos of the I. Puluj Museum and the locations where the busts are located, were filmed. Based on the photo materials of the exposition, a panoramic tour of the museum has been created (available on the university website at puluj-museum.tntu.edu.ua). The exhibition stands described above can be viewed by users in the form of high-resolution digital copies from any laptop or mobile device. The tour is also supplemented with links to multimedia content with instruments and devices, designed by I. Puluj (based on archival photographs of the Puluj physical laboratory at the Prague Polytechnic, see photo tab [9, p. 32]). Using the available archival materials, 3D-models of lamps constructed by Puluj were created (rendering was carried out with the help of specialized software Autodesk 3ds MAX 2021 as well as ZBrush, and texturized in Adobe Substance Painter). Digital copies of scientific papers and real museum stands with rare photographs (see Figures 5, 6) are located in a virtual space developed with Unity engine use (including the Probuilder package for landscapes and interiors). Figure3: Electronic fund of Ivan Puluj in the institutional depository of TNTU [15] Figure 4: Start page of the Ivan Puluj panoramic museum with a video tour and integrated 3D models of monuments It is planned to distribute the exposition of the virtual museum in accordance with the historical and geographical periods of I. Puluj's activity (Ternopil region, Vienna, Strasbourg, Prague, etc.). For the sake of completeness and authenticity, some 3D-models for the museum, in particular busts and monuments to I. Puluj, were reproduced by photogrammetry using programs for 3D-processing of photographs (3DF Zephyr, Meshroom, Meshmixer). Worthwhile to note, 3DF Zephyr performs rendering with the advanced proprietary algorithm, known in computer vision as Multiview Stereo, designed to highly accurate extract dense point clouds from a collection of 2D images. Also, the proprietary algorithm for mesh extraction in 3DF Zephyr ensures extracting the surface in high detail. Filming in Hrymayliv and Ternopil made it possible to obtain (see Figs. 2, 6) scanned models of monuments with detailed textures. The virtual space is converted into an Android application that supports the functionality of virtual reality glasses (see Fig. 7, the avatar is controlled by the movements of the head based on changing the position of the sensors of the mobile device). Figure 5: Screenshot of the virtual space of the Ivan Puluj digital museum with a model of a bust, a Puluj lamp and a fragment of the exposition related to the history of X-rays discovery. Figure 6: General plan of the browser version of the Ivan Puluj digital museum virtual exposition, developed with Unity instruments and integrated into the website with use of WebGL technology. Figure 7: Screenshot of a user view of the exposition of the Ivan Puluj digital museum through virtual reality glasses Figure 8: Screenshots of a mobile device with the "Monuments to Ivan Puluj" augmented reality application To obtain the museum's exhibits as augmented reality objects, printed photo materials (postcards, calendars, souvenir magnets) were used, which contain a number of image-templates that act as visual markers for mobile devices (such a scheme has been successfully tested on 3D memory models of monuments to I. Puluj in Ternopil region, see Fig. 8). The real stands and objects presented in the panoramic museum also contain markers that will provide additional information through a visitor's smartphone or tablet (analog of a virtual guide). 4. Conclusions Summarizing, we note that the use of extended reality means (through the components of virtual and augmented reality) allows for the proper reconstruction of devices and techniques that were crucial to the technological progress of civilization and contributes to a better understanding of the historical context. Main applied result of our work is the design of the framework for development of the complex of interrelated components for virtual museums and collections. The proof-of-concept web portal with 3D content, created during this study as an extension of the traditional information databases, will help to increase the tourist attractiveness of Ternopil region, the birthplace of Ivan Puluj and preserve cultural heritage in digital formats. The format of the digital museum should be used as a modern addition to the existing stationary exhibitions, as it brings a number of significant benefits, including greater accessibility, elements of interactivity of the virtual exhibition, the ability to actively promote during educational activities, such as field trips and mobile exhibitions, scientific picnics, mobile lectures, etc. 5. Acknowledgements The authors of the article and the Cyber-Physical Systems Laboratory are grateful to Ternopil Ivan Puluj National Technical University for financial support of the project. 6. References [1] H. Parsinejad, I. Choi, and M.Yari, Production of Iranian Architectural Assets for Representa- tion in Museums: Theme of Museum-Based Digital Twin, Body, Space & Technology 20 (2021) 61– 74. doi: https://doi.org/10.16995/bst.364 [2] M. Duguleana, M. Carrozzino, M. Gams, I. Tanea, VR Technologies in Cultural Heritage, VRTCH 2018, Springer, Cham, Commun. in Comp. and Inform. Science 904 (2019). [3] T. Caciora, G.V. Herman, A. Ilies et al, The Use of Virtual Reality to Promote Sustainable Tourism: A Case Study of Wooden Churches Historical Monuments from Romania, Remote Sens- ing13 (2021) 1758. doi: https://doi.org/10.3390/rs13091758 [4] M. Koehl, and N. Brigand, Combination of virtual tours, 3d model and digital data in a 3d archaeological knowledge and information system, Int. Arch. Photogramm. Remote Sens. Spatial Inf. Sci., XXXIX-B4 (2012) 439–444. doi: 10.5194/isprsarchives-XXXIX-B4-439-2012 [5] R. Bakhmut, N. Kunanets, A. Rzheuskyi, V. Pasichnyk, V. Kut, O. Volkov, I. Tsmots. For- mation of Virtual Tours with the Use of Augmented Reality Information Technology, Proceedings of the 5th International Conference on Computational Linguistics and Intelligent Systems (2021) 1431- 1443. [6] O. Kramar, Yu. Drohobytskiy, Yu. Skorenkyy et al, Augmented Reality-assisted Cyber- Physical Systems of Smart University Campus, IEEE 15th International Conference on Computer Sciences and Information Technologies (2020) 309-313, doi: 10.1109/CSIT49958.2020.9321951 [7] R. Wojciechowski, K. Walczak, M. White and W. Cellary, Building Virtual and Augmented Reality museum exhibitions, Proceedings of the ninth international conference on 3D Web technolo- gy, New York, NY, USA (2004) 135–144. doi: https://doi.org/10.1145/985040.985060 [8] M. White, P. Petridis, F. Liarokapis and D. Pletinckx Multimodal Mixed Reality Interfaces for Visualizing Digital Heritage, International Journal of Architectural Computing 5(2007). 322-337. doi: 10.1260/147807707781514986 [9] R. Gajda, R. Plyatsko, Ivan Puluj: life and works, Lviv, NTSh, 2019 [in Ukrainian]. [10] V. Schenderovsky, Ukrainian scientists in global science, Kyyiv, Prostir, 2019 [in Ukrainian]. [11] N. Maiellaro, A.Varasano, and S.Capotorto Digital Data, Virtual Tours, and 3D Models Inte- gration Using an Open-Source Platform, VR Technologies in Cultural Heritage. VRTCH 2018. Sprin- ger, Cham, Commun. in Comp. and Inform. Science 904 (2019) 148-164. [12] S.Gonizzi Barsanti, G.Caruso, L.L.Micoli et al, 3D Visualization of Cultural Heritage Arte- facts with Virtual Reality devices, Int. Arch. Photogramm. Remote Sens. Spatial Inf. Sci., XL-5/W7 (2015) 165–172. [13] E. Tesinska, Johann Puluj (1845–1918): his career and the “invisible cathode rays”, IUCr Newsletter 28 (2020). URL: https://www.iucr.org/news/newsletter/etc/articles?issue=148371&result_138339_result_page=26 [14] M. L'Annunziata, Radioactivity: Introduction and History, From the Quantum to Quarks, Elsevier, 2016. [15] Electronic Fund of Ivan Puluj. URL: http://elartu.tntu.edu.ua/handle/123456789/329 [16] Yu. Holovach, R. Plyatsko, H. Svarnyk,Peter Puluj and Ivan Puluj’s archive.Preprint 20-01U (2020), URL: https://www.icmp.lviv.ua/sites/default/files/preprints/pdf/2001U.pdf [in Ukrainian]. [17] O. Zbozhna, Ivan Puluj, Letters, Ternopil, Volia, 2007 [in Ukrainian]. [18] V.Schenderovsky (Ed), Ivan Puluj, Collection of works, Kyyiv, Rada, 1996 [in Ukrainian]. [19] O. Rokitsky, N. Rokitska, V. Lazariuk, Exposition of Ivan Puluj museum in Ternopil Ivan Puluj National Technical University, Collection of scientific works of Ternopil branch of NTSh, Ter- nopil, Terno-graph, Vol. 8, Museums of Ternopil Region (2013), 415-421 [in Ukrainian].