Mixed Reality (MR) is a technique that combines elements of virtual reality and augmented reality to create an immersive experience in a digital environment that is superimposed on the real world. This allows users to interact with virtual objects in a physical space in a natural and fluid way. Some common applications include video games, entertainment, design, and education. The Zitacuaro Technological Institute is a Higher Education Institution located in the east of the State of Michoacán, with 32 years of foundation; The physical infrastructure is housed in the Ex-Hacienda of Manzanillos, which dates back to the end of the 19th century and has a unique location and beauty in the region. Currently, the technological institution is considered the best higher education institution in the East of Michoacán, thanks to the consolidation of its Educational Programs, the development of research projects, innovation, entrepreneurship and national and international links. In this work, a virtualization strategy based on MR is implemented for the university campus. With the development of the project, a unique user experience will be generated, contemplating virtual and physical environments throughout the campus, using virtual reality and augmented reality techniques. Through the application, any user physically (local), and globally (remote), will be able to experience a tour of the campus facilities with locally enriched virtual elements and remote virtual models. The implementation of the application aims to strengthen national and international collaboration strategies with universities and research centers, and as support for comprehensive activities, positioning and promotion of the educational offer.
Information and Communication Technologies (ICTs) are modernizing and adding new,
increasingly surprising utilities. They have not only allowed remote work in educational
institutions but also advanced new ways of seeing reality, as is the case of Virtual Reality (VR),
Augmented Reality (AR) and Mixed Reality (MR) technologies [
AR is a technology that allows digital information to be superimposed on the real world. This
is achieved through the use of devices such as smartphones, tablets, special glasses, among others,
that provide a view of reality with additional digital elements. AR is used in a variety of fields such
as education, medicine, advertising, entertainment, robotics, design, among others [
The main objective of this research is design and implement a Mixed Reality strategy that allows users to interact in an enriched physical and virtual environment of the facilities of the Zitacuaro Technological Institute using Virtual Reality and Augmented Reality techniques. The main goals of this project is to implement an MR application that includes the facilities of the Zitacuaro Technological Institute based on a virtual reality model that provides users (students and interested parties) with a unique and significant experience. The virtual tour with MR will allow users to experiment with different technologies and systems interactively in real time. Additionally, the inclusion of virtual and augmented elements will make the experience more engaging and motivating. The implementation of the application aims to strengthen national and international collaboration strategies with universities and research centers, and as support for comprehensive activities, positioning and promotion of the educational offer.
The use of VR, AR and MR technologies in HEIs is a growing trend in recent years. These tools have allowed greater inclusion of technology in the teaching and learning process, providing students with a more interactive and enriching experience. In the field of education, MR has been used to create immersive learning experiences that combine the virtual world with the real world, allowing students to explore and experiment with concepts and theories more effectively. VR has been used to create completely virtual learning environments that allow students to explore scenarios and situations that would be impossible or dangerous in the real world. Finally, AR has been used to overlay digital information over the real world, allowing students to gain additional, more detailed information about an object or situation in real time.
At a global level, there are different universities that have adopted RM as a pedagogical tool in their institutions, below are some examples: •
University of Bristol (United Kingdom). The University of Bristol has implemented a mixed reality experience for its aviation engineering program [7]. • New York University (USA). New York University has used mixed reality in its architecture program to provide students with a virtual experience in building construction and design. • University of California, Los Angeles (USA). The University of California, Los Angeles has used mixed reality in its health program to provide students with a virtual experience in healthcare [9]. • In Mexico, some universities have investigated or implemented MR applications in their teaching, for example: o The National Autonomous University of Mexico (UNAM), the Autonomous University of Nuevo León (UANL), the Metropolitan Autonomous University (UAM), and the Ibero-American University (IBERO) have carried out various research on the topic. o According to [10] the National Autonomous University of Mexico (UNAM) has investigated the application of mixed reality in the teaching of mechanical engineering and medicine. o Another example is the Autonomous University of Nuevo León (UANL), which has implemented mixed reality in its computer systems engineering courses to improve the learning experience of its students.
These are just a few of the many universities that are exploring and using mixed reality in their teaching. In this sense and to strengthen national and international collaboration strategies, and in support of comprehensive activities, positioning and promotion of the educational offer, the Zitacuaro Technological Institute joins the universities that will be making use of Mixed Reality technology.
In the current market there are several devices for viewing mixed reality available. Below are some of these:
Microsoft HoloLens: Microsoft HoloLens is an autonomous MR device developed and produced by Microsoft, it is within the family of smart glasses, it uses the Windows 10 operating system. This technology tracks images through its camera, allowing both vision of the real world as well as the virtual world by projecting multidimensional color images known as holograms through its very high definition screen, hence the name HoloLens [11]. Magic Leap One: Magic Leap one is a device created by the Magic Leap company. Currently there are 2 versions for application developers and for users. The technology combines scene modeling, gesture recognition, object tracking, virtual object design, among others. It has a wide field of vision, a very realistic virtual design and something very important is that it is open source [12].
Oculus Quest 2: It is a device manufactured by Oculus belonging to the company Meta, it is an independent device that does not require external connections or wiring, since it has its own hardware including screen, processor, sensors and headphones [13].
In relation to software, there are various applications on the market, both downloadable and online, available to immerse yourself in MRI and make use of these technologies. • • • •
Microsoft Dynamics 365 Remote Assist: is an RM application designed for remote collaboration and technical support. It allows users to share their point of view with remote experts and receive real-time instructions by overlaying virtual objects on the real environment.
Wayfair Spaces: is an MR application that allows users to visualize virtual furniture and decorations in their real environment. Allows you to try out how the products would look in your home before purchasing.
SketchAR: it is a MR application that uses object tracking technology to help users draw and learn to draw in the real world. Projects a virtual image onto paper or other surface to guide the user through the drawing process.
Spatial: it is an MR remote collaboration application that allows users to interact and collaborate with virtual objects and people in a shared environment. It allows you to hold virtual meetings and share content in an immersive way.
The methodology selected for creating the RM strategy includes the following stages: Design and planning: In this stage, the objectives of the application, the target audience and the content that you want to display in the MR are determined. The user interface structure is also planned and technical requirements are specified. For RM design and planning, a usercentered approach is used, where the user is involved in the design process to ensure that the application is relevant and easy to use [14]. A game-based approach is also applied, where game design techniques are used to create engaging and motivating MR experiences [15]. The hardware and software necessary for the application, the user's space and movement requirements, as well as security and privacy are considered [14]. 3d modeling: it is a basic tool in creating MR experiences. 3D modeling allows you to create virtual objects and scenarios that can be integrated into physical reality through MR devices. These 3D models are used to generate the information necessary for real-time visualization and user interaction. There are 3D modeling software that is used to create content for MR. Some examples include Autodesk Maya, Blender, SketchUp, and 3D Studio Max. In the case of the project, SKETCHUP is used, which offers a variety of tools and techniques to create and edit 3D models, as well as animate and export them in device-compatible MR formats. Regarding the methodology to create 3D models for MR, the prototype-based approach will be used, where a rapid 3D model is created and iteratively improved [11].
Programming and development: it is the main stage to create interactive and enriching experiences. It is used the C++ and JavaScript languages that are managed by the UNITY platform. Software development tools are also used to create content and applications such as Vuforia, ARToolKit, and HoloLens as these tools provide a set of functions and features specific to MR development, such as mark detection, gesture recognition, and position tracking. To make the process more efficient, an agile methodology is used, such as SCRUM, since it better adapts to the environment of uncertainty and constant change of emerging technologies such as mixed reality [16].
Testing and debugging: this stage ensures that the parts function correctly and offers a satisfactory user experience.
Publication and distribution of the application.
QR code printing: for access to MR on mobile devices.
The planning stage of the project included the creation of a model in Unity software, designed in Sketchup and rendered with V-Ray, that is capable of being introduced into the university campus through the Internet, in such a way that users can carry out a virtual tour in three dimensions. The areas for the virtual visit consider the exterior of the buildings that make up fraction “A” of the campus, in total there are 13 buildings. Figure 1 shows, in general terms, the area of the university campus to be virtualized.
In the design stage, the possible implementation options for the software to be built were analyzed, as well as deciding its general structure. Design is a complex stage and its process must be carried out iteratively. The design of each building is required with its characteristics closest to reality. The design considers only the exterior part of each building, for this there are various modeling tools such as 3D Max, ArchiCAD, Blender, Maya, etc. Autodesk 3D, V-Ray, Maverick, Arnol, Blender, etc. can be used for rendering. To combine design and rendering, Unity, Unreal, Corona SDK, GoDot, among others, can be used. The first step was to obtain the overall plan of the campus in AutoCAD which was already created (see Figure 2).
Subsequently, photographs were taken of the entire university campus to design them in 3D (see Figure 3).
Subsequently, modeling was performed using Sketchup software (see Figure 4).
In the rendering stage, V-Ray software implemented in Sketchup was used (see Figure 5).
In the integration stage, the Unity software was used to later carry out the programming tasks (see Figure 6).
The use of tools such as Sketchup for the modeling of each building has been the best option since it allows modeling identical to the original, in addition to its ease of manipulation and the few resources it requires to work; once modeled, the rendering continues with the V-Ray software, which is integrated into Sketchup as a plugin and it is only a matter of configuring it to be able to render. This software does require extensive graphic resources on the computer, such as a graphics card and RAM memory. With the generated file it is incorporated or added to Unity, which is where the tour is finally processed. In the design and rendering stages, high-end hardware was used: an AlienWare Aurora R7 I7 8th computer, GTX 1070, with 26 Gb of Ram, 1 Tb of hard drive, Intel UHD Graphics 630 NVIDIA GeForce GTX 1070 (8 Gb) of video), all this resulted in faster and more agile work.
Figure 7 shows the first results of the 3D modeling of the Zitacuaro Technological Institute; as can be seen, the entire fraction “A” is designed, in the background you can see part of the campus (without rendering), auditorium, buildings, computer center, address, etc.
The application allows you to display descriptive information about strategic areas of the campus when users are near each area. Figure 9 shows the descriptive information of Building K.
Figure 10 shows the use of AR, giving information about the directors of the Zitacuaro Technological Institute, when approaching the address building using the Vuforia tool.
A Mixed Reality application was generated that allows virtual tours for remote users located anywhere in the world (using VR) and physically (using AR). The entire university campus of the Zitacuaro Technological Institute was designed and rendered in fraction “A”. The technologies used to implement the application were Sketchup, V-Ray, Vuforia and Unity. Currently the application is hosted on the institute's server, fulfilling the function of dissemination and knowledge of the physical facilities of the university campus; the application allows users to take virtual tours through the entire physical facilities of the campus and physically (visitor) using AR, in addition to providing additional information in the languages of Spanish, English, French, Portuguese and German. The future work is to implement various Artificial Intelligence technologies (artificial vision and natural language processing), database storage for better interaction with users, as well as the extension to the “B” fraction of the university campus.
The main author and co-authors of the work thank CONAHCYT for the scholarship awarded to carry out master's studies and all the support granted by the Zitacuaro Technological Institute. [7] Yee, R. P., & Bailenson, J. N. (2007). The implications of immersive virtual reality for electronic commerce: The example of tourism and travel. International Journal of Human-Computer Studies, 65(11), 1023-1038. [8] Ryder, C. (January 26, 2023). To Mars and Beyond: NYU Tandon and partners selected as new hub for VR/AR in NYC. Tandon School of Engineering. https://engineering.nyu.edu/news/mars-and-beyond-nyu-tandon-and-partners-selectednew-hub-vrar-nyc. [9] Riva, G. (2017). The impact of virtual reality on healthcare and medical education. The Lancet
Digital Health, 1(1), e1-e8. [10] Zaragoza Pérez, Rosangela y Cuevas Escudero, Ana Leticia. (2020, noviembre). Realidad aumentada en la enseñanza. Revista Digital Universitaria (rdu), 21(6). doi: http://doi.org/10.22201/cuaieed.16076079e.2020.21.6.9 [11] A. J. Martín García. “Aplicación de realidad mixta para la asistencia en tareas de ensamblaje”.
Escuela Técnica Superior de Ingenieros Industriales, Universidad Politécnica de Madrid.
Madrid, España. 2019. [12] Soto Ramos, M. (2019). Development of mixed reality applications using the magic leap one device. Universidad Autónoma de Madrid. [13] CARNEVALE, Arianna, et al. Realidad virtual para la rehabilitación del hombro: Evaluación de la precisión de oculus quest 2. Sensores, 2022, vol. 22, n.º 15, p. 5511. [14] Keshavarz, A., Kalantari, B. y Gao, Y. (2018). Un enfoque de diseño centrado en el usuario para sistemas de realidad mixta. Transacciones IEEE en sistemas hombre-máquina, 48(4), 447-456. [15] Tello, D. C. (2018). Realidad mixta/híbrida para la enseñanza de la animación en la educación superior. Recuperado de: http://hdl.handle.net/20.500.12749/2660. [16] CUÉLLAR-ROJAS, Óscar-Andres, et al. Caracterización del uso de tecnologías inmersivas aplicadas en geoparques: Realidad Virtual, Realidad Aumentada, Realidad Mixta, Técnicas de visualización y Gamificación. Revista Ibérica de Sistemas e Tecnologias de Informação, 2021, no E45, p. 285-302.