Navigation has always been an object of interest to scientists and business industry representatives. There are plenty of ready-to-use applications that use GPS data, designed to make user's navigation easier. Augmented Reality is currently one of the most popular upcoming technologies most commonly known for its use within games and advertising. By combining navigation with augmented reality, it could be possible to obtain new user friendly applications which will be able to quickly help users in everyday navigation. In this study the applied aspects of information system development for routing and visualization of augmented reality routes have been considered. The relevance of the study has been proved by the results of the survey among first-year students of Khmelnytskyi National University (KhNU) on the necessity of assistance in navigation during the first year of the study. An information system in the form of a mobile application has been developed, which provides assistance in routing in real time and reproducing the saved routes using augmented reality technology. Smart campus, Augmented reality (AR), Navigation systems, iOS, Mobile Application IntelITSIS'2022: 3d International Workshop on Intelligent Information Technologies and Systems of Information Security, March 23-25, ORCID: 0000-0003-2905-0215 (O. Pavlova); 0000-0002-0775-1347(A. Bashta); 0000-0003-4472-592X(S.Kravchuk); 0000-0001-98195069 (Y. Hnatchuk) 0000-0003-3239-8255(H. El Bouhissi);
navigation [
The concept of smart cities became quite widespread over the last decade. It includes facilities for dwellers such as smart bus stops, smart parking, inclusive access to city buildings and smart
Currently relevant is the issue of navigating unfamiliar places at short distances, such as a hospital area with plenty of buildings or university campus with lots of different buildings (dormitories, educational buildings, library etc.), where GPS-navigators do not always give accurate
Nowadays students care for the digitalization and sustainability of their university campuses, work on green-tech projects and want to see their university modern and technological. The territory of Khmelnytsky National University is large enough and occupies 81602.95 square meters. The institution accepts more than 500 new first-year students and about 30 foreign students annually, the survey among the students has been conducted. 78 Ukrainian and 34 foreign students took part in the survey. The results of the survey are shown in Figure 1.
2022 Copyright for this paper by its authors.
According to the survey, 89,5% of Ukrainian first-year students and 100% of foreign first year students consider the campus of Khmelnytskyi National University to be large. 63,2% of Ukrainian first-year students and 71,4% of foreign first year students responded that they needed some help in navigation (finding the necessary academic buildings, hostel, sport building etc) at first.
78,9% of Ukrainian first-year students and 85,7% of foreign first year students responded that in their opinion freshmen need some help navigating the campus of KhNU.
84,2% of Ukrainian first-year students and 85,7% of foreign first year students consider that it would be relevant and useful to create an augmented reality based application for laying the most popular routes on the campus of KhNU to facilitate the navigation of first-year students and their parents during their first stay on the territory of KhNU.
Considering the relevance of this issue, it was decided to develop the information system for route planning and visualization in the form of mobile application. For the developing augmented reality technologies and GPS data have been chosen.
Therefore, the aim of this work is: 1) сonduct the analysis of modern technologies for navigation using augmented reality and GPS data; 2) develop the client-based part in the form of mobile app which provides planning and visualization of routes; 3) сonduct the experiment on route planning and visualization in Khmelnytskyi National University campus.
Augmented Reality is an area that is closely related to Virtual Reality in that both utilize a subset
of the same tools but for different purposes [
Master Degree
Thesis Master Degree
Thesis
A summarizing study
A summarizing study
Method of
Research A Qualitative
set of interviews Quantitative and qualitative
study, development of
AR-based mobile app for
Android OS Literature
study Literature
study
An investigative study
A quantitative/ qualitative set of interviews - AR operates and is used in a 3D environment.
Analysis of sources (Table 1) has been conducted to learn the state-of-the arts in Augmented Reality domain.
Table 1 Literature review Augmented Reality domain for navigation
Source
Purpose
Design
Results
Conclusion
As can be seen from Table 1, review of the literature and academic studies in the AR domain gives
merely theoretical background. Surveys of the research literature suggest that AR currently greatly
increases driver’s attention. Research in the AR and VR domain is moving fast and several institutes
and companies like Tesla, BMW, Pioneer, and Toyota are working on development in AR navigation.
[
Table 2 shows that AR is used not only for in-built navigation systems in driverless cars, but also for custom use in the form of mobile applications. The analysis has shown that AR + Android OS are the most frequently used combination for mobile development. Therefore it was decided to use the combination of such technologies as: iOS operating system, Swift programming language and augmented reality library ARKit, since there are not so many currently existing iOS based mobile applications and there is still the necessity of research and development.
Logo
Free or paid free free free, in-App purchases The addition is in BETA version and works with Locus Map Pro application.
In Locus Map Free, its usage is limited to 1 minute.
Description Android and iOS-based mobile application for indoor navigation through Gatwick Airport This Android-based application uses smartphone's GPS and camera to implement an augmented reality-powered car navigation system Android-based live maps for navigation. Use in trips or while traveling. Search makes it easy to plan trips and itineraries.
Additon for Android-based Locus Map application that enables visualization of selected points on the device screen with camera view - in augmented reality.
Useful during town sightseeing tours, on viewtowers, for geocaching or for simple guidance to any point. The add-on is in BETA version and works with
Locus Map Pro application reality-based technology for route planning and
As can be seen from Figure 2, AR-based system uses real-time video stream as input data. After processing the video and adding augmented reality elements to build the route that involved the user and environment, the route is being built and saved in the application. This route can be reproduced again once a user needs it or transferred to another user and opened with the same application. Reproducing the ready-built route, that is the output data of the system, also requires user and environment engagement.
To automate the work of the AR-based technology, on which the work of mobile application is based, it is necessary to make the decomposition of the system. The structure of the technology is shown in Figure 3. The work of the technology requires the camera of the smartphone to be enabled.
First, iOS requires the camera authorization from the user, and shows the “Allow” button. Every
time when a user opens the application this requirement is shown and needs to be checked by the user.
Then, the CoreLocation framework is used to get user location from the iOS system. After that, the
recording can be started — camera session starts, camera preview layer is displayed on the screen.
User has one option - tap on the “Start” button, then the 3, 2, 1 timer is displayed and the user can
start going to any point and record their way. This entire route is recorded using small 3D objects,
which are called SCNodes. SCNodes are parts of SceneKit framework [
Then all these nodes are connected to one UIBezierPath object. All the information is converted to container NSSecureCoding, which can be saved to device memory. All the routes are available to save and can be reproduced when needed. User can reproduce any needed recorded route from point to point in real time, using AR-based Data on Device location and orientation in Camera View.
To work with augmented reality technology using the ARKit library, it should be noted that during
the experiment (setting a 3D route) the phone will be in three-dimensional space (Figure 4), so it is
necessary to consider several aspects. The coordinate system used in ARKit is a right-hand coordinate
system. The reduction of the first coordinate of an augmented reality object is to bring together three
coordinate systems - the world coordinate system, the object coordinate system and the camera
coordinate system in one point [
The world coordinate system is determined when the user's device recognizes the outer space to start the session. An object that is placed in augmented reality space is given an absolute position in this coordinate system.
The coordinate system of the object in which the visualization is performed: the object must be placed in the world coordinate system, while having its absolute position.
The coordinate system of the camera must match the coordinate system of the object. After
launching ARKit, the user moves the smartphone in space. At this time, the camera changes the
coordinates from the world coordinate system and the object is already in the coordinate system of the
camera [
The relationship between the camera's coordinate system and the screen's coordinate system can be expressed using the formula (Figure 5).
Thus, the matrix of the projection of the image on the screen P can be determined by the
Formula 1:
= (
0
0
0
0
0
0
0
−1
0
0
0
0 ( + )⁄( − )
2 ⁄( − )
)
(1)
window by the formula winZ = (q.z + 1) / 2.
the z axis. Namely,
where x and y regulate the angle of view and the ratio of the sides, f and n - the depth of distance
and approximation to a certain distance, respectively [
Thus the projection of a point in the world coordinate system z = (0 0 z 1) T is given by calculating p = P * z, and then using the perspective distribution q = p / p.w. Next we need to calculate the depth of the Following these formulas, we can derive a relationship between winZ and the z coordinate of our point on winZ = (p.z / p.w + 1) / 2 p.z / p.w = 2 * winZ-1
p.w = -z p.z = z * (f + n) / (n-f) + 2fn / (n-f) (z (f + n) / (n-f) + 2fn / (n-f)) / (- z) = 2winZ-1
z = fn / (f * winZ-n * winZ-f) or equivalent to winZ = f (n + z) / (f-n) z) (2)
It is notable that when z = -f, then winZ = 1, and when z = -n, then winZ = 0. Meanwhile, we have the inverse relationship (Figure 6):
During the research, an augmented reality-based information system has been developed in the form of a mobile application for iOS using Swift programming language and ARKit library. This system allows to record the route from point A to point B in real time, save it and display it using augmented reality (the direction of movement from point A to point B is shown by arrows) at the request of the user. All user-recorded routes are stored in the mobile application database and available for sharing for users who have the app installed on their phones.
In the course of the research, an augmented reality-based information system has been developed in the form of a mobile application for iOS using Swift programming language, ARKit library and SceneKit framework. This system allows to record the route from point A to point B in real time, save it and display it, using augmented reality (the direction of movement from point A to point B is shown by arrows) at the request of the user. All user-recorded routes are stored in the mobile application database and available for sharing for users who have the app installed on their phones.
The experiments have been conducted over the campus of Khmelnytskyi National University (KhNU). The following technologies have been used to accomplish the task of the research: iOS operating system, Swift programming language and ARKit augmented reality library. Two of the most popular routes were plotted on the campus plan (Figure 7).
The route marked yellow is the route from academic building 3, where the administration of KhNU is located and the classes are held, to Hostel 3, where the students of IT faculty live.
The route marked red is the route from Hostel 4, where foreign students live, through the botanical garden passing the workshors base, where some practical classes on Technical disciplines are being held, towards academic building 3.
Figure 8 shows interface windows of the developed AR-based information system for route planning and visualization. a) main screen (after the authorization); b) main menu with the saved routes; c) the process of routing using augmented reality. a) b) c) based mobile application for routing and visualization the routes.
The quality and the efficiency of ARroute application was comparatively evaluated using the main
performance parameters, such as Memory usage, Application Launch Performance, Memory Leaks,
maximum CPU usage, Energy Impact and quantity of frames per second (FPS). Proposed architecture
and software development model in this paper works quite well as compared to the other similar
applications. The comparative analysis has been conducted between ARroute and Feed Me
application [
Figure 9 shows that panorama views and 3D routing provided by Google Maps use already taken and saved by other user videos. For the experiment we tested 3D route across Botanical Garden. The time stamp shows that this video has been taken in April 2019. Therefore we can conclude that Google Maps do not provide real time 3D routes paving. In return ARroute provides real time route paving. It helps user to intuitively build the route using augmented reality markers. The saved route can be reproduced using a device camera in real time once needed.
Thus, the proposed augmented reality-based information system for routing and route visualization provides quick and accurate route paving and saving the route with the following display at the request of the user.
During the study the survey among students was conducted, which showed the relevance and the necessity of the research and development of the proposed mobile application. Also the literature analysis and analysis of already existing AR-based mobile applications provided the conclusions about the technologies that have been used for the development of the proposed augmented realitybased information system for routing and route visualization.
The performance testing, which has been performed during the experiment, showed that the proposed ARroute application has high application launch performance, no memory leaks, low energy consumption and high image processing frequency.The developed application works quite well, has a user friendly and intuitive interface.
The further efforts of the authors will be directed to improving the existing algorithms for the work with augmented reality technology, conducting more experiments and improving the proposed mobile application.