This study aims to improve the existing indoor spatial cognition systems for hearing and visually impaired people, especially in unfamiliar spaces where outdoor tools are not functional. The experiment conducted in the academic multistage building of the University of Central Asia (Naryn campus, Kyrgyz Republic) demonstrates a positioning accuracy of 100 % for indoor localization. For this purpose, two Java Android applications were developed on the Android 10 operating system. The first application is employed to design a WiFi heatmap of the building using the Java Android class WifiManager to gather information about BSSIDs (Basic Service Set Identifiers) of WiFi networks indoors (this task might be done by any project team member). The second application also implements the principle of the pentest passive reconnaissance and provides audio feedback to the visually impaired and textual/photo information to people with disabling hearing loss. Java Android applications are freeware and runnable on over 81 % of Android devices and 71 % of smartphones worldwide as of June 2024.
The World Health Organization reported that over 2.2 billion and 430 million people worldwide have
vision impairment and disabling hearing loss, respectively [
Existing assistive solutions, such as DeafSpace [
They have distinctive drawbacks, such as requiring the installation of new equipment like Bluetooth low-energy beacons and complicated algorithms like image processing, that demand highperformance devices.
This study addresses an important social problem of the indoor spatial cognition of HVIs. Passive
reconnaissance with a WiFi heatmap as the main outcome is the distinctive feature and advantage
of the proposed approach inspired by the pentest ethical hacking methods [
The final product is an Android .apk file, which can be shared via Google Play Store, university website, and/or direct copying. It is designed to run on approximately 81 % of Android devices as of June 2024, as it was developed for the Android operating system (OS) 10.
To support the indoor spatial cognition of HVIs, a new Java Android mobile application was
developed to find the location of HVIs inside a multistage s contribution jointly
includes the following:
1. Affordability: the software was developed in the Android Studio 4.0 free of charge, and it is
planned to be shared without any cost.
2. User-friendly interface: textual and photo information is shown to the hearing impaired [
This paper is organized as follows: Section 2 presents the analysis of previous studies in the context of the active or passive reconnaissance pentest techniques.
It also introduces the general architecture of the assistive system for the HVI indoor spatial cognition.
Section 3 discusses three main phases of the project lifecycle: mapping the academic multistage building of the University of Central Asia; designing the WiFi heatmap and indoor spatial cognition of HVIs using two developed Java Android applications.
Section 4 describes the successful experiment conducted at the University of Central Asia: indoor spatial cognition of HVIs using a Java Android Application and the WiFi heatmap. Results and discussion are presented in Section 5. Conclusions are summarized in Section 6.
Spatial awareness is a crucial and most difficult task in HVI navigation. Existing positioning
methodologies might be divided into four main categories [
Global navigation system-based methods, such as the pseudolite GPS [
Microelectromechanical sensor systems and magnetic field distribution-based methods require
frequent recalibration, which can be quite complex after software release and updates. Computer
vision needs high-performance soft-/hardware to provide accurate real-time feedback. Authors have
a positive experience with QR codes [
However, they may require additional space and permissions that organizations, as discussed in this study, may not always allow.
This study focuses only on wireless networks, as the end user relies solely on a wireless connection to locate indoor areas in the academic multistage building.
The analysis of previous studies showed that two opposite techniques, active and passive
reconnaissance, are employed to scan wireless networks inside multistage buildings. In active
reconnaissance, additional equipment like Bluetooth low-energy beacons and RFID (Radio Frequency
Identification) tags, such as presented in [
Typically, network administrators avoid using the same Basic Service Set Identifiers (BSSIDs) with Media Access Control (MAC) physical addresses when the network is reconfigured and/or access points or wireless routers are replaced.
The key benefit of this technique is its independence from the host network topology and characteristics such as Bluetooth addresses and BSSIDs.
A major drawback is that the installation of additional equipment is not free and is usually not allowed by the organization's policies.
In this study, a passive reconnaissance technique [
Regardless of the localization technique, the mobile application user interface should include
the following features to ensure accessibility for HVIs [
Visually impaired: the audio [19] should be played, and the button as a user interface element should be placed next to the smartphone screen border (the top in this project) to perform the scan of the WiFi host network.
Hearing impaired: textual and visual (photos in this project) information should be displayed on the smartphone screen.
The general architecture of an assistive system for the HVI indoor spatial cognition based on the mobile application, the passive reconnaissance, and the above-stated analysis of previous studies is shown in Figure 1. The backend of the project, i.e., the database with BSSIDs and indoor locations, can be implemented as a part of the mobile application or using a Backend-as-a-Service app development platform like Google Firebase with a cloud-hosted NoSQL Realtime Database. In the current version, the first approach is employed since the network administrator can change the MAC address of the WiFi interface to the required one. The mobile application uses the database and the BSSIDs inside a building to identify the location and provide the textual/visual/audio information to HVI.
The project lifecycle consists of three main phases: mapping [20] the academic multistage building of the University of Central Asia; designing the WiFi heatmap and indoor spatial cognition of HVIs using the first and second developed Java Android applications, respectively.
The University of Central Asia has several constructions on the Naryn campus, but only the academic multistage building (it is marked out with the red arrow in Figure 2) is challenging for the HVI spatial cognition, as other facilities are not typically visited by newcomers. In Figure 2, the photo was captured by the flagship triple-camera drone DJI Mavic 3 Pro Cine by author Nurlan Shaidullaev on June 14, 2024. Academic building 1, 2, and 3 apartments of teaching staff; 4, 5, and 6 central corridor; 7, 8, 9, and 40 main entrance; 19 computer class; 20, 21, and 22 canteen; 23, 24, and 25 library. • • • 10, 11, and 12 classes; 13, 14, and 15 central corridor; 16, 17, and 18 apartments of teaching staff; 35 yellow classroom; 36, 37, 38, and 39 corridor to dorms. offices of teaching staff; central corridor; apartments. • 39 • 1 • 2 • 3 • 18 • 17 • 16 • 4 • 22 • 25 • 15 • 38 • 5 • 21 • 24 • 14 • 37
The distance between points ranges from 15 m to 20 m around the academic building. In this study, the positioning accuracy takes binary values: if the positioning information is accurate within 10 m of the point, then the accuracy is 100 %; otherwise, it is 0 %. • 6 • 20 • 23 • 13 • 36 • 40 • 10 • 11 • 12 • 9 • 8 • 7 • 19 • 35
The WiFi heatmap is designed using the first developed Java Android application. A Java Android class WifiManager provides information about the SSID (Service Set Identifier), BSSID, and RSSI (Received Signal Strength Indication) of all available WiFi connections at specific points. Figure 6 shows two screenshots examples of SSIDs, BSSIDs, and RSSIs at points 6 and 23. • 26 • 27 • 28 • 34 • 33 • 32 • 31 • 30 • 29
Analysis of BSSIDs in the academic building of the University of Central Asia indicates that most of points 1-40 (see Figures 4, 5, and 6) have unique BSSIDs. An example for the point 1 is as follows (SSID, BSSID, RSSI): • • • • • •
However, several points from different locations may be connected to an access point with the same BSSID: 1. Points 5, 21, and 22: t
the HVI. 2. Points 6, 7, and 23: t : Campus administration & is delivered to the HVI. This case is presented in Figure 6: WiFi connections at these points have at least two common BSSIDs d8:84:66:0f:7d:93 and d8:84:66:0f:7d:90 with quite good RSSI -63.
3. Points 24, 38, and 39: t
The experiment was conducted in the academic multistage building of the University of Central Asia. The WiFi heatmap was used to find the HVI location inside a building. For this purpose, the second Java Android Application was developed. It uses the Java Android class WifiManager to gather the information about SSID, BSSID, and RSSI of all available WiFi connections at specific points. Figure 7 presents four screenshots examples of the successful HVI localization inside a building at point 19 (smartphones are Doogee S96 Pro with Android 10 OS and Samsung M31 SM-M315F/DSN with Android 12 OS) and outdoors . The information (text and photo) is friendly displayed to the hearing impaired, and the audio .mp3 file, i.e., the speech navigation [21], is friendly played to the visually impaired. The audio can be easily switched off on the Android smartphone if necessary. Experimental results demonstrated a positioning accuracy of 100 % for indoor localization.
Figure 7 shows the prospect for future development the constraint layout looks different on screens of non-identical smartphones.
This study presents an assistive system to support the indoor spatial cognition of HVIs. Two Java Android mobile applications were developed to design a WiFi heatmap and to find the location of HVI inside a multistage building. The main result jointly includes the following:
An experiment conducted at the University of Central Asia shows a positioning accuracy of 100 % for indoor localization.
A user-friendly interface provides audio feedback to the visually impaired and textual/photo information to people with disabling hearing loss.
The software was developed in Android Studio 4.0 free of charge, and it is planned to be shared without any cost.
Two questions were raised during the discussion of the presented project at the departments of Computer Science (University of Central Asia) and Computer Systems and Networks (Kryvyi Rih National University):
The constraint layout of the Java Android mobile application appears differently on screens of non-identical smartphones, which should be improved for consistency. A suggestion was
A) to calculate layout parameters based on screen size to ensure uniformity across different smartphones.
The Android mobile applications were developed using the classical imperative approach with the Java programming language. However, it was suggested that the future development of the project should employ a declarative methodology, such as the Kotlin-based Jetpack Compose, which is currently prevalent in Meta Company.
B)
C)
D)
In this study, two Java Android applications (design of a WiFi heatmap and localization of HVIs) were developed to assist the indoor spatial cognition of HVIs using the WiFi heatmap of the academic multistage building of the University of Central Asia (Naryn campus, Kyrgyz Republic). The main contribution jointly includes the following: a positioning accuracy of 100 % for indoor localization; a user-friendly interface that provides audio feedback to the visually impaired and textual/photo information to people with disabling hearing loss; affordable (free-software license) and widely accessible (the developed Java Android applications might be run on over 81 % of Android devices as of June 2024).
The main advantage of the developed assistive system is that no additional equipment is required to be installed since up-to-date Android smartphones can scan WiFi networks and identify BSSIDs employing passive reconnaissance techniques (a Java Android class WifiManager in this project). The Android platform is widely used in the market nowadays, and hence the developed mobile applications might run on over 71 % of mobile devices worldwide as of June 2024. The most likely area for further development of this study is to enhance the Java Android application by adding a navigation component and ensuring that the layout design is responsive to the size of the smartphone screen.
This study and the research behind it have been inspired by the active social position of the Ukrainian Association of the Blind and the Kyrgyz Society of Blind and Deaf. The authors sincerely appreciate their suggestions and assistance in the project development. [18] V. Nair, G. Olmschenk, W.H. Seiple, and Z. Zhu, ASSIST: Evaluating the Usability and Performance of an Indoor navigation assistant for blind and visually impaired people, Assistive Technology 34 3 (2020) 289-299. doi:10.1080/10400435.2020.1809553. [19] H. Jabnoun, M.A. Hashish, and F. Benzarti, Mobile Assistive Application for Blind People in Indoor Navigation. In: M. Jmaiel, M. Mokhtari, B. Abdulrazak, H. Aloulou, and S. Kallel (Eds.) The Impact of Digital Technologies on Public Health in Developed and Developing Countries, ICOST 2020, volume 12157 of Lecture Notes in Computer Science, Springer, Cham. doi:10.1007/978-3-030-51517-1_36. [20] T. Alhmiedat, A.A. Taleb, and G. Samara, A Prototype Navigation System for Guiding Blind People Indoors using NXT Mindstorms, International Journal of Online and Biomedical Engineering 9 5 (2013) 52-58. doi:10.3991/ijoe.v9i5.2848. [21] R. Ivanov, Indoor Navigation System for Visually Impaired, in: Proceedings of the 11th International Conference on Computer Systems and Technologies and Workshop for PhD Students in Computing on International Conference on Computer Systems and Technologies, Sofia Bulgaria, 2010, pp. 143-149. doi:10.1145/1839379.1839405.