3D Photorealistic Radio Map Generation Using Gaussian Splatting toward On-site Surveys Yuma Toyoda1,∗ , Sho Maeoki1 , Naoya Yokono1 and Wataru Yamada1 1 NTT DOCOMO, INC. Chiyoda-ku, Tokyo, Japan Abstract Thanks to mobile network operators who maintain their networks, we can communicate anywhere with high speed and stability over them. Radio maps, data sets of radio information at each location, are essential to maintain communication quality in cellular networks. Radio maps are typically collected by measuring radio information and plotting locations manually in on-site surveys; however, this investigation work is a hard burden. Although several studies have been proposed to efficiently generate radio maps by automatically acquiring location information, those methods are limited to constructing 2D maps or necessitate expensive equipment. This paper proposes a time-efficient and low-cost method to generate a radio map with a 3D photorealistic map (i.e., a 3DP radio map) using an omnidirectional camera and a smartphone. Intuitive comprehension of radio conditions and the measurement environment with the 3DP radio map can help maintainers to analyze and improve communication states. The experiments demonstrated the efficiency of radio map generation by the proposed method and revealed the beneficial factors of the 3DP radio map for improving communication quality. Keywords Radio Map, Cellular, 3D Mapping, Gaussian Splatting, Computer Vision 1. Introduction Mobile network operators (MNOs) maintain their networks to address the demand for high- capacity communications using smartphones. Thanks to their constant effort, we can enjoy high-quality and stable communication. The MNOs collect radio maps [1], data sets of radio information at each position in measure- ment areas, to maintain communication quality in their networks. The characteristics of radio propagation in the cellular networks can be observed from radio maps. Based on the analysis of radio maps, MNOs devise ways for specific purposes, such as adjusting the parameters of existing base stations or installing new base stations. There are two typical approaches to obtain radio maps. One is the automatic collection of communication status from mobile devices. This approach automatically collects radio information by monitoring the communication status between base stations and mobile devices of massive users. Although this approach facilitates the grasp of radio conditions in large areas, Proceedings of the Work-in-Progress Papers at the 14th International Conference on Indoor Positioning and Indoor Navigation (IPIN-WiP 2024),October 14- 17, 2024, Hong Kong ∗ Corresponding author. Envelope-Open yuuma.toyoda.hx@nttdocomo.com (Y. Toyoda); syou.maeoki.rz@nttdocomo.com (S. Maeoki); naoya.yokono.kz@nttdocomo.com (N. Yokono); wataruyamada@acm.org (W. Yamada) Orcid 0000-0002-0050-2890 (Y. Toyoda) © 2024 Copyright for this paper by its authors. Use permitted under Creative Commons License Attribution 4.0 International (CC BY 4.0). CEUR ceur-ws.org Workshop ISSN 1613-0073 Proceedings (a) Video Capturing and (b) Processing of Video Data (d) Generating 3DP Radio Map Radio Measurement Structure Gaussian from Motion Splatting 360-degree (SfM) 3D Sparse 3D Photorealistic 3D Photorealistic Radio Map Omnidirectional Camera Video Data Point Cloud Data Spatial Data (3DP Radio Map) Camera Position Data Smartphone Measurement (c) Processing of Radio Information Device Radio Information Radio Map Joining Process elapsed time bands rsrp … X Y Z • Elapsed time ・ Radio information and 2.0 [1] -97 1.19 -3.78 -0.20 ・ Radio conditions are visualized • Bands 4.0 [1] -95 1.27 -3.60 -0.19 • RSRP position data are associated 6.0 [1] -94 … 1.33 -3.38 -0.18 • etc. according to the elapsed time in 3D photorealistic space. 8.0 [1] -93 1.46 -3.22 -0.16 10.0 [1] -93 1.57 -3.11 -0.18 ︙ Figure 1: Overview of the proposed method. Proposed method consists of four main steps. it is not suitable for detailed analysis of a specific small area due to its inability to provide enough data where the radio wave information is required to be collected. The other is manual on-site surveys by MNO’s maintainers. The maintainers survey radio conditions on site with smartphones. In a typical survey procedure, the maintainers collect radio information at each position in the target area while manually plotting measurement points. Although this survey is a hard burden, it is still conducted, especially for some facilities with high priority (e.g., train stations and event venues), as it can provide detailed radio maps. Exploring alternatives to the manual plotting approach, several studies have proposed ways to reduce the burden of manual on-site surveys [2, 3]. For example, 2D light detection and rangings (LiDARs) or smartphones are used to automatically plot measurement points for radio information. Point cloud data acquired from 2D LiDARs or image data acquired from smartphone cameras are used as input to the simultaneous localization and mapping (SLAM) algorithm. Consequently, a 2D map and a radio map can be generated at the same time. Moreover, a 3D LiDAR-based method [4] has also been proposed to generate radio maps. With this method, a 3D map of the surveyed area can be obtained, which is impossible with 2D LiDAR and smartphone-based methods. With the generated 3D map, we can observe the structure and arrangement of objects and partitions in the target area, based on which we can analyze radio conditions without going there again. Additionally, the 3D reconstructed map can be used to simulate radio wave propagation by ray tracing. In spite of such advantages, unfortunately, 3D LiDAR is very expensive and specialized equipment. For this reason, conducting large-scale and frequent on-site surveys may present significant challenges due to the need for extensive preparation of numerous devices and the associated high costs. Furthermore, if it is leveraged for optimizing the radio environment, the generated 3D map is ought to be composed of photorealistic point cloud data that allows for intuitive recognition of the target area. The acquisition of high-density point clouds for generating 3D photorealistic data requires even more expensive 3D LiDARs. We propose a time-efficient and low-cost method to generate a radio map and a 3D photoreal- istic map using an omnidirectional camera and a smartphone. In this paper, the generated radio map together with a 3D photorealistic map are referred to as the 3D photorealistic radio map (3DP radio map) as shown in Figure 1. The proposed method initially captures 360-degree video data and measures radio information simultaneously. A measured radio map is superimposed on a reconstructed 3D photorealistic map. The 3D map is reconstructed by Gaussian Splatting method [5]. The 3DP radio map provides an intuitive recognition of radio environments in the target area; hence, it helps radio surveyors to analyze radio conditions and consider methods to improve the quality of radio communications. We conduct two experiments to evaluate the time-efficiency in generating a radio map and the helpfulness in analyzing radio conditions of the proposed method. The first experiment is a comparison of the proposed method and the manual plotting method for the same duration of time at the same on-site survey location. The second experiment is that we interview three experts of on-site radio surveys, where we request them to fill out questionnaires and hold discussions regarding the effects of 3D photorealistic data under the same observing radio conditions. The experimental results demonstrate that the proposed method can obtain approximately 8.2 times more measurement points in the same measurement time compared to the method of plotting locations manually. Furthermore, the 3D photorealistic information in the measurement environment is revealed to be more expressive and suggestive in the analysis of radio conditions using radio maps than the information simply given by a radio map based on a 2D map. 2. Proposed Method The proposed method overview is illustrated in Figure 1. There are primarily four main processes in the generation of the 3DP radio map: video capturing and radio measurement, processing of video data, processing of radio information, and generating a 3DP radio map. First, video data and radio information are collected as illustrated in Figure 1a. Next, position data and 3D photorealistic spatial data are obtained based on the captured video data as shown in Figure 1b. In parallel, the measured radio information is processed into a radio map as in Figure 1c. Subsequently, a 3DP radio map is generated by combining the radio map and the 3D photorealistic spatial data. Video Capturing and Radio Measurement. We prepare a device that combines an omni- directional camera and a smartphone to simultaneously capture video data and measure radio information in the target area, as shown in Figure 1a. While holding this device, we walk around the target area to collect video data and radio information. We slowly and carefully move around the area during the measurement to reduce the blurring of omnidirectional videos, which results in improving the quality of 3D reconstructed data. Note that the coordinates of the omnidirectional camera are regarded as the position data for the radio map, because the camera and the smartphone are fixed at close range. Processing of Video Data. From the acquired video data, we generate position data and 3D photorealistic spatial data (i.e. 3D reconstructed data by gaussian splatting) for the 3DP radio map, as shown in Figure 1b. As a pre-processing step, the video captured by an omnidirectional camera, which contains 360-degree information, is divided into eight 2D sequence images with different angles every 45 degrees in the horizontal direction. The processed image sequence data is fed into a structure from motion (SfM) algorithm [6, 7], and consequently camera coordinates and sparse 3D point cloud data are generated. Then, Gaussian Splatting is performed with the obtained camera positions and the sparse 3D point cloud as input data to reconstruct 3D photorealistic data. Processing of Radio Information. Combining the radio information collected from the camera position data calculated in the previous process, we produce a radio map for the 3DP radio map, as illustrated in Figure 1c. The radio information includes various data significant for the analysis of radio conditions represented by RSRP, RSRQ, and bands. The elapsed time from the start of measurement is also recorded to construct a radio map. Given the elapsed time as a key, the radio information is joined to the position data. Note that the measurement interval for cellular data is constrained to once per approximately two seconds due to Android specifications. Generating 3DP radio map. The 3DP radio map is finally generated by combining the 3D photorealistic spatial data and the radio map, as shown in Figure 1d. The generated 3DP radio map enables the observation of radio information in the measurement area, which is helpful for the analysis of radio conditions. 3. Evaluation We conducted two experiments to evaluate how much time for the on-site radio surveys can be reduced by the proposed method and how the analysis for improvement of communication quality can be facilitated with the 3DP radio map. The one experiment campared the number of measurement points between the proposed method and a manual plotting method in radio map generation. The other was a comparison of the 3DP radio map and the conventional 2D radio map, where the radio map is visualized on a 2D map, through questionnaires and interviews with experts involved in on-site radio surveys. 3.1. Experimental Setup We collected data for generating a radio map by walking around in an event room approximately 10 × 20 meters, as shown in Figure 2a. The measurement was performed along a route to cover the entire space, as illustrated in Figure 2b. The RICOH THETA X was adopted as an omnidirectional camera for capturing 360-degree video data and the Google Pixel 8 Pro was used as a smartphone for measuring radio information. The radio information was acquired using the TelephonyManager class 1 provided by the Android SDK. 3.2. Measurement time in on-site radio surveys We compared the number of measurement points of radio maps to evaluate whether the proposed method can reduce the burdens of the manual on-site surveys. We conducted the experiment of radio map generation to compare the measurement points collected over the same duration of time for both the proposed method and the manual plotting method in the on-site radio survey. 1 https://developer.android.com/reference/android/telephony/TelephonyManager (a) (b) (c) Measurement points are plotted Measurement Start/End Point on a printed 2D map Corresponding time is recorded on a printed sheet Figure 2: (a) Image of the measurement environment. (b) Measurement route. (c) Image of a clipboard used for the manual plotting method. The experiment consists of the following two steps. The first step was measurement using the proposed method. We performed the measurement with the proposed method while slowly moving along the route shown in Figure 2b. The measurement took 6 minutes and 40 seconds. For the second step, we employed the manual plotting method, a typical method in on-site radio surveys for producing a radio map. We plotted the locations at where radio information was measured on a 2D map. The location and the corresponding time for constructing a radio map were recorded by hand on a printed map and sheet, both of which are attached to a clipboard during the measurement as depicted in Figure 2c. The measurement time was ensured to be the same 6 minutes and 40 seconds as for the proposed method. The number of measurement points obtained by the proposed method was 188, whereas the manual plotting method resulted in 23 measurement points, as shown in Figure 3. The experiment results demonstrate that the proposed method can obtain approximately 8.2 times more measurement points over the same duration of time compared to the manual plotting method. These results indicate that the proposed method can generate dense radio maps with a greater number of measurement points in a shorter period of time compared to the manual plotting method, resulting in a reduction of the burdens in the manual on-site surveys. 3.3. How 3DP radio map facilitates analysis and discussion Questionnaires and interviews were conducted with on-site radio survey experts to evaluate which aspects of the 3DP radio map are practical in the study for improving communication (a) (b) Number of measurement points: 188 Number of measurement points: 23 Figure 3: (a) Measurement points by the proposed method. (b) Measurement points by the manual plotting method. quality. 3.3.1. Settings We asked three experts working on on-site radio surveys in the MNO to review both the conventional 2D radio map and the 3DP radio map in terms of their advantages for analyzing and improving radio conditions. The experts had two to five years of experience in on-site radio surveys and analysis for communication quality improvement. As the conventional 2D radio map, images of radio wave information and measurement location visualized on a 2D map as illustrated in Figure 4a was shown to the experts. Subsequently, we presented the experts with the 3DP radio map, as shown in Figure 4b. We requested the experts to complete the questionnaire after reviewing each radio map. The questionnaire consisted of six questions in total, including the four categories described below. • Recognition of the survey environment appearance (Q1): Whether the experts can grasp the appearance and the structure of the target area. • Comprehension of radio conditions (Q2-Q4): Whether the experts can interpret the distribution of reference signal received power (RSRP), bands, and physical cell id (PCI). • How suggestive for improvement approaches (Q5): Whether the experts can come up with some plans for improving radio conditions. • Overall assessment (Q6): Which method the experts prefer to adopt for on-site radio surveys. Questionnaires were rated on the 5-point Likert scale. A rating closer to 1 indicates that the conventional 2D radio map is superior, whereas a rating closer to 5 indicates that the proposed 3DP radio map is superior. After questionnaire completion, we conducted group interviews with the three experts to explore the reasons for their responses to questionnaires, as well as the advantages and potential improvements of the proposed method. (a) (b) (c) 3DP radio map Conventional 2D radio map Q6 Q4 dio c Q5 prov Q2 Q1 viron Q3 dio c ra im rad en ra :O : C ond : H em : C nd : R me : C ond i o om itio ow en om itio ve ec nt om itio c o ra og ap pre ns pre ns pre ns su t ap ll a nit pe gg pro he (PC he (R he (Ba sse ion ar es ns ns SR ns tive ache ssm of anc ion I) ion P) ion nds) the e of for s of en of su t rve y Figure 4: Results of the experiments. (a) Radio map visualized on a 2D map. (b) 3DP radio map by the proposed method. (c) Questionnaire results. Note that the RSRP visualization at each measurement point in (a) and (b) are colored black to red. 3.3.2. Results The results of the questionnaire regarding Q1 showed that the experts unanimously rated the proposed method as excellent in terms of recognizing the appearance of the target environment as shown in Figure 4c. The experts mentioned that the 3DP radio map is beneficial for their work thanks to its intuitive visual information on the target area. According to their comments, this could be explained that the 3DP radio map can reveal the placement and height of objects such as desks, chairs, and partitions. We assume that the photorealistic visualization of the target area can help analyze radio conditions without being on-site from the results and comments of Q1. Regarding the comprehension of radio conditions, there were different opinions between those who responded that the proposed method is superior and those who answered that the conventional 2D radio map is better as illustrated in Q2 and Q3 of Figure 4c. Subject C who rated the proposed method highly appreciated the ability to visualize the strength and distribution of radio signals in a 3D space. On the other hand, subject A, who supported the 2D radio map, and subject B, who considered both the same, argued that even though the 3DP radio map is preferred in terms of comprehending local radio conditions, the 2D map provides a more holistic view of the target area that helps us comprehend the overall distribution of radio signals. The superiority of the conventional method was also demonstrated in Q4 because the radio map with 2D map is sufficient for comprehending the PCI distribution. As a comprehensive insight into the interpretation of radio conditions, these results suggest that the conventional 2D map is superior to the 3DP radio map in terms of overall visibility of the radio wave situation, whereas the 3DP radio map is superior in terms of recognizing the structure of the target area for considering radio wave propagation and interference. Regarding how suggestive for improving communication quality, the result showed the superiority of the proposed method as shown in Q5 of Figure 4c. The experts evaluated that the photorealistic 3D data, which contains the arrangement of windows and the height of objects, is valuable for examining approaches to communication quality. Therefore, the 3DP radio map enables the experts to analyze radio propagation and interference in the target area. Particularly, the experts advocated that the photorealistic appearance provoked discussions about the placement of base stations for improving radio quality. Finally, in terms of the overall assessment, the 3DP radio maps showed a tendency to be superior as illustrated in Figure 4c (Q6). The experts indicated that the visualization of radio conditions on a 3D space in the measurement environment contributes to the beneficial con- sideration of approaches for improving communication quality. These results suggest that the intuitive comprehension of radio conditions in the target area provided by the 3DP radio map facilitates suggestions to improve communication quality compared to conventional 2D radio maps. We consider that combining the conventional 2D radio maps and the 3DP radio map will be effective for analyzing radio conditions in the target area because each of them has different effective usage scenarios. These two radio maps can be used together to help MNOs maintain their networks in the future. 4. Conclusion In this paper, we proposed a method for generating a 3DP radio map (i.e., a radio map with 3D photorealistic spatial data), that provides important insights into the consideration of approaches for improving communication quality. The proposed method generates a 3DP radio map that visually represents measured radio conditions in a photorealistic 3D data by simultaneously capturing omnidirectional video data and collecting radio information. This method enables efficient generation of a radio map useful for on-site radio surveys without the need for expensive equipment. Through experiments, we demonstrated that the proposed method is capable of efficiently generating radio maps. Moreover, the intuitive visual information of radio conditions provided by the 3DP radio map is more effective in the analysis for improving communication quality through the evaluation by experts of on-site radio surveys. We will consider a measurement method that take into account the influences of orientation and placement of a smartphone on the acquired radio information as a further study. In addition, we will explore the application of the proposed method to automatic measurement of radio conditions using robots and to fingerprint-based localization using measured radio information. References [1] D. Romero, S.-J. Kim, Radio Map Estimation: A data-driven approach to spectrum cartogra- phy, IEEE Signal Processing Magazine 39 (2022). [2] D.-I. Nastac, E.-S. Lehan, F. A. Iftimie, O. Arsene, B. Cramariuc, Automatic Data Acquisition with Robots for Indoor Fingerprinting, in: International Conference on Communications (COMM), IEEE, 2018, pp. 321–326. [3] T. Liu, X. Zhang, Q. Li, Z. Fang, A Visual-based Approach for Indoor Radio Map Construction Using Smartphones, Sensors 17 (2017). [4] Y.-C. Lee, W. Yu, 3D Portable Mapping System to Build Radio Fingerprints and Spatial Map, in: International Conference on Information and Communication Technology Convergence (ICTC), 2020, pp. 1641–1643. [5] B. Kerbl, G. Kopanas, T. Leimkühler, G. Drettakis, 3D Gaussian Splatting for Real-Time Radiance Field Rendering, ACM Transactions on Graphics 42 (2023). [6] J. L. Schönberger, J.-M. Frahm, Structure-from-Motion Revisited, in: Conference on Computer Vision and Pattern Recognition (CVPR), 2016, pp. 4104–4113. [7] J. L. Schönberger, E. Zheng, M. Pollefeys, J.-M. Frahm, Pixelwise View Selection for Un- structured Multi-View Stereo, in: European Conference on Computer Vision (ECCV), 2016, pp. 501–518.