This study proposes and evaluates a method to estimate indoor position by environmental ultrasound as latent information in a target environment. The method converts environmental ultrasounds into spectrograms and builds a nonlinear regression model based on the spectrograms using a convolutional neural network to estimate the indoor position and direction. This study evaluated the proposed method's estimation accuracy of the position and direction in a room. In a position estimation experiment using ultrasounds recorded at 20 measuring point in the target room, a position was estimated based on a regression model trained with training data. The model has a root mean squared error (RMSE) of 1.01 m for test data. This result demonstrates that the proposed method is effective for indoor position estimation. In a direction estimation experiment, the estimation error was 78.81° for test data. This result shows that it is difficult to estimate the direction of an ultrasonic microphone used in the experiment.
Indoor position estimation techniques are being studied at present, but many of them require installation of devices such as beacons. We think that information which already exists in an indoor environment makes it possible to perform indoor position estimation which keeps cost down. In this research, we focused on environmental ultrasound as latent information in an indoor environment. Environmental ultrasound refers to a mixture of ultrasounds emitted from indoor devices such as air conditioning equipment and switching power supply.
Tsuchiya et al. have shown that it is possible to identify multiple rooms by using
environmental ultrasound [
In the case of a method using radio waves, a method based on triangulation is often used if the position of the transmitting source is known. A method of triangulation mainly uses received signal strength or time of flight between a transmitting source and a receiving terminal. When the position of the transmitting source is unknown, many methods based on position fingerprinting are used. In the case of using environmental ultrasounds, estimation based on position fingerprinting is performed.
There are many methods that use Wi-Fi, which is already installed in many indoor
environments [
Tsuchiya et al. showed that environmental ultrasounds in rooms have unique
characteristics and demonstrated that analysis of ultrasounds could distinguish multiple
rooms with an accuracy of 97.5% [
We propose a method to estimate the position and direction within a room using environmental ultrasound. The method is partially based on the work of Tsuchiya et al. but differs in that we estimate the coordinates and azimuth of a recoding ultrasonic microphone by using non-linear regression models. The regression models are built by CNN with spectrograms of ultrasound. Since our method is a type of fingerprinting, we need to collect ultrasounds at various positions and directions to build non-linear regression models for coordinates and azimuth. 3.1
An example of a spectrogram is shown in Fig.1. A spectrogram is a heat map representing relationships among time, frequency, and intensity. In the spectrogram, the horizontal axis represents time, the vertical axis represents a frequency band, and the gray scale represents intensity. Short-term changes such as beats of fan noises of an air conditioner can be important features of ultrasound fingerprinting. The spectrogram can include such importance features which cannot be expressed by simple acoustic features such as Fourier coefficients for a period. 3.2
CNN is one of deep learning algorithms that show high accuracy in image recognition. CNN performs feature extraction in the image by the convolution layer and the pooling layer that allows misalignment. We experimented to evaluate the effectiveness of position and direction estimation by the proposed method. This section describes the experiment and its results. We collected ultrasounds in one room. The recorded environmental ultrasound is converted into a spectrogram. Pairs of the spectrogram and recording position of ultrasound were accumulated to a dataset. We divide the created data set into training data and test data. Non-linear regression models of the positions were built by CNN. The features of the spectrogram for each position of training data were learned. Recording positions of test data are estimated using the created models, and the estimation error is evaluated. 4.2
Ultrasounds were collected in a room whose dimensions are 5.93 m long × 4.73 m wide × 2.60 m high. The room has 0.3 m × 0.4 m pillars at its four corners, which are indicated by black rectangles in Fig.2(a).
A Dodotronic Ultramic 250k ultrasonic microphone was used as the recording device. Since the maximum sampling rate of the microphone is 250 kHz, ultrasounds can be recorded up to 125 kHz. The microphone was connected to a notebook-type personal computer (PC) which captured and saved ultrasounds. The PC was fixed at the center of the room to reduce the influence of any ultrasound generated by the PC.
We placed 20 measuring points indicated by dots in Fig.2(a), which were points at the intersections on a 1.2 × 1.2 m grid, within a room, and performed recording at the 20 points. Also, to evaluate the influence of recording direction, recordings in four directions were collected at each measuring point as shown in Fig.2(a). This dataset which contains four direction data is called dataset R-all.
The length of one recording was two seconds, and we repeatedly recorded ultrasounds at one-second intervals. In this experiment, we only used sound data of 20k to 125 kHz because a spectrum of the audible range is affected by sounds caused by daily human activities (e.g., human voices).
To investigate the influence of recording direction on position estimation, we compared the estimation errors of the datasets for each direction and all directions. We refer to the set of all data recorded with direction 0° as dataset R0, which contains 10,000 recordings. Similarly, sets of data recorded with directions 90°, 180°, and 270° are referred to respectively as datasets R1, R2, and R3. For dataset R-all, 200 data were collected in each direction, 800 data in each position, 16,000 data for all. We used 80 % of the data in each direction at each point as training data and the remaining 20% as test data.
0.6 1.8 3.0 4.2 (m)
window 0 0.6 1.8 3.0 5.4 (m)
PC door
= Measuring Point (a).Position Estimation Root mean squared error (RMSE) is used as a quantitative measure of estimation error. RMSE shows an average of distance between coordinates of a ground truth and an estimation. The unit of distance is meter in this research.
In this experiment, we configure two non-linear regression model which estimate X and Y coordinate. We defined RMSE of X and Y coordinates independently by (1), (2) as RMSEx and RMSEy where a both of data i is respectively ( , ) and ( ̂ , ̂ ) and the number of data is n. RMSE between ground truth positions and estimated positions is defined (3).
RMSE = RMSE = √ = 1( ̂ − )⁄ √ = 1( ̂ − )⁄ RMSE = √RMSE 2 + RMSE 2 (1) (2) (3) 4.6
horizontal and vertical axes of the graph represent respectively the number of epochs of learning and RMSE. The solid and dotted lines in Fig.3(a) indicate RMSE for the training data and the test data, respectively. At epoch 100, the RMSE of the training data was 0.15 m, and that of the test data was 1.01 m. values of dataset R0, R1, R2, and R3, each of which includes data for one direction, is lower than the RMSE values of dataset R-all, which includes all of four direction data. (m) 4.5 4.0 3.5 3.0 E 2.5 SM 2.0 R 1.5 1.0 0.5 0.0 (m) 1.4 test
R0
R1
R2
R3
R-all test In the experiment using the dataset R-all that mixed the recording data of four directions, RMSE for training data was about 0.15 m. This result shows that learning epoch is sufficient and the indoor environment ultrasounds had different characteristics for each position.
However, RMSE for test data fluctuated between 1 m and 1.5 m, as shown in Fig.3(a). Moreover, the estimation performance for training data was higher than that for test data. Those suggest that the non-linear regression model fell into overfitting.
The RMSE of R0, R1, R2, and R3 using only unidirectional data is lower than that of R-all. This result suggests that recording directions affect position estimation accuracy. If we get orientation information, for example, from a magnetic field sensor in a smartphone, we could improve the accuracy and robustness to microphone direction of the proposed method. 5
From result of position Estimation, recording direction of microphone effect position estimation accuracy. In this section, we performed direction estimation experiment by proposed method. 5.1
We followed the same procedure as the experiment described in Section 4. The same microphone and room as the position estimation experiment were used for the recording. In this experiment, we estimate indoor direction by non-linear regression model.
Ultrasounds were recorded in 16 directions from the center of the room, as shown in Fig.2(b). An ultrasonic microphone was placed on a desk at the center of the room. The recording angle was varied from 0 to 337.5° in increments of 22.5°.
Dataset contains 1,000 items of data in each direction, for a total of 16,000 items. We used 80% of the data in each direction as training data and the remaining 20% as test data. 5.3
Fig. 4. shows the transition of the estimation error. The horizontal axis of the graph represents the number of epochs of learning and the vertical axis represents estimation error. The unit of estimation error is a degree. The solid and dotted lines in the graph indicate the estimation error for the training data and test data of datasets, respectively. At epoch 100, the estimation error for the test data was respectively 78.81°. (degree) 110
100 ro 90 err 80 on 70 ita 60 itm 50 sE 40 0 20 40
60 training 80 test 100 As a result of direction estimation error for test data is about 80°. In the position experiment, Fig.3(b) shows that environmental ultrasound recorded in 90 ° units has different features. If the direction estimation result is used to specify the recording direction of the microphone in position estimation, an error of about 90 degrees is insufficient in accuracy from the R-all result. Therefore, further investigation is needed to clarify the cause. This paper proposed and evaluated a method to estimate indoor position and direction within a room based on environmental ultrasounds. In a position estimation, the results suggest that environmental ultrasound can be used to estimate a position in one room and that estimation accuracy can be improved by utilizing orientation information together. In a direction estimation, it was difficult to estimate the direction of an ultrasonic microphone used in the experiment.