Pedestrian Dead Reckoning(PDR) is one of the promising technology for indoor localization. Most of the current PDR techniques can be categorized as the strapdown algorithm and the step-and-heading algorithm [1]. The strapdown algorithms require precise accuracy of the sensor devices to realize the accurate localization. However, most of current smartphones equipped with MEMS sensors do not have enough precision for the double integration, it causes a timecumulative drift-error. On the other hand, the step-and-heading PDR algorithm has the major difficulty in robust estimation of the step length and the step detection.
Step length depends on several parameters such as person's height, walking speed and type of gait. So, in the conventional method [5] [15], it is not easy to estimate step length without using user dependent information. For the step detection, distinguishing "stamp" with usual "walk" is very difficult.
In this paper, we propose an end-to-end walking speed estimation method for smartphone PDR by using DualCNN-LSTM. By estimating pedestrian's walking speed directly from accelerometer sensor data, we do not have to consider about parameters such as step length, person's height nor type of gait. This means we do not have to consider about user dependent information which affects walking-speed parameters. To adapt machine learning algorithm for end-toend speed estimation, we address two problems: 1) how we collect ground truth data of pedestrian's speed and trajectory for training data, and 2) how we design neural network for achieving high accuracy. To collect data for training, we leverage Google Tango [6] for recognizing trajectory, and we calculate pedestrian's speed data by using matrix manipulation with Karman filter. To achieve integration of step-detection and step-length estimation as speed estimation with machine learning technique, we employ LSTM(Long Short Term Memory) [14] with convolutional neural network called CNN-LSTM [12]. Additionally, we extend CNN-LSTM with the fusion of short term convolutional features and long term convolutional features. So we call our network as DualCNN-LSTM. Through our experiments, we confirmed that our method achieves higher precision such as 6.51% error rate compared to 17.55% of existing method .
There are large amount of study which handles pedestrian localization systems [1]. One of the successful PDR is based on ZUPT(Zero Velocity Updates) [2] method which use fixed sensors on the foot [17]. But this method cannot utilize smartphone because it requires to fix the sensors on the foot. Most of smartphone PDR researches use step-and-heading algorithm. For the step detection, Alzantot [5] utilize finite automaton with peak detection. Also, there are several PDR competitions [15] which collects several algorithms to evaluate them under the same condition. In addition, there is a step-length estimation method which utilize stacked autoencoders [11]. These works challenged to increase accuracy of PDR. however, still not achieved enough accuracy for real-world deployment.
In addition, recent advancement of deep learning technology enables end-toend machine learning on different domains [3] [4]. We obtain various technical hints from these researches. One of the most famous end-to-end machine learning system is "Deep Speech" [18] which enables end-to-end speech recognition. By utilizing fully connected layer and bi-directional Recurrent Neural Network, they enabled learning from unaligned transcribed audio dataset.
Our long-term objective is to establish a method for end-to-end PDR which inputs accelerometer and gyro sensor data and outputs relative position movement. However, this paper focuses to estimate pedestrian's speed by end-to-end manner, and calculating trajectory with the speed and heading data. Thus, we propose speed-and-heading PDR algorithm. To best of our knowledge, recognizing heading can be achieved with high accuracy. Compared with heading, current PDR methods' inaccuracy is caused by failing estimation of step length and step counts. So we divide the problem into simpler components -one is the end-to-end horizontal walking speed estimation, and the other is the horizontal heading estimation. This method is different from conventional step-and-heading approach because we don't have to estimate the step count and the step length. Our end-to-end walking speed estimation method inputs accelerometer sensor data and directly outputs terminal movement speed.
To achieve the objective, we address two problems to be solved for end-toend speed estimation. First problem is how we prepare training data for end-toend speed estimation. To adapt machine learning for speed estimation, we need to collect dataset which includes pedestrian's (i.e., smartphone's) trajectory, accelerometer data, gyro sensor data, and speed. It is difficult to get these data from smartphone directly, so that we leverage Google Tango and analyze it's data for preparing speed as ground truth. Second problem is how we model neural network for estimating speed. We surveyed different methods of deep neural network for activity recognition area. Through the survey and our initial experiments with different models of networks, we decided to extend CNN-LSTM for end-to-end speed estimation (see Fig. 1).
End-to-end machine learning of PDR requires ground truth data of the precise terminal location with sensor inputs. In this paper, we employ Google Tango enabled smartphone (Lenovo PHAB2 Pro) with original location data logger software and HASC Logger [7] . Google Tango utilizes vision tracking called "VSLAM" with sensor fusion technology. By using Google Tango, we can obtain 3D trajectory of terminal position. The location measurement error of Google Tango in our pre-experiment is less than 30cm, and also in the evaluation literature [3]. So we use Google Tango tracking data as a ground truth data of the terminal location. We have collected 79 different routes by 5 subject who is equipped with 3 smartphones simultaneously. In our data collection, subjects are ordered to perform different type of gaits such as fast walk, normal walk, slow walk, and stamp. Details of the collected PDR dataset is shown in Table 1.
Based on the collected data, we have to estimate pedestrian's speed. In this paper, we focus two-dimensional trajectory. However, speed vector which is calculated from Tango's location data cannot be used directly because Tango exports data which includes 3 dimensional data. Therefore, we applied Karman filter based method [16] to estimate and remove data of gravity direction with considering noise reduction [8]. We calculate 2 dimensional moving vector by using gravity direction vector g which is estimated from Karman filter as following:
We use composition of calculated speed vector as ground truth data for speed estimation in the following section. Fig. 2 shows the overview of the process of extracting horizontal speed.
To model the walking speed, we employ CNN-LSTM [12] which is successfully used for activity recognition and other temporal signal processing methods. Additionally, we use fusion layer to capture short and long term features of walking
We have evaluated our proposed method with conventional automaton based speed estimation method [8]. For the evaluation dataset, we use our collected PDR dataset and large indoor pedestrian sensing corpus HASC-IPSC [10]. HASC-IPSC is a corpus for indoor localization but not for real-time location estimation. So HASC-IPSC only contains 3D routes without time-stamp. Table 2 shows the detail of HASC-IPSC.
For the comparison, we use conventional state-machine based PDR.
Step detection finite state automaton is shown in Fig. 4. State transition of automaton by norm is depicted in Fig. 5. Parameter set for the automaton is shown in table 3.
For the input of the state machine, we use 100Hz resampled 3 axis accelerometer sensor data and norm. We suppress the high-frequency noise of the sensor data by low-pass filter using FFT (higher than 8Hz for pocket, and others for 10Hz). Additionally, we limit the least time span of steps to be more than 0.5sec to avoid error detection. For the walking speed estimation, we use step length as person's stature × 0.46.
In this paper, we report the detailed result of our PDR dataset for evaluation. We divide the 5 subjects PDR dataset for 4 subjects for learning, and 1 subject for test, which results 198 learning files and 36 test files. For the evaluation metrics, we employ the following metrics called PIEM(Path Independent Evaluation Metrics) [9] -Average moving distance error (AMDE), Moving distance error rate for each meter (MDEM), and Moving distance error rate for each second (MDES).
For AMDE, we calculate total distance error by using estimated walking speed and elapsed time. For MDEM and MDES, we first create a scatter plot from moving distance error and ground truth distance, or elapsed time. Then we obtain the error rate from the slope of the line regressed by the least square estimate method. Result of the evaluation with our PDR dataset is shown in Table 4. All results of DualCNN-LSTM show better performance than Automaton based method. Effect of the type of gait is shown in Table 5. Automaton based method cannot handle the "stamp", so it increases estimation error. Fig. 6 and Fig. 7 show examples of the speed estimation results of conventional automaton based method and DualCNN-LSTM. Fig. 8 and Fig. 9 show the results of estimated moving paths with ground truth. To plot the moving path, we integrate estimated walking speed with moving direction which is calculated from horizontal angular velocity. Fig. 6 and Fig. 8 is for "walk", and Fig. 7 and Fig. 9 is for "stamp". Automaton based method cannot clearly distinguish the "stamp" with "walk", so it sometimes outputs incorrect speed in "stamp"(like in Fig. 7).
We propose an end-to-end pedestrian walking speed estimation method using DualCNN-LSTM. By using Google Tango for collecting the corpus of pedestrian's 3D-location with sensor data, our learning method achieved higher precision such as 6.51% error rate.
Acknowledgment. This work is supported by JSPS KAKENHI Grant Number JP17H01762.