Pedestrian Dead-Reckoning Using Multiple Low-Cost Inertial/Magnetic Sensors on Lower Limbs Chandra Tjhai1,2[0000−0002−6640−9875] and Kyle O’Keefe1,3[0000−0003−2123−2372] 1 Position, Location, And Navigation (PLAN) Group, Department of Geomatics Engineering, Schulich School of Engineering, University of Calgary, Alberta, Canada 2 chandra.tjhai@ucalgary.ca 3 kpgokeef@ucalgary.ca Abstract. This paper presents a pedestrian dead-reckoning method that uses seven low-cost wearable sensors distributed on the pelvis, thighs, shanks, and feet. A commercial inertial sensor is also used and mounted on the right foot for the comparison purposes. The proposed method uses the gait kinematics of the lower limbs to compute the positioning solution by estimating the joint angles and step lengths. The position- ing solution is compared with Kalman filter based solutions computed from the two types of foot-mounted sensors. Indoor walking experiments with linear and rectangular trajectories are conducted to evaluate the proposed method. The results indicate that the low-cost wearable multi- sensor system is able to detect the turning motion at corners and provides a 2D position error of less than 1% after walking for 36 m. 1 Introduction Over the past two decades, pedestrian navigation systems have gone into rapid development due to the emergence of micro-electromechanical systems (MEMS). This technology has enabled the miniaturization of many navigation sensors in- cluding accelerometers, gyroscopes, magnetometers, and barometers. Most MEMS sensors are as small as integrated circuit chips, lightweight and low-cost and found in consumer devices. These low-cost inertial sensors are available in the market with a price tag of 20 US dollars or less. The motion of a pedestrian can be estimated using the knowledge of the gait kinematics of the lower limb segments during walking. Some of the works in pedestrian navigation can be found in these references [4, 5]. This paper demon- strates the capability of using such a low-cost wearable multi-sensor system to track pedestrian motion and compares the results to a typical foot-mounted sin- gle sensor solution. The proposed algorithm uses a model of the gait kinematics and inertial and magnetic measurements to estimate the change of position and orientation of the pedestrian. The wearable sensors are distributed on the lower limbs with one sensor on each segment. 2 C. Tjhai and K. O’Keefe 2 Methodology 2.1 Inertial-Based Foot Position Tracking There are two advantages in tracking pedestrian motion using foot-mounted sensors. First, the walking gait can be easily detected using the change of direc- tion in the gyroscope measurement, especially the sagittal component. Second, the stance phase during walking introduces a quasi-static period where pseudo- measurements of zero velocity and zero angular rate can be applied as Kalman filter updates. The Kalman filter formulation is similar to the implementation found in the references [3, 5] and only the zero velocity and zero angular rate updates are implemented. 2.2 Foot Gait Segmentation Human walking gait is a cyclic process that alternates between swing and stance phases. A walking cycle can be defined as two consecutive gait phases, a pair of swing and stance in any order. The stance phase is marked by a heel-strike event and ended by a toe-off event while the swing phase is vice versa. The proposed method used in this paper requires the information about when the heel-strike and toe-off events occur. One way is to detect these foot gait events using inertial sensor mounted on the foot. The detection method for the swing and stance phases is based on the work reported in [7] while the foot gait detection method is similar to the method reported in [8]. 2.3 Kinematics of Lower Limbs The human gait kinematic model can be modelled using a skeleton consisting of seven segments connected by joints. These segments are the pelvis, the left and right thighs, the left and right shanks, and the left and right feet. This paper uses a simplified skeletal model that replaces the feet with an extended shank from knee to heel, as shown in Fig. 1a, to reduce the complexity. The model has four joints: two hips (HL and HR) and two knees (KL and KR). The hip joints are connected by the pelvis and are allowed to move with three degrees of freedom. Meanwhile, the knees are modelled as single axis revolute joints which can only rotate in the sagittal plane. The computation of forward kinematics requires one end of the links to be fixed while the other end is free to move. This condition fits the cyclic nature of the walking cycle. The computation sets the fixed end to be the standing foot while the other end be the swinging foot and alternates between steps. The mechanization of the forward kinematics starts from the fixed foot and follows the skeletal linkages until the other foot. Based on the model shown in Fig. 1a, this computation requires all three attitude angles of the pelvis sensor and the pitch angles of the thigh and shank sensors. Assuming the right foot is in stance Pedestrian Dead-Reckoning Using Multiple Low-Cost Sensors 3 HL HR HL Z KR KL HR KL FR Y Heel-Strike KR Stance FL FR X Toe-Off (a) Human skeletal (b) Step sizes extracted from mechanization model of lower limbs. of forward kinematics of lower limbs. Fig. 1: Forward kinematics of lower limbs phase and the left foot is in swing phase, the forward kinematics equations can be written as follows pnF L/F R = pnF R +Cnb,SR `bSR +Cnb,T R `bT R +Cnb,P `bP +Cnb,T L `bT L +Cnb,SL `bSL (1) where the subscripts are described by Fig. 1a. The position vectors are expressed in the local level frame, denoted by pn• , and the symbol Cnb,• denotes the direction cosine matrix transforming from body frame to local level frame. This matrix is computed based on the estimated attitude angles. The lengths of body segments are expressed in the body frame and written as `b• . For the case where the left foot is in stance phase and the right foot is in swing phase, the forward kinematics equations need to be adjusted by swapping the subscripts L and R. The starting point (pnF R or pnF L ) is assumed to be the local origin position in each mechanization. The attitude estimation algorithm used in this paper is based on the gra- dient descent algorithm proposed by Madgwick et al. [6]. The step sizes can be extracted from the mechanization of the forward kinematics between toe-off and heel-strike events of each swing phase. A stride length is the foot position displacement from toe-off to heel-strike while a step length can be extracted in two different ways: the position displacement of the hip during stance phase or the swing phase, and the diagonal distances between the two feet at toe-off and heel-strike. Fig. 1b sketches a visualization of forward kinematics and the computations of step and stride lengths. 2.4 Pedestrian Dead-Reckoning Filter The kinematics of lower limbs can be used to estimate the step length and its corresponding azimuth at each step. A pedestrian dead-reckoning filter can then be implemented using these two observations. A simplified Kalman filter is 4 C. Tjhai and K. O’Keefe adopted from the reference [2] and modified for 2D positioning. The filter state is propagated using the following equation     N̂ ; Ê; ψ̂ k+1 = N̂k + s cos ψ̂k ; Êk + s sin ψ̂k ; ψ̂k (2) where s represents a constant step length estimated assuming the pedestrian moves at 1 m/s. The measurements are the step length and heading estimated by the kinematics of the lower limb segments.  T zk = s̃k ψ̃k (3) 3 Experiments 4.658 m 2.828 m 42 43 44 01 02 03 N 1.743 m ORIGIN E 04 05 06 07 08 09 10 11 12 13 15 16 10.945 m 14 17 18 19 20 21 8.117 m 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 (a) Low-cost wearable sensors distributed 6.401 m on lower-limbs and Xsens MTi-G-710 on (b) Office room layout and experiment right foot. walking paths. Fig. 2: Indoor experiment setup In the experiments, one sensor was mounted on each foot, shank, and thigh, and two sensors were mounted on back of the pelvis as shown in Fig. 2a. One of the pelvis sensors is redundant and only one is used in the computation. These wearable sensors were mounted such that the right leg had MPU-9250 modules while the left leg had MPU-6050 modules, while the pelvis had one of each. Both sensor modules are manufactured by TDK Invensense. The sampling rate was set at 50 Hz and two microcontrollers were used to collect the data send to a computer for logging. Additionally an Xsens MTi-G-710 was mounted on the right foot below the low-cost module to use for comparison purposes. The experiments were conducted in an empty office room with approximate dimensions of 6.5 × 11 m. There were 44 reference points marked on the ground with one 1 m spacing. The room layout and walking paths are shown in Fig. 2b. Two walking scenarios were conducted: straight line walk and rectangular loop. Pedestrian Dead-Reckoning Using Multiple Low-Cost Sensors 5 In the straight line walk, the test subject walk with approximately equal step lengths of 75 cm while the rectangular walk consisted of approximately 75 steps in total. In addition to the low cost and commercial inertial sensors, the test subject was also video recorded in order to compare the estimated leg segment angles with those obtained from the wearable multi-sensor system. 4 Results and Discussions 4.1 Orientations of Lower Limb Segments 10 Roll [deg] 0 -10 -20 16 17 18 19 20 21 22 23 24 25 26 10 Pitch [deg] 0 -10 -20 16 17 18 19 20 21 22 23 24 25 26 10 Yaw [deg] 0 -10 -20 16 17 18 19 20 21 22 23 24 25 26 Time [s] (a) Pose estimation using OpenPose li- brary. (b) Attitude angles of pelvis. 40 OpenPose OpenPose Madgwick AHRS 50 Left Pitch [deg] Left Pitch [deg] Madgwick AHRS 20 0 0 -20 -50 -40 17 18 19 20 21 22 23 24 17 18 19 20 21 22 23 24 OpenPose 40 OpenPose Madgwick AHRS 50 Right Pitch [deg] Right Pitch [deg] Madgwick AHRS 20 0 0 -20 -50 -40 17 18 19 20 21 22 23 24 17 18 19 20 21 22 23 24 Time [s] Time [s] (c) Thigh pitch angles. (d) Shank pitch angles. Fig. 3: Estimated lower limb orientation angles during straight line walk. To check the validity of the estimated pitch angles from thighs and shanks, a publicly released computer vision library (OpenPose [1]) is used to estimate the 2D human pose and extract the pitch angles at each frame and the results are visualized in Fig. 3. Fig. 3b shows the results of pelvis attitude angles for walking on a straight line. There is a small difference between the initial and final heading angles, and the alternation of heading angle is not symmetric due to accumulated 6 C. Tjhai and K. O’Keefe gyroscope and magnetometer errors. The pitch angles estimated by OpenPose and sensor fusion algorithm are plotted in Fig. 3c and Fig. 3d. These two results have a good agreement between the two estimated pitch angles. The OpenPose results have larger variation on the thigh segment and smaller variation on the shank due to the fact that the 2D projected area of the thigh is both larger and less rectangular than the shank. There are some outliers due to the failure of the classification algorithm in differentiating between the left and the right limbs (shown by the first foot step in Fig. 3a). 4.2 Forward Kinematics The equations of the forward kinematics require two inputs, the segment attitude angles and the segment lengths. The attitude angles are estimated using the gradient descent algorithm. The segment lengths can be obtained by measuring the limb segments directly. 3 2 1 0 MPU-9250 EKF-ZUPT -1 North [m] ZUPT -2 -3 1 -4 Up [m] 0.5 0 -5 -0.5 -6 -7 -4 -8 -2 0 2 4 6 East [m] Reference Path MPU-9250 EKF-ZUPT 2D PDR East [m] Xsens EKF-ZUPT R. Foot Kinematics 0 -3 North [m] MPU-9250 INS-ZUPT Start/Stop (a) Straight line walk. (b) Rectangular path walk. Fig. 4: Comparison of position trajectories estimated by different methods. The results of mechanizing the forward kinematics are plotted in Fig. 4a as position trajectories of the lower limb joints and heels. Based on the position of the right foot, the final position solution has drifted 0.42 m to the left and 0.56 m downward. The total forward travelled distance error is 0.08 m over 4.5 m. The vertical error is due to the effect of ignoring the foot segments in the forward kinematics model. During walking, the foot segment rotates the heel upward after each heel-strike. Treating this foot rotation as quasi-static motion causes the position solution to drift on the vertical axis. The other components Pedestrian Dead-Reckoning Using Multiple Low-Cost Sensors 7 of the position errors are caused by the accumulation of the errors of the attitude angles. 4.3 Comparison of Position Solutions in a linear path For comparison, the typical foot-mounted EKF algorithm is implemented an applied both the MPU-9250 and Xsens MTi-G-710 data. The filter is tuned such that an optimal solution can be obtained by selecting the values of the process noise matrix and measurement covariance. For both sensors, the process noise models of the attitude and the velocity are modelled as random walks driven by the noises in the gyroscope and accelerometer. The augmented bias states are simply modelled as random constants. The results shown in Fig. 4a indicate that there are significant differences in the position solutions computed with the EKF-based algorithm (Section 2.1). The zero measurement updates cannot correct the error drift on the low- cost MPU-9250 sensor that is significantly larger than the accumulated error on the Xsens. The purple line (INS-ZUPT) represents the foot position solu- tion computed by mechanizing the inertial equations with zero velocity pseudo- measurement. This computation strategy is better for the low-cost inertial sen- sors than applying the EKF framework. The Fig. 4a also indicates that the kinematics of the lower limbs can be used to improve the positioning solution of the low-cost sensors. The angular rates and accelerations estimated from the inertial sensors can accumulate huge errors when going through multiple time integrals to get the position estimates. By using the kinematics of lower limbs, there is only one time integral function used in order to estimate the attitude angles from the angular rate measurements. 4.4 Pedestrian Dead-Reckoning on a rectangular path The motion data collected during a rectangular walk are processed using different algorithms described in Section 2. The size of the rectangular path is about 4 m × 5 m. The test subject starts and ends the walking experiment at the same location. The EKF filters of the two foot-mounted sensors are tuned with the same values with the straight line walk. The performance comparison is tabulated in Table 1. The result computed by mechanizing the inertial equation with ZUPT (INS-ZUPT) has a smaller error, but the foot-mounted sensor is not able to estimate the heading accu- rately. The PDR results do not represent the best possible PDR solution due to the simplicity of the filter architecture. This simple filter was implemented to demonstrate that the step length and heading estimated by the equations of the forward kinematics can be used as the inputs to the navigation filter. The kinematics of lower limb can detect the turns at the corners despite using such a simple attitude estimator. The heading is mainly estimated from using using the pelvis mounted sensor. The error of the forward kinematics is small. Thus, this computation strategy is able to estimate the step lengths. 8 C. Tjhai and K. O’Keefe Table 1: Positioning errors from different methods after walking for 36 m. Methods Positioning Error (% of total travelled distance) MPU-9250 EKF-ZUPT 5.98 Xsens EKF-ZUPT 1.16 MPU-9250 INS-ZUPT 0.72 Pelvis 2D PDR 4.89 R. Foot Kinematics 0.74 5 Summary and Future Work This paper proposes a method to use step lengths and orientation angles as the inputs to the pedestrian dead-reckoning filter. These gait parameters are estimated using the gait kinematics of the lower limbs. The kinematics of lower limbs are able to detect turns and provide positioning error of < 1% of total travelled distance while using low cost inertial sensors. The results of this paper cannot assess the complete performance of a PDR system due to the small testing area. Future work will include larger experiment area and vertical displacement. References 1. OpenPose, https://github.com/CMU-Perceptual-Computing-Lab/openpose, Last accessed 19 July 2019 2. Ahmed, D.B., Diaz, E.M.: Loose Coupling of Wearable-Based INSs with Automatic Heading Evaluation. Sensors 17(11), 2534 (2017). https://doi.org/10.3390/s17112534 3. Dehkordi, M.B., Frisoli, A., Sotgiu, E., Loconsole, C.: Pedestrian indoor naviga- tion system using inertial measurement unit. International Journal of Sensor Net- works and Data Communications 3(1), 1–9 (2014). https://doi.org/10.4172/2090- 4886.1000115 4. Foxlin, E.: Pedestrian tracking with shoe-mounted inertial sensors. IEEE Computer Graphics and Applications 25(6), 38–46 (2005). https://doi.org/10.1109/MCG.2005.140 5. Jiménez, A.R., Seco, F., Prieto, J.C., Guevara, J.: Indoor Pedestrian Navigation Using an INS/EKF Framework for Yaw Drift Reduction and A Foot-Mounted IMU. In: 2010 7th Workshop on Positioning, Navigation and Communication. pp. 135–143 (March 2010). https://doi.org/10.1109/WPNC.2010.5649300 6. Madgwick, S.O.H., Harrison, A.J.L., Vaidyanathan, R.: Estimation of IMU and marg orientation using a gradient descent algorithm. In: 2011 IEEE International Conference on Rehabilitation Robotics. pp. 1–7 (Jun 2011). https://doi.org/10.1109/ICORR.2011.5975346 7. Skog, I., Handel, P., Nilsson, J.O., Rantakokko, J.: Zero-velocity detection - an algorithm evaluation. IEEE Transactions on Biomedical Engineering 57(11), 2657– 2666 (Nov 2010). https://doi.org/10.1109/TBME.2010.2060723 8. Tjhai, C., O’Keefe, K.: Step-Size Estimation Using Fusion of Multiple Wearable Inertial Sensors. In: 2017 International Conference on Indoor Positioning and Indoor Navigation (IPIN). pp. 1–8 (Sep 2017). https://doi.org/10.1109/IPIN.2017.8115926