With the continuous development of smart wearable devices and with the emergence of the Internet of things era, the need for both indoor and outdoor positioning technology has become increasingly imperative. Wearable watches can enable the positioning of their movement tracks and recognize the user's movement patterns and provide services such as safety monitoring in specific scenarios such as construction sites and tunnels. However, current smart watches have some problems, such as inaccurate sensor data collection, unstable positioning effect, low accuracy of human posture recognition, especially, the performance of step detection based on wearable sensors is affected by the complex human motion. This paper proposes a low-cost and high-precision gait detection model according to Long Short-Term Memory (LSTM) neural network, the motion features provided by pre-processed acceleration vector are extracted for model training and prediction, which realizes the step detection rate of more than 98.5% and outperforms the existing step recognition algorithms and achieves a relatively accurate step notation through a large amount of real-world experiment on smart watch, laying the foundation for subsequent indoor positioning.
Emerging technologies including artificial intelligence (AI), blockchain, big data, cloud computing, 5G, and wearable technology are quickly becoming prominent subjects of interest in a variety of businesses due to the Internet industry's rapid growth. In the 1960s, the Massachusetts Institute of Technology Media Lab proposed the innovative concept of "wearable technology" [1]. Wearable technology embeds a series of tiny sensors and cutting-edge technologies into people's daily wearables to make them intelligent, which not only maintains the function of the wearable itself, but also gives the function of smart devices, which greatly enriches people's life experience [2].
Currently, a variety of micro-electromechanical systems (MEMS) sensors are integrated internally to enhance the user's ability to sense the external environment [3]. They support handsfree use, and their lightweight and simplicity allows users to enjoy the various experiences brought by the devices while walking or being busy. Wearable technology is evolving rapidly, yet it is still in the developmental stage. While wearable devices bring good experiences to users, they also reveal some problems. At present, power consumption has been the drawback of wearable devices. Miniaturized wearable devices integrate various additional functions, resulting in an increase in the workload of the microcontroller, which ultimately leads to a lack of endurance of the devcie, so that the user cannot experience the device for a long time [4].
Specially, positioning technology contributes an important function of wearable devices. Combined navigation is a method mainly used for precise positioning and attitude fixing. Since the 1960s, Western industrial powers including the United States, Germany, and the United Kingdom have made extensive use of combinatorial navigation in the domains of guiding, ships, and airplanes due to the development of tiny sensors and contemporary control theory [5]. The introduction of Kalman filtering technology greatly corrects the error parameters generated by the combined navigation system, and the navigation systems complement each other, making the accuracy of the combined navigation system far exceed that of any single navigation system [6].
In general, there are three aspects of wireless-based indoor localization systems, including spatial variability, temporal variability, and device variability. When electromagnetic waves are transmitted, reflection, refraction, and diffraction can produce power loss and multipath effects that alter the amplitude and phase of the waves. This phenomenon is known as spatial variation [7]. Both large- and small-scale spatial changes are possible. While phase cancellation or phase lengthening of the signal owing to multipath effects causes small-scale spatial variations, path loss and obstacle occlusion of the signal are the major causes of large-scale spatial variations [8]. The term "temporal variation" describes how radio propagation effects change over time in interior environments. This might involve big, sudden changes like moving furniture or altering the architecture of a building, as well as tiny, recurring changes like door opening and closing, people moving about, or transient signal interference from other wireless devices [9]. Because the inside environment can vary over time, temporal fluctuations in indoor localization systems are vital to take into account. Device variation is the term used to describe circumstances in which the behavior of various devices varies. Specifications for wireless devices of the same technology might vary even amongst companies, particularly when it comes to transmitter gain, receiver sensitivity, and antenna radiation direction map. When placed in various containers or linked to different antennas, gadgets of the same brand may exhibit varied radiation patterns [10].
Gait detection technologies based on accelerometers primarily utilize accelerometers to capture the acceleration signals generated during human movement. By analyzing these signals, the technologies can identify and assess gait patterns. These techniques are widely applied in fields such as medical health, sports science, and virtual reality. Below are some common gait detection technologies based on accelerometers:
1) Time-domain Analysis: This is the most straightforward method, which involves analyzing the characteristics of acceleration signals in the time domain, such as step length, speed, and frequency. Time-domain analysis typically includes calculating statistical parameters of acceleration signals, such as peak values, mean, and standard deviation, to identify gait characteristics [11].
2) Frequency-domain Analysis: By performing a Fourier transform on the acceleration signals, this method converts them from the time domain to the frequency domain to analyze the spectral characteristics of the signals. This approach can identify periodic features of gait, such as gait frequency, as well as the stability and symmetry of the gait [12].
3) Time-frequency Domain Analysis: Combining time-domain and frequency-domain analyses, techniques such as wavelet transform are used to obtain both time and frequency information of the signal simultaneously. This method is suitable for analyzing non-stationary signals, providing more information about the dynamic changes in gait [13].
4) Machine Learning Methods: By collecting a large amount of acceleration data, machine learning algorithms (such as Support Vector Machines, Random Forests, Deep Learning, etc.) can be used to train models to automatically recognize gait patterns. This approach can handle complex gait features, improving the accuracy and robustness of recognition [14].
5) Dynamic Time Warping (DTW): This is an algorithm that considers the time scalability of time series data, dynamically aligning time series to match gait cycles of different time lengths, used for recognizing and comparing gait patterns [15].
6) Sensor Fusion: In addition to using accelerometers alone, combining them with other sensors such as gyroscopes and magnetometers, and fusing data from different sensors, can improve the accuracy and reliability of gait detection [16].
Each of these technologies has its advantages and limitations. In practice, the appropriate method or a combination of multiple methods is usually chosen based on specific needs and conditions to achieve the best gait detection results.
This work focuses on a simplified and accurate step detection algorithm based on the single accelerometer integrated in the smart wearable device. The innovations of this work are described as:
(
(
(
The structure of this paper is organized as follows: Section II presents the data pre -processing, feature extraction and methodology. Section III provides real-world experiments for accuracy and robustness evaluation of proposed algorithm. Section IV concludes our work and presents the future work.
This section focus on the data pre-processing, features extraction and modelling, and proposed LSTM network based step detection algorithm.
The foundation and root of human posture recognition is data acquisition. In this paper, the movement parameters of the human wrist acceleration data are used to identify the human body's movement mode and state. Three human body movement modes are defined for more accurate step detection: walking, static, and falling. The human body and wrist will be in different states under these different movement states and modes.
The raw data in this paper is mainly through the watch's three -axis acceleration sensor to realtime acquisition of human motion data, to obtain the dynamic acceleration of the device, each time the data collection is worn by the user in the left wrist for data collection, the collection frequency is generally set to more than twice the frequency of the wrist movement, to avoid the overlap of the acceleration data signal. According to related literature research [17], the frequency of daily human movement is generally lower than 20 HZ, combined with multiple experimental control, set a fixed acquisition frequency of 50 HZ for each experiment, i.e., to collect data of about 50 sampling points per second.
According to the characteristics of the three human movement modes, the following can be observed: when the human body is in walking mode, the wrist mainly swings slowly with small amplitude, and the range of its acceleration and angular velocity changes is small; when the human body is at rest, the human body does not experience obvious acceleration or deceleration. As a result, the acceleration data has a very small range of variations and is generally characterized by relatively smooth and stable signals, and because the human body does not produce significant movements, the acceleration data is relatively smooth and has a low noise level. This lessens interference and erratic swings in the data. The human body experiences a sudden acceleration peak and a large acceleration change when it falls suddenly. This is because the body loses balance during the transition from a vertical position to a down position, tilting quickly downward and falling to the ground. The acceleration sensor records this abrupt change in peak acceleration. This is demonstrated by the acceleration data having a significant number of high-frequency components as a result of impact vibration, which frequently exhibit abrupt variations and discontinuities.
As shown in Figure.1, for the experimental user to wear on the left wrist collected under the walking, stationary, fall three kinds of motion state data, three different curves are the acceleration sensor X-axis, Y-axis, Z-axis signal waveform, the horizontal axis of the coordinates according to the time sequence of the sorting of the sampling points, the vertical axis of the sensor acceleration data, from the figure can be found in the different motion state of the some regular changes, the characteristics of these characterizations provide a series of proofs and controls for the subsequent pattern recognition.
(a) Falling down (b) Static
After preprocessing, the internal errors and interference of the data were significantly reduced, while retaining the actual valid information. These data signals are essentially time series and contain a series of informative features of human movement. However, these features often involve multiple behavioral states and cannot be extracted and used directly. Therefore, before extracting different features, the whole data needs to be processed by window segmentation in order to extract feature values that can characterize the complete action from the segmented data pieces.
Sliding windows, event-defined windows, and activity-defined windows are now the three primary types of classical windowing systems [18]. Sliding window is a commonly used windowing technique which splits the data according to a fixed window size and moves it sequentially by sliding the window. This windowing technique can cover the entire data series and there is an overlap between the windows. Sliding windows are very effective in extracting localized features in time series data, but may lead to data redundancy. Event windows are a technique for defining window boundaries based on specific events or markers. The data is segmented into different windows based on the time of the event or other characteristics. Event windows can capture key action segments more accurately and are suitable for scenarios that need to be analyzed for specific events. Action windows are windows that are defined based on the movements or activities of the human body. The data is divided into different windows based on the body's movement patterns or specific activity states. Action windows are capable of extracting features related to human movements and are used for tasks such as action recognition and behavior analysis. To synthesize the main application scenarios and situations, the window processing of sliding window is adopted for the preprocessed data.
A crucial stage in the process of recognizing human posture is the extraction and selection of feature value quality, which is essential for differentiating the data. Even when the signals are preprocessed and window segmented, the resulting window data is still unable to accurately depict the human body's behavioral state. Therefore, further processing and transformation of the data is required to form a feature set that accurately describes the behavioral actions of the human body and extract relevant features from it [19].
Three categories of features—time-domain, frequency-domain, and time-frequency-domain—are the subject of current study [20-22]. Theoretically, when the number of extracted features increases, the set of features formed also becomes larger, which improves the ability to accurately describe human posture recognition. However, the amounts of signals contained between different features varies, and there may be redundant information in the high-dimensional feature set, which ultimately reduces the accuracy and recognition rate, so the extracted features need to be further optimized. Commonly used methods include Principal Component Analysis (PCA) [23] and Independent Component Analysis (ICA) [24].
Gait detection mainly refers to analyzing and recognizing the movements and patterns of the human body while walking to determine the individual's physical state, motor state, number of steps walked, walking step length, motor ability and a series of possible abnormalities. Gait detection is important in many fields, including medicine, biomechanics, sports science, architecture, and security.
Gait detection can provide information for the assessment of an individual's health status and the diagnosis of disease. For example, certain neurological disorders (e.g., Parkinson's disease) or musculoskeletal disorders (e.g., scoliosis) can be detected by analyzing gait patterns; it can also be used for motion analysis and rehabilitation monitoring. For athletes and sports rehabilitation patients, understanding their gait patterns can help assess motor skills, improve training programs, and monitor rehabilitation progress; due to the individual uniqueness of gait and patterns, it can be used for identification and security monitoring. In the field of security, gait recognition can be used for pedestrian identity verification, which can be used for security access control and crime prevention; gait detection can also be used for motion control and the design and development of intelligent assistive devices. For example, in smart prosthetic limbs or exoskeleton systems, realtime monitoring of gait patterns can be used to realize precise control and adaptive adjustment of the device, improving the user's athletic ability and quality of life. Most importantly, for indoor and outdoor navigation and positioning, by analyzing people's gait patterns, it is possible to understand people's walking habits and behaviors in the built environment. This information is useful for designing indoor and outdoor spaces, path planning, stair and ramp design, and facility layout. For example, understanding people's gait and walking speed can help designers determine appropriate pathway widths and flow layouts to ensure people's comfort and safety in the built environment.
This study uses a Long Short-Term Memory (LSTM) network to accomplish footstep detection. The fundamental use of LSTM, a unique kind of Recurrent Neural Network (RNN) architecture, is the modeling of time series and their distant relationships. It performs exceptionally well in time series prediction, processing, classification, and learning from data [25-26].
Accelerometer data serves as the foundation for conventional step detection techniques. These techniques, which have limited precision, often find the peak or over-zero point of acceleration to estimate the step length. Numerous academics have begun to use deep learning architectures in the field of pedestrian navigation as a result of the advancements in deep learning technology. Among them, LSTM networks are frequently employed for sequential data recognition due to their superior handling of the "long-term dependency" issue. LSTM networks are ideally suited to assess the step detection problem since human walking is a continuous and periodic motion [11-16].
The step length detecting network's architecture is displayed below [17]:
The triaxial accelerometer BMA423's output provides information to the network, but it also
includes high frequency components and noise. The acceleration data is initially down-sampled to
a sampling rate of 50 Hz in order to lessen the impact of noise. The acceleration data during a step
period is aggregated into a single input sample, where each input is represented by a 6x3 matrix, in
order to enhance the feature representation. The output of the z-axis is often greater than that of
the x- and y-axes because accelerometers detect particular force. A normalization layer is added to
the input data to equalize it and balance the impact of each dimension's weights. The normalizing
layer formula is as follows:
a = a − amean (
amax − amin where a indicates the acceleration vector, amax and amin indicate the maximum and minimum values, amean indicates the mean value.
Assume that an is the initial input data and that a/n is the normalized data, where mean is the average value of a. Since multilayer LSTMs perform better than single layer LSTMs, the LSTM layer—which is made up of four LSTM units —is an essential component of the feature extraction process. Employing four layers of LSTM facilitates the learning of higher order features and enhances efficiency. In order to transfer the features to the output space and minimize their dimensionality, fully linked layers are employed. The network employs binary cross entropy as its loss function:
1 N
f = ∑ −[ yi * log yˆi + (1− yi ) log(1− yˆi )] (
N i=0 where f indicates the loss function. Assume that yi represents the label of the training data (1 for true and 0 for false), yˆi represents the probability of prediction, and N indicates the length of the training data. The network produces a one-dimensional tensor as its output; if the value is more than 0.5, it indicates that a gait has been identified; if it is less than 0.5, no gait has been detected. Since every input has a corresponding output, the output frequency of the proposed LSTM network's is 50 Hz.
This section designs comprehensive experiments for the accuracy evaluation and comparison of proposed LSTM-based step detection framework, and a real-world dataset is prepared for accuracy evaluation.
Every time you finish a walking step, you will tap the watch screen. The watch will recognize this tap and change the value of the label, which is continuously recorded, from 0 to 1 and back to 0. The training data are the triaxial acceleration data in the walking state for a period of time in a specific scenario and the triaxial acceleration data labels that record the number of valid steps. It creates a period of labeled data at the conclusion of the measurement, capturing both the actual walking and the watch throughout this time. accompanying the measurement, a period of real walking label data and the three-axis acceleration data that the watch continually recorded throughout this time are formed.
The loss function of binary cross entropy chosen during the training process of this model shows obvious convergence at the end of 500th epoch training, as shown in Figure. 3:
To ensure that our suggested method is feasible, we compared it with two standard step detection techniques. Peak and over-zero acceleration detection is the basis for the traditional techniques for counting steps with watches and bracelets (peak detection [27] and over-zero detection [28]), and also the deep-learning approaches including one-dimensional convolutional neural network model (1D-CNN) [29] and Multilayer perceptron (MLP) [30] are compared at the same time. On the other hand, we performed experimental measurements using a very accurate detection technique that was tailored for the inexpensive three-axis acceleration BMA423 sensor. This algorithm produced a step count that was around 30% off from the real number of steps. Walking, running, and stair climbing were all detectable by the step counter and pedometer. Moreover, the gait identification algorithm's accuracy, which relied on the LSTM neural network model, varied by less than 5% from the real number of steps. Comparing this performance to the improved sensor algorithm, there was a noticeable improvement. We carried out three tests under different scenarios with varying amounts of stages to confirm the correctness of our approach. The Table. 1 below displays the outcomes of the experiment. The table shows that the LSTM-based approaches fared better in terms of accuracy than both conventional and deep-learning based methods, with our method obtaining an step detection accuracy of higher than 98.5%. Because there were no irregular motions in Experiment 1, such as turning, the improved performance of all approaches may be attributed to the regularity of the motion
LSTM 1D-CNN MLP Traditional Peak
In the era of smart wearables and the Internet of Things, the demand for precise indoor and
outdoor positioning technologies has surged. Wearable devices, like smartwatches, offer the
potential for tracking movement and recognizing user patterns, crucial for safety monitoring in
environments like construction sites and tunnels. However, challenges persist with smartwatches,
including inaccurate sensor data, unstable positioning, and poor posture recognition, particularly in
complex motion scenarios. This study introduces a cost-effective, high-accuracy gait detection
model based on the LSTM network. By extracting motion features from pre-processed acceleration
vectors for training and prediction, the model achieves a step detection accuracy exceeding 98.5%,
surpassing existing algorithms. Extensive real-world testing on smartwatches demonstrates its
superior step recognition capabilities, setting a solid foundation for enhanced indoor positioning.
Our future work will focus on utilizing more advanced deep learning models to enhance the
accuracy of pedestrian gait detection under complex motion patterns and to overcome the
interference of random movements. Moreover, current pedestrian gait detection models based on
deep learning heavily rely on datasets and annotation information. In the future, we will explore the
use of small-sample or pre-trained models to achieve gait detection with minimal labeling.
[4] Lara O D, Labrador M A. A survey on human activity recognition using wearable sensors. IEEE
communications surveys & tutorials, 2012, 15(
Pattern Analysis and Machine Intelligence, 2020, 44(4): 2074-2088. [10] Zhou M, Li Y, Tahir M J, et al. Integrated statistical test of signal distributions and access point contributions for Wi-Fi indoor localization. IEEE Transactions on Vehicular Technology, 2021, 70(5): 5057-5070. [11] Zhang M, Wang Q, Liu D, et al. Real-time gait phase recognition based on time domain features of multi-MEMS inertial sensors. IEEE Transactions on Instrumentation and Measurement, 2021, 70: 1-12. [12] Fu C, Li Q, Shen M, et al. Realtime robust malicious traffic detection via frequency domain analysis[C]//Proceedings of the 2021 ACM SIGSAC Conference on Computer and Communications Security. 2021: 3431-3446. [13] Jia Y, Guo Y, Wang G, et al. Multi-frequency and multi-domain human activity recognition based on SFCW radar using deep learning. Neurocomputing, 2021, 444: 274-287. [14] Qiu S, Zhao H, Jiang N, et al. Multi-sensor information fusion based on machine learning for real applications in human activity recognition: State-of-the-art and research challenges.
Information Fusion, 2022, 80: 241-265. [15] Huang L, Zheng J, Hu H. A gait phase detection method in complex environment based on
DTW-mean templates. IEEE sensors journal, 2021, 21(13): 15114-15123.
[16] Cornacchia M, Ozcan K, Zheng Y, et al. A survey on activity detection and classification using
wearable sensors. IEEE Sensors Journal, 2016, 17(
Based on Peak‐Valley Detection. Journal of Sensors, 2020, 2020(