In this paper, we consider the possibility of distinguishing the movements of a person and people by their gait based on data obtained from the accelerometer of a wearable device. A mobile phone was used as a wearable device. The paper considers the features of recognizing human movements based on a wearable device. A recognition algorithm based on a neural network with preliminary data processing and correlation analysis is proposed. The volume of the training sample consisted of 32 subjects with various physiological characteristics. The sample size in the subgroup of four people ranged from 2000 to 3000 movements. The main motor patterns for classification were the movements performed when walking in a straight line and stairs with a load (a bag with a laptop weighing 3.5 kg) and without it. The direct propagation network is chosen as the basic structure for the neural network. The neural network has 260 input neurons, 100 neurons in one hidden layer, and 4 neurons in the output layer. When training the neural network, the gradient reverse descent function was used. Crossentropy was used as an optimization criterion. The activation function of the hidden layer was a sigmoid, and the output layer was a normalized exponential function. The presented algorithm makes it possible to distinguish between subjects when performing different movements in more than 90% of cases. The practical application of the results of the work is relevant for automated information systems of the medical, law enforcement and banking sectors. Automation, information system, movements, wearable device, phone, authentication,
parameters, classification, distinction
The development of intelligent technologies in the field of processing, analysis and forecasting is reflected in more and more aspects of human life. There are many methods of artificial intelligence: methods of analysis and construction of expert systems, reasoning by analogy, Bayesian networks, behavioral methods, neural networks, fuzzy systems, evolutionary methods, pattern recognition, heuristic programming, multi-agent approach and etc. Artificial neural networks are one of the promising areas of development. The integration of neural networks into complex systems allows you to solve problems with unknown patterns, adapt to changes. When implemented in hardware, they have a high potential in performance and fault tolerance. With the help of neural networks, such tasks as clustering and classification, categorization, approximation, forecasting, optimization are solved [1]. The correct choice of the neural network structure and its training algorithm allows you to approach the object of research more flexibly and find optimal solutions. The introduction of intelligent technologies into information systems opens a new stage of development of the latter. In particular, automated information systems that collect and process biometric information, including access control and management systems, the banking and medical sectors, are a promising direction [2, 3, 4]. The desire to obtain biometric data, remote biometrics and the development of wearable devices require the
2021 Copyright for this paper by its authors. development of new algorithms for collecting and analyzing biometric data. There are many ways to register human parameters [5, 6, 7, 8, 9]. Currently, the most common method of obtaining biometric data is using a video camera and a microphone [10, 11, 12]. In addition, other methods of collecting biometric indicators are actively developing. These methods are based on sensors built into personal portable devices, such as a mobile phone or a smart bracelet [13]. The purpose of this work is to develop technologies for collecting and analyzing biometric data in automated information systems to improve the quality of user identification and authentication procedures, increase the effectiveness of personalized health monitoring by identifying differences and changes in human movement parameters based on the accelerometer data of a wearable device.
The task of recognizing human movements (both recognizing movements in one person and movements in different people – recognizing people) is reduced to their clustering and further classification. During recognition, the received data about objects is divided into subsets. Within each subset, the data samples should have the maximum similarity, and the data belonging to different classes should have the maximum differences. Subsequently, the new data is evaluated as belonging to one of the classes (subsets). The partitioning takes place on the basis of features. The input data can be: a set of features, a distance matrix, time series [14]. The main methods of data classification are: decision tree, Bayesian classifier, nearest neighbor method, support vector machine method, random forest method, gradient descent and booting, logistic regression, tobit and probit [15].
The classification of human movements according to the data of one sensor is incorrect, because the resulting sets overlap very much with each other. The solution of the problem makes sense under the condition of classification of a limited number of movements. In this work, the classification uses signs that determine the features of a person's gait in various conditions and indirectly include the parameters of individual limb movements. That is, when moving, a large number of muscles, ligaments and joints are involved in the work [16, 17]. In addition, the accuracy of the classification is affected by the features of the method of registering movements, which are associated with changes in the registration process of the location and orientation in the space of the sensor itself (which can be performed without walking), the design and metrological features of the mobile device (the location and type of the accelerometric sensor), the transformation of the projection of the acceleration of free fall into motion parameters.
To obtain a set of signs on the basis of which it is possible to classify movements, studies were conducted on a group of people aged from 15 to 67 years. In total, 32 male and female people with different physiological characteristics (height, weight, posture) participated in the research. For the experiment, the subjects were divided into subgroups (according to similar physiological characteristics) of 4 people in order to analyze the quality of distinguishing the subjects and their exercises from each other on the basis of the developed classifier. The volume of the training sample was formed from movements that were performed under various conditions: the form of clothing (loose and tight, different types of shoes), the location of the phone (front and back pockets of pants, near the ear). The volume of the training sample within each subgroup ranged from 2000 to 3000 movements. The main motor patterns for classification were the movements performed when walking in a straight line and stairs with a load (a bag with a laptop weighing 3.5 kg) and without it. Thus, the main movements performed in a person's daily life were included in the training sample. The measurements were made using the accelerometer of a mobile phone. This is due to its high prevalence at the present time compared to smart bracelets. In addition, mobile phones, in comparison with smart bracelets, have more developed functionality and extensive software in which the results of work can be implemented.
As a result of the conducted researches, it was found that the probability of correct discrimination by the average value is no more than 0.19 when using a threshold detector. The correct difference in the standard deviation is no more than 0.13. In the worst-case scenario (in loose clothing and sneakers), based on the correlation receiver, at least 90% of movements can be distinguished at a threshold value of the correlation coefficient of more than 0.8, in good conditions (tight clothing, shoes), this percentage is achieved at a threshold of 0.7.
For each user, movement patterns are selected and a neural network classifier is trained. The duration of the maximum time of the template is taken as the value of the time window. All patterns are normalized in amplitude and supplemented with zeros up to the duration of the maximum pattern (time normalization). Further, during the operation, the data read from the phone's accelerometer is processed within the time window. Each component (axis) of the accelerometer is processed separately. This makes it possible to bring the dependence of the data on the sensor orientation to zero.
The read data located within the time window is normalized. The average value and the standard deviation of the time window data are estimated. After that, the correlation value of the time window data and each of the templates is evaluated. If the correlation value exceeds the set threshold, the neural network classifier is launched. During the research, it was found that the optimal threshold for the correlation coefficient is a value from 0.75 to 0.8. Conducting a preliminary correlation analysis allows you to discard some of the noise signals and improve the quality of the classifier. An example of the structure of a neural network for distinguishing subjects within a group is shown in Figure 2.
The direct propagation network is chosen as the basic structure for the neural network. The number of inputs is 260, of which 256 belong to the analyzed time window with a dimension of 256 of the reference, the following parameters are fed to the remaining inputs: the correlation coefficient, the average value, the standard deviation of the template and the data of the time window. The size of the analyzed time window (256 counts) was chosen as the maximum duration of the performed movement from all the exercises and persons of this research. In the future, in practice, this value may increase.
The neural network has 260 input neurons, 100 neurons in one hidden layer, and 4 neurons in the output layer. The number of neurons of the output layer (4 pieces) corresponds to the number of persons in the subgroup (each subgroup includes subjects with the closest physiological characteristics). The distinction of this neural network is carried out for each subgroup separately. The distinction of persons between subgroups is carried out using a correlation receiver. When training the neural network, the gradient reverse descent function was used. The entire sample was divided into a training sample, a sample for verification and testing in the following ratio: 70%, 15%, 15%. Cross-entropy was used as an optimization criterion. The activation function of the hidden layer was a sigmoid, and the output layer was a normalized exponential function.
The results of testing the neural network are shown in Table 1 with a different number of neurons in the hidden layer.
Thus, as a result of the conducted research, it was found that even in the worst-case scenario, in which the subjects were in loose clothing and sneakers (which adds more noise components to the accelerometer signal and smoothes individual gait features by softening the step), the presented algorithm allows to distinguish the subjects when performing various movements in more than 90% of cases. This also applies to the most monotonous movements – going down and up the stairs and when walking with a mobile phone near your ear. The distinction of the subject's own movements is 100%.
In the course of the research, it was found that the use of a single accelerometric sensor of a mobile phone makes it possible to distinguish between individual movements of a person and movements made by different people. It should be noted that the proposed algorithm for distinguishing movements allows you to separate one movement from another under different conditions, which are characterized by the degree of fit of clothing to the body, the complexity of the path. To improve the quality of distinguishing movements and people, the neural network can be additionally trained in the process of functioning. However, for further development, it is necessary to conduct a larger number of subjects with different physiological characteristics and include a larger number of different interfering factors in the experimental methodology. Nevertheless, the implementation and development of the proposed algorithm in practice will increase the functionality of automated information systems of the medical, law enforcement and banking sectors.
The work was carried out with the financial support of the grant of the President of the Russian Federation No. MK-1558.2021.1.6.