The paper is devoted to the problem of the recognition of a person by gait using a video recorded in the optical range. The method proposedin this paper consists in the detection of a moving person on a video sequence with the subsequent size normalization and dimensionality reduction using the principal component analysis technique. The person classification was carried out using the support vector machine. The experimental studies performed using the CASIA GAIT dataset allowed us to determine the best values of the method parameters. The obtained results showed that with a small number of classes, high classification accuracy canbe achieved.
The identification of a person by its biometric parameters is popular and widely used all over the world at present. Such specific features as face image, voice timbre, fingerprints, iris pattern and even gait are used for the identification of a person. Although the use of fingerprints or the iris pattern makes it possible to identify a person with little or no error, contactless and remote identification methods are of considerable interest. In this regard, especially important is the problem of recognizing the person using his gait.
Considering the gait as a set of poses and movements, we can distinguish two most common ways
of recording (capturing) such information: video [
Considering that the gait allows to identify the person even in cases where it is not possible to produce it in other ways (the object is at a distance, it is impossible to obtain a high-quality image of the face, etc.), the use of a video, for example, from CCTV cameras is of particular interest.
To date, various methods have been used to solve the problem of the person identification on a video by gait.
The approach used in [
The first step of the approach proposed in the paper [
In the paper [
In the paper [
In general, it can be noted that the problem of recognizing a person by gait attracts the attention of an increasing number of researchers. At the same time, considerable attention is paid to both the methods of the feature description of motion and the choice of effective classification methods.
In this paper, to solve the problem of identification of a person by gait, we follow the general
approach used in the above studies [
The paper has the following structure. Section 2 is devoted to the description of the method used in the paper. Section 3 describes the results of experiments. The paper ends up with the conclusion. The list of used literature is given at the end of the paper.
The method developed in this paper consists of the following steps:
- detection and segmentation of a moving person in the video sequence,
-normalization of the frame size of the selected video sequence fragment,
-dimensionality reduction of the selected video sequence fragment,
-classification of video sequences.
2.1. Detection and segmentation of a moving person on a video sequence
At the first stage of the developed method the moving person is allocated on the video sequence.
When the video sequence source is a video surveillance camera, background subtraction methods are
used most frequently. The main idea of the methods of this class is to use a certain background model
and to decide whether the particular pixel belongs to the background or a moving object. This decision
is based on the correspondence of the pixel to the background model. The background model is
gradually refined. Although the time-averaged observation image can be used as a background model
in simplest applications, better results in this problem are given by more complex models, for example
[
In this paper, we use the background subtraction algorithm based on the mixture of Gaussian
distributions (Gaussian mixture model, GMM) [
As a result of this stage, the set of masks corresponding to individual frames of the video sequence is formed. Each mask reflects the result of the segmentation of a frame into the foreground area corresponding to a moving person and the background. An example obtained using the selected method is shown in figure 1.
At the second stage of the method, obtained masks are processed as follows. First, the center of mass
for each foreground region is calculated. Then the linear dimensions (size) of the region are
determined, and a framing (truncation of the mask image) is performed. After that, the cropped image
is resized to the specified size. The described scheme is shown in the figure 2.Taking into account the
time coordinate, the dimensionality of the sequence of masks, which describes the movement of a
person, remains high even after the size normalization. In this regard, the third stage reduces the
dimensionality of data describing the movement of a person.
2.3. Dimensionality reduction using the principal component analysis technique
To reduce the dimensionality of multidimensional data, both linear and nonlinear methods are used.
The most commonly used are linear methods, such as the principal component analysis (PCA) [
In this paper, we use the principal component analysis technique, as the most often used in such
cases (see, for example, [
In this paper, before reducing the dimensionality of data, for each sequence of frames we form a set of subsequences of a fixed length. To do this, we successively select subsequences of the predefined length k with the step s starting from the beginning of the whole sequence (see figure 3).
For each selected subsequence, the vector of features is formed as follows: each normalized frame of the subsequence is expanded into a row, and the rows obtained for individual frames are concatenated to each other.
The feature vectors of all sequences for different persons form the input matrix for the principal component analysis technique. When principal components are found, the projection of feature vectors onto the first N principal components is taken as a feature description.
The features obtained as a result of the principal component analysis are used to train the support
vector machine (SVM) [
The description given above is valid for the training mode, in which the parameters of the dimensionality reduction procedure (PCA) and classifier (SVM) are configured. In the testing mode, the data is processed in the same way, except that the parameters of the linear transformation (which is used to reduce the dimensionality) are fixed to the values obtained in the training mode, and the classification is performed by the trained SVM classifier.
The described above methods were implemented in C ++ using the OpenCV library. A PC based on Intel Core i5-3470 CPU 3.2 GHz was used to perform experimental studies.
For the experimental study, the video sequences from the open CASIA GAIT dataset [
In this work, we used sequences of 25 persons, in which the shooting angle is 90 degrees, people are depicted in normal clothes and without bags. There were 6 sequences in each class. The length of each sequence was not less than 60 frames. Classes were divided into training and test samples of 3 sequences each.
To estimate the quality of the considered methods, we used the classification accuracy, defined as the proportion of correctly classified sequences.
In the first experiment, the dependence of the classification accuracy on the maximum shift of the subsequences from the beginning of the sequences was investigated (parameter m in figure 3). In this experiment, to reduce the learning time used in the selection of subsequences, the shift step s was 3 frames. Thus, the maximum step of the subsequences m also changed in step 3, taking values from 0 to 15. The last value was determined from the length k of the generated subsequences (k = 45 frames), and the minimum length of the sequences selected for the experiments (n=60 frames).
The experimental results are shown in figures 4 and 5. It was experimentally determined that the accuracy of the classification as a whole increases with increasing maximum shift. This observation is quite expected, since the greater the maximum shift is used in the formation of characteristics, the more "complete" is the feature description of the video sequence. This growth is also accompanied by an increase in processing time, as the number of processed subsequences increases.
99 % ,no98,5 i t ica 98 f i s sa97,5 l c fyo 97 c rau96,5 c c A 96 0 3 6 9 12
15
To choose the value of the step s (see figure 3) we considered rather small values from 1 to 4, which allow us to obtain a more “dense” feature description of the video sequence. Our preliminary experiments showed that the best classification accuracy is achieved at small values of the shift s. Taking into account also the computation time, we ended up with the value s=2.For the method proposed in Section 2.3, we studied the dependence of the classification accuracy on the
32 64
128 256
As it can be seen from the figure, the best values of the classification accuracy are achieved for 64dimensional feature vectors. The increase in dimensionality is accompanied by the expected increase in processing time, although the changes are not very significant. dimensionality of feature vectors (output dimensionality of the PCA technique). The results of the experiments are shown in figures 6 and 7.
20 25
In the next experiment, we considered the dependence of the classification accuracy on the number of classes (persons). The experiment was carried out for 5, 10, 15, 20 and 25 classes, and other parameters remained fixed. In particular, the step s was equal to 2 frames, the maximum shift m of the beginning of the extracted subsequences was equal to 15 frames, and the dimensionality of feature vectors was equal to 64. The results of the experiment are shown in the figures 8 and 9.
15 5 10 20 25
It is worth noting that a direct experimental comparison to other works seems to be quite a
challenge in connection with the different data sets used, as well as the potential differences in the
experimental conditions. The closest approach to the proposed one is described in the paper [
Thus, it can be said that the results obtained in this paper correspond to the current state in the considered field of research.
As it can be seen in the figure 9, the processing takes an increasing amount of time as the number of classes increases. Considerable time is taken in the training mode. This fact becomes especially important in scenarios when the number of classes changes dynamically and it is required to re-train the system regularly.
In this paper we proposed the method for human identification by gait. The proposed method consists in the detection of a moving person on a video sequence with the subsequent normalization of size, generation of subsequences, dimensionality reduction using the principal component analysis technique, and classification using the support vector machine.
The experiments performed on the CASIA GAIT dataset allowed to determine the best values of the parameters of the proposed method.
The drawbacks of the proposed method include its long operating time. In connection with this, a
promising line of research is speeding up this method. Another possible direction of further research is
the recognition of human actions and the detection of abnormal behavior (see, for example, [
The reported study was funded by RFBR according to the research project №17-29-03190.