The paper considers the problem of a person identi cation by gait using a video sequence. It provides a comparative analysis of two methods. The rst method is proposed in this paper and consists in the detection of a moving person on a video sequence with the subsequent size normalization, generation of subsequences, and dimensionality reduction using the principal component analysis technique. The person classi cation is carried out using the support vector machine. In the second method, the known approach based on the hidden Markov model is used. CASIA GAIT dataset is used in this paper to compare the above methods. It was shown that the proposed method outperforms the HMM-based technique and provides high classi cation accuracy on the considered dataset.
The identi cation of a person by its biometric parameters is popular and widely used all over the world at present. Such speci c features as a face image, voice timbre, ngerprint, iris pattern and even gait are used for identi cation of a person. Although the use of ngerprints or iris patterns makes it possible to identify a person with little or no error, contactless and remote identi cation methods are of considerable interest. In this regard, especially important is the problem of recognizing a person using his gait.
Considering a gait as a set of poses and movements, we can distinguish two
most common ways of recording (capturing) such information: video [
Considering that a gait allows to identify a person even in cases where it is not possible to produce it in other ways (the person is at a distance, it is impossible to obtain the high-quality image of the face, etc.), the use of video, for example, from CCTV cameras is of particular interest.
To date, various methods have been used to solve the problem of identi cation of a person on video by gait.
The approach used in [
The rst step of the approach proposed in the paper [
In the paper [
In the paper [
In general, it is worth noting that the problem of recognizing a person by gait attracts the attention of an increasing number of researchers. Considerable attention is paid to both feature description techniques and to the choice of e ective classi cation methods.
In this paper, to solve the problem of identi cation of a person by gait, we
follow the general approach used in the above studies [
The proposed approach di ers from the above mentioned papers.
The proposed technique is compared to the HMM-based technique. The paper shows that the proposed method provides higher classi cation accuracy with a relatively small number of classes.
The paper has the following structure. Section 2 is devoted to the description of the method used in the paper. Section 3 describes the results of experiments. The paper ends up with the conclusion. The list of references is given at the end of the paper.
2.1 Identi cation of a moving person using principal components analysis and support vector machine The method developed in this paper consists of the following steps (see Figure 1): - detection and segmentation of a moving person in the video sequence, - normalization of the size of detected fragments, - generation of subsequences of video frames and dimensionality reduction of generated subsequences, - classi cation of video sequences. Detection and segmentation of a moving person on a video sequence. At the rst stage of the developed method, a moving person is detected on a video sequence. Background subtraction methods are most frequently used to detect moving objects if the video sequence was obtained using video surveillance camera. The main idea of the methods of this class is to use a certain background model and to decide whether a particular pixel belongs to the background or a moving object. This decision is based on the correspondence of the pixel to the background model. The background model is gradually re ned. Although the time-averaged image can be used as a background model in simplest applications, better results in this problem are given by more complex models, for example [7{9].
In this paper, we use the background subtraction algorithm based on a
mixture of Gaussian distributions (Gaussian mixture model, GMM) [
To choose this particular background subtraction technique, we took into
account both our preliminary experiments and the experience of using this method
by other researchers [
Upon completion of the rst stage of the method, a sequence of masks corresponding to individual frames of the video sequence is formed. Each mask re ects the result of the segmentation of the corresponding video frame into the foreground region (moving person) and the background.
Normalization of the size of detected fragments. At the second stage of the method, obtained masks are processed as follows. First, the center of mass for each foreground region is calculated. Then the linear dimensions (size) of the region are determined, and a framing (truncation of the mask image) is performed. After that, the cropped image is resized to the speci ed size. The described scheme is shown in Figure 2. Taking into account the time coordinate, the dimensionality of the sequence of masks, which describes the movement of a person, remains high even after size normalization. In this regard, the third stage reduces the dimensionality of data describing the movement of a person.
Dimensionality reduction using the principal component analysis
technique. To reduce the dimensionality of multidimensional data, both linear and
nonlinear methods are used. The most commonly used are linear methods, such
as the principal component analysis (PCA) [
In this paper, we use the principal component analysis technique, as the
most often used in such cases (see, for example, [
In this paper, before reducing the dimensionality of data, we form a set of subsequences of a xed length for each sequence of frames. To do this, we successively select subsequences of the prede ned length k with the step s starting from the beginning of the whole sequence (see Figure 3).
For each selected subsequence, the vector of features is formed as follows: each normalized frame of the subsequence is expanded into a row, and the rows obtained for individual frames are concatenated to each other.
The feature vectors of all sequences for di erent persons form the input matrix for the principal component analysis technique. When principal components are found, the projection of feature vectors onto the rst N principal components is taken as a feature description.
Classi cation of video sequences. The features obtained as a result of
the principal component analysis are used to train the support vector machine
(SVM) [
The description given above is valid for the training mode, in which the parameters of the dimensionality reduction procedure (PCA) and classi er (SVM) are con gured. In the testing mode, the data is processed in the same way, except that the parameters of the linear transformation (which is used to reduce the dimensionality) are xed to the values obtained in the training mode, and the classi cation is performed by the trained SVM classi er. 2.2
Identi cation of a moving person using hidden Markov models
In this paper, we consider the method described in [
A Hidden Markov Model (HMM) is de ned as = ( ,A,B), where is the
initial state distribution, A is the state transition probability matrix, and B is
the observation probability matrix. As the gait of a person consists of a sequence
of cyclic movements, the HMM can be naturally adapted to solve the considered
task. To built the HMM model one has to determine the number n of states
Si; i = 1::n of the model, and estimate initial , transition A, and observation
B probability matrices. In according to [
HMM initialization. The HMM is built for each person separately. The initial state probabilities are set to i = 1=n; i = 1::n, as the rst frame of an observed sequence can correspond to any state si. The transition probability matrix A = fai;j g is de ned as: ai;i = 0:5 and ai;i+1 = 0:5; 1 i < n, an;1 = 0:5. The observation probability matrix B = fbi;j g; i = 1::n; j = 1::m for a given set of frames X = xj ; j = 1::m is de ned as bi;j = e
D(ei;xj): (1) Here and are some constants, D() is a distance metric. In this paper we used Euclidean distance as a metric. Above mentioned constants are selected so that Pm j=1 bi;j = 1; i = 1::n. The described HMM is shown in the Figure 4.
As it can be seen from the above expression, we rst need to de ne
exemplar images ei; i = 1::n to estimate the observation probabilities. To do this, we
use the following procedure proposed in [
Examples of estimated exemplar images are shown in Figure 5.
HMM training. The initial HMM model described above can be further re ned
according to [
1. The most probable sequence of states is estimated for each training
sequence of frames using the Viterbi algorithm [
2. The exemplar images are re ned according to the estimation made at the previous step of the algorithm.
3. The transition probability matrix A is re ned using the Baum-Welch
algorithm [
Classi cation of video sequences. Given a number of trained HMM models
k; k = 1::K, and a test frame sequence, X = fxj g; j = 1::m, we select the model,
which gives maximum probability p(Xj k) of producing the test sequence. The
probability for each trained model is computed using the forward algorithm [
The described above methods were implemented in C ++ using the OpenCV library. A PC based on Intel Core i5-3470 CPU 3.2 GHz was used to perform experimental studies.
For the experimental study, the video sequences from the open CASIA GAIT
dataset [
For the method proposed in Section 2.1, we studied the dependence of the classi cation accuracy on the dimensionality of feature vectors (output dimensionality of the PCA technique). The results of the experiments are shown in the Figure 6. As it can be seen from the gure, the best values of the classi cation accuracy are achieved for 64-dimensional feature vectors. The increase in dimensionality is accompanied by the expected increase in processing time, although the changes are not very signi cant.
In the second experiment, we considered the dependence of the classi cation accuracy on the number of classes (persons). The experiment was carried out for 5, 10, 15, 20 and 25 classes, and other parameters remained xed. In particular, the step s was equal to 2 frames, the maximum shift m of the beginning of the extracted subsequences was equal to 15 frames, the dimensionality of feature vectors was equal to 64. The results of the experiment are shown in the Figure 7. This experiment was conducted on the rst three sequences of random classes with subsequent averaging.
It is worth noting that a direct experimental comparison to other works
seems to be quite a challenge in connection with the di erent data sets used,
as well as the potential di erences in the experimental conditions. The closest
approach to the proposed one is described in the paper [
As it can be seen in Figure 7, the processing takes an increasing amount of time as the number of classes increases. Considerable time is taken in the training mode. This fact becomes especially important in scenarios when the number of classes changes dynamically, and it is required to re-train the system regularly.
To compare the proposed method to the known approach, we implemented the HMM-based technique described in Section 2.2. The results of the experiments are shown in the Figure 8. As you can see, the HMM-based approach provided acceptable accuracy for the considered number of classes. But these values are signi cantly lower then the accuracy demonstrated by the proposed approach. In this paper, we proposed the method for human identi cation by gait. The proposed method consists of the detection of a moving person on a video sequence with the subsequent normalization of size, generation of subsequences, dimensionality reduction using the principal component analysis technique, and classi cation using the support vector machine. The experiments performed on the CASIA GAIT dataset allowed to determine the best values of the parameters of the proposed method and to compare the proposed method to the HMMbased technique. It was shown that the proposed method outperforms implemented HMM-based technique and provides high classi cation accuracy on the considered dataset. In particular, for 25 classes, the accuracy was 98,8% that corresponds to the current state of research.
The drawbacks of the proposed method include its long operating time. In connection with this, a promising line of research is speeding up this method. Acknowledgments The reported study was funded by RFBR according to the research project no. 17-29-03190-o .