In this study, a micro-Doppler radar-based method for classifying gaits with and without smartphone texting while walking (i.e., the classification of normal pedestrians and smartphone zombies) is proposed. The motion features in the gait of the participants texting on a smartphone while walking were collected using the micro-Doppler radar as time-frequency distributions (spectrograms), which showed the velocity variation of body parts with respect to time. We adopted a method of inputting spectrogram images obtained from a short-time Fourier transform of the received signals of the Doppler radar to a convolutional neural network. Experimental results showed that the proposed method achieved accurate classification with over 80 % accuracy.
With the increasing use of smartphones, the number of people texting on smartphones while walking
(called "smartphone zombie") has increased, which has led to many accidents, such as collisions and
falling. Various approaches have been studied to detect the action of people using smartphones while
walking to avoid such accidents and to develop warning and collision avoidance systems. For example,
a method that uses built-in smartphone sensors has been proposed [
In contrast, radar techniques can measure motion information without these problems. In particular,
the micro-Doppler radar technique is a promising candidate for accurate pedestrian sensing because of
its capability to remotely measure the time variation in the velocities of human body parts [
In this paper, we present a method for classifying the gaits of common pedestrians and smartphone zombies using micro-Doppler radar data. The micro-Doppler radar data of the gaits of participants walking normally and participants on a smartphone while walking were collected and analyzed. Then, an accuracy of more than 80 % was demonstrated in classifications by inputting a convolutional neural network (CNN) to time-frequency distribution (spectrogram) images calculated from radar-received signals. We investigated the classification accuracies using representative architectures of CNNs, such as VGGNet, ResNet, and AlexNet, and revealed an efficient CNN model for micro-Doppler radar-based detection of smartphone zombies.
2.2.
We then generated images of the time-frequency distribution (spectrogram images) of the received
signals using a short-time Fourier transform (STFT), similar to our previous study [
Figure 2 shows examples of the generated spectrogram images of walking motion types (a) and (b).
The frequencies of the spectrogram correspond to the target velocity, components in the spectrogram
images with larger received power correspond to the motion of the torso, and accompanying relatively
larger frequency variations correspond to the forward motion of the legs. Although motion type (b)
(smartphone zombie) tended to have a slightly lower frequency compared with motion type (a)
(common walking), clear differences between the two types could not be confirmed in the spectrogram
images. This is because the essential differences between these two types of walking motion were less
visible in the spectrograms. The simplest difference between these types was the presence of an
armswinging motion. However, the arm motions were not clearly confirmed in the spectrograms because
the received powers of the echoes from the arms were quite small compared to those from the torso and
legs [
(a) (b) 2.3.
To extract and use the slight differences between the spectrogram images of the two groups, we used
a CNN for feature extraction and gait classification. Figure 3 shows an outline of the proposed
classification method. The CNN was used as the deep-learning method for classification, similar to
previous studies on Doppler radar-based personal identification [
To evaluate the classification accuracy, we performed a hold-out validation. In each architecture, the CNN was trained using 80 % of the generated images, and the remaining 20 % were used as test data. Ten hold-out validation trials were conducted by randomly varying the training and test data. The mean and standard deviation of the classification accuracies across all trials were calculated.
Table 1 presents the test results for the classification of walking motion types (a) and (b) using various CNN architectures. LeNet, AlexNet, and ResNet-50 achieved accurate classification with over 80 % accuracy. The highest accuracy of 85.7 % was achieved using AlexNet. The accuracy of ResNet50 was relatively stable, with a smaller standard deviation. These results indicate that the CNN with spectrogram images accurately classified the gaits of the common pedestrian and smartphone zombie groups, even though their differences in the images were minor. The results for LeNet and AlexNet show that a relatively simple and lower-layered network can sufficiently capture the gait features of smartphone zombies. However, relatively deeper ResNet-50 results were stable, which implies that such a network may be effective when the data of a larger number of participants and training samples are used; this should be an important future direction of this study.
The effectiveness of the proposed method using AlexNet, which achieved the highest accuracy, is discussed using the confusion matrix and convergence curve. Table 2 presents the confusion matrix for the results using AlexNet and indicates that there are some cases in which the use of the smartphone did not change the speed of the torso and leg motions; therefore, these data were not correctly classified. However, a sensitivity of 84 % and a specificity of 86 % were achieved. Thus, the proposed method can accurately screen for smartphone zombies. Figure 4 shows an example of the convergence curves of the proposed method using AlexNet and ResNet-50. For both networks, the accuracy converged to approximately 50 epochs during the test process, and overfitting was not confirmed. Thus, the proposed method effectively trained the classification model. Although AlexNet achieved a better accuracy result, the steady state accuracy of ResNet-50 was stable compared to that of AlexNet, and these results indicate a tendency similar to that in Table 1, which indicates the possibility of the effectiveness of a deeper network.
Finally, we discuss the mechanism of accurate gait classification in our results. As shown in Section
2.2, the differences between the images of the two walking types were not clear because the motion
differences in the arms and head were not sufficiently measured. However, our results indicated that a
classification with an accuracy of more than 80 % was achieved using the spectrogram images that
mainly reflected the motions of the torso and legs. It appears that the proposed method extracted features
corresponding to slight differences in such motions using the CNN. A biomechanical study clarified
slight reductions in gait speed and changes in the tibialis anterior and gastrocnemius when texting on a
smartphone while walking for young adults [
In this study, the gaits of common pedestrians and smartphone zombies were measured using the micro-Doppler radar, and images of spectrograms (time-frequency distributions) of the radar received signals were used for classification. By applying a CNN using the image spectrograms as the input data, a classification accuracy of 86% was achieved using the experimental data. Thus, the effectiveness of the proposed method for remotely detecting the action of texting on a smartphone while walking was verified.
In our experiments, all participants were approximately 20 years old and were measured in the same experimental environment. Therefore, the future direction of our study is to measure various types of participants of different ages and in different environments. In addition, experiments on multiple pedestrian situations are important to demonstrate the practicality of the proposed method.
We appreciate all the participants for their engagement in this study.