Although existing object detection algorithms have achieved good results, it is still a challenge to effectively detect small-scale pedestrians in real time. Aiming at the problems of complex structure, large number of parameters and high missed detection rate of small targets in existing pedestrian detection algorithms, the YOLO-DSRF pedestrian detection algorithm is proposed. On the basis of YOLOv4, the depth separation convolution was first introduced to significantly reduce the amount of parameters and computation of the model, and the channel attention mechanism was introduced into the network to improve the influence of important channel features on the network, a feature fusion module was designed in the backbone network by merging deep and shallow features to effectively extract target semantic information and location information, and introduce a receptive field module in the detection head to simulate the human receptive field to enhance feature extraction capability for small targets. For training and verification on the Caltech dataset, compared with the original algorithm, the number of parameters is reduced by 65.2%, and the running speed on the GPU is increased by 20%. The AP is roughly the same. The algorithm proposed in this paper can effectively reduce the model complexity while ensuring accuracy. Thereby increasing the running speed.
Object detection has always been a hot research direction in the field of computer vision. Its task is
to accurately identify and locate all objects in an image or video. Pedestrian detection is an important
branch of object detection, which is widely used in re-identification, and intelligent monitoring[
The feature extraction network of the pedestrian detection algorithm has a low downsampling factor and a small receptive field, and pays more attention to small-scale pedestrian targets. However, due to the weak ability to represent semantic information, it is easy to detect false object. Deep features have large downsampling multiples and large receptive fields, which may easily lead to misse a lot of smallscale pedestrian targets. To solve the above problems, some scholars proposed FPN (Feature Pyramid Networks)[10], which fuses the features of deep and shallow layers to improve the detection effect, but the detection speed of this kind of algorithm is low and cannot be detected in real-time. The YOLOv3[11] proposed by Redmon et al. uses the DarkNet-53[12] residual network for feature extraction and combines the FPN network to significantly improve the small target detection performance. Bochkovskiy A improved on YOLOv3 and proposed YOLOv4[13]. Using CSPDarkNet[14]as the backbone network, using SPP+PAN to fuse feature maps of different sizes, greatly improving the accuracy of model detection, but its complex structure and large number of parameters are difficult to deploy on mobile terminals or embedded devices.
In order to reduce the complexity of the YOLOv4 model and improve the network running speed.
Wang et al.[
In order to make yolo more effective in detecting small targets, Gao et al. [
This paper proposes the YOLO-DSRF (Depthwise separable convolution & SE attention module & Receptive field module & Feature fusion module) algorithm by improving YOLOv4. First, replacing traditional convolution with depthwise separation convolution reduces the amount of parameters and improves the running speed. Second, in order to fully extract the features of small objects, the SE attention mechanism is introduced to suppress the influence of noise and enhance the learning of important channel features. Third, the FFM module is designed to integrate deep and shallow features, and to strengthen the influence of shallow features on geometric information. and semantic information representation ability. Finally, adding a receptive field module to the detection head of small-scale targets to enhance feature extraction for small-scale targets. Finally, the effectiveness of the proposed algorithm is verified on the Caltech dataset.
The proposed YOLO-DSRF algorithm is improved by YOLOv4, and the improved part is the dashed box in Figure 1. Depth separable convolution is introduced to replace the 3*3 convolution kernel in the original network, and SE attention mechanism, RFB module, and top-down feature fusion module are added
The depthwise separation convolution is a lightweight method for embedded devices proposed by
Sandler et al. in MobileNet [
Paying attention to important parts and ignoring irrelevant parts is the attention mechanism. SENet
[
The receptive field module was proposed by Liu et al. [19] in RFBNet. Inspired by the receptive field structure in the human visual system, it simulates the group receptive field in the human superficial retinal image. As shown in Figure 4, its construction is similar to the Inception structure. A similar multi-branch convolution module that captures multi-scale information. In addition, dilated convolution is introduced to expand the sampling range to extract finer features of the target. Since there are few features of small-scale pedestrians, in order to enhance the network's feature representation for small objects, this paper introduces RFB in YOLOv4 to improve the network's detection effect on small objects.
The shallow features extracted by the network have low downsampling times and high resolution for detecting small-scale pedestrian targets. They have strong geometric representation capabilities, but weak semantic representation capabilities. The deep features extracted by the network have small downsampling multiples and low resolution to detect large-scale pedestrian targets. They have strong semantic representation ability, but weak geometric representation ability. In order to enhance the feature representation ability of the network to achieve accurate pedestrian detection, this paper designs a top-down FFM as shown in Figure 5(a) and a bottom-up FFM as shown in Figure 5(b). Strengthen the fusion of deep features and shallow features in the backbone network, and enhance the semantic information representation ability of shallow features to improve the detection effect of small-scale pedestrian targets. Check the effect. The module uses P3, P4, and P5 in Figure 1 as input. It can be seen from Figure 5 that the top-down FFM first upsamples P5 and stacks P4, and then performs convolution to achieve feature integration and output P4' to replace the original network. P4; the convolution output P3' performed after stacking the resulting P4' upsampling with P3 replaces P3 in the original network. Bottom-up FFM is to downsample the P3 feature map with a convolution with a stride of 2 and stack it with P4, and then perform the convolution output to integrate the output P4', stack the resulting P4' downsampling with P5, and convolve the output P5 ', both operations strengthen the fusion of shallow and deep features in the backbone network, and enhance the geometric and semantic representation capabilities of features, thereby improving the accuracy of network detection.
Fig 5 Feature fusion module
The experiment uses the Caltech[
The experimental environment in this paper is the GNU operating system, the graphics card is Tesla V100s, the CUDA version is 11.0, the Pytorch 1.8 deep learning framework, and the compiled language is Python3.6.8. The experiments use the Adam optimizer with a batch size of 8 and a cosine annealing learning rate with an initial learning rate of 0.001.
This paper uses five indicators of P (Precision), R (Recall), AP(Average Precision), PARAM (Parameters), FPS (Frame Per Second) five indicators to evaluate network performance, The calculation formulas of P , R and AP are shown in formula (1-3).
= = = ∗ 100% ∗ 100% （ ） （1） （2） （3）
TP (True Positive) is the number of targets correctly detected by the model, FP (False Positive) is the number of falsely detected targets, and FN (False Negative) is the number of false detections and missed detections. AP is the area under the P-R curve, which is used to measure the detection capability of the network; the parameter quantity is used to measure the complexity of the network; FPS is the number of pictures processed per second, which is used to measure the computing power of the network.
R(%) 76.04 72.19 73.41 74.18 74.46
In order to verify the effectiveness of the algorithm proposed in this paper, an ablation experiment is designed. First, YOLO-D replaces the 3*3 convolution in the network with a depthwise separable convolution, and then introduces the SE attention mechanism. , RFB, and the two FFMs designed in this paper. The results are shown in Table 1. The introduction of depthwise separable convolution greatly reduces the amount of network parameters, which is only 27.9% of YOLOv4. 2.5%, and Recall decreased by nearly 4%. The introduction of the SE attention mechanism increases the number of parameters by 14.11M, the AP increases by 0.93%, and the Recall increases by 1.22%, which proves that the SE attention mechanism can enhance the feature extraction ability. The introduction of RFB in the network detection head only increases the amount of parameters by 0.35M, the Recall is increased by about 2%, and the AP is increased by 1.1%, which proves that RFB can more effectively extract the features of small targets. YOLO-D-FEM1 introduces the bottom-up feature fusion module designed in this paper, the parameters are increased by 12.6M, the AP is increased by 1.52%, and the Recall is increased by 2.27%. YOLO-D-FFM2 introduces the top-down feature fusion module designed in this paper. The number of parameters is increased by 5.04M, the AP is increased by 1.47%, and the Recall is increased by 2.28%. It is proved that the two modules designed in this paper can improve the representation ability of semantic information and geometric information by fusing deep and shallow features in the backbone network. In addi-tion, based on YOLO-D-SE, RFB, bottom-up FFM, and topdown FFM are introduced respectively. Compared with YOLO-D-SE, AP increased by 0.89%, 0.95%, 1.3%; Recall increased by 0.89%, 0.95%, 1.3%. Finally, for the algorithm YOLO-DSRF proposed in this paper, compared with YOLO-D, AP, Recall, and Precision have been improved by 2.35%, 3.52%, and 1.23 respectively; compared with YOLOv4, the difference in AP is only 0.15%; the difference in Recall is only 0.33 %; increased by 0.07%. Compared with YOLO, the number of parameters is increased by 19.49M, which is only 35.8% of YOLOv4. Runs 20% faster on GPU
In view of the complex structure of the YOLOv4 algorithm, the poor real-time performance of mobile devices and the insufficient extraction of small target pedestrian features, this paper adopts the depth separable convolution to replace the ordinary convolution, and introduces the SE attention mechanism, RFB and FFM designed in this paper. The designed YOLOv4-DSRF algorithm greatly reduces the amount of parameters and computation, and enhances the feature extraction capability of the network, especially the feature extraction capability for small targets. However, limited by the low resolution of the feature map extracted by the network, it is not conducive to the feature analysis of small objects. In the future research work, we will study the construction of a high-resolution lightweight network. Also, it is very important to study the appropriate feature fusion method to improve the detection of small objects. optimizing the feature fusion method to further improve the detection effect of small target pedestrians.
region proposal networks[J].IEEE Transactions on Pattern Analysis & Machine Intelligence， [8]. LIU W，ANGUELOV D，ERHAN D，et al.SSD：single shot multibox detector[C]//European
Conference on Computer Vision.Cham：Springer，2016：21-37. [9]. REDMON J，DIVVALA S，GIRSHICK R，et al.You only look once：unified，real-time object detection[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition，2016：779-788.
Conference on Computer Vision and Pattern Recognition，2017：2117-2125. [11].REDMON J，FARHADI A.YOLOv3：an incremental improvement[J].arXiv：1804.02767， 2018. [12].KIM K J，KIM P K，CHUNG Y S，et al.Performance enhancement of YOLOv3 by adding prediction layers with spatial pyramid pooling for vehicle detection[C]// 2018 15th IEEE International Conference on Advanced Video and Signal Based Surveillance（AVSS），2018：1-6. [13].BOCHKOVSKIY A，WANG C Y，LIAO H Y M.YOLOv4： optimal speed and accuracy of [19].Liu S，Huang D.Receptive field block net for accurate and fast object detetion[C]//European Con