This article shows an effective implementation of the algorithm for detection of road signs using video obtained by a camera installed in a vehicle. Road signs detection and recognition are implemented using CUDA and operate in real-time on a mobile GPU. Vehicle speed is used to predict the position of road signs in the adjacent frames in a video sequence. The experimental results confirmed the high efficiency of the developed road signs system.
In the past 10 years, the problem of detection of road signs has been solved by many
scientists in different countries [
Recent development of modern mobile processors allowed many car manufacturers to install computer vision systems in automobiles. Among other tasks, such systems also solve the road signs recognition task. However, the main problem of currently provided systems is the low detection accuracy, as well as the inability of some systems to recognize some geographically-specific road signs, for example Russian signs. This article describes the developed technology to detect and recognize road signs in real-time. The developed system also uses the speed obtained from the vehicle. It makes it possible to predict not only the presence of the object, but also its size and exact location in the adjacent frames. Thus, detection accuracy is improved, while the computational complexity is the same.
To provide a driver with actual information about currently applicable road signs, the system should recognize the localized object. After the localization procedure, a small image of a sign is cut from the original image. The size of this image is scaled to 31x31 pixels, which is enough for recognition. In fact, recognition of objects in such small images does not cause any difficulties, since all possible images of road signs are known in advance. In addition, in the case of successful localization procedures, the exact coordinates of a road sign are known. Thus, the template matching algorithm is used at the classification step. This algorithm shows good results in combination with the localization step.
In order to train and test the developed algorithms, we used the German Traffic Sign
Detection Database [
The developed technology described in this paper consists of three steps: image preprocessing, the localization of a road sign, and its classification.
To extract red and blue colors from a source image, the HSV color space is used in
the preprocessing step. Due to errors of camcorder sensor or small colored objects in
the images, after applying a threshold filter, some spot noise may occur. To remove
this noise, the algorithm described in [
Paper [
To classify the found object, the algorithm uses specially prepared etalon binary images. Fig. 2 shows several such images, which are actually the cut inner area of road signs. For recognition of a sign, a simple subtraction of two images is applied and then the pair with the lowest value is chosen, which will point to the highest proximity to a certain etalon. In the case of large values, the algorithm will give a false detection message, as none of the etalon images suits the localized area. This means that the found the area most likely does not contain a road sign.
The execution time for such classification in average is 1.2 ms using 32 types of etalon images. Here, each single etalon image type corresponds to a unique image of a road sign. There can be an unlimited number of etalon images of each type in order to consider as many road signs variants as possible. For example, this implementation of the method uses 5 etalon images for every road sign type, for a total of 160 etalon images.
To assess the accuracy of the detection algorithm, the German annotated image data
base containing images with road signs was used [
Sliding window + SVM [
with preprocessing
without preprocessing
Viola-Jones [
HOG [
The modified GHT method was implemented using CUDA technology. Table 2 shows the results of performance improvement. To measure the performance, the mobile processor Nvidia Tegra K1 was used, which contains 192 graphics cores for computation using CUDA technology and 4 CPU cores of ARM architecture.
The article describes a technology of detection and recognition of road signs. To localize a road sign, a modification of the generalized Hough transform has been developed taking into account the constraints on the executing time of a single frame. The algorithm shows effective results and works well with the preprocessed images. Tracking using the value of the current car velocity has improved the performance of the system, as the search area in the adjacent frames can be significantly reduced. In addition, the presence of the sign in the sequence of adjacent frames in predicted areas significantly increases the confidence of correct detection. Classification of localized road signs ensures that the whole procedure of recognition of road signs is successful. Testing the described algorithms using video sequences has proved the ability to process high-resolution video stream in real-time. The detection range of road signs when using a FullHD camera is up to 50 m.
Massively multithreaded programming technology CUDA was used to speed up the implementation of the developed algorithms. When using a mobile GPU Nvidia Tegra K1, the acceleration reaches 15 times.
This work was supported by Project #RFMEFI57514X0083 by the Ministry of Education and Science of the Russian Federation.