With the rapid growth of transport number on our streets, the need for finding a vacant parking spot today could most of the time be problematic, but even more in the coming future. Smart parking solutions have proved their usefulness for the localization of unoccupied parking spots. Nowadays, surveillance cameras can provide more advanced solutions for smart cities by finding vacant parking spots and providing cars safety in the public parking area. Based on the analysis, Google Cloud Vision technology has been selected to develop a cyber-physical system for smart parking based on computer vision technology. Moreover, a new model based on the fine-tuned convolutional neural network has been developed to detect empty and occupied slots in the parking lot images collected from the KhNUParking dataset. Based on the achieved results, the performance of parking lots' detections can be simplified, and its accuracy improved. The Google Cloud Vision technology as parking slots detector and a pre-trained convolutional neural network as a feature extractor and a classifier were selected to develop a cyber-physical system for smart parking. As a result of the computational investigation, the proposed fine-tuned CNN managed to process 66 parking slots in roughly 0.14 seconds on a single GPU with an accuracy of 85.4%, demonstrating decent performance and practical value. Overall, all considered approaches contain strengths and weaknesses and might be applied to the task of parking slots detection depending on the number of images, CCTV angle, and weather conditions. Video-image processing, smart parking, deep learning, convolutional neural network, IntelITSIS'2022: 3rd International Workshop on Intelligent Information Technologies & Systems of Information Security, March 23-25, ORCID: 0000-0003-3609-112X (P. Radiuk); 0000-0001-7019-0354 (O. Pavlova); 0000-0003-3239-8255 (H. El Bouhissi); 0000-0002-
In recent years, the issue of creating smart parking has become highly essential, especially in large cities. As the number of cars has rapidly increased over the last few years (see Fig. 1), so does the need for parking spaces and search facilities. Assuming that the average driver spends 20 minutes searching for such a place every day, about 120 hours a year could be spent on something more useful.
As shown in Fig. 1, most vehicles entering Ukraine have been newly imported automobiles (red part of the column) and used cars from Europe and the USA (green part of the column). At the same time, it is noticeable that from 2016 to 2020, the import of new vehicles remains at about the same level, but the share of imports of used cars is gradually increasing. Such an outcome was due to changes in the legislation on customs clearance of vehicles imported from abroad on Nov. 25, 2018; a law was passed to simplify the procedure for customs clearance of used cars imported from abroad
2022 Copyright for this paper by its authors. [1]. Nowadays, there are many smart parking projects, but ready-for-use examples can be counted on the fingers of one hand, and information about the cost-effective aspect of their implementation is generally minimal. It should be noted that when designing such tools, the most significant financial part of the development is born by the software, not hardware. After considering and comparing different parking detection techniques [2], a conclusion has been made that the method of smart parking using visual surveillance based on an external closed-circuit television (CCTV) is much more effective than others, considering most of the factors considered.
The scientific community has actively investigated and proposed novel CV methods and approaches to identifying and demarcating parking areas. Traditional visual-based techniques for detecting parking slots are divided into line-based [5] and marking-point-based [6]. Line-based approaches first construct visible lines in an image around a region of interest (ROI) using various CV features, such as the Canny edge detector [7], Laplacian operator [8], and Haar Cascade [9]. Next, the parameters of the detected lines are predicted using a line fitting algorithm to draw boundaries around the ROI. Similar to line-based approaches, marking-point-based ones seek for marking points in an image around an ROI using, for example, Harris edge detector [10] or boosting decision tree [11], and then use a pattern matching technique [12] or combining line detection [13] to locate a targeted parking space. Even though such traditional techniques of detecting parking spaces provide decent results, they are susceptible to changes in the environment and, therefore, not applicable to component cases of delimitation of parking areas.
Overall, table 1 presents decent studies that have been conducted over the past years to find the best approach for parking slots detection and vehicle recognition.
This ensemble approach allows identifying a vehicle or a parking slot from any angle of view; the use of imposed features ensures the detection accuracy of an individual vehicle of roughly 100%.
neural network are fine-tuned according to the characteristics of the parking spaces, leading to a high precision rate of 99.63%.
maximum recognition accuracy of about 100% by the fixed CCTV position and constant light intensity).
the benchmark dataset may not be practical for recognizing real outdoor parking lot.
In multiple-vehicle detection, the superimposed features sometimes do not distinguish between similar edges of objects, leading to the detection of two vehicles as one.
parking spaces, as the entry point markings are faint, and the parking space is less than the threshold.
adapted to a specific parking space; even minor changes within the parking lot can adversely affect classification accuracy.
As can be seen from Table 1, a deep learning (DL) approach, particularly deep convolutional neural networks (CNNs), has been most frequently used over the past five years and has shown the most robust recognition of parking lots, among other approaches. For example, DeepPS [12] is the first multi-module information system based on DL algorithms for identifying parking spaces. This system is based on two visual-based technologies: the well-known OpenCV library to describe the marking points in an image around ROI and CNN to identify the target features of vehicles in an image and match the paired marking points with the identified features. Overall, ensemble approaches based on deep neural networks demonstrate the best performance in detecting parking slots and vehicles in different environmental conditions.
Therefore, considering the abovementioned analysis, two visual-based technologies were defined as the most effective for parking lot detection – OpenCV and CNN. 2.1.
Over the past decades, the OpenCV computer vision library [18] has become the leading technology in the image processing domain. This tool set serves as a so-called infrastructure for applying CV techniques in information systems. OpenCV is used, among other things, to resize input images, convert them to vector form, and detect the features of target objects in the image. At the same time, one of the most popular approaches to detecting features in the image today is DL, in particular, CNN [19]-[21].
The CNN model combines many functional operations that transmit the input image as feature vectors into the resulting data to estimate the belonging of the identified objects to predefined classes. The CNN architecture utilized in this study is from the authors’ previous work [22] and is depicted in Fig. 2.
According to the classification results in [23], we conclude that combining the GCV API system and OpenCV + CNN tools may achieve more robust performance and higher classification accuracy. 2.2.
Another equally well-known image recognition technology is the Google Cloud Vision API (GCV API) [24]. The GCV API is a de facto set of prepared machine learning models and algorithms that service users can quickly implement to meet their business needs. The principle of the GCV API is to perform two steps: 1) assigning labels to the original image; 2) automatic recognition of objects in the image by predefined classes. The GCV API is a universal classifier that identifies various moving and still objects in an image.
In [23], we conducted a preliminary experiment: ten images were used from the video surveillance camera of one of the parking lots of Khmelnytskyi National University. The images were preliminarily prepared: the contours were cropped to bring the focus as close as possible to the location of the cars. In addition, the objects in the image were magnified to increase the likelihood of finding the object.
The experiment was to test the same image using two of the most popular image recognition technologies. The object identification results on the target image, performed using OpenCV + CNN and GCV API technologies, are shown in Fig. 3.
Fig. 3 shows that GCV API technology coped much better with the task of identifying cars on the image (Fig. 3a) than OpenCV + CNN technology (Fig. 3b).
Hence, the GCV API system as a parking slots detector and a pre-trained CNN as a feature extractor were chosen to develop a cyber-physical system for smart parking.
The authors compiled Dataset (KhNUParking) from the collected images extracted from an external closed-circuit television (CCTV). The CCTV was installed on Campus 3 of Khmelnytskyi National University, Ukraine. The images show parking spaces of the outdoor parking lot between campuses 3 and 4 of the university (Fig. 4).
(a) (b) Figure 4: The samples of the KhNUParking dataset presenting targeted parking spaces: (a) – almost all parking lots are empty, (b) – nearly all are fully occupied
The initial KhNUParking was collected of 100 images extracted from a CCTV, each of which was 853 × 480 pixels, and then was split into training (70%) and validation (30%) subsets. An additional subset of 100 images was created to test the classification models. Furthermore, actual annotations of the parking slots, ground boxes (33 slots), and the occupancy (3300) were employed to assess the proposed approach’s accuracy.
PKLot: a subset of 390 randomly sampled images of 1280 × 720 pixels was collected from the PKLot dataset [25]. It must be noted that in the original PKLot dataset, parked vehicles are displayed from up to down.
Experimental setup: all computational experiments were performed on the Python v3.8 stack with Keras as the back end. The calculations were executed on 8-core Ryzen 2700 and a single GPU card GeForce GTX1080 with 8 GB of memory.
Methodology: the proposed approach for CV technology is depicted in Fig. 5.
In this work, we utilized a neural network model based on pre-trained CNN as a feature extractor and a two-layer perceptron as a classification module. The pre-trained CNN contained 1000 classes (pre-trained with the ImageNet dataset). To prepare the model for detecting occupied and empty parking spaces, the last fully connected layers in the network were replaced with two classes that correspond to “Empty” or “Occupied.” In this work, the testing models were evaluated by several statistical indicators and run-time, an average time in seconds to read images from the hard disk and crop them. Statistical measurements used in this study are defined as:
Accuracy = Precision =
TP + TN TP + TN + FP + FN
TP + TN TP + TN + FP + FN , , Recall = 1 =
TP TP + FP TP ,
,
TP + FN
(
The data augmentation technique was also performed on the fine-tuning dataset to reduce overfitting.
Two transformations were applied: 1) reflection along X and Y axes and 2) change of the X and Y scales of the images. Furthermore, the input images were resized to 128 × 128 to fit the input of the fine-tuned CNN.
The network was pre-trained with a stochastic gradient descent with a momentum of 0.8, a learning rate of 0.005, and a batch size of 64; training epochs were set to 20. The pre-training process took roughly 50 minutes on a single GPU. Fig. 6 shows the training and validation accuracy and loss curves.
The prepared fine-tuned CNN was tested on the set of 100 KhNUParking images in the following matter. At first, in each of 100 images, 33 individual parking slots were cropped and then passed to the fine-tuned CNN to perform the classification task.
The actual samples of the KhNUParking dataset contained the status of 3300 occupied/empty spaces and 33 ground boxes of the parking slots.
Here, the delineations of parking slots are presented as the so-called bounding boxes that also crop the individual parking slots.
A bounding box is determined by [x, y, w, h], where [x, y] – the coordinates of the middle of the boxes, and [w, h] – the width and height. Fig. 7 presents the classification results obtained by the testing dataset.
As it is seen from Fig. 7, 748 empty parking spaces and 2069 occupied parking lots were correctly identified; meanwhile, 321 vacant lots were classified wrongly, and 162 occupied spaces were recognized as open. So, the overall classification accuracy was 85.34%. According to the obtained classification results, the proposed fine-tuned CNN makes more mistakes and thus less accurate in identifying empty parking spaces.
Upon visualizing a few of the wrongly identified parking spaces in Fig. 8, it was observed that those parking lots mostly contained parts of vehicles, people, or other objects inside the image crop. Finally, Fin. 9 shows the visual representation of the classified parking slots.
Several models, namely AlexNet [19], VGG-16 [20], and MobileNetV2 [21], were compared with the fined-tuned CNN in terms of their efficiency and accuracies.
The classification results obtained from all models are shown in Table 2.
Table 2 presents the values of statistical measurements (
Google Cloud Vision showed worse performance than the analogs in these experiments yet retained appropriate generalizing ability over diverse parking spaces. In conclusion, all considered approaches contain strengths and weaknesses and might be applied to the task of parking slots detection depending on the number of images, CCTV angle, and weather conditions.
Overall, the proposed fine-tuned CNN could process 66 parking slots in roughly 0.14 seconds on a single GPU with an accuracy of 85.4%, demonstrating decent performance and practical value.
Therefore, during the study, an analysis of information technologies for image recognition based on computer vision was conducted. Based on the analysis, Google Cloud Vision technology has been selected to develop a cyber-physical system for smart parking based on computer vision technology. A new model based on the fine-tuned convolutional neural network has been developed to detect empty and occupied slots in the parking lot images collected from the KhNUParking dataset. Based on the achieved results, the performance of parking lots’ detections can be simplified, and its accuracy improved. It was also concluded that the Google Cloud Vision technology as parking slots detector and a pre-trained convolutional neural network as a feature extractor and classification were decided to develop a cyber-physical system for smart parking. As a result of the computational investigation, the proposed fine-tuned CNN managed to process 66 parking slots in roughly 0.14 seconds on a single GPU with an accuracy of 85.4%, demonstrating decent performance and practical value.
Further investigation will be devoted to developing the server- and client-based parts as a mobile app that tracks the availability of vacant places at the university’s parking lot. 6. References [8] M. Noor and A. Shrivastava, Automatic parking slot occupancy detection using Laplacian operator and morphological kernel dilation, in 2021 10th IEEE International Conference on Communication Systems and Network Technologies (CSNT-2021), (2021) 825–831. doi:10.1109/CSNT51715.2021.9509620. [9] I. M. Hakim, D. Christover, and A. M. Jaya Marindra, Implementation of an image processing based smart parking system using Haar-Cascade method, in 2019 IEEE 9th Symposium on Computer Applications Industrial Electronics (ISCAIE-2019) (2019) 222–227.
doi:10.1109/ISCAIE.2019.8743906. [10] J. S. L. Tang and S. Manickam, Parking lot occupancy detection using image overlay and intersection technique with Harris corner detector, J. Eng. Technol., 11 1 SE-Articles (2020) 3752. URL: https://jet.utem.edu.my/jet/article/view/5876 (accessed Oct. 25, 2021) [11] R. Sun, G. Wang, W. Zhang, L.-T. Hsu, and W. Y. Ochieng, A gradient boosting decision tree based GPS signal reception classification algorithm, Appl. Soft Comput., 86 (2020) 105942. doi:10.1016/j.asoc.2019.105942. [12] L. Zhang, J. Huang, X. Li, and L. Xiong, Vision-based parking-slot detection: A DCNN-based approach and a large-scale benchmark dataset, IEEE Trans. Image Process., 27 11 (2018) 5350– 5364. doi:10.1109/TIP.2018.2857407.
W. Li, L. Cao, L. Yan, C. Li, X. Feng, and P. Zhao, Vacant parking slot detection in the around view image based on deep learning, Sensors, 20 7 (2020) 2138.doi:10.3390/s20072138. [13] J. D. Trivedi, M. Sarada Devi, and D. H. Dave, Different modules for car parking system demonstrated using Hough transform for smart city development, in Intelligent Manufacturing and Energy Sustainability, 169 (2020) 109-121. doi:10.1007/978-981-15-1616-0_11. [14] H. Canli and S. Toklu, “Deep learning-based mobile application design for smart parking,” IEEE
Access, 9 (2021) 61171–61183. doi:10.1109/ACCESS.2021.3074887. [15] G. Manjula, G. Govinda Rajulu, R. Anand, and J. T. Thirukrishna, Implementation of smart parking application using IoT and machine learning algorithms, in Computer Networks and Inventive Communication Technologies, (2022) 247-257. doi:10.1007/978-981-16-3728-5_18. [16] C. Min, J. Xu, L. Xiao, D. Zhao, Y. Nie, and B. Dai, Attentional graph neural network for parking-slot detection, IEEE Robot. Autom. Lett., 6 2 (2021) 3445–3450 doi:10.1109/LRA.2021.3064270. [17] S. Gollapudi, OpenCV with Python, in Learn Computer Vision Using OpenCV, Apress,
Berkeley, CA, (2019) 31-50. doi:10.1007/978-1-4842-4261-2_2. [18] A. Krizhevsky, I. Sutskever, and E. G. Hinton, ImageNet classification with deep convolutional neural networks, Commun. ACM, 60 6 84-90. doi:10.1145/3065386. [19] K. Simonyan and A. Zisserman, Very deep convolutional networks for large-scale image recognition, in 3rd International Conference on Learning Representations (ICLR-2015), (2014) 1-14. doi:10.48550/arxiv.1409.1556. [20] M. Sandler, A. Howard, M. Zhu, A. Zhmoginov, and L. Chen, MobileNetV2: Inverted residuals and linear bottlenecks, in 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR-2018), (2018) 4510-4520. doi:10.1109/CVPR.2018.00474. [21] O. Barmak and P. Radiuk, Web-based information technology for classifying and interpreting early pneumonia based on fine-tuned convolutional neural network, Comput. Syst. Inf. Technol., 3 1 (2021) 12-18. doi:10.31891/CSIT-2021-3-2. [22] V. Avsiyevych and V. Kovalenko, Analysis of information technology for smart parking based on artificial neural networks, in XIII All-Ukrainian Scientific and Practical Conference “Actual Problems of Computer Science” (APCS-2021), (2021) 12-14.
Vision AI Google Cloud, Google, Inc, 2021. URL: https://cloud.google.com/vision (accessed Dec. 17, 2021). [23] P. R. L. De Almeida, L. S. Oliveira, A. S. Britto, E. J. Silva, and A. L. Koerich, PKLot - A robust dataset for parking lot classification, Expert Syst. Appl., 42 11 (2015) 4937-4949. doi:10.1016/J.ESWA.2015.02.009.