Survey on Testing of Autonomous Driving Systems Wenjie Chen Zeyu Ma Chao Wang Software Engineering Institute Software Engineering Institute Software Engineering Institute Shanghai Key Laboratory of Computer Shanghai Key Laboratory of Computer Shanghai Key Laboratory of Computer Software Testing & Evaluation Software Testing & Evaluation Software Testing & Evaluation Shanghai Development Center of Shanghai Development Center of Shanghai Development Center of Computer Software Technology Computer Software Technology Computer Software Technology Shanghai, China Shanghai, China Shanghai, China cwj@sscenter.sh.cn mzy@sscenter.sh.cn wc@sscenter.sh.cn Mingang Chen Lizhi Cai Software Engineering Institute Software Engineering Institute Shanghai Key Laboratory of Computer Shanghai Key Laboratory of Computer Software Testing & Evaluation Software Testing & Evaluation Shanghai Development Center of Shanghai Development Center of Computer Software Technology Computer Software Technology Shanghai, China Shanghai, China cmg@sscenter.sh.cn clz@sscenter.sh.cn Abstract—Before autonomous driving vehicles are II. TESTING PROCESS OF AUTONOMOUS DRIVING SYSTEMS commercialized, they need to undergo a series of rigorous tests. This paper first proposes the general process of autonomous Autonomous driving system testing is an important driving system testing, and then summarizes the research procedure in the development process of autonomous vehicles, progress of autonomous driving vehicle testing from the model, it mainly focuses on testing the functional suitability, simulation, network, and vehicle levels, and analyzes the reliability and security defined in the ISO/IEC 25010 standard. characteristics of various testing technologies. Finally, this The testing process of the autonomous driving system paper gives suggestions on the development of autonomous includes model testing, simulation testing, cybersecurity driving testing technology. testing, and field testing, as shown in Fig. 1. Model testing, simulation testing and field testing mainly test functional Keywords—autonomous driving, adversarial examples, suitability and reliability of the autonomous driving system, simulation testing, cybersecurity and cybersecurity testing mainly tests security of the autonomous driving system. I. INTRODUCTION In recent years, the pervasive and tremendous breakthroughs of deep neural networks (DNNs) promote the Model testing development of autonomous driving technology. However, sometimes it is difficult to guarantee the reliability of the Simulation testing autonomous driving system. Banerjee et al. investigated the Testing causes of 5,328 failures from autonomous driving systems of Results Cybersecurity Scenario 12 AV manufacturers[1]. As high as 64% of the failures were testing Database found to be caused by the bugs in the machine learning system. The industry and academia have strived to improve the safety Field testing of the autonomous driving system, they have done a lot of research on testing autonomous driving system. This paper summarizes the current research of autonomous driving testing technology from four aspects: Fig. 1. Testing process of autonomous driving systems model testing, simulation testing, cybersecurity testing, and field testing, combined with the system and software quality The testing process of the autonomous driving system is a model defined in the ISO/IEC 25010 standard. Research on continuous cycle. During model testing, the perception model model testing is mainly focused on the adversarial attack. At is tested using test data sets and adversarial examples. Then present, there are already some works that can generate the simulation platform is used to test the autonomous driving adversarial examples for street signs and billboards, leading to system and each control unit. Cybersecurity testing mainly misclassification of the autonomous driving system. In terms focuses on the network security of the autonomous driving of simulation testing, many companies have developed system and its interface. Field testing is to test the autonomous simulation platforms and scenario databases. The research of driving vehicle in a proving-ground. During the test, a cybersecurity testing focuses on the vulnerability detection scenario database composed of environment, traffic and data protection of the terminals and interfaces in the participants, sensors, and other data is generated. The results vehicle network. Field testing mainly focuses on the of tests will also be used to update the scenario database, such construction of a proving ground for autonomous driving. as adding some newly discovered dangerous scenes. This work was funded by Ministry of Science and Technology of the People's Republic of China Program (2018YFB1403405) and Science and Technology Commission of Shanghai Municipality Program (21511101204). Copyright © 2021 for this paper by its authors. Use permitted under Creative Commons License Attribution 4.0 International (CC BY 4.0). III. TESTING TECHNOLOGY FOR AUTONOMOUS DRIVING SYSTEMS A. Testing for autonomous driving models Testing of autonomous driving system models is performed by testing datasets, adversarial examples, and other methods against perception models. Most current perception models are implemented by DNNs, but due to the large dimensions of input data for models, DNNs are vulnerable to adversarial examples, which is a great concern for the prospect Fig. 2. Attacking stop signs with stickers[6] of autonomous driving. Many studies have shown that the object detection model used for autonomous driving can be Aishan Liu et al. proposed PS-GAN, which can generate easily deceived by the existing adversarial attack methods, adversarial patches, innovatively combining GAN network thereby causing errors in autonomous driving decision- and attention mechanism[7]. PS-GAN can capture the making.Current adversarial attack methods in the field of sensitivity of spatial distribution to obtain the optimal attack autonomous driving are mainly divided into attacks on the locations in order to enhance the attack capability of the object detection model and attacks on LIDAR. patches and also ensure a reasonable appearance, as in Fig. 3. In 2017, a team of researchers at Columbia University However, this method does not guarantee that the patch is proposed DeepXplore[2], an automated DNN testing system always on the target object. that uses several existing test inputs as seeds, then modifies the test inputs in a continuous loop iteratively through a gradient ascent method to maximize the difference between the output of the model under test and the output of other similar models. Eventually, this system is able to generate a new set of test data, which can trigger errors in the judgment of the model under test. Experiments on the Udacity challenge dataset showed that the system is capable of making the Fig. 3. The adversarial patches generated by PS-GAN[7] DNN-based autonomous driving systems' prediction of the car's steering angle incorrect by changing the lightness and Zelun Konget al. proposed PhysGAN, an algorithm that darkness of the input images, adding noise, and so on. The generates adversarial examples with physical world resilience same group followed up with DeepTest[3], removing the for autonomous driving systems in a continuous manner[8]. requirement that DeepXplore must provide multiple DNNs Unlike PS-GAN, the input to PhysGAN is a given scene (e.g. with similar functions. DeepTest also makes some changes for the content of a billboard, as in Fig. 4), so the generated the generation of test data for autonomous driving systems, adversarial examples are more realistic. What differs from resulting in a more efficient system for generating data for other adversarial attack methods which aim at detection extreme scenarios. classifiers is that PhysGAN attacks autonomous driving navigation systems, which are regression models. So the DeepXplore and DeepTest can generate a large number of authors use the mean squared error and the maximum error of adversarial examples, but these are very different from real the angle of autonomous driving navigation evaluate the result. scenes, and many extreme scenes such as rainy and foggy days are tough to generate by simple image transformation. DeepRoad[4] uses Generative Adversarial Networks (GAN) to generate more realistic transformed images of rainy and snowy days, compare the steering angle prediction results of the generated images with results of the original images, which can find the scenes that cause errors of DNN in the autonomous driving. Zhou et al. proposed DeepBillboard, which generates real- world adversarial examples of billboards that could trigger steering errors in autonomous driving systems[5]. It Fig. 4. Attacking autonomous driving navigation systems via PhysGAN on the McDonald's ads[8] demonstrates the possibility of generating real physical-world adversarial examples for actual autonomous driving systems. The above studies aim at vision-based autonomous driving Kevin et al. perform a physical-world robustness attack on systems, but LiDAR-Adv[9], a joint study of University of parking road signs (by sticking adversarial patches at specified Michigan, UIUC, and Baidu, breaks through LiDAR systems. locations on the road signs) to misclassify stop signs as other The researchers connected perturbations of a 3D target to a signs of the specified category with 100% probability, as in LiDAR scan (or point cloud) by modeling a differentiable Fig. 2[6]. The authors proposed the RP2 algorithm: Firstly, LiDAR renderer. They then used the differentiable proxy build a model to quantify the target object's physical changes function to produce 3D feature aggregations and designed (including changes in distance, angle, illumination, etc., and different losses to ensure that 3D adversarial examples were transformations such as random cropping of images and smooth. In a specific experiment, the researchers compared a changes in luminance), then construct a mask of the target normal box with a 3D printed adversarial example on the object to discover "vulnerable regions" on which the attack is Apollo Autopilot system. They found that the LIDAR- achieved by masking. equipped car did not detect the target until it approached the adversarial example. In contrast, the car detected the normal Tencent and Baidu have also released their own databases of box at a long distance. autonomous driving scenarios. The University of Toronto, Princeton University in 2) Simulation platform conjunction with Uber also proposed a generic 3D adversarial The autonomous driving simulation platform is a system object generator to fool LIDAR detectors[10]. In particular, that tests autonomous driving functions by simulating traffic the authors placed a generated pseudo-object on top of any scenes, vehicle movements, and sensor signals. Its main target vehicle to completely hide the vehicle, resulting in an functions include restoring static scenes and dynamic scenes, 80% success rate of not being detected by LIDAR detectors, camera and radar simulation, and vehicle dynamics simulation. as shown in Fig. 5. At present, many companies and institutions have developed their autonomous driving simulation platforms. Carla[12] is an open source free autopilot simulator based on unreal engine. It supports flexible configuration of sensors, environmental states, dynamic and static traffic participants and maps, and can control simulated vehicles through Python or C language API. Autoware[13] is an open source software for automatic driving technology research. It includes four modules: localization, detection, prediction and planning, and control. It supports path planning, traffic signal detection, lane detection, virtual reality and other functions. PreScan[14] is a widely used vehicle driving simulation software product of Siemens. It supports the simulation of multiple functions such as camera, radar, LiDAR, GPS, and vehicle-to-vehicle communication, and can simulate simple traffic scenarios. Fig. 5. Placing an antagonistic object on the target vehicle made the SiVIC[15] is similar to PreScan, but it can provide more vehicle "invisible" to LIDAR[10] realistic and complete sensor models. Google developed the simulation platform Carcraft, based on the scenario data B. Simulation testing for autonomous driving systems collected by Waymo, combined with high-precision map information, to realistically simulate the real traffic The simulation test is to test the autonomous driving environment. NVIDIA released the cloud-based NVIDIA system and each control unit through the simulation platform. Drive Constellation simulation system in 2018, which can Currently, about 90% of autonomous driving tests are generate realistic data, create various test environments, completed through simulation platforms. Both Scenario simulate various weather conditions such as rain and snow, database and simulation platform are required for simulation simulate different roads and terrains, and simulate dazzling testing. light during the day and limited vision at night. Microsoft 1) Creating Scenario Database open-sourced the cross-platform Unreal Engine simulator A scenario is the overall description of the autonomous AirSim in 2017, which supports simulations of drones and vehicle and environment components over a period, which is autonomous driving. It can create a highly realistic traffic abundant and complex. The scenario database is a database environment and simulate vehicles and sensors. LG Silicon composed of a series of test scenarios that meet certain test Valley Lab released the open-source autonomous driving requirements. The construction of the scenario database is simulator LGSVL Simulator in early 2019, which supports generally divided into four steps[11]: data collection, data sensor simulation and editable maps, vehicles, weather, traffic cleaning, information labeling, and scenario clustering. The flow, pedestrians, etc. source of scenario data mainly includes real-world driving Tencent released the autonomous driving simulation data and simulation data synthesized from real-world driving platform TAD Sim in 2018, which combines professional data. The test results of the simulation test will also be used to game engines, industrial level vehicle dynamics models, update the scenario database as required. integrated virtual and real traffic flow and other technologies. At present, many auto companies and autonomous driving Baidu's self-developed autonomous driving simulation system solution providers have established their own scenario AADS includes a data-driven traffic flow simulation databases. Waymo collects simulation scenarios based on framework and a scene picture synthesis framework based on field tests. After testing autonomous vehicles on public roads image rendering. Researchers of Jilin University and proving ground, Waymo accumulates thousands of independently developed the PanoSim, a virtual autonomous scenario data, creates virtual scenarios based on these data, driving test platform[16]. They analyzed the driving habits of and produces more scenarios by modifying scenario drivers based on the platform and proposed ADAS control parameters. Waymo has released a database of more than one strategies that consider different driving habits. thousand scenarios. Automotive Data of China Co., Ltd. has C. Cybersecurity testing for autonomous driving systems initially built a simulation testing scenario database that includes nearly 500,000 kilometers of driving data and traffic The autonomous driving system is deployed in the rules, covering important cities such as Beijing, Tianjin, and Intelligent Connected Vehicle(ICV), which can not only assist Shanghai. China Automotive Engineering Research Institute or replace drivers to control a vehicle through advanced Co., Ltd. released the "China Typical Scenario Database onboard sensors, controllers, actuators, and other devices but V2.0" in 2019, including hundreds of standard traffic rules also integrate modern communication and network scenarios, 3,000 empirical scenarios, 50,000 functional technologies to realize Vehicle-to-everything(V2X), complex scenarios, and 150 accident scenarios. Companies such as environmental perception, decision making and cooperative control of multi-vehicle, etc. The vehicle network technology To test the four types of cybersecurity testing objects for provides the possibility that crackers hack into the autonomous driving systems, several researchers not only autonomous driving system or ICV system. Crackers can slow applied existing traditional effective and valuable down, stop the engine, brake, or do other malicious operations cybersecurity testing methods to cloud service platforms and to the vehicles by hacking into the cloud account. They may mobile phone terminals, but also proposed testing methods for also hack into the mobile application to take over the control the items of autonomous driving systems that require special of vehicle such as unlocking and starting engine remotely; attention. Wu Lingyun et al. proposed a random forest-based implant malicious files into the in-vehicle networking (IVN) CAN bus message anomaly detection method model, which through USB storage media to control the vehicle, etc. effectively detects anomalous data on ICV and improves Yoshiyasu Takefuji listed several cybersecurity incidents, vehicle operation security[18]. Mazloom S et al. created a such as white hat hackers stopped the engine of a car on the malicious demo application, loaded on a mobile terminal. It highway through a remote man-in-the-middle attack in 2015; uses the open MirrorLink interface on an IVI to connect to a Ford car's parking assistance module could be forced to mobile phone and discovers a heap overflow vulnerability that intervene the control of steering wheel through the CAN allows an attacker to obtain the control flow of a privileged command named "0x0081"[17]. process executing on the IVI[19]. It will further allow malicious attacks on the controller of the autonomous driving The frequent occurrence of security accidents has system. Testers can use the same principle to test whether the accelerated the in-depth examination of cybersecurity issues dangerous interface of IVI has been closed using the relevant of ICV. The security of autonomous driving systems requires Payload. the support of layered distributed technology, including the security of in-vehicle module systems, network-side D. Field testing for autonomous driving systems interaction and cloud information processing. The security of Field testing is to test the autonomous vehicle on the real- each layer guarantees the security of ICV and connected world road, typically in a proving ground. The autonomous vehicle system, and ensures the safety of the autonomous vehicle must be tested in many scenarios and environments in driving system ultimately. a limited field. Backend The United States and the European Union have built Autonomous driving system cybersecurity Mobile phone terminal Cloud service platform some proving ground for autonomous driving testing. The Smart Road was built in Virginia by renovating part of the Communication Network highway, which is 2.2 miles long and can simulate rainy and testing objects Network Cellular Network foggy weather by spraying water mist. The Mcity proving ground in Michigan contains pavements of different materials, In-vehicle Network (CAN Bus) and is equipped with abundant traffic signs, signal lights, tunnels, and other traffic elements. Google rents the Castle Air In-vehicle terminal Force Base in California to test its autonomous vehicles. There are various streets, highways, traffic lights, traffic Car T-Box IVI ECU GateWay/... roundabouts, etc. inside the proving ground, as well as rainy weather simulators. The AstaZero Proving Ground in Sweden Fig. 6. Objects of autonomous driving system cybersecurity testing includes urban roads, highways, multi-lane parallel road, roundabouts and intersections, and has become a research and According to the structure of vehicle security operation development platform for autonomous driving safety center (VSOC) defined in the white book, "Setting the technology. Shanghai, Hangzhou, Wuhan, Shenzhen and Standard for Connected Cars' Cybersecurity", the objects of some other cities in China also plan to build proving ground testing autonomous driving system cybersecurity can be for autonomous driving testing. classified into cloud service platform, mobile phone terminal, IV. CONCLUSION in-vehicle terminal, and communication network (Fig. 6). Specific testing points for these objects include the following: Testing and verifying the safety of autonomous driving systems is an important prerequisite for running autonomous a) Cloud service platform testing concerns more about the vehicles on the road. The difficulty of testing is increasing as traditional Web vulnerability and the transmission security the level of autonomous driving increases. Currently, the between the cloud and the other two terminals; industry and academia have carried out a lot of research on b) Mobile phone terminal has become the standard testing autonomous driving systems and developed configuration of ICV so that testers should evaluate whether corresponding tools and technologies to test autonomous the communication key and communication protocol between driving models, systems, networks, and vehicles. A lot of the mobile and vehicle terminal can be cracked by achievements have been made in adversarial example technologies, analyze the communication protocol, and use it generation, simulation platform development, network to forge malicious requests for vehicle control; vulnerability analysis and proving ground construction. However, there are still some unresolved problems in c) In-vehicle terminal testing objects include In-Vehicle autonomous driving system testing. For example, the Infotainment (IVI), Telematics-Box (T-Box), sensors, cooperation mechanism of developing the scenario database external interfaces, and other components. Generally, IVI, T- is still inefficient, and the standard of autonomous driving Box and other components contain operating systems, in- system evaluation is not established yet. In the future, it is vehicle APPs, and a large number of third-party libraries; necessary to establish a set of testing standards and tool chains d) Communication networks are tested for authentication, for autonomous driving systems to provide forceful supports transmission encryption, and protocol security. for the development and implementation of autonomous [9] Yulong Cao, Chaowei Xiao, Dawei Yang, Jing Fang, Ruigang Yang, driving technology. Mingyan Liu, and Bo Li. "Adversarial objects against LiDAR-Based autonomous driving systems," arXiv:1907.05418, 2019. REFERENCES [10] Tu J, Ren M, Manivasagam S, et al. "Physically realizable adversarial examples for lidar object detection," Proceedings of the IEEE/CVF [1] S. S. Banerjee, S. Jha, J. Cyriac, Z. T. Kalbarczyk, and R. K.Iyer. Conference on Computer Vision and Pattern Recognition. 2020, pp. "Hands off the wheel in autonomous vehicles?: A systems perspective 13716-13725. on over a million miles of field data," In 2018 48th Annual IEEE/IFIP [11] Zhang, Z., Shi, J., Guo, K., and Wang, J. "Research on construction International Conference on Dependable Systems and Networks (DSN), method and application of autonomous driving test scenario database," 2018, pp. 586–597. CICTP 2020. 2020, pp. 311-323. [2] Kexin Pei, Yinzhi Cao, Junfeng Yang, and Suman Jana. "DeepXplore: [12] Dosovitskiy, A., Ros, G., Codevilla, F., Lopez, A. and Koltun, V.. Automated whitebox testing of deep learning systems," In Proceedings "CARLA: An open urban driving simulator," Proceedings of the 1st of the 26th Symposium on Operating Systems Principles,ACM, 2017, Annual Conference on Robot Learning, in PMLR, 2017, pp. 1-16. pp. 1–18. [13] S. Kato, S. Tokunaga, Y. Maruyama, S. Maeda, M. Hirabayashi, Y. [3] Tian, Y., Pei, K., Jana, S., and Ray, B. "Deeptest: Automated testing of Kitsukawa, A. Monrroy, T. Ando, Y. Fujii, and T. Azumi, "Autoware deep-neural-network-driven autonomous cars," Proceedings of the on board: Enabling autonomous vehicles with embedded systems," In 40th international conference on software engineering. 2018, pp. 303- Proceedings of the 9th ACM/IEEE International Conference on Cyber- 314. Physical Systems (ICCPS2018), 2018, pp. 287-296. [4] Zhang, M., Zhang, Y., Zhang, L., Liu, C., and Khurshid, S. "DeepRoad: [14] Gietelink O J. "Design and validation of advanced driver assistance GAN-based metamorphic testing and input validation framework for systems," 2007. autonomous driving systems," 2018 33rd IEEE/ACM International Conference on Automated Software Engineering (ASE). IEEE, 2018, [15] Gruyer D, Pechberti S, Glaser S. "Development of full speed range pp. 132-142. ACC with SiVIC, a virtual platform for ADAS prototyping, test and evaluation," 2013 IEEE Intelligent Vehicles Symposium (IV). 2013, pp. [5] Zhou H, Li W, Kong Z, et al. "Deepbillboard: Systematic physical- 100-105. world testing of autonomous driving systems," 2020 IEEE/ACM 42nd International Conference on Software Engineering (ICSE). IEEE, 2020, [16] Shanshan Wang. "PanoSim: A new generation of advanced automobile pp. 347-358. intelligent driving simulation system," Review of Science and Technology, 2015, no. 10, pp. 68-71. [6] K. Eykholt et al. "Robust physical-world attacks on deep learning visual classification," 2018 IEEE/CVF Conference on Computer [17] Y. Takefuji, "Connected vehicle security vulnerabilities Vision and Pattern Recognition, 2018, pp. 1625-1634. [Commentary]," in IEEE Technology and Society Magazine, 2018, vol. 37, no. 1, pp. 15-18. [7] Liu, A., Liu, X., Fan, J., Ma, Y., Zhang, A., Xie, H., and Tao, D. "Perceptual-Sensitive GAN for generating adversarial patches," [18] Lingyun W U , Guihe Q , He Y U. "Anomaly detection method for in- Proceedings of the AAAI Conference on Artificial Intelligence, 2019, vehicle CAN bus based on random forest," Journal of Jilin Vol. 33, No. 01, pp. 1028-1035. University(Science Edition), 2018, vol. 56, no. 3, pp. 663-668. [8] Z. Kong, J. Guo, A. Li and C. Liu. "PhysGAN: Generating physical- [19] Mazloom, S., Rezaeirad, M., Hunter, A., and McCoy, D. "A security world-resilient adversarial examples for autonomous driving," 2020 analysis of an in-vehicle infotainment and app platform," In 10th IEEE/CVF Conference on Computer Vision and Pattern Recognition {USENIX} Workshop on Offensive Technologies ({WOOT} 16), (CVPR), 2020, pp. 14242-14251. 2016.