Recently technological advancements in the automobile and transportation sector have gained significant interest from governments, industry leaders, and citizens. Together with Autonomous Vehicles (AV) and Connected Vehicles (CV), Connected-Autonomous Vehicles (CAV) have made a revolution in these sectors. Emergency Vehicles (EVs), such as ambulances, fire trucks, and patrol cars, are essential to our daily trafic life. Each of EVs has a diferent purpose, but all have their urgency and importance, and any time passing may cause the death of life. Thus, whenever other vehicle drivers encounter an EV on the road, they must yield to the EVs. Therefore, a CAV system that can detect EVs will significantly improve these issues. According to the Society of Automotive Engineers International (SAE), in today's autonomous vehicles, most of them are less than Level 5, and car manufacturers assume the driver will take back control. Still, most autonomous vehicles mainly rely on their vision sensor instead of their sound sensor. Thus, when the system notifies the driver that the EVs are already close to them, it may be dangerous for the driver, pedestrians, and passengers in the vehicle. This paper proposes a conceptual framework and discusses a related methodology to support such a real-time emergency response model for CAV. connected-autonomous vehicle, emergency vehicle, emergency response, real-time response, driving assistance technology, the Doppler Efect, exceptional handling, control strategy, machine learning from governments, industry leaders, and citizens. Au- (SAV) should be considered at least Level 2 of vehicle ∗Corresponding author.
and transportation sector have gained significant interest
tonomous Vehicles (AV) technology enables vehicles to be
controlled by precise, fast responding computers instead
of error-prone and slowly responding human beings;
Connected Vehicles (CV) allow infrastructure units and
vehicles to share high-resolution information through
wireless connectivity that can communicate to support
interaction with their internal and external environments
in real-time, e.g., for trafic systems and between
indiwhich integrates the best of both AVs and CVs, have
revolutionized these sectors [
According to the report of the Society of Automotive
THECOG 2022: Transforms in behavioral and afective computing,
citizens by ofering them better and more efective
transportation services, such as dramatically reducing car
crashes and driver fatigue [
Many of today’s automated vehicles lose track of the Control Units (ECUs) with relevant standards or codes
lane position when the lane markings are absent. For ex- such as the National Electrical Code/National Fire
Proample, erroneous lane marking recognition contributed tection Association (NFPA) 70, Canadian Electrical Code
to a fatal crash of Tesla cars in California in 2018 [
On the one hand, these connections are necessary to The remainder of the paper is organized as follows: provide basic (e.g., driving) and advanced features (e.g., Section 2 reviews the related scientific works present in autonomous driving and entertainment) to involved per- the literature in this work’s area. Section 3 presents a sons, such as drivers, passengers, and pedestrians. proposed conceptual cooperative framework and method
The application of CAV in real-time response for Emer- ology for real-time emergency response for Connectedgency Vehicles (EVs) has resulted in great improvements Autonomous Vehicles. Section 4 concludes the paper, in the eficiency of the process. EVs, such as ambulances, presenting a brief discussion and the found limitations. ifre trucks, and police cars, are essential to our daily trafifc life. Each EV has a diferent purpose, but all have their urgency and importance for emergency response and 2. Literature Review saving a life. Thus, whenever a vehicle driver encounters an EV on the road, the driver must yield to the EV in a Safety risks may bring implications for CAV passengers, safe condition. other vehicles, pedestrians, and road infrastructure to
Usually, the EV has vision and audio devices to remind understand human aspects and perceptions towards CAV,
drivers and pedestrians of their existence, but these de- such as trust [11, 12], driving style [13], and physical
vices may sometimes be less efective in a noisy surround- safety of pedestrians and city infrastructure [14]. These
ing environment [
States [
This paper aims to study a real-time emergency re- environments or even disaster areas, and intersections sponse model for CAV (SAE Level of autonomy 3), specif- are the most frequent places for EVs to be involved in ically for EVs, using a hybrid approach embedding either car accidents [15]. In the physical feature, fire trucks and vision and sounds sensors, aiming to detect and localize ambulances have a larger volume and are more likely to the EVs by the vision and siren detection system, which cause danger when they are driven together with other will be discussed and analyzed the increase of accuracy vehicles. Summing up the above factors, we can underof distance and identification measures to ensure sustain- stand the threat of encountering EVs on the road. Their able and safe operation during normal and emergency task nature is more dangerous than general road driving, conditions and consideration of current related codes and especially since most vehicle conflicts occur at intersecstandards environment. tions. Therefore, if the trafic signal light can respond to
The paper will survey and evaluate CAV based on a the situation on the road and save time on the ambulance, it can also reduce the risk in the intersections. According station, for example. In the context of smart technolto the task of EVs, there is usually time urgency, so for the ogy, the interface may have access to the fridge or food surrounding vehicles, the best way to respond is to slow storage information to add a stop at the supermarket or down and stop as soon as possible so that emergency grocery store so that the human can purchase supplies. vehicles do not have to be distracted by other vehicles. These contextual GPS scenarios can ofer more efective However, even the Intelligent Transport Systems (ITS), itineraries for the drivers. They can include everything which include the protocols of communications between from picking up colleagues to sharing a ride for work to CAVs and the intelligent trafic, could be compromised by syncing the driver’s agenda or adapting routes to trafic cyber-attacks becoming susceptible to safety risks [16]. information, among other individual behaviors.
Advancements in the CAV industry also open oppor- In previous research, the methods for detecting sirens tunities for creating a new profile of drivers. Among the from EVs can be divided into two diferent approaches. most promising approaches, CAV is the first alternative The first approach is to identify whether the data confor independent visually impaired drivers [17]. Accord- tains siren sound based on the siren’s characteristics, ing to the World Health Organization (WHO), more than such as high-frequency and low-frequency, or cyclical 1 billion people live with some visual impairment in the nature [26, 27]. However, this method does not perform world [18]. WHO shows that 36 million of them are blind, well in noisy environments, especially in urban areas. and the majority of those people are over 50 years old. In- The second approach is to extract the siren signal from deed, population aging is a worldwide phenomenon that background noise [28]. For example, Fazenda et al. emis expected to bring economic consequences [19]. Build- ployed the least mean squared algorithm to create a noise ing CAV that the elderly population can use can help the canceller to extract the target signals [29]. Nishimura et industry to overcome the economic challenge. However, al. proposed a data embedding method for the vehicle to this end, accessibility of CAV becomes imperative for location into the siren sound [30]. This research tends the sector [17]. Besides the elderly and people with visual to extract the siren signals from background noise, inimpairments, advances in CAV also open opportunities cluding lots of parts in real trafic life, by considering to enhance children’s transportation. For example, au- the Doppler Efect. The Doppler Efect is that the sound tonomous school buses may benefit their independent frequency will vary according to diferent speeds, so the transportation, or, even, parents may use private CAV to siren frequency in real life may difer from the spectrum drive their kids to school. we observe. Referring to the siren datasets we collected
Driving essentials scenarios include, but are not lim- from the Web, Schröder et al. showed that some audio ited to, studies on Computer Vision (CV) to enhance CAV software could help to mimic the Doppler Efect in the capabilities of (partial or complete) self-driving, Artificial datasets, such as Adobe Audition 1.0 [31]. Intelligence (AI), Cloud computing, and machine learn- Our hybrid approach aims to localize the siren by ing, among other computational domains concerned to a time delay estimation method and a sound intensity enhance essential driving functionalities [20, 21, 22, 23]. probe method in the Path Planning function. For examEssential functionalities cover GPS services (to allow au- ple, Fazenda et al. showed that the accuracy of the time tonomous driving) and a range of sensing technologies delay estimation method is better in the distance between suitable for driver’s and passenger identification, includ- the emergency vehicle and the driver in a long distance. ing vehicle and road infrastructure detection and parking But if it is in a short distance, the sound intensity probe assistance. This scenario involves Computer Vision tech- method can also get a higher accuracy [29]. niques and route generation, which come from Computer Diferent projects have made great eforts to advance Science and Automation Engineering backgrounds. An- in the CAV areas. Big technology and automotive comother technology, such as the blockchain, has already panies and universities have been working together to been embedded in CAV, considering its eficient perfor- advance the projects, designing vehicles and developing mance mechanisms for decentralized distributed storage diferent algorithms and drive systems split into diferent and security management of big data [24, 25]. levels of autonomy.
The main stakeholders in this real-time emergency Uber, in partnership with Carnegie Mellon Univerresponse scenario are the automobile sector, the mobile sity; Lift together with General Motors; and Didi, with application market, and average drivers and passengers. Japanese automotive companies, have been building selfAn interface for this scenario can assist drivers in se- driving cars and ride-sharing services with level 4 and lecting and monitoring their driving routes, including planning to build level 5 of autonomy. Uber, Lift and contextual stops for either safety or personal matters. Didi are companies that provide mobility services and The emergency response model component may collect have a great power of trafic data collection, which is CAV’s contextual information, like the vehicle’s fuel or the key to developing and improving their automation another engine status. Then, it uses such information to system and models [32]. AutoX, in 2018, built an addetermine if the GPS route must add a stop at the gas vanced full-stack self-driving AI platform in partnership
with Alibaba Group, Chery automotive, NVIDIA, and other companies to build SAE Level 2 and 3 assistivedriving vehicles, including RoboTaxis and RoboTrucks.
The companies’ idea for the next decade is to develop a CAVs are under various scenarios and operating condifull (Level 5) autonomous car, beginning in 2021 with the tions of residential, industrial facilities, transportation ifrst Fully Driverless RoboTaxi Service to The Public in electrification, and grid-connected integration. ThereChina [32, 33]. fore, it should provide a comprehensive evaluation of the
Google Waymo, in partners with automotive com- safety risks during the operation of CAV by identifying
panies such as Fiat-Chrysler, Audi, Toyota, and Jaguar, hazards and estimating risks in diferent operating
condihas been working on self-driving vehicles of autonomy tions and modes. Codes and standards roadmap should
level 4, operating the Waymo Driver, a commercial au- be performed based on fault propagation modeling and
tonomous ride-hailing service, in San Francisco, Cali- analysis, simulation and evaluation will be presented and
fornia. Recently in 2020, Waymo started the operation analyzed by independent protection layers for all
possiof Waymo Via, transporting commercial goods that use ble normal and abnormal operating conditions. Besides,
autonomous vans and trucks [
To enhance situational awareness in CAV, our pro- according to their ability to transition with city speed
posed model will also incorporate the research work in restrictions during an emergency.
computer vision tools such as geolocalized photos and During autonomous driving, one of the most
dangervideos of the situations into the proposed model [
This research will consider the operation of CAVs that vehicle’s ability to adapt to road circumstances.
Accordface some technical challenges, such as the instability in ing to recent literature, the Path Planning component of
some operating conditions due to the dynamic response CAV comprises three functions: Mission Planning,
Beof emergency trafic scenarios on a real-time basis. Most havioral Planning and Motion Planning. A typical task
of the CAV mainly rely on their vision sensor instead of each function is outlined as follows: (1) The Mission
of their sound sensor. One of the related research areas Planner: High-level decisions such as determining pickup
is the hearing impaired [
In Figure 1, Our proposed EVs identification process will start by detecting the approaching vehicle based on visual and sound sensors. There are usually two ways to detect EVs. One is to see their unique appearance, and the other is to hear their siren sound. In the siren sound detection, we first identify which type of emergency vehicle it belongs to and extract the specific siren sound from the background noise. After we extracted the siren sound, we used the time delay estimation method and sound intensity probe method to localize the direction of the sound. From the previous research [29], the time delay estimation method has better performance in long distances than the sound intensity probe method. Thus, we can decide the direction of the siren sound detection.
gency vehicle and use the image we captured and the distance we detect from Light Detection and Ranging (LIDAR) to localize the direction and the position of the EVs. Combine the above information and use the GPS to help predict the possible path of EVs and provide driver information so that appropriate responses can be made as soon as possible. If our route is interleaved with the route of EVs, we will respond. If there is no conflict, we will continue to observe but do not require a response. Basically, we hope we can detect both sound and visual detection to give complete information to the driver. If we can not detect the precise position, our system can probably predict the possible position to give the most appropriate response.
algorithm. Table 1 is a brief conceptual introduction. When we detect approaching EVs, we first evaluate our speed. If our speed is higher than a certain speed, we gradually decrease the speed to maintain safety. Then when the position of EVs has been confirmed, the system starts to perform the lane change to yield to EVs. The method of changing lanes will first detect vehicles in the vicinity, according to LIDAR, which can detect vehicles within a radius of 100 meters.
Result: def yieldToEVs(): Matrix = detectSurroundVehicle(); # By Using LIDAR to detect vehicles changeLane(Matrix); # change lane according to the vehicle matrix while Emergency Vechicles is detected do if speed is above certain speed then slowDown(); # slow down to certain speed if EVs’ location is confirmed then yieldToEVs(); vehicleStop(); end end end Algorithm 1: Algorithm of Emergency Vehicles Response
vehicle matrix according to our lane and then plan how
to yield EVs based on the vehicle matrix. In Canada, the
Ministry of Transportation [
create a comprehensive safety standard and system. Figure 2 is our conceptual cooperative CAV system; all the CAVs can do collaborative sense and computation. In addition to the communication between vehicle and vehicle (V2V), the communication between the vehicle and the trafic lights (V2I) can also significantly save time for the EVs to reach the destination. Usually, the biggest problem encountered by EVs is the trafic jam on the road or the danger encountered when executing ‘code 3 running’, on the way to an emergency. Our EVs response ADAS can collaborate with other ADAS, such as Adaptive Cruise Control, Autonomous Emergency Braking, and Lane Change Assistant, to give corresponding responses automatically. As for the interaction between 4. Conclusion humans and vehicles, in the process of automated response, humans need to supervise and be able to inter- The operation of CAVs faces some technical challenges, vene. For example, in Tesla’s autopilot system, people such as the instability in some operating conditions due need to put their hands on the steering wheel to ensure to the dynamic response of trafic scenarios, such as emersafety when performing lane changes. Accidents caused gency vehicle response. Therefore, the SAE classifies the by automated failures in the response of EVs can seri- vehicle from Level 0 to 5 based on the automation capaously afect lives. At present, a few fatal vehicle accidents bilities. In today’s CAV, most of them are in Level 3, and occur on autonomous vehicles. Therefore, before we can car manufacturers assume the driver will take back confully guarantee the driving safety of autonomous vehi- trol during the emergency in real time. However, drivers cles, appropriate human supervision and intervention should stay aware of automation limitations, and the manare necessary. ufacturers should make a warning system that can give
ADAS can outperform humans in some automated op- warnings far ahead of time. Nowadays, most CAV mainly
erations, thus promoting trafic safety and resulting in rely on their vision sensor instead of their sound sensor.
the development of autonomous vehicles. To achieve Our hybrid approach aims to detect and localize the EVs
the driving safety of autonomous cars, some standard- by the vision and siren detection system. We believe
ized methodologies were created. Projects such as the that detecting the approaching EVs using both vision
Waymo Driver and AutoX developed the ADAS of their and sound sensors are essential in CAV, increasing the
Self-driving cars using the ISO 26262 “Road Vehicles - accuracy of distance and identification measures. The
reFunctional Safety,” which presents guidelines applied to search direction covers codes and standards for all control
safety-related systems that include one or more electri- and communication functions in related ECUs through
cal and/or electronic systems in automobiles. Due to the operating process of CAV, which should be tested for
the necessity of dynamically and automatically driving accuracy and strength to support the requirements of the
awareness, with lanes and vehicle detection, as well as design, development, operation, and evaluation of the
collision avoidance, the CAV systems have continuously model. The requirements can fit together with standards
improved their automobile intelligence and connectivity (e.g., the working document ISO 21434 “Road Vehicles –
capabilities, increasing the focus on cybersecurity. Stan- Cyber Security Engineering”) and functional safety
redardized methods of cybersecurity, such as the ISO 21434 quirements (e.g., ISO 26262 “Road vehicles - Functional
“Road Vehicles - Cyber Security Engineering,” have been safety”) within the North American regulatory structure
adopted, which specifies requirements for the whole life and utility requirements.
cycle of automotive products of engineering-related
cybersecurity risk management for road vehicle electrical
and electronic systems and their components and inter- Acknowledgments
faces [
Our system development will comply with ISO 26262 Zayed University, United Arab Emirates. and ISO 21434. The software development life cycle in ISO 26262, from design to implantation and validation, will be followed. In addition to software development, appropriate information management, including process [9] Road vehicles - Functional safety, Technical Report and transportation engineering (English edition) 6 ISO Standard No.26262:2018, International Organi- (2019) 109–131.
zation for Standardization, 2018. [22] H. Gao, B. Cheng, J. Wang, K. Li, J. Zhao, D. Li, Ob[10] Road vehicles Cybersecurity system evaluation ject classification using cnn-based fusion of vision method, Technical Report ISO Standard No. and lidar in autonomous vehicle environment, IEEE 21434:2021, International Organization for Stan- Transactions on Industrial Informatics 14 (2018) dardization, 2021. 4224–4231. [11] H. Abraham, C. Lee, S. Brady, C. Fitzgerald, [23] M. R. T. Hossai, M. A. Shahjalal, N. F. Nuri, Design B. Mehler, B. Reimer, J. F. Coughlin, Autonomous of an iot based autonomous vehicle with the aid vehicles, trust, and driving alternatives: A survey of of computer vision, in: 2017 International Conferconsumer preferences, Massachusetts Inst. Technol, ence on Electrical, Computer and Communication AgeLab, Cambridge 1 (2016) 2018–12. Engineering (ECCE), IEEE, 2017, pp. 752–756. [12] K. Lazanyi, G. Maraczi, Dispositional trust—do we [24] G. Drakopoulos, E. Kafeza, H. Al Katheeri, Proof trust autonomous cars?, in: 2017 IEEE 15th In- systems in blockchains: A survey, in: 2019 4th ternational Symposium on Intelligent Systems and South-East Europe Design Automation, Computer Informatics (SISY), IEEE, 2017, pp. 000135–000140. Engineering, Computer Networks and Social Media [13] C. Basu, Q. Yang, D. Hungerman, M. Singhal, A. D. Conference (SEEDA-CECNSM), IEEE, 2019, pp. 1–6.
Dragan, Do you want your autonomous car to drive [25] T. Jiang, H. Fang, H. Wang, Blockchain-based interlike you?, in: Proceedings of the 2017 ACM/IEEE net of vehicles: Distributed network architecture International Conference on Human-Robot Interac- and performance analysis, IEEE Internet of Things tion, 2017, pp. 417–425. Journal 6 (2018) 4640–4649. [14] A. Millard-Ball, Pedestrians, autonomous vehicles, [26] S. Kiran, M. Supriya, Siren detection and driver and cities, Journal of planning education and re- assistance using modified minimum mean square search 38 (2018) 6–12. error method, in: 2017 International Conference [15] H. Hsiao, J. Chang, P. Simeonov, Preventing emer- On Smart Technologies For Smart Nation (Smartgency vehicle crashes: status and challenges of TechCon), IEEE, 2017, pp. 127–131. human factors issues, Human factors 60 (2018) [27] J.-J. Liaw, W.-S. Wang, H.-C. Chu, M.-S. Huang, C.1048–1072. P. Lu, Recognition of the ambulance siren sound [16] B. B. Gupta, A. Gaurav, E. C. Marín, W. Alhalabi, in taiwan by the longest common subsequence, in: Novel graph-based machine learning technique to 2013 IEEE International Conference on Systems, secure smart vehicles in intelligent transportation Man, and Cybernetics, IEEE, 2013, pp. 3825–3828. systems, IEEE Transactions on Intelligent Trans- [28] L. Marchegiani, P. Newman, Listening for sirens: portation Systems (2022). Locating and classifying acoustic alarms in city [17] M. Bonani, R. Oliveira, F. Correia, A. Rodrigues, scenes, IEEE Transactions on Intelligent TransT. Guerreiro, A. Paiva, What my eyes can’t see, a portation Systems (2022). robot can show me: Exploring the collaboration [29] B. Fazenda, H. Atmoko, F. Gu, L. Guan, A. Ball, between blind people and robots, in: Proceedings Acoustic based safety emergency vehicle detection of the 20th International ACM SIGACCESS Con- for intelligent transport systems, in: 2009 ICCASference on Computers and Accessibility, 2018, pp. SICE, IEEE, 2009, pp. 4250–4255.
15–27. [30] A. Nishimura, Encoding data by frequency modu[18] World Health Organization, Blindness lation of a high-low siren emitted by an emergency and vision impairment, 2021. URL: https: vehicle, in: 2014 Tenth International Conference //www.who.int/news-room/fact-sheets/detail/ on Intelligent Information Hiding and Multimedia blindness-and-visualimpairment. Signal Processing, IEEE, 2014, pp. 255–259. [19] G. Carbonaro, E. Leanza, P. McCann, F. Medda, De- [31] J. Schröder, S. Goetze, V. Grützmacher, J. Anemüller, mographic decline, population aging, and modern Automatic acoustic siren detection in trafic noise ifnancial approaches to urban policy, International by part-based models, in: 2013 IEEE International Regional Science Review 41 (2018) 210–232. Conference on Acoustics, Speech and Signal Pro[20] A. Dominguez-Sanchez, M. Cazorla, S. Orts- cessing, IEEE, 2013, pp. 493–497.
Escolano, Pedestrian movement direction recog- [32] C. Badue, R. Guidolini, R. V. Carneiro, P. Azevedo, nition using convolutional neural networks, IEEE V. B. Cardoso, A. Forechi, L. Jesus, R. Berriel, T. M. transactions on intelligent transportation systems Paixao, F. Mutz, et al., Self-driving cars: A sur18 (2017) 3540–3548. vey, Expert Systems with Applications 165 (2021) [21] D. Elliott, W. Keen, L. Miao, Recent advances in 113816.
connected and automated vehicles, journal of trafic [33] AutoX, The autox safety factor technical re