Benefits and Challenges of Smart Highways for the User

Abstract

In order to improve the user experience on the highway, a smart highway sensor system can be connected to the car and other devices. It can provide the driver with the fastest and safest journey on the highway by improved route planning. This paper discusses the potential benefits and challenges of the use of a connected highway sensor system. The provided information is based on a focus group of experts (N=9) in the scope of the project Providentia. Within this project, a sensor system is built on the highway to track trafic objects. Diferent user groups that potentially benefit of a smart highway system are identified: Drivers of autonomous and semi-autonomous vehicles, highway operators and teleoperators of autonomous cars. The main benefit for users is the redundant sensor system that provides a far-reaching view. This additional information about the highway situation enables improved route planning for connected vehicles.

CCS CONCEPTS

• Computing methodologies → Modeling and simulation; • Humancentered computing → Visualization systems and tools; • Hardware → Emerging technologies.

INTRODUCTION

Smart Infrastructure connects trafic lights with cars, for example, to provide users with infrastructure information such as a red trafic light. On the highway, sensors can collect information about the vehicles or transmit information about the highway to the user. One example is a smart highway system that collects information about road usage to power on or power of lighting on the highway [ 14 ]. Within the project Providentia [ 7 ] a sensor system is built to observe the trafic on the highway. One measurement point consists of four radars and four cameras to cover far and near range in both directions (see Figure 1 and Figure 2). Within Providentia a distance of approximately 2.2 kilometers of a highway is covered. As a result, the vehicles’ velocities and positions can be tracked. IUI Workshops’19, March 20, 2019, Los Angeles, USA Copyright © 2019 for the individual papers by the papers’ authors. Copying permitted for private and academic purposes. This volume is published and copyrighted by its editors.

This information can be used in diferent ways to benefit the driver. Many applications are not immediately apparent to the driver driving on the highway. As an indirect beneficiary e.g. the highly automated vehicle of the driver uses the infrastructure information to improve the driving performance. By transmitting information from the infrastructure to the vehicle the sensor data of the vehicle can be improved. The highly automated vehicle itself can then send information to the infrastructure which adds to the data of infrastructure. A direct benefit would occur in case of the driver informing him/herself of the route by infrastructure sensors. A ifrst study was conducted to assess the improvement of the driving performance by informing the driver about the highway situation in a dangerous scenario [ 16 ]. In Figure 3 the driver of the car gets this visualization of the highway in an augmented representation within the cockpit. In both cases, the infrastructure data extends the knowledge of the highway situation. If a person is sitting in an autonomous car, this knowledge can be used to validate the sensor information of the autonomous car. In a manually steered car, the information can be used to warn the driver of trafic situations on the highway. One trafic situation is a fast braking car further ahead. If this car brakes right in front of you the vehicle sensors would be able to detect the reduction of the velocity. In case of a braking car three cars ahead of the driver, the scenario would just be detectable by connected vehicles or infrastructure sensors. Then the sensors need to transmit this information in order to shorten the reaction and thus braking time of the driver. In this scenario, the trafic management can prevent a braking cascade by informing all vehicles and drivers of this scenario. If there is bad weather on the highway the driver can be warned of an accident that is not yet well visible. These examples show the direct benefits of this system for the driver. Nonetheless, you need an intelligent interface to warn the driver and adjust the route or driving behavior of the vehicle accordingly. Within the project Providentia a smart voice assistant is implemented so people can already inform themselves about the test bed. With that you can, for example, ask, how many vehicles are on the route and which lane is best in order to get fastest to the destination. This smart infrastructure holds challenges like the construction and the costs thereof or the architecture of those systems. Nevertheless, there are several benefits that might not be directly apparent to the user of smart infrastructure. 2 In the following the benefits and possible use cases of a sensor system on the highway, as identified in a focus groups (N = 9), is presented. The focus group consists of sensor, infrastructure, and data fusion experts. A first brainstorming session was conducted by all participants in order to identify user groups and use cases of the infrastructure system. This brainstorming session was realized by asking the participants of use cases and user groups that potentially benefit from a highway sensor system. In a second session, the participants were divided into three groups. Each group was told to brainstorm use cases for diferent user groups. One group, for example, thought of use cases for teleoperated drivers, one about the driver and co-driver of a manually driven, highly automated an autonomous vehicle (SAE Level 2-5 [ 8 ]) and one about operators of the highway. 3

BENEFITS OF SMART INFRASTRUCTURE

FOR THE USER

The benefits that arise of the use of the smart infrastructure can be provided to diferent user groups of the system. Those are: Drivers of manual, semi-autonomous and autonomous cars (SAE Level 2-5 [ 8 ]), the operator of the highway and a teleoperator of an autonomous car. 3.1

Driver of manual, semi-autonomous and autonomous cars

Manually driven vehicles do not have much sensor information but can benefit from infrastructure information by brought-in devices. In case of the driver using his/her phone for navigational information, infrastructure information can be provided to the driver. Warnings of dangerous trafic situations or route information can be transmitted. The driver of a semi-autonomous car can be warned of diferent trafic scenarios that can be classified by the information provided by the sensor system. Following incomplete list includes some examples of trafic scenarios: • Standing Vehicle

In case of a vehicle that breaks down on the highway, the other cars on the highway can be warned that a vehicle stands on the highway. • Ghost driver

In case of a driver that mistakenly enters the highway in the wrong way, the sensor system can identify this vehicle and warn other vehicles on the highway of the ghost driver. • Bad weather

During bad weather, the orientation on the highway might be dificult for the driver. In this case, the surrounding trafic and the distance between vehicles can be provided to the user. By combining sensor data from the vehicles on the road and sensor data from Providentia, the accuracy of detected objects could be increased. The cameras that use a deep learning based object detection approach and the radars that provide object detection fuse the data in a data fusion unit. By combining the strengths of the sensors (cameras are good classification sensors, radars determine velocities and angles) a more robust system that covers all weather conditions is used than the sensor systems of single vehicles [ 7 ]. • Early Warning

In case of a highway sensor system that is distributed along several kilometers of the highway the driver can be warned early on of possible accidents or dangerous situations along the route [ 16 ]. The infrastructure system has the advantage of having an overall overview of accurate information about the trafic on the highway. Today’s advanced driver assistance systems (ADAS) detect objects and trafic situations in close proximity and warn the driver of braking situations in close proximity of the ego vehicle. Providentia though is able to detect braking scenarios that lead to brake cascades. A brake cascade originates from a fast braking car and can lead to trafic jams and accidents of following vehicles. An early warning of fast braking cars can, therefore, result in smoother trafic flow and a faster and safer journey.

The driver of an autonomous car benefits from smart infrastructure rather indirectly as he/she does not distinguish between the sources that the vehicle needs to drive. The sensor system on the highway is a redundant sensor system that can be used by the algorithm of the autonomous car to validate its own sensor information or extend the sensor information of the car. Autonomous trucks that are connected to other trucks via sensors can drive in a platoon on the highway. Infrastructure sensors provide a way to add sensor information and control values to those platoons. By optimized trafifc flow, those platoons can get information about the route ahead to improve their route planning. Current projects that research vehicle platooning have the goal to maintain a fixed gap between vehicles or perform evasive maneuvers such as emergency braking [ 1 ]. 3.2

Operator of the highway

The operator of the highway needs to maintain the highway, know about damages and accidents in order to redirect the tracfi flow. The additional information collected by smart infrastructure can be transmitted to the operator in order to improve trafic planning. Emergency vehicles can be supervised from far in order to find the best and fastest way to an accident. Currently, the trafic on the highway is directed manually by changing speed limitations or by indicating that the emergency lane can be used by vehicles. Trafic flow management could be enabled by smart infrastructure by predicting the vehicles trajectories and behavior. By providing the operator with an accurate trafic density on the road, the average speed of the vehicles and predicted maneuvers part of the trafic lfow control could be automated. The attention of the driver can be directed to critical situations on the highway and as a consequence make the reaction time faster. 3.3

Teleoperator of an autonomous car

Autonomous cars will likely face limits in operability in certain situations in which their sensors break down or there is not enough environment information to securely operate the car. In such situations, a teleoperator might be able to steer the car to its destination [ 15 ]. By providing the teleoperator information about the surroundings of the car he/she is able to steer the car from the distance for example by joystick [ 12 ]. Even if the sensors of the vehicle itself do not work, the surroundings of the car are monitored by infrastructure sensors and therefore do not face the same limitations as the car sensors. Through this redundant interface, the lost information of the car can be replaced. The visualization of the highway prevents situations in which the occupants of the car are not able to continue their journey. 4

INTERFACE DESIGN

In an interface that provides the user with information about the infrastructure, more information can be communicated. Therefore the existing design spaces for driver-based automotive user interfaces [ 10 ] and windshield applications [ 6 ] need to be extended to consider design dimensions covering use cases of infrastructure information. 4.1

Driver of a semi-autonomous and autonomous car

Even though design spaces for windshield applications and driverbased automotive user interfaces exist [ 10 ][ 6 ], there is not yet a design space for a highly automated vehicles. To put an emphasis on infrastructure interaction, the design dimensions need to be adjusted according to the input of the infrastructure information. As in theory the information of the whole length of the highway can be detected, scenario information also needs to be accompanied by distance information. If the scenario is close by, the notification modality needs to difer from the modality of notifications of scenarios in a far distance. Otherwise, the occupant of the vehicle can not estimate if immediate action needs to be taken or if the scenario might change over time. Predicting whether a scenario will change in time for the vehicle to arrive at that predicted point on the highway could help the driver make an informed decision about the route to take. In case of an accident, the algorithm could calculate, that it might take some time to clear the highway. Therefore the prediction could be very certain that the driver will lose time on the route. In case of fog in the morning in the far distance the prediction could communicate to the driver that the fog will probably lift itself before the vehicle arrives at the scene. It is imaginable that the user interface needs to inform the driver of out-of-view scenarios on the highway in case of more infrastructure information. 4.2 Operators of highways observe the highway and control it in a limited way. The velocity of the trafic can sometimes be regulated and the emergency lane can be blocked or authorized for trafic 1. The design of control rooms consists of several working desks that have screens to show control applications or the videos of the highway [ 9 ] [ 11 ]. The challenges of large screen applications are the loss of orientation on large screens. Looking for the mouse cursor on large displays creates high physical demand. One possibility of improving control operators input techniques is eye tracking as suggested by Lischke et al. [ 11 ]. Some control rooms, such as nuclear power industries, have a strong focus on safety and performance of operators [ 13 ]. Therefore the control rooms need to be designed in a way that operators have no spatial constraint and have fast interaction possibilities. In the automotive domain especially tunnels need to be observed and in case of an emergency need to be closed for incoming trafic right away. Therefore the operator needs the information of dangerous scenarios on the highway right away. Then emergency vehicles can be informed and trafic can be controlled. 4.3

Teleoperator of an autonomous car

Teleoperation is needed in several contexts, e.g. drone control, robot control in space operations or medicine. In automotive vehicles, the design of teleoperation interfaces is under research [ 4 ]. Teleoperated driving requires a network that has a high uplink data rate and a much lower downlink data rate according to Boban et al. [ 3 ]. To get an experience similar to that of a regular driver of the car, several sensors (two or more cameras and other sensor information) need to transmit their information to the teleoperator interface. Berggren et al. test a teleoperated bus on a test bed [ 2 ]. To transmit the environment information and steering relevant information of the bus to the teleoperator, a driving simulator interface is used. The operator sees a camera image of the cockpit of the bus on a screen in front of him/her. The input control can be manipulated by a steering wheel that is handled by the remote operator. The bus speeds up to 20 km/h. In another study by Georg et al. [ 5 ] teleoperated driving with head-mounted displays is compared with teleoperated driving with conventional computer screens. Even though they did not find significant diferences between the two output modalities, the participants thought the top down view of the vehicle surroundings was helpful. With Providentia this topdown view and all other visualization angles could be realized in teleoperated driving. Situation awareness would then increase if the whole environment would be visualized. 5

CHALLENGES

Even though the benefits of the system promise great potential in the future of autonomous driving there are some challenges 1http://www.stmb.bayern.de/vum/strasse/verkehrsmanagement/verkehrssteuerung/index.php, Accessed: 2019-02-13 that need to be addressed. In order to use the system as redundant sensor system for a teleoperator, the system needs to be spread to the whole length of the considered highway. Teleoperated driving is preferably latency free, therefore a minimal delay between the steering decision of the teleoperator and the movement of the car would be desirable [ 5 ]. The user might not have transparency of the used system that is in control of his/her vehicle. In order to regain control of the vehicle this information might be necessary though. Therefore, a clear and transparent communication of the used systems is necessary to gain trust of the user. In order to prevent others to harmfully interact with the system, safety is also an important topic that needs to be addressed. If wrongful information gets distributed the algorithm of the car might make a decision that might result in an accident or undesired behavior. Therefore, the accuracy of the sensors and the classified and identified vehicles on the road must be high enough in order to make accurate steering decisions. 6

CONCLUSION

Smart infrastructure provides great potential for users of the road. Facing new technology such as autonomous cars, connected cars, brought-in devices or platooning the users of the highway will have diferent levels of autonomy. Therefore a perfectly controlled trafic flow is not possible. By providing all users with information independent of their vehicles’ abilities, a shared knowledge base is created. This knowledge base can be retrieved by the system itself but also by the driver on the highway. A brought-in device such as a phone or tablet, for example, can retrieve information like the accurate trafic situation on the highway. This provides the user of the infrastructure system with more trafic information in order to improve the safety of the drivers on the road. 7

ACKNOWLEDGEMENT

This research has been funded by the Federal Ministry of Transport and Digital Infrastructure in the project Providentia. I am particularly grateful for the assistance given by the Providentia Team, namely Vincent Aravantinos, Maida Bakovic, Markus Bonk, Martin Büchel, Dhiraj Gulati, Gereon Hinz, Annkathrin Krämmer, Juri Kuhn, Venkat Lakshmi, Daniel Malovetz, Philipp Quentin, Maximilian Schnettler and Christoph Schöller and their contribution to this paper.

References

[1] Carl Bergenhem , Steven Shladover, Erik Coelingh, Christofer Englund, and Sadayuki Tsugawa . 2012 . Overview of platooning systems . In Proceedings of the 19th ITS World Congress, Oct 22-26 , Vienna, Austria ( 2012 ).

[2] Viktor Berggren , Aneta Vulgarakis, Athanasios Karapantelakis, Elena Fersman, Keven Wang, Leonid Mokrushin, Nicolas Schrammar, and Rafia Inam . 2019 . 5G teleoperated vehicles for future public transport . https://www.ericsson.com/en/ blog/2017/6/5g-teleoperated -vehicles-for-future-public-transport . Accessed: 2019-02-14.

[3] Mate Boban , Apostolos Kousaridas, Konstantinos Manolakis, Josef Eichinger, and Wen Xu . 2018 . Connected roads of the future: Use cases, requirements, and design considerations for vehicle-to-everything communications . ieee vehicular technology magazine 13 , 3 ( 2018 ), 110 - 123 .

[4] Martijn Bout , Anna Pernestål Brenden, Maria Klingegård, Azra Habibovic, and Marc-Philipp Böckle . 2017 . A Head-Mounted Display to Support Teleoperations of Shared Automated Vehicles . In Proceedings of the 9th International Conference on Automotive User Interfaces and Interactive Vehicular Applications Adjunct. ACM , 62 - 66 .

[5] Jean-Michael Georg , Johannes Feiler, Frank Diermeyer, and Markus Lienkamp . 2018 . Teleoperated Driving, a Key Technology for Automated Driving? Comparison of Actual Test Drives with a Head Mounted Display and Conventional Monitors . In 2018 21st International Conference on Intelligent Transportation Systems (ITSC) . IEEE, 3403 - 3408 .

[6] Renate Haeuslschmid , Bastian Pfleging, and Florian Alt . 2016 . A design space to support the development of windshield applications for the car . In Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems. ACM , 5076 - 5091 .

[7] Gereon Hinz , Josef Eichinger, Martin Büchel , and Alois Knoll . 2017 . Proactive Video-based Use of Telecommunications Technologies in Innovative Motorway Scenarios . 5th GI/ITG KuVS Fachgespräch Inter-Vehicle Communication ( 2017 ), 41 .

[8] SAE International . 2016 . Taxonomy and definitions for terms related to driving automation systems for on-road motor vehicles .

[9] Toni Ivergard and Brian Hunt . 2008 . Handbook of control room design and ergonomics: a perspective for the future . CRC Press.

[10] Dagmar Kern and Albrecht Schmidt . 2009 . Design space for driver-based automotive user interfaces . In Proceedings of the 1st International Conference on Automotive User Interfaces and Interactive Vehicular Applications. ACM , 3 - 10 .

[11] Lars Lischke , Sven Mayer, Andreas Preikschat, Markus Schweizer, Ba Vu, Paweł W WoŹniak, and Niels Henze . 2018 . Understanding Large Display Environments: Contextual Inquiry in a Control Room . In Extended Abstracts of the 2018 CHI Conference on Human Factors in Computing Systems. ACM, LBW134.

[12] Chang Liu , Hong Yu Zheng, and Chang Fu Zong . 2013 . Study on Control Strategy of Steer-by-Wire Vehicle with Joystick . In Advanced Materials Research , Vol. 694 . Trans Tech Publ, 2061 - 2064 .

[13] Michael N Louka , Morten A Gustavsen, and Svein Tore Edvardsen. 2006 . Using virtual reality to support multi-participant human-centered design processes for control room design. In Proceedings of 5th international topical meeting on nuclear plant instrumentation, controls, and human machine interface technology (NPIC&HMIT 2006) at the American Nuclear Society 2006 meeting , November. American Nuclear Society, Albuquerque.

[14] Ahmad M Mustafa , Omar M Abubakr , Ahmed H Derbala , Essam Ahmed, and Bassem Mokhtar . 2017 . Towards a Smart Highway Lighting System Based on Road Occupancy: Model Design and Simulation . In Applications for Future Internet . Springer, 22 - 31 .

[15] Tito Tang , Frederic Chucholowski, and Markus Lienkamp . 2014 . Teleoperated driving basics and system design . ATZ worldwide 116 , 2 ( 2014 ), 16 - 19 .

[16] Gesa Wiegand , Christian Mai, Yuanting Liu, and Heinrich Hußmann . 2018 . Early Take-Over Preparation in Stereoscopic 3D . In Adjunct Proceedings of the 10th International Conference on Automotive User Interfaces and Interactive Vehicular Applications. ACM , 142 - 146 .