Autonomous driving has evolved with the advancements in AI, sensor technology, and intelligent control in a multidisciplinary intersection of research [1]. While the concept was first implemented decades ago, technological developments have led to increased attention and research in autonomous driving systems (ADS) [2].
An ADS generally has six stages or layers, with each stage feeding new information into the next stages: (1) the sensor and hardware layer for gathering data about static and dynamic objects from the environment, (2) the perception layer for performing object tracking and detection tasks, (3) the localization and mapping layer for getting the vehicle's position in the environment, (4) the assessment layer for estimating the overall risk and performing predictions to avoid accidents, (5) the path planning and decision-making layer for getting the shortest path from start to end point without collisions, and (6) the control layer which uses actions to control the vehicle to perform the path [3].
The main focus of this paper is on the decision-making layer; however, the aim is not to attain the shortest path but rather to utilize hybrid intelligence (HI), or hybrid augmented intelligence (HAI) as it is referred to in some papers [1], for planning the best route according to multiple human-machine criteria. This is because decision-making and planning can evolve by augmenting human and machine intelligence into HI. Human drivers have the capability to improve the performance of autonomous driving in realworld traffic, and their feedback should be introduced into the learning process of ADSs [1].
Nevertheless, although research extensively covers tech- Envelope nada.sanad@oulu.fi (N. Elgendy); anna.teern@oulu.fi (A. Teern); pertti.seppanen@oulu.fi (P. Seppänen); tero.paivarinta@oulu.fi (T. Päivärinta) Orcid 0000-0001-6765-017X (N. Elgendy); 0000-0002-8214-3181 (A. Teern); 0000-0002-4289-2487 (P. Seppänen); 0000-0002-7477-0783 (T. Päivärinta) nologies, methods, techniques, and algorithms for enhancing vehicle autonomy and performance, research on hybrid intelligence autonomous driving systems (HI-ADS) is limited. Within such research, there is a further gap on how to design HI-ADS in practice and the relevant requirements and necessary knowledge. Therefore, the research question of this paper is as follows:
"What are the design and knowledge requirements for human-machine hybrid intelligence in autonomous driving systems?" Expert interviews were conducted within the scope of the 6G Visible project to answer the research question and propose a set of design requirements and knowledge requirements for HI-ADS. The paper is structured as follows. Section 2 provides the theoretical background on ADS and human-machine HI. Section 3 covers the research method and design, including the research process, a description of the 6G Visible project and the route planning and change decision use case, and a summary of the interviews conducted. Subsequently, the results are presented and discussed in Section 4. Finally, Section 5 concludes the paper and suggests future work.
ADSs are defined by the Society of Automotive Engineers (SAE) as "vehicle driving automation systems that perform part or all of the dynamic driving task on a sustained basis" [4]. There are six levels of driving automation from Levels 0 to 5. Level 0 systems exhibit no driving automation. Level 1 systems include primitive driver assistance, while Level 2 systems have partial driving automation into which advanced driver assistance systems are integrated. Level 3 systems exhibit conditional driving automation where drivers should be ready to respond and take over. On the other hand, Level 4 systems have high driving automation and do not require human attention but can only operate in limited domains and infrastructures. Finally, Level 5 systems are those with full driving automation [4] [5].
The scope of this paper focuses on Level 1, 2, and 3 ADSs, which require some form or degree of human supervision and intervention. In the future, this research can be developed for Level 4 ADSs. However, Level 0 and 5 ADSs, which either require full or no driver control and intervention, are outside the scope of this research.
The ability of autonomous driving to handle dynamic driving environments is based on two fundamental components: decision-making and motion planning. In decision-making, information is acquired on traffic perception and vehicle states, and driving tasks are accomplished by generating desired driving behavior. Next, motion planning occurs by calculating the desired trajectory or vehicle actuator commands [1].
Higher levels of automation are more complex, although still lacking in outperforming human drivers, and require a perception system (which perceives data through sensors from the environment), traffic rules interpreter, and a vehicle controller in addition to the decision system which controls vehicle behavior and what reaction the system should perform [6].
Despite recent technological advancements, many tasks are yet unsolvable by machines alone and require sociotechnical ensembles of humans and machines, or HI systems [7]. HI emphasizes human-AI complementarity by utilizing the senses, perceptions, emotional intelligence, and social skills of humans with the advanced processing, computational, and pattern detection capabilities of AI [8]. Furthermore, such human-machine collaboration leads to collaborative rationality, which is no longer bounded by the limitations of each and can lead to higher quality and more informed data-driven decisions [9]. Accordingly, a human and machine agent collaborate together, informed by data from various sources. Collaborative rationality and/or HI then facilitate data-driven decision-making, and the results of the decisions and their evaluation provide feedback that can be fed back into the system to enable new types of learning [10].
Many application areas for HI have been discussed throughout the literature. For example, HI can be useful for autonomous vehicles, which require accurate and up-to-date comprehensive maps to provide real-time environmental information. Human agents could provide information and knowledge about the streets, such as rush hour situations, detours in various times and conditions, and their experiences and recommendations for navigation. On the other hand, real-world information could be provided by traffic cameras and monitoring sensors, as well as satellites and global maps. Advanced algorithms can then suggest navigation controls based on fusing both human and machine intelligence [11].
HI can lead to a semi-autonomous driving architecture through which human and machine agents can cooperatively achieve driving tasks with better system performance than would be achieved by each on their own [12]. Furthermore, data from human guidance and real-time evaluation reflects the driving preferences of humans and can improve the learning quality of machine agents [1]. Nevertheless, such systems are very complex, require high levels of information fusion, and are encumbered by several challenges, such as the integration of complicated systems, ensuring robustness and reliability in complex scenarios and weather conditions, incorporating human cognition and knowledge with existing information for decision-making, and data privacy protection [12].
While some studies focus on augmenting human feedback to reinforcement learning in the decision-making and planning of autonomous vehicles [1], human input is only observed in ex-post evaluation of the vehicle's decisions, and human knowledge is not augmented ex-ante into the decision. Other studies aim to model human psychology and cognition into intelligent decision-making systems for autonomous driving [6]. However, such research mainly focuses on enhancing the autonomy, algorithms, and performance of the vehicle (e.g. [13][2] [3]) without harnessing the capabilities and knowledge of human drivers during the collaborative driving and decision-making process.
Another study emphasized the importance of HI for autonomous urban vehicle control through a theoretical example, with no practical guidelines on the design requirements of such systems [11]. A theoretical architecture for HI in autonomous driving was also proposed, along with several research challenges [12]. Nonetheless, the requirements for designing such systems in practice are not elaborated. Hence, this research aims to define the design and knowledge requirements for HI in ADSs.
A design science research (DSR) methodology is planned to be adopted for designing a HI-ADS. This paper constitutes the initial stages of defining the requirements of the solution artifact [14] according to the practical needs derived from the environment and the knowledge gained from the knowledge base [15].
First, the 6G Visible project within which this research is conducted is explained. Next, the data-driven decisionmaking use case, which was determined to narrow the scope of the research, is elaborated. Consequently, data was collected through several semi-structured interviews, which are covered in subsection 3.5.
The 6G Visible project aims to develop advanced solutions for autonomous and semi-autonomous driving while considering the capabilities of 6G network technologies to support real-time, safety-critical services. These solutions aim to provide visibility for objects not visible to drivers or detectable by existing vehicle sensors, even in various weather conditions. This involves creating dynamic, real-time models of the environment, traffic, and weather conditions, as well as detecting obstacles for autonomous driving. Furthermore, it involves augmenting human rationality and knowledge with machine rationality and artificial intelligence while utilizing various data and knowledge sources (e.g. vehicle, sensor, traffic, weather, feedback) to enhance data-driven decision-making in autonomous driving. The research is conducted through a collaboration between software engineering/information systems, the Finnish Meteorological Institute (FMI), and leading industry collaborators in AI, digital twins, and autonomous driving.
Current autonomous vehicle solutions rely on static environmental models (such as maps of city streets and buildings, etc.) and integrated sensors on the vehicle. While they work well for many applications, they are limited in providing real-time information and rapidly changing situations beyond the sensor's range. Furthermore, icy and harsh weather conditions can affect driving and road conditions in various ways that may not traditionally be accounted for. Hence, real-time dynamic environmental data, including weather adaptation, is essential to enhance autonomous systems.
To narrow the scope of the research, three data-driven decision-making use cases were determined based on their dependence on the changing conditions of the external environment. First, in the route planning and change use case, the HI system should determine the optimal route and adapt to complex road situations, changes in the environment, and unexpected weather conditions. Second, in the parking use case, the system should support automating the parking process according to the relevant external conditions. Third, in the weather and lighting adaptation use case, the system should adjust to various weather and lighting conditions based on data and the environment.
The first data-driven decision-making use case, route planning and change, was selected as the most comprehensive and highest-priority use case to start with.
Finding the route on the road network from the origin to the final destination is the responsibility of the global planner in the ADS, where high performance has become an industry standard through advancements in GPS and offline map navigation systems. However, depending on the country or location, road hierarchies differ, and the shortest path may not be the fastest or most desirable. Furthermore, there are other considerations, such as avoiding obstacles, satisfying optimization criteria, and understanding road semantics, lanes, and drivable surfaces [5].
In many countries, such as Finland, weather conditions complicate autonomous driving. Winter conditions that cause slippery and icy road surfaces and low visibility due to fog, heavy snowfall, or blizzards and blowing snow are challenging for ADSs and may require making sudden or precautionary changes to particular routes. Precise and real-time weather information is necessary, along with information about road conditions, obstacles, flooding, or other unexpected situations. Additionally, instruments that detect the driving circumstances, such as sensors, cameras, LiDAR, and radar, are crucial for autonomous driving and yet face many vulnerabilities in harsh weather and winter conditions [16].
An iterative top-down approach was adopted for designing semi-structured, qualitative interview questions and for data collection, as shown in Figure 1. Rather than defining the available data that can be used to support the data-driven decision, we started by defining the decision scenario and Local taxi company context and drilling down to the knowledge and then the data required to support such decisions. By following this approach, we can focus our resources to utilize only the necessary knowledge and data. The interview questions were structured into themes according to the components of the framework proposed in Section 3.1. and the elements of data-driven decisionmaking [1]. Hence, there were questions about the decision, decision-making process, knowledge, data, human decision-maker, machine, and decision evaluation as separate attributes.
Deductive thematic analysis [17] was then applied to group the requirements discussed during the interviews according to the predefined themes.
A total of 6 interviews, each approximately 2 hours long, were conducted as summarized in Table 1. Accordingly, the interviewees were provided with the interview questions, and open discussions were held.
The dynamic route planning and change decision scenario is a complex data-driven decision with various internal and external factors. The considerations that must be accounted for within the decision scenario that resulted from the interviews are synthesized below. They are categorized into themes based on the seven attributes upon which the interviews were structured.
The decisions involve selecting and adapting the optimal route based on the combination of a variety of factors, including weather conditions, road friction, road conditions, visibility, accident alerts, traffic information, speed limits, familiarity with the route, road maintenance, traffic jams, construction or roadwork, rest points, personal preferences such as scenery and facilities, how busy the route is, and the driver's urgency. Additionally, considerations like vehicle type (personal vs. public transport), driver experience, and road safety are important to account for. The necessity for route adaptation or change usually arises from emergent conditions such as changes in weather, mostly based on road friction from icy conditions, sudden traffic changes, accidents or emergencies, or alterations in driver preferences or requests.
Human preferences, experience, and input play a large role in route planning and change. For instance, drivers may prefer particular routes based on services (e.g. certain gas stations), familiarity with the route, avoidance of roadwork, or other preferences, regardless of the expected time for taking the route or other variables. Additionally, icy roads complicate driving and are difficult to account for. They require consideration of the road lanes, which are not visible in such conditions, friction, speed, traffic, and other variables.
For example, one interviewee remarked, "We have tested the automatic vehicle with machine vision that works during summer somehow, but not very reliably even then, and winter would be impossible because it's trying to search for the white [lane] lines. " Additionally, sensors are not always accurate (e.g. may be blocked and sense a normal road as icy). This usually requires an experienced driver who can realize whether or not the road is actually icy.
As another example, one interviewee stated, "I know I don't change the route because basically, I know that there's no traffic jam there. There's, for some reason, a slow car driving there, and it's giving false readings to that server, and the server is sending to my GPS in my own vehicle data that there's a traffic jam in front, because there are slow cars there…it's not really necessary to drive 20-30 kilometers more because of that. "
Hence, traffic congestion may be due to a few cars which are slow (e.g. due to weather or road conditions) and may not require a route change. However, the system might usually suggest changing to another, longer route due to traffic congestion. An experienced driver would know that this is not actual congestion, but due to a slow car which will pass soon. Accordingly, it would not be necessary to change to a less desirable route.
While route planning and optimization is an important aspect for autonomous driving, both for individuals and public personal transportation, such as taxis, it is currently mainly managed by the personal knowledge and experience of the drivers.
The decision-making process requires environmental awareness and starts with the collection of data regarding available routes, estimated travel times, and current and predicted road and weather conditions. This data is then evaluated against a set of criteria that may include safety considerations, efficiency metrics, and personal preferences. Accordingly, the alternatives may be prioritized, and a route choice is made. The process should be iterative and allow reassessing and modifying the decision and plans based on incoming information and changing conditions.
The process is highly dependent on the level of automation of the ADS. Lower levels require more human involvement in the decision-making process, while higher levels would necessitate more automated processes with less human intervention.
Informed decision-making in the context of route planning and change requires access to and understanding of various knowledge areas. Essential knowledge includes access to and interpretation of reliable traffic, road condition, and weather information, as well as their implications on the route, safety, and driving speed. Furthermore, while knowledge about other vehicles is crucial, current vehicleto-vehicle communication is limited.
Additionally, the application of real-time data from weather stations and the utilization of algorithms for rapid data analysis and interpretation are critical for making informed decisions about route adaptations. Human intuition and experience remain to be important sources of knowledge and understanding of the environmental context.
The decision is heavily reliant on multiple data sets, which include accurate GPS and location data, sensor, camera, and LiDAR data, detailed road weather information, traffic congestion reports, construction and accident updates, visibility (e.g. fog or increased sunshine), and precise road surface temperatures and conditions (e.g. ice, snow, or water presence). Nevertheless, GPS routes, and maps are not always precise. One interviewee commented, "We need accurate GPS routes, but the maps aren't usually so accurate. You need to go and measure them yourself with [real time kinematic] RTK GPS, usually. "
Regarding weather data, one interviewee said, "We need to be wary in situations where it is below 0 degrees [C] or close to 0 degrees, and then we have observations, and we also model the road conditions; which means if there is ice or snow or water on the road. "
The sources of data range from the vehicle and its sensors and systems to external weather stations or other services and data sources. Derived or calculated variables are also necessary, such as friction levels, driving condition indices, and precipitation information. One interviewee informed, "We also calculate these kinds of more advanced variables like friction that tell us if the friction value is low, then it's slippery, and if it's high, then it's not slippery and also gives this kind of three-level index that is the driving condition: normal, bad, or very bad. Precipitation information is probably important, like if there's a storm or something. "
Additionally, for route changes, new information acquired en route is crucial. The availability of traffic accident data, changes in weather forecasts (according to an interviewee, "They [weather data] can also change because sometimes the forecast is wrong and then we suddenly no-tice…sometimes there can be these kinds of exceptional changes. "), and real-time traffic flow data further inform decision-making. Metadata and decision variables can be recorded in logs to monitor the decision and inform future decisions.
The role of the human is highly dependent on the level of ADS automation. Within the scope of this research, humans play a critical role in overseeing and finalizing decisions, with the ability to adjust or override automated suggestions based on their personal judgment, experience, and real-time observations. The human decision-maker generally evaluates automated recommendations, verifies machine observations (e.g. from sensors), ensures safety, and considers personal preferences or knowledge of local conditions. Some limitations to human drivers exist. For example, not all drivers are effective at predicted driving, or predicting and reacting to driving scenarios, due to limitations in cognitive and memory functions. Furthermore, emotions may also lead to poor decisions. For example, if there is a traffic delay, a driver's mood may become negatively affected and lead to rash decisions that wouldn't have otherwise been made.
Nevertheless, while such human errors and biases exist, these can be reduced through the provision of simplified information and reliable data to support decision-making. For example, knowing what is causing the traffic delay and possible predictions may prevent negative moods and lead to better decisions. Additionally, more minor errors can automatically be corrected or alerted to by the vehicle, such as lane keeping. For example, an interviewee added, "I would say in normal cars, there are cameras monitoring humans, so if they fall asleep or something like that, then their vehicle can try to stop at least some of them…Of course, the lane keeping and stuff like that, there's a lot of that already in modern cars. "
Furthermore, humans may have more experience as well as environmental and situational awareness than machines. For example, if the ADS suggests a route change based on a traffic jam, a human who has more experience with that particular route may know that the traffic jam is due to a slow car, which will clear up quickly, and that the current route remains more optimal than a route change.
The role of the machine is also dependent on the level of the ADS automation, with higher levels requiring larger machine roles. Nevertheless, the current status of technology requires human oversight, validation, and control. Thus, humans generally make the final decisions. Machines can support (or make) the decision by suggesting (or taking) routes based on programmed criteria, preferences, safety, and efficiency, while drawing on quality data and updated maps. However, current systems remain subpar in terms of data and recommendation reliability, availability of more comprehensive information and environmental awareness, consideration of a variety of human preferences, and effective and efficient communication with the human driver.
Particularly in route change scenarios, such decisions need to be made on the spot and communicated to the driver in the fastest, simplest, and least disturbing way possible. Additionally, there should be some degree of transparency for the human driver to understand why the route is being changed, and to be given the chance to override the decision as long as it does not compromise safety requirements.
On the other hand, vehicles need more situational awareness about the environment and decision context, as is emphasized by Siili Auto. While not all available data is used by the vehicle, such data may be crucial to other vehicles. Accordingly, there should be vehicle-to-vehicle communication, and data can be shared across a network.
Some companies, such as Elektrobit, have developed automated driving software that aims to make the vehicle aware of the surroundings and road ahead, see beyond the range of sensors through predictions, provide precise positioning, and model the vehicle environment. With such features, vehicle capabilities may be enhanced and provide more knowledge to the HI-ADS for supporting data-driven decision making. Nevertheless, renewing vehicle-level instrumentation, which deploys the latest technologies, takes time.
After the data-driven decision is made, it should be evaluated, either during the route or after the destination is reached. It can be evaluated through several metrics or criteria, which include passenger satisfaction, safety perceptions, and the accuracy of estimated versus actual travel times. Manual mechanisms for feedback, such as user ratings or direct input on satisfaction levels, could help in refining future decision-making processes and should be supported by the system. Additionally, more automated feedback mechanisms should also be considered.
Moreover, although it is not generally considered, the impact of the decision on the external environment and other vehicles should be accounted for. For example, actions such as unnecessary overtaking may also delay other vehicles and negatively impact traffic and should be evaluated as such. However, humans and current technologies may not consider the effect or consequences of the decision on others or the external environment.
Furthermore, evaluation could be conducted by incorporating external data sources, such as weather or accident data, to perform a more comprehensive evaluation. Local forecasts can be compared to observation data (e.g., roads that were predicted to be slippery but weren't, or vice versa) to enhance future decisions, not only for the vehicle but for other vehicles as well. However, more global evaluation would require communication with other servers, weather stations, and other vehicles.
Based on the results of the interviews and considerations discussed in the previous subsection, the requirements that h Define evaluation criteria and feedback mechanisms. Incorporate various aspects for evaluation, including impact and consequences on the internal and external environments. Provide the capability to detect, reduce, and learn from both human and machine decision-making and errors. should be supported when designing a HI-ADS are summarized in Table 2.
For representing human and machine knowledge in the system, knowledge graphs (KG) can be used to structure the knowledge and enable HI systems. A KG is a large-scale knowledge base commonly used for intelligent applications, which comprises a large number of entities and the relationships between them [18]. The KG construction process is as follows. Starting with the knowledge acquisition, the data for building the ontology is acquired. Knowledge fusion tasks are then performed. The knowledge is stored and represented, and then processed to create and update an ontology. Consequently, after the ontology is built, the knowledge can be utilized, and further iterations can occur to evolve the KGs [19].
Such aspects can serve as a basis for augmenting human and machine knowledge within the HI-ADS to support datadriven decision-making and enable knowledge sharing and utilization.
As an initial stage, the knowledge requirements that were determined from the interviews are summarized in Table 3 and can subsequently be considered for building the ontology.
Based on the knowledge requirements and through further research and work, an initial ontology is developed. However, the ontology is not presented within the scope of this paper and will be developed further in the future for constructing the KGs within the HI-ADS.
The main contributions of this paper are two-fold: the design requirements in Table 2 and the knowledge requirements in Table 3.
The design requirements cover a range of multi-faceted attributes that should be reflected in the system, as opposed to focusing only on vehicular requirements, technological advancements, or planning and prediction algorithms (e.g. [13][3][5]). While such focused research is necessary for advancing ADSs, it generally does not consider the human aspect or additional decision-making factors. This paper contributes to such a gap by proposing a more comprehensive viewpoint to the data-driven decision scenario.
Accordingly, the attributes considered in the design requirements include the decision, decision-making process, knowledge, data, human, machine, and decision evaluation considerations.
A set of knowledge requirements for the system is subsequently proposed, from which an ontology can be devel-oped, and the data required by the system can further be determined. By adopting a top-down approach, resources can be focused on gathering only the necessary data and knowledge that support the data-driven decision.
Therefore, the design and knowledge requirements contribute to theory by establishing the initial objectives of a design science artifact, which can be developed in future research. Furthermore, they support a more comprehensive and sociotechnical view for the practical development and implementation of hybrid intelligence in autonomous driving systems beyond the prevailing and somewhat myopic focus on vehicular capabilities.
Moreover, this paper supports existing research that emphasizes the need for improving situational awareness of the vehicle and its understanding of the environment [2], incorporating human feedback into the learning process for HI-ADSs [1], and considering various factors from multiple systems and utilizing human cognition [12]. Such needs are reflected in the design and knowledge requirements and extended upon with additional considerations regarding the data-driven decision context. Additionally, determining the knowledge requirements is an important, although commonly overlooked, prerequisite for supporting the development of knowledge graphs within the HI-ADSs.
In this paper, we aimed to determine the design and knowledge requirements for developing HI-ADSs. Six expert interviews were conducted within the scope of the 6G Visible project. Consequently, following a top-down approach, considerations for a dynamic route planning and change decision scenario were defined based on seven attributes: the decision, decision-making process, knowledge, data, human decision-maker, machine, and decision evaluation. From these considerations, the design requirements were determined, followed by the knowledge requirements for HI-ADSs Future work involves building an ontology and knowledge graph based on the knowledge requirements, merging the main concepts from the data-driven decision-making and evaluation model in [10] and the KG construction process in [19] into a conceptual framework for augmenting human-machine intelligence and knowledge, and designing a HI-ADS based on the design requirements. Finally, data privacy, legal, and ethical regulations and requirements were not covered within the scope of this research and may be considered in the future.
This research was partially funded by Business Finland. We would like to acknowledge the Finnish Meteorological Institute, Siili Auto, Elektrobit, and the interviewees from the local taxi company and driving school in Finland for their cooperation.