Design of AI-based lane changing modules in connected and autonomous vehicles: a survey Bharathkumar Hegde1 , Melanie Bouroche1 1 Trinity College Dublin, Dublin, Ireland Abstract Lane changing is one of the complex driving tasks as it requires the vehicle to be aware of its highly- dynamic surrounding environment, make decisions, and enact them in a timely manner. By exploiting both sensors and inter-vehicle communication, Connected and Autonomous Vehicles (CAVs) have the potential to significantly improve lane changing safety and efficiency. The complexity of the task and the real-time requirements make lane-changing a problem particularly suited to Artificial Intelligence (AI) approaches. In this paper, we survey the design of AI-based Lane-Changing(LC) modules for CAVs. First, we identify the key factors that can influence the design of an LC module. Next, we survey recent developments in AI-based lane changing. Finally, we analyse these approaches along the dimensions of the key influencing factors and summarise the challenges that are yet to be addressed and opportunities that can guide the future developments in AI-based LC modules. Keywords Connected and autonomous vehicle (CAV), Lane change, Artificial intelligence, Deep learning (DL), Intelligent transportation system (ITS) 1. Introduction Autonomous Vehicles (AVs) are one of the major components of a rapidly developing Intelligent Transportation System (ITS). Developments in the communication technology are expected to complement the development of the AV technology. Therefore, advancements in Connected and Autonomous Vehicles (CAVs) are expected to improve the performance in the driving tasks required for achieving autonomy of level of 3 and above [1]. Currently, in the commercial market, Tesla Model S has achieved an autonomy level of 2.5 and the Audi A-8 has achieved level 3 autonomy in driving [2] by automating major driving tasks. Whereas, a fully autonomous vehicle (SAE’s level 5 autonomy) should be capable of performing all driving tasks safely and efficiently in all kinds of environment. Among the driving tasks, lane changing is one of the complex tasks for CAVs and a challenging problem for researchers [3]. By planning and coordinating lane changes, CAVs might be able to improve the traffic flow at both microscopic and macroscopic level. The macroscopic traffic level benefits may include increased safety, traffic efficiency, and road capacity [4] and the microscopic traffic level benefits may include increased comfort for travellers with minimal speed variation and reduced travel ATT’22: Workshop Agents in Traffic and Transportation, July 25, 2022, Vienna, Austria Envelope-Open hegdeb@tcd.ie (B. Hegde); melanie.bouroche@tcd.ie (M. Bouroche) GLOBE https://www.scss.tcd.ie/Melanie.Bouroche/ (M. Bouroche) Orcid 0000-0002-2085-7867 (B. Hegde); 0000-0002-5039-0815 (M. Bouroche) © 2022 Copyright for this paper by its authors. Use permitted under Creative Commons License Attribution 4.0 International (CC BY 4.0). CEUR Workshop Proceedings http://ceur-ws.org ISSN 1613-0073 CEUR Workshop Proceedings (CEUR-WS.org) delay [5]. To plan a lane changing manoeuvre, CAVs use information collected by sensors and other vehicles in a highly-dynamic environment [6]. An LC module uses this information to make lane change decisions. Some of the popular methods used to design an LC module are game theory [7], controller optimisation [8], and AI [9]. Recently, AI has been used more often to design LC modules, as the recent developments in AI have proven useful to make fast decisions in dynamic environments with a large set of parameters. Real-world traffic is very dynamic and a large set of parameters may be considered to perform a lane change. Parameters may include the position, speed, and heading of the ego vehicle and surrounding vehicles [10]. Moreover, an LC module needs to make intelligent trade-offs to improve the possibility of achieving safety along with other objectives of lane changes, such as improving mobility, comfort of travel, fuel efficiency, and reducing emissions. Therefore, AI is a promising option for designing LC modules, which can make efficient lane change decisions in complex traffic environments to achieve multiple objectives. Most survey papers have reviewed the application of AI in wider fields such as ITS [11], CAVs [12], or V2X communication [9]. Conversely, this paper focuses on the design of AI-based modules for CAV lane changing controllers. Specifically, the main contributions of this work are: • Identify the key factors that can influence the design of an AI-based LC module. • Provide insights into the recent developments in the design of AI-based LC module along the dimensions of the key influencing factors. • Summarise the challenges and opportunities in the design of AI-based LC modules. Challenges provide the research gaps that are yet to be addressed, and opportunities identify the possibilities that can guide the future developments of AI-based LC modules. The remainder of the paper is structured as follows. Section 2 provides background details related to the development of lane change and the architecture of lane changing in CAVs. Section 3 presents the key factors that influence the design of AI-based LC modules. A review of AI-based LC modules is presented in Section 4. Finally, Section 5 reviews the approaches discussed before summarising the challenges and possible opportunities in the design of AI-based lane changing modules. 2. Background This section discusses the development of lane changing models and a general architecture of a CAV lane change. The development of lane changing models lists some of the standard lane changing models used for traffic simulations. The general architecture of a CAV lane change provides its components and describes how they are related. 2.1. Development of lane changing models An LC model usually encodes a rational decision to change lanes based on various parameters that describe the environment around a vehicle. The first known LC model is the Gipps lane changing model [13]. The Gipps model is based on maintaining a desired speed and being in the correct lane for an upcoming desired manoeuvre. LC models developed based on the Gipps model are classified as Gipps-type LC models. To overcome the limitations of Gipps-type LC models, which were deterministic, rule-based models were proposed [14]. Rule-based models consist of a decision process defined in four steps: decision to consider a lane change, choice of the target lane, search for an acceptable gap, and executing the lane change [15]. Considering the probabilistic approach for lane changing decision instead of deterministic lane changing decision as in Gipps type LC model, was one of the distinct features that made the rule-based model more realistic [15]. While the Gipps-type LC models consider only the vehicle speed, Kesting et al. proposed a novel incentive-based lane changing model, MOBIL (Minimising Overall Braking Induced by Lane change)[16], which considers the acceleration of the vehicles as well [15]. The MOBIL model makes a lane change decision based on the probability of advantages and disadvantages of the lane change, based on the accelerations of the vehicles. In addition to the acceleration, MOBIL model also considers factors such as politeness and the right-left lane bias (which restricts overtaking from the right side, eg in Germany,) [15]. These considerations enable easy integration of MOBIL with car-following models such as the Intelligent Driver Model (IDM) [17]. A lane changing model, named LC2013, considers the intention of changing lanes using a decision-tree algorithm [18]. The intention for a lane change can be to reach a specific destination, to overtake a slow vehicle, to cooperate with other vehicles, or to follow local traffic regulations. In LC2013 LC model, CAVs coordinate by sharing their intentions during lane changing manoeuvres. The LC2013 model is integrated in the Simulation of Urban MO- bility (SUMO) simulation framework and allows customisation to simulate regulatory traffic restrictions, such as the restriction on overtaking from the right side as in Germany. To conclude, the models discussed above have been used as standard lane changing models in popular traffic simulators and as a baseline to validate recent LC modules. These standard LC models are designed to achieve a single objective, that is, to make a safe lane change decision. However, recent AI-based LC modules aim to achieve safety along with other objectives such as improving mobility, comfort, and fuel efficiency. A detailed discussion of recent AI-based LC modules is presented in Section 4. 2.2. Architecture of CAV lane changing The architecture of CAV lane changing typically consist of four major components, namely perception, communication, lane changing, and vehicular control as shown in Figure 1 [12]. The perception module creates a perception of the environment around the vehicle by com- bining the inputs from various sensors such as LiDAR, RADAR, camera, GPS, IMU, etc. The vehicle-to-everything (V2X) communication module provides interfaces to communicate with other components of the Intelligent Transport System (ITS), such as other vehicles, road side unit (RSU), mobile edge computing (MEC) server, cloud server, etc [19]. The lane changing module integrates inputs from the perception module and information collected through the communication module to make a lane changing decision and plan trajectories for the execution of lane changes [12]. The lane changing module can be implemented using either a centralised [20, 21] or a decentralised [10, 22] architecture. The Vehicular control module includes the Figure 1: Architecture of CAV lane changing lateral and longitudinal control of the vehicle which executes the instructions from the lane change module [2]. 3. Dimensions of the survey In this survey, we explore the design of AI-based LC modules for CAVs along four dimensions. These dimensions are objectives of the lane change, lane change scenarios, architecture, and mixed traffic consideration. They are some of the key factors that influence the design of AI-based LC modules implemented in CAVs. 3.1. Lane changing objective The objectives of lane changing can be broadly classified as safety, mobility, comfort, and sustainability. Safety is one of the major objectives considered in LC modules. Safety objectives mainly focus on lane changing with minimal risk to avoid collisions [5, 4, 3, 23, 24]. However, a CAV should not compromise on mobility, while trying to improve safety. Mobility objectives consider improving traffic throughput [10, 25] and average speed [21] of the vehicle, and avoiding stop-go traffic [25]. To improve mobility, a CAV can make unnecessary lane change manoeuvres [5] or accelerate and decelerate frequently, causing discomfort to passengers [24, 26]. Therefore, achieving travel comfort is another potential objective of LC modules. Furthermore, some of the LC modules consider sustainability [27] as one of their objective, as lane changes can also affect overall fuel efficiency and emissions [10]. 3.2. Lane change scenario The lane change scenario can be defined based on the motive of a vehicle to perform a lane change. The motive to change lane can be broadly categorised as discretionary lane change, mandatory lane change, and lane change in bottleneck sections. Dynamics of vehicle movement and the parameters considered for the lane change decision making differ for each of these categories of lane change. Hence, the lane change scenario can be one of the factors to consider while designing an LC module. An optional lane change by a vehicle, for the benefit of its own or other vehicles in traffic, is considered a Discretionary Lane Change (DLC). DLCs often result in increased speed for the ego vehicles, and they may have various positive impacts on the traffic at the macroscopic level, such as increasing road capacity, increasing traffic throughput, minimising traffic jam propagation, etc. DLCs focus primarily on safety and achieving macroscopic objectives like increased driving comfort, mobility, or throughput [5, 7, 26]. On occasions, a vehicle may be required to change the lane to reach a desired destination; such lane changes are classified as Mandatory Lane Changes (MLCs). Some examples of MLCs include changing lane to enter a highway, exit a highway, or before reaching an intersection for a turn. Since lane change is mandatory in these cases, the vehicle may need to execute a risky lane change, especially in high traffic. MLC by a vehicle may affect the other vehicles in traffic, therefore, the ideal MLC controller should be capable of ensuring safety even under risky situations and it should have minimal negative impact on the mainstream traffic flow of the highway [22, 21, 10]. Similar to an MLC, a vehicle will be changing lanes when the current lane reaches a dead end or merges into an adjacent lane. Such lane changes can be categorised as lane change in bottleneck sections. In bottleneck sections, coordination among vehicles plays a key role as the vehicles changing lane will interrupt the main traffic flow. A bottleneck may be created because of construction works, reduced road space, vehicle broken-down, or accidents. Hence, bottleneck sections are often not observed in advance. Therefore, lane changes in bottleneck sections may need to be handled differently compared to an MLC. Typically, the LC modules for bottleneck sections aim to achieve a smooth traffic flow with less congestion, and increase traffic throughput by avoiding stop-go traffic [20, 25]. 3.3. Architecture The architecture of an LC module can be classified as centralised or decentralised, depending on its placement within the CAV lane change architecture. In a centralised architecture, the LC module can be placed in an RSU, an edge server, or a cloud server which can be a centralised controller. A centralised controller can integrate the information from traffic participants and use it for trajectory planning and lane changing decisions. Furthermore, the controller may suggest state changes, such as path, velocity, etc. [28] or lane change decisions to a CAV [21]. Centralised controllers have been found to achieve better results in completing cooperative objectives [29]. However, one disadvantage of the centralised architecture is the challenge of scaling the central server based on the variation in the traffic flow. Additionally, it adds an extra overhead for installing and maintaining a wide scale infrastructure. As the central entity can be a bottleneck of the system, the centralised infrastructure is prone to failures and network congestion. On the other hand, the decentralised architecture can be implemented by placing the LC module in individual CAVs. The LC module along with the perception module and the vehic- ular control module can form an autonomous controller. These autonomous controllers can communicate through a direct V2V communication interface or through the network infras- tructure to collect the information needed from other CAVs for trajectory planning and lane change decisions [30]. The necessary information required from other CAVs may include state information, trajectory plan, traffic information, etc. An LC module can collect this information and act independently or interact with other CAVs to achieve a cooperative decision [28]. In a decentralised architecture, lane change execution can be faster, as the LC module and the vehicular control module are placed in the same CAV [31]. This approach can significantly improve the scalability of the module based on traffic demands. Furthermore, ITS infrastructure, such as RSU or MEC servers, can be used to offload some of the resource-intensive computations [28]. However, one of the key challenges in a decentralised architecture is to achieve consensus among multiple CAVs [30]. 3.4. Mixed traffic consideration A mixed traffic scenario refers to a traffic environment consisting of vehicles having various levels of connectivity and automation [32]. As a wider adoption of CAVs can be a slow process, considerable market penetration of CAVs is only expected to happen by 2040-2050 [33]. Thus, CAVs will coexist with HDVs in the foreseeable future [34] and consideration of mixed traffic is necessary to design a practical LC module. In mixed traffic, creating a perception of the surrounding environment can be a complex problem [3]. Furthermore, the performance analysis of the module in mixed traffic, with a variable penetration rate of CAVs, can provide a practical estimate of the minimum percentage of CAVs required for the module to perform effectively [20]. 4. Applications of AI for lane changing In recent publications, AI is widely used in various applications of ITS [11]. However, only a limited number of research works focus on using AI-based LC modules for CAVs. These LC modules can be categorised as Deep Reinforcement Learning (DRL), swarm intelligence, and federated learning. 4.1. Deep reinforcement learning Deep reinforcement learning (DRL) is a combination of deep learning and Reinforcement Learning (RL). Deep learning is a method to extract knowledge from a large input data using multilayer network of processing nodes, called neurons [9]. This network of neurons is widely known as Neural Networks (NN). The Convolution Neural Network (CNN), the Recurrent Neural Network (RNN), and the Graph Convolution Network (GCN) are some of the well- known examples of NNs used in deep learning [11]. RL is a method of learning through experience. Usually an RL defines a state space to represent possible states of the system, an action space to represent a set of possible actions, and a reward for an action in the action space. With repeated training, a given state can be mapped to the best action that maximises the reward. For a large state space and action space, RL would be inefficient, because the search tree would increase exponentially. To handle a large number of states and actions, RL can make use of deep learning to map states to the actions with the best possible reward. A CAV (agent) may have to consider a large state space and a large action space to make efficient lane changes to achieve a specific set of objectives. Thus, recent research works have used DRL to implement LC modules in CAVs. DRL can be a suitable option for CAV controllers as they are capable of learning from a dynamic environment with a large action space and state space [5]. Moreover, the DRL can be trained using simulations at lower costs. They can provide fast inference, scale easily, and outperform humans with instantaneous and reliable decision making capabilities [21]. For these reasons, DRL has been one of the popular choices to solve the challenges related to lane changing in CAVs, especially lane change decision making. Some of the research works have designed DRL-based modules for making lane change decisions, using various formulations of state space and action space [5, 21, 23]. Conversely, other methods have used DRL only for a sub-task within a complex LC module. The sub-task can be trajectory planning [23] or trajectory prediction of surrounding vehicles [24]. In general, applications of DRL for CAV lane changes can be broadly grouped based on the type of learning approach, such as Deep Q-Network (DQN) and Actor-Critic (AC) network. 4.1.1. Deep Q-Network DQNs can be applied to map a high-dimensional input space to a discrete action space, based on a policy [35]. This makes them suitable for high-level decision making for CAV lane changes (change left, right or stay in the same lane), as it might depend on a variety of inputs recorded from local sensors and surrounding vehicles [31]. DQNs have been successfully applied to CAVs lane changing, with reward functions that account for safety, mobility and comfort to achieve lane changing objectives [36, 26]. Although a DQN seems to be useful for lane change decision making modules, some challenges need to be addressed for effectively using it. One of the challenge can be dynamic state space of the CAVs, because of which DQN inputs can be of variable size [5]. DQNs, however, require inputs of fixed size. This challenge can be addressed by encoding the dynamic state space with variable length to a set of parameters with a fixed length. For example, Dong et al. used three NNs to encode each component of a dynamic state space, which contains the state of a CAV, the states of the surrounding vehicles and the states of the downstream vehicles [5]. This LC module, however, does not take advantage of the possibility of collaboration among CAVs. To enable collaboration between CAVs, Chen et al. uses Graph Convolution Networks (GCNs) to include topological information about traffic to make collaborative lane change decisions. The GCN is implemented in a centralised unit to encode dynamic input data and topological information to a set of fixed length parameters, which are used as input to a DQN [21]. On the other hand, a decentralised approach was used by Yu et al., for encoding the dynamic traffic topology as a Dynamic Coordination Graph (DCG) to achieve collaborative lane change decisions [26]. In summary, DQN-based LC modules can be a good option for single-step lane change decision making in CAVs. The implementations of DQN for CAV lane changing address the limitation of fixed length input and achieve coordination among CAVs using innovative methods. However, existing DQN implementations do not consider continuous controls such as acceleration, which could be an important factor, as it can be used to create appropriate gaps to allow collaborative lane changes [37]. 4.1.2. Actor-Critic Network The Actor-Critic Network (ACN) is an extension of DQN which implements the Actor-Critic (AC) algorithm [38]. The AC is a type of RL algorithm that consists of policy (actor) and value (critic) functions [39]. Policy functions use optimisation methods such as the Deterministic Policy Gradient (DPG) or the Deep DPG (DDPG) to estimate a policy in the continuous action space. Optimisation methods, however, suffer from high variance to estimate the gradient, as a result learning can be slow [39]. On the other hand, value functions use Temporal Difference (TD) learning to reduce variance in the expected return. Hence, the AC algorithm, which combines optimisation method and TD learning, can quickly converge to learn a policy for the continuous action space. Overall, ACNs can provide the combined advantage of AC algorithm and DQN to design a CAV controller, which can handle a large state space and a continuous action space. Existing ACN implementations aim to achieve a balance between the scalability of the LC module and cooperation among CAVs based on the requirements of the lane change scenario. Since ACNs allow learning a policy in a continuous action space, they can be used to adjust the continuous variables of CAV control, such as acceleration or speed, to enable cooperation between CAVs by creating the necessary gaps to allow safe lane changes. Cooperation among CAVs can be enabled by using a centralised LC controller, but this compromises scalability. Conversely, there is a good possibility to improve scalability with a decentralised LC controller, but a cooperation mechanism would have to be implemented explicitly. For example, an LC module can implement cooperation among CAVs by using a centralised ACN-based controller to adjust the speed of CAVs in a congested highway bottleneck [20]. Cooperation among CAVs would be necessary in a congested bottleneck scenario as vehicles need to create gaps that allow safe merging of vehicle into the main stream. Therefore, to enforce the cooperation among CAVs, a centralised solution can be an ideal option. However, cooperation can be induced among CAVs using a decentralised LC module. An example of a decentralised LC module was developed by Ren et al. for lane merging in a work zone section. This module uses ACN to adjust the acceleration of the CAV to allow cooperative lane changes in a work zone section [25]. Overall, for CAV lane changes in a work zone section or a bottleneck section, both centralised and decentralised architecture can be used to implement cooperation among CAVs with the ACN-based LC module. For lane changes on a highway or in a weaving section of the highway, a decentralised approach would enable an independent strategy for each vehicle [10]. An example of a de- centralised LC module for lane changes in a weaving section of a highway is the multi-agent DRL module proposed by Hou and Graf, which uses an ACN to make lane change decisions and speed adjustments to allow cooperation among vehicles [10]. This decentralised module relies on global state information to make its decisions. As global state information may need to be obtained from an external centralised system, it could compromise the scalability of the LC module. On the other hand, a shared ACN can also be used to implement cooperative lane change among CAVs, without compromising scalability. Zhou et al. proposed a cooperative and decentralised LC module [3]. This LC module uses a shared ACN to make lane change decisions and control vehicle speed. Furthermore, the module achieves cooperation and improved per- formance compared to the individual ACN implementation. Overall, ACN-based LC modules, designed mainly for MLC and DLC in highway traffic, can provide scalable cooperation. Although ACN-based LC modules provide some advantages compared to DQN-based LC modules, they suffer from some limitations. ACN-based LC modules provide various ways to implement cooperation among CAVs. Moreover, they consider lane changing scenario as well to choose the appropriate architecture for an LC module, such as centralised or decentralised. However, the ACN-based LC modules discussed above assume that lane change is executed in a single step, and consider the LC module as a single concrete module. These assumptions limit the possibility of including additional functionalities, such as planning the lane change trajectory, predicting the trajectory of other vehicles, or negotiating combined lane change trajectories to improve the performance of an LC module. To overcome these limitations, a modular lane change approach can be used [40]. In the modular lane change approach, the LC module can be a combination of different methods to achieve the best overall results. Such sub modules can have their own way of handling a specific task such as lane change decision making, trajectory planning or predicting the probable trajectory of other vehicles which might add additional benefits to improve the performance of the LC module. For example, Liao et al. proposed an online model to predict the possibility of lane changes by surrounding vehicles. This model is a combination of two sub-modules [24]. The first sub-module uses a Long-Short Term Memory(LSTM) network and the second sub-module uses Inverse Reinforcement Learning (IRL) to predict the trajectory of the vehicle. The predictions generated from this module can be used to improve the performance of the LC module. Another example that uses modular approach consists of a high-level Finite State Machine (FSM) module for lane change decision making and a low level ACN to perform safe lane changing manoeuvres [23]. In general, the modular approach seems to be a promising trend for AI-based LC modules as it opens up new dimensions to improve the efficiency of an LC module. 4.2. Swarm intelligence Swarm intelligence is a method to achieve collective intelligence in a group of things (in case of ITS they can be vehicles, infrastructure, or actuators) without a central controlling agent. Therefore, use of swarm intelligence in the V2X paradigm provides various advantages such as scalability, fault tolerance, adaptation, modularity, and autonomy of each agent [41]. Some examples of the swarm intelligence algorithm can be Particle Swarm Optimisation (PSO) to solve optimal point problems, Ant Colony Optimisation (ACO) for graph optimisation problems, and swarmcasting for distributed media sharing problems [9]. In the V2X paradigm, swarm intelligence can be applied to perform a collective task by all vehicles using the communication environment. Mostly, swarm intelligence is applied in communication technologies such as AntNet [9]. To our knowledge, swarm intelligence has not been applied to lane changing. However, swarm intelligence was used by Bang and Ahn to design a platooning strategy for CAVs [42]. The objectives of the platooning strategy are similar to the objectives of the LC modules, such as to improve traffic efficiency, safety and stability. In addition, the platooning strategy is based on longitudinal control of CAVs with simple formulations compared to learning-based modules. It could be interesting to investigate the possibility of using a similar swarm intelligence strategy for designing CAV lane changes. 4.3. Federated learning Federated learning is a fairly new branch of artificial intelligence that allows distributed training to create a global model. The agents can use the knowledge aggregated in a global model to make the best decisions in unseen situations. The key ideas behind federated learning are local computation and model transmission [43]. These ideas can reduce the privacy risks concerning local data. Moreover, federated learning can significantly reduce training time as the model can be trained in parallel using multiple agents. Wireless connectivity in CAVs can be leveraged to implement a federated learning-based CAV controller [44]. Using federated learning to design a CAV controller may have various advantages compared to traditional AI based controllers. Significant amounts of data are required to train traditional AI based controllers. On the other hand, a federated learning-based controller may depend on local data and updates from the global model, thus reducing storage requirements in a CAV. Moreover, a federated learning-based controller is expected to adapt well in various traffic environments [44]. For example, Zeng et al. used federated learning framework to effectively design a longitudinal control for CAVs to reduce accidents, road congestion, and improve traffic throughput. Even though the federated learning framework allows distributed training, it requires a central unit to aggregate the model updates from all agents. 5. Challenges and opportunities Although every LC module has specific advantages of their own, they suffer from some limita- tions. Some of these limitations can be observed from Table 1, which summarises the AI-based approaches discussed before, according to the dimensions presented in Section 3. To overcome these limitations, several challenges need to be addressed. This section provides a summary of these limitations and challenges, as well as possible considerations that can contribute to the development of efficient and practical lane-changing solutions for CAVs. From Table 1 we can observe some trends that highlight the limitations of AI-based LC modules along each dimension of this survey. Among objectives of LC modules, improving safety and mobility are the main objectives in most AI-based LC modules. In addition to these objectives, sustainability is also one of the main priorities of ITS [45]. However, only a limited number of LC modules have considered sustainability as their objective. Sustainability considerations such as energy utilisation and emissions from vehicles at a societal level may be significantly affected by the increase of CAVs in traffic. Therefore, sustainability considerations would be a valuable addition to the LC module and increase the chances of its acceptance in society. Table 1 AI-based CAV LC modules Lane change Mixed Reference Year Objectives AI method Architecture scenario Traffic safety Yu et al. 2020 DQN Discretionary Decentralised No mobility comfort Dong et al. 2021 safety DQN Discretionary Decentralised Yes comfort safety Graph NN + Chen et al. 2021 Mandatory Centralised Yes mobility DQN comfort Ha et al. 2020 safety GCN + ACN Bottleneck Centralised Yes mobility Ren et al. 2020 safety ACN Bottleneck Decentralised No mobility safety Zhou et al. 2021 ACN Discretionary Decentralised Yes mobility comfort Hou and Graf 2021 mobility ACN Mandatory Decentralised No sustainability safety Hybrid: Hwang et al. 2022 Discretionary Decentralised Yes mobility FSM + ACN safety Hierarchial: Liao et al. 2022 Mandatory Decentralised Yes mobility LSTM + IRL safety Swarm Bang and Ahn 2017 - Decentralised Yes mobility Intelligence safety Federated Zeng et al. 2021 - Decentralised Yes mobility Learning In terms of AI methods, most AI approaches to lane changing in CAVs use DRL. DRL, however, requires a significant amount of dedicated computing capacity for training and execution. Very few LC modules have provided the hardware specification of the machine on which the simulation was executed, and it has not been investigated whether the computing requirements are likely to be available in an individual CAV. Other emerging AI-based methods, such as swarm intelligence and federated learning, can potentially train high-quality controllers with minimal computing power requirements for individual CAVs. Swarm intelligence is currently used in ITS for network congestion control [9, 46] and for designing longitudinal control to create platoons with safe gaps to allow lane change by other vehicles [42], however, the application of swarm intelligence in lateral CAV control has not yet been evaluated. Similarly, federated learning can allow training a high-quality model with a distributed training mechanism [44], while preserving the privacy of individual CAVs. Current applications of federated learning in CAVs address only longitudinal CAV control, so its application to lateral CAV control needs to be investigated. Most of the LC modules presented in Table 1 are designed for a specific lane change scenario. Although some LC modules consider a generic approach, their evaluation considers only single or simplified traffic scenarios. In a real-world situation, a CAV may need to perform lane changes in different scenarios in a single journey. Therefore, a practical LC module needs to consider all possible scenarios of lane change in its design. This consideration can be implemented by using a generic LC module which can adapt to all scenarios of lane change. For the architecture of the LC modules, the decentralised architecture is a popular choice. This could be due to the high cost and time required to deploy the ITS infrastructure, which is necessary to support centralised LC modules [21]. The ITS infrastructure may include edge servers, roadside units, centralised servers, and V2I communication infrastructure. On the other hand, while decentralised architecture may not require any high-cost external infrastructure, establishing reliable coordination among CAVs is challenging. Most AI-based LC modules have considered operation in mixed traffic, though some have left it for future work [10]. Simulation of the mixed traffic scenario was modelled using the baseline car-following and lane changing models (MOBIL, LC2013) for HDVs in most cases. However, using the same standard driving model for HDVs may not reflect realistic mixed traffic. It is important to design a realistic mixed traffic scenario for simulation that can accurately predict the effect of CAV driving on the traffic [34]. Therefore, uncertainties must be considered in HDV models to create a realistic simulation environment with mixed traffic. Beyond these challenges, some assumptions of AI-based LC modules for CAVs may limit their applicability to practical solutions. Specifically, most of the LC modules surveyed in this paper assume the LC module to be a single concrete unit that can make lane change decisions and control acceleration. Moreover, they assume a lane change to complete in a single time step. In practical situations, however, a lane change is a complex task for CAVs, and might require the interaction of multiple independent processing modules, such as lane change decision making, trajectory planning, predicting changes in the environment, negotiating a lane change, etc. Therefore, a modular approach, which provides flexibility in developing a module in multiple dimensions, is likely to be more suitable for building a practical and realistic LC module. Acknowledgments The authors wish to thank the editors and anonymous reviewers for their valuable comments and helpful suggestions which greatly improved the paper’s quality. This work was supported by the SFI Centre for Research Training in Advanced Networks for Sustainable Societies (ADVANCE CRT), Ireland under the Grant number 18/CRT/6222. References [1] SMMT, Connected and Autonomous Vehicles Position Paper, Technical Re- port, SMMT, 2017. URL: https://www.smmt.co.uk/wp-content/uploads/sites/2/ SMMT-CAV-position-paper-final.pdf. [2] J. Wang, J. Liu, N. Kato, Networking and Communications in Autonomous Driving: A Survey, IEEE Communications Surveys Tutorials 21 (2019) 1243–1274. doi:10.1109/COMST. 2018.2888904 , conference Name: IEEE Communications Surveys Tutorials. [3] W. Zhou, D. Chen, J. Yan, Z. Li, H. Yin, W. Ge, Multi-agent Reinforcement Learning for Cooperative Lane Changing of Connected and Autonomous Vehicles in Mixed Traffic, arXiv:2111.06318 [cs] (2021). URL: http://arxiv.org/abs/2111.06318, arXiv: 2111.06318. [4] H. Yu, R. Jiang, Z. He, Z. Zheng, L. Li, R. Liu, X. Chen, Automated vehicle-involved traffic flow studies: A survey of assumptions, models, speculations, and perspectives, Transporta- tion Research Part C: Emerging Technologies 127 (2021) 103101. URL: https://linkinghub. elsevier.com/retrieve/pii/S0968090X21001224. doi:10.1016/j.trc.2021.103101 . [5] J. Dong, S. Chen, Y. Li, R. Du, A. Steinfeld, S. Labi, Space-weighted information fusion using deep reinforcement learning: The context of tactical control of lane-changing autonomous vehicles and connectivity range assessment, Transportation Research Part C: Emerging Technologies 128 (2021) 103192. URL: https://www.sciencedirect.com/science/article/pii/ S0968090X21002084. doi:10.1016/j.trc.2021.103192 . [6] Z. Wang, Y. Wu, Q. Niu, Multi-Sensor Fusion in Automated Driving: A Survey, IEEE Access 8 (2020) 2847–2868. doi:10.1109/ACCESS.2019.2962554 , conference Name: IEEE Access. [7] Y. Zheng, W. Ding, B. Ran, X. Qu, Y. Zhang, Coordinated decisions of discretionary lane change between connected and automated vehicles on freeways: a game theory- based lane change strategy, IET Intelligent Transport Systems 14 (2020) 1864–1870. URL: https://onlinelibrary.wiley.com/doi/abs/10.1049/iet-its.2020.0146. doi:10.1049/iet- its. 2020.0146 , _eprint: https://onlinelibrary.wiley.com/doi/pdf/10.1049/iet-its.2020.0146. [8] M. F. Keskin, B. Peng, B. Kulcsar, H. Wymeersch, Altruistic Control of Connected Auto- mated Vehicles in Mixed-Autonomy Multi-Lane Highway Traffic, IFAC-PapersOnLine 53 (2020) 14966–14971. URL: https://linkinghub.elsevier.com/retrieve/pii/S2405896320326215. doi:10.1016/j.ifacol.2020.12.1990 . [9] W. Tong, A. Hussain, W. X. Bo, S. Maharjan, Artificial Intelligence for Vehicle-to- Everything: A Survey, IEEE Access 7 (2019) 10823–10843. doi:10.1109/ACCESS.2019. 2891073 , conference Name: IEEE Access. [10] Y. Hou, P. Graf, Decentralized Cooperative Lane Changing at Freeway Weaving Areas Using Multi-Agent Deep Reinforcement Learning, arXiv:2110.08124 [cs] (2021). URL: http://arxiv.org/abs/2110.08124, arXiv: 2110.08124. [11] M. Veres, M. Moussa, Deep Learning for Intelligent Transportation Systems: A Survey of Emerging Trends, IEEE Transactions on Intelligent Transportation Systems 21 (2020) 3152–3168. doi:10.1109/TITS.2019.2929020 , conference Name: IEEE Transactions on Intelligent Transportation Systems. [12] Z. Wang, X. Shi, X. Li, Review of Lane-Changing Maneuvers of Connected and Automated Vehicles: Models, Algorithms and Traffic Impact Analyses, Journal of the Indian Institute of Science 99 (2019) 589–599. URL: https://doi.org/10.1007/s41745-019-00127-7. doi:10. 1007/s41745- 019- 00127- 7 . [13] P. G. Gipps, A model for the structure of lane-changing decisions, Transportation Research Part B: Methodological 20 (1986) 403–414. URL: https://www.sciencedirect.com/science/ article/pii/0191261586900123. doi:10.1016/0191- 2615(86)90012- 3 . [14] Q. Yang, H. N. Koutsopoulos, A Microscopic Traffic Simulator for evaluation of dynamic traf- fic management systems, Transportation Research Part C: Emerging Technologies 4 (1996) 113–129. URL: https://www.sciencedirect.com/science/article/pii/S0968090X9600006X. doi:10.1016/S0968- 090X(96)00006- X . [15] Z. Zheng, Recent developments and research needs in modeling lane changing, Transporta- tion Research Part B: Methodological 60 (2014) 16–32. URL: https://www.sciencedirect. com/science/article/pii/S019126151300218X. doi:10.1016/j.trb.2013.11.009 . [16] A. Kesting, M. Treiber, D. Helbing, General Lane-Changing Model MOBIL for Car- Following Models, Transportation Research Record 1999 (2007) 86–94. URL: https: //doi.org/10.3141/1999-10. doi:10.3141/1999- 10 , publisher: SAGE Publications Inc. [17] S. Ulbrich, M. Maurer, Towards Tactical Lane Change Behavior Planning for Automated Vehicles, in: 2015 IEEE 18th International Conference on Intelligent Transportation Systems, 2015, pp. 989–995. doi:10.1109/ITSC.2015.165 , iSSN: 2153-0017. [18] J. Erdmann, SUMO’s Lane-changing model, in: M. Behrisch, M. Weber (Eds.), LECTURE NOTES IN CONTROL AND INFORMATION SCIENCES, volume 13, Springer Verlag, Berlin, 2015, pp. 105–123. URL: http://link.springer.com/chapter/10.1007/978-3-319-15024-6_7. [19] J. He, K. Yang, H.-H. Chen, 6G Cellular Networks and Connected Autonomous Vehicles, arXiv:2010.00972 [cs, eess] (2020). URL: http://arxiv.org/abs/2010.00972, arXiv: 2010.00972. [20] P. Y. J. Ha, S. Chen, J. Dong, R. Du, Y. Li, S. Labi, Leveraging the Capabilities of Connected and Autonomous Vehicles and Multi-Agent Reinforcement Learning to Mitigate Highway Bottleneck Congestion, arXiv:2010.05436 [cs, eess] (2020). URL: http://arxiv.org/abs/2010. 05436, arXiv: 2010.05436. [21] S. Chen, J. Dong, P. Y. J. Ha, Y. Li, S. Labi, Graph neural network and reinforce- ment learning for multi-agent cooperative control of connected autonomous vehicles, Computer-Aided Civil and Infrastructure Engineering 36 (2021) 838–857. URL: http: //onlinelibrary.wiley.com/doi/abs/10.1111/mice.12702. doi:10.1111/mice.12702 , _eprint: https://onlinelibrary.wiley.com/doi/pdf/10.1111/mice.12702. [22] Z. Wang, X. Shi, X. Zhao, X. Li, Modeling decentralized mandatory lane change for connected and autonomous vehicles: An analytical method, Transportation Research Part C: Emerging Technologies 133 (2021) 103441. URL: https://www.sciencedirect.com/ science/article/pii/S0968090X21004319. doi:10.1016/j.trc.2021.103441 . [23] S. Hwang, K. Lee, H. Jeon, D. Kum, Autonomous Vehicle Cut-In Algorithm for Lane- Merging Scenarios via Policy-Based Reinforcement Learning Nested Within Finite- State Machine, IEEE Transactions on Intelligent Transportation Systems (2022) 1–13. URL: https://ieeexplore-ieee-org.elib.tcd.ie/document/9729796. doi:10.1109/TITS.2022. 3153848 , conference Name: IEEE Transactions on Intelligent Transportation Systems. [24] X. Liao, Z. Wang, X. Zhao, Z. Zhao, K. Han, P. Tiwari, M. Barth, G. Wu, Online Prediction of Lane Change with a Hierarchical Learning-Based Approach, 2022. [25] T. Ren, Y. Xie, L. Jiang, Cooperative Highway Work Zone Merge Control Based on Reinforcement Learning in a Connected and Automated Environment, Transportation Research Record 2674 (2020) 363–374. URL: https://doi.org/10.1177/0361198120935873. doi:10.1177/0361198120935873 , publisher: SAGE Publications Inc. [26] C. Yu, X. Wang, X. Xu, M. Zhang, H. Ge, J. Ren, L. Sun, B. Chen, G. Tan, Distributed Multiagent Coordinated Learning for Autonomous Driving in Highways Based on Dy- namic Coordination Graphs, IEEE Transactions on Intelligent Transportation Systems 21 (2020) 735–748. doi:10.1109/TITS.2019.2893683 , conference Name: IEEE Transactions on Intelligent Transportation Systems. [27] Y. Lin, P. Wang, M. Ma, Intelligent Transportation System(ITS): Concept, Challenge and Opportunity, in: 2017 ieee 3rd international conference on big data security on cloud (bigdatasecurity), ieee international conference on high performance and smart computing (hpsc), and ieee international conference on intelligent data and security (ids), 2017, pp. 167–172. doi:10.1109/BigDataSecurity.2017.50 . [28] B. Häfner, V. Bajpai, J. Ott, G. A. Schmitt, A Survey on Cooperative Architectures and Ma- neuvers for Connected and Automated Vehicles, IEEE Communications Surveys Tutorials (2021) 1–1. doi:10.1109/COMST.2021.3138275 , conference Name: IEEE Communications Surveys Tutorials. [29] Y. Yang, R. Luo, M. Li, M. Zhou, W. Zhang, J. Wang, Mean Field Multi-Agent Reinforcement Learning, in: Proceedings of the 35th International Conference on Machine Learning, PMLR, 2018, pp. 5571–5580. URL: https://proceedings.mlr.press/v80/yang18d.html, iSSN: 2640-3498. [30] P. Shi, B. Yan, A Survey on Intelligent Control for Multiagent Systems, IEEE Transac- tions on Systems, Man, and Cybernetics: Systems 51 (2021) 161–175. doi:10.1109/TSMC. 2020.3042823 , conference Name: IEEE Transactions on Systems, Man, and Cybernetics: Systems. [31] X. Liao, X. Zhao, Z. Wang, K. Han, P. Tiwari, M. J. Barth, G. Wu, Game Theory-Based Ramp Merging for Mixed Traffic With Unity-SUMO Co-Simulation, IEEE Transactions on Systems, Man, and Cybernetics: Systems (2021) 1–12. URL: https://doi-org.elib.tcd.ie/ 10.1109/TSMC.2021.3131431. doi:10.1109/TSMC.2021.3131431 , conference Name: IEEE Transactions on Systems, Man, and Cybernetics: Systems. [32] M. Shen, C. R. He, T. Molnar, A. H. Bell, G. Orosz, Energy-efficient Connected Cruise Control with Lean Penetration of Connected Vehicles, Technical Report arXiv:2205.03473, arXiv, 2022. URL: http://arxiv.org/abs/2205.03473, arXiv:2205.03473 [cs, eess] type: article. [33] Z. Vander Laan, P. Schonfeld, Modeling heterogeneous traffic with cooperative adap- tive cruise control vehicles: A first-order macroscopic perspective, Transportation Planning and Technology 43 (2020) 113–140. URL: https://doi.org/10.1080/03081060. 2020.1717127. doi:10.1080/03081060.2020.1717127 , publisher: Routledge _eprint: https://doi.org/10.1080/03081060.2020.1717127. [34] M. Garg, C. Johnston, M. Bouroche, Can Connected Autonomous Vehicles really improve mixed traffic efficiency in realistic scenarios?, in: 2021 IEEE International Intelligent Transportation Systems Conference (ITSC), 2021, pp. 2011–2018. doi:10.1109/ITSC48978. 2021.9565068 . [35] V. Mnih, K. Kavukcuoglu, D. Silver, A. A. Rusu, J. Veness, M. G. Bellemare, A. Graves, M. Riedmiller, A. K. Fidjeland, G. Ostrovski, S. Petersen, C. Beattie, A. Sadik, I. Antonoglou, H. King, D. Kumaran, D. Wierstra, S. Legg, D. Hassabis, Human-level control through deep reinforcement learning, Nature 518 (2015) 529–533. URL: http://www.nature.com/articles/ nature14236. doi:10.1038/nature14236 , number: 7540 Publisher: Nature Publishing Group. [36] J. Dong, S. Chen, Y. Li, P. Y. J. Ha, R. Du, A. Steinfeld, S. Labi, Spatio-weighted information fusion and DRL-based control for connected autonomous vehicles, in: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), 2020, pp. 1–6. doi:10.1109/ITSC45102.2020.9294550 . [37] Y. Ali, Z. Zheng, M. Mazharul Haque, M. Yildirimoglu, S. Washington, Understanding the discretionary lane-changing behaviour in the connected environment, Accident Analysis & Prevention 137 (2020) 105463. URL: https://www.sciencedirect.com/science/article/pii/ S0001457519312771. doi:10.1016/j.aap.2020.105463 . [38] T. P. Lillicrap, J. J. Hunt, A. Pritzel, N. Heess, T. Erez, Y. Tassa, D. Silver, D. Wierstra, Continuous control with deep reinforcement learning, arXiv:1509.02971 [cs, stat] (2019). URL: http://arxiv.org/abs/1509.02971, arXiv: 1509.02971. [39] I. Grondman, L. Busoniu, G. A. D. Lopes, R. Babuska, A Survey of Actor-Critic Re- inforcement Learning: Standard and Natural Policy Gradients, IEEE Transactions on Systems, Man, and Cybernetics, Part C (Applications and Reviews) 42 (2012) 1291–1307. doi:10.1109/TSMCC.2012.2218595 , conference Name: IEEE Transactions on Systems, Man, and Cybernetics, Part C (Applications and Reviews). [40] H. Xu, Y. Zhang, C. G. Cassandras, L. Li, S. Feng, A bi-level cooperative driving strategy allowing lane changes, Transportation Research Part C: Emerging Technologies 120 (2020) 102773. URL: https://www.sciencedirect.com/science/article/pii/S0968090X20306835. doi:10.1016/j.trc.2020.102773 . [41] I. Kassabalidis, M. El-Sharkawi, R. Marks, P. Arabshahi, A. Gray, Swarm intelligence for routing in communication networks, in: GLOBECOM’01. IEEE Global Telecommunications Conference (Cat. No.01CH37270), volume 6, 2001, pp. 3613–3617 vol.6. doi:10.1109/ GLOCOM.2001.966355 . [42] S. Bang, S. Ahn, Platooning Strategy for Connected and Autonomous Vehicles: Transition from Light Traffic, Transportation Research Record 2623 (2017) 73–81. URL: https://doi. org/10.3141/2623-08. doi:10.3141/2623- 08 , publisher: SAGE Publications Inc. [43] C. Zhang, Y. Xie, H. Bai, B. Yu, W. Li, Y. Gao, A survey on federated learning, Knowledge- Based Systems 216 (2021) 106775. URL: https://www.sciencedirect.com/science/article/pii/ S0950705121000381. doi:10.1016/j.knosys.2021.106775 . [44] T. Zeng, O. Semiari, M. Chen, W. Saad, M. Bennis, Federated Learning on the Road: Autonomous Controller Design for Connected and Autonomous Vehicles, arXiv:2102.03401 [cs, eess] (2021). URL: http://arxiv.org/abs/2102.03401, arXiv: 2102.03401. [45] L. Zhao, Y. Jia, Intelligent transportation system for sustainable environment in smart cities, The International Journal of Electrical Engineering & Education (2021) 0020720920983503. URL: https://doi.org/10.1177/0020720920983503. doi:10.1177/0020720920983503 , pub- lisher: SAGE Publications Ltd STM. [46] H. Ma, S. Li, E. Zhang, Z. Lv, J. Hu, X. Wei, Cooperative Autonomous Driving Oriented MEC-Aided 5G-V2X: Prototype System Design, Field Tests and AI-Based Optimization Tools, IEEE Access 8 (2020) 54288–54302. doi:10.1109/ACCESS.2020.2981463 , conference Name: IEEE Access.