=Paper=
{{Paper
|id=Vol-3173/paper7
|storemode=property
|title=Design of AI-Based Lane Changing Models in Connected and Autonomous Vehicles: a Survey
|pdfUrl=https://ceur-ws.org/Vol-3173/7.pdf
|volume=Vol-3173
|authors=Bharathkumar Hegde,Melanie Bouroche
|dblpUrl=https://dblp.org/rec/conf/ijcai/HegdeB22
}}
==Design of AI-Based Lane Changing Models in Connected and Autonomous Vehicles: a Survey==
https://ceur-ws.org/Vol-3173/7.pdf
Design of AI-based lane changing modules in
connected and autonomous vehicles: a survey
Bharathkumar Hegde1 , Melanie Bouroche1
1
Trinity College Dublin, Dublin, Ireland
Abstract
Lane changing is one of the complex driving tasks as it requires the vehicle to be aware of its highly-
dynamic surrounding environment, make decisions, and enact them in a timely manner. By exploiting
both sensors and inter-vehicle communication, Connected and Autonomous Vehicles (CAVs) have the
potential to significantly improve lane changing safety and efficiency. The complexity of the task and
the real-time requirements make lane-changing a problem particularly suited to Artificial Intelligence
(AI) approaches. In this paper, we survey the design of AI-based Lane-Changing(LC) modules for CAVs.
First, we identify the key factors that can influence the design of an LC module. Next, we survey recent
developments in AI-based lane changing. Finally, we analyse these approaches along the dimensions of
the key influencing factors and summarise the challenges that are yet to be addressed and opportunities
that can guide the future developments in AI-based LC modules.
Keywords
Connected and autonomous vehicle (CAV), Lane change, Artificial intelligence, Deep learning (DL),
Intelligent transportation system (ITS)
1. Introduction
Autonomous Vehicles (AVs) are one of the major components of a rapidly developing Intelligent
Transportation System (ITS). Developments in the communication technology are expected to
complement the development of the AV technology. Therefore, advancements in Connected
and Autonomous Vehicles (CAVs) are expected to improve the performance in the driving tasks
required for achieving autonomy of level of 3 and above [1]. Currently, in the commercial
market, Tesla Model S has achieved an autonomy level of 2.5 and the Audi A-8 has achieved level
3 autonomy in driving [2] by automating major driving tasks. Whereas, a fully autonomous
vehicle (SAE’s level 5 autonomy) should be capable of performing all driving tasks safely and
efficiently in all kinds of environment. Among the driving tasks, lane changing is one of the
complex tasks for CAVs and a challenging problem for researchers [3].
By planning and coordinating lane changes, CAVs might be able to improve the traffic flow
at both microscopic and macroscopic level. The macroscopic traffic level benefits may include
increased safety, traffic efficiency, and road capacity [4] and the microscopic traffic level benefits
may include increased comfort for travellers with minimal speed variation and reduced travel
ATT’22: Workshop Agents in Traffic and Transportation, July 25, 2022, Vienna, Austria
Envelope-Open hegdeb@tcd.ie (B. Hegde); melanie.bouroche@tcd.ie (M. Bouroche)
GLOBE https://www.scss.tcd.ie/Melanie.Bouroche/ (M. Bouroche)
Orcid 0000-0002-2085-7867 (B. Hegde); 0000-0002-5039-0815 (M. Bouroche)
© 2022 Copyright for this paper by its authors. Use permitted under Creative Commons License Attribution 4.0 International (CC BY 4.0).
CEUR
Workshop
Proceedings
http://ceur-ws.org
ISSN 1613-0073
CEUR Workshop Proceedings (CEUR-WS.org)
delay [5]. To plan a lane changing manoeuvre, CAVs use information collected by sensors and
other vehicles in a highly-dynamic environment [6]. An LC module uses this information to
make lane change decisions. Some of the popular methods used to design an LC module are
game theory [7], controller optimisation [8], and AI [9].
Recently, AI has been used more often to design LC modules, as the recent developments
in AI have proven useful to make fast decisions in dynamic environments with a large set of
parameters. Real-world traffic is very dynamic and a large set of parameters may be considered
to perform a lane change. Parameters may include the position, speed, and heading of the
ego vehicle and surrounding vehicles [10]. Moreover, an LC module needs to make intelligent
trade-offs to improve the possibility of achieving safety along with other objectives of lane
changes, such as improving mobility, comfort of travel, fuel efficiency, and reducing emissions.
Therefore, AI is a promising option for designing LC modules, which can make efficient lane
change decisions in complex traffic environments to achieve multiple objectives.
Most survey papers have reviewed the application of AI in wider fields such as ITS [11],
CAVs [12], or V2X communication [9]. Conversely, this paper focuses on the design of AI-based
modules for CAV lane changing controllers. Specifically, the main contributions of this work
are:
• Identify the key factors that can influence the design of an AI-based LC module.
• Provide insights into the recent developments in the design of AI-based LC module along
the dimensions of the key influencing factors.
• Summarise the challenges and opportunities in the design of AI-based LC modules.
Challenges provide the research gaps that are yet to be addressed, and opportunities
identify the possibilities that can guide the future developments of AI-based LC modules.
The remainder of the paper is structured as follows. Section 2 provides background details
related to the development of lane change and the architecture of lane changing in CAVs.
Section 3 presents the key factors that influence the design of AI-based LC modules. A review
of AI-based LC modules is presented in Section 4. Finally, Section 5 reviews the approaches
discussed before summarising the challenges and possible opportunities in the design of AI-based
lane changing modules.
2. Background
This section discusses the development of lane changing models and a general architecture of a
CAV lane change. The development of lane changing models lists some of the standard lane
changing models used for traffic simulations. The general architecture of a CAV lane change
provides its components and describes how they are related.
2.1. Development of lane changing models
An LC model usually encodes a rational decision to change lanes based on various parameters
that describe the environment around a vehicle. The first known LC model is the Gipps lane
changing model [13]. The Gipps model is based on maintaining a desired speed and being in
the correct lane for an upcoming desired manoeuvre. LC models developed based on the Gipps
model are classified as Gipps-type LC models. To overcome the limitations of Gipps-type LC
models, which were deterministic, rule-based models were proposed [14]. Rule-based models
consist of a decision process defined in four steps: decision to consider a lane change, choice of
the target lane, search for an acceptable gap, and executing the lane change [15]. Considering
the probabilistic approach for lane changing decision instead of deterministic lane changing
decision as in Gipps type LC model, was one of the distinct features that made the rule-based
model more realistic [15].
While the Gipps-type LC models consider only the vehicle speed, Kesting et al. proposed a
novel incentive-based lane changing model, MOBIL (Minimising Overall Braking Induced by
Lane change)[16], which considers the acceleration of the vehicles as well [15]. The MOBIL
model makes a lane change decision based on the probability of advantages and disadvantages
of the lane change, based on the accelerations of the vehicles. In addition to the acceleration,
MOBIL model also considers factors such as politeness and the right-left lane bias (which
restricts overtaking from the right side, eg in Germany,) [15]. These considerations enable easy
integration of MOBIL with car-following models such as the Intelligent Driver Model (IDM)
[17].
A lane changing model, named LC2013, considers the intention of changing lanes using
a decision-tree algorithm [18]. The intention for a lane change can be to reach a specific
destination, to overtake a slow vehicle, to cooperate with other vehicles, or to follow local
traffic regulations. In LC2013 LC model, CAVs coordinate by sharing their intentions during
lane changing manoeuvres. The LC2013 model is integrated in the Simulation of Urban MO-
bility (SUMO) simulation framework and allows customisation to simulate regulatory traffic
restrictions, such as the restriction on overtaking from the right side as in Germany.
To conclude, the models discussed above have been used as standard lane changing models
in popular traffic simulators and as a baseline to validate recent LC modules. These standard LC
models are designed to achieve a single objective, that is, to make a safe lane change decision.
However, recent AI-based LC modules aim to achieve safety along with other objectives such
as improving mobility, comfort, and fuel efficiency. A detailed discussion of recent AI-based LC
modules is presented in Section 4.
2.2. Architecture of CAV lane changing
The architecture of CAV lane changing typically consist of four major components, namely
perception, communication, lane changing, and vehicular control as shown in Figure 1 [12].
The perception module creates a perception of the environment around the vehicle by com-
bining the inputs from various sensors such as LiDAR, RADAR, camera, GPS, IMU, etc. The
vehicle-to-everything (V2X) communication module provides interfaces to communicate with
other components of the Intelligent Transport System (ITS), such as other vehicles, road side
unit (RSU), mobile edge computing (MEC) server, cloud server, etc [19]. The lane changing
module integrates inputs from the perception module and information collected through the
communication module to make a lane changing decision and plan trajectories for the execution
of lane changes [12]. The lane changing module can be implemented using either a centralised
[20, 21] or a decentralised [10, 22] architecture. The Vehicular control module includes the
Figure 1: Architecture of CAV lane changing
lateral and longitudinal control of the vehicle which executes the instructions from the lane
change module [2].
3. Dimensions of the survey
In this survey, we explore the design of AI-based LC modules for CAVs along four dimensions.
These dimensions are objectives of the lane change, lane change scenarios, architecture, and mixed
traffic consideration. They are some of the key factors that influence the design of AI-based LC
modules implemented in CAVs.
3.1. Lane changing objective
The objectives of lane changing can be broadly classified as safety, mobility, comfort, and
sustainability. Safety is one of the major objectives considered in LC modules. Safety objectives
mainly focus on lane changing with minimal risk to avoid collisions [5, 4, 3, 23, 24]. However, a
CAV should not compromise on mobility, while trying to improve safety. Mobility objectives
consider improving traffic throughput [10, 25] and average speed [21] of the vehicle, and avoiding
stop-go traffic [25]. To improve mobility, a CAV can make unnecessary lane change manoeuvres
[5] or accelerate and decelerate frequently, causing discomfort to passengers [24, 26]. Therefore,
achieving travel comfort is another potential objective of LC modules. Furthermore, some of
the LC modules consider sustainability [27] as one of their objective, as lane changes can also
affect overall fuel efficiency and emissions [10].
3.2. Lane change scenario
The lane change scenario can be defined based on the motive of a vehicle to perform a lane
change. The motive to change lane can be broadly categorised as discretionary lane change,
mandatory lane change, and lane change in bottleneck sections. Dynamics of vehicle movement
and the parameters considered for the lane change decision making differ for each of these
categories of lane change. Hence, the lane change scenario can be one of the factors to consider
while designing an LC module.
An optional lane change by a vehicle, for the benefit of its own or other vehicles in traffic,
is considered a Discretionary Lane Change (DLC). DLCs often result in increased speed for the
ego vehicles, and they may have various positive impacts on the traffic at the macroscopic
level, such as increasing road capacity, increasing traffic throughput, minimising traffic jam
propagation, etc. DLCs focus primarily on safety and achieving macroscopic objectives like
increased driving comfort, mobility, or throughput [5, 7, 26].
On occasions, a vehicle may be required to change the lane to reach a desired destination;
such lane changes are classified as Mandatory Lane Changes (MLCs). Some examples of MLCs
include changing lane to enter a highway, exit a highway, or before reaching an intersection
for a turn. Since lane change is mandatory in these cases, the vehicle may need to execute a
risky lane change, especially in high traffic. MLC by a vehicle may affect the other vehicles
in traffic, therefore, the ideal MLC controller should be capable of ensuring safety even under
risky situations and it should have minimal negative impact on the mainstream traffic flow of
the highway [22, 21, 10].
Similar to an MLC, a vehicle will be changing lanes when the current lane reaches a dead
end or merges into an adjacent lane. Such lane changes can be categorised as lane change in
bottleneck sections. In bottleneck sections, coordination among vehicles plays a key role as
the vehicles changing lane will interrupt the main traffic flow. A bottleneck may be created
because of construction works, reduced road space, vehicle broken-down, or accidents. Hence,
bottleneck sections are often not observed in advance. Therefore, lane changes in bottleneck
sections may need to be handled differently compared to an MLC. Typically, the LC modules
for bottleneck sections aim to achieve a smooth traffic flow with less congestion, and increase
traffic throughput by avoiding stop-go traffic [20, 25].
3.3. Architecture
The architecture of an LC module can be classified as centralised or decentralised, depending
on its placement within the CAV lane change architecture. In a centralised architecture, the LC
module can be placed in an RSU, an edge server, or a cloud server which can be a centralised
controller. A centralised controller can integrate the information from traffic participants and
use it for trajectory planning and lane changing decisions. Furthermore, the controller may
suggest state changes, such as path, velocity, etc. [28] or lane change decisions to a CAV [21].
Centralised controllers have been found to achieve better results in completing cooperative
objectives [29]. However, one disadvantage of the centralised architecture is the challenge of
scaling the central server based on the variation in the traffic flow. Additionally, it adds an extra
overhead for installing and maintaining a wide scale infrastructure. As the central entity can
be a bottleneck of the system, the centralised infrastructure is prone to failures and network
congestion.
On the other hand, the decentralised architecture can be implemented by placing the LC
module in individual CAVs. The LC module along with the perception module and the vehic-
ular control module can form an autonomous controller. These autonomous controllers can
communicate through a direct V2V communication interface or through the network infras-
tructure to collect the information needed from other CAVs for trajectory planning and lane
change decisions [30]. The necessary information required from other CAVs may include state
information, trajectory plan, traffic information, etc. An LC module can collect this information
and act independently or interact with other CAVs to achieve a cooperative decision [28]. In
a decentralised architecture, lane change execution can be faster, as the LC module and the
vehicular control module are placed in the same CAV [31]. This approach can significantly
improve the scalability of the module based on traffic demands. Furthermore, ITS infrastructure,
such as RSU or MEC servers, can be used to offload some of the resource-intensive computations
[28]. However, one of the key challenges in a decentralised architecture is to achieve consensus
among multiple CAVs [30].
3.4. Mixed traffic consideration
A mixed traffic scenario refers to a traffic environment consisting of vehicles having various
levels of connectivity and automation [32]. As a wider adoption of CAVs can be a slow process,
considerable market penetration of CAVs is only expected to happen by 2040-2050 [33]. Thus,
CAVs will coexist with HDVs in the foreseeable future [34] and consideration of mixed traffic
is necessary to design a practical LC module. In mixed traffic, creating a perception of the
surrounding environment can be a complex problem [3]. Furthermore, the performance analysis
of the module in mixed traffic, with a variable penetration rate of CAVs, can provide a practical
estimate of the minimum percentage of CAVs required for the module to perform effectively
[20].
4. Applications of AI for lane changing
In recent publications, AI is widely used in various applications of ITS [11]. However, only a
limited number of research works focus on using AI-based LC modules for CAVs. These LC
modules can be categorised as Deep Reinforcement Learning (DRL), swarm intelligence, and
federated learning.
4.1. Deep reinforcement learning
Deep reinforcement learning (DRL) is a combination of deep learning and Reinforcement
Learning (RL). Deep learning is a method to extract knowledge from a large input data using
multilayer network of processing nodes, called neurons [9]. This network of neurons is widely
known as Neural Networks (NN). The Convolution Neural Network (CNN), the Recurrent
Neural Network (RNN), and the Graph Convolution Network (GCN) are some of the well-
known examples of NNs used in deep learning [11]. RL is a method of learning through
experience. Usually an RL defines a state space to represent possible states of the system, an
action space to represent a set of possible actions, and a reward for an action in the action
space. With repeated training, a given state can be mapped to the best action that maximises
the reward. For a large state space and action space, RL would be inefficient, because the search
tree would increase exponentially. To handle a large number of states and actions, RL can make
use of deep learning to map states to the actions with the best possible reward. A CAV (agent)
may have to consider a large state space and a large action space to make efficient lane changes
to achieve a specific set of objectives. Thus, recent research works have used DRL to implement
LC modules in CAVs.
DRL can be a suitable option for CAV controllers as they are capable of learning from a
dynamic environment with a large action space and state space [5]. Moreover, the DRL can
be trained using simulations at lower costs. They can provide fast inference, scale easily, and
outperform humans with instantaneous and reliable decision making capabilities [21]. For
these reasons, DRL has been one of the popular choices to solve the challenges related to lane
changing in CAVs, especially lane change decision making.
Some of the research works have designed DRL-based modules for making lane change
decisions, using various formulations of state space and action space [5, 21, 23]. Conversely,
other methods have used DRL only for a sub-task within a complex LC module. The sub-task
can be trajectory planning [23] or trajectory prediction of surrounding vehicles [24]. In general,
applications of DRL for CAV lane changes can be broadly grouped based on the type of learning
approach, such as Deep Q-Network (DQN) and Actor-Critic (AC) network.
4.1.1. Deep Q-Network
DQNs can be applied to map a high-dimensional input space to a discrete action space, based
on a policy [35]. This makes them suitable for high-level decision making for CAV lane changes
(change left, right or stay in the same lane), as it might depend on a variety of inputs recorded
from local sensors and surrounding vehicles [31]. DQNs have been successfully applied to CAVs
lane changing, with reward functions that account for safety, mobility and comfort to achieve
lane changing objectives [36, 26].
Although a DQN seems to be useful for lane change decision making modules, some challenges
need to be addressed for effectively using it. One of the challenge can be dynamic state space of
the CAVs, because of which DQN inputs can be of variable size [5]. DQNs, however, require
inputs of fixed size. This challenge can be addressed by encoding the dynamic state space with
variable length to a set of parameters with a fixed length. For example, Dong et al. used three
NNs to encode each component of a dynamic state space, which contains the state of a CAV, the
states of the surrounding vehicles and the states of the downstream vehicles [5]. This LC module,
however, does not take advantage of the possibility of collaboration among CAVs. To enable
collaboration between CAVs, Chen et al. uses Graph Convolution Networks (GCNs) to include
topological information about traffic to make collaborative lane change decisions. The GCN is
implemented in a centralised unit to encode dynamic input data and topological information to
a set of fixed length parameters, which are used as input to a DQN [21]. On the other hand, a
decentralised approach was used by Yu et al., for encoding the dynamic traffic topology as a
Dynamic Coordination Graph (DCG) to achieve collaborative lane change decisions [26].
In summary, DQN-based LC modules can be a good option for single-step lane change decision
making in CAVs. The implementations of DQN for CAV lane changing address the limitation of
fixed length input and achieve coordination among CAVs using innovative methods. However,
existing DQN implementations do not consider continuous controls such as acceleration, which
could be an important factor, as it can be used to create appropriate gaps to allow collaborative
lane changes [37].
4.1.2. Actor-Critic Network
The Actor-Critic Network (ACN) is an extension of DQN which implements the Actor-Critic
(AC) algorithm [38]. The AC is a type of RL algorithm that consists of policy (actor) and value
(critic) functions [39]. Policy functions use optimisation methods such as the Deterministic
Policy Gradient (DPG) or the Deep DPG (DDPG) to estimate a policy in the continuous action
space. Optimisation methods, however, suffer from high variance to estimate the gradient, as a
result learning can be slow [39]. On the other hand, value functions use Temporal Difference
(TD) learning to reduce variance in the expected return. Hence, the AC algorithm, which
combines optimisation method and TD learning, can quickly converge to learn a policy for the
continuous action space. Overall, ACNs can provide the combined advantage of AC algorithm
and DQN to design a CAV controller, which can handle a large state space and a continuous
action space.
Existing ACN implementations aim to achieve a balance between the scalability of the LC
module and cooperation among CAVs based on the requirements of the lane change scenario.
Since ACNs allow learning a policy in a continuous action space, they can be used to adjust
the continuous variables of CAV control, such as acceleration or speed, to enable cooperation
between CAVs by creating the necessary gaps to allow safe lane changes. Cooperation among
CAVs can be enabled by using a centralised LC controller, but this compromises scalability.
Conversely, there is a good possibility to improve scalability with a decentralised LC controller,
but a cooperation mechanism would have to be implemented explicitly.
For example, an LC module can implement cooperation among CAVs by using a centralised
ACN-based controller to adjust the speed of CAVs in a congested highway bottleneck [20].
Cooperation among CAVs would be necessary in a congested bottleneck scenario as vehicles
need to create gaps that allow safe merging of vehicle into the main stream. Therefore, to
enforce the cooperation among CAVs, a centralised solution can be an ideal option. However,
cooperation can be induced among CAVs using a decentralised LC module. An example of a
decentralised LC module was developed by Ren et al. for lane merging in a work zone section.
This module uses ACN to adjust the acceleration of the CAV to allow cooperative lane changes in
a work zone section [25]. Overall, for CAV lane changes in a work zone section or a bottleneck
section, both centralised and decentralised architecture can be used to implement cooperation
among CAVs with the ACN-based LC module.
For lane changes on a highway or in a weaving section of the highway, a decentralised
approach would enable an independent strategy for each vehicle [10]. An example of a de-
centralised LC module for lane changes in a weaving section of a highway is the multi-agent
DRL module proposed by Hou and Graf, which uses an ACN to make lane change decisions
and speed adjustments to allow cooperation among vehicles [10]. This decentralised module
relies on global state information to make its decisions. As global state information may need
to be obtained from an external centralised system, it could compromise the scalability of the
LC module. On the other hand, a shared ACN can also be used to implement cooperative lane
change among CAVs, without compromising scalability. Zhou et al. proposed a cooperative and
decentralised LC module [3]. This LC module uses a shared ACN to make lane change decisions
and control vehicle speed. Furthermore, the module achieves cooperation and improved per-
formance compared to the individual ACN implementation. Overall, ACN-based LC modules,
designed mainly for MLC and DLC in highway traffic, can provide scalable cooperation.
Although ACN-based LC modules provide some advantages compared to DQN-based LC
modules, they suffer from some limitations. ACN-based LC modules provide various ways to
implement cooperation among CAVs. Moreover, they consider lane changing scenario as well
to choose the appropriate architecture for an LC module, such as centralised or decentralised.
However, the ACN-based LC modules discussed above assume that lane change is executed
in a single step, and consider the LC module as a single concrete module. These assumptions
limit the possibility of including additional functionalities, such as planning the lane change
trajectory, predicting the trajectory of other vehicles, or negotiating combined lane change
trajectories to improve the performance of an LC module.
To overcome these limitations, a modular lane change approach can be used [40]. In the
modular lane change approach, the LC module can be a combination of different methods
to achieve the best overall results. Such sub modules can have their own way of handling a
specific task such as lane change decision making, trajectory planning or predicting the probable
trajectory of other vehicles which might add additional benefits to improve the performance
of the LC module. For example, Liao et al. proposed an online model to predict the possibility
of lane changes by surrounding vehicles. This model is a combination of two sub-modules
[24]. The first sub-module uses a Long-Short Term Memory(LSTM) network and the second
sub-module uses Inverse Reinforcement Learning (IRL) to predict the trajectory of the vehicle.
The predictions generated from this module can be used to improve the performance of the
LC module. Another example that uses modular approach consists of a high-level Finite State
Machine (FSM) module for lane change decision making and a low level ACN to perform safe
lane changing manoeuvres [23]. In general, the modular approach seems to be a promising
trend for AI-based LC modules as it opens up new dimensions to improve the efficiency of an
LC module.
4.2. Swarm intelligence
Swarm intelligence is a method to achieve collective intelligence in a group of things (in case
of ITS they can be vehicles, infrastructure, or actuators) without a central controlling agent.
Therefore, use of swarm intelligence in the V2X paradigm provides various advantages such
as scalability, fault tolerance, adaptation, modularity, and autonomy of each agent [41]. Some
examples of the swarm intelligence algorithm can be Particle Swarm Optimisation (PSO) to
solve optimal point problems, Ant Colony Optimisation (ACO) for graph optimisation problems,
and swarmcasting for distributed media sharing problems [9].
In the V2X paradigm, swarm intelligence can be applied to perform a collective task by
all vehicles using the communication environment. Mostly, swarm intelligence is applied in
communication technologies such as AntNet [9]. To our knowledge, swarm intelligence has
not been applied to lane changing. However, swarm intelligence was used by Bang and Ahn
to design a platooning strategy for CAVs [42]. The objectives of the platooning strategy are
similar to the objectives of the LC modules, such as to improve traffic efficiency, safety and
stability. In addition, the platooning strategy is based on longitudinal control of CAVs with
simple formulations compared to learning-based modules. It could be interesting to investigate
the possibility of using a similar swarm intelligence strategy for designing CAV lane changes.
4.3. Federated learning
Federated learning is a fairly new branch of artificial intelligence that allows distributed training
to create a global model. The agents can use the knowledge aggregated in a global model to
make the best decisions in unseen situations. The key ideas behind federated learning are local
computation and model transmission [43]. These ideas can reduce the privacy risks concerning
local data. Moreover, federated learning can significantly reduce training time as the model can
be trained in parallel using multiple agents.
Wireless connectivity in CAVs can be leveraged to implement a federated learning-based
CAV controller [44]. Using federated learning to design a CAV controller may have various
advantages compared to traditional AI based controllers. Significant amounts of data are
required to train traditional AI based controllers. On the other hand, a federated learning-based
controller may depend on local data and updates from the global model, thus reducing storage
requirements in a CAV. Moreover, a federated learning-based controller is expected to adapt well
in various traffic environments [44]. For example, Zeng et al. used federated learning framework
to effectively design a longitudinal control for CAVs to reduce accidents, road congestion, and
improve traffic throughput. Even though the federated learning framework allows distributed
training, it requires a central unit to aggregate the model updates from all agents.
5. Challenges and opportunities
Although every LC module has specific advantages of their own, they suffer from some limita-
tions. Some of these limitations can be observed from Table 1, which summarises the AI-based
approaches discussed before, according to the dimensions presented in Section 3. To overcome
these limitations, several challenges need to be addressed. This section provides a summary of
these limitations and challenges, as well as possible considerations that can contribute to the
development of efficient and practical lane-changing solutions for CAVs.
From Table 1 we can observe some trends that highlight the limitations of AI-based LC
modules along each dimension of this survey. Among objectives of LC modules, improving
safety and mobility are the main objectives in most AI-based LC modules. In addition to
these objectives, sustainability is also one of the main priorities of ITS [45]. However, only a
limited number of LC modules have considered sustainability as their objective. Sustainability
considerations such as energy utilisation and emissions from vehicles at a societal level may be
significantly affected by the increase of CAVs in traffic. Therefore, sustainability considerations
would be a valuable addition to the LC module and increase the chances of its acceptance in
society.
Table 1
AI-based CAV LC modules
Lane change Mixed
Reference Year Objectives AI method Architecture
scenario Traffic
safety
Yu et al. 2020 DQN Discretionary Decentralised No
mobility
comfort
Dong et al. 2021 safety DQN Discretionary Decentralised Yes
comfort
safety Graph NN +
Chen et al. 2021 Mandatory Centralised Yes
mobility DQN
comfort
Ha et al. 2020 safety GCN + ACN Bottleneck Centralised Yes
mobility
Ren et al. 2020 safety ACN Bottleneck Decentralised No
mobility
safety
Zhou et al. 2021 ACN Discretionary Decentralised Yes
mobility
comfort
Hou and Graf 2021 mobility ACN Mandatory Decentralised No
sustainability
safety Hybrid:
Hwang et al. 2022 Discretionary Decentralised Yes
mobility FSM + ACN
safety Hierarchial:
Liao et al. 2022 Mandatory Decentralised Yes
mobility LSTM + IRL
safety Swarm
Bang and Ahn 2017 - Decentralised Yes
mobility Intelligence
safety Federated
Zeng et al. 2021 - Decentralised Yes
mobility Learning
In terms of AI methods, most AI approaches to lane changing in CAVs use DRL. DRL, however,
requires a significant amount of dedicated computing capacity for training and execution.
Very few LC modules have provided the hardware specification of the machine on which the
simulation was executed, and it has not been investigated whether the computing requirements
are likely to be available in an individual CAV. Other emerging AI-based methods, such as swarm
intelligence and federated learning, can potentially train high-quality controllers with minimal
computing power requirements for individual CAVs. Swarm intelligence is currently used in
ITS for network congestion control [9, 46] and for designing longitudinal control to create
platoons with safe gaps to allow lane change by other vehicles [42], however, the application
of swarm intelligence in lateral CAV control has not yet been evaluated. Similarly, federated
learning can allow training a high-quality model with a distributed training mechanism [44],
while preserving the privacy of individual CAVs. Current applications of federated learning in
CAVs address only longitudinal CAV control, so its application to lateral CAV control needs to
be investigated.
Most of the LC modules presented in Table 1 are designed for a specific lane change scenario.
Although some LC modules consider a generic approach, their evaluation considers only single
or simplified traffic scenarios. In a real-world situation, a CAV may need to perform lane changes
in different scenarios in a single journey. Therefore, a practical LC module needs to consider all
possible scenarios of lane change in its design. This consideration can be implemented by using
a generic LC module which can adapt to all scenarios of lane change.
For the architecture of the LC modules, the decentralised architecture is a popular choice.
This could be due to the high cost and time required to deploy the ITS infrastructure, which is
necessary to support centralised LC modules [21]. The ITS infrastructure may include edge
servers, roadside units, centralised servers, and V2I communication infrastructure. On the other
hand, while decentralised architecture may not require any high-cost external infrastructure,
establishing reliable coordination among CAVs is challenging.
Most AI-based LC modules have considered operation in mixed traffic, though some have left
it for future work [10]. Simulation of the mixed traffic scenario was modelled using the baseline
car-following and lane changing models (MOBIL, LC2013) for HDVs in most cases. However,
using the same standard driving model for HDVs may not reflect realistic mixed traffic. It is
important to design a realistic mixed traffic scenario for simulation that can accurately predict
the effect of CAV driving on the traffic [34]. Therefore, uncertainties must be considered in
HDV models to create a realistic simulation environment with mixed traffic.
Beyond these challenges, some assumptions of AI-based LC modules for CAVs may limit their
applicability to practical solutions. Specifically, most of the LC modules surveyed in this paper
assume the LC module to be a single concrete unit that can make lane change decisions and
control acceleration. Moreover, they assume a lane change to complete in a single time step. In
practical situations, however, a lane change is a complex task for CAVs, and might require the
interaction of multiple independent processing modules, such as lane change decision making,
trajectory planning, predicting changes in the environment, negotiating a lane change, etc.
Therefore, a modular approach, which provides flexibility in developing a module in multiple
dimensions, is likely to be more suitable for building a practical and realistic LC module.
Acknowledgments
The authors wish to thank the editors and anonymous reviewers for their valuable comments and
helpful suggestions which greatly improved the paper’s quality. This work was supported by the
SFI Centre for Research Training in Advanced Networks for Sustainable Societies (ADVANCE
CRT), Ireland under the Grant number 18/CRT/6222.
References
[1] SMMT, Connected and Autonomous Vehicles Position Paper, Technical Re-
port, SMMT, 2017. URL: https://www.smmt.co.uk/wp-content/uploads/sites/2/
SMMT-CAV-position-paper-final.pdf.
[2] J. Wang, J. Liu, N. Kato, Networking and Communications in Autonomous Driving: A
Survey, IEEE Communications Surveys Tutorials 21 (2019) 1243–1274. doi:10.1109/COMST.
2018.2888904 , conference Name: IEEE Communications Surveys Tutorials.
[3] W. Zhou, D. Chen, J. Yan, Z. Li, H. Yin, W. Ge, Multi-agent Reinforcement Learning for
Cooperative Lane Changing of Connected and Autonomous Vehicles in Mixed Traffic,
arXiv:2111.06318 [cs] (2021). URL: http://arxiv.org/abs/2111.06318, arXiv: 2111.06318.
[4] H. Yu, R. Jiang, Z. He, Z. Zheng, L. Li, R. Liu, X. Chen, Automated vehicle-involved traffic
flow studies: A survey of assumptions, models, speculations, and perspectives, Transporta-
tion Research Part C: Emerging Technologies 127 (2021) 103101. URL: https://linkinghub.
elsevier.com/retrieve/pii/S0968090X21001224. doi:10.1016/j.trc.2021.103101 .
[5] J. Dong, S. Chen, Y. Li, R. Du, A. Steinfeld, S. Labi, Space-weighted information fusion using
deep reinforcement learning: The context of tactical control of lane-changing autonomous
vehicles and connectivity range assessment, Transportation Research Part C: Emerging
Technologies 128 (2021) 103192. URL: https://www.sciencedirect.com/science/article/pii/
S0968090X21002084. doi:10.1016/j.trc.2021.103192 .
[6] Z. Wang, Y. Wu, Q. Niu, Multi-Sensor Fusion in Automated Driving: A Survey, IEEE
Access 8 (2020) 2847–2868. doi:10.1109/ACCESS.2019.2962554 , conference Name: IEEE
Access.
[7] Y. Zheng, W. Ding, B. Ran, X. Qu, Y. Zhang, Coordinated decisions of discretionary
lane change between connected and automated vehicles on freeways: a game theory-
based lane change strategy, IET Intelligent Transport Systems 14 (2020) 1864–1870. URL:
https://onlinelibrary.wiley.com/doi/abs/10.1049/iet-its.2020.0146. doi:10.1049/iet- its.
2020.0146 , _eprint: https://onlinelibrary.wiley.com/doi/pdf/10.1049/iet-its.2020.0146.
[8] M. F. Keskin, B. Peng, B. Kulcsar, H. Wymeersch, Altruistic Control of Connected Auto-
mated Vehicles in Mixed-Autonomy Multi-Lane Highway Traffic, IFAC-PapersOnLine 53
(2020) 14966–14971. URL: https://linkinghub.elsevier.com/retrieve/pii/S2405896320326215.
doi:10.1016/j.ifacol.2020.12.1990 .
[9] W. Tong, A. Hussain, W. X. Bo, S. Maharjan, Artificial Intelligence for Vehicle-to-
Everything: A Survey, IEEE Access 7 (2019) 10823–10843. doi:10.1109/ACCESS.2019.
2891073 , conference Name: IEEE Access.
[10] Y. Hou, P. Graf, Decentralized Cooperative Lane Changing at Freeway Weaving Areas
Using Multi-Agent Deep Reinforcement Learning, arXiv:2110.08124 [cs] (2021). URL:
http://arxiv.org/abs/2110.08124, arXiv: 2110.08124.
[11] M. Veres, M. Moussa, Deep Learning for Intelligent Transportation Systems: A Survey
of Emerging Trends, IEEE Transactions on Intelligent Transportation Systems 21 (2020)
3152–3168. doi:10.1109/TITS.2019.2929020 , conference Name: IEEE Transactions on
Intelligent Transportation Systems.
[12] Z. Wang, X. Shi, X. Li, Review of Lane-Changing Maneuvers of Connected and Automated
Vehicles: Models, Algorithms and Traffic Impact Analyses, Journal of the Indian Institute
of Science 99 (2019) 589–599. URL: https://doi.org/10.1007/s41745-019-00127-7. doi:10.
1007/s41745- 019- 00127- 7 .
[13] P. G. Gipps, A model for the structure of lane-changing decisions, Transportation Research
Part B: Methodological 20 (1986) 403–414. URL: https://www.sciencedirect.com/science/
article/pii/0191261586900123. doi:10.1016/0191- 2615(86)90012- 3 .
[14] Q. Yang, H. N. Koutsopoulos, A Microscopic Traffic Simulator for evaluation of dynamic traf-
fic management systems, Transportation Research Part C: Emerging Technologies 4 (1996)
113–129. URL: https://www.sciencedirect.com/science/article/pii/S0968090X9600006X.
doi:10.1016/S0968- 090X(96)00006- X .
[15] Z. Zheng, Recent developments and research needs in modeling lane changing, Transporta-
tion Research Part B: Methodological 60 (2014) 16–32. URL: https://www.sciencedirect.
com/science/article/pii/S019126151300218X. doi:10.1016/j.trb.2013.11.009 .
[16] A. Kesting, M. Treiber, D. Helbing, General Lane-Changing Model MOBIL for Car-
Following Models, Transportation Research Record 1999 (2007) 86–94. URL: https:
//doi.org/10.3141/1999-10. doi:10.3141/1999- 10 , publisher: SAGE Publications Inc.
[17] S. Ulbrich, M. Maurer, Towards Tactical Lane Change Behavior Planning for Automated
Vehicles, in: 2015 IEEE 18th International Conference on Intelligent Transportation
Systems, 2015, pp. 989–995. doi:10.1109/ITSC.2015.165 , iSSN: 2153-0017.
[18] J. Erdmann, SUMO’s Lane-changing model, in: M. Behrisch, M. Weber (Eds.), LECTURE
NOTES IN CONTROL AND INFORMATION SCIENCES, volume 13, Springer Verlag, Berlin,
2015, pp. 105–123. URL: http://link.springer.com/chapter/10.1007/978-3-319-15024-6_7.
[19] J. He, K. Yang, H.-H. Chen, 6G Cellular Networks and Connected Autonomous Vehicles,
arXiv:2010.00972 [cs, eess] (2020). URL: http://arxiv.org/abs/2010.00972, arXiv: 2010.00972.
[20] P. Y. J. Ha, S. Chen, J. Dong, R. Du, Y. Li, S. Labi, Leveraging the Capabilities of Connected
and Autonomous Vehicles and Multi-Agent Reinforcement Learning to Mitigate Highway
Bottleneck Congestion, arXiv:2010.05436 [cs, eess] (2020). URL: http://arxiv.org/abs/2010.
05436, arXiv: 2010.05436.
[21] S. Chen, J. Dong, P. Y. J. Ha, Y. Li, S. Labi, Graph neural network and reinforce-
ment learning for multi-agent cooperative control of connected autonomous vehicles,
Computer-Aided Civil and Infrastructure Engineering 36 (2021) 838–857. URL: http:
//onlinelibrary.wiley.com/doi/abs/10.1111/mice.12702. doi:10.1111/mice.12702 , _eprint:
https://onlinelibrary.wiley.com/doi/pdf/10.1111/mice.12702.
[22] Z. Wang, X. Shi, X. Zhao, X. Li, Modeling decentralized mandatory lane change for
connected and autonomous vehicles: An analytical method, Transportation Research
Part C: Emerging Technologies 133 (2021) 103441. URL: https://www.sciencedirect.com/
science/article/pii/S0968090X21004319. doi:10.1016/j.trc.2021.103441 .
[23] S. Hwang, K. Lee, H. Jeon, D. Kum, Autonomous Vehicle Cut-In Algorithm for Lane-
Merging Scenarios via Policy-Based Reinforcement Learning Nested Within Finite-
State Machine, IEEE Transactions on Intelligent Transportation Systems (2022) 1–13.
URL: https://ieeexplore-ieee-org.elib.tcd.ie/document/9729796. doi:10.1109/TITS.2022.
3153848 , conference Name: IEEE Transactions on Intelligent Transportation Systems.
[24] X. Liao, Z. Wang, X. Zhao, Z. Zhao, K. Han, P. Tiwari, M. Barth, G. Wu, Online Prediction
of Lane Change with a Hierarchical Learning-Based Approach, 2022.
[25] T. Ren, Y. Xie, L. Jiang, Cooperative Highway Work Zone Merge Control Based on
Reinforcement Learning in a Connected and Automated Environment, Transportation
Research Record 2674 (2020) 363–374. URL: https://doi.org/10.1177/0361198120935873.
doi:10.1177/0361198120935873 , publisher: SAGE Publications Inc.
[26] C. Yu, X. Wang, X. Xu, M. Zhang, H. Ge, J. Ren, L. Sun, B. Chen, G. Tan, Distributed
Multiagent Coordinated Learning for Autonomous Driving in Highways Based on Dy-
namic Coordination Graphs, IEEE Transactions on Intelligent Transportation Systems 21
(2020) 735–748. doi:10.1109/TITS.2019.2893683 , conference Name: IEEE Transactions
on Intelligent Transportation Systems.
[27] Y. Lin, P. Wang, M. Ma, Intelligent Transportation System(ITS): Concept, Challenge and
Opportunity, in: 2017 ieee 3rd international conference on big data security on cloud
(bigdatasecurity), ieee international conference on high performance and smart computing
(hpsc), and ieee international conference on intelligent data and security (ids), 2017, pp.
167–172. doi:10.1109/BigDataSecurity.2017.50 .
[28] B. Häfner, V. Bajpai, J. Ott, G. A. Schmitt, A Survey on Cooperative Architectures and Ma-
neuvers for Connected and Automated Vehicles, IEEE Communications Surveys Tutorials
(2021) 1–1. doi:10.1109/COMST.2021.3138275 , conference Name: IEEE Communications
Surveys Tutorials.
[29] Y. Yang, R. Luo, M. Li, M. Zhou, W. Zhang, J. Wang, Mean Field Multi-Agent Reinforcement
Learning, in: Proceedings of the 35th International Conference on Machine Learning,
PMLR, 2018, pp. 5571–5580. URL: https://proceedings.mlr.press/v80/yang18d.html, iSSN:
2640-3498.
[30] P. Shi, B. Yan, A Survey on Intelligent Control for Multiagent Systems, IEEE Transac-
tions on Systems, Man, and Cybernetics: Systems 51 (2021) 161–175. doi:10.1109/TSMC.
2020.3042823 , conference Name: IEEE Transactions on Systems, Man, and Cybernetics:
Systems.
[31] X. Liao, X. Zhao, Z. Wang, K. Han, P. Tiwari, M. J. Barth, G. Wu, Game Theory-Based
Ramp Merging for Mixed Traffic With Unity-SUMO Co-Simulation, IEEE Transactions
on Systems, Man, and Cybernetics: Systems (2021) 1–12. URL: https://doi-org.elib.tcd.ie/
10.1109/TSMC.2021.3131431. doi:10.1109/TSMC.2021.3131431 , conference Name: IEEE
Transactions on Systems, Man, and Cybernetics: Systems.
[32] M. Shen, C. R. He, T. Molnar, A. H. Bell, G. Orosz, Energy-efficient Connected Cruise
Control with Lean Penetration of Connected Vehicles, Technical Report arXiv:2205.03473,
arXiv, 2022. URL: http://arxiv.org/abs/2205.03473, arXiv:2205.03473 [cs, eess] type: article.
[33] Z. Vander Laan, P. Schonfeld, Modeling heterogeneous traffic with cooperative adap-
tive cruise control vehicles: A first-order macroscopic perspective, Transportation
Planning and Technology 43 (2020) 113–140. URL: https://doi.org/10.1080/03081060.
2020.1717127. doi:10.1080/03081060.2020.1717127 , publisher: Routledge _eprint:
https://doi.org/10.1080/03081060.2020.1717127.
[34] M. Garg, C. Johnston, M. Bouroche, Can Connected Autonomous Vehicles really improve
mixed traffic efficiency in realistic scenarios?, in: 2021 IEEE International Intelligent
Transportation Systems Conference (ITSC), 2021, pp. 2011–2018. doi:10.1109/ITSC48978.
2021.9565068 .
[35] V. Mnih, K. Kavukcuoglu, D. Silver, A. A. Rusu, J. Veness, M. G. Bellemare, A. Graves,
M. Riedmiller, A. K. Fidjeland, G. Ostrovski, S. Petersen, C. Beattie, A. Sadik, I. Antonoglou,
H. King, D. Kumaran, D. Wierstra, S. Legg, D. Hassabis, Human-level control through deep
reinforcement learning, Nature 518 (2015) 529–533. URL: http://www.nature.com/articles/
nature14236. doi:10.1038/nature14236 , number: 7540 Publisher: Nature Publishing
Group.
[36] J. Dong, S. Chen, Y. Li, P. Y. J. Ha, R. Du, A. Steinfeld, S. Labi, Spatio-weighted information
fusion and DRL-based control for connected autonomous vehicles, in: 2020 IEEE 23rd
International Conference on Intelligent Transportation Systems (ITSC), 2020, pp. 1–6.
doi:10.1109/ITSC45102.2020.9294550 .
[37] Y. Ali, Z. Zheng, M. Mazharul Haque, M. Yildirimoglu, S. Washington, Understanding the
discretionary lane-changing behaviour in the connected environment, Accident Analysis
& Prevention 137 (2020) 105463. URL: https://www.sciencedirect.com/science/article/pii/
S0001457519312771. doi:10.1016/j.aap.2020.105463 .
[38] T. P. Lillicrap, J. J. Hunt, A. Pritzel, N. Heess, T. Erez, Y. Tassa, D. Silver, D. Wierstra,
Continuous control with deep reinforcement learning, arXiv:1509.02971 [cs, stat] (2019).
URL: http://arxiv.org/abs/1509.02971, arXiv: 1509.02971.
[39] I. Grondman, L. Busoniu, G. A. D. Lopes, R. Babuska, A Survey of Actor-Critic Re-
inforcement Learning: Standard and Natural Policy Gradients, IEEE Transactions on
Systems, Man, and Cybernetics, Part C (Applications and Reviews) 42 (2012) 1291–1307.
doi:10.1109/TSMCC.2012.2218595 , conference Name: IEEE Transactions on Systems,
Man, and Cybernetics, Part C (Applications and Reviews).
[40] H. Xu, Y. Zhang, C. G. Cassandras, L. Li, S. Feng, A bi-level cooperative driving strategy
allowing lane changes, Transportation Research Part C: Emerging Technologies 120
(2020) 102773. URL: https://www.sciencedirect.com/science/article/pii/S0968090X20306835.
doi:10.1016/j.trc.2020.102773 .
[41] I. Kassabalidis, M. El-Sharkawi, R. Marks, P. Arabshahi, A. Gray, Swarm intelligence for
routing in communication networks, in: GLOBECOM’01. IEEE Global Telecommunications
Conference (Cat. No.01CH37270), volume 6, 2001, pp. 3613–3617 vol.6. doi:10.1109/
GLOCOM.2001.966355 .
[42] S. Bang, S. Ahn, Platooning Strategy for Connected and Autonomous Vehicles: Transition
from Light Traffic, Transportation Research Record 2623 (2017) 73–81. URL: https://doi.
org/10.3141/2623-08. doi:10.3141/2623- 08 , publisher: SAGE Publications Inc.
[43] C. Zhang, Y. Xie, H. Bai, B. Yu, W. Li, Y. Gao, A survey on federated learning, Knowledge-
Based Systems 216 (2021) 106775. URL: https://www.sciencedirect.com/science/article/pii/
S0950705121000381. doi:10.1016/j.knosys.2021.106775 .
[44] T. Zeng, O. Semiari, M. Chen, W. Saad, M. Bennis, Federated Learning on the Road:
Autonomous Controller Design for Connected and Autonomous Vehicles, arXiv:2102.03401
[cs, eess] (2021). URL: http://arxiv.org/abs/2102.03401, arXiv: 2102.03401.
[45] L. Zhao, Y. Jia, Intelligent transportation system for sustainable environment in smart cities,
The International Journal of Electrical Engineering & Education (2021) 0020720920983503.
URL: https://doi.org/10.1177/0020720920983503. doi:10.1177/0020720920983503 , pub-
lisher: SAGE Publications Ltd STM.
[46] H. Ma, S. Li, E. Zhang, Z. Lv, J. Hu, X. Wei, Cooperative Autonomous Driving Oriented
MEC-Aided 5G-V2X: Prototype System Design, Field Tests and AI-Based Optimization
Tools, IEEE Access 8 (2020) 54288–54302. doi:10.1109/ACCESS.2020.2981463 , conference
Name: IEEE Access.