<?xml version="1.0" encoding="UTF-8"?>
<TEI xml:space="preserve" xmlns="http://www.tei-c.org/ns/1.0" 
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" 
xsi:schemaLocation="http://www.tei-c.org/ns/1.0 https://raw.githubusercontent.com/kermitt2/grobid/master/grobid-home/schemas/xsd/Grobid.xsd"
 xmlns:xlink="http://www.w3.org/1999/xlink">
	<teiHeader xml:lang="en">
		<fileDesc>
			<titleStmt>
				<title level="a" type="main">Design of AI-based lane changing modules in connected and autonomous vehicles: a survey</title>
			</titleStmt>
			<publicationStmt>
				<publisher/>
				<availability status="unknown"><licence/></availability>
			</publicationStmt>
			<sourceDesc>
				<biblStruct>
					<analytic>
						<author>
							<persName><forename type="first">Bharathkumar</forename><surname>Hegde</surname></persName>
							<email>hegdeb@tcd.ie</email>
							<affiliation key="aff0">
								<orgName type="institution">Trinity College Dublin</orgName>
								<address>
									<settlement>Dublin</settlement>
									<country key="IE">Ireland</country>
								</address>
							</affiliation>
						</author>
						<author>
							<persName><forename type="first">Melanie</forename><surname>Bouroche</surname></persName>
							<email>melanie.bouroche@tcd.ie</email>
							<affiliation key="aff0">
								<orgName type="institution">Trinity College Dublin</orgName>
								<address>
									<settlement>Dublin</settlement>
									<country key="IE">Ireland</country>
								</address>
							</affiliation>
						</author>
						<title level="a" type="main">Design of AI-based lane changing modules in connected and autonomous vehicles: a survey</title>
					</analytic>
					<monogr>
						<idno type="ISSN">1613-0073</idno>
					</monogr>
					<idno type="MD5">031FC8CFB6A70DB6E5C0BB1D7208306E</idno>
				</biblStruct>
			</sourceDesc>
		</fileDesc>
		<encodingDesc>
			<appInfo>
				<application version="0.7.2" ident="GROBID" when="2023-03-24T08:08+0000">
					<desc>GROBID - A machine learning software for extracting information from scholarly documents</desc>
					<ref target="https://github.com/kermitt2/grobid"/>
				</application>
			</appInfo>
		</encodingDesc>
		<profileDesc>
			<textClass>
				<keywords>
					<term>Connected and autonomous vehicle (CAV)</term>
					<term>Lane change</term>
					<term>Artificial intelligence</term>
					<term>Deep learning (DL)</term>
					<term>Intelligent transportation system (ITS) Orcid 0000-0002-2085-7867 (B. Hegde); 0000-0002-5039-0815 (M. Bouroche)</term>
				</keywords>
			</textClass>
			<abstract>
<div xmlns="http://www.tei-c.org/ns/1.0"><p>Lane changing is one of the complex driving tasks as it requires the vehicle to be aware of its highlydynamic surrounding environment, make decisions, and enact them in a timely manner. By exploiting both sensors and inter-vehicle communication, Connected and Autonomous Vehicles (CAVs) have the potential to significantly improve lane changing safety and efficiency. The complexity of the task and the real-time requirements make lane-changing a problem particularly suited to Artificial Intelligence (AI) approaches. In this paper, we survey the design of AI-based Lane-Changing(LC) modules for CAVs. First, we identify the key factors that can influence the design of an LC module. Next, we survey recent developments in AI-based lane changing. Finally, we analyse these approaches along the dimensions of the key influencing factors and summarise the challenges that are yet to be addressed and opportunities that can guide the future developments in AI-based LC modules.</p></div>
			</abstract>
		</profileDesc>
	</teiHeader>
	<text xml:lang="en">
		<body>
<div xmlns="http://www.tei-c.org/ns/1.0"><head n="1.">Introduction</head><p>Autonomous Vehicles (AVs) are one of the major components of a rapidly developing Intelligent Transportation System (ITS). Developments in the communication technology are expected to complement the development of the AV technology. Therefore, advancements in Connected and Autonomous Vehicles (CAVs) are expected to improve the performance in the driving tasks required for achieving autonomy of level of 3 and above <ref type="bibr" target="#b0">[1]</ref>. Currently, in the commercial market, Tesla Model S has achieved an autonomy level of 2.5 and the Audi A-8 has achieved level 3 autonomy in driving <ref type="bibr" target="#b1">[2]</ref> by automating major driving tasks. Whereas, a fully autonomous vehicle (SAE's level 5 autonomy) should be capable of performing all driving tasks safely and efficiently in all kinds of environment. Among the driving tasks, lane changing is one of the complex tasks for CAVs and a challenging problem for researchers <ref type="bibr" target="#b2">[3]</ref>.</p><p>By planning and coordinating lane changes, CAVs might be able to improve the traffic flow at both microscopic and macroscopic level. The macroscopic traffic level benefits may include increased safety, traffic efficiency, and road capacity <ref type="bibr" target="#b3">[4]</ref> and the microscopic traffic level benefits may include increased comfort for travellers with minimal speed variation and reduced travel delay <ref type="bibr" target="#b4">[5]</ref>. To plan a lane changing manoeuvre, CAVs use information collected by sensors and other vehicles in a highly-dynamic environment <ref type="bibr" target="#b5">[6]</ref>. An LC module uses this information to make lane change decisions. Some of the popular methods used to design an LC module are game theory <ref type="bibr" target="#b6">[7]</ref>, controller optimisation <ref type="bibr" target="#b7">[8]</ref>, and AI <ref type="bibr" target="#b8">[9]</ref>.</p><p>Recently, AI has been used more often to design LC modules, as the recent developments in AI have proven useful to make fast decisions in dynamic environments with a large set of parameters. Real-world traffic is very dynamic and a large set of parameters may be considered to perform a lane change. Parameters may include the position, speed, and heading of the ego vehicle and surrounding vehicles <ref type="bibr" target="#b9">[10]</ref>. Moreover, an LC module needs to make intelligent trade-offs to improve the possibility of achieving safety along with other objectives of lane changes, such as improving mobility, comfort of travel, fuel efficiency, and reducing emissions. Therefore, AI is a promising option for designing LC modules, which can make efficient lane change decisions in complex traffic environments to achieve multiple objectives.</p><p>Most survey papers have reviewed the application of AI in wider fields such as ITS <ref type="bibr" target="#b10">[11]</ref>, CAVs <ref type="bibr" target="#b11">[12]</ref>, or V2X communication <ref type="bibr" target="#b8">[9]</ref>. Conversely, this paper focuses on the design of AI-based modules for CAV lane changing controllers. Specifically, the main contributions of this work are:</p><p>• Identify the key factors that can influence the design of an AI-based LC module.</p><p>• Provide insights into the recent developments in the design of AI-based LC module along the dimensions of the key influencing factors. • Summarise the challenges and opportunities in the design of AI-based LC modules.</p><p>Challenges provide the research gaps that are yet to be addressed, and opportunities identify the possibilities that can guide the future developments of AI-based LC modules.</p><p>The remainder of the paper is structured as follows. Section 2 provides background details related to the development of lane change and the architecture of lane changing in CAVs. Section 3 presents the key factors that influence the design of AI-based LC modules. A review of AI-based LC modules is presented in Section 4. Finally, Section 5 reviews the approaches discussed before summarising the challenges and possible opportunities in the design of AI-based lane changing modules.</p></div>
<div xmlns="http://www.tei-c.org/ns/1.0"><head n="2.">Background</head><p>This section discusses the development of lane changing models and a general architecture of a CAV lane change. The development of lane changing models lists some of the standard lane changing models used for traffic simulations. The general architecture of a CAV lane change provides its components and describes how they are related.</p></div>
<div xmlns="http://www.tei-c.org/ns/1.0"><head n="2.1.">Development of lane changing models</head><p>An LC model usually encodes a rational decision to change lanes based on various parameters that describe the environment around a vehicle. The first known LC model is the Gipps lane changing model <ref type="bibr" target="#b12">[13]</ref>. The Gipps model is based on maintaining a desired speed and being in the correct lane for an upcoming desired manoeuvre. LC models developed based on the Gipps model are classified as Gipps-type LC models. To overcome the limitations of Gipps-type LC models, which were deterministic, rule-based models were proposed <ref type="bibr" target="#b13">[14]</ref>. Rule-based models consist of a decision process defined in four steps: decision to consider a lane change, choice of the target lane, search for an acceptable gap, and executing the lane change <ref type="bibr" target="#b14">[15]</ref>. Considering the probabilistic approach for lane changing decision instead of deterministic lane changing decision as in Gipps type LC model, was one of the distinct features that made the rule-based model more realistic <ref type="bibr" target="#b14">[15]</ref>.</p><p>While the Gipps-type LC models consider only the vehicle speed, Kesting et al. proposed a novel incentive-based lane changing model, MOBIL (Minimising Overall Braking Induced by Lane change) <ref type="bibr" target="#b15">[16]</ref>, which considers the acceleration of the vehicles as well <ref type="bibr" target="#b14">[15]</ref>. The MOBIL model makes a lane change decision based on the probability of advantages and disadvantages of the lane change, based on the accelerations of the vehicles. In addition to the acceleration, MOBIL model also considers factors such as politeness and the right-left lane bias (which restricts overtaking from the right side, eg in Germany,) <ref type="bibr" target="#b14">[15]</ref>. These considerations enable easy integration of MOBIL with car-following models such as the Intelligent Driver Model (IDM) <ref type="bibr" target="#b16">[17]</ref>.</p><p>A lane changing model, named LC2013, considers the intention of changing lanes using a decision-tree algorithm <ref type="bibr" target="#b17">[18]</ref>. The intention for a lane change can be to reach a specific destination, to overtake a slow vehicle, to cooperate with other vehicles, or to follow local traffic regulations. In LC2013 LC model, CAVs coordinate by sharing their intentions during lane changing manoeuvres. The LC2013 model is integrated in the Simulation of Urban MObility (SUMO) simulation framework and allows customisation to simulate regulatory traffic restrictions, such as the restriction on overtaking from the right side as in Germany.</p><p>To conclude, the models discussed above have been used as standard lane changing models in popular traffic simulators and as a baseline to validate recent LC modules. These standard LC models are designed to achieve a single objective, that is, to make a safe lane change decision. However, recent AI-based LC modules aim to achieve safety along with other objectives such as improving mobility, comfort, and fuel efficiency. A detailed discussion of recent AI-based LC modules is presented in Section 4.</p></div>
<div xmlns="http://www.tei-c.org/ns/1.0"><head n="2.2.">Architecture of CAV lane changing</head><p>The architecture of CAV lane changing typically consist of four major components, namely perception, communication, lane changing, and vehicular control as shown in Figure <ref type="figure" target="#fig_0">1</ref>  <ref type="bibr" target="#b11">[12]</ref>. The perception module creates a perception of the environment around the vehicle by combining the inputs from various sensors such as LiDAR, RADAR, camera, GPS, IMU, etc. The vehicle-to-everything (V2X) communication module provides interfaces to communicate with other components of the Intelligent Transport System (ITS), such as other vehicles, road side unit (RSU), mobile edge computing (MEC) server, cloud server, etc <ref type="bibr" target="#b18">[19]</ref>. The lane changing module integrates inputs from the perception module and information collected through the communication module to make a lane changing decision and plan trajectories for the execution of lane changes <ref type="bibr" target="#b11">[12]</ref>. The lane changing module can be implemented using either a centralised <ref type="bibr" target="#b19">[20,</ref><ref type="bibr" target="#b20">21]</ref> or a decentralised <ref type="bibr" target="#b9">[10,</ref><ref type="bibr" target="#b21">22]</ref> architecture. The Vehicular control module includes the </p></div>
<div xmlns="http://www.tei-c.org/ns/1.0"><head n="3.">Dimensions of the survey</head><p>In this survey, we explore the design of AI-based LC modules for CAVs along four dimensions. These dimensions are objectives of the lane change, lane change scenarios, architecture, and mixed traffic consideration. They are some of the key factors that influence the design of AI-based LC modules implemented in CAVs.</p></div>
<div xmlns="http://www.tei-c.org/ns/1.0"><head n="3.1.">Lane changing objective</head><p>The objectives of lane changing can be broadly classified as safety, mobility, comfort, and sustainability. Safety is one of the major objectives considered in LC modules. Safety objectives mainly focus on lane changing with minimal risk to avoid collisions <ref type="bibr" target="#b4">[5,</ref><ref type="bibr" target="#b3">4,</ref><ref type="bibr" target="#b2">3,</ref><ref type="bibr" target="#b22">23,</ref><ref type="bibr" target="#b23">24]</ref>. However, a CAV should not compromise on mobility, while trying to improve safety. Mobility objectives consider improving traffic throughput <ref type="bibr" target="#b9">[10,</ref><ref type="bibr" target="#b24">25]</ref> and average speed <ref type="bibr" target="#b20">[21]</ref> of the vehicle, and avoiding stop-go traffic <ref type="bibr" target="#b24">[25]</ref>. To improve mobility, a CAV can make unnecessary lane change manoeuvres <ref type="bibr" target="#b4">[5]</ref> or accelerate and decelerate frequently, causing discomfort to passengers <ref type="bibr" target="#b23">[24,</ref><ref type="bibr" target="#b25">26]</ref>. Therefore, achieving travel comfort is another potential objective of LC modules. Furthermore, some of the LC modules consider sustainability <ref type="bibr" target="#b26">[27]</ref> as one of their objective, as lane changes can also affect overall fuel efficiency and emissions <ref type="bibr" target="#b9">[10]</ref>.</p></div>
<div xmlns="http://www.tei-c.org/ns/1.0"><head n="3.2.">Lane change scenario</head><p>The lane change scenario can be defined based on the motive of a vehicle to perform a lane change. The motive to change lane can be broadly categorised as discretionary lane change, mandatory lane change, and lane change in bottleneck sections. Dynamics of vehicle movement and the parameters considered for the lane change decision making differ for each of these categories of lane change. Hence, the lane change scenario can be one of the factors to consider while designing an LC module.</p><p>An optional lane change by a vehicle, for the benefit of its own or other vehicles in traffic, is considered a Discretionary Lane Change (DLC). DLCs often result in increased speed for the ego vehicles, and they may have various positive impacts on the traffic at the macroscopic level, such as increasing road capacity, increasing traffic throughput, minimising traffic jam propagation, etc. DLCs focus primarily on safety and achieving macroscopic objectives like increased driving comfort, mobility, or throughput <ref type="bibr" target="#b4">[5,</ref><ref type="bibr" target="#b6">7,</ref><ref type="bibr" target="#b25">26]</ref>.</p><p>On occasions, a vehicle may be required to change the lane to reach a desired destination; such lane changes are classified as Mandatory Lane Changes <ref type="bibr">(MLCs)</ref>. Some examples of MLCs include changing lane to enter a highway, exit a highway, or before reaching an intersection for a turn. Since lane change is mandatory in these cases, the vehicle may need to execute a risky lane change, especially in high traffic. MLC by a vehicle may affect the other vehicles in traffic, therefore, the ideal MLC controller should be capable of ensuring safety even under risky situations and it should have minimal negative impact on the mainstream traffic flow of the highway <ref type="bibr" target="#b21">[22,</ref><ref type="bibr" target="#b20">21,</ref><ref type="bibr" target="#b9">10]</ref>.</p><p>Similar to an MLC, a vehicle will be changing lanes when the current lane reaches a dead end or merges into an adjacent lane. Such lane changes can be categorised as lane change in bottleneck sections. In bottleneck sections, coordination among vehicles plays a key role as the vehicles changing lane will interrupt the main traffic flow. A bottleneck may be created because of construction works, reduced road space, vehicle broken-down, or accidents. Hence, bottleneck sections are often not observed in advance. Therefore, lane changes in bottleneck sections may need to be handled differently compared to an MLC. Typically, the LC modules for bottleneck sections aim to achieve a smooth traffic flow with less congestion, and increase traffic throughput by avoiding stop-go traffic <ref type="bibr" target="#b19">[20,</ref><ref type="bibr" target="#b24">25]</ref>.</p></div>
<div xmlns="http://www.tei-c.org/ns/1.0"><head n="3.3.">Architecture</head><p>The architecture of an LC module can be classified as centralised or decentralised, depending on its placement within the CAV lane change architecture. In a centralised architecture, the LC module can be placed in an RSU, an edge server, or a cloud server which can be a centralised controller. A centralised controller can integrate the information from traffic participants and use it for trajectory planning and lane changing decisions. Furthermore, the controller may suggest state changes, such as path, velocity, etc. <ref type="bibr" target="#b27">[28]</ref> or lane change decisions to a CAV <ref type="bibr" target="#b20">[21]</ref>. Centralised controllers have been found to achieve better results in completing cooperative objectives <ref type="bibr" target="#b28">[29]</ref>. However, one disadvantage of the centralised architecture is the challenge of scaling the central server based on the variation in the traffic flow. Additionally, it adds an extra overhead for installing and maintaining a wide scale infrastructure. As the central entity can be a bottleneck of the system, the centralised infrastructure is prone to failures and network congestion.</p><p>On the other hand, the decentralised architecture can be implemented by placing the LC module in individual CAVs. The LC module along with the perception module and the vehicular control module can form an autonomous controller. These autonomous controllers can communicate through a direct V2V communication interface or through the network infrastructure to collect the information needed from other CAVs for trajectory planning and lane change decisions <ref type="bibr" target="#b29">[30]</ref>. The necessary information required from other CAVs may include state information, trajectory plan, traffic information, etc. An LC module can collect this information and act independently or interact with other CAVs to achieve a cooperative decision <ref type="bibr" target="#b27">[28]</ref>. In a decentralised architecture, lane change execution can be faster, as the LC module and the vehicular control module are placed in the same CAV <ref type="bibr" target="#b30">[31]</ref>. This approach can significantly improve the scalability of the module based on traffic demands. Furthermore, ITS infrastructure, such as RSU or MEC servers, can be used to offload some of the resource-intensive computations <ref type="bibr" target="#b27">[28]</ref>. However, one of the key challenges in a decentralised architecture is to achieve consensus among multiple CAVs <ref type="bibr" target="#b29">[30]</ref>.</p></div>
<div xmlns="http://www.tei-c.org/ns/1.0"><head n="3.4.">Mixed traffic consideration</head><p>A mixed traffic scenario refers to a traffic environment consisting of vehicles having various levels of connectivity and automation <ref type="bibr" target="#b31">[32]</ref>. As a wider adoption of CAVs can be a slow process, considerable market penetration of CAVs is only expected to happen by 2040-2050 <ref type="bibr" target="#b32">[33]</ref>. Thus, CAVs will coexist with HDVs in the foreseeable future <ref type="bibr" target="#b33">[34]</ref> and consideration of mixed traffic is necessary to design a practical LC module. In mixed traffic, creating a perception of the surrounding environment can be a complex problem <ref type="bibr" target="#b2">[3]</ref>. Furthermore, the performance analysis of the module in mixed traffic, with a variable penetration rate of CAVs, can provide a practical estimate of the minimum percentage of CAVs required for the module to perform effectively <ref type="bibr" target="#b19">[20]</ref>.</p></div>
<div xmlns="http://www.tei-c.org/ns/1.0"><head n="4.">Applications of AI for lane changing</head><p>In recent publications, AI is widely used in various applications of ITS <ref type="bibr" target="#b10">[11]</ref>. However, only a limited number of research works focus on using AI-based LC modules for CAVs. These LC modules can be categorised as Deep Reinforcement Learning (DRL), swarm intelligence, and federated learning.</p></div>
<div xmlns="http://www.tei-c.org/ns/1.0"><head n="4.1.">Deep reinforcement learning</head><p>Deep reinforcement learning (DRL) is a combination of deep learning and Reinforcement Learning (RL). Deep learning is a method to extract knowledge from a large input data using multilayer network of processing nodes, called neurons <ref type="bibr" target="#b8">[9]</ref>. This network of neurons is widely known as Neural Networks (NN). The Convolution Neural Network (CNN), the Recurrent Neural Network (RNN), and the Graph Convolution Network (GCN) are some of the wellknown examples of NNs used in deep learning <ref type="bibr" target="#b10">[11]</ref>. RL is a method of learning through experience. Usually an RL defines a state space to represent possible states of the system, an action space to represent a set of possible actions, and a reward for an action in the action space. With repeated training, a given state can be mapped to the best action that maximises the reward. For a large state space and action space, RL would be inefficient, because the search tree would increase exponentially. To handle a large number of states and actions, RL can make use of deep learning to map states to the actions with the best possible reward. A CAV (agent) may have to consider a large state space and a large action space to make efficient lane changes to achieve a specific set of objectives. Thus, recent research works have used DRL to implement LC modules in CAVs.</p><p>DRL can be a suitable option for CAV controllers as they are capable of learning from a dynamic environment with a large action space and state space <ref type="bibr" target="#b4">[5]</ref>. Moreover, the DRL can be trained using simulations at lower costs. They can provide fast inference, scale easily, and outperform humans with instantaneous and reliable decision making capabilities <ref type="bibr" target="#b20">[21]</ref>. For these reasons, DRL has been one of the popular choices to solve the challenges related to lane changing in CAVs, especially lane change decision making.</p><p>Some of the research works have designed DRL-based modules for making lane change decisions, using various formulations of state space and action space <ref type="bibr" target="#b4">[5,</ref><ref type="bibr" target="#b20">21,</ref><ref type="bibr" target="#b22">23]</ref>. Conversely, other methods have used DRL only for a sub-task within a complex LC module. The sub-task can be trajectory planning <ref type="bibr" target="#b22">[23]</ref> or trajectory prediction of surrounding vehicles <ref type="bibr" target="#b23">[24]</ref>. In general, applications of DRL for CAV lane changes can be broadly grouped based on the type of learning approach, such as Deep Q-Network (DQN) and Actor-Critic (AC) network.</p></div>
<div xmlns="http://www.tei-c.org/ns/1.0"><head n="4.1.1.">Deep Q-Network</head><p>DQNs can be applied to map a high-dimensional input space to a discrete action space, based on a policy <ref type="bibr" target="#b34">[35]</ref>. This makes them suitable for high-level decision making for CAV lane changes (change left, right or stay in the same lane), as it might depend on a variety of inputs recorded from local sensors and surrounding vehicles <ref type="bibr" target="#b30">[31]</ref>. DQNs have been successfully applied to CAVs lane changing, with reward functions that account for safety, mobility and comfort to achieve lane changing objectives <ref type="bibr" target="#b35">[36,</ref><ref type="bibr" target="#b25">26]</ref>.</p><p>Although a DQN seems to be useful for lane change decision making modules, some challenges need to be addressed for effectively using it. One of the challenge can be dynamic state space of the CAVs, because of which DQN inputs can be of variable size <ref type="bibr" target="#b4">[5]</ref>. DQNs, however, require inputs of fixed size. This challenge can be addressed by encoding the dynamic state space with variable length to a set of parameters with a fixed length. For example, Dong et al. used three NNs to encode each component of a dynamic state space, which contains the state of a CAV, the states of the surrounding vehicles and the states of the downstream vehicles <ref type="bibr" target="#b4">[5]</ref>. This LC module, however, does not take advantage of the possibility of collaboration among CAVs. To enable collaboration between CAVs, Chen et al. uses Graph Convolution Networks (GCNs) to include topological information about traffic to make collaborative lane change decisions. The GCN is implemented in a centralised unit to encode dynamic input data and topological information to a set of fixed length parameters, which are used as input to a DQN <ref type="bibr" target="#b20">[21]</ref>. On the other hand, a decentralised approach was used by Yu et al., for encoding the dynamic traffic topology as a Dynamic Coordination Graph (DCG) to achieve collaborative lane change decisions <ref type="bibr" target="#b25">[26]</ref>.</p><p>In summary, DQN-based LC modules can be a good option for single-step lane change decision making in CAVs. The implementations of DQN for CAV lane changing address the limitation of fixed length input and achieve coordination among CAVs using innovative methods. However, existing DQN implementations do not consider continuous controls such as acceleration, which could be an important factor, as it can be used to create appropriate gaps to allow collaborative lane changes <ref type="bibr" target="#b36">[37]</ref>.</p></div>
<div xmlns="http://www.tei-c.org/ns/1.0"><head n="4.1.2.">Actor-Critic Network</head><p>The Actor-Critic Network (ACN) is an extension of DQN which implements the Actor-Critic (AC) algorithm <ref type="bibr" target="#b37">[38]</ref>. The AC is a type of RL algorithm that consists of policy (actor) and value (critic) functions <ref type="bibr" target="#b38">[39]</ref>. Policy functions use optimisation methods such as the Deterministic Policy Gradient (DPG) or the Deep DPG (DDPG) to estimate a policy in the continuous action space. Optimisation methods, however, suffer from high variance to estimate the gradient, as a result learning can be slow <ref type="bibr" target="#b38">[39]</ref>. On the other hand, value functions use Temporal Difference (TD) learning to reduce variance in the expected return. Hence, the AC algorithm, which combines optimisation method and TD learning, can quickly converge to learn a policy for the continuous action space. Overall, ACNs can provide the combined advantage of AC algorithm and DQN to design a CAV controller, which can handle a large state space and a continuous action space.</p><p>Existing ACN implementations aim to achieve a balance between the scalability of the LC module and cooperation among CAVs based on the requirements of the lane change scenario. Since ACNs allow learning a policy in a continuous action space, they can be used to adjust the continuous variables of CAV control, such as acceleration or speed, to enable cooperation between CAVs by creating the necessary gaps to allow safe lane changes. Cooperation among CAVs can be enabled by using a centralised LC controller, but this compromises scalability. Conversely, there is a good possibility to improve scalability with a decentralised LC controller, but a cooperation mechanism would have to be implemented explicitly.</p><p>For example, an LC module can implement cooperation among CAVs by using a centralised ACN-based controller to adjust the speed of CAVs in a congested highway bottleneck <ref type="bibr" target="#b19">[20]</ref>. Cooperation among CAVs would be necessary in a congested bottleneck scenario as vehicles need to create gaps that allow safe merging of vehicle into the main stream. Therefore, to enforce the cooperation among CAVs, a centralised solution can be an ideal option. However, cooperation can be induced among CAVs using a decentralised LC module. An example of a decentralised LC module was developed by Ren et al. for lane merging in a work zone section. This module uses ACN to adjust the acceleration of the CAV to allow cooperative lane changes in a work zone section <ref type="bibr" target="#b24">[25]</ref>. Overall, for CAV lane changes in a work zone section or a bottleneck section, both centralised and decentralised architecture can be used to implement cooperation among CAVs with the ACN-based LC module.</p><p>For lane changes on a highway or in a weaving section of the highway, a decentralised approach would enable an independent strategy for each vehicle <ref type="bibr" target="#b9">[10]</ref>. An example of a decentralised LC module for lane changes in a weaving section of a highway is the multi-agent DRL module proposed by Hou and Graf, which uses an ACN to make lane change decisions and speed adjustments to allow cooperation among vehicles <ref type="bibr" target="#b9">[10]</ref>. This decentralised module relies on global state information to make its decisions. As global state information may need to be obtained from an external centralised system, it could compromise the scalability of the LC module. On the other hand, a shared ACN can also be used to implement cooperative lane change among CAVs, without compromising scalability. Zhou et al. proposed a cooperative and decentralised LC module <ref type="bibr" target="#b2">[3]</ref>. This LC module uses a shared ACN to make lane change decisions and control vehicle speed. Furthermore, the module achieves cooperation and improved performance compared to the individual ACN implementation. Overall, ACN-based LC modules, designed mainly for MLC and DLC in highway traffic, can provide scalable cooperation.</p><p>Although ACN-based LC modules provide some advantages compared to DQN-based LC modules, they suffer from some limitations. ACN-based LC modules provide various ways to implement cooperation among CAVs. Moreover, they consider lane changing scenario as well to choose the appropriate architecture for an LC module, such as centralised or decentralised. However, the ACN-based LC modules discussed above assume that lane change is executed in a single step, and consider the LC module as a single concrete module. These assumptions limit the possibility of including additional functionalities, such as planning the lane change trajectory, predicting the trajectory of other vehicles, or negotiating combined lane change trajectories to improve the performance of an LC module.</p><p>To overcome these limitations, a modular lane change approach can be used <ref type="bibr" target="#b39">[40]</ref>. In the modular lane change approach, the LC module can be a combination of different methods to achieve the best overall results. Such sub modules can have their own way of handling a specific task such as lane change decision making, trajectory planning or predicting the probable trajectory of other vehicles which might add additional benefits to improve the performance of the LC module. For example, Liao et al. proposed an online model to predict the possibility of lane changes by surrounding vehicles. This model is a combination of two sub-modules <ref type="bibr" target="#b23">[24]</ref>. The first sub-module uses a Long-Short Term Memory(LSTM) network and the second sub-module uses Inverse Reinforcement Learning (IRL) to predict the trajectory of the vehicle. The predictions generated from this module can be used to improve the performance of the LC module. Another example that uses modular approach consists of a high-level Finite State Machine (FSM) module for lane change decision making and a low level ACN to perform safe lane changing manoeuvres <ref type="bibr" target="#b22">[23]</ref>. In general, the modular approach seems to be a promising trend for AI-based LC modules as it opens up new dimensions to improve the efficiency of an LC module.</p></div>
<div xmlns="http://www.tei-c.org/ns/1.0"><head n="4.2.">Swarm intelligence</head><p>Swarm intelligence is a method to achieve collective intelligence in a group of things (in case of ITS they can be vehicles, infrastructure, or actuators) without a central controlling agent. Therefore, use of swarm intelligence in the V2X paradigm provides various advantages such as scalability, fault tolerance, adaptation, modularity, and autonomy of each agent <ref type="bibr" target="#b40">[41]</ref>. Some examples of the swarm intelligence algorithm can be Particle Swarm Optimisation (PSO) to solve optimal point problems, Ant Colony Optimisation (ACO) for graph optimisation problems, and swarmcasting for distributed media sharing problems <ref type="bibr" target="#b8">[9]</ref>.</p><p>In the V2X paradigm, swarm intelligence can be applied to perform a collective task by all vehicles using the communication environment. Mostly, swarm intelligence is applied in communication technologies such as AntNet <ref type="bibr" target="#b8">[9]</ref>. To our knowledge, swarm intelligence has not been applied to lane changing. However, swarm intelligence was used by Bang and Ahn to design a platooning strategy for CAVs <ref type="bibr" target="#b41">[42]</ref>. The objectives of the platooning strategy are similar to the objectives of the LC modules, such as to improve traffic efficiency, safety and stability. In addition, the platooning strategy is based on longitudinal control of CAVs with simple formulations compared to learning-based modules. It could be interesting to investigate the possibility of using a similar swarm intelligence strategy for designing CAV lane changes.</p></div>
<div xmlns="http://www.tei-c.org/ns/1.0"><head n="4.3.">Federated learning</head><p>Federated learning is a fairly new branch of artificial intelligence that allows distributed training to create a global model. The agents can use the knowledge aggregated in a global model to make the best decisions in unseen situations. The key ideas behind federated learning are local computation and model transmission <ref type="bibr" target="#b42">[43]</ref>. These ideas can reduce the privacy risks concerning local data. Moreover, federated learning can significantly reduce training time as the model can be trained in parallel using multiple agents.</p><p>Wireless connectivity in CAVs can be leveraged to implement a federated learning-based CAV controller <ref type="bibr" target="#b43">[44]</ref>. Using federated learning to design a CAV controller may have various advantages compared to traditional AI based controllers. Significant amounts of data are required to train traditional AI based controllers. On the other hand, a federated learning-based controller may depend on local data and updates from the global model, thus reducing storage requirements in a CAV. Moreover, a federated learning-based controller is expected to adapt well in various traffic environments <ref type="bibr" target="#b43">[44]</ref>. For example, Zeng et al. used federated learning framework to effectively design a longitudinal control for CAVs to reduce accidents, road congestion, and improve traffic throughput. Even though the federated learning framework allows distributed training, it requires a central unit to aggregate the model updates from all agents.</p></div>
<div xmlns="http://www.tei-c.org/ns/1.0"><head n="5.">Challenges and opportunities</head><p>Although every LC module has specific advantages of their own, they suffer from some limitations. Some of these limitations can be observed from Table <ref type="table" target="#tab_0">1</ref>, which summarises the AI-based approaches discussed before, according to the dimensions presented in Section 3. To overcome these limitations, several challenges need to be addressed. This section provides a summary of these limitations and challenges, as well as possible considerations that can contribute to the development of efficient and practical lane-changing solutions for CAVs.</p><p>From Table <ref type="table" target="#tab_0">1</ref> we can observe some trends that highlight the limitations of AI-based LC modules along each dimension of this survey. Among objectives of LC modules, improving safety and mobility are the main objectives in most AI-based LC modules. In addition to these objectives, sustainability is also one of the main priorities of ITS <ref type="bibr" target="#b44">[45]</ref>. However, only a limited number of LC modules have considered sustainability as their objective. Sustainability considerations such as energy utilisation and emissions from vehicles at a societal level may be significantly affected by the increase of CAVs in traffic. Therefore, sustainability considerations would be a valuable addition to the LC module and increase the chances of its acceptance in society. In terms of AI methods, most AI approaches to lane changing in CAVs use DRL. DRL, however, requires a significant amount of dedicated computing capacity for training and execution. Very few LC modules have provided the hardware specification of the machine on which the simulation was executed, and it has not been investigated whether the computing requirements are likely to be available in an individual CAV. Other emerging AI-based methods, such as swarm intelligence and federated learning, can potentially train high-quality controllers with minimal computing power requirements for individual CAVs. Swarm intelligence is currently used in ITS for network congestion control <ref type="bibr" target="#b8">[9,</ref><ref type="bibr" target="#b45">46]</ref> and for designing longitudinal control to create platoons with safe gaps to allow lane change by other vehicles <ref type="bibr" target="#b41">[42]</ref>, however, the application of swarm intelligence in lateral CAV control has not yet been evaluated. Similarly, federated learning can allow training a high-quality model with a distributed training mechanism <ref type="bibr" target="#b43">[44]</ref>, while preserving the privacy of individual CAVs. Current applications of federated learning in CAVs address only longitudinal CAV control, so its application to lateral CAV control needs to be investigated.</p><p>Most of the LC modules presented in Table <ref type="table" target="#tab_0">1</ref> are designed for a specific lane change scenario. Although some LC modules consider a generic approach, their evaluation considers only single or simplified traffic scenarios. In a real-world situation, a CAV may need to perform lane changes in different scenarios in a single journey. Therefore, a practical LC module needs to consider all possible scenarios of lane change in its design. This consideration can be implemented by using a generic LC module which can adapt to all scenarios of lane change.</p><p>For the architecture of the LC modules, the decentralised architecture is a popular choice. This could be due to the high cost and time required to deploy the ITS infrastructure, which is necessary to support centralised LC modules <ref type="bibr" target="#b20">[21]</ref>. The ITS infrastructure may include edge servers, roadside units, centralised servers, and V2I communication infrastructure. On the other hand, while decentralised architecture may not require any high-cost external infrastructure, establishing reliable coordination among CAVs is challenging.</p><p>Most AI-based LC modules have considered operation in mixed traffic, though some have left it for future work <ref type="bibr" target="#b9">[10]</ref>. Simulation of the mixed traffic scenario was modelled using the baseline car-following and lane changing models (MOBIL, LC2013) for HDVs in most cases. However, using the same standard driving model for HDVs may not reflect realistic mixed traffic. It is important to design a realistic mixed traffic scenario for simulation that can accurately predict the effect of CAV driving on the traffic <ref type="bibr" target="#b33">[34]</ref>. Therefore, uncertainties must be considered in HDV models to create a realistic simulation environment with mixed traffic.</p><p>Beyond these challenges, some assumptions of AI-based LC modules for CAVs may limit their applicability to practical solutions. Specifically, most of the LC modules surveyed in this paper assume the LC module to be a single concrete unit that can make lane change decisions and control acceleration. Moreover, they assume a lane change to complete in a single time step. In practical situations, however, a lane change is a complex task for CAVs, and might require the interaction of multiple independent processing modules, such as lane change decision making, trajectory planning, predicting changes in the environment, negotiating a lane change, etc. Therefore, a modular approach, which provides flexibility in developing a module in multiple dimensions, is likely to be more suitable for building a practical and realistic LC module.</p></div><figure xmlns="http://www.tei-c.org/ns/1.0" xml:id="fig_0"><head>Figure 1 :</head><label>1</label><figDesc>Figure 1: Architecture of CAV lane changing</figDesc><graphic coords="4,89.29,84.19,416.68,208.17" type="bitmap" /></figure>
<figure xmlns="http://www.tei-c.org/ns/1.0" type="table" xml:id="tab_0"><head>Table 1</head><label>1</label><figDesc>AI-based CAV LC modules</figDesc><table><row><cell>Reference</cell><cell cols="2">Year Objectives</cell><cell>AI method</cell><cell>Lane change scenario</cell><cell>Architecture</cell><cell>Mixed Traffic</cell></row><row><cell>Yu et al.</cell><cell>2020</cell><cell>safety mobility</cell><cell>DQN</cell><cell>Discretionary</cell><cell cols="2">Decentralised No</cell></row><row><cell></cell><cell></cell><cell>comfort</cell><cell></cell><cell></cell><cell></cell><cell></cell></row><row><cell>Dong et al.</cell><cell cols="2">2021 safety comfort</cell><cell>DQN</cell><cell>Discretionary</cell><cell cols="2">Decentralised Yes</cell></row><row><cell>Chen et al.</cell><cell>2021</cell><cell>safety mobility</cell><cell>Graph NN + DQN</cell><cell>Mandatory</cell><cell>Centralised</cell><cell>Yes</cell></row><row><cell></cell><cell></cell><cell>comfort</cell><cell></cell><cell></cell><cell></cell><cell></cell></row><row><cell>Ha et al.</cell><cell cols="2">2020 safety mobility</cell><cell cols="2">GCN + ACN Bottleneck</cell><cell>Centralised</cell><cell>Yes</cell></row><row><cell>Ren et al.</cell><cell cols="2">2020 safety mobility</cell><cell>ACN</cell><cell>Bottleneck</cell><cell cols="2">Decentralised No</cell></row><row><cell>Zhou et al.</cell><cell>2021</cell><cell>safety mobility</cell><cell>ACN</cell><cell>Discretionary</cell><cell cols="2">Decentralised Yes</cell></row><row><cell></cell><cell></cell><cell>comfort</cell><cell></cell><cell></cell><cell></cell><cell></cell></row><row><cell cols="3">Hou and Graf 2021 mobility sustainability</cell><cell>ACN</cell><cell>Mandatory</cell><cell cols="2">Decentralised No</cell></row><row><cell>Hwang et al.</cell><cell cols="2">2022 safety mobility</cell><cell>Hybrid: FSM + ACN</cell><cell>Discretionary</cell><cell cols="2">Decentralised Yes</cell></row><row><cell>Liao et al.</cell><cell cols="2">2022 safety mobility</cell><cell>Hierarchial: LSTM + IRL</cell><cell>Mandatory</cell><cell cols="2">Decentralised Yes</cell></row><row><cell cols="3">Bang and Ahn 2017 safety mobility</cell><cell>Swarm Intelligence</cell><cell>-</cell><cell cols="2">Decentralised Yes</cell></row><row><cell>Zeng et al.</cell><cell cols="2">2021 safety mobility</cell><cell>Federated Learning</cell><cell>-</cell><cell cols="2">Decentralised Yes</cell></row></table></figure>
		</body>
		<back>

			<div type="acknowledgement">
<div xmlns="http://www.tei-c.org/ns/1.0"><head>Acknowledgments</head><p>The authors wish to thank the editors and anonymous reviewers for their valuable comments and helpful suggestions which greatly improved the paper's quality. This work was supported by the SFI Centre for Research Training in Advanced Networks for Sustainable Societies (ADVANCE CRT), Ireland under the Grant number 18/CRT/6222.</p></div>
			</div>

			<div type="references">

				<listBibl>

<biblStruct xml:id="b0">
	<monogr>
		<author>
			<persName><surname>Smmt</surname></persName>
		</author>
		<ptr target="https://www.smmt.co.uk/wp-content/uploads/sites/2/SMMT-CAV-position-paper-final.pdf" />
		<title level="m">Connected and Autonomous Vehicles Position Paper, Technical Report</title>
				<imprint>
			<publisher>SMMT</publisher>
			<date type="published" when="2017">2017</date>
		</imprint>
	</monogr>
</biblStruct>

<biblStruct xml:id="b1">
	<analytic>
		<title level="a" type="main">Networking and Communications in Autonomous Driving: A Survey</title>
		<author>
			<persName><forename type="first">J</forename><surname>Wang</surname></persName>
		</author>
		<author>
			<persName><forename type="first">J</forename><surname>Liu</surname></persName>
		</author>
		<author>
			<persName><forename type="first">N</forename><surname>Kato</surname></persName>
		</author>
		<idno type="DOI">10.1109/COMST.2018.2888904</idno>
	</analytic>
	<monogr>
		<title level="m">conference Name: IEEE Communications Surveys Tutorials</title>
				<imprint>
			<date type="published" when="2019">2019</date>
			<biblScope unit="volume">21</biblScope>
			<biblScope unit="page" from="1243" to="1274" />
		</imprint>
	</monogr>
</biblStruct>

<biblStruct xml:id="b2">
	<monogr>
		<author>
			<persName><forename type="first">W</forename><surname>Zhou</surname></persName>
		</author>
		<author>
			<persName><forename type="first">D</forename><surname>Chen</surname></persName>
		</author>
		<author>
			<persName><forename type="first">J</forename><surname>Yan</surname></persName>
		</author>
		<author>
			<persName><forename type="first">Z</forename><surname>Li</surname></persName>
		</author>
		<author>
			<persName><forename type="first">H</forename><surname>Yin</surname></persName>
		</author>
		<author>
			<persName><forename type="first">W</forename><surname>Ge</surname></persName>
		</author>
		<idno type="arXiv">arXiv:2111.06318</idno>
		<idno>arXiv:</idno>
		<ptr target="2111.06318" />
		<title level="m">Multi-agent Reinforcement Learning for Cooperative Lane Changing of Connected and Autonomous Vehicles in Mixed Traffic</title>
				<imprint>
			<date type="published" when="2021">2021</date>
		</imprint>
	</monogr>
</biblStruct>

<biblStruct xml:id="b3">
	<analytic>
		<title level="a" type="main">Automated vehicle-involved traffic flow studies: A survey of assumptions, models, speculations, and perspectives</title>
		<author>
			<persName><forename type="first">H</forename><surname>Yu</surname></persName>
		</author>
		<author>
			<persName><forename type="first">R</forename><surname>Jiang</surname></persName>
		</author>
		<author>
			<persName><forename type="first">Z</forename><surname>He</surname></persName>
		</author>
		<author>
			<persName><forename type="first">Z</forename><surname>Zheng</surname></persName>
		</author>
		<author>
			<persName><forename type="first">L</forename><surname>Li</surname></persName>
		</author>
		<author>
			<persName><forename type="first">R</forename><surname>Liu</surname></persName>
		</author>
		<author>
			<persName><forename type="first">X</forename><surname>Chen</surname></persName>
		</author>
		<idno type="DOI">10.1016/j.trc.2021.103101</idno>
		<ptr target="https://linkinghub.elsevier.com/retrieve/pii/S0968090X21001224.doi:10.1016/j.trc.2021.103101" />
	</analytic>
	<monogr>
		<title level="j">Transportation Research Part C: Emerging Technologies</title>
		<imprint>
			<biblScope unit="volume">127</biblScope>
			<biblScope unit="page">103101</biblScope>
			<date type="published" when="2021">2021</date>
		</imprint>
	</monogr>
</biblStruct>

<biblStruct xml:id="b4">
	<analytic>
		<title level="a" type="main">Space-weighted information fusion using deep reinforcement learning: The context of tactical control of lane-changing autonomous vehicles and connectivity range assessment</title>
		<author>
			<persName><forename type="first">J</forename><surname>Dong</surname></persName>
		</author>
		<author>
			<persName><forename type="first">S</forename><surname>Chen</surname></persName>
		</author>
		<author>
			<persName><forename type="first">Y</forename><surname>Li</surname></persName>
		</author>
		<author>
			<persName><forename type="first">R</forename><surname>Du</surname></persName>
		</author>
		<author>
			<persName><forename type="first">A</forename><surname>Steinfeld</surname></persName>
		</author>
		<author>
			<persName><forename type="first">S</forename><surname>Labi</surname></persName>
		</author>
		<idno type="DOI">10.1016/j.trc.2021.103192</idno>
		<ptr target="https://www.sciencedirect.com/science/article/pii/S0968090X21002084.doi:10.1016/j.trc.2021.103192" />
	</analytic>
	<monogr>
		<title level="j">Transportation Research Part C: Emerging Technologies</title>
		<imprint>
			<biblScope unit="volume">128</biblScope>
			<biblScope unit="page">103192</biblScope>
			<date type="published" when="2021">2021</date>
		</imprint>
	</monogr>
</biblStruct>

<biblStruct xml:id="b5">
	<analytic>
		<title level="a" type="main">Multi-Sensor Fusion in Automated Driving: A Survey</title>
		<author>
			<persName><forename type="first">Z</forename><surname>Wang</surname></persName>
		</author>
		<author>
			<persName><forename type="first">Y</forename><surname>Wu</surname></persName>
		</author>
		<author>
			<persName><forename type="first">Q</forename><surname>Niu</surname></persName>
		</author>
		<idno type="DOI">10.1109/ACCESS.2019.2962554</idno>
	</analytic>
	<monogr>
		<title level="m">conference Name: IEEE Access</title>
				<imprint>
			<date type="published" when="2020">2020</date>
			<biblScope unit="volume">8</biblScope>
			<biblScope unit="page" from="2847" to="2868" />
		</imprint>
	</monogr>
</biblStruct>

<biblStruct xml:id="b6">
	<analytic>
		<title level="a" type="main">Coordinated decisions of discretionary lane change between connected and automated vehicles on freeways: a game theorybased lane change strategy</title>
		<author>
			<persName><forename type="first">Y</forename><surname>Zheng</surname></persName>
		</author>
		<author>
			<persName><forename type="first">W</forename><surname>Ding</surname></persName>
		</author>
		<author>
			<persName><forename type="first">B</forename><surname>Ran</surname></persName>
		</author>
		<author>
			<persName><forename type="first">X</forename><surname>Qu</surname></persName>
		</author>
		<author>
			<persName><forename type="first">Y</forename><surname>Zhang</surname></persName>
		</author>
		<idno type="DOI">10.1049/iet-its.2020.0146</idno>
		<ptr target="https://onlinelibrary.wiley.com/doi/pdf/10.1049/iet-its.2020.0146" />
	</analytic>
	<monogr>
		<title level="j">IET Intelligent Transport Systems</title>
		<imprint>
			<biblScope unit="volume">14</biblScope>
			<biblScope unit="page" from="1864" to="1870" />
			<date type="published" when="2020">2020</date>
		</imprint>
	</monogr>
</biblStruct>

<biblStruct xml:id="b7">
	<analytic>
		<title level="a" type="main">Altruistic Control of Connected Automated Vehicles in Mixed-Autonomy Multi-Lane Highway Traffic</title>
		<author>
			<persName><forename type="first">M</forename><forename type="middle">F</forename><surname>Keskin</surname></persName>
		</author>
		<author>
			<persName><forename type="first">B</forename><surname>Peng</surname></persName>
		</author>
		<author>
			<persName><forename type="first">B</forename><surname>Kulcsar</surname></persName>
		</author>
		<author>
			<persName><forename type="first">H</forename><surname>Wymeersch</surname></persName>
		</author>
		<idno type="DOI">10.1016/j.ifacol.2020.12.1990</idno>
		<ptr target="https://linkinghub.elsevier.com/retrieve/pii/S2405896320326215.doi:10.1016/j.ifacol.2020.12.1990" />
	</analytic>
	<monogr>
		<title level="j">IFAC-PapersOnLine</title>
		<imprint>
			<biblScope unit="volume">53</biblScope>
			<biblScope unit="page" from="14966" to="14971" />
			<date type="published" when="2020">2020</date>
		</imprint>
	</monogr>
</biblStruct>

<biblStruct xml:id="b8">
	<analytic>
		<title level="a" type="main">Artificial Intelligence for Vehicle-to-Everything: A Survey</title>
		<author>
			<persName><forename type="first">W</forename><surname>Tong</surname></persName>
		</author>
		<author>
			<persName><forename type="first">A</forename><surname>Hussain</surname></persName>
		</author>
		<author>
			<persName><forename type="first">W</forename><forename type="middle">X</forename><surname>Bo</surname></persName>
		</author>
		<author>
			<persName><forename type="first">S</forename><surname>Maharjan</surname></persName>
		</author>
		<idno type="DOI">10.1109/ACCESS.2019.2891073</idno>
	</analytic>
	<monogr>
		<title level="j">IEEE Access</title>
		<imprint>
			<biblScope unit="volume">7</biblScope>
			<biblScope unit="page" from="10823" to="10843" />
			<date type="published" when="2019">2019</date>
		</imprint>
	</monogr>
	<note>conference Name: IEEE Access</note>
</biblStruct>

<biblStruct xml:id="b9">
	<monogr>
		<title level="m" type="main">Decentralized Cooperative Lane Changing at Freeway Weaving Areas Using Multi-Agent Deep Reinforcement Learning</title>
		<author>
			<persName><forename type="first">Y</forename><surname>Hou</surname></persName>
		</author>
		<author>
			<persName><forename type="first">P</forename><surname>Graf</surname></persName>
		</author>
		<idno type="arXiv">arXiv:2110.08124[cs</idno>
		<idno>arXiv:</idno>
		<ptr target="2110.08124" />
		<imprint>
			<date type="published" when="2021">2021</date>
		</imprint>
	</monogr>
</biblStruct>

<biblStruct xml:id="b10">
	<analytic>
		<title level="a" type="main">Deep Learning for Intelligent Transportation Systems: A Survey of Emerging Trends</title>
		<author>
			<persName><forename type="first">M</forename><surname>Veres</surname></persName>
		</author>
		<author>
			<persName><forename type="first">M</forename><surname>Moussa</surname></persName>
		</author>
		<idno type="DOI">10.1109/TITS.2019.2929020</idno>
	</analytic>
	<monogr>
		<title level="m">conference Name: IEEE Transactions on Intelligent Transportation Systems</title>
				<imprint>
			<date type="published" when="2020">2020</date>
			<biblScope unit="volume">21</biblScope>
			<biblScope unit="page" from="3152" to="3168" />
		</imprint>
	</monogr>
</biblStruct>

<biblStruct xml:id="b11">
	<analytic>
		<title level="a" type="main">Review of Lane-Changing Maneuvers of Connected and Automated Vehicles: Models, Algorithms and Traffic Impact Analyses</title>
		<author>
			<persName><forename type="first">Z</forename><surname>Wang</surname></persName>
		</author>
		<author>
			<persName><forename type="first">X</forename><surname>Shi</surname></persName>
		</author>
		<author>
			<persName><forename type="first">X</forename><surname>Li</surname></persName>
		</author>
		<idno type="DOI">10.1007/s41745-019-00127-7</idno>
		<ptr target="https://doi.org/10.1007/s41745-019-00127-7.doi:10.1007/s41745-019-00127-7" />
	</analytic>
	<monogr>
		<title level="j">Journal of the Indian Institute of Science</title>
		<imprint>
			<biblScope unit="volume">99</biblScope>
			<biblScope unit="page" from="589" to="599" />
			<date type="published" when="2019">2019</date>
		</imprint>
	</monogr>
</biblStruct>

<biblStruct xml:id="b12">
	<analytic>
		<title level="a" type="main">A model for the structure of lane-changing decisions</title>
		<author>
			<persName><forename type="first">P</forename><forename type="middle">G</forename><surname>Gipps</surname></persName>
		</author>
		<idno type="DOI">10.1016/0191-2615(86)90012-3</idno>
		<ptr target="https://www.sciencedirect.com/science/article/pii/0191261586900123.doi:10.1016/0191-2615(86)90012-3" />
	</analytic>
	<monogr>
		<title level="j">Transportation Research Part B: Methodological</title>
		<imprint>
			<biblScope unit="volume">20</biblScope>
			<biblScope unit="page" from="403" to="414" />
			<date type="published" when="1986">1986</date>
		</imprint>
	</monogr>
</biblStruct>

<biblStruct xml:id="b13">
	<analytic>
		<title level="a" type="main">A Microscopic Traffic Simulator for evaluation of dynamic traffic management systems</title>
		<author>
			<persName><forename type="first">Q</forename><surname>Yang</surname></persName>
		</author>
		<author>
			<persName><forename type="first">H</forename><forename type="middle">N</forename><surname>Koutsopoulos</surname></persName>
		</author>
		<idno type="DOI">10.1016/S0968-090X(96)00006-X</idno>
		<ptr target="https://www.sciencedirect.com/science/article/pii/S0968090X9600006X.doi:10.1016/S0968-090X(96)00006-X" />
	</analytic>
	<monogr>
		<title level="j">Transportation Research Part C: Emerging Technologies</title>
		<imprint>
			<biblScope unit="volume">4</biblScope>
			<biblScope unit="page" from="113" to="129" />
			<date type="published" when="1996">1996</date>
		</imprint>
	</monogr>
</biblStruct>

<biblStruct xml:id="b14">
	<analytic>
		<title level="a" type="main">Recent developments and research needs in modeling lane changing</title>
		<author>
			<persName><forename type="first">Z</forename><surname>Zheng</surname></persName>
		</author>
		<idno type="DOI">10.1016/j.trb.2013.11.009</idno>
		<ptr target="https://www.sciencedirect.com/science/article/pii/S019126151300218X.doi:10.1016/j.trb.2013.11.009" />
	</analytic>
	<monogr>
		<title level="j">Transportation Research Part B: Methodological</title>
		<imprint>
			<biblScope unit="volume">60</biblScope>
			<biblScope unit="page" from="16" to="32" />
			<date type="published" when="2014">2014</date>
		</imprint>
	</monogr>
</biblStruct>

<biblStruct xml:id="b15">
	<analytic>
		<title level="a" type="main">General Lane-Changing Model MOBIL for Car-Following Models</title>
		<author>
			<persName><forename type="first">A</forename><surname>Kesting</surname></persName>
		</author>
		<author>
			<persName><forename type="first">M</forename><surname>Treiber</surname></persName>
		</author>
		<author>
			<persName><forename type="first">D</forename><surname>Helbing</surname></persName>
		</author>
		<idno type="DOI">10.3141/1999-10</idno>
		<ptr target="https://doi.org/10.3141/1999-10.doi:10.3141/1999-10" />
	</analytic>
	<monogr>
		<title level="j">Transportation Research Record</title>
		<imprint>
			<biblScope unit="page" from="86" to="94" />
			<date type="published" when="1999">1999. 2007</date>
			<publisher>SAGE Publications Inc</publisher>
		</imprint>
	</monogr>
</biblStruct>

<biblStruct xml:id="b16">
	<analytic>
		<title level="a" type="main">Towards Tactical Lane Change Behavior Planning for Automated Vehicles</title>
		<author>
			<persName><forename type="first">S</forename><surname>Ulbrich</surname></persName>
		</author>
		<author>
			<persName><forename type="first">M</forename><surname>Maurer</surname></persName>
		</author>
		<idno type="DOI">10.1109/ITSC.2015.165</idno>
	</analytic>
	<monogr>
		<title level="m">IEEE 18th International Conference on Intelligent Transportation Systems</title>
				<imprint>
			<date type="published" when="2015">2015. 2015</date>
			<biblScope unit="page" from="989" to="995" />
		</imprint>
	</monogr>
</biblStruct>

<biblStruct xml:id="b17">
	<analytic>
		<title level="a" type="main">SUMO&apos;s Lane-changing model</title>
		<author>
			<persName><forename type="first">J</forename><surname>Erdmann</surname></persName>
		</author>
		<idno type="DOI">10.1007/978-3-319-15024-6_7</idno>
		<ptr target="http://link.springer.com/chapter/10.1007/978-3-319-15024-6_7" />
	</analytic>
	<monogr>
		<title level="s">LECTURE NOTES IN CONTROL AND INFORMATION SCIENCES</title>
		<editor>M. Behrisch, M. Weber</editor>
		<imprint>
			<biblScope unit="volume">13</biblScope>
			<biblScope unit="page" from="105" to="123" />
			<date type="published" when="2015">2015</date>
			<publisher>Springer Verlag</publisher>
		</imprint>
	</monogr>
</biblStruct>

<biblStruct xml:id="b18">
	<monogr>
		<author>
			<persName><forename type="first">J</forename><surname>He</surname></persName>
		</author>
		<author>
			<persName><forename type="first">K</forename><surname>Yang</surname></persName>
		</author>
		<author>
			<persName><forename type="first">H.-H</forename><surname>Chen</surname></persName>
		</author>
		<idno type="arXiv">arXiv:2010.00972</idno>
		<idno>arXiv:</idno>
		<ptr target="2010.00972" />
		<title level="m">6G Cellular Networks and Connected Autonomous Vehicles</title>
				<imprint>
			<date type="published" when="2020">2020</date>
		</imprint>
	</monogr>
	<note>cs, eess</note>
</biblStruct>

<biblStruct xml:id="b19">
	<monogr>
		<author>
			<persName><forename type="first">P</forename><forename type="middle">Y J</forename><surname>Ha</surname></persName>
		</author>
		<author>
			<persName><forename type="first">S</forename><surname>Chen</surname></persName>
		</author>
		<author>
			<persName><forename type="first">J</forename><surname>Dong</surname></persName>
		</author>
		<author>
			<persName><forename type="first">R</forename><surname>Du</surname></persName>
		</author>
		<author>
			<persName><forename type="first">Y</forename><surname>Li</surname></persName>
		</author>
		<author>
			<persName><forename type="first">S</forename><surname>Labi</surname></persName>
		</author>
		<idno type="arXiv">arXiv:2010.05436</idno>
		<idno>arXiv:</idno>
		<ptr target="2010.05436" />
		<title level="m">Leveraging the Capabilities of Connected and Autonomous Vehicles and Multi-Agent Reinforcement Learning to Mitigate Highway Bottleneck Congestion</title>
				<imprint>
			<date type="published" when="2020">2020</date>
		</imprint>
	</monogr>
	<note>cs, eess</note>
</biblStruct>

<biblStruct xml:id="b20">
	<analytic>
		<title level="a" type="main">Graph neural network and reinforcement learning for multi-agent cooperative control of connected autonomous vehicles</title>
		<author>
			<persName><forename type="first">S</forename><surname>Chen</surname></persName>
		</author>
		<author>
			<persName><forename type="first">J</forename><surname>Dong</surname></persName>
		</author>
		<author>
			<persName><forename type="first">P</forename><forename type="middle">Y J</forename><surname>Ha</surname></persName>
		</author>
		<author>
			<persName><forename type="first">Y</forename><surname>Li</surname></persName>
		</author>
		<author>
			<persName><forename type="first">S</forename><surname>Labi</surname></persName>
		</author>
		<idno type="DOI">10.1111/mice.12702</idno>
		<ptr target="https://onlinelibrary.wiley.com/doi/pdf/10.1111/mice.12702" />
	</analytic>
	<monogr>
		<title level="j">Computer-Aided Civil and Infrastructure Engineering</title>
		<imprint>
			<biblScope unit="volume">36</biblScope>
			<biblScope unit="page" from="838" to="857" />
			<date type="published" when="2021">2021</date>
		</imprint>
	</monogr>
</biblStruct>

<biblStruct xml:id="b21">
	<analytic>
		<title level="a" type="main">Modeling decentralized mandatory lane change for connected and autonomous vehicles: An analytical method</title>
		<author>
			<persName><forename type="first">Z</forename><surname>Wang</surname></persName>
		</author>
		<author>
			<persName><forename type="first">X</forename><surname>Shi</surname></persName>
		</author>
		<author>
			<persName><forename type="first">X</forename><surname>Zhao</surname></persName>
		</author>
		<author>
			<persName><forename type="first">X</forename><surname>Li</surname></persName>
		</author>
		<idno type="DOI">10.1016/j.trc.2021.103441</idno>
		<ptr target="https://www.sciencedirect.com/science/article/pii/S0968090X21004319.doi:10.1016/j.trc.2021.103441" />
	</analytic>
	<monogr>
		<title level="j">Transportation Research Part C: Emerging Technologies</title>
		<imprint>
			<biblScope unit="volume">133</biblScope>
			<biblScope unit="page">103441</biblScope>
			<date type="published" when="2021">2021</date>
		</imprint>
	</monogr>
</biblStruct>

<biblStruct xml:id="b22">
	<analytic>
		<title level="a" type="main">Autonomous Vehicle Cut-In Algorithm for Lane-Merging Scenarios via Policy-Based Reinforcement Learning Nested Within Finite-State Machine</title>
		<author>
			<persName><forename type="first">S</forename><surname>Hwang</surname></persName>
		</author>
		<author>
			<persName><forename type="first">K</forename><surname>Lee</surname></persName>
		</author>
		<author>
			<persName><forename type="first">H</forename><surname>Jeon</surname></persName>
		</author>
		<author>
			<persName><forename type="first">D</forename><surname>Kum</surname></persName>
		</author>
		<idno type="DOI">10.1109/TITS.2022.3153848</idno>
		<ptr target="https://ieeexplore-ieee-org.elib.tcd.ie/document/9729796.doi:10.1109/TITS.2022.3153848" />
	</analytic>
	<monogr>
		<title level="m">IEEE Transactions on Intelligent Transportation Systems</title>
				<imprint>
			<date type="published" when="2022">2022</date>
			<biblScope unit="page" from="1" to="13" />
		</imprint>
	</monogr>
</biblStruct>

<biblStruct xml:id="b23">
	<monogr>
		<title level="m" type="main">Online Prediction of Lane Change with a Hierarchical Learning-Based Approach</title>
		<author>
			<persName><forename type="first">X</forename><surname>Liao</surname></persName>
		</author>
		<author>
			<persName><forename type="first">Z</forename><surname>Wang</surname></persName>
		</author>
		<author>
			<persName><forename type="first">X</forename><surname>Zhao</surname></persName>
		</author>
		<author>
			<persName><forename type="first">Z</forename><surname>Zhao</surname></persName>
		</author>
		<author>
			<persName><forename type="first">K</forename><surname>Han</surname></persName>
		</author>
		<author>
			<persName><forename type="first">P</forename><surname>Tiwari</surname></persName>
		</author>
		<author>
			<persName><forename type="first">M</forename><surname>Barth</surname></persName>
		</author>
		<author>
			<persName><forename type="first">G</forename><surname>Wu</surname></persName>
		</author>
		<imprint>
			<date type="published" when="2022">2022</date>
		</imprint>
	</monogr>
</biblStruct>

<biblStruct xml:id="b24">
	<analytic>
		<title level="a" type="main">Cooperative Highway Work Zone Merge Control Based on Reinforcement Learning in a Connected and Automated Environment</title>
		<author>
			<persName><forename type="first">T</forename><surname>Ren</surname></persName>
		</author>
		<author>
			<persName><forename type="first">Y</forename><surname>Xie</surname></persName>
		</author>
		<author>
			<persName><forename type="first">L</forename><surname>Jiang</surname></persName>
		</author>
		<idno type="DOI">10.1177/0361198120935873</idno>
		<ptr target="https://doi.org/10.1177/0361198120935873.doi:10.1177/0361198120935873" />
	</analytic>
	<monogr>
		<title level="j">Transportation Research Record</title>
		<imprint>
			<biblScope unit="volume">2674</biblScope>
			<biblScope unit="page" from="363" to="374" />
			<date type="published" when="2020">2020</date>
			<publisher>SAGE Publications Inc</publisher>
		</imprint>
	</monogr>
</biblStruct>

<biblStruct xml:id="b25">
	<analytic>
		<title level="a" type="main">Distributed Multiagent Coordinated Learning for Autonomous Driving in Highways Based on Dynamic Coordination Graphs</title>
		<author>
			<persName><forename type="first">C</forename><surname>Yu</surname></persName>
		</author>
		<author>
			<persName><forename type="first">X</forename><surname>Wang</surname></persName>
		</author>
		<author>
			<persName><forename type="first">X</forename><surname>Xu</surname></persName>
		</author>
		<author>
			<persName><forename type="first">M</forename><surname>Zhang</surname></persName>
		</author>
		<author>
			<persName><forename type="first">H</forename><surname>Ge</surname></persName>
		</author>
		<author>
			<persName><forename type="first">J</forename><surname>Ren</surname></persName>
		</author>
		<author>
			<persName><forename type="first">L</forename><surname>Sun</surname></persName>
		</author>
		<author>
			<persName><forename type="first">B</forename><surname>Chen</surname></persName>
		</author>
		<author>
			<persName><forename type="first">G</forename><surname>Tan</surname></persName>
		</author>
		<idno type="DOI">10.1109/TITS.2019.2893683</idno>
	</analytic>
	<monogr>
		<title level="m">conference Name: IEEE Transactions on Intelligent Transportation Systems</title>
				<imprint>
			<date type="published" when="2020">2020</date>
			<biblScope unit="volume">21</biblScope>
			<biblScope unit="page" from="735" to="748" />
		</imprint>
	</monogr>
</biblStruct>

<biblStruct xml:id="b26">
	<monogr>
		<author>
			<persName><forename type="first">Y</forename><surname>Lin</surname></persName>
		</author>
		<author>
			<persName><forename type="first">P</forename><surname>Wang</surname></persName>
		</author>
		<author>
			<persName><forename type="first">M</forename><surname>Ma</surname></persName>
		</author>
		<idno type="DOI">10.1109/BigDataSecurity.2017.50</idno>
		<title level="m">Intelligent Transportation System(ITS): Concept, Challenge and Opportunity, in: 2017 ieee 3rd international conference on big data security on cloud (bigdatasecurity), ieee international conference on high performance and smart computing (hpsc), and ieee international conference on intelligent data and security (ids)</title>
				<imprint>
			<date type="published" when="2017">2017</date>
			<biblScope unit="page" from="167" to="172" />
		</imprint>
	</monogr>
</biblStruct>

<biblStruct xml:id="b27">
	<analytic>
		<title level="a" type="main">A Survey on Cooperative Architectures and Maneuvers for Connected and Automated Vehicles</title>
		<author>
			<persName><forename type="first">B</forename><surname>Häfner</surname></persName>
		</author>
		<author>
			<persName><forename type="first">V</forename><surname>Bajpai</surname></persName>
		</author>
		<author>
			<persName><forename type="first">J</forename><surname>Ott</surname></persName>
		</author>
		<author>
			<persName><forename type="first">G</forename><forename type="middle">A</forename><surname>Schmitt</surname></persName>
		</author>
		<idno type="DOI">10.1109/COMST.2021.3138275</idno>
	</analytic>
	<monogr>
		<title level="m">conference Name: IEEE Communications Surveys Tutorials</title>
				<imprint>
			<date type="published" when="2021">2021</date>
			<biblScope unit="page" from="1" to="1" />
		</imprint>
	</monogr>
</biblStruct>

<biblStruct xml:id="b28">
	<analytic>
		<title level="a" type="main">Mean Field Multi-Agent Reinforcement Learning</title>
		<author>
			<persName><forename type="first">Y</forename><surname>Yang</surname></persName>
		</author>
		<author>
			<persName><forename type="first">R</forename><surname>Luo</surname></persName>
		</author>
		<author>
			<persName><forename type="first">M</forename><surname>Li</surname></persName>
		</author>
		<author>
			<persName><forename type="first">M</forename><surname>Zhou</surname></persName>
		</author>
		<author>
			<persName><forename type="first">W</forename><surname>Zhang</surname></persName>
		</author>
		<author>
			<persName><forename type="first">J</forename><surname>Wang</surname></persName>
		</author>
		<ptr target="iSSN:2640-3498" />
	</analytic>
	<monogr>
		<title level="m">Proceedings of the 35th International Conference on Machine Learning</title>
				<meeting>the 35th International Conference on Machine Learning<address><addrLine>PMLR</addrLine></address></meeting>
		<imprint>
			<date type="published" when="2018">2018</date>
			<biblScope unit="page" from="5571" to="5580" />
		</imprint>
	</monogr>
</biblStruct>

<biblStruct xml:id="b29">
	<analytic>
		<title level="a" type="main">A Survey on Intelligent Control for Multiagent Systems</title>
		<author>
			<persName><forename type="first">P</forename><surname>Shi</surname></persName>
		</author>
		<author>
			<persName><forename type="first">B</forename><surname>Yan</surname></persName>
		</author>
		<idno type="DOI">10.1109/TSMC.2020.3042823</idno>
	</analytic>
	<monogr>
		<title level="m">conference Name: IEEE Transactions on Systems, Man, and Cybernetics: Systems</title>
				<imprint>
			<date type="published" when="2021">2021</date>
			<biblScope unit="volume">51</biblScope>
			<biblScope unit="page" from="161" to="175" />
		</imprint>
	</monogr>
</biblStruct>

<biblStruct xml:id="b30">
	<analytic>
		<title level="a" type="main">Game Theory-Based Ramp Merging for Mixed Traffic With Unity-SUMO Co-Simulation</title>
		<author>
			<persName><forename type="first">X</forename><surname>Liao</surname></persName>
		</author>
		<author>
			<persName><forename type="first">X</forename><surname>Zhao</surname></persName>
		</author>
		<author>
			<persName><forename type="first">Z</forename><surname>Wang</surname></persName>
		</author>
		<author>
			<persName><forename type="first">K</forename><surname>Han</surname></persName>
		</author>
		<author>
			<persName><forename type="first">P</forename><surname>Tiwari</surname></persName>
		</author>
		<author>
			<persName><forename type="first">M</forename><forename type="middle">J</forename><surname>Barth</surname></persName>
		</author>
		<author>
			<persName><forename type="first">G</forename><surname>Wu</surname></persName>
		</author>
		<idno type="DOI">10.1109/TSMC.2021.3131431</idno>
		<ptr target="https://doi-org.elib.tcd.ie/10.1109/TSMC.2021.3131431.doi:10.1109/TSMC.2021.3131431" />
	</analytic>
	<monogr>
		<title level="m">conference Name: IEEE Transactions on Systems, Man, and Cybernetics: Systems</title>
				<imprint>
			<date type="published" when="2021">2021</date>
			<biblScope unit="page" from="1" to="12" />
		</imprint>
	</monogr>
</biblStruct>

<biblStruct xml:id="b31">
	<monogr>
		<author>
			<persName><forename type="first">M</forename><surname>Shen</surname></persName>
		</author>
		<author>
			<persName><forename type="first">C</forename><forename type="middle">R</forename><surname>He</surname></persName>
		</author>
		<author>
			<persName><forename type="first">T</forename><surname>Molnar</surname></persName>
		</author>
		<author>
			<persName><forename type="first">A</forename><forename type="middle">H</forename><surname>Bell</surname></persName>
		</author>
		<author>
			<persName><forename type="first">G</forename><surname>Orosz</surname></persName>
		</author>
		<idno type="arXiv">arXiv:2205.03473</idno>
		<idno>arXiv:2205.03473</idno>
		<ptr target="http://arxiv.org/abs/2205.03473" />
		<title level="m">Energy-efficient Connected Cruise Control with Lean Penetration of Connected Vehicles</title>
				<imprint/>
	</monogr>
	<note type="report_type">Technical Report</note>
	<note>cs, eess] type: article</note>
</biblStruct>

<biblStruct xml:id="b32">
	<analytic>
		<title level="a" type="main">Modeling heterogeneous traffic with cooperative adaptive cruise control vehicles: A first-order macroscopic perspective</title>
		<author>
			<persName><forename type="first">Z</forename><surname>Vander Laan</surname></persName>
		</author>
		<author>
			<persName><forename type="first">P</forename><surname>Schonfeld</surname></persName>
		</author>
		<idno type="DOI">10.1080/03081060.2020.1717127</idno>
		<idno>doi:10.1080/03081060.2020.1717127</idno>
		<ptr target="https://doi.org/10.1080/03081060.2020.1717127" />
	</analytic>
	<monogr>
		<title level="j">Transportation Planning and Technology</title>
		<imprint>
			<biblScope unit="volume">43</biblScope>
			<biblScope unit="page" from="113" to="140" />
			<date type="published" when="2020">2020</date>
		</imprint>
	</monogr>
</biblStruct>

<biblStruct xml:id="b33">
	<analytic>
		<title level="a" type="main">Can Connected Autonomous Vehicles really improve mixed traffic efficiency in realistic scenarios?</title>
		<author>
			<persName><forename type="first">M</forename><surname>Garg</surname></persName>
		</author>
		<author>
			<persName><forename type="first">C</forename><surname>Johnston</surname></persName>
		</author>
		<author>
			<persName><forename type="first">M</forename><surname>Bouroche</surname></persName>
		</author>
		<idno type="DOI">10.1109/ITSC48978.2021.9565068</idno>
	</analytic>
	<monogr>
		<title level="m">2021 IEEE International Intelligent Transportation Systems Conference (ITSC)</title>
				<imprint>
			<date type="published" when="2021">2021</date>
			<biblScope unit="page" from="2011" to="2018" />
		</imprint>
	</monogr>
</biblStruct>

<biblStruct xml:id="b34">
	<analytic>
		<title level="a" type="main">Human-level control through deep reinforcement learning</title>
		<author>
			<persName><forename type="first">V</forename><surname>Mnih</surname></persName>
		</author>
		<author>
			<persName><forename type="first">K</forename><surname>Kavukcuoglu</surname></persName>
		</author>
		<author>
			<persName><forename type="first">D</forename><surname>Silver</surname></persName>
		</author>
		<author>
			<persName><forename type="first">A</forename><forename type="middle">A</forename><surname>Rusu</surname></persName>
		</author>
		<author>
			<persName><forename type="first">J</forename><surname>Veness</surname></persName>
		</author>
		<author>
			<persName><forename type="first">M</forename><forename type="middle">G</forename><surname>Bellemare</surname></persName>
		</author>
		<author>
			<persName><forename type="first">A</forename><surname>Graves</surname></persName>
		</author>
		<author>
			<persName><forename type="first">M</forename><surname>Riedmiller</surname></persName>
		</author>
		<author>
			<persName><forename type="first">A</forename><forename type="middle">K</forename><surname>Fidjeland</surname></persName>
		</author>
		<author>
			<persName><forename type="first">G</forename><surname>Ostrovski</surname></persName>
		</author>
		<author>
			<persName><forename type="first">S</forename><surname>Petersen</surname></persName>
		</author>
		<author>
			<persName><forename type="first">C</forename><surname>Beattie</surname></persName>
		</author>
		<author>
			<persName><forename type="first">A</forename><surname>Sadik</surname></persName>
		</author>
		<author>
			<persName><forename type="first">I</forename><surname>Antonoglou</surname></persName>
		</author>
		<author>
			<persName><forename type="first">H</forename><surname>King</surname></persName>
		</author>
		<author>
			<persName><forename type="first">D</forename><surname>Kumaran</surname></persName>
		</author>
		<author>
			<persName><forename type="first">D</forename><surname>Wierstra</surname></persName>
		</author>
		<author>
			<persName><forename type="first">S</forename><surname>Legg</surname></persName>
		</author>
		<author>
			<persName><forename type="first">D</forename><surname>Hassabis</surname></persName>
		</author>
		<idno type="DOI">10.1038/nature14236</idno>
		<ptr target="7540Publisher" />
	</analytic>
	<monogr>
		<title level="j">Nature</title>
		<imprint>
			<biblScope unit="volume">518</biblScope>
			<biblScope unit="page" from="529" to="533" />
			<date type="published" when="2015">2015</date>
			<publisher>Nature Publishing Group</publisher>
		</imprint>
	</monogr>
</biblStruct>

<biblStruct xml:id="b35">
	<analytic>
		<title level="a" type="main">Spatio-weighted information fusion and DRL-based control for connected autonomous vehicles</title>
		<author>
			<persName><forename type="first">J</forename><surname>Dong</surname></persName>
		</author>
		<author>
			<persName><forename type="first">S</forename><surname>Chen</surname></persName>
		</author>
		<author>
			<persName><forename type="first">Y</forename><surname>Li</surname></persName>
		</author>
		<author>
			<persName><forename type="first">P</forename><forename type="middle">Y J</forename><surname>Ha</surname></persName>
		</author>
		<author>
			<persName><forename type="first">R</forename><surname>Du</surname></persName>
		</author>
		<author>
			<persName><forename type="first">A</forename><surname>Steinfeld</surname></persName>
		</author>
		<author>
			<persName><forename type="first">S</forename><surname>Labi</surname></persName>
		</author>
		<idno type="DOI">10.1109/ITSC45102.2020.9294550</idno>
	</analytic>
	<monogr>
		<title level="m">IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC)</title>
				<imprint>
			<date type="published" when="2020">2020. 2020</date>
			<biblScope unit="page" from="1" to="6" />
		</imprint>
	</monogr>
</biblStruct>

<biblStruct xml:id="b36">
	<analytic>
		<title level="a" type="main">Understanding the discretionary lane-changing behaviour in the connected environment</title>
		<author>
			<persName><forename type="first">Y</forename><surname>Ali</surname></persName>
		</author>
		<author>
			<persName><forename type="first">Z</forename><surname>Zheng</surname></persName>
		</author>
		<author>
			<persName><forename type="first">M</forename><surname>Mazharul Haque</surname></persName>
		</author>
		<author>
			<persName><forename type="first">M</forename><surname>Yildirimoglu</surname></persName>
		</author>
		<author>
			<persName><forename type="first">S</forename><surname>Washington</surname></persName>
		</author>
		<idno type="DOI">10.1016/j.aap.2020.105463</idno>
		<ptr target="https://www.sciencedirect.com/science/article/pii/S0001457519312771.doi:10.1016/j.aap.2020.105463" />
	</analytic>
	<monogr>
		<title level="j">Accident Analysis &amp; Prevention</title>
		<imprint>
			<biblScope unit="volume">137</biblScope>
			<biblScope unit="page">105463</biblScope>
			<date type="published" when="2020">2020</date>
		</imprint>
	</monogr>
</biblStruct>

<biblStruct xml:id="b37">
	<monogr>
		<author>
			<persName><forename type="first">T</forename><forename type="middle">P</forename><surname>Lillicrap</surname></persName>
		</author>
		<author>
			<persName><forename type="first">J</forename><forename type="middle">J</forename><surname>Hunt</surname></persName>
		</author>
		<author>
			<persName><forename type="first">A</forename><surname>Pritzel</surname></persName>
		</author>
		<author>
			<persName><forename type="first">N</forename><surname>Heess</surname></persName>
		</author>
		<author>
			<persName><forename type="first">T</forename><surname>Erez</surname></persName>
		</author>
		<author>
			<persName><forename type="first">Y</forename><surname>Tassa</surname></persName>
		</author>
		<author>
			<persName><forename type="first">D</forename><surname>Silver</surname></persName>
		</author>
		<author>
			<persName><forename type="first">D</forename><surname>Wierstra</surname></persName>
		</author>
		<idno type="arXiv">arXiv:1509.02971[cs,stat</idno>
		<idno>arXiv:</idno>
		<ptr target="1509.02971" />
		<title level="m">Continuous control with deep reinforcement learning</title>
				<imprint>
			<date type="published" when="2019">2019</date>
		</imprint>
	</monogr>
</biblStruct>

<biblStruct xml:id="b38">
	<analytic>
		<title level="a" type="main">A Survey of Actor-Critic Reinforcement Learning: Standard and Natural Policy Gradients</title>
		<author>
			<persName><forename type="first">I</forename><surname>Grondman</surname></persName>
		</author>
		<author>
			<persName><forename type="first">L</forename><surname>Busoniu</surname></persName>
		</author>
		<author>
			<persName><forename type="first">G</forename><forename type="middle">A D</forename><surname>Lopes</surname></persName>
		</author>
		<author>
			<persName><forename type="first">R</forename><surname>Babuska</surname></persName>
		</author>
		<idno type="DOI">10.1109/TSMCC.2012.2218595</idno>
	</analytic>
	<monogr>
		<title level="m">conference Name: IEEE Transactions on Systems, Man, and Cybernetics, Part C (Applications and Reviews</title>
				<imprint>
			<date type="published" when="2012">2012</date>
			<biblScope unit="volume">42</biblScope>
			<biblScope unit="page" from="1291" to="1307" />
		</imprint>
	</monogr>
</biblStruct>

<biblStruct xml:id="b39">
	<analytic>
		<title level="a" type="main">A bi-level cooperative driving strategy allowing lane changes</title>
		<author>
			<persName><forename type="first">H</forename><surname>Xu</surname></persName>
		</author>
		<author>
			<persName><forename type="first">Y</forename><surname>Zhang</surname></persName>
		</author>
		<author>
			<persName><forename type="first">C</forename><forename type="middle">G</forename><surname>Cassandras</surname></persName>
		</author>
		<author>
			<persName><forename type="first">L</forename><surname>Li</surname></persName>
		</author>
		<author>
			<persName><forename type="first">S</forename><surname>Feng</surname></persName>
		</author>
		<idno type="DOI">10.1016/j.trc.2020.102773</idno>
		<ptr target="https://www.sciencedirect.com/science/article/pii/S0968090X20306835.doi:10.1016/j.trc.2020.102773" />
	</analytic>
	<monogr>
		<title level="j">Transportation Research Part C: Emerging Technologies</title>
		<imprint>
			<biblScope unit="volume">120</biblScope>
			<biblScope unit="page">102773</biblScope>
			<date type="published" when="2020">2020</date>
		</imprint>
	</monogr>
</biblStruct>

<biblStruct xml:id="b40">
	<analytic>
		<title level="a" type="main">Swarm intelligence for routing in communication networks</title>
		<author>
			<persName><forename type="first">I</forename><surname>Kassabalidis</surname></persName>
		</author>
		<author>
			<persName><forename type="first">M</forename><surname>El-Sharkawi</surname></persName>
		</author>
		<author>
			<persName><forename type="first">R</forename><surname>Marks</surname></persName>
		</author>
		<author>
			<persName><forename type="first">P</forename><surname>Arabshahi</surname></persName>
		</author>
		<author>
			<persName><forename type="first">A</forename><surname>Gray</surname></persName>
		</author>
		<idno type="DOI">10.1109/GLOCOM.2001.966355</idno>
	</analytic>
	<monogr>
		<title level="m">GLOBECOM&apos;01. IEEE Global Telecommunications Conference (Cat. No.01CH37270)</title>
				<imprint>
			<date type="published" when="2001">2001</date>
			<biblScope unit="volume">6</biblScope>
			<biblScope unit="page" from="3613" to="3617" />
		</imprint>
	</monogr>
</biblStruct>

<biblStruct xml:id="b41">
	<analytic>
		<title level="a" type="main">Platooning Strategy for Connected and Autonomous Vehicles: Transition from Light Traffic</title>
		<author>
			<persName><forename type="first">S</forename><surname>Bang</surname></persName>
		</author>
		<author>
			<persName><forename type="first">S</forename><surname>Ahn</surname></persName>
		</author>
		<idno type="DOI">10.3141/2623-08</idno>
		<ptr target="https://doi.org/10.3141/2623-08.doi:10.3141/2623-08" />
	</analytic>
	<monogr>
		<title level="j">Transportation Research Record</title>
		<imprint>
			<biblScope unit="volume">2623</biblScope>
			<biblScope unit="page" from="73" to="81" />
			<date type="published" when="2017">2017</date>
			<publisher>SAGE Publications Inc</publisher>
		</imprint>
	</monogr>
</biblStruct>

<biblStruct xml:id="b42">
	<analytic>
		<title level="a" type="main">A survey on federated learning</title>
		<author>
			<persName><forename type="first">C</forename><surname>Zhang</surname></persName>
		</author>
		<author>
			<persName><forename type="first">Y</forename><surname>Xie</surname></persName>
		</author>
		<author>
			<persName><forename type="first">H</forename><surname>Bai</surname></persName>
		</author>
		<author>
			<persName><forename type="first">B</forename><surname>Yu</surname></persName>
		</author>
		<author>
			<persName><forename type="first">W</forename><surname>Li</surname></persName>
		</author>
		<author>
			<persName><forename type="first">Y</forename><surname>Gao</surname></persName>
		</author>
		<idno type="DOI">10.1016/j.knosys.2021.106775</idno>
		<ptr target="https://www.sciencedirect.com/science/article/pii/S0950705121000381.doi:10.1016/j.knosys.2021.106775" />
	</analytic>
	<monogr>
		<title level="j">Knowledge-Based Systems</title>
		<imprint>
			<biblScope unit="volume">216</biblScope>
			<biblScope unit="page">106775</biblScope>
			<date type="published" when="2021">2021</date>
		</imprint>
	</monogr>
</biblStruct>

<biblStruct xml:id="b43">
	<monogr>
		<author>
			<persName><forename type="first">T</forename><surname>Zeng</surname></persName>
		</author>
		<author>
			<persName><forename type="first">O</forename><surname>Semiari</surname></persName>
		</author>
		<author>
			<persName><forename type="first">M</forename><surname>Chen</surname></persName>
		</author>
		<author>
			<persName><forename type="first">W</forename><surname>Saad</surname></persName>
		</author>
		<author>
			<persName><forename type="first">M</forename><surname>Bennis</surname></persName>
		</author>
		<idno type="arXiv">arXiv:2102.03401</idno>
		<idno>arXiv:</idno>
		<ptr target="2102.03401" />
		<title level="m">Federated Learning on the Road: Autonomous Controller Design for Connected and Autonomous Vehicles</title>
				<imprint>
			<date type="published" when="2021">2021</date>
		</imprint>
	</monogr>
	<note>cs, eess</note>
</biblStruct>

<biblStruct xml:id="b44">
	<analytic>
		<title level="a" type="main">Intelligent transportation system for sustainable environment in smart cities</title>
		<author>
			<persName><forename type="first">L</forename><surname>Zhao</surname></persName>
		</author>
		<author>
			<persName><forename type="first">Y</forename><surname>Jia</surname></persName>
		</author>
		<idno type="DOI">10.1177/0020720920983503</idno>
		<ptr target="pub-lisher:SAGEPublicationsLtdSTM" />
	</analytic>
	<monogr>
		<title level="j">The International Journal of Electrical Engineering &amp; Education</title>
		<imprint>
			<biblScope unit="page">0020720920983503</biblScope>
			<date type="published" when="2021">2021</date>
		</imprint>
	</monogr>
</biblStruct>

<biblStruct xml:id="b45">
	<analytic>
		<title level="a" type="main">Cooperative Autonomous Driving Oriented MEC-Aided 5G-V2X: Prototype System Design, Field Tests and AI-Based Optimization Tools</title>
		<author>
			<persName><forename type="first">H</forename><surname>Ma</surname></persName>
		</author>
		<author>
			<persName><forename type="first">S</forename><surname>Li</surname></persName>
		</author>
		<author>
			<persName><forename type="first">E</forename><surname>Zhang</surname></persName>
		</author>
		<author>
			<persName><forename type="first">Z</forename><surname>Lv</surname></persName>
		</author>
		<author>
			<persName><forename type="first">J</forename><surname>Hu</surname></persName>
		</author>
		<author>
			<persName><forename type="first">X</forename><surname>Wei</surname></persName>
		</author>
		<idno type="DOI">10.1109/ACCESS.2020.2981463</idno>
	</analytic>
	<monogr>
		<title level="j">IEEE Access</title>
		<imprint>
			<biblScope unit="volume">8</biblScope>
			<biblScope unit="page" from="54288" to="54302" />
			<date type="published" when="2020">2020</date>
		</imprint>
	</monogr>
	<note>conference Name: IEEE Access</note>
</biblStruct>

				</listBibl>
			</div>
		</back>
	</text>
</TEI>
