Rapid urbanization and the resulting traffic congestion have made the reduction of transport‑related carbon‑dioxide (CO2) emissions a pivotal goal for sustainable‑city initiatives. This paper introduces an artificial‑intelligence (AI) traffic‑signal control system that exploits deep reinforcement learning (DRL) to achieve dynamic, emission‑aware phase scheduling at signalized intersections. Unlike conventional fixed‑time or actuated controllers, the proposed framework continuously perceives multi‑modal state information, queue length, average speed, arrival rates and projected emergency‑vehicle trajectories, through a digital twin fed by microscopic traffic simulation. A convolutional DRL agent, trained with proximal‑policy optimization and shaped by a composite reward that penalizes total CO₂, queue growth and delay while rewarding throughput, learns an adaptive policy that balances environmental and mobility objectives. In controlled experiments mirroring a mid‑size European arterial, the AI agent lowers cumulative CO2 emissions by 18% without sacrificing pedestrian service levels. The learned policy exhibits robust generalization under varying traffic demand patterns and mild sensor noise, suggesting strong transferability.
Optimizing traffic light systems to curb carbon dioxide (CO2) emissions is a pressing challenge,
essential not only for current urban sustainability but also for the long-term preservation of
ecosystems and the global biosphere [
One of the most promising techniques for intelligent traffic signal management is reinforcement
learning (RL), particularly its deep learning-enhanced variant, deep reinforcement learning (DRL)
[
The primary contribution of this study is the development of a DRL-based simulation framework for determining traffic light control policies at a single intersection, with a targeted goal of minimizing CO2 emissions.
The remainder of the article is organized as follows: the “Related Works” section surveys recent efforts in traffic signal control aimed at emission reduction. The “Materials and Methods” section introduces the proposed model tailored for local road intersections. Finally, “Results and Discussion” presents the simulation outcomes and their interpretation.
This section synthesizes prior investigations into traffic light signal control focused on reducing
CO2 emissions. A common thread among these studies is the widespread adoption of the SUMO
simulation model [
For example, reference [
In [
Using SUMO, study [16] illustrated that adaptive traffic lights markedly lower fuel use and emissions compared to fixed systems, achieving reductions in travel time by 82.84% and CO2 emissions by 51.2%. In [17], an intelligent traffic system prioritizing emergency vehicles was introduced, capable of adjusting signal timings based on vehicle classification (e.g., “emergency” versus “passenger”).
Work [18] tackled traffic light arrangements to ease congestion, proposing models to optimize
signal timing graphs for specific goals [19], with SUMO assessing their efficacy. Study [
Focusing on isolated intersections, [20] optimized traffic light signals using a double deep Qnetwork (DDQN), an RL variant. Notably, [15] stressed the importance of tailoring reward functions in RL to alig n agent actions with CO2 reduction objectives, testing various reward structures with SUMO and its emission model to gauge performance sensitivity. Reference [21] offered a novel traffic light control alternative by incorporating predictive noise modeling with RL and a SeqtoSeq-LSTM framework [22], significantly improving noise levels, CO2 emissions, and fuel efficiency over traditional systems.
Further insights come from works [23–25], which showcase successful DQN applications to analogous challenges. Study [23] introduced a “pressure” concept for regional signal coordination, supported by extensive SUMO experiments, including a real-world Manhattan scenario with 2510 traffic lights. Reference [24] provided a toolkit for modern DQN implementations, while [25] leveraged value-based RL for online learning with interpretable policy functions [26]. These findings highlight the demand for intelligent traffic light systems that reduce CO2 emissions while addressing vehicle and pedestrian queues, emergency vehicle access [27], and other factors, necessitating further exploration. The prevalent use of RL and SUMO underscores their promise.
Thus, this work is aimed at advancing sustainable development by curbing CO2 emissions through an intelligent traffic signal control model tailored for complex terrains, balancing the needs of drivers, pedestrians, and emergency services. This will be accomplished by devising a simulation and DQN model for local intersection traffic light control to minimize CO2 emissions.
In this study, we delineate the elements of a holistic approach crafted by the authors to address traffic light control at local intersections, targeting CO2 emission reductions (see Figure 1). A key attribute of this method is the creation of a unified metric and solution that, beyond lowering CO2 emissions, accounts for complex road topographies (e.g., slopes); drivers’ desires for swift destination arrivals; pedestrians’ needs, especially near crowded locales like schools and malls; rapid transit for emergency services; the presence of heavy vehicles; and road emergencies. Moreover, this approach is engineered for adaptability to encompass additional priorities as they arise.
Note that for traffic signal control (TSC), one of the most widely used solution tools is the
reinforcement learning algorithm, namely its modern implementation Deep Reinforcement Learning
(DRL), since DRL can learn to adapt to changing traffic demands [
Figure 1 shows the decomposition of the proposed approach into stages. Note that this article presents research results for stages 3 and 4. Other stages will be considered in future studies.
To test the approach’s feasibility, it is proposed to simulate traffic through an intersection using Algorithm 1.
Simulating the movement of cars through intersections.
1: Input: intersection properties and traffic scenarios 2: Initialize event: Evnt about the appearance of a new car, relative to the start of the simulation with the following properties: lane from which the car is declared; lane into which the car is leaving; speed of the car 3: Set time counter at 0 4: For simulation 5: // Check if there is an event that corresponds to the current simulation time. If there is such an event, add the corresponding car to the simulation on the specified lane with the specified parameters: 6: If .Time = current simulation time Then Add the car to the lane 7: // Update for each vehicle inside the simulation: 8: If the vehicle is near the intersection Then: 9: // Checking the condition of the traffic light: 10: If the green signal allows movement in its direction Then: 11: A car crosses an intersection into a lane 12: If the signal is red then: 13: The car stops 14: If the car is not near the intersection then: 15: If the path is free then: 16: The car continues to move at the current speed 17: If Obstacle Ahead Then: 18: The car stops 19: // Updating the state of the traffic light (at each moment of time we check if it is time to change the phase of the traffic light) 20: If it’s time to change the phase of the traffic light Then: 21: Moving the traffic light to the next phase ℎ, , ) channels, for the lanes,
2 emissions, considering the properties of the observation environment.
In Stage 3, it is proposed to synthesize a reward function for the intersection to reduce 2
Note that the motion simulation iteration is performed every ∆ t, but the reward is calculated not for each iteration, but after N∆t, i.e. the system is controlled for a predetermined number of iterations and the state of the system changes during this period at certain points in time. an integral indicator consisting of the following intermediate components:
(reward at certain steps of the simulation) is proposed to be defined as where is a weighting factor determined empirically, is a fine or reward.
In Stage 4, it is proposed to synthesize a DQN model to achieve the goal of reducing taking into account the properties of the observation environment and the integral loss function. It should be noted that the study hypothesized that optimal solutions for traffic light control should be sought not for a specific intersection, but for a sequence of intersections.
The study proposes to use the independent DQN (IDQN) model for a sequence of intersections and vehicle traffic scenarios. The essence of IDQN is that we will use several networks – each intersection’s own DQN. An approach is proposed where each agent learns independently and simultaneously its own policy, considering other agents as part of the environment.
In IDQN, each agent observes the observation space of its intersection and chooses a separate action and receives a reward. Since the agent “sees” the observation space only from its own perspective, IDQN can be implemented by assigning each agent its own observation history of actions. In deep reinforcement learning, this can be implemented by giving each agent “its own” DQN to perform the observations and produce the actions.
So, we will form IDQN models for intersection sequences and traffic scenarios: = ∑ , (1) 2 emissions, = , 3, = { 1, 2, … , }, , 1, , 2, ,
(2) space (at each observation time) in the form of a matrix ( where – is the number of intersections in the sequence, × – is the number of lanes that the traffic lights regulate, – is the agent’s observation × ), where
– is the number of – is the number of properties – is a convolutional layer that detects spatial dependencies between lanes, takes into account the relationships between the properties of the lanes and prepares data for fully connected layers that make decisions about changing the phases of the traffic light; convolution with a kernel (2×2), stride (stride) 1 and no padding (padding) is used, (ReLU(x) = max(0, x) i.e. this function replaces all values less than zero with zero and leaves values greater than zero unchanged), – is a layer that converts the output of the 2 layer into a one-dimensional vector of size 64 ∗ ( − 1) ∗ ( − 1), which is then fed to the input of the fully connected layers, vector of dimension 64 ∗ ( 1, − 1) ∗ ( 2,
3are fully connected layers: the first layer converts a − 1) into a vector of 64 elements, the second layer supports dimension 64, the third linear layer outputs a vector whose size is equal to the number of actions, – is a transfer function that the final Next, we present an Algorithm 2 for training IDQN models (2).
– is a layer that processes the output of the last linear layer, ensuring values for discrete actions (traffic light phase switching) are obtained.
Training Reinforcement Deep Learning Models .
1: Input: properties of the intersection and traffic light and properties of the traffic scenario at each moment in time of the simulation according to Algorithm 1 2: For i=1… 3: Forming the input matrix for 4: The input matrix InputLayer passes through the convolutional layer Conv2d to extract features and learn patterns between traffic lanes, and then through the Flatten layer and fully connected layers (Linear1, Linear2, Linear3), which form a vector of Q-values. 5: Each element of the vector Q corresponds to the expected reward for a particular action, so its size is equal to the number of possible actions. 6: Based on the obtained Q-values, an action is selected – that is, the action where Q is maximum. The observation space determines the number of possible actions. It is used to form the last linear layer of the network. Each element of this layer corresponds to a Q-value for a specific action. 7: End For 8: Apply the Reward function – the reward is received from the environment after performing the action and is used to correct Q-values during training. 9: Imprint: Trained Models
Thus, M-networks learn to predict the future reward for each action, allowing for optimal
decisions. Note that during training M uses: reward (from the environment), next state (calculated
after performing the action) obtained from motion simulation, discount factor γ∈[
To validate the proposed approach, a simple intersection with one lane in each direction was used (Figure 2).
Through this intersection, the flow of vehicles with different 2 emissions were simulated using Algorithm 1. 12 routes were defined (from each direction to each possible target direction), and traffic flows were generated in four-time intervals (0–25000, 25000–50000, 50000–75000, 75000– 100000 seconds). For each flow, the following were specified: route (in direction of travel), simulation start and end times, intensity, initial speed, lane position, and lane selection.
Thus, vehicles were generated at different times and with different intensities, following specific routes through the intersection.
Then, the IDQN model trained by Algorithm 2 on the generated traffic was compared with known similar algorithms [23–25], according to the criteria: (a) 2 emissions, (2) the total waiting time of all cars, and (3) the total time required for all cars to pass through the intersection.
For the experimental study presented below, the following component reward functions were used: (1): intersection, 2 – total emissions
ℎ – total length of the queue at the – total waiting time at the intersection, – average speed at − 3 3 = −0.1 ∙ the intersection. Accordingly: − 1 1 = −1 ∙ /10 2, − 4 4 = 0.1 ∙
2
/10 3, − 2 2 = −0.1 ∙ /10 2.
Next, we present the obtained results. Figure 3 shows a comparison of emissions for four approaches (approaches [23–25] and ours) after 100 training cycles. The vertical axis shows the level in mg, and the horizontal axis shows the training cycles of the neural network. As can be seen 2 ℎ from the graph, for our proposed approach,
2 approaches (ours and approaches [23–25] after 100 training cycles.
Next, we present the results obtained. Figure 3 shows a comparison of emissions are the lowest.
2 2 emissions for four
2 level in mg, and the horizontal axis shows the neural network training cycles. From Figure 3, for our proposed approach, 2 emissions are the lowest. cycles. The vertical axis shows the total waiting time of the full simulation in seconds, and the horizontal axis shows the neural network training cycles. As can be seen from the graph, the proposed approach performs at the same level as other approaches.
(a) (b) for all cars to travel through the intersection emissions. axis plots time in seconds, and the horizontal axis plots neural network training cycles. As can be seen from the graph, the proposed approach performs slightly better than the others.
The results obtained confirm the ability of the proposed intelligent model to obtain optimal traffic light signal control for a local intersection using simulation modeling and DQN to reduce
The results in Table 2 show the ability of our approach to reduce 2 emissions by 18%. The limitations might include the individual DQN for each intersection, the lack of synchronization between intersections, and the non-transparency of decision-making in the DQN model. This study presents an intelligent traffic light control approach based on DRL, specifically employing a DQN, to reduce CO2 emissions while considering the interests of all road users. The primary contribution lies in the development and simulation-based validation of a DQN model and a tailored loss function that enables adaptive traffic signal control at a local intersection. Through extensive simulation experiments using realistic traffic patterns, the proposed model achieved a measurable reduction in CO2 emissions by approximately 18%, highlighting the potential of DRL in addre ssing environmental and operational challenges in urban traffic management. The limitations include the need to train a separate DQN model for each intersection, the absence of coordination between adjacent intersections affecting overall traffic flow, and the lack of interpretability in the model’s decision-making process, which hinders transparency, trust, and practical deployment.
Future work will focus on addressing these challenges by incorporating real-time input data from surveillance cameras and other sensors to enable online learning and context-aware signal control. 2, mg
This research was supported by the U_CAN –“Towards carbon neutrality of Ukrainian cities” project, under Grant Agreement No. 101148374, funded by the Horizon Europe program.
The authors have not employed any Generative AI tools. [14] Z. Wang, L. Xu, J. Ma, Carbon dioxide emission reduction-oriented optimal control of traffic signals in mixed traffic flow based on deep reinforcement learning, Sustainability 15.24 (2023) 16564. doi:10.3390/su152416564. [15] M. Schumacher, C. M. Adriano, H. Giese, Challenges in reward design for reinforcement learning-based traffic signal control: An investigation using a CO2 emission objective, SUMO Conf. Proc. 4 (2023) 131–151. doi:10.52825/scp.v4i.222. [16] P. C. Sherimon, V. Sherimon, J. Joy, A. M. Kuruvilla, G. Arundas, Efficient deep learning methods for detecting road accidents by analyzing traffic accident images, Int. J. Comput. 23.3 (2024) 440–449. doi:10.47839/ijc.23.3.3664. [17] A. Sachenko, V. Kochan, V. Turchenko, Intelligent distributed sensor network, in: IMTC/98 Conference Proceedings. IEEE Instrumentation and Measurement Technology Conference. Where Instrumentation is Going (Cat. No.98CH36222), IEEE, New York, NY, USA, 1998, pp. 60– 66. doi:10.1109/IMTC.1998.679663. [18] H. Abohashima, A. Eltawil, M. Gheith, A mathematical programming model and a firefly-based heuristic for real-time traffic signal scheduling with physical constraints, IEEE Access 9 (2021) 128314–128327. doi:10.1109/access.2021.3112600. [19] E. Manziuk, O. Barmak, I. Krak, O. Mazurets, T. Skrypnyk, Formal model of trustworthy artificial intelligence based on standardization, in: Proc. of the 2nd International Workshop on Intelligent Information Technologies & Systems of Information Security with CEUR-WS, CEUR-WS.org, Aachen, 2021, pp. 190–197. URL: https://ceur-ws.org/Vol-2853/short18.pdf. [20] T. A. Haddad, D. Hedjazi, S. Aouag, A new deep reinforcement learning-based adaptive traffic light control approach for isolated intersection, in: 2022 5th International Symposium on Informatics and its Applications (ISIA), IEEE, New York, NY, USA, 2022, pp. 1–6. doi:10.1109/isia55826.2022.9993598. [21] C. Ounoughi, G. Touibi, S. B. Yahia, EcoLight: Сco-friendly traffic signal control driven by urban noise prediction, in: Database and Expert Systems Applications. DEXA 2022, Vol. 13426: Lecture Notes in Computer Science, Springer International Publishing, Cham, 2022, pp. 205–219. doi:10.1007/978-3-031-12423-5_16. [22] I. Sutskever, O. Vinyals, Q. V. Le, Sequence to sequence learning with neural networks, in: Proceedings of the 28th international conference on neural information processing systems volume 2, MIT Press, Cambridge, MA, USA, 2014, pp. 3104–3112. doi:10.5555/2969033.2969173. [23] C. Chen, H. Wei, N. Xu, G. Zheng, M. Yang, Y. Xiong, K. Xu, Z. Li, Toward a thousand lights: Decentralized deep reinforcement learning for large-scale traffic signal control, Proc. AAAI Conf.
Artif. Intell. 34.04 (2020) 3414–3421. doi:10.1609/aaai.v34i04.5744. [24] J. Ault, J. Hanna, G. Sharon, Learning an interpretable traffic signal control policy, in: Proc. of the 19th International Conference on Autonomous Agents and MultiAgent Systems (AAMAS ‘20), International Foundation for Autonomous Agents and Multiagent Systems, Richland, SC, USA, 2020, pp. 88–96. URL: https://ifaamas.org/Proc./aamas2020/pdfs/p88.pdf. [25] S. Vladov, L. Scislo, V. Sokurenko, O. Muzychuk, V. Vysotska, S. Osadchy, A. Sachenko, Neural network signal integration from thermogas-dynamic parameter sensors for helicopters turboshaft engines at flight operation conditions, Sensors 24.13 (2024) 4246. doi:10.3390/s24134246. [26] P. Radiuk, O. Barmak, E. Manziuk, I. Krak, Explainable deep learning: A visual analytics approach with transition matrices, Mathematics 12.7 (2024) 1024. doi:10.3390/math12071024. [27] J. Ault, G. Sharon, Reinforcement learning benchmarks for traffic signal control, in: Proc. of the Thirty-fifth Conference on Neural Information Processing Systems (NeurIPS 2021) Datasets and Benchmarks Track, 2021, pp. 1–11. URL: https://people.engr.tamu.edu/guni/papers/NeurIPSsignals.pdf. [28] V. Mnih, K. Kavukcuoglu, D. Silver, E. Al, Human-level control through deep reinforcement learning, Nature 518.7540 (2015) 529–533. doi:10.1038/nature14236. [29] P. Alvarez Lopez, A. Banse, M. Barthauer, M. Behrisch, B. Couéraud, J. Erdmann, Y.-P. Flötteröd, R. Hilbrich, R. Nippold, P. Wagner, Simulation of urban mobility (SUMO), Software, v. 1.21.0, Zenodo, 2024. doi:10.5281/zenodo.13907886. [30] D. Krajzewicz, M. Behrisch, P. Wagner, R. Luz, M. Krumnow, Second generation of pollutant emission models for SUMO, in: Modeling Mobility with Open Data, Lecture Notes in Mobility, Springer International Publishing, Cham, 2015, pp. 203–221. doi:10.1007/978-3-319-15024-6_12.