The primary objective of the AIDECC (Argumentation for Informed Decisions with Applications to Energy Consumption in Computing) project is to enhance energy efficiency in neural network training by integrating argumentation-based causal discovery and machine learning. The project's planned activities include a theoretical investigation into abstract argumentation and its role in causal discovery, a detailed analysis of relevant datasets, and the development of structural causal models for identifying energy reduction interventions.
In recent years, Deep Neural Networks (DNNs) [1] have been extensively adopted across various data-driven fields, including computer vision [2], natural language processing [3], personalised recommendation [4], and speech recognition [5]. The growth in these areas has led to an increased reliance on powerful, parallel GPU clusters for training DNN models. However, this surge in computational demand has significant energy implications [6]. The (recently started) AIDECC project aims to achieve a more sustainable approach to AI development, focusing on reducing energy consumption. To this end, the project focuses on investigating causal relationships related to energy consumption in DNN training. Through causal studies, the project seeks to design targeted interventions to reduce energy use, thereby directly contributing to decreased waste and pollutant emissions associated with electricity production.
Three main objectives can be identified for the project. In accordance with the EU AI Act, the project aims to devise a human-centric and ethically aligned approach, where the system's decisions are always subject to human review and interpretation.
A second key objective is acquiring a situational understanding of energy consumption in neural network training, providing insights into the underlying mechanisms and their interactions. This means identifying which variables affect energy use and understanding the causal relationships and rationale behind these effects.
Finally, the project aims to identify strategies to reduce energy consumption and waste in training neural networks. This involves a deep understanding of their computational and energy dynamics along with an awareness of the practical constraints and operational environments in which these networks are deployed. This analysis must consider real-world variables such as hardware limitations, scalability, and the diverse nature of neural network applications across different industries.
To achieve these objectives, argumentation theory [7] is adopted as a fundamental component for causal discovery. Within this paradigm, a causal link is considered a provisional argument subject to a potential series of dialectical interactions [8]. For instance, the arguments from circumstantial evidence consider the spatial and temporal proximity of events, repeated co-occurrences, and similarities between cause and effect. The arguments based on contrastive evidence rely on observations of covariation and change, looking at how alterations in one event may result in changes in another. Also, arguments from causal explanations delve into the underlying mechanisms and processes, considering the absence of alternative explanations and the typical effects known to follow certain causes.
Argumentation fosters a balanced and comprehensive view, considering both the energy implications and the potential compromises in performance or accuracy, facilitating informed and sustainable decision-making through a nuanced discussion (the first objective). It is also instrumental in enhancing causal discovery, especially in scenarios with scarce data, unveiling the causal relationships between the architectural choices and hyperparameters on learning process efficiency (the second objective). Furthermore, argumentative causal models support potential interventions to reduce energy consumption in DNN training while engaging in informed discussions and debates about various interventions' merits and potential trade-offs (the third objective).
The project comprises three working packages.
We plan to expand upon the state-of-the-art on causal discovery and argumentation-based machine learning to support informed human decision-making. In particular, WP1 will focus on the theoretical foundations of argumentation and its application in identifying and evaluating causal relationships from data. It will also explore integrating argumentative methods with machine learning algorithms, illustrating how this combination can lead to more robust and transparent models.
We plan to analyse relevant datasets1 using the advanced argumentative techniques developed as part of WP1.
The insights gleaned from Work Packages 1 (WP1) and 2 (WP2) will be instrumental in developing a structural causal model. This model will map out the causal relationships between different variables for which we can build arguments supporting the claim that they influence energy consumption.
The energy-intensive nature of training complex neural networks has significant environmental implications, primarily due to the substantial carbon footprint associated with high energy consumption. Through causal studies, the project aims to formulate policies and design targeted interventions to reduce energy use, thereby directly contributing to a decrease in pollutant emissions associated with electricity production.
This work has been supported by the EU NEXTGENERATIONEU program within the PNRR Future Artificial Intelligence -FAIR project (PE0000013, CUP H23C22000860006), Objective 10: Abstract Argumentation for Knowledge Representation and Reasoning, specifically by the project Argumentation for Informed Decisions with Applications to Energy Consumption in Computing-AIDECC (CUP D53C24000530001).