Fairness and explainability are important issues that must be addressed in multi-agent reinforcement learning (MARL) systems. In this study, we propose a novel approach that directly incorporates fairness constraints and layer-wise relevance propagation (LRP) into multi-agent training. Through the proposed method, explainability and fairness can be addressed simultaneously, improving the interpretability of agent's decisions and guaranteeing that agents are assigned tasks equitably. We evaluate the performance of the proposed method based on a resource allocation problem. The results show average fairness and explainability ratings of 0.921 and 0.931, respectively. Preliminary results show that this strategy greatly enhances system fairness and explainability while maintaining a competitive average system reward. Furthermore, by encouraging eficient resource use, the proposed method advances the principles of green artificial intelligence.
The rapid advancement of artificial intelligence (AI) and, in particular, of machine learning (ML) has
transformed numerous sectors. Given the tremendous impact these technologies have on
decisionmaking processes, the concepts of fairness and explainability in AI have become a necessity. AI systems
have been shown to have biases [
On the other hand, the explainability of AI systems is another important research issue [
Multi-agent reinforcement learning (MARL) is a powerful way to deal with complex and dynamic
environments where multiple agents interact and learn simultaneously. MARL can be used to generate
through continuous learning and adaptation. The underlying interactions and dependencies among
the agents, make it dificult to achieve both fairness and explainability simultaneously. The literature
extensively addresses both fairness [
Green AI highlights the need for AI systems that are not only powerful but also sustainable in terms of the environment. As an example, lets consider the following multi-agent reinforcement learning framework: Example 1. Think of a MARL system where agents cooperate to accomplish activities in a shared environment. The capability level of each agent ∈ {1, . . . , } indicates its task-performance capacity (e.g., hardware availability). The dificulty level determines the characteristics of each task ∈ {1, . . . , }. The task complexity and agent capacity match to determine the agent’s reward for completing a task. Each agent aims to choose tasks that maximize its predicted cumulative reward, ( ).
Agents are encouraged to choose tasks that are in line with their capabilities, which promotes efective resource utilization. This is achieved by using the match between the agent’s capability and the task’s dificulty as a metric for a given reward. The conventional objective is to maximize the total cumulative rewards ∑︀
=1 ( ). Unlike the traditional methods, we propose to add fairness constraints to ensure equitable (individual) reward distribution among agents and explainability constraints that guarantee transparent decision-making processes.
This fair allocation of rewards among agents, independent of their innate capabilities, contributes to preventing disadvantages for agents with lower capabilities. This is significant because many stakeholders, geographical areas, or systems with difering resources and technological capabilities may be represented by these agents. Let’s consider “World Community Grid” in our Example 1 as a sizable distributed computer network for scientific research. The agents stand in for individual computers or small clusters that have been donated by various global players. These systems’ computing power and energy eficiency are reflected in their capability levels. If there was no fairness restriction, the system might constantly assign the hardest jobs to the agents with the highest capabilities, which would result in high energy usage and participation barriers. It should be emphasized that the major idea behind this study is to integrate fairness and explainability into MARL. Depending on the application, the fairness constraint may be defined diferently.
On the other hand, understanding why certain decisions are made can lead to insights for improving the system, potentially identifying more eficient allocation strategies. By combining fairness and explainability, one can help to design more sustainable, eficient, and trustworthy AI systems that adhere closely to the ideals of Green AI.
In this study, we address individual fairness in the context of algorithmic fairness, meaning similar
agents should receive similar treatment [
The structure of this paper is as follows: In Section 2, we briefly address the most recent related works and the preliminary. Section 3 outlines the problem statement, objective function, and proposed method. Section 4 presents the experimental setup and results, demonstrating the eficacy of the proposed algorithm. Finally, Section 5 concludes the paper, discussing the limits and future research direction.
Cimpean et al. [
Explainability in AI is critical for understanding decision-making processes. Traditional approaches
often focus on post-hoc explanations, where the decisions of pre-trained models are interpreted.
However, such methods come short in dynamic interactions where interpretable and transparent decisions
must be made in a continuous fashion. Current research eforts, like LRP [
In short, the existing methods mainly focus on fairness and explainability as independent issues.
Therefore, we propose a single framework that could handle both in multi-agent systems simultaneously.
Continuing, we provide some preliminary definitions:
• Agent: A multi-agent system entity that learns and makes decisions according to its environment
and follows policy. An agent can be represented by a tuple ⟨, , , ⟩ where is the set
of states, is the set of actions the agent can take, : → is the policy that maps states to
actions, and : × → R is the reward function that provides feedback to the agent based
on its actions in a given state.
• Environment: The environment or context within which agents act and interact. It is typically
modeled by a Markov decision process (MDP) defined by a tuple ⟨, , , ⟩ where is the set
of all possible states of the environment, is the set of all possible actions that agents can take,
: × × → [
We consider a set of agents interacting within a common environment. Each agent , where ∈ [1.. ], acts according to a policy that specifies its actions as a function of observed states of the environment. The goal is to maximize the collective reward function while maximizing the agents’ overall performance under fairness and explainability constraints. The problem is defined as follows:
max ∑︁ ( ),
1, 2,..., =1
subject to: (1, 2, . . . , ) ≥ , ( ) ≥ ,
∀,
(1)
where ( ) is the expected cumulative reward (objective function) of agent , measures the fairness
across the agents, is the threshold of fairness, measures the explainability of the decisions, and is
the minimal level of explainability. The fairness constraint requires the equal treatment of all agents
in terms of the rewards they receive for their actions. The explainability for the actions performed by
agents is obtained by using the Layer-wise Relevance Propagation (LRP) method [
← + () ·
where () is the learning rate and is the immediate reward. It should be noted that calculating ( )
requires considering the entire sequence of states, actions, and rewards, which can be computationally
expensive and impractical for frequent updates. Therefore, in Algorithm 1, we use as the immediate
reward for each agent. To evaluate fairness, we consider Jain’s fairness metric [
1In our experiment, we use a constant value for the learning rate. 2We use a softmax strategy, however other strategies such as -greedy could also be considered. (2) (3) (4) (5) (6) (1, 2, . . . , ) = (∑︀=1 ( ))2 ∑︀ =1 ( )2 A score close to 1 indicates high fairness, meaning all agents are receiving rewards that are close to the average. In Algorithm 1 (lines 14–16), we check to make sure that the fairness is above the given threshold. In cases where the fairness is lower than the threshold, we perform the “Adjust” mechanism to ensure equitable reward distribution among agents. As a sample for “Adjust” mechanism, lets consider the code that we implemented in our experiment as given in Appendix.
To evaluate the explainability, we use the layer-wise relevance propagation (LRP) scores , . The relevance metric is defined in Equation (5):
|(, ; )| , = ∑︀ |(, ; )| where (, ; ) is the Q-value for agent taking action in its current state , and the summation in the denominator is over all possible actions for agent in state . We assign relevance scores based on the absolute of Q-values for each action. Actions with higher absolute Q-values are considered more relevant to the agent’s decision-making process. We consider the Entropy of this relevance (Equation (6)), where () is the entropy of the relevance scores for agent . Entropy is a quantitative indicator of relevance scores that makes comparing explainability amongst agents simple.
The explainablity metric ( ) is defined in Equation (7), where is the number of features or inputs considered by the model. A higher value (close to 1) indicates that behaviors are more explainable. () ( ) = 1 − log2() (7) In Algorithm 1, we compare the explainablity with the predefined threshold (lines 19-21). In case that the ( ) is less than the given threshold, we perform “Explainability boost”, and prioritize the activities of the most relevant agents. A simple code for this part as implemented in our experiment is given in the appendix.
Algorithm 1 Proposed Algorithm for Multi-Agent Systems
In lines 22-24 of Algorithm 1, we perform “partially reset”. We perform this operation in a defined number of iterations to prevents the agents from converging to suboptimal policies by encouraging them to explore a wider range of actions. A code is given in the appendix. We also perform “pruning” (lines 25-27 of Algorithm 1) as a mechanism to enhance the eficiency of the learning process. By utilizing this mechanism, we remove the less relevant connections in the policy parameters . A sample code performed in our experiment is given in the appendix.
We evaluate the performance of the proposed algorithm and compare it with a typical RL approach
for our presented Example 1. We consider = 10 agents and 100 tasks, assuming each task has a
dificulty level (we randomly assign a value in the range [
As can be seen in Table 2, the proposed method achieves significantly higher fairness compared to the base method. This means that the rewards are distributed more equitably among agents. The proposed method also excels in explainability. This suggests that the decisions made by agents in the proposed method are more transparent compared to those in typical reinforcement learning. These achievements comes with a price and the total average reward of the base method is higher. However, this slightly decrees in the total reward, enables the system to satisfy fairness, and provides better explainablity.
This study proposed a new approach for integrating fairness and explainability in multi-agent reinforcement learning systems. Our approach increased agent transparency by using LRP and systematic fairness requirements. The experimental results showed that, while keeping a competitive average system reward, our proposed method greatly increases both explainability and system fairness, with average scores of 0.921 and 0.931, respectively. The proposed method is a straightforward illustration of how explainability and fairness might be combined in multi-agent systems. However, the proposed algorithm needs to be improved with more dynamic tasks. The eficacy and scalability of proposed method needs to be verified by experimenting on large and varied datasets. A thorough investigation is also needed to evaluate the efects of hyperparameters on the algorithm’s performance. As the future work, we expect that adding more dynamic and complex metrics for fairness and explainability could improve the system’s equity and transparency. We would investigate the proposed concept in the practical fields, including healthcare, banking, and self-governing systems.
We acknowledge the support of the PNRR project FAIR - Future AI Research (PE00000013), Spoke 9 - Green-aware AI, under the NRRP MUR program funded by the NextGenerationEU. One of authors was also funded by the Next Generation EU -Italian NRRP, Mission 4, Component 2, Investment 1.5, call for the creation and strengthening of ‘Innovation Ecosystems’, building ‘Territorial R&D Leaders’ (Directorial Decree n. 2021/3277) - project Tech4You - Technologies for climate change adaptation and quality of life improvement, n. ECS0000009. This work reflects only the authors’ views and opinions, neither the Ministry for University and Research nor the European Commission can be considered responsible for them.
During the preparation of this work, the authors used Quillbot and Grammarly in order to: Grammar and spelling check, and re-phrasing sentences. After using these tool, the authors reviewed and edited the content as needed and take full responsibility for the publication’s content.
Appendix # Adjust fairness if fairness_score < fairness_threshold: max_agent = np.argmax(agent_rewards) min_agent = np.argmin(agent_rewards) adjustment = (agent_rewards[max_agent] - agent_rewards[min_agent]) * alpha policy_params[:, max_agent] -= adjustment policy_params[:, min_agent] += adjustment # Explainability boost for task_id in range(num_tasks): top_agent = np.argmax(relevance_scores[task_id]) policy_params[task_id] *= 0.5 policy_params[task_id][top_agent] += 0.5 # Periodic Reset if iteration % 100 == 0:
policy_params = 0.7 * policy_params + 0.3 * np.random.rand(num_tasks, num_agents) # pruning if iteration % 50 == 0: threshold = np.percentile(policy_params, 50) policy_params[policy_params < threshold] = 0