eXplainable AI (XAI) has emerged as a research direction in response to the lack of explainability and interpretability of AI models [6]. XAI aims to enhance the transparency of AI models, ML in particular, by providing human decision-makers with insights into the inner workings of ML models [21]. XAI makes ML outputs more interpretable and comprehensible by demystifying the complex processes within ML models. Approaches such as feature importance, example-based and counterfactual explanations have been developed for that purpose [1]. XAI seeks to bridge the gap between the technical complexity of these models and the need for human-understandable outputs by unravelling the intricacies of ML model decisions.
Despite the potential benefits of XAI, many existing XAI models lack practical usability and fail to improve human-AI performance [1]. Decision-makers often perceive explanations as tools designed for data scientists and ML engineers [7], leading to disinterest and reluctance to engage with XAI interfaces [8]. Decision-makers also report not seeing explanations as motivating to learn and solve problems; they show a lack of interest and curiosity unless these explanations align with their initial expectations [1]. This is consistent with findings from cognitive psychology showing that users tend to focus on features that have apparent value for their decision-making process [8]. Research in Human-Computer Interaction (HCI) has identified several user issues in human-XAI interaction, such as misinterpreting explanations, highlighting the need for improved design solutions [9]. Explanations might also be ignored if they are overly abstract, as people tend to prioritise concrete information instead [8].
Researchers have suggested various approaches to operationalise XAI in human-AI settings. For instance, contextualising XAI design by incorporating domain-related information and empirical knowledge has shown promising results in enhancing user satisfaction and understanding [10]. Additionally, employing contrastive explanations and juxtaposing features can help develop an expert ability to notice salient features and anomalous events [11]. To enhance critical engagement with explanations and mitigate over-reliance on AI recommendations, incorporating cognitive forcing [12] and nudging [8] design elements into the XAI interface has been empirically tested. These elements have been shown to discourage decision-makers from blindly accepting AI suggestions and instead prompt them to evaluate recommendations thoughtfully. Furthermore, methodological approaches have been suggested to help UX designers operationalise XAI methods on the XAI user interface level [12]. These approaches have been shown to improve human-XAI interaction and increase participant engagement with AI explanations.
Interestingly, much of the existing research in Human-Centered eXplainable AI focuses on oneto-one interactions between the XAI and human decision-makers. In other words, it focuses on how individual decision-makers interact with AI explanations and use them to make decisions in human-AI settings. However, much of real-world scenarios often involve many-to-one relationships in a group-AI interaction, where a group of individuals collaborate to make collective decisions using the XAI interface. Figure 1 explains this relationship. Group-AI interaction refers to the collaboration and interaction between groups of human decision-makers and AI systems in decision-making processes [17]. In this context, groups may include executive committees, boards, teams of professionals, or any collective of individuals tasked with making decisions within an organisation or context [15]. Group-AI interaction has the potential to leverage the capabilities of AI technologies alongside collective human expertise to enhance decision-making outcomes. It has also been shown to exceed the accuracy of human-AI collaboration by bringing different expertise and perspectives of humans involved in the decision-making process [7]. Individuals in the group can be assigned specialised roles and responsibilities related to interacting with AI systems, interpreting AI-generated insights, and integrating them into decision-making processes [1]. This division of responsibilities ensures that each member contributes their expertise effectively, mitigating the risks associated with human-AI collaboration [1].
In this paper, we argue that designing XAI for group-AI interaction requires distinct approaches and careful considerations compared to the ones used in human-AI interaction. This latebreaking contribution synthesises insights from collective decision-making and Human-Centered XAI literature to discuss challenges inherent in transitioning XAI from human-AI to group-AI interaction. These encompass the complexities of group dynamics, the potential amplification of cognitive biases, issues surrounding trust, as well as the critical facets of XAI evaluation in the context of group-AI. While acknowledging the possibility of additional challenges, our discussion provides an initial framework for contrasting the nuanced design requirements of XAI in facilitating AI-assisted decision-making within group settings.
Comparison of One-to-One and Many-to-One Interactions with an XAI Interface. In the One-to-One interaction mode, a single user interacts directly with the XAI interface, receiving explanations tailored to their specific queries or actions. In contrast, the Many-to-One interaction mode involves multiple users interacting with the XAI interface simultaneously, with explanations generated to accommodate diverse user inputs and preferences.
In group decision-making, individuals often wield varying degrees of influence and expertise, regardless of the context-be it a professional team, a board of directors, or a community organisation [19]. This diversity encompasses a nuanced interplay of individual personalities, social structures, power dynamics, and communication patterns within the group [21]. Such complexity in group diversity further impacts the interaction between groups and AI, particularly in XAI contexts. This diversity underscores the necessity for XAI systems to accommodate inclusive explanations tailored to the diverse needs and backgrounds of group members [20]. For example, individuals with varying levels of familiarity with AI and machine learning concepts may necessitate explanations that are lucid, accessible, and devoid of technical jargon. Moreover, the diversity within groups may prompt XAI developers to accommodate diverse learning styles and linguistic preferences, thereby enhancing the accessibility of XAI explanations [5]. This entails providing explanations in multiple formats or languages and integrating interactive features to facilitate engagement and comprehension among all group members [11]. Additionally, the array of group interactions may be further complicated by individual attitudes and perceptions toward AI technology [22]. Cultural values and norms, for instance, have been demonstrated to influence attitudes toward AI [1]. While some individuals may embrace XAI as a valuable tool for enhancing decision-making capabilities, others may harbour scepticism or resistance due to concerns about job displacement, loss of autonomy, or ethical implications. Consequently, XAI for group-AI interaction must address these diverse perspectives and cultivate a culture of trust, transparency, and open communication to mitigate resistance to XAI adoption and foster constructive collaboration within the group.
Biases in group-AI interaction can be more pronounced than human-AI interaction, presenting significant challenges for the design and development of XAI systems [4] [17]. One prevalent bias is groupthink, where group members prioritise consensus and overlook dissenting viewpoints to maintain harmony [23]. In this context, if explanations do not encourage critical thinking and challenge groupthink, individuals within the group may unquestioningly accept AI-generated insights without thorough examination explanations [13]. XAI systems should not only explain recommendations but also encourage scrutiny and diverse viewpoints [26]. This could involve presenting multiple explanations, highlighting uncertainties, and actively soliciting feedback from users with varying perspectives. XAI systems may also implement mechanisms for independent review and validation. For instance, introducing a Devil's Advocacy role within the team can challenge group consensus and encourage critical evaluation of XAI explanations [24]. This individual identifies potential flaws or biases, fostering a more balanced consideration of decision options.
Another bias that could impact XAI design in group-AI interaction scenarios is the equality bias. It refers to the tendency for individuals to downplay their expertise or to weigh everyone's opinion equally, regardless of competence or expertise [27]. This bias can have detrimental effects on decision-making processes, particularly when there is a genuine disparity in knowledge or experience within the group. In the context of XAI, the equality bias could be amplified when group members defer too readily to the AI recommendations, regardless of their domain expertise or experience in the subject matter [28]. For example, suppose a group of healthcare professionals is using an XAI system to diagnose patients. In that case, individuals with specialised medical knowledge may inadvertently downplay their expertise and defer to the AI's recommendations and explanations, even when they have valid insights or concerns that should be taken into account. XAI design shall account for such bias by designing explanations that encourage individuals to recognise and value their expertise and insights, as well as those of others within the group. Additionally, fostering a culture of collaboration and open communication within the group can help ensure that diverse perspectives and expertise are taken into account when making decisions with the assistance of AI systems. This may involve providing XAI explanations that highlight the knowledge and contributions of individual group members, as well as mechanisms for facilitating constructive dialogue and debate within the group.
Trust has been a crucial element in human-AI interaction, influencing the dynamics and effectiveness of human-AI teams [1]. When considering group-AI interaction, trust dynamics become more complex, involving not only trust between group members and AI but also among group members themselves. It has been discussed that incorporating XAI into group decisionmaking processes can impact trust dynamics among group members [25]. The integration of XAI into group decision-making processes introduces new dimensions to these dynamics, with potential implications for team cohesion and effectiveness. Suppose an explanation contradicts the opinions or recommendations of certain group members, it could create tensions or conflicts within the group, undermining trust and cohesion. In addition, some scenarios could involve individuals within the group perceiving XAI explanations as more reliable or objective than human judgments, which may lead to a shift in trust dynamics within the group [9]. To navigate these complexities and foster trust among group members, XAI development needs to consider the social dynamics of group interaction. Open communication about XAI's role in decisionmaking and clear explanations of its outputs are crucial for trust calibration. Additionally, establishing protocols for interpreting XAI explanations in context, along with mechanisms for addressing conflicts arising from AI-influenced decisions, can safeguard against trust erosion. This ensures that XAI's benefits are harnessed without jeopardizing ethical principles or human values.
Evaluating XAI for group interaction presents distinct challenges compared to traditional XAI evaluations focused on individual users. Traditionally, XAI examines how individuals interact with AI systems, understand explanations, and make decisions based on them [1] [6]. Here, evaluation metrics assess explanation clarity, relevance, and user satisfaction [9] [10]. Trust and interpersonal dynamics are also crucial factors, with conflicts arising from discrepancies between user expectations and AI behaviour, requiring strategies for resolution [1] [8]. However, in group-AI interaction, evaluation extends beyond individual users. We need to consider the entire group ecosystem. This includes how AI-generated explanations are communicated within the group, how they impact group cohesion and communication patterns, and how conflicts are resolved among members, considering factors like power dynamics and individual expertise [18]. Additionally, group-XAI evaluation involves understanding the social influence of explanations on the group's decision-making dynamics. This encompasses considerations of scalability (how well does XAI adapt to groups of varying sizes?) and consensus-building (how can XAI support groups in achieving agreement despite diverse perspectives?) [18]. Therefore, evaluating XAI for group interaction demands methodologies that account for the complexities of social interactions and group dynamics. This might involve incorporating social network analysis to understand how information flows within the group and identify potential bottlenecks or communication silos. Longitudinal studies could be conducted to assess the impact of XAI on group performance and decision-making quality over time. Ultimately, understanding these differences is crucial for designing effective XAI systems that empower both individual users and collaborative decision-making processes, while mitigating potential pitfalls and fostering a healthy group environment.
In conclusion, the integration of Machine Learning (ML) into decision-making processes across various sectors has prompted the development of XAI to address concerns regarding ethics, legality, explainability, and safety. In this paper, we showed that much of the existing research focuses on one-to-one interactions between XAI interfaces and individual decision-makers. However, many real-world scenarios require the interaction between a group of humans and the XAI interface. Building on the current research on Human-Centered XAI, this paper has discussed four key considerations when transitioning from human-AI to group-AI interaction in the context of XAI. These challenges include complexities in group dynamics, cognitive bias amplification, trust issues within the group, and group-centric evaluation. By drawing upon current work in Human-Centered XAI research, we contribute to advancing the field and facilitate discussions on group-XAI interaction. This paper calls for further research in this area to enhance the effectiveness and usability of XAI in collaborative decision-making settings, ultimately leading to more informed and successful outcomes in various domains.