rejections. Research on human-computer interaction emphasise the importance of reliability in hybrid decision-making systems. Trust hinges on the performance and trustworthiness of AI, achievable through accuracy metrics, confidence scores, eXplainable AI, and abstention mechanisms. This study presents an explainable abstaining classifier named Learning to Reject via Local Rule-based Explanations (L2loRe), a novel approach that leverages the distance between data points and counterfactuals to evaluate the confidence of predictions, thus facilitating the formulation of a rejection policy and generating clear explanations for the reasoning behind predictions or Research on human-computer interaction emphasizes that a safe and efective utilization of Artificial Intelligence (AI) in decision-making requires human agents to properly rely on AI systems, which in turn is achieved by building the hybrid system for appropriate trust [1]. Key factors influencing trust in automation include performance and transparency, typically conveyed via accuracy metrics, confidence scores, and explanations [ 2]. The inclusion of an abstention mechanism can further improve the reliability of the system [3]. When the AI system lacks suficient confidence in its predictions or the impact of errors could be serious, it may be more prudent to refrain from making a prediction and instead direct the input to a more advanced system or a human agent. Several approaches, referred to as “Learning to Abstain” (L2A, [4]), have adopted this mechanism. Nonetheless, limited attention has been directed towards a significant drawback of L2A, namely, the opaqueness of the rejection policy, which may ultimately undermine human trust and satisfaction with the automated system.
interpretable confidence score described by the distance between data points and their corresponding counterfactual instances. L2lore leverages this distance as a proxy to quantify the certainty level of a prediction and to reject it accordingly.
Overall, the contributions of this work can be summarized as follows: i) We present a novel strategy to fine-tune an existing pre-trained classifier with local rule-based explanations; i) we propose to use the distance between data points and their corresponding counterfactuals as a confidence metric to define a rejection policy for a given classifier;
i) we generate human-understandable explanations to enrich the outcome of the classifier in case of rejection.
CEUR
ceur-ws.org
Problem setting. The goal of L2R algorithms is to learn a model that consists of two components, namely a predictor and a rejection policy . The former is defined as a function ∶ → , where denotes the feature space and the target space, while the latter is generally defined (at inference time) as ∶ → {0, 1}
. In the case of dependent rejectors, depends also on the predictor through a confidence function ∶ → ℝ
+ and a certain threshold . The composed system is then defined by a function ∶ → ⋃{∅} such that: () = { ∅ () if (; otherwise.
, ) = 1, In other words, given an instance , if the rejection policy rejects it, then no prediction is made and the instance is directly deferred to a downstream agent (e.g., a human decision-maker); conversely, if accepts , then the prediction function is applied to and the outcome () is observed. In this study, we assume to be a binary or multiclass classifier. Ideally, should be able to accurately capture the decision boundary of in order to reject the examples on which is prone to make mistakes, while accepting those where a correct prediction is more probable. In accordance with the terminology in [4], our proposed method implements a dependent and staged (i.e., learnt post-hoc with respect to the classifier) abstention policy for the rejection of ambiguous samples. L2loRe is model agnostic for classification models on tabular data.
Architecture. L2loRe implements the following sequential functions: 1. Learning a confidence function that quantifies the uncertainty associated with the prediction of a test example by measuring its proximity to its closest counterfactual. Intuitively, the distance between an instance and its counterfactual might serve as a proxy for the confidence of the original classifier. The first step to learn counterfactual rules = { 1, … } and a set of counterfactual instances () = { 1, … , } obtained by applying the counterfactual rule to the original instance to get . This step largely relies XAI method LORE [9]. Successively, the confidence function ∶ × → ℝ+ is the generation of a set of is estimated as: (, ()) ≔ ( − ∈ min ). 2. Learning the rejection policy. The rejection policy is formulated in such a way that data points whose distance from their closest counterfactual is less than a specified threshold are deemed too uncertain to be predicted by the classifier and are thus rejected. Formally: (; , ) = { 1 if (, ()) < , 0 otherwise.
The rejection threshold is chosen as the solution of the following constrained optimization problem: minimize arg max ( )
∈[0,1] subject to ( ) < max
max > 1 − 1 − ( )
(rejection rate) (misprediction error) where ∶ [0, 1] → ℝ + is a measure defined as a function of the rejection threshold (e.g., one of the three L2R evaluation metrics suggested in [10] ), max is an upper bound for the rejection rate, and max is an upper bound for the proportion of misprediction. In order to determine the optimal value for , we employed a heuristic approach that involved assessing the performance of in correspondence of a number of candidate rejection thresholds (i.e., distance values). These are sampled from a gamma distribution fitted on the vector of confidence scores { (, ())} ∈ . 3. Generating explanations. L2loRe additionally provides a textual explanation for the decision/abstention. This can be achieved by directly exploiting the underlying LORE architecture that provides counterfactual explanations regarding the existence of high confident counterfactual points in the very close proximity of the decision boundary of the classifier.
Experiments. In order to perform experiments with L2loRe, we selected three binary (Compas [11], Adult [12], German Credit [13]) and three multiclass (Wine [14], Abalone [15], Student [16]) tabular datasets, most of which contained a combination of categorical and continuous variables. As a first preliminary analysis, we examined how the performance of L2loRe varies as a function of the rejection rate over the aforementioned datasets. As it can been seen in Figure 2, L2loRe improved the performance of the classification task over all datasets. Secondly, we fine-tuned the rejection threshold . Given the the optimal choice of for all datasets, we then computed and analyzed the classification-rejection scores, as reported in Table 1. Specifically, we set max = 0.6 and max = 0.3 as upper bounds to the rejection and misprediction rate, respectively, and optimized with respect to each of the three L2R metrics defined in [ 10]. Figure 3 displays the explanation provided by L2loRe in the a case of an rejected instanced selected from the Compas dataset.
Based on these preliminary findings, the next steps will involve conducting in-depth experiments to assess L2loRe from both quantitative and qualitative perspectives. In addition, we suggest that a potentially promising avenue for future research could be extending L2loRe within the L2D framework. By incorporating a deferral policy that learns from and accounts for human expected performance, this scenario should also explain why a specific human agent or machine was considered more suitable for making a prediction. (2004) 50–80. 3 (2022) 100455.
pp. 854–859.
Evaluation of L2loRe with optimal choice of the rejection threshold .
This work has been supported by the European Union under Horizon Europe Programme Grant Agreement no. 101120763 (TANGO). Views and opinions expressed are however those of the author(s) only and do not necessarily reflect those of the European Union or the European Health and Digital Executive Agency (HaDEA). Neither the European Union nor the granting authority can be held responsible for them. This work has been realised also thanks to ERC-2018-ADG GA 834756 (XAI), by HumanE-AI-Net GA 952026, by the Partnership Extended PE00000013 - “FAIR - Future Artificial Intelligence Research” Spoke 1 “Human-centered AI”, by “SoBigData++: European Integrated Infrastructure for Social Mining and Big Data Analytics” ( http://www.sobigdata.eu ), GA 871042 and by NextGenerationEU - National Recovery and Resilience Plan, PNRR) - Project: “SoBigData.it - Strengthening the Italian RI for Social
[1] J. D. Lee, K. A. See, Trust in automation: Designing for appropriate reliance, Hum. Factors 46 [2] J. Zerilli, U. Bhatt, A. Weller, How transparency modulates trust in artificial intelligence, Patterns [3] K. Hendrickx, L. Perini, D. V. der Plas, W. Meert, J. Davis, Machine learning with a reject option: a survey, Mach. Learn. 113 (2024) 3073–3110.
for hybrid decision making systems, CoRR abs/2402.06287 (2024). [4] C. Punzi, R. Pellungrini, M. Setzu, F. Giannotti, D. Pedreschi, AI, meet human: Learning paradigms [5] A. Artelt, B. Hammer, ”even if ...” - diverse semifactual explanations of reject, in: SSCI, IEEE, 2022, [6] A. Artelt, J. Brinkrolf, R. Visser, B. Hammer, Explaining reject options of learning vector quantization classifiers, in: IJCCI, SCITEPRESS, 2022, pp. 249–261. [7] A. Artelt, R. Visser, B. Hammer, Model agnostic local explanations of reject, in: ESANN, 2022. [8] S. Singla, N. Murali, F. Arabshahi, S. Triantafyllou, K. Batmanghelich, Augmentation by counterfactual explanation -fixing an overconfident classifier, in: WACV, IEEE, 2023, pp. 4709–4719. [9] R. Guidotti, A. Monreale, S. Ruggieri, F. Naretto, F. Turini, D. Pedreschi, F. Giannotti, Stable and actionable explanations of black-box models through factual and counterfactual rules, Data Min.
Knowl. Discov. 38 (2024) 2825–2862. [10] F. Condessa, J. M. Bioucas-Dias, J. Kovacevic, Performance measures for classification systems with rejection, Pattern Recognit. 63 (2017) 437–450. [11] J. Dressel, H. Farid, The accuracy, fairness, and limits of predicting recidivism, Science Advances 4 (2018). doi:10.1126/sciadv.aao5580. [12] B. Becker, R. Kohavi, Adult, UCI Machine Learning Repository, 1996. URL: https://doi.org/10.24432/
C5XW20. [13] H. Hofmann, Statlog (German Credit Data), UCI Machine Learning Repository, 1994. URL: https: //doi.org/10.24432/C5NC77. [14] P. Cortez, A. Cerdeira, F. Almeida, T. Matos, J. Reis, Wine Quality, UCI Machine Learning Repository, 2009. URL: https://doi.org/10.24432/C56S3T. [15] W. Nash, T. Sellers, S. Talbot, A. Cawthorn, W. Ford, Abalone, UCI Machine Learning Repository, 1995. URL: https://doi.org/10.24432/C55C7W. [16] V. Realinho, M. V. Martins, J. Machado, L. M. T. Baptista, Predict students’ dropout and academic success, https://doi.org/10.24432/C5MC89, 2021. Accessed on YYYY-MM-DD.