Explaining outliers' occurrence and mechanisms is crucial across various domains, as malfunctions, frauds, and threats require valid explanations for effective countermeasures. With the increasing use of sophisticated Machine Learning techniques to identify anomalies, explaining their presence becomes more challenging. Our proposed Decision Tree Outlier Regressor (DTOR) addresses this challenge by providing rule-based explanations for individual data points using anomaly scores from a detection model. By leveraging a Decision Tree Regressor to compute estimation scores and extracting relative paths, DTOR illustrates its effectiveness across different anomaly detectors and diverse datasets, including those with numerous features.
Internal audit in the banking sector is crucial for evaluating operational integrity and efficiency, assessing internal controls, risk management processes, and regulatory compliance. Anomaly detection techniques play a vital role in identifying atypical patterns and outliers within data populations analyzed for audit purposes, assisting in risk mitigation and fraud detection. However, ensuring the effective utilization of these techniques requires the ability to explain why certain records are considered anomalies, particularly for internal auditors with limited data analytics expertise [1,2].
Among various anomaly detection techniques, Isolation Forest [3], One-Class SVM [4], and Gaussian Mixture Models [5] are prominent anomaly detection techniques widely employed in practical applications [6,7]. These methods leverage diverse mathematical principles to detect anomalies efficiently. However, their interpretability may be limited, necessitating explainable artificial intelligence (XAI) techniques to elucidate model decisions, ensure transparency, and enhance trust in AI-driven decisions [8,9,10].
To meet this requirement, we introduce a novel model-agnostic XAI framework specifically designed for anomaly detection in the banking sector. Unlike conventional XAI methods that primarily focus on feature importance (e.g., SHAP and DIFFI [8,9]), our framework generates easily understandable rules to elucidate model predictions, thereby enhancing transparency and fostering trust in AI-driven decisions. Notable techniques such as LORE, RuleXAI, and Anchors [11,12,13] exemplify this approach.
Our approach aims to bridge the divide between interpretability and effectiveness in anomaly detection by offering human-understandable rules that clarify the rationales behind anomalous predictions. Relevant works in this domain include [14] and [15], focusing on online anomaly explanation and providing a survey of explainable anomaly detection methods, respectively. By harnessing rule-based explanations, our XAI framework ensures transparency and accessibility in the decision-making process of anomaly detection models for data scientists, domain experts, and colleagues in the banking industry.
Our novel XAI method, inspired by the principles of the Isolation Forest algorithm, takes advantage of the concept of isolating anomalies with minimal cuts in the feature space. To provide clear explanations for anomaly detection decisions, we use decision tree regressors. In our approach, a decision tree regressor is trained to learn the anomaly scores assigned to each data point generated by the Anomaly Detector. Notably, during training, we introduce a weighted loss function that gives a significantly higher weight to the data point under consideration. This weighting scheme ensures that the decision tree regressor prioritizes accurate estimation of the anomaly score for the target data point, thereby improving the interpretability and reliability of the local explanation. After training the decision tree, extracting the path of the datapoint can provide an interpretable rule for the anomaly score (algorithm 1). The implementation of DTOR at the following link can be accessed online 1
This section delineates the configurations of three Anomaly Detector models trained on two public datasets and one private dataset from Intesa Sanpaolo (see Table 1), offering explanations using both Anchors and DTOR. The DTOR method and the experiments conducted on public datasets are available in the GitHub repository accessible via the following link: https://github. com/rcrupiISP/DTOR.
The DTOR approach generates explanations for a given instance.
input : (π₯ e , π¦ Λe): the sample to be explained along its corresponding score from the AD;
Utilizing the novel XAI technique across various datasets aims to assess its effectiveness in explaining different types of anomalies learned by unsupervised Machine Learning models. The chosen Anomaly Detector models include IF, One-class SVM, and GMM [17]. Default parameters were opted for, as the primary objective of this study is to comprehend the explanation rather than optimize a performance metric specific to the dataset problem. Therefore, three distinct models were chosen to reason in different ways. The dataset was partitioned into training and testing sets. Specifically, the test set comprises 50 samples from each dataset, containing both anomalies and normal data points. The anomalies for GMM are defined to represent 5% of the training set, as well as for the isolation forest using the contamination hyperparameter set to 0.05. Default hyperparameters were retained for the SVM (kernel: radial basis function, π = 0.5, representing the upper bound on the fraction of training errors), resulting in anomalies representing about 50% of the training set.
We explore various explainability techniques, focusing on rule-based explanations due to challenges in interpreting feature importance methods like SHAP and DIFFI, especially with high-dimensional datasets. Initially, Anchors were used to explain the banking dataset, but we found limitations, such as the inability to reason on regression tasks and constraints in model implementation, leading to the development of DTOR. In addition to Anchors and DTOR, we considered LORE and RuleXAI. However, LORE requires extensive hyperparameter tuning, increasing implementation complexity. Additionally, RuleXAI is not actively maintained, with outdated Python library requirements. For future work, we plan to compare DTOR with other explainability techniques.
We adopt a perspective of providing rule-based explanations to Data Scientists, summarizing examples in Table 2 with four key metrics: execution time, coverage, and rule length. For DTOR, we set specific hyperparameters tailored to the banking dataset, ensuring both quantity and quality of explanations. However, a dataset-specific approach is crucial to identifying the optimal anomaly detector and evaluating explanation quality effectively. The hyperparameters for DTOR are carefully chosen, with the max depth set to 8, the min impurity decrease to 10 β5 , and the weight π½ for learning the rule to 0.1 * π , suitable for unbalanced datasets with anomalies. DTOR estimates the anomaly score rather than a binary output, and the same threshold used in anomaly detection models is applied to determine anomalies. While not detailed here, each rule output by DTOR provides both precision and average anomaly score, enhancing informativeness.
The findings derived from the DTOR algorithm provide significant insights into both anomaly detection and explainability methodologies. Notably, we observed a consistent trend towards shorter explanations for anomalies across various anomaly detection (AD) models and datasets, as evidenced by examples in Table 2, particularly instances with IDs 1 and 2. Conversely, instance ID 3 presents a lengthier explanation. This observation may align with the strategy employed by the Isolation Forest, which aims to isolate anomalies through a minimal number of steps. DTOR, by design, follows a similar path, leveraging the locally trained decision tree to isolate the sample. If the sample is an outlier, it can be easily separated with fewer steps, whereas non-outliers may require more complex separation. It's worth noting that our comparison was conducted against a surrogate classifier model, while our contribution introduces a surrogate regressor model. This distinction allows us not only to provide the rule but also to estimate the anomaly detection (AD) score, offering nuanced insights beyond binary classification tasks. Instance ID 1 showcases three distinct explanations, underscoring the variability introduced by different AD models and potential feature correlations. This phenomenon illustrates the Rashomon effect in explainability [18], where multiple plausible explanations coexist.
Although the execution time for generating explanations typically falls within seconds, it slightly increases for the banking dataset due to its larger sample size, necessitating additional computational resources. Looking ahead, further analysis is warranted to delve into these explanations in depth and compare them with state-of-the-art rule-based explainability techniques. Key metrics such as precision, coverage, and stability will be evaluated to assess the effectiveness of DTOR and its potential advantages over existing methods. For a more detailed analysis on the state of the art, performance and comparison experiments with Anchors, please refer to [19].