Explaining outliers' occurrence and mechanisms is crucial across various domains, as malfunctions, frauds, and threats require valid explanations for efective countermeasures. With the increasing use of sophisticated Machine Learning techniques to identify anomalies, explaining their presence becomes more challenging. Our proposed Decision Tree Outlier Regressor (DTOR) addresses this challenge by providing rule-based explanations for individual data points using anomaly scores from a detection model. By leveraging a Decision Tree Regressor to compute estimation scores and extracting relative paths, DTOR illustrates its efectiveness across diferent anomaly detectors and diverse datasets, including those with numerous features.
Internal audit in the banking sector is crucial for evaluating operational integrity and eficiency,
assessing internal controls, risk management processes, and regulatory compliance. Anomaly
detection techniques play a vital role in identifying atypical patterns and outliers within data
populations analyzed for audit purposes, assisting in risk mitigation and fraud detection.
However, ensuring the efective utilization of these techniques requires the ability to explain
why certain records are considered anomalies, particularly for internal auditors with limited
data analytics expertise [
Among various anomaly detection techniques, Isolation Forest [
To meet this requirement, we introduce a novel model-agnostic XAI framework specifically
designed for anomaly detection in the banking sector. Unlike conventional XAI methods that
primarily focus on feature importance (e.g., SHAP and DIFFI [
Our approach aims to bridge the divide between interpretability and efectiveness in anomaly
detection by ofering human-understandable rules that clarify the rationales behind anomalous
predictions. Relevant works in this domain include [
Our novel XAI method, inspired by the principles of the Isolation Forest algorithm, takes advantage of the concept of isolating anomalies with minimal cuts in the feature space. To provide clear explanations for anomaly detection decisions, we use decision tree regressors. In our approach, a decision tree regressor is trained to learn the anomaly scores assigned to each data point generated by the Anomaly Detector. Notably, during training, we introduce a weighted loss function that gives a significantly higher weight to the data point under consideration. This weighting scheme ensures that the decision tree regressor prioritizes accurate estimation of the anomaly score for the target data point, thereby improving the interpretability and reliability of the local explanation. After training the decision tree, extracting the path of the datapoint can provide an interpretable rule for the anomaly score (algorithm 1). The implementation of DTOR at the following link can be accessed online 1
This section delineates the configurations of three Anomaly Detector models trained on two public datasets and one private dataset from Intesa Sanpaolo (see Table 1), ofering explanations using both Anchors and DTOR. The DTOR method and the experiments conducted on public datasets are available in the GitHub repository accessible via the following link: https://github. com/rcrupiISP/DTOR.
Algorithm 1: The DTOR approach generates explanations for a given instance. def explain_instance: input : (e, ˆe): the sample to be explained along its corresponding score from the AD; (t, ˆt): a train set and its corresponding scores from the AD; : training weight associated to e; h: list of parameters of the decision tree; output : a list of rules explaining the instance (e, ˆe) ← len(t); model ← DecisionTreeRegressor(h); /* append the sample in the train set */ ˆ ← concat t with e; ˆ ← concat ˆt with ˆe; /* build the array of weights that gives more importance in the loss function to the sample e */ ← concat 1 with ; /* train the DT to the weighted configuration */ model.fit( (ˆ, ˆ), sample_wights=); /* retrieve the path taken by e in the decision tree rule ← _ℎ(model, e); return rule */
Utilizing the novel XAI technique across various datasets aims to assess its efectiveness in
explaining diferent types of anomalies learned by unsupervised Machine Learning models. The
chosen Anomaly Detector models include IF, One-class SVM, and GMM [
We explore various explainability techniques, focusing on rule-based explanations due to challenges in interpreting feature importance methods like SHAP and DIFFI, especially with high-dimensional datasets. Initially, Anchors were used to explain the banking dataset, but we 1https://github.com/rcrupiISP/DTOR.
Dataset
Lymphography (L) 100,000 214 148 26 9 19
anomaly identification and improved client analysis to discover probable instances of fraud or criminal conduct.
the USA Forensic Science Service and includes six diferent glass kinds, each distinguished by its oxide composition.
obtained from the University
found limitations, such as the inability to reason on regression tasks and constraints in model implementation, leading to the development of DTOR. In addition to Anchors and DTOR, we considered LORE and RuleXAI. However, LORE requires extensive hyperparameter tuning, increasing implementation complexity. Additionally, RuleXAI is not actively maintained, with outdated Python library requirements. For future work, we plan to compare DTOR with other explainability techniques.
We adopt a perspective of providing rule-based explanations to Data Scientists, summarizing examples in Table 2 with four key metrics: execution time, coverage, and rule length. For DTOR, we set specific hyperparameters tailored to the banking dataset, ensuring both quantity and quality of explanations. However, a dataset-specific approach is crucial to identifying the optimal anomaly detector and evaluating explanation quality efectively. The hyperparameters for DTOR are carefully chosen, with the max depth set to 8, the min impurity decrease to 10− 5, and the weight for learning the rule to 0.1 * , suitable for unbalanced datasets with anomalies. DTOR estimates the anomaly score rather than a binary output, and the same threshold used in anomaly detection models is applied to determine anomalies. While not detailed here, each rule output by DTOR provides both precision and average anomaly score, enhancing informativeness.
The findings derived from the DTOR algorithm provide significant insights into both anomaly detection and explainability methodologies. Notably, we observed a consistent trend towards shorter explanations for anomalies across various anomaly detection (AD) models and datasets, as evidenced by examples in Table 2, particularly instances with IDs 1 and 2. Conversely, instance ID 3 presents a lengthier explanation. This observation may align with the strategy employed by the Isolation Forest, which aims to isolate anomalies through a minimal number of steps. DTOR, by design, follows a similar path, leveraging the locally trained decision tree to isolate the sample. If the sample is an outlier, it can be easily separated with fewer steps, whereas non-outliers may require more complex separation. It’s worth noting that our comparison was conducted against a surrogate classifier model, while our contribution introduces a surrogate regressor model. This distinction allows us not only to provide the rule but also to estimate the anomaly detection (AD) score, ofering nuanced insights beyond binary classification tasks. Instance ID 1 showcases three distinct explanations, underscoring the variability introduced by diferent AD models and potential feature correlations. This phenomenon illustrates the Rashomon efect in explainability [ 18], where multiple plausible explanations coexist.
Although the execution time for generating explanations typically falls within seconds, it slightly increases for the banking dataset due to its larger sample size, necessitating additional computational resources. Looking ahead, further analysis is warranted to delve into these explanations in depth and compare them with state-of-the-art rule-based explainability techniques. Key metrics such as precision, coverage, and stability will be evaluated to assess the efectiveness of DTOR and its potential advantages over existing methods. For a more detailed analysis on the state of the art, performance and comparison experiments with Anchors, please refer to [19]. [18] M. G. M. M. Hasan, D. Talbert, Mitigating the rashomon efect in counterfactual explanation: A game-theoretic approach, in: The International FLAIRS Conference Proceedings, volume 35, 2022. [19] R. Crupi, A. D. Sabatino, I. Marano, M. Brinis, L. Albertazzi, A. Cirillo, A. C. Cosentini, Dtor: Decision tree outlier regressor to explain anomalies, arXiv preprint arXiv:2403.10903 (2024).