This extended abstract overviews the work presented in [1] where an extension of Trepan is proposed. Trepan is a seminal global explanation approach that extracts surrogate decision trees from black-box models. Trepan was extended to take into account explicit knowledge, modeled by means of ontologies, to extract human-understadable explanations. Trepan is a tree induction algorithm that recursively extracts decision trees from black-box classifiers [ 2]. The algorithm is model-agnostic, and it can be applied to explain any black-box classifier (e.g., Multi-Layer Perceptron, Random Forest). Trepan combines the learning of the decision tree with a trained machine learning classifier (the oracle). The proposed extension of the Trepan algorithm, called Trepan Reloaded, uses a modified information gain that, in the creation of split nodes, gives priority to features associated with more general concepts defined in a domain ontology. This was achieved by means of an information content measure defined using the refinement operators [3]. The perceived understandability of the extracted explanations by humans was tested by means of a user study with four diferent tasks. Results were evaluated in terms of response times and correctness, subjective ease of understanding and confidence, and similarity of free text responses. The results showed that decision trees generated with Trepan Reloaded, taking into account domain knowledge, were significantly more understandable than those generated by standard Trepan. The enhanced understandability of post-hoc explanations was achieved with little compromise on the accuracy with which the surrogate decision trees replicate the behaviour of the original neural network models.
1. Extended Abstract