In recent years, machine learning has made significant advancements in various fields, including image recognition. However, the complexity of these models often makes it dificult for users to understand the reasoning behind their predictions. This is especially true for sketch recognition, where the ability to understand and explain the model's decision-making process is crucial. To address this issue, our research focuses on developing an explainable machine learning framework for sketch recognition. The framework incorporates techniques such as feature visualization and feature attribution methods which provide insights into the model's decision-making process. The goal of this research is to not only improve the performance of sketch recognition models but also to increase their interpretability, making them more usable and trustworthy for users.
The task of creating diagrams on a computer using a traditional mouse and keyboard can be a dificult task compared to the ease of drawing with a pen and paper. To bridge this gap, stylus-based devices are used to provide a similar user experience to paper-based sketching. Recognizing these sketches, or identifying elements in the drawing, can enhance the user experience by allowing for advanced functionalities such as automatic beautification, intelligent editing, and animation of the content. However, a challenge in the field of sketch recognition is maintaining high accuracy while still allowing for a free-sketch environment similar to traditional pen and paper. Even though recognition techniques have become more sophisticated, it is dificult to understand the inner workings of blackbox machine learning methods [1, 2, 3]. Without a deeper understanding, it is hard to make substantial improvements to the recognition algorithm’s accuracy. In this research, we applied explainable AI techniques to assist in understanding how a machine learning based sketch recognition algorithm classifies instances. We believe this use of explainable AI (XAI) will lead to improved accuracy in future sketch recognition techniques. The main contributions of this work can be summarized as follow: • Inside the blackbox: We are able to provide insights into the inner workings of a blackbox machine learning model for sketch recognition.
By using techniques such as feature visualization and feature attribution methods like SHAP, we are able to provide a clear understanding of the model’s decision-making process. This is important because it allows researchers to understand how the model is able to classify sketches and identify the important features that contribute to the predictions. • Methodology for understanding the blackbox: We have outlined a methodology that can be used in future to understand blackbox sketch reocognisers. By incorporating interpretable models and feature attribution methods, we are able to provide a transparent and understandable explanation of the model’s decision-making process.
This is important because it allows users to trust the model’s predictions and understand why they are being made.
This work is still in progress and the above contributions are important to the sketch community so that we can understand how and why blackbox algorithms work. By providing insights into the inner workings of the model, we are promoting trust in the algorithms and allowing researchers to improve the decisions being made by the recognisers. This is crucial for the further development and use of blackbox algorithms for sketch recognition.
The remainder of the paper is organised as follows. In towards more sophisticated blackbox machine learning section 2 we discuss related work. Section 3 describes methods which produce higher accuracy rates [1]. Howour methodology. Section 4 presents the results of our ex- ever, although these blackbox algorithms might produce periments. Section 6 concludes the paper with directions high recognition rates, interpreting the results and how for future work. the classifications are made s becoming far more dificult. The use of XAI techniques on such blackbox models can assist in interpreting results and therefore lead to the 2. Related work design of more successful sketch recognisers, as has been illustrated other areas of research such as healthcare [18].
In recent years, there has been increasing interest in developing models that are not only accurate but also interpretable. The concept of model interpretability can 3. Materials and methodology be classified into two categories: global interpretability and local interpretability [4]. Global interpretability en- This section presents the details of our proposed methodables users to understand the overall structure of a model, ology. while local interpretability focuses on the reasoning behind a model’s decision for a specific input. Various 3.1. Datasets techniques have been developed in Explainable Artificial Intelligence (XAI) to enhance model interpretability, The chosen datasets are all full diagrams containing including Attention Mechanisms [5], LIME (Local Inter- shapes and text together (as opposed to isolated shapes pretable Model-agnostic Explanations) [6], Saliency Maps or text). They were chosen to represent a large variation [7], Counterfactual Analysis [8], Model Distillation [9], of diagram domains, as we seek to investigate domain and SHapley Additive exPlanations (SHAP) [10] . SHAP independent systems. They include examples of conhas gained increasing attention as it provides both global nected (e.g. directed graphs) and unconnected diagram and local interpretability by assigning an importance domains (e.g. user interface). They also include variations value to each feature in a prediction through the calcula- in the placement of text, such as those with text inside tion of the average marginal contribution of the feature in shapes (e.g. organisation), outside shapes (e.g. Euler), or all possible coalitions. It can measure feature importance annotated connectors (e.g. process diagrams). Table 1 for any model and handle interactions among features summarises datasets used in this research. [10].
Previous research in XAI for blackbox machine Table 1 learning-based sketch recognition algorithms has mainly Number of participants and strokes per dataset focused on visualising Convolutional Neural Networks Dataset # Participants # Text # Shape # Total (CNN’s), which is an image-based recognition approach. TUrsaeirninintegrface [1] 20 4354 671 5025 Peters et al. [11] produce videos using the dimensional- DOirrgeacnteisdatgiroanp[h1[]1] 2200 1106948 365047 1571085 ity reduction method, UMAP, to visualise neuron activ- EVRer[i1fi9c]ation 33 2143 1050 3193 ity in the training process. Mopuri et al. [12] are able PTerosctienssg[19] 33 2674 1195 3869 to highlight discriminative regions of images classified TMoi-nddo-mlisatp[1[]1] 2200 11781105 230614 12911719 by the CNN by examining the forward pass operation. UML class [1] 20 1481 383 1864 Theodorus et al. [13] compare an interpretable model, Euler [20] 9 60 60 120 BagNet, to blackbox CNN’s. They use a score to rank the interpretability of a model based on heatmaps of discriminative regions of an image. Cai et al. [14] focus 3.2. Feature Library on end-user interaction with a sketch recognition sys- For reliable and accurate recognition a set of quality featem by providing two example-based explanations for tures must be supplied to the algorithms. We employed predictions, normative and comparative. Normative ex- Blagojevic et al’s [21] digital ink feature library for our planations show examples from the target class (using experiments. This library contains 114 features each the ground truth), while comparative explanations show measuring unique characteristics of each stroke such as examples of the closest predicted classes. curvature, density, direction, intersections, pressure, size,
To our knowledge there have not been explorations temporal and spatial context and time/speed. into the use of XAI for other blackbox sketch recognition approaches, such as blackbox feature-based techniques e.g. using support vector machines [15] or ensembles [1]. 3.3. Classification methods While there are feature based approaches that are easier to interpret [16, 17], research directions have steered We have used Extra-trees classifier, which is generally considered to be a black box technique because it uses an ensemble of decision trees to make predictions. Decision where, where ∑︀() represents the summation over all trees are a type of supervised learning algorithm that possible sets of feature indices, is a set of feature indices is used to classify instances based on their features. It and is a subset of . || represents the cardinality of works by recursively partitioning the feature space into set , which is the number of elements in the set and ||! smaller regions, known as leaves, and making predictions represents the factorial of ||. (|| − | |)! represents the based on the majority class within each leaf [22]. The factorial of (|| − | |), which is the number of elements Extra-trees classifier makes predictions based on a com- in the set minus the number of elements in the subset . bination of features, and it may be dificult to determine | |! represents the factorial of | |, which is the number which features are most important or how they are being of elements in the subset . () represents the average weighted by the model [23]. prediction of the model when input features indexed by are set to their baseline values. ( ) represents 3.4. Methodology the average prediction of the model when input features indexed by are set to their baseline values and feature This section presents a framework for improving the in- is set to its actual value for sample . ( ( ) − ()) terpretability of any sketch classification system (SCS). represents the diference between the average prediction The framework is designed to enhance the transparency of the model when input features indexed by are set to of SCS, which is crucial for human operators making their baseline values and feature is set to its actual value decisions. The framework, as shown in Figure 1, com- for sample , and the average prediction of the model prises of two parts: the traditional structure of SCS on the when input features indexed by are set to their baseline left, and the interpretability-enhancing component on values. the right. The traditional structure includes the dataset, In simple terms, equation 1 provides a local explanatrained classification models, and predictions. tion i.e. provide an understanding of how each feature
The focus of the interpretability framework is on pro- is impacting the prediction for a specific sample. Global viding local and global explanations, using the SHAP explanations provide an understanding of the overall method, to improve experts’ trust in the SCS. The main importance of each feature and how it impacts the preidea behind SHAP values is to calculate the contribution dictions of the model across all samples. The global SHAP of each feature to the prediction of a specific sample. The values are calculated for each feature and they represent SHAP value for a feature is defined as the average dif- the average change in the model output caused by setference between the prediction of the model with that ting feature i to its actual value, while holding all other feature and the prediction of the model without that features fixed at their baseline values across all samples. feature, for all possible coalitions of features. Mathemati- The equation for the global SHAP value of feature is: cally, the SHAP value for a feature for sample denoted ⎡ ⎤ as (, ), is defined as: () = ∑︁()⎣(({}) − ())⎦ (2) (, ) = ∑︁()⎣⎡ (|(||−||!) |)! × | |! × (︁ ( ∪ {}) − ())︁ ⎦⎤ (1) In equation 2, represents the index of the sample being evaluated, and is a set of feature indices. The term () represents the average prediction of the model when input features indexed by are set to their baseline values.
On the other hand, ( ) represents the average prediction of the model when input features indexed by are set to their baseline values, and feature is set to its actual value for the sample .
This section describes the experimental setup, performance metrics used to evaluate the proposed approach and lastly, observed results are discussed in detail. This study was carried out using 2.3 GHz 8-core Intel i9 processor with 16 GB memory on Big Sur 11.4 operating system. The proposed approach is developed using Python programming language with several statistical and visualization packages such as Sckit-learn, Numpy, Pandas, Tensorflow, SHAP [24] and Matplotlib. In this work, we have used the Accuracy, Precision, Recall, F1-score for binary-class classification (text/shape). 4.1. Discussion We have made a number of diferent observations to understand the performance implications both during the training and testing phases. Table 2 presents the classification outcomes for various diagram datasets. The table shows the performance of the model on diferent datasets in terms of accuracy, precision, recall, and F1-score. From the table, it can be seen that the model performed well Figure 4: To-do list, Mind-map and UML class diagrams and was successful in identifying shape strokes and text datasets strokes. Additionally, the table illustrates that text strokes had a higher recall rate compared to shape strokes, meaning that the model was able to correctly identify a higher the model’s decision-making in more detail, the second percentage of text strokes than shape strokes out of all part of the framework is used to provide global and local the text strokes that were present in the dataset. How- explanations of the model’s predictions. Global explaever, it is important to note that the accuracy, recall, nations provide an understanding of the overall feature precision, and F1-score can only provide an overall per- importance and how it impacts the predictions of the formance metric for the model and it does not explain model across all samples. Local explanations provide an the reasoning behind its decision-making. To understand understanding of how each feature is impacting the prediction for a specific sample. This can provide insights into which features are most important for the model’s predictions and how the model is making its decisions. Figure 7: Local explanation visualisation behind wrong text The beeswarm plots presented in Figure 2 through Fig- sample ure 5 shows how each feature contributes to the overall output of a black-box model, providing a means to interpret the model’s global explanations for each dataset.
It is a combination of a scatter plot and a violin plot, where each dot represents a sample, and the y-axis represents the feature importance. To avoid overlapping, the dots in Figures 2 to 5 are horizontally jittered, and their colors represent the actual value of the feature for the corresponding sample; red dots indicate high feature values, while blue dots indicate low feature values. The violin plot in each figure shows the number of samples with similar feature values and can also identify outliers.
These figures display the twenty most important features extracted from the extra tree classifier for the shape class in various datasets. Each point in the figures represents a Shapley value for a feature per sample, and the features are arranged in descending order of importance. For example, Figures 2 to 5 reveal that LogLongestSideRect is the top feature for the extra tree classifier, and the model will consider data points as shape if this feature has a larger value.
The local explanation is provided by Figure 7 and Fig- Figure 8: Misclassified shape strokes ure 9 through a visual representation of the contribution of each feature to the model’s predictions for individual samples. It also provides a reference point by showing the baseline which is the average prediction of the model when all features are set to their baseline values. It can Figure 9: Local explanation visualisation behind wrong shape be used to interpret the predictions of a black-box model sample and to identify any potential issues with the model’s decision-making. The length of arrows tells about importance of feature in the prediction i.e. long arrows for the model’s decision. For example, Figure 6 shows have a large efect on the prediction. These features are text stroke classified as shape by the model. The Figure likely to be the most important for the model’s decision. 7 shows an explanation behind wrong prediction. In this Features with short arrows have a small efect on the specific example, it can be seen that the Average Density prediction. These features are likely to be less important of Close Strokes had a stronger influence on the model’s decision to classify the stroke as a shape, while the length of next stroke had a stronger influence on the model’s decision to classify it as text. In addition above two features are most important feature for model’s decision for this particular instance. Similarly, Figure 8 show instances of arrows that were incorrectly classified as text, whereas Figure 9 providing an explanation for one arrow that was incorrectly classified as text.
This study aims to enhance the interpretability of sketch recognition models, as many machine learning models in this field do not provide any insight into the reasoning behind their decisions. Future work in this field could include the following. Firstly, XAI techniques could be applied to other types of models to better understand and interpret their predictions. In this work we focused on using XAI techniques to interpret and explain the predictions of a black-box ensemble learning model. However, there are many other types of models, such as deep neural networks and support vector machines that could also benefit from the use of XAI techniques. Secondly, using a combination of XAI techniques could be explored. Each XAI technique provides diferent types of explanations therefore combining them can further enhance our understanding of the model’s decision-making by providing a more complete picture of the model’s predictions. Lastly, building on Cai et als work [14], user feedback can be further incorporated to better understand how users interpret the explanations provided by the models. [16] D. Rubine, Specifying gestures by example (1991) 329–337. URL: http://doi.acm.org/10.1145/122718. 122753. doi:10.1145/122718.122753. [17] A. Bhat, T. Hammond, Using entropy to distinguish shape versus text in hand-drawn diagrams, in: IJCAI International Joint Conference on Artificial Intelligence, volume 9, 2009, pp. 1395–1400. [18] G. Yang, Q. Ye, J. Xia, Unbox the black-box for the medical explainable ai via multi-modal and multicentre data fusion: A mini-review, two showcases and beyond, Information Fusion 77 (2022) 29– 52. URL: https://www.sciencedirect.com/science/ article/pii/S1566253521001597. doi:https://doi. org/10.1016/j.inffus.2021.07.016. [19] P. Schmieder, Comparing basic shape classifiers: A platform for evaluating sketch recognition algorithms, 2009. [20] M. Wang, Exploring intuitive methods for creating euler and spider diagrams, 2011. [21] R. Blagojevic, S. H.-H. Chang, B. Plimmer, The Power of Automatic Feature Selection: Rubine on Steroids, in: M. Alexa, E. Y.-L. Do (Eds.), Eurographics Workshop on Sketch-Based Interfaces and Modeling, The Eurographics Association, 2010. doi:10.2312/SBM/SBM10/079-086. [22] I. Goodfellow, Y. Bengio, A. Courville, Deep Learning, MIT Press, 2016. http: //www.deeplearningbook.org. [23] J. Younas, M. I. Malik, S. Ahmed, F. Shafait, P. Lukowicz, Sense the pen: Classification of online handwritten sequences (text, mathematical expression, plot/graph), Expert Systems with Applications 172 (2021) 114588. URL: https://www.sciencedirect.com/science/ article/pii/S0957417421000294. doi:https: //doi.org/10.1016/j.eswa.2021.114588. [24] S. M. Lundberg, B. Nair, M. S. Vavilala, M. Horibe, M. J. Eisses, T. Adams, D. E. Liston, D. K.-W. Low, S.-F. Newman, J. Kim, et al., Explainable machinelearning predictions for the prevention of hypoxaemia during surgery, Nature Biomedical Engineering 2 (2018) 749.