Results⋆

Abstract

In recent years, knowledge extraction approaches have been adopted to distill the medical knowledge included in clinical reports. In this regard, the Semantic Knowledge Extractor Tool (SKET) has been introduced for extracting knowledge from pathology reports, leveraging a hybrid approach that combines unsupervised rule-based techniques with pre-trained Machine Learning (ML) models. Since ML models are usually based on probabilistic/statistical approaches, their predictions cannot be easily understood, especially for what concerns their underlying decision mechanism. To explain the SKET's decisionmaking process, we propose SKET eXplained (SKET X), a web-based system providing visual explanations in terms of the models, rules, and parameters involved for each prediction. SKET X is designed for pathologists and experts to ease the comprehension of SKET predictions, increase awareness, and improve the efectiveness of the overall knowledge extraction process according to the pathologists' feedback. To assess the learnability and usability of SKET X, we conducted a user study designed to collect useful suggestions from pathologists and domain experts to further improve the system.

Keywords

• 0000-0001-5015-5498 (F
• Giachelle)
• 0000-0003-0362-5893 (S
• Marchesin)
• 0000-0002-0877-7063 (G
• Silvello)

1. Introduction

In the last decades, eXplainable Artificial Intelligence ( XAI) approaches have gained increasingly importance to face the lack of interpretability and explainability of AI models relying on Machine Learning (ML) and Deep Learning (DL) methods [ 2 ]. Indeed, in the medical domain, where ML and DL based methods for information extraction and retrieval are gaining popularity [ 3, 4, 5 ], the transparency of models and their decision processes is essential to promote trustworthy AI [ 6, 7 ]. In this regard, the High-Level Expert Group on Artificial Intelligence ( AI HLEG), set up by the European Commission, recently published a set of ethics guidelines for trustworthy AI, requiring that “algorithmic processes need to be transparent and decisions explainable” [ 8 ]. XAI approaches to support physicians and medical experts in the comprehension of algorithm predictions are urgently needed due to the increasing application of AI in the medical domain, especially for diagnostics [ 9 ]. In particular, explainability techniques can be useful in the Digital Pathology (DPATH) domain, where most of the approaches for image analysis are DL-based. Yet, they are efective but their comprehension can be challenging for humans due to their black-box nature [ 10, 11, 12 ]. In this context, Pocevičiūtė et al. emphasizes the importance of understanding why a specific prediction has been made in order to trust machine predictions exploited for diagnostic purposes [ 13 ]. Moreover, from a recent interview presented by Evans et al., it emerges that pathologists clearly prefer visual explanations for XAI [ 14 ]. As a prominent example, HistoMaprTM is a proprietary explainability tool for DPATH designed to support pathologists during the annotation activities of histology images, by means of visual explanations [ 15 ].

In the DPATH domain the Semantic Knowledge Extractor Tool (SKET) has been introduced to extract meaningful information – i.e., concepts, entity mentions, and labels – from pathology reports provided in natural language [ 1 ]. SKET performs the knowledge extraction process and produces weak annotations (labels) that can be used to train image classification algorithms for supporting the decision-making process in the DPATH domain [16]. However, since SKET adopts a hybrid approach that combines unsupervised rule-based techniques with pre-trained ML models, understanding the rationale behind SKET’s predictions may not be obvious for humans, despite it is crucial in the medical domain to have transparent models in order to trust their results. To explain SKET’s results, we designed and developed SKET eXplained (SKET X) a web-based system that exploits Visual Analytics (VA) techniques to provide the pathologists and experts with visual analyses and explanations regarding the underlying SKET’s decision process.

2. SKET X

SKET X is a web-based tool that aims to visually explain SKET predictions in order to ease their comprehension and support pathologists and domain experts in understanding the underlying machine decision mechanism [ 1 ]. SKET X is available at http://w3id.org/sketx1 and provides a visual interface to interact with an online instance of SKET; it can be used to continuously refine SKET parameters and rules in a human-in-the-loop interaction so that to improve SKET’s efectiveness progressively. SKET X allows the users to explain the rationale employed to determine the Named Entity Recognition and Linking (NER+L) pipeline’s outputs, including the concepts and the corresponding mentions identified by SKET in the provided clinical reports. Through SKET X, users can get useful insights about SKET’s knowledge extraction process and the resulting outputs. Specifically, users can visually identify the diferent components (e.g., models, rules, and parameters) activated during the knowledge extraction process. Thereby, users can easily understand why a certain output has been generated for the given clinical reports. SKET X exploits VA techniques to make SKET’s outputs visually intuitive via interactive interfaces. SKET X allows the users to execute SKET several times as independent pipelines considering diferent models, parameters, and data. Thereby, users can easily compare SKET’s results using the dedicated interface tab Compare which displays the results of two pipelines considered in the comparison in two vertical panels arranged side-by-side. Moreover, users 1Access provided with credentials demo/demo can compare variations of the same pipeline which is executed with diferent configuration parameters to assess their impact on the overall efectiveness of the knowledge extraction process. As a result, pathologists’ feedback can be exploited to refine the rules considered in predictions, leading to improved efectiveness of the knowledge extraction process.

Figure 1 shows the main interface of SKET X which is organized in six tabs: Overview, Input, Output, Params, Analytics, and Compare. The current active tab in figure is Analytics which allows the user to analyze the outputs for the current phase (e.g., Entity Linking (EL)) of the knowledge extraction process. The interface is divided in three major parts: (A) report section displaying information for the current report as well as controls to navigate among the reports; (B) Sankey diagram presenting the SKET’s rules for the current phase and highlighting the subset of rules activated for determining the current outputs; (C) section presenting the SKET’s outputs. We can observe that SKET identifies several concepts including Biopsy of Colon, Mild Colon Dysplasia, and Moderate Colon Dysplasia that have been identified using the SKET’s rules highlighted in the Sankey diagram on the right side.

User study To evaluate SKET X in terms of learnability, usability, and user satisfaction we conducted a user study to collect feedback from pathologists and experts to improve SKET X accordingly. Overall, nine participants were involved in the user study. We provided the participants anonymized credentials to access SKET X and a private link to an online form where they could answer predefined questions and provide feedback. The user study was organized into two parts, designed to measure the learnability and usability of SKET X, respectively. The learnability part focused on assessing the users’ confidence and awareness with accomplishing the following predefined tasks with SKET X: (i) analyzing the mentions/concepts identified by SKET; (ii) analyzing the labels (weak annotations) produced by SKET and (iii) answer questions about the results produced by SKET. To guide and support the users in the analysis process, we provided two explanatory videos. Then, the collected answers of each user were compared with the correct ones to assess the user proficiency with SKET X in the analysis and interpretation of SKET’s results. Secondly, we evaluated SKET X in terms of usability and user satisfaction using the System Usability Scale (SUS), that is considered an industry standard for assessing systems’ usability [17]. In this regard, we computed the average SUS score and it emerges that the usability of SKET X is quite good with a score equal to 66.7.

3. Conclusions

We introduced SKET X, a web-based system designed to explain the outputs generated by SKET through visual interactive interfaces. SKET X aims at simplifying the comprehension of SKET outputs as well as supporting pathologists and domain experts in their analysis, thus increasing awareness and understanding of the machine decision process. SKET X exploits VA techniques to explain why a specific prediction of SKET has been made and which are the roles of its diferent components involved in the knowledge extraction process. Hence, SKET X allows expert users to not only comprehend SKET results but also get valuable insights concerning the knowledge extraction process. Moreover, we assessed SKET X in terms of usability and learnability, by conducting a user study with digital pathology experts. To measure the SKET X’s learnability, we asked the participants to complete a sequence of analysis tasks by employing SKET X and then we asked them to answer a multiple-choice questionnaire. Thereby, we evaluated the number of tasks completed correctly by each user and thus the degree of understanding. From the answers and the explanations collected, we observed that almost all the participants correctly understood how to use SKET X to explain SKET results. Moreover, we asked the participants to answer a set of multiple choice questions to appraise user satisfaction and system usability according to the SUS scale. Finally, we collected useful suggestions from pathologists and other experts to identify the key necessities and foster further advancements in the design of transparent and explainable models/algorithms for DPATH.

Acknowledgments

The work was supported by the ExaMode project as part of the EU H2020 program under Grant Agreement no. 825292. Intelligence and Machine Learning for Digital Pathology, Springer, 2020, pp. 204–227. [16] N. Marini, S. Marchesin, S. Otálora, M. Wodzinski, A. Caputo, M. van Rijthoven, W. Aswolinskiy, J. M. Bokhorst, D. Podareanu, E. Petters, S. Boytcheva, G. Buttafuoco, S. Vatrano, F. Fraggetta, J. der Laak, M. Agosti, F. Ciompi, G. Silvello, H. Muller, M. Atzori, Unleashing the potential of digital pathology data by training computer-aided diagnosis models without human annotations, npj Digital Medicine 5 (2022). URL: http://dx.doi.org/10.1038/s41746-022-00635-4. doi:1 0 . 1 0 3 8 / s 4 1 7 4 6 - 0 2 2 - 0 0 6 3 5 - 4 . [17] J. Brooke, et al., Sus: A quick and dirty usability scale, Usability evaluation in industry 189 (1996) 4–7.

References

[1] S. Marchesin , F. Giachelle , N. Marini , M. Atzori , S. Boytcheva , G. Buttafuoco , F. Ciompi , G. M. Di Nunzio , F. Fraggetta , O. Irrera , H. Müller , T. Primov , S. Vatrano , G. Silvello, Empowering digital pathology applications through explainable knowledge extraction tools , Journal of Pathology Informatics ( 2022 ) 100139 . URL: https://www.sciencedirect.com/ science/article/pii/S2153353922007337. doi:h t t p s : / / d o i . o r g / 1 0 . 1 0 1 6 / j . j p i . 2 0 2 2 . 1 0 0 1 3 9 .

[2] U. Kamath , J. Liu, Explainable Artificial Intelligence: An Introduction to Interpretable Machine Learning , Springer, 2021 . doi:1 0 . 1 0 0 7 / 9 7 8 - 3 - 0 3 0 - 8 3 3 5 6 - 5 .

[3] S. Marchesin , G. Silvello, TBGA: a large-scale gene-disease association dataset for biomedical relation extraction , BMC Bioinform . 23 ( 2022 ) 111 . URL: https://doi.org/10.1186/ s12859-022 -04646-6. doi:1 0 . 1 1 8 6 / s 1 2 8 5 9 - 0 2 2 - 0 4 6 4 6 - 6 .

[4] M. Agosti , S. Marchesin , G. Silvello, Learning Unsupervised Knowledge-Enhanced Representations to Reduce the Semantic Gap in Information Retrieval , ACM Trans. Inf. Syst . 38 ( 2020 ).

[5] F. Giachelle , S. Marchesin , G. Silvello , O. Alonso , Searching for reliable facts over a medical knowledge base , in: Proc. of the 46th International ACM SIGIR Conference on Research and Development in Information Retrieval , SIGIR 2023 , Taipei, Taiwan, July 23-27 , 2023 , ACM, 2023 .

[6] S. Kundu , Ai in medicine must be explainable , Nature medicine 27 ( 2021 ) 1328 - 1328 .

[7] H. Müller , A. Holzinger , M. Plass , L. Brcic , C. Stumptner , K. Zatloukal , Explainability and causability for artificial intelligence-supported medical image analysis in the context of the european in vitro diagnostic regulation , New Biotechnology 70 ( 2022 ) 67 - 72 .

[8] G. Sartor, The impact of the General Data Protection Regulation (GDPR) on artificial intelligence , Technical Report, Panel for the Future of Science and Technology (STOA) , 2020 .

[9] A. Holzinger , G. Langs, H. Denk , K. Zatloukal , H. Müller , Causability and explainability of artificial intelligence in medicine , WIREs Data Mining Knowl. Discov . 9 ( 2019 ). URL: https://doi.org/10.1002/widm.1312. doi:1 0 . 1 0 0 2 / w i d m . 1 3 1 2 .

[10] A. Holzinger , B. Malle , P. Kieseberg , P. M. Roth , H. Müller , R. Reihs , K. Zatloukal , Towards the augmented pathologist: Challenges of explainable-ai in digital pathology , CoRR abs/1712 .06657 ( 2017 ).

[11] A. Holzinger , From machine learning to explainable ai , 2018 World Symposium on Digital Intelligence for Systems and Machines (DISA) ( 2018 ) 55 - 66 . URL: https://doi.org/10.1109/ DISA. 2018 . 8490530 . doi:1 0 . 1 1 0 9 / D I S A . 2 0 1 8 . 8 4 9 0 5 3 0 .

[12] A. Holzinger , Explainable AI and multi-modal causability in medicine, i-com 19 ( 2021 ) 171 - 179 . URL: https://doi.org/10.1515/icom-2020 -0024. doi:1 0 . 1 5 1 5 / i c o m - 2 0 2 0 - 0 0 2 4 .

[13] M. Pocevičiūtė , G. Eilertsen, C. Lundström , Survey of xai in digital pathology , in: Artificial intelligence and machine learning for digital pathology , Springer, 2020 , pp. 56 - 88 .

[14] T. Evans , C. O. Retzlaf , C. Geißler , M. Kargl , M. Plass , H. Müller , T.-R. Kiehl , N. Zerbe , A. Holzinger , The explainability paradox: Challenges for xai in digital pathology , Future Generation Computer Systems 133 ( 2022 ) 281 - 296 .

[15] A. B. Tosun , F. Pullara , M. J. Becich , D. Taylor , S. C. Chennubhotla , J. L. Fine , Histomapr™: An explainable ai (xai) platform for computational pathology solutions , in: Artificial