Pathology is a fundamental element of modern medicine that determines the final diagnosis and portrays the prognosis in most medical conditions. Due to continuous improvements in AI capabilities (e.g., object recognition and image processing), intelligent systems are bound to play a key role in augmenting pathology research and clinical practices. Despite the pervasive deployment of computational approaches in similar fields such as radiology, there has been less success in integrating AI in clinical practices and histopathologic diagnosis. This partly has to do with the opacity of end-to-end AI systems, which raises issues of interoperability and accountability of medical practices. In this article, we draw on interactive machine learning to take advantage of AI in digital pathology in an attempt to open the Blackbox of AI and generate a more effective partnership between pathologists and AI systems based on the metaphors of parameterization and implicitization. Artificial intelligence, Medical diagnosis, Pathology, Histopathologic diagnosis, End-to-end
Implicitization partnership
The application of AI for medical diagnosis is expanding rapidly [
Applications of AI in routine pathology; however, are constrained by some key challenges. These
can include infrastructural deficiencies such as limited digitization practices, reliable computational
infrastructures, or lack of reliable data storage. However, a closer examination into limited applications
of computer-mediated tools in pathology, particularly AI systems, reveals deeper issues than failed
technology and points to practice-level dynamics [
The mysterious and versatile power of neural networks is exactly what makes them blackboxes to humans, but as AI is making inroads into various domains and providing unprecedented performance,
2020 Copyright for this paper by its authors.
it leaves medical domain experts with ‘why’ questions. The job of these experts is often founded on
offering explanations and taking responsibility for decisions being made [
An important fact to emphasize is that pathologic diagnoses rarely fall into Boolean answers (e.g.,
benign, or malignant). In real-life clinical practice, pathologists often build on a diverse and complex
set of background information and foundations including their broad understanding of the clinical
contexts, the patient's history, and years of tacit knowledge [
In what follows we provide an overview of the common end-to-end approach towards the application of AI in medical diagnoses and juxtapose it with our approach, which centers on explainable AI. 2. End-to-end AI
The typical uses of deep learning focus on training and optimizing a model based on training data, mostly medical images, that have been labeled by human experts, here clinicians/pathologists. The label can include information related to patient outcomes, clinical classifications, and image annotations. Some studies even go around the issue of pixel-wise manual annotations by pathologists, train AI models on large training data sets (e.g., whole slide images) and offer automatic extraction and identification of features [e.g., 5, 11].
In this typical work system, the role of the human expert is either eliminated or relegated to 1) trainer of the algorithmic system, and 2) validator of the system’s recommendations and decisions (the loop marked in purple in Fig 1). It is important to note that beyond this stage in building and training endto-end AI systems, the pathologist is not replaced in clinical practices but provided with automated binary decisions. Such routines of AI systems arguably increase the speed and efficiency since the pathologists receive a final diagnosis. However, as the AI-driven diagnoses are not evidence-based, the clinicians cannot interact with machines and their logic and therefore there are few opportunities for mutual learning. Due to the Blackbox nature of the AI system, there is not much room for learning on the part of the human expert. In essence, both AI and human experts retain tacit knowledge which is difficult to transfer to the other party.
Our approach is informed and inspired by current research on interactive machine learning
(interactive ML), which is focused on training and optimizing algorithms through intuitive human
computer interface; and integrating users’ feedback into informing features. In interactive ML, human
experts are not just labelers or annotators, but they serve as the primary driver, guide, and active
explorer; she interacts with data and may directly contribute to feature extraction [
We contribute to interactive ML as this research stops short of presenting a more comprehensive work system, one that integrates nuanced organizational dynamics like developing a broader and embedded work system of collaboration between humans and AI. Current research on interactive ML typically stays at the level of user-interface interaction. 3.1.
We propose a framework for developing explainable AI workflows using a case study of grading of
meningioma (see [
We articulate the symbiotic interaction between pathologists and the AI system through the
continuous process of parameterization and implicitization. These two concepts are used as metaphors
to explain the reciprocal process through which the two partners work together and contribute to
explainable AI workflows. Inspired by the mathematical process of parameterization and
implicitization (mostly used in geometry), we describe how an effective partnership between humans
and AI can help reduce uncertainty and complexity as key factors that riddle the efficacy of decision
making in clinical settings and elsewhere. Humans have unique capacities in dealing with uncertainty
whereas AI systems are more competent in handling the complexity of information [
Parameterization is the process of identifying and expressing implicit equations using manifolds or a variety of parameters. Parameterization is instrumental in defining the state or quality of a system. Feynman parametrization, for example, helps express and evaluate loop integrals in so-called Feynman diagrams. Parameterization reduces complexity. Complexity refers to the abundance of variables and their associations. Parameterization ‘divides and conquers’ the problem area; that is, it breaks it down to sub-problems (parameters) so that these parameters and their associations become simple enough to be observed and understood directly.
Human experts' contributions are still irreplaceable in two ways: (1) Pathologists initiate the criteria
for a specific diagnosis; choosing ‘good’ parameters (features) are always the first step in interactive
ML and require domain and problem-specific knowledge [
AI provides two contributions to the parameterization process. (1) It extracts features/parameters defined by pathologists (e.g., it showcases the structure of cells in the data with nuclei larger than certain diameters ) (2) it could produce new knowledge (latent knowledge) by discovering new parameters and new relationships between parameters unnoticed or unknown by pathologists. Due to its analytical superiority in parsing and analyzing a large number of features and data points, AI can uncover parameters and associations that were not considered salient before. Humans decide if such new parameters/relationships discovered by AI are meaningful and valuable. The valuable new knowledge will be fed back to the algorithm for better diagnosis accuracy.
As noted, AI helps the discovery of associations between parameters decided by human experts as well as the association and parameters discovered by mass image processing and parametrization producing new knowledge. In cited work of grading meningioma, the algorithm looks for all the features that are compatible with a grade 2 meningioma and provides the histopathologic data such as brain invasion, specific subtypes, and all other features that qualify a grade 2 meningioma. The AI focuses on the criteria that are used by pathologists to grade the meningioma, not on a black-box process with which a pathologist has no way of interaction. The system receives feedback from pathologists based on their confirmation or rejection of an extracted feature; this leads to the improvement of the characteristic detection by the system.
In addition, AI can reduce complexity by adapting to evidence-based feature extraction and diagnosis. The advantage of the system is the speed of pattern recognition and characteristics that could take hours for a pathologist to go over. For instance, accurate detection of small foci of spontaneous necrosis is a challenging task that can take over an hour for a pathologist to check the entire set of slides. The system does the accurate detection task with high speed and provides the pathologist with all spontaneous necrosis candidates and the pathologist confirms or rejects the findings. And once again, such an interaction allows for the system to quickly get trained and apply newly discovered information with the help of the pathologist. 3.3.
Implicitization is the inverse process of parameterization. Implicitization refers to converting back
parameters and their association to a single implicit equation. Implicitization is key in keeping the
holistic nature of a system in view (the system is bigger than the sums of its elements). In the
decisionmaking context, humans enjoy a competitive edge in generating and maintaining a holistic and abstract
view, which is central in overcoming uncertainty (defined as the lack of information about all
alternatives or their consequences). The histological diagnosis, empowered by AI, offers a crucial but
inexorably limited perspective. This piece of analysis must be complemented by a more comprehensive
synthesis of the broader context to construct the final integrated diagnosis. The broader context consists
of many factors (see Fig 3) and the synthesis that goes into the diagnosis report often include
information such as sub-type/variant of cancer, grade, stage, as well as prognosis and recommendations
relative to the further management/treatment decisions. AI algorithms in current forms lack this
humanlevel general intelligence, and only perform efficiently and with high accuracy in narrow and specific
(and often binary) tasks of histological diagnosis. This is what AI researchers refer to as “weak AI”
[
This noted, AI can potentially contribute to the implicitization process by helping human experts in visualizing the associations between many parameters (i.e., patterns of associations) and understanding the ways interactions among those parameters/features or their unique combinations may give rise to the problem at hand.
In this workflow, the AI is utilized to recognize morphology and the final decision is made by the pathologists who make the final diagnosis and assemble the final report for the clinicians and patients. The pathologist decides on the accuracy of the results by going over all the findings detected and presented by the AI system. Such an approach is empowered by close and transparent interaction between the AI and pathologists.
The holistic view possessed by medical experts is often of an implicit/tacit nature and derives from
an intuitive decision-making style [
Parameterization and Implicitization help to open the Blackbox of AI-empowered systems and fulfill the vision of human-AI partnership. This approach reinforces the mutual learning between human experts and AI: The inner circle in figure 2 facilitates mutual learning through continuous parameterization and implicitization. Finally, the interpretability achieved here enables higher levels of trust and accountability as the human expert retakes the helm in understanding how the whole work system arrives at a decision. In this new work system, each partner brings unique capabilities and comparative advantage to the table (see Table 1).
In contrast to the end-to-end approach, our approach places humans at the center of “the learning loop”. Rather than consumers of the AI-generated diagnosis or trainers of the AI system; they occupy a more critical role (beyond feeding the ML model) and learn alongside the machine as the process of human-AI interaction is parameterized and unpacked. Such collaboration enables pathologists to discover new parameters and relationships identified by machines while they are enabled to feedback their verdict into the system.
Contribution of our approach to computer-mediated pathology is twofold. First, it provides a more effective approach towards human-AI interaction, one that reinforces and elevates the role of pathologists as human experts. Second, we offer a middle ground between (1) end-to-end AI and (2) manually engineered feature extraction by human experts. In our approach, we draw on the strength of both methods while overcoming their inherent limitations.
In this approach, AI closely collaborates with humans and contributes to parameterization and differential diagnosis by 1) capturing the complexity of multiple factors lying outside the cognitive processing of humans, 2) discovering new parameters, and finding connections between multiple parameters and evidence, and 3) learning alongside pathologists by using their inputs. In addition, AI can facilitate knowledge transfer among pathologists. Parameterization and finding criteria (as well as corresponding evidence) help experienced pathologists in the process of knowledge transfer to novices and students. Over the years these experts tend to internalize knowledge in a way that transcends explicitly connecting evidence and parameters; so, in their diagnostic approach, some of these experienced experts are less attentive to individual parameters and instead rely on holistic, implicit and automatic evaluations that do not easily lend themselves to explicit knowledge transfer and explanation. Through AI-enabled parameterization, interactions between experts and learners are facilitated.
Lack of interpretability and transparency stands in the way of clinical adoptions of these systems.
This symbiotic relationship presented here could clarify the unique contributions of both humans and
machines and raise trust in the application of AI in pathology. Our approach takes up the challenge of
interpretability and accountability, as common end-to-end approaches based on artificial neural
networks “do not provide a verifiable path to understanding the rationale behind its decisions” [
We appreciate Hongyan Gu, Yifan Xu, and Xiang 'Anthony' Chen for their contribution to this research project.