The COVID-19 disease caused by the SARS-CoV-2 virus rst appeared in Wuhan, China, and is considered a serious disease due to its high permeability, and contagiousness. The similarity of COVID19 disease with other lung infections, along with its high spreading rate, makes the diagnosis di cult. Solutions based on machine learning techniques achieved relevant results in identifying the correct disease and providing early diagnosis, and can hence provide signi cant clinical decision support; however, such approaches su er from the lack of proper means for interpreting the choices made by the models, especially in case of deep learning ones. With the aim to improve interpretability and explainability in the process of making quali ed decisions, we designed a system that allows a partial opening of this black box by means of proper investigations on the rationale behind the decisions. We tested our approach over arti cial neural networks trained for multiple classi cation based on Chest X-ray images; our tool analyzed the internal processes performed by the networks during the classi cation tasks to identify the most important elements involved in the training process that in uence the network's decisions. We report the results of an experimental analysis aimed at assessing the viability of the proposed approach.
The Novel Coronavirus, that reportedly started to infect human individuals at
the end of 2019, rapidly caused a pandemic, as the infection can spread quickly
from individual to individual in the community [
Early and automatic diagnoses are relevant to control the epidemic, paving
the way to timely referral of patients to quarantine, rapid intubation of serious
Copyright c 2020 for this paper by its authors. Use permitted under Creative
Commons License Attribution 4.0 International (CC BY 4.0)
cases in specialized hospitals, and monitoring of the spread of the disease. Since
the disease heavily a ects human lungs, analyzing Chest X-ray images of the
lungs may prove to be a powerful tool for disease investigation. Several methods
have been proposed in the literature in order to perform disease classi cation
from Chest X-ray images, especially based on deep learning approaches [
In this work, we investigate the use of convolutional neural networks (CNNs)
with the aim to perform multiple-disease classi cation from Chest X-ray images.
Diseases that are a matter of concern for our experiments are COVID-19,
Viral Pneumonia and Streptococcus Pneumonia. Notably, although these diseases
are characterized by pulmonary in ammation caused by di erent pathogens,
Streptococcus Pneumonia and Viral Pneumonia have similar clinical symptoms
to COVID-19 [
We analyze the CNNs-based model to identify the mechanisms and the motivations steering neural networks decisions in classi cation task. In particular, we use gradient visualization techniques to produce coarse localization maps highlighting the image regions most likely to be referred to by the model when the classi cation decision is taken. The highlighted areas are then used to discover (i) patterns in Chest X-ray images related to a speci c disease, and (ii) correlation between these areas and classi cation accuracy, by analyzing a possible performance worsening after their removal.
The remainder of the paper is structured as follows. We rst brie y report on related work in Section 2; in Section 3 we then provide a detailed description of our approach, that has been assessed via a careful experimental activity, which is discussed in Section 4; we analyze and discuss results in Section 5, eventually drawing our conclusions in Section 6. 2
In this section we present state-of-the-art methods used to (i) perform disease classi cation through Chest X-ray images and (ii) provide interpretability and explainability of the rationale behind the decisions performed.
Disease Classi cation. Deep learning-based models recently achieved
promising results in image-based disease classi cation. These models, such as CNNs [
Explainability of deep learning model. In the last year, attempts at
understanding neural networks decision-making have raised a lot of interest in
the scienti c community. Several approaches have been proposed to visualize the
behavior of a CNN by sampling image patches that maximize the activation of
hidden units [
In this work, we propose the use of Deep Learning approach to perform multiple disease classi cation using Chest X-ray. Additionally, we take advantage of a novel technique for analyzing the internal processes and the decision performed by a neural network during the training phase. 3 3.1
Classi cation Considering that other diseases appear similar to COVID-19 Pneumonia, including other Coronavirus infections and community-acquired Pneumonia such
Fig. 2: Architecture of the network Inception-v3. as Streptococcus, the distinction between these is extremely important and necessary, especially during a pandemic. Therefore, our purpose is to automatically identify the \correct" disease in Chest X-ray images.
In order to achieve this goal, we train CNN to classify patients according to 3 similar-based symptoms diseases (i.e., COVID-19, Viral Pneumonia, Streptococcus Pneumonia) and Healthy patients.
The herein proposed approach, illustrated in Fig. 1, is based on: (i) Multipledisease classi cation using CNNs, and (ii) Visual Explanations using GradCAM to indicate the discriminative image regions used by the CNN.
In order to classify patients, we used and compared the results of four neural
networks chosen on the basis of the good performance obtained on the
ImageNet data set over several competitions [
DenseNet networks [
The structure of Inception-v3 is shown in Fig. 2. The Inception modules (Inception A, Inception B and Inception C) are well-designed convolution modules that can both generate discriminatory features and reduce the number of parameters. Each Inception module is composed of several convolutional layers and pooling layers in parallel. The network is composed of 3 Inception A modules, 5 Inception B modules, and 2 Inception C modules that are stacked in series. The input size used is 224 224 and, after the Inception modules and convolutional layers, the feature map dimensions were 5 5 with 2.048 channels. Afterwards, we added 3 fully connected layers to the end of the Inception modules, and, nally, a softmax layer was added as a classi er outputting a probability for each class, and the one with the highest probability was chosen as the predicted class. 3.2
Visual Explanations We used GradCAM to identify visual features in the input able to explain result process achieved during the multiple classi cation. The overall structure of GradCAM is showed in Fig. 3. In particular, it uses the gradient information owing into the last convolutional layer of the CNN to assign importance values to each neuron. GradCAM is applied to a trained neural network with xed weights. Given a class of interest c, let yc the raw output of the neural network, that is, the value obtained before the application of softmax used to trasform the raw score into a probability. GradCAM performs the following three steps: 1. Compute Gradient of yc w.r.t. feature maps activation Ak, for any arbic trary k, of a convolutional layer (i.e., Ayk ). This gradient value depends on the input image chosen; indeed, the input image determines both the feature maps Ak and the nal class score yc that is produced. 2. Calculate Alphas by Averaging Gradients over the width dimension (indexed by i) and the height dimension (indexed by j) to obtain neuron importance weights kc, as follows: kc = global average pooling 1 zX}|X{ yc Z i j Aik;j ;
gradient|s v{iza }backprop where Z is a constant (i.e., number of pixels in the activation map). 3. Calculate Final GradCAM Heatmap by performing a weighted combination of the feature map activations Ak as follows:
LcGradCAM = ReLU (X kcAk);
k lin|ear co{mzbinat}ion where kc is a di erent weight for each k, and ReLU is the Recti ed Linear Unit operation used to emphasize only the positive values and to convert the negative values into 0.
(b) Viral Pneumonia (c) Streptococcus
Pneumonia (d) Healthy patients We describe next the setting of the experimental analysis performed in order to assess the viability of our approach. The dataset was split into training (80%) and testing (20%) sets; the 20% of the training set is used as validation set, in order to monitor the training process and prevent over tting.
All experiments have been performed on a machine equipped with a 12 x86 64 Intel(R) Core(TM) CPUs @3.50GHz, running GNU/Linux Debian 7 and using CUDA compilation tools, release 7:5, V 7:5:17 NVIDIA Corporation GM 204 on GeForce GTX 970.
Fine-tuning For the training phase we performed hyperparameters optimization. DenseNet 169 was trained with both optimizers Adam and SGD and for each optimizer 7 learning rate were tried. The best performance is obtained with the following con guration, trained for 300 epochs: Adam optimizer, learning rate 10 5, batch size 16, and binary cross-entropy as loss function.
The con guration of networks was modi ed in terms of the number of nodes or levels to optimize the performance. We empirically changed the number of layers and we trimmed network size by pruning nodes to improve computational performance and identify those nodes which would not noticeably a ect network performance. However, since we performed the experiments using well-know networks already optimized, we achieved the best performance using the standard con guration as originally proposed by respective authors.
We performed 10-fold cross-validation in order to choose the parameter value that gives the lowest cross-validation average error; experiments were performed on the very same machine with the same con guration of the other approaches. 4.3
Performance Metrics We assessed the e ectiveness of our approach by measuring Area Under the Curve (AUC) and Recall, especially focusing on the last one; indeed, in this context, the most important thing is to minimize False Negatives (i.e., disease is present but is not identi ed).
Let T P be a True Positive, T N a True Negative, F N a False Negative, and F P a False Positive, a ROC curve is a plot of true positive fractions (Se = T PT+PF N ) versus false positive fractions (Sp = 1 T NT+NF P ) by varying the threshold on the probability map. Closer a curve approaches the top left corner, then better is the performance of the system.
The Area Under the Curve (AUC), which is 1 for a perfect system, is a single
measure to quantify this behavior [
Recall (Rec = T PT+PF N ) considers prediction accuracy among only actual positives and explain how correct our prediction is among all people. 5
Table 2 and Table 3 report classi cation results after 10-fold cross-validation for all datasets in terms of Recall and AUC, respectively. Even though promising results are achieved in all DenseNet-based experiments, DenseNet 169 shows the most e cient architecture: if reports AUC mean value of 0:95 and Recall mean value of 0:90 over all the classes: hence, it was the one selected for the study.
The herein proposed approach achieves promisingly results; in particular, DenseNet 169 achieves the best performance on COVID-19 dataset (i.e., Recall mean value: 0:99 and AUC: 0:99), instead, it decreases on Viral Pneumonia dataset (i.e., Recall mean value: 0:83 and AUC: 0:92), and in classifying Healthy patients (i.e., Recall mean value of 0:80 and AUC: 0:89).
Analyzing the results, we see that Viral Pneumonia is often confused with Streptococcus Pneumonia, due to overlapping imaging characteristics and thus resulting in a Recall mean value always less than 0:90 in all experiments performed. It is worth noting that, in general, the extraction of CT scan images from published articles, rather than from actual sources, might lessen image quality, thus a ecting performance of the machine learning model.
A visual inspection of the GradCAM results con rms the quality of the model; indeed, our model exhibits strong classi cation criteria in the Chest region (see Fig. 5). In particular, red areas refer to the parts where the attention is strong, while blue areas refer to weaker attention. In general, the warmer the color, the more important are the features highlighted for the network.
Moreover, in order to con rm that the identi ed portions are actually signi cant, for each dataset we selected and removed the 40% of highlighted elements; this threshold was selected empirically after several experiments. A substantial decrease of Recall (on average around 10%) is shown using COVID-19, Viral Pneumonia and Streptococcus Pneumonia (i.e., p-value < 0.05 for paired t-test computed before and after images cutting); no statistical changes are shown using dataset of Healthy patients. This result suggests that GradCAM is able to identify the important elements involved in the training process and, consequently, responsibility for this diminishment is due to images cutting by which we removed the peculiar characteristic of the disease. (b) Viral Pneumonia (c) Streptococcus Pneumonia (d) Healthy patients In this work we exploit the use of CNNs and visual explanation techniques to estimate diagnosis using Chest X-ray and to analyze the internal processes performed by a neural network during the training phase with the aim of improving explainability in the process of making quali ed decisions. Basically, we try to identify the most important regions that in uence the network's decisions.
We ne-tuned the approach by means of accurate experimental activities; in particular, we classi ed four di erent disease datasets, and four di erent CNNs for the classi cation.
Experimental results show that our proposal is robust and it is able to identify speci c regions that are crucial in the neural network decision-making process, thus improving explainability. Indeed, classi cation accuracy is lower when highlighted regions are removed from the input images; this suggests the importance of these areas in disease classi cation and the possibility to consider the set of elements identi ed as potential disease markers.
In context where early and accurate medical diagnosis of speci c pathologies are essential, our method proves that visual explanation method combined with machine learning techniques can be used to provide solid disease classi cations and automatically discover new bio-markers by interpreting network decisions.
As future work is concerned, we plan to investigate misclassi cation errors and improve the generalization capability of the model. Our e orts will also include the interaction with physicians, so that proper medical expertise can be used to judge and better assess the quality of the regions highlighted by the proposed approach.