The Covid-19 pandemic has spread quickly, making identification of the virus critically important in assisting overburdened healthcare systems. Numerous techniques have been used to identify Covid-19, of which the Polymerase chain reaction (PCR) test is the most common. However, obtaining results from the PCR test can take up to two days. An alternative is to use X-ray images of the subject's chest area as inputs to a deep learning neural networks algorithm. The two problems with this approach are the choice of architecture and the method used to deal with the imbalanced data. In this study a comparative analysis of a standard convolutional neural network (CNN) and a number of transfer learning algorithms with a range of imbalanced data techniques was conducted to detect Covid-19 from a data set of chest x-ray images. This data set was an amalgamation of two data sets extracted from the Kaggle Covid-19 open source data repository and non-Covid illnesses taken from the National Institute of Health. The resulting data set was had over 115k records and 15 diferent type of findings ranging from no-illness to illnesses such as Covid-19, emphysema and lung cancer. This study addresses the problem of class imbalance on the largest data set used for x-ray detection of Covid-19 by combining undersampling and oversampling methods. The results showed that a CNN model in conjunction with these random over and under sampling methods outperformed all other candidates when identifying Covid-19 with a F1-score of 93%, a precision of 90% and a recall of 91%.
The novel coronavirus illness of 2019 has created a substantial and immediate threat to global health. The outbreak has paralyzed the world economy and created widespread disruptions to people’s lives. According to the World Health Organization (WHO), the coronavirus pandemic is putting unprecedented and escalating strain on healthcare systems around the world.
Covid-19 has been diagnosed using a range of medical imaging modalities,
blood tests (CBCs) and PCR. According to the WHO, all diagnoses of Covid-19
must be validated by reverse-transcription polymerase chain reaction tests. PCR
techniques are performed manually and have a high degree of inaccuracy with a
positivity rate of only 63% [
Deep learning may be used to make predictions based on medical images by
extracting distinctive characteristics such as shape and spatial rotation from the
images [
For this study, a comparative analysis of a standard convolution neural network (CNN) and number of transfer learning algorithms with a range of imbalanced data techniques was conducted to detect Covid-19 from a data set of chest x-ray images. Each algorithm and imbalance combination will be assessed using the F1-score, Precision, Recall and the area under the curve (AUC).
In section 2, past research is examined that employed deep neural networks to identify Covid-19 and other chest illnesses. The data sets used are described in section 3 and the methodology is detailed in section 4. Finally, our findings and analysis based on the model and data set are presented in section 5. 2
Researchers have developed deep learning techniques to detect and diagnose Covid-19 based on x-ray images of the chest. This literature review, describes the development of deep learning techniques, to detect Covid-19 and the methods used to handle an imbalanced data set. 2.1
A Generative Adversarial Network, is introduced by Loey et al., for pre-processing
chest x-ray images [
A combination of a CNN and a Long Short-Term Memory (LSTM), was
used by Islam et al., to detect Covid-19 using x-ray images [
An inception model was proposed by Das et al., to screen Covid-19 from other
non-Covid cases like pneumonia, tuberculosis, and healthy cases [
A CNN was implemented by Wang et al., to train the model over 13,975 chest
x-ray images to detect Covid-19 [
A concatenated network was developed by Rahhimzadeh et al., with
Xception and ResNet50V2 models to classify Covid-19 chest x-ray images [
A Deep Bayes-SqueezeNet model is proposed by Ucar et al., to detect
Covid19 images [
A deep neural network was designed by Khan et al., to automatically detect
and diagnose Covid-19 images [
The system proposed by T. Ozturk at al., includes seven pre-trained models:
VGG19, MobileNetV2, InceptionV3, ResNetV2, DenseNet201, Xception, and
InceptionResNetV2 [
A VGG16 deep transfer learning model was considered by Singh et al., to
detect Covid-19 from CT scans [
A review of the application of Machine-Learning and Deep-Learning in the
diagnosis of Covid-19 through x-ray and CT images was conducted by
MohammadRahimi et al [
During the pre-processing step, Lin et al., used two undersampling techniques
that involved clustering [
Sun et al, proposed an ensemble method to convert the imbalanced data
sets into multiple balanced data sets and then built classifiers on them using a
classification algorithm [
Nanni et al., proposed a neighborhood balanced bagging technique to modify
samples according to the class distribution in their neighborhood using two
approaches [
In this section, an overview of the approaches utilised in this study is given.
For the identification and categorization of the Covid-19 illness, two types of
chest x-ray image data sets are used in our study. The Covid-19 images are
taken from the Covid-19 chest x-ray image data set which is freely available on
Kaggle [
From Table 1, it can be seen that just over half the observations have been
classified under the ’healthy’ or ‘No findings’ label. The data set allows for
multiple label peer samples, however, a ‘No findings’ can only occur on its own. A
little over half of the observations in the general data set represent the absence
of disease and 3616 subjects had Covid-19. This amalgamated data set can be
found in a Github repository [
Each picture sample is accompanied by numerous fields in each of the two data sets. These variables contain patient information such as gender, ID and age, as well as information on the images perspective (PA or AP). The number of images per class is presented in Table 1.
The x-ray images used were taken at diferent orientations, positions, there was some variation in size. The following pre-processing steps were taken: resize the images to equal length and width; Gaussian Blur was used to remove the noise from the sample image; smooth the edges of the images to allow easy detection; segmentation was applied (histogram equalization), to normalize the brightness and contrast. 3.3
In general, for machine learning or deep learning imbalanced learning problems
are dealt with by using resampling [
In this section, we present the diferent resampling methods for handling imbalance in image data classification.
Oversampling Oversampling is a strategy that adds more samples to an imbalance class. Many studies have used this strategy to solve classification challenges for extremely imbalanced, image data sets.
There are many oversampling methods available which are designed as
variations of the oversampling technique SMOTE such as: Borderline-SMOTE,
Synthetic Sampling (ADASYN), SVM-SMOTE [
Instead of smaller particular decision areas, synthetic methods cause a
classifier to produce bigger and less specific decision regions [
Undersampling Random undersampling in the training data set removes certain data points from the majority class. This results in a decreased number of instances in the majority class of the training data set. This method is repeated until each class has the same number of examples. This approach is suitable for dealing with the imbalance when there are a suficient number of samples in the minority class that holds all the necessary information.
Undersampling approaches, may, on the other hand, result in a loss of
information, when data from the majority class are removed, leading to under-fitting.
No method exists to recognize or preserve information-rich instances from the
majority class, since examples are eliminated at random [
There are diferent approaches available making use of undersampling e.g.
TomekLinks. These approaches are used to solve problems with imbalanced data
sets; however, they are confined to moderate or low skewed data sets and are
not recommended for highly imbalanced data sets [
In order to avoid a low level of recall due to undersampling, this study applied
a combination of oversampling and undersampling, similar to the work by Sun
et al [
Convolutional Neural Network In this study, a convolutional neural network architecture was constructed to detect Covid-19 from chest x-ray images. The network has 14 layers: six convolutional layers, four pooling layers, four dropout layers, one fully connected layer, and one output layer. Each convolution block is made up of two 2-D CNNs and one pooling layer, followed by a dropout layer with a 25% dropout rate. The 5 x 5 kernel is used for feature extraction with an ReLU activation function at the convolutional layer. The 3 x 3 kernel is used by the max pooling layer to reduce the size of the input image. Finally, after extracting the features, it is connected to fully connected layers for making the prediction.
Transfer Learning Transfer learning is a deep learning approach that involves
training a neural network model on a problem that is similar to the one being
solved [
In addition to the developed CNN model, four pre-trained models for
classiifcation have been used with applied transfer learning: ResNet50, NasNetLarge,
Xception and InceptionV3. ResNet50 is made up of convolutional layers with
an average pooling layer at the end. It uses residual learning as a building
component. CIFAR10 and ImageNet are two types of architectures that are part of
NasNetLarge [
Convolutional layers with a ReLU activation function are used in the
intermediate flow on their own. Convolutional, max pooling and global average
pooling layers, with a ReLU activation function, make up the exit flow at the
end of the architecture [
To implement this structure, models were derived from the Python Keras package and initialized with a shape of (80,80,1). The transfer learning models are capable of categorizing a maximum of 1000 classes, however, for our purposes all that was needed was two. Therefore the transfer learning models were modiifed by substituting the fully connected layer with one that has only outputs. 3.5
The CNNs performances were assessed by using the following metrics: sensitivity,
recall, precision and F1-Score (F-Measure). All of these metrics may be
evaluated upon testing the model. The diferent outcomes for recall and precision is
discussed below [
The Receiver Operating Characteristic (ROC) curve expresses the
performance of the classification CNN model used in this study. A graphical approach
is used that works at all classification thresholds. It is able to achieve this by
graphing the True Positive Rate and False Positive Rate. The metric gives a
summary of the performance of a classiefir over all possible thresholds. Similar
to the ROC is the Area Under the ROC curve (AUC) which examines the entire
two-dimensional area underneath the entire ROC curve covering from (0,0) to
(1,1). This metric is efective at checking the proper or wellness and the quality
of our model’s prediction performance [
The CNN and pre-trained models developed in this study, receive pre-processed grayscale images as their input and binary classification is performed to distinguish between Covid and non-Covid cases. Meanwhile, the model is trained for 100 epochs and a batch size of 64 for each experiment.
To handle the imbalance problem, we tested the proposed model with a combination of oversampling and undersampling as described in Section 3.3. The new population of data was used for training the model. The obtained results are displayed in Table 2. The CNN model that uses mixed sampling outperforms the other combination of algorithms tested.
In machine learning, learning curves are a common diagnostic tool for algorithms that learn progressively from a training data set. After each update during training, the model can be assessed on the training data set and a hold out validation data set. Graphs of the measured performance can be generated to display the learning curves. In this study, the training performances for both SMOTE and Random Oversampling (ROS) were tested. A comparison of a learning curve is shown in Fig.4. Looking at these graphs, mixed sampling with ROS performs well and there is no sign of overfitting or underfitting. On the other hand, the mixed sampling with SMOTE graph, suggests that the model often lfuctuates at diferent epochs.
The classification evaluation metrics are summarised in the confusion matrix which results in the True positive, True Negative, False Positive and False Negative shown in Fig. 2.
The CNN model outperforms mixed sampling with random oversampling approach by achieving 90% precision, 91% recall (this avoids unnecessary false alarms) and 93% F1-score which measures the misclassified cases.
In the second phase of testing, pre-trained models were trained on diferent combinations of resampling techniques. The results from this analysis were not very significant. The large number of classes and continual overfitting of the models were the main factors contributing to these uninformative results. 5
Medical image analysis and feature extraction, which are used to diagnose a wide range of chest illnesses, have benefited greatly from deep learning. CNN architectures are well-known for their capacity to learn and predict mid-level and high-level visual representations.
In this paper, a combination of over and under sampling methods has been used to address the problem of class imbalance on the largest data set used for x-ray detection of Covid-19. Image pre-processing was performed as described in Section 3.3 to remove unnecessary details and each model was trained with all the combinations of the resampled data. The proposed modelling analysis outperformed many of the existing methods.
The data set used here, was comparatively large relative to other research conducted into x-ray classification of Covid-19. The presence of many diferent illnesses in the data set poses a higher challenge in classification compared to those works that used fewer disease types. Satisfactory results of F1-score, precision and recall have been obtained when compared to those obtained by previous works. For the purposes of reproducibility and experimental validation, both the data set and the code have been made publicly available.
Due to the exponential growth in Covid-19 reported cases and treatments throughout the world, the content of Covid-19 databases are continuously growing by significant amounts. As a result, future research could focus on improving the architecture of the deep-learning model proposed in this study, as well as testing its robustness on even larger data sets.
In future work, the study could be expanded to explore the use of One-ClassClassification algorithms used for imbalanced data sets, in conjunction with the techniques that have been proposed in this paper.