A proven way to detect various injuries: from fractures to heart failure, is an X-ray. However, because this examination method depends on the doctor's visual analysis, it can lead to misdiagnosis, that is, the case when the early stage of pneumonia will not be recognized and treatment will be ineffective. This study proposes using a convolutional neural network to classify chest X-rays to solve this problem. To do this, we analyzed the materials on the classification using neural networks for different areas of computer vision. In particular, convolutional neural networks for medical use are considered. The classification model of images on a database that included 112 thousand captions and 30 thousand unique patients is trained. High accuracy values of 0.93 and completeness of 0.99 models were obtained. An analysis of the literature on the convolutional neural networks was performed. Their shortcomings are taken into account, and a new optimization approach is proposed. The classification results were compared with a parallel approach on a GPU and a sequential on a CPU. The model trains on the GPU is 6.13 times faster than on the CPU based on the proposed algorithm. Computer vision, image classification model, parallelization, acceleration, GPU, CPU.
When it comes to identifying images, it is straightforward for us humans to recognize and
distinguish the different features of the depicted objects. All because our brains are constantly
subconsciously training on the same set of data, we can easily distinguish between various entities. In
contrast, the computer looks at the world around it differently: it is an array of numerical values that
form the critical aspects of an image or video that it tries to recognize. The principle by which the
system interprets the picture is radically different from how people do it. To vision, a computer needs
image recognition algorithms to analyze and understand what is happening in an image or its
sequence. An excellent example is the identification of pedestrians and vehicles, which is possible due
to the preliminary categorization and sorting of millions of prints – data provided by users [
Medicine is a clear favorite for areas that require a reliable image identification system while
generating a large amount of data on which you can train the same recognition [
2021 Copyright for this paper by its authors.
The classification task is for a computer to analyze an image and assign it to an appropriate class,
usually setting a label to a particular image. For us, the classification of images is elementary, but it is
also a great example of the Moravec paradox – that simple for humans is difficult for artificial
intelligence [
Early image classification depended on pure pixels – the computer split the image provided into
individual pixels. The problem with this approach is that two photos of the same object may look
different. Different backgrounds, angles, poses, and many other factors made it difficult for computers
to recognize and categorize images. This problem is designed to correct deep learning [
Unfortunately, the task of classification is quite resource-intensive and when using weak computing power or a consistent algorithm to obtain the result can take a long time. In such cases, optimization or the possibility of using better resources is usually considered.
The aim of this study is to propose a parallel algorithm and analyze the advantages obtained in solving the problem of classifying X-ray images to detect pneumonia using the CPU and GPU.
Classification of images in training often involves the use of convolutional neural networks
(CNN). The latter proved their power when AlexNet won the ImageNet competition by a wide margin
from other, more traditional neural networks. Since then, CNN has become one of the most promising
machine learning algorithms. It is widely used to solve problems involving large-scale datasets.
However, learning deep convolutional neural networks on large datasets is a highly intensive
computational task and requires much time to learn. That is why the algorithm is subjected to
parallelization, which reduces the load on one core, dividing the work between several [
There are two approaches: model parallelism, i.e., the model is divided between several computing
nodes and trained on the same data, and data parallelism, if the data is distributed on several nodes
and the same model is used for learning. Hybrid approaches using both parallelisms have also been
proposed. Examples of hybrid systems are papers [
One of CNN's most popular learning algorithms is stochastic gradient descent (SGD). It was
demonstrated in the article [
In paper [
Another example of an article related to medicine and the parallelism of CNN is [20]. But the method used in this work is a hybrid in which there is CNN and RNN.
Therefore, in the analysis of publications, no work was found that would demonstrate the parallelization of CNN on the CPU and GPU to classify pneumonia.
Some authors argue that parallelism of large-scale data is harmful and leads to deterioration of results; others say it is beneficial [21-23]. The absolute competitiveness of parallelization can show today, as we can obtain large amounts of data in free access, allowing more free experiments. Hence, it is not surprising that the literature remains ambiguous because all of the above methods have pros and cons.
Deep convolutional neural networks are used in many tasks, such as image classification, work with sound, etc. [24-27]. Although CNN was developed in 2012, it is gaining real popularity right now. The main factors for CNN's success are the availability of large data sets and the high performance of modern computer systems.
For high results, this neural network requires large amounts of data. For example, one of the earliest CNNs, LeNet-5 [28], was taught to recognize handwritten numbers using the MNIST dataset, containing about 60 000 images in a training set and 10 000 images in a test set. The CIFAR-10 dataset consists of 60 000 32x32 color images, including 50 000 training images and 10 000 test images. We took an NIH Chest X-rays database for our research, including 112 thousand highresolution pictures of 30 thousand different patients [29].
The database, which is considered in this paper, contains images of chest X-ray examinations, one of the most common medical examinations. One of the main obstacles to creating large data sets of Xray images is the lack of resources to label many photos. Before releasing this dataset, Openi [30] was the largest publicly available source of chest X-rays, with 4 143 images available.
The NIH chest X-ray data set consists of 112 120 X-rays with disease labels of 30 805 unique patients [29].
Thus, the neural network's task is to divide all images into two classes: pneumonia and healthy, based on notions of chest X-rays (see Figure 1).
Since this paper will compare the proposed parallel approach with the standard use of CNN, we first consider the operation of the convolutional neural network algorithm.
In tightly coupled neural networks, neurons are divided into groups that form successive layers. In them, each unit is connected to each neuron from neighboring layers. An example of this network is shown below in Figure 2:
This approach works well to solve a classification problem based on a limited set of defined characteristics. But the situation is complicated when the data to be classified are images. We could transmit the brightness of each pixel as separate units to the input of our dense network, but then for it to work, it must contain tens or even hundreds of millions of neurons. One way to solve this problem and reduce the network is to reduce the scale of the photo itself, but then we lose information. Therefore, convolutional neural networks are used to solve this problem.
To begin with, to analyze the operation of convolutional networks, you need to describe the data structure with which they will work. For convenience, the image is stored in a 2d matrix, each of which characterizes a particular image pixel. The image consists of three such matrices for the RGB model, each of which corresponds to a specific channel - red, green, and blue. If the image is black and white, then one matrix is used. Each of the digits is in the range from 0 to 255.
The main element in the work of CNN is a matrix, which is a filter or core. To process it, we pass our image matrix and convert it based on filter values. The following formula calculates the value of the object map. The input of the image is denoted by f and the kernel by h. The indices of the rows and columns of the result matrix are denoted by m and n, respectively.
[ , ] = ( ∗ ℎ)[ , ] = ∑ ∑ ℎ[ , ] [ − , − ].
After placing our filter over the selected pixels, we take each value and multiply it by the corresponding value from the kernel. Summarize all the results and write in the appropriate place on the map of the initial characteristics. The filter is above pixels that have tens and zeros. Let each pixel multiply by the corresponding one in the filter and add all the results. We write down this result in a matrix. We move the kernel and repeat the previous steps. In the end, we get a matrix with new data. Because the image decreases with each subsequent iteration, the number of image convolutions is limited. Also, following the movement of the filter, the influence of pixels located on the outskirts is observed. They are much smaller than the center of the image. In this way, part of the information present in the picture is lost.
A frame is added to the image matrix, mainly filled with zeros, to solve this problem. Depending we use fillings or not, we deal with two types of convolutions - original and same. Valid – the original image is used. Same – uses a frame around the original image so that when convolving, the output gives a matrix of the same size as the original image. For the second case, the width of the frame must be equal to the following value: = −1 2 where – filling, and – size of the filter (usually this value is odd).
One of the essential hyperparameters of the convolutional layer is the length of the step along which the filter should move. For the CNN architecture, this parameter is vital. If you want to overlap the receptive fields less or the spatial dimensions of the function map to perform the exercise, you need to increase the step. The following function is used to calculate the size of the output matrix: + 2 − + 1]. steps: result. and
Convolution over volume – is an essential concept because it allows you to work with color images and apply multiple filters within one layer. But keep in mind that the number of channels contained in the filter and the images must match. To use various filters in the same image, we collapsed filters separately, and the results are combined at the end. We can find the size of the tensor we obtain using the following formula: [ , , ] ∗ [ , , ] = [[ +2 − + 1] , [ +2 −
+ 1] , ], p is the fill used, s is the step used, is the number of filters. where is the size of the image, is the size of the filter, is the number of channels in the image,
Forward propagation consists of two steps. The first is the calculation of intermediate values of , which we obtain by convolving the input data from the previous layer by the tensor and then adding the offset . The second is the application of the nonlinear activation function to our intermediate value, which is denoted by the letter g. The following equations can demonstrate these [ ] = [ ] ⋅ [ −1] + [ ] [ ] = [ ]( [ −1]).
Now let's move on to the vital attributes in the complex layers. First, all neurons in convolutional layer
are interconnected. Second, some neurons have the same weight. This shows that the convolution has reduced the parameters to be studied. It is also worth noting that one value from the filter affects each element of the object map, which is crucial in backpropagation.
Consider the algorithm of the network in reverse propagation. This algorithm aims to calculate the derivatives and then use them to update the values of the parameters in a process called gradient descent. We want to assess the impact of parameter changes on the resulting feature map and the final
The problem of inverse propagation is to calculate the partial derivatives of cost functions: which will be transferred to the previous layer. At the entrance we have [ ] – which are derivatives related to the parameters of the current layer, as well as values [ ] the intermediate value
by applying the activation functions to the input tensor ′( [ ]). According to the chain rule, the result of this operation will be used later. [ ] = [ ]. The first step is to obtain Then the matrix operation is applied - full convolution. For this operation, we use a core that is [ ] [ −1] [ ] ∗ rotated 180 degrees. As a result, we have: where previous layer.
− filter, and [ ] = ∑ ℎ =0 ∑ =0
⋅ [ ] [ , ], [ ] [ , ] is a scalar that belongs to the partial derivative obtained from the
In addition to convolution layers, CNN very often uses so-called aggregation layers. They are mainly used to reduce the size of the tensor and speed up calculations. These layers are superficial – we need to divide our image into different regions and then perform a specific operation for each of these parts. For example, for the highest layer of the pool, we select the maximum value from each area and place it in the appropriate place at the output. As in the case of the convolutional layer, we have two hyperparameters available – filter size and pitch. Last but not least, if you are merging for a multi-channel image, the merging for each channel should be done separately.
As we can see, CNN learning can be pretty slow due to the number of calculations required for each iteration. Therefore, to speed up the work, it would be advisable to carry out parallel computing.
In parallel calculations, the problem is divided into independent smaller subtasks that run simultaneously. The results obtained are recombined or synchronized to formulate the impact of the initial count. The number of tasks into which computations can be divided depends on the number of cores. Which in turn are contained in the equipment. Unlike CPUs, which process operations sequentially, computationally tricky tasks on a GPU are distributed among thousands of processors, allowing you to perform calculations much faster [31].
Keras and TensorFlow technologies will be used to parallelize the algorithm. These technologies use the capabilities of parallel processing of the graphics processor. Due to the CUDA, computational tasks are performed sequentially on the CPU due to the C ++ software interface or in parallel on the GPU.
As mentioned above, a filter matrix with dimension n*n is used for a classical convolutional neural network. This process can be seen in Figure 3. This matrix is multiplied by a matrix of image pixels. These calculations are performed sequentially and independently of each other, which means that no analysis depends on the results of any other count. This lets us see that the convolution operation can be accelerated using a parallel programming approach and GPUs. This operation takes the most time in this algorithm and is the most voluminous because the image size is 500x500.
We can conclude that it is better to perform parallel calculations using graphics processors. To test this hypothesis, we will conduct experiments and test the results and learning time of the convolutional neural network sequentially on the CPU and in parallel on the graphics.
Intel (R) Core (TM) i7-10750H CPU with the following characteristics is used for numerical experiments:
After training, the following results were obtained: 0.1751
0.14
As shown from Table 3 and Figure 3, the losses in the training and validation samples decrease at the same rate in both the GPU and CPU versions. This means that according to the first indicator, the loss function, there is no difference in the models to use.
The second indicator that can be used to monitor the change in the model is its accuracy. The task is to maximize it, ideally to reduce it to 1, i.e., 100% accuracy.
As shown from Table 4 and Figure 6, the accuracy of the training and validation samples increases at the same rate in both the GPU and CPU versions. This means that for the second indicator, accuracy, there is no difference in the models to use.
Given the same rate of convergence of the model on both the CPU and GPU, an indicator that becomes important is the time spent on training one era, which is equal to 263 steps. Epochs
6 GPU
CPU
From Table 5, Figure 7-8, the average time spent on training 1 step with a GPU - 0.347 sec/step; CPU – 2 sec/step, the average time spent on training 1 era with GPU - 91.3 sec/epoch, and CPU 559.6 sec/epoch. This means that the model trains on the GPU 6.13 times faster than on the CPU.
One of the tasks is to train the model to classify chest X-rays to detect pneumonia. To prepare the model for 10 epochs using a GPU, because as shown above, it gives the same result 6.13 times faster.
The results presented in the previous section showed the advantage of parallel computing on the GPU overusing a serial algorithm on the CPU. Using the GPU for training took an average of 91.3 sec/epoch, and the CPU was 559.6 sec/epoch, which is 6.13 times faster for each period than the time it took the CPU to perform the same calculations. At the same time, the losses in the training and validation samples decrease at the same rate, and the accuracy in both instances increases at the same rate regardless of which processor was used, i.e., the model was equally well trained on both CPU and GPU.
It can also be concluded that the time required to move data from the CPU to the GPU, recombination or synchronization of data obtained from parallel computations to form the result of the initial calculation is not a significant threat to the speed of the program when processing large amounts of data. Fully compensated during the counts - so much faster they run on the GPU than on the CPU.
Figure 9 shows the result of the classification of some cases taken from the dataset. Regarding the work of the model itself, it is worth noting once again its high metrics of accuracy, precision, completeness, and the weighted average value of accuracy and recall (f1 score). As already mentioned, the use of classification in medicine requires or not the highest possible indicators. We can assume that our model meets this need, showing an 18% error in the absence of pneumonia and only 0.77% when lung damage was still present on X-ray.
However, there is room for improvement. In particular, the next step may be to reduce the error in the absence of pneumonia. By increasing the dataset or creating a more complex model, this can be done.
By performing convolutional neural network training to classify chest X-ray images on the CPU, and in combination with the GPU, it was shown that the training on the GPU is faster. In this particular case, training on the NVIDIA GeForce GTX 1650 Ti GPU was 6.13 times faster than on the Intel (R) Core (TM) i7-10750H CPU only. And while setting up a GPU requires extra time, reducing the training time of a convolutional neural network becomes very significant in real learning scenarios, when training a single neural network can take days. In this case, training on the CPU can last a week, when the use of the GPU will reduce this time to one day.
A convolutional neural network was constructed to classify chest X-rays to detect pneumonia, and an accuracy of 0.93 was obtained, with a recall of 0.99. Only three patients with pneumonia are misclassified, which is essential in the medical field because it is better to misclassify a healthy patient and pay more attention to him than to miss a patient who may get complications because of it. [20]. Yao, Hongdou et al. Parallel Structure Deep Neural Network Using CNN and RNN with an Attention Mechanism for Breast Cancer Histology Image Classification. Cancers vol. 11, 12 1901, (2019), doi.org/10.3390/cancers11121901. [21]. Elnashar, Alaa. To Parallelize or Not to Parallelize, Speed Up Issue. International Journal of
Distributed and Parallel Systems (IJDPS) Vol. 2, № 2, March 2011, pp. 14-28, (2011). [22]. Parallel computing and its advantage and disadvantage, 2018.
https://www.geekboots.com/story/parallel-computing-and-its-advantage-and-disadvantage. [23]. Martinovic, Goran, Zdravko Krpic and Snjezana Rimac-Drlje. Parallelization Programming
Techniques: Benefits and Drawbacks. (2010). [24]. Lee, S., Agrawal, A., Balaprakash, P., Choudhary, A., & Liao, W. K. Communication-Efficient Parallelization Strategy for Deep Convolutional Neural Network Training. In Proceedings of MLHPC 2018: Machine Learning in HPC Environments, Held in conjunction with SC 2018: The International Conference for High Performance Computing, Networking, Storage and Analysis, pp. 47-56, (2019), doi.org/10.1109/MLHPC.2018.8638635. [25]. Bird, Jordan & Faria, Diego & Manso, Luis & Ayrosa, Pedro & Ekárt, A. A study on CNN image classification of EEG Signals represented in 2D and 3D. Journal of Neural Engineering, 18(2), (2021), doi.org/10.1088/1741-2552/abda0c. [26]. Sharma, Atul & Phonsa, Gurbakash. Image Classification Using CNN. SSRN Electronic Journal. Proceedings of the International Conference on Innovative Computing & Communication (ICICC) 2021, 5 p., (2021). [27]. Palanisamy, Kamalesh & Singhania, Dipika & Yao, Angela. Rethinking CNN Models for Audio
Classification. 2020, 8 p., arXiv: 2007.11154v2 [cs.CV] 13 Nov 2020. [28]. Yann, LeCun, Leon, Bottou, Yoshua, Bengio, and Patrick, Haffner. Gradient-Based Learning
Applied to Document Recognition. Proc. of the IEEE, pp. 1-46, (1998). [29]. NIH Chest X-ray Dataset. URL: https://www.kaggle.com/nih-chest-xrays/data. [30]. Openi, chest X-ray collection. URL: https://openi.nlm.nih.gov/. [31]. Mochurad, L.I. Optimization of numerical solution of model problems on the basis of parallel calculations. Chapter 1: monograph. Lviv: PE “BONA Publishing House”, 208 p., (2021).