DCGAN for Enhancing Eye Diseases Classification Mahmoud Smaidaa, Serhii Yaroshchaka, Youness El Bargc a The national university of water and Environmental Engineering, Rivne, Ukraine b Multi-disciplinary Faculty Laraxche, Larache, Morocco Abstract One of the most important factors in image classification is the amount of data, especially in medical images. However, In the medical field, obtaining these datasets remains a challenge. In this work, we present Deep Convolutional Generative Adversarial Network (DCGAN) method that generate synthetic medical images. In addition, using GMD (Glaucoma, Myopia and Diabetic retinopathy) model to improve eye diseases classification with and without synthetic medical images. Four types of eye diseases, Glaucoma, Myopia, Diabetic retinopathy and Normal are our dataset. Our method is demonstrated on a limited dataset of eye disease (300 Glaucoma, 300 Myopia, 300 Diabetic retinopathy and 300 Normal). Firstly, we exploit DCGAN to generate synthetic medical images. then we utilize GMD method for eye diseases classification. finally, training our method using original data and synthetic medical images, then compare performance. The accuracy of the model had improved significantly from 76.58% in training set and 76.42% in validation set, to 80.45% in training set and 83.74% in validation set. We suggest that this work can be applied to other image classification models such as Vgg16, Inception v3, ResNet to enhance the accuracy. Keywords 1 Eye diseases, Deep learning, Myopia, DCGAN, Glaucoma, Diabetic retonapthy 1. Introduction One of the main challenges in the field of medical imaging is how to deal with small data sets, especially when we use supervised learning to classify images. In medical imaging assignments, explanatory reports are made by specialists with experience in his or her specialty. Although, some of medical data sets are able to be used online, and major challenges have been declared, most data sets remain limited and only apply to specific medical issues. Collecting medical data is a complex and costly procedure that requires collaboration between researchers and clinics [1]. Many researchers try to solve this problem with augmentation diagrams, including making some adjustments to the images of the data set such as rotation, cropping and size. Using such redundant data to improve network training has become a standard procedure for computer vision tasks, recently one of the most important method used is GANs. Generative Adversarial Networks (GANs), are a type of machine learning network that Ian Goodfellow and colleagues invented in 2014. Two neural networks compete with each other in a game (meaning game theory, often but not always in the form of a zero-sum game). The aim of it is to creating fabricated data similar to real data, which are difficult for a human or mechanical observer to distinguish between them. This technique learns to generate new data with the same statistical properties of a training set. For example, a GAN trained in photographs can create new images that CMIS-2021: The Fourth International Workshop on Computer Modeling and Intelligent Systems, April 27, 2021, Zaporizhzhia, Ukraine EMAIL: m.e.smaida@nuwm.edu.ua (M. Smaida); s.v.yaroshchak@nuwm.edu.ua (S. Yaroshchak); youness.elbarg@etu.uae.ac.ma (Y. El Barg) ORCID: 0000-0002-5552-2768 (M. Smaida); 0000-0001-9576-2929 (S. Yaroshchak); 0000-0002-6386-5402 (Y. El Barg) © 2020 Copyright for this paper by its authors. Use permitted under Creative Commons License Attribution 4.0 International (CC BY 4.0). CEUR Workshop Proceedings (CEUR-WS.org) look real to human observers, and have many photorealistic characteristics. Although it was originally proposed as a form of the generative model of unsupervised learning, GANs have also proven useful for semi-supervised learning, fully supervised learning, and reinforcement learning. At the 2016 symposium, AI expert Yann LeCun described GANs as "the coolest idea in the field of machine learning in the past 20 years" [2]. The Generative Adversarial Network consists of two separate neural networks, Generator (G) and Discriminator (D) constantly competing against each other, the generator, which tries to generate examples that seem the real data, it works to deceive the discriminator, while the role of discriminative network is to distinguish between the real and fake data. a vector of random number will be feeds to the generator, and the outputs are unreal synthetic data denoted by G (z), the discriminator feeds real data and the output of generator. The output of the discriminator is the possibility whether the entered data is real or fake, the output denoted by D (x). The output D (x) represents the probability that x will be a real image if it is 1, which means 100% is a real image, and the output is 0 which means that the image cannot be real. During training, the goal of creating the Generator is to try to create real images to deceive the Discriminator. The goal of Discriminator is to separate the Generated images from the real images as closely as possible. What is the outcome of the final match? In the most perfect case, Generator can create a real and fake G (z) images. For D it is difficult to determine whether the image generated by G is real so D (G (z)) = 0.5. In this paper, we investigate the problem of lack of medical images such as eye diseases for image classification. We propose Generative Adversarial networks (GANs) to increase our dataset in order to improve the accuracy of classification using our model (Glaucoma, Myopia and Diabetic retinopathy model). The model based on CNN for image classification. In this way, our goal is achieved and we obtain the generative model G, which can be used to create images in order to increase our dataset [4]. 1.1. How GAN is work: The above is just a rough overview of the basic principles of GAN, how can it be described in mathematical language? Here is a straightforward excerpt from the formula from Ian Goodfellow's paper, which called objective function of GAN [3]: min max V ( D, G )  E x  pdata ( x ) log( D( x)  E z  pz ( z ) [log(1  D(G ( z )))] (1) G D You only need to analyze this formula:  The complete formula consists of two periods. X represents the real image; Z represents the noise input to the G network.  G (z) is a fake image generated by the generator G.  D (x) is the probability of that X came from the real data (0~1), the discriminator should classify a real image as real. And this value should be close to 1.  Purpose of G as mentioned above. D (G (z)) is the probability that a D network governs whether the image generated by G is real. In other words, G wants D (G (z)) to be as large as possible (close to 1), and V (D, G) will be smaller at this time. So, we see that the forward tag of the formula is min_G.  Purpose of D: is simply a classifier. It tries to distinguish real data from the data created by the generator. train D to classify real images as real, D(x) should be close to 1. Train D to classify fake images as fake, D (G (z)) should be close to 0. Figure 1: Adversarial Nets Framework 1.2. Training process in GAN: Since the generative model and the discriminant model in the generative adversarial network are completely independent models and cannot be trained at the same time, separate repetitive training is used. For the discriminant, the label for the true sample is marked with the number 1, and the label for the generated sample is marked as 0, regardless of the quality of the sample generated. Updating the discriminator by ascending its stochastic gradient: 1 m  d  log D( x (i ) )  log(1  D(G ( z (i ) )))] m i 1 (2) Updating the generator by descending its stochastic gradient: 1 m  g  log(1  D(G ( z ( i ) ))) (3) m i 1 We train D with the hope that V (G, D) is close to 1, so we add a gradient (ascending). When training G in the second step, V (G, D) is as small as possible close to 0, so the gradient is subtracted. And then the entire training process rotates. 1.3. Different types of GANs There have been many types of GAN since its first appearance in 2014, including Convolutional GAN (CGAN), Deep Convolutional GANs (DCGANs), Least Squares GANs(LSGAN), Boundary Equilibrium GANs(BEGAN), StackGAN, InfoGANs, Wasserstein GANs (WGAN) and DiscoGANs. The dcgan that has been used in this paper will be explained briefly [4], [5], [6], [7], [8]. 1.3.1. Concept of DCGAN DCGAN is Deep Convolutional Generative Adversarial Network, the same as GAN, so we won't repeat it here. It only replaces the G and D above with two convolutional neuronal networks (CNN). But the change is not directly enough, DCGAN made some changes to the structure of the convolutional neural network to improve sample quality and convergence velocity. These changes include:  Instead of using pooling layers use stride convolutions (discriminator) and fractional-stride convolutions (generator).  Use batch normalization in both D and G.  Remove the FC layer and make the network a fully convolutional network.  The G ReLU network is used as the activation function, and the last layer uses tanh.  Use LeakyReLU as the activation function in the D network. Figure 2 shows G-Network in DCGAN [3]. Figure 2: G‐Network in DCGAN 2. Related work several researches have been published in the study of GAN. Most of these studies done recently. Emphasis will be placed on increasing the sample size of the database, a few reviews are as follows: Frid-Adar, Maayan, et al. [9] In this article, the authors proposed an enlargement scheme based on a combination of standard imaging and artificial liver lesions using GANs to improve the classification of liver lesions. Using the GAN, they pooled high-quality focal liver lesions from CT images and developed CNN for the liver lesion classification task and augmented the CNN training set using synthetic data created to improve classification outcomes. The classification efficiency gave a sensitivity of 78.6% and a specificity of 88.4%. By adding the synthetic data, they obtained a sensitivity of 85.7% and a specificity of 92.4%. Iqbal, Talha, and Hazrat Ali. [10] in this work authors propose a new GAN for Medical Images (MIGAN). The MIGAN generates synthetic retinal image with their segmented masks, which will be used to apply a controlled analysis of medical images. One of the features of the proposed model is the ability to learn features from a few dataset. Shmelkov, Konstantin, Cordelia Schmid, and Karteek Alahari. [11] The authors suggest two measures based on the classification of images, GANTrain and GANTest, which come close to recall and precision. They introduce new assessment metrics to compare class conditional GAN structure with GANTrain and GANTest scores. They used NN architecture for the classification of images for these two measurements. Authors train their networks with generated images to calculate the GANTrain. Then they obtained performance on a test set which collected from real images. Bowles, Christopher, et al. [12] this article explains the feasibility of injecting synthetic data obtained from the GAN into the original datasets in two brain segmentation problems, resulting in an improvement in the dice similarity rate by 1 to 5 percentage points under various conditions. They used a Progressive Growing of GANs (PGGAN) network to generate synthetic data. PGGAN was chosen, because the ability of training at large image sizes. They added the synthetic data with the real data to increase their dataset. The best result obtained when the authors used GAN + Rotation augmentation. Zeid Baker, Mousa. [13] the researcher in this paper used Generative Adversarial Networks to generate synthetic images similar to the real dataset in order to expand a dataset. they used two types of experiments were achieved, the first is using fine-tune a Deep Convolutional Generative Adversarial Network for a specific dataset, while the second experiment was used to analyze how synthetic data affect the accuracy in a classification task. authors used three types of datasets MNIST, Fashion-MNIST and Flower photos. The authors conclusion that DCGAN leads to an increase the model accuracy depends on the type of the dataset and the data preprocessing plays a big roll on a DCGANs performance, as for almost any ML algorithm. Wu, Qiufeng, Yiping Chen, and Jun Meng. [14] in this work, authors used Generated images augmented by deep convolutional generative adversarial networks (DCGAN) and original images to Identify Tomato Leaf Disease. They used GoogLeNet classifier to training and testing 5 classes of tomato leaf images, this model achieved accuracy of 94.33%. the authors result that images generated by DCGAN not only enlarge the size of the data set, but also have the characteristics of diversity, which makes the model have a good generalization effect. 3. Material and method 3.1. Datasets The dataset was analyzed and preprocessed of four different classes which contain 1692 of Glaucoma, Myopia, Diabetic retinopathy and Normal images, were selected for this study to aim for the highest variance among classes. where 1200 images used for training, 246 images for testing and 246 images used for validation purpose, which mean 15% of the total images were used for testing and 15% for validation purpose. All the images were collected in total from Kaggle dataset and iChallenge-GON Comprehension, in high resolution images. Figure 3: Images of Normal Fundus, Glaucoma, Diabetic retinopathy and Myopia The size of all images should be the same, colored and png format. In machine learning and deep learning, accuracy of model will be affected if the samples are not equally distributed [15]. Table 1 Eye diseases samples in original dataset Diseases Training Validation Glaucoma 300 48 Diabetic retinopathy 300 66 Myopia 300 66 Normal 300 66 Total 1200 246 In addition, 300 images have been used of each class to train our model. There are two issue should be considered: The number of parameters when the number of network layers increases, and the small number of eye diseases which we have collected. That will lead to the overfit under the influence of many parameters and few datasets. Increasing of the dataset is an effective way to solve this issue. In this paper, authors proposed DCGAN network to increase eye diseases dataset by generate synthetic dataset. Therefore, the training data set, and validation set will be increased (data enhancement). 3.2. Model building The main goal in this work is to generate images that look like realistic images in each class in order to solve the problem of insufficient medical images. in addition, using GMD model to classify eye diseases with and without DCGAN. 3.2.1. GMD model As shown in Figure 4, the size of input image is set to 64×64 pixels with 3 RGB channels. To extract the features from the image, two convolution layers were used: the first is 256 filters of size 3×3 pixels and the second is 128 filters of size 3×3 pixels. For the pooling layer, a window of size 2×2 pixels were used, which compresses the original image size for further processing. After that, another three convolution layers were used of 128, 64, and 32 filters with size 3×3 pixels with a maximum pooling size 2×2 pixels. Then, fully connection is used (Dense 100 units and 60 units) and output layer (4 units) to predict the eye cases. CNNs adjust their filter weights through backpropagation, which means that after the forward pass, the network is able to look at the loss function and make a backward pass to update the weights. The GMD model performance was evaluated against training, testing, and validation datasets using the accuracy measure, which reflects the ability of the model to classify the whole database into four categories with almost no error. The following represents the accuracy measure equation [16]: (4) Where, TP is True Positive: Your predicted positive and it is true, TN is True Negative: Your predicted negative and it is true, FP is False Positive: your predicted positive but it is false and False Negative FN: Your predicted negative but it is false. Figure 4: Block diagram of GMD model 3.2.2. DCGAN architecture GAN consists of two separate neural networks, Generator (G) and Discriminator (D) constantly competing against each other, the generator, which tries to generate examples that seem the real data, it works to deceive the discriminator, while the role of discriminative network is to distinguish between the real and fake data. a vector of random number will be feeds to the generator, and the outputs are unreal synthetic data denoted by G (z), the discriminator feeds real data and the output of generator. The output of the discriminator is the possibility whether the entered data is real or fake, the output denoted by D (x). The output D (x) represents the probability that x will be a real image if it is 1, which means 100% is a real image, and the output is 0 which means that the image cannot be real [17]. Objective function and how GAN is work, we mentioned above in introduction in section 1. DCGAN is Deep Convolutional GAN, it only replaces the G and D with two convolutional neuronal networks (CNN). Compared with GAN, DCGAN made some changes to the CNN architecture to improve sample quality and convergence speed. These changes are mentioned above in section 1.3.1. In this paper, the block diagram of eye diseases identification is shown in Figure 5. We also will interduce the proposed generator and discriminator network used by referring to the DCGAN architectures as shown in Figure 6: Figure 5: Block diagram of eye diseases identification Figure 6: discriminator (left) and generator architecture (right) 4. Experiments and Results According to the models explained above, all these models were implemented by Python language (Google Colab) using dual Graphics Processing Unit (GPU). The main objective of this paper is to increase our dataset using DCGAN. These data will be feds to GMD model in order to improve eye diseases classification. Accuracy will be measured before and after adding synthetic dataset. Dataset has been explained in section 3.1. it consists of four types of eye diseases, Glaucoma, Diabetic retinopathy, Myopia and Normal eyes. Table 1 shows the number of samples of each class in training set and validation set. 4.1. DCGAN results DCGAN is to increase our dataset in order to improve the diagnosis of eye diseases. The schema of our experiment as shown in Figure 5, has been applied. We utilized our dataset which consists of 1200 training samples, 246 validation samples of eye diseases size 64x64 coloured images for four general classes. We used the same network structure for the generator and discriminator as shown in figure 6, and training our medical images, the parameters of the discriminator and generator are updated in the same number of iterations. During this process BCElose will be calculated on all real and fake batch, and calculate gradients for D and G in backward propagation. # (1) Update D network: maximize log(D(x)) + log(1 - D(G(z))) errD_fake.backward() ## Train with all-real batch D_G_z1 = output.mean().item() netD.zero_grad() # Add the gradients from the all-real and all-fake batches # Format batch errD = errD_real + errD_fake real_cpu = data[0].to(device)[:1] # Update D b_size = real_cpu.size(0) optimizerD.step() label = torch.full((b_size,), real_label, dtype=torch.float, de # (2) Update G network: maximize log(D(G(z))) vice=device) netG.zero_grad() # Forward pass real batch through D label.fill_(real_label) # fake labels are real for generator c output = netD(real_cpu).view(-1) ost # Calculate loss on all-real batch # Since we just updated D, perform another forward pass of all- errD_real = criterion(output, label) fake batch through D # Calculate gradients for D in backward pass output = netD(fake.to(device)[:1]).view(-1) errD_real.backward() # Calculate G's loss based on this output D_x = output.mean().item() errG = criterion(output, label) ## Train with all-fake batch # Calculate gradients for G # Generate batch of latent vectors errG.backward() noise = torch.randn(b_size, nz, 1, 1, device=device) D_G_z2 = output.mean().item() # Generate fake image batch with G # Update G fake = netG(noise) optimizerG.step() label.fill_(fake_label) # Classify all fake batch with D output = netD(fake.detach().to(device)[:1]).view(-1) # Calculate D's loss on the all-fake batch errD_fake = criterion(output, label) # Calculate the gradients for this batch For the discriminant, the label for the true sample is marked with the number 1, and the label for the generated sample is marked as 0, regardless of the quality of the sample generated. So, D(x) should be close to 1 and D(G(z)) should be close to 0. In our experiment we save just the images that D(G(z))>=0.76 which close to 1. x=0.76 if errD_fake >= float(x) or errD_fake == errD_real : vutils.save_image(vutils.make_grid(fake[:1].to(device)[:1], padding=2, normalize=True), "%s/fake_samples_epoch_%03d.p ng" % ("/content/drive/MyDrive/GAN2/FakeImages", epoch), normalize = False) Adam was used as the gradient method for learning parameters of model. Its initial learning rate is 0002. optimizerD = optim.Adam(netD.parameters(), lr=lr, betas=(beta1, 0.999)) optimizerG = optim.Adam(netG.parameters(), lr=lr, betas=(beta1, 0.999)) The number of epochs we have used is 4000, that is not mean 4000 images will be generated, because we save the images that D(G(z))>=0.76. Table 2 shows the synthetic images and total images which we obtained. Figure 7 shows samples of the fake and real images. Table 2 Eye diseases samples after using DCGAN Diseases synthetic data using Original data + synthetic data DCGAN Training Validation Glaucoma 318 618 48 Diabetic retinopathy 294 594 66 Myopia 428 728 66 Normal 306 606 66 Total 1346 2546 246 Figure 7: sample of real and fake image The figures below show the loss during training in both Generator and Discriminator for all classes: Figure 8: Loss in Glaucoma Figure 9: Loss in Diabetic Retonapthy Figure 10: Loss in Myopia Figure 11: Loss in Normal 4.2. GMD model Results with Original data GMD model has been applied, Constant learning rate is the default learning rate schedule in SGD. Both momentum and decay rate are set to zero. Learning rate was set to 0.001 which showed a good performance to start. In constant learning rate schedule, the model was defined and the learning rate value was set to constant as follows. # define CNN_ model model1 = CNN_model() epochs = 100 # define SGD optimizer and set to default except lr learning_rate = 0.001 sgd = SGD(lr=learning_rate, momentum=0.0, decay=0.0, nesterov=False). Then, compiling, fitting GMD model and plotting the model accuracy as shown in Figure.12. We obtained Accuracy on training set 76.58% and Accuracy on validation set 76.42%. Figure 12: model accuracy using constant learning rate without data generated 4.3. GMD model Results with synthetic data The model has been applied and constant learning rate is the default learning rate schedule in SGD as mentioned in paragraph 4.1. Then, compiling, fitting our model and plotting the model accuracy as shown in Figure.13. The result obtained, accuracy on training set 80.45% and accuracy on validation set 83.74%. Figure 13: model accuracy using constant learning rate with data generated 5. Conclusion In this research, a Deep Convolutional Generative Adversarial Network, and GMD model based on Keras, TensorFlow and pycharm has been deployed using Python on a database of four types of ophthalmic cases: Glaucoma, Myopia, Diabetic retinopathy, and Normal eyes. The implemented DCGAN and GMD model have been trained using GPU in Google Colab, in order to increase our dataset to improve eye diseases classification. In conclusion, DCGAN strongly can be used for generating high quality of medical images and increasing training data. Furthermore, the performance of GMD model with original data and with synthetic images data were compared. We confirmed that with the help of DCGAN, GMD model (CNN model) can yield a better accuracy compared to traditional methods. Moreover, the accuracy of the model had improved significantly from 76.58% in training set and 76.42% in validation set, to 80.45% in training set and 83.74% in validation set. Based on this work, we suggest that DCGAN can be optimized for multi-class generator. We also suggest that this work can be applied to other image classification models such as Vgg16, Inception v3, ResNet to enhance the accuracy. We noticed that the images we have obtained from DCGAN had some noise. Therefore, in the next paper we will use denoising Autoencoder to enhance these images to improve the model accuracy. 6. References [1] H. Greenspan, B. van Ginneken, and R. M. Summers, “Guest editorial deep learning in medical imaging: Overview and future promise of an exciting new technique,” IEEE Transactions on Medical Imaging, vol. 35, no. 5, pp. 1153–1159, May 2016. [2] Mustaffa Hussain, GANs-What and Where?, 2020. URL: https://medium.com/thecyphy/gans- what-and-where-b377672283c5. [3] Goodfellow, Ian J., et al. "Generative adversarial networks." arXiv preprint arXiv:1406.2661 (2014). [4] Radford, Alec, Luke Metz, and Soumith Chintala. "Unsupervised representation learning with deep convolutional generative adversarial networks." arXiv preprint arXiv:1511.06434 (2015). [5] Mirza, Mehdi, and Simon Osindero. "Conditional generative adversarial nets." arXiv preprint arXiv:1411.1784 (2014). [6] Mao, Xudong, et al. "Least squares generative adversarial networks." Proceedings of the IEEE international conference on computer vision. 2017. [7] Berthelot, David, Thomas Schumm, and Luke Metz. "Began: Boundary equilibrium generative adversarial networks." arXiv preprint arXiv:1703.10717 (2017). [8] Martín Arjovsky, Soumith Chintala, and Léon Bottou. Wasserstein Generative Adversarial Networks. International Conference on Machine Learning, ICML, 2017. [9] Frid-Adar, Maayan, et al. "Synthetic data augmentation using GAN for improved liver lesion classification." 2018 IEEE 15th international symposium on biomedical imaging (ISBI 2018). IEEE, 2018. [10] Iqbal, Talha, and Hazrat Ali. "Generative adversarial network for medical images (MI- GAN)." Journal of medical systems 42.11 (2018): 1-11. [11] Shmelkov, Konstantin, Cordelia Schmid, and Karteek Alahari. "How good is my GAN?." Proceedings of the European Conference on Computer Vision (ECCV). 2018. [12] Bowles, Christopher, et al. "Gan augmentation: Augmenting training data using generative adversarial networks." arXiv preprint arXiv:1810.10863 (2018). [13] Zeid Baker, Mousa. "Generation of Synthetic Images with Generative Adversarial Networks." (2018). [14] Wu, Qiufeng, Yiping Chen, and Jun Meng. "DCGAN-based data augmentation for tomato leaf disease identification." IEEE Access 8 (2020): 98716-98728. [15] F. J. Pulgar, A. J. Rivera, F. Charte, and M. J. del Jesus, ‘‘On the impact of imbalanced data in convolutional neural networks performance,’’ in Proc. Int. Conf. Hybrid Artif. Intell. Syst., vol. 10334. Cham, Switzerland: Springer, 2017, pp. 220–232. [16] Visa, Sofia, et al. "Confusion Matrix-based Feature Selection." MAICS 710 (2011): 120-127. [17] M. Mirza and S. Osindero, ‘‘Conditional generative adversarial nets,’’ 2014, arXiv:1411.1784. [Online]. Available: http://arxiv.org/abs/ 1411.1784 [18] Kong, Quan, et al. "Active generative adversarial network for image classification." Proceedings of the AAAI Conference on Artificial Intelligence. Vol. 33. No. 01. 2019. [19] Khalifa, Nour Eldeen M., et al. "Detection of coronavirus (covid-19) associated pneumonia based on generative adversarial networks and a fine-tuned deep transfer learning model using chest x-ray dataset." arXiv preprint arXiv:2004.01184 (2020).