In this work, we proposed a new approach to analyzing eye fundus images that relies upon the use of a convolutional neural network (CNN). The CNN architecture was constructed, followed by network learning on a balanced dataset composed of four classes of images, composed of thick and thin blood vessels, healthy areas, and exudate areas. The learning was conducted on 12x12 images because an experimental study showed them to be optimal for the purpose. The test error was no higher than 4% for all sizes of the samples. Segmentation of eye fundus images was performed using the CNN. Considering that exudates are a primary target of laser coagulation surgery, the segmentation error was calculated on the exudate class, amounting to 5%. In the course of this research, the HSL color system was found to be most informative, using which the segmentation error was reduced to 3%.
A severe consequence of diabetic retinopathy (DRP) is vision loss. DRP affects all parts of the retina,
leading to macular edema, which in turn causes fast worsening of eyesight [
Convolutional neural networks are the choice of preference when dealing with object classification
[
In Ref. [
In this work, we study a class of eye fundus images with pathological changes that can be found at different stages of the disease. Manifestations of the diabetic retinopathy include exudates, which cause the retina thickening (Figure 1). In general, the image of an eye fundus with pathology contains four classes of objects, such as thick and thin blood vessels, healthy areas and exudate zones.
The initial data for analysis contained 11 training datasets of various size. All datasets were balanced and in total contained 534 images. For the purposes of the present work, the CNN training was conducted on four above-described classes of eye fundus images. The initial dataset consisted of 75 percent of training images and 25 percent of test images. To prevent overtraining, a control dataset was also used. A 3x3 convolution kernel was chosen because it is optimal for 12x12 images. The CNN architecture was constructed empirically so as the required accuracy of no less than 96 % is ensured. Table 1 gives architecture of the empirically constructed convolutional neural network. With this architecture, a recognition accuracy of 99.3% was attained, which is the best recognition result for the four above-mentioned classes of images. Figure 2 depicts a plot for training the CNN model in each training epoch.
Layer number 1 1 2 2 2 2 3
Layers Convolutional Activation Convolutional Activation Dropout MaxPooling Convolutional
To attain a recognition certainty of 95 %, the CNN was put through 120 training runs on the initial images of all sizes. Figure 3 shows an average training result for each image size.
The results in Figure 2 show that the highest classification accuracy is attained for 12x12 images.
For the experiments, datasets were formed containing four above-described classes of 12x12 images, using which the best result of CNN testing is achieved (Figure 3). In this study, the segmentation of eye fundus images was conducted via deep learning. Shown in Fig. 4b is the result of CNN-aided image segmentation. With a view of estimating the CNN-aided segmentation error, a manual segmentation by an expert ophthalmologist was introduced as a reference image (Figure 4c). The study was conducted on the exudates class, which had been singled out into a separate image (Figure 4d). The error of CNN-aided segmentation of the said exudate areas was calculated relative to the expert estimate. The result of comparison of the exudation areas highlighted by CNN and the expert is shown in Table 2. Using the data from Table 2, a CNN-aided segmentation error for the exudates was defined as E = ( k + t ) NM and amounted to 7% (where N×M is the image size, k is the number of expert-highlighted pixels that CNN failed to recognize as exudates, t is the number of exudate pixels recognized by CNN but missing from the expert's image). The error of first kind, defined as E1 = l F , where l is the number of falsely recognized exudates classes and F is the total number of exudatecontaining pixels in the expert's image, amounted to 5%.
In the process of exudates area identification, color plays a key role. The segmentation error can be
significantly reduced by operating in particular color spaces. It has been established [
Areas Exudates area in the expert's image Exudates area in the CNN-aided image Total exudates area Expert's exudates areas omitted by CNN CNN-highlighted exudates areas missing in the expert's image
Veracity of CNN-aided exudate highlighting has been confirmed by comparison of histograms of CNN-aided and expert's images (Figure 4), which were superimposed for each corresponding channel of HSL color system, with the expert-based histograms marked as green bars, and the CNN-based histograms marked red (Figure 6). The expert-based histograms define an interval of values for the affected fundus areas. From the histograms, the CNN-aided interval of exudates area is seen to be narrower than that obtained based on expert's estimates. The histogram regions corresponding to the false CNN-aided classification are within intervals shown by rectangles (Figure 6). Table 3 gives segmentation errors calculated for each channel of the НSL color model. The data in Table 3 suggest that the H channel is the most informative channel with the least segmentation error. d) e) f) Figure 5. Expert-highlighted affected fundus areas for the HSL color model for the channel (a) H, (b) S, and (c) L; CNN-highlighted affected fundus areas for the HSL color model for the channel (d) H, (e) S, and (f) L.
Channel Segmentation error for the exudates class, % c) Figure 6. Image histograms obtained by using an expert opinion and CNN: а) Н channel, b) S channel, and c) L channel.
In this work, a convolutional neural network (CNN) has been applied to the analysis of an eye fundus image. CNN architecture has been constructed, allowing a testing error of no more than 4% to be attained. Based on a 3x3 convolution kernel, CNN training was conducted on 12x12 images, thus enabling the best result of CNN testing to be achieved. CNN-aided segmentation of the input image conducted in this work has shown the CNN to be capable of identifying all training dataset classes with high accuracy. The segmentation error was calculated on the exudates class, which is key for laser coagulation surgery. The segmentation error on the exudates class was 7 %, with the error of first kind being 5 %.
In the study, we utilized the HSL color model because it renders color characteristics of eye blood vessels and exudates most adequately. We have demonstrated the H channel to be most informative, with the segmentation error amounting to 3 %.
This work was financially supported by the Russian Foundation for Basic Research under grant # 1929-01135, # 17-01-00972 and by the Ministry of Science and Higher Education within the State assignment to the FSRC “Crystallography and Photonics” RAS.