Diabetic Retinopathy (DR) is a complication of the diabetes, caused by a damage to the blood vessels in the light-sensitive tissue of the retina: consequently it affects the eyes, determining blindness and visual impairment. On the basis of the last report of the International Diabetes Federation, 537 million adults (20-79 years) were living with diabetes (1 in 10) in 2021, being the cause of 6.7 million deaths (1 every 5 seconds). Taking into account that the number of people affected by DR is predicted to increase to 643 million by 2030 and 700 million by 2045, it is clear that effective screening of potential DR patients is of utmost importance. While direct and indirect ophthalmoscopy are the main methods for evaluating DR, artificial intelligence is on the rise in vision care. DR is detectable by analyzing data from patients' fundus photographs, and is therefore a disease that artificial intelligence tools can effectively support. In this paper, we present some numerical results obtained in the classification between eye fundi of healthy individuals and of people with severe DR, using a Multiple Instance Learning approach.
Diabetes is a chronic disease related to the production of insulin, a hormone that regulates the glucose levels in the blood. Unfortunately it is on the rise worldwide, as confirmed by the last International Diabetes Federation (IDF) report [1], showing that the number of diabetes cases has significantly increased in the recent decades, with projections indicating that there could be 600 million people with diabetes by 2035 and 700 million by 2045.
A common and serious complication of the diabetes is the Diabetic Retinopathy (DR), which approximately affects 30% of diabetic patients, being the primary cause of blindness among people aged 20-74. DR occurs when the blood vessels in the retina, the light-sensitive tissue at the back of the eye, are damaged due to diabetes. At the time of their first diabetes diagnosis, up to 21% of patients also present retinopathy. In many cases, people with DR may not notice symptoms until significant damage has already occurred. Therefore, it is straightforward that an early diagnosis of DR is critical, as it leads to a better response to the treatment and to a more positive prognosis. Screening can be carried out through fundus examination or retinography, with the latter being more commonly recommended and forming the basis of current DR screening protocols. All diabetes and ophthalmology associations currently recommend annual retinography screening for all diabetic patients from the first diagnosis. Therefore, there is a need to develop tools that allow for early and accurate diagnosis, which can assist specialists and enable the release of low-cost solutions for effective self-diagnosis.
While direct and indirect ophthalmoscopy is the primary method for evaluating DR, Machine Learning (ML) and Artificial Intelligence (AI) have become increasingly popular in the eye care sector: they utilize advanced algorithms to face the screening of specific pathologies by analyzing vast amounts of clinical data, with the ultimate goal of achieving tasks with minimal human involvement. Different solutions for data and image analysis, based on ML and AI, have been provided in the literature: they are extensively used as medical information tools for data prediction and diagnostic procedures [2], as well as in applications for specific lesion characterization and feature detection [3].
In this paper we focus on the DR detection by image analysis of the eye fundus, using a Multiple Instance Learning (MIL) technique and focusing on the binary classification case, aimed at discriminating patients with severe DR from those ones without DR.
The paper is organized as follows. In the next section, we highlight which are the visual features that are indicative of the presence of DR pathology. In Section 3 we describe the MIL paradigm and the related technique used for our experiments, which are presented in Section 4. Finally, in Section 5 some conclusions are drawn.
Micro blood vessels continuously supply blood to the retina, which maintains the blood sugar level. However, high levels of sugar in the blood can cause the tiny blood vessels that nourish the retina to become blocked, cutting off its blood supply. In response, the eye attempts to grow additional blood vessels that do not develop properly and can easily leak.
The most common scale used by clinicians to grade DR is the International Clinical Diabetic Retinopathy (ICDR) scale, which has vfie grades: no DR, mild DR, moderate DR, severe DR, and proliferative DR. Each stage of the disease is associated with specific features. These features include microaneurysms, exudates, hemorrhages, abnormal blood vessels, and neovascularization. The presence or occurrence of one, two, or all of these characteristics in the retina determines the stage of DR [4].
In the early stages of diabetic retinopathy, patients may not experience any noticeable symptoms. In fact, the number of undiagnosed diabetic cases is in general high. However, as the condition progresses, patients may begin to experience a range of symptoms, including distorted or blurred and fluctuating vision, dark or empty areas in their vision, spots or dark strings that float in their vision (known as floaters), and a progressive loss of visual acuity.
Figure 1 a) depicts a fundus photograph of a person’s left eye with mild (early) nonproliferative diabetic retinopathy and centre-involving diabetic macular edema (DME). The image highlights small red "dots" (microaneurysms) and "blots" (hemorrhages) with typical examples marked by white arrowheads.
Figure 1 b) is a diagrammatic representation of an advanced proliferative diabetic retinopathy with ischemic maculopathy. The image shows laser scars resulting from panretinal laser photocoagulation, including areas of focal grid laser treatment within the macula.
The standard features together with the corresponding DR type are reported in Table 1.
small microaneurysms hemorrhages, exudates, microaneurysms retinal ischemias and abnormal features abnormal blood vessels, all abnormal features
Mild NPDR Moderate NPDR
Severe NPDR
PDR
For individuals with uncontrolled diabetes, it is recommended to undergo annual fundus screening to facilitate early identification and treatment of diabetic retinopathy, which can help to decrease the disease burden in the community. There has been an increased interest in using completely automated AI-based grading systems for grading pictures. Such technology would enable real-time determination of whether a patient requires referral, and it may be much less expensive compared to having ophthalmologists conduct the screening.
In April 2018, the US Food and Drug Administration (FDA) certified an AI algorithm created by IDX Systems Corporation (IDX) 1 for use with the Topcon Fundus camera to detect diabetic retinopathy. Several studies have been conducted to develop AI-based algorithms for detecting DR, including models based on microaneurysms and hemorrhages identification by Wong et al. [5] and an approach for detecting exudation and blood vessels using morphological component analysis by Imani et al. [6]. Adaptive thresholding is used to create the vessel map. Using inverse surface thresholding, Yazid et al. [7] were able to identify hard exudation and the optic disc.
Ting et al. [8] conducted a significant research in Singapore using numerous retinal pictures to validate Deep Learning (DL) and found that it has a high sensitivity and specificity in detecting 1IDX was a healthcare software technology company founded in 1969 acquired by General Electric and incorporated into its GE Healthcare business unit in 2006.
DR and other eye disorders. However, the problem of the interpretability of the results by specialists arises. The physicians express reservations about solutions that work like a “black box” and whose diagnoses are not based on the features arising from the medical protocols of reference. Interpretability is not just a matter of intellectual curiosity, but is a critical factor due to the significant impact medical choices have on patients’ lives, and the associated risks and responsibilities that clinicians bear [9].
The way is paving for the adoption of hybrid frameworks, which use DL algorithms for the preprocessing [10] and segmentation steps of the medical images [11, 12] and then adopt advanced ML algorithms and heuristics [13, 14, 15] for the classification steps. The adoptable features can therefore also be defined by specialists and not just already extracted from a system that works like a “black box”. The classification performances can also give important results using ensemble learning configurations [ 16, 17]. In this sense, what is presented in this work is meant to be functional for the creation of hybrid solutions.
Multiple Instance Learning (MIL) [18] is a classification approach aimed at discriminating among sets of points. In the MIL terminology, the sets are called bags and the points inside them are called instances. The main characterization of a MIL approach with respect to a classical supervised classification technique resides in the fact that the learning phase is performed by exploiting only the class labels of the entire bags, since the class label of each single instance is unknown.
MIL plays a relevant role in medical image and video analysis (see [19]) and especially in diagnostics by images analysis, which is a fundamental field supporting physicians to have early diagnoses.
The most common MIL problem treated in the literature is the binary case (i.e. two classes of bags), where it is possible to have also more than two classes of instances. For such problems, there are mainly three types of approaches (see for example [20, 21]). The first one is the bag-level approach, where each bag is treated as a global entity, while the second one is the instance-level approach (see for example [22, 23, 24, 25, 26]) where the classification is performed in the instance space, obtaining the bag label as aggregation of the labels of the corresponding instances. The third type of approach (see for example [27]) is a compromise between the two previous ones and consists in representing each or some bag by one of its instances, involved successively in the classification process.
The MIL technique adopted in this work for DR by means of image detection is the mi-SPSVM (multiple instance - Semi-Proximal Support Vector Machine) algorithm [23, 24], which is an instance-level approach solving binary MIL problems characterized by two classes of instances. In the literature, such kind of MIL problems are based on the so-called standard MIL assumption, stating that a bag is considered positive if it contains at least a positive instance and it is negative otherwise.
Algorithm mi-SPSVM has been introduced in [23] and it exploits the good properties exhibited for supervised classification by the SVM (Support Vector Machine) technique in terms of accuracy and by the PSVM (Proximal Support Vector Machine) approach [28] in terms of efficiency. It computes a separating hyperplane
(, ) =△ { ∈ R | + = 0}, solving, at each iteration, the following quadratic optimization problem: ⎧ ⎪⎪ min ⎪ ,, ⎪ ⎪ ⎪ ⎪ ⎪ ⎪ ⎪ ⎪ ⎨ ⎪ ⎪ ⎪ ⎪ ⎪ ⎪ ⎪ ⎪ ⎪ ⎪ ⎪ ⎩ 1 ⃦⃦ ⃦⃦ 2 2 ⃦⃦ ⃦⃦ + ∑︁ 2 + ∑︁ 2 ∈+
∈−
= 1 − ( + ) ∈ +
≥ 1 + ( + ) ∈ −
≥ 0 ∈ − ,
(
by varying the sets + and − , which contain the indexes of the instances currently considered
positive and negative, respectively. The positive parameter tunes the weight between the
maximization of the margin, obtained by minimizing the Euclidean norm of (, ), and the
minimization of the misclassification errors of the instances s. At the initialization step, +
contains the indexes of all the instances of the positive bags, while − contains the indexes of all
the instances of the negative bags. Once an optimal solution, say (* , * , * ), to problem (
− := − ∪ ¯ ¯ = { ∈ + ∖ * | * + * ≤ −
1}, * = {* , = 1, . . . , | * * + * ≤ −
1} * =△ arg ∈(m+a∩x+){* + * }
Some comments on the updating of the sets + and − are in order. A particular role in the
definition of the set ¯ is played by the set * , introduced for taking into account the satisfaction
of the standard MIL assumption, stating that, for each positive bag, at least one instance must
be positive. At the current iteration, the set * is the index set (subset of +) corresponding
to the instances closest, for each positive bag, to the current hyperplane (* , * ) and strictly
lying in the negative side with respect to it. If an index, say * ∈ * , corresponding to one of
such instances entered the set − , all the instances of the -th positive bag would be considered
negative by problem (
Algorithm mi-SPSVM, summarized in the previous section, has been tested on a random sample of eye fundus images drawn from the public dataset Messidor, available at the web page https://www.adcis.net/ en/third-party/messidor/. It has been run on a Windows 11 system with a 2.21 GHz processor, using the same Matlab implementation adopted in [29, 30].
The Messidor database [31] consists of 1200 color images of the back of the eye (the posterior pole), taken by three ophthalmology departments using a color video camera mounted on a Topcon TRC NW6 retinograph. In correspondence to each image, two types of diagnoses have been provided by medical experts: a grade for retinopathy and a risk assessment for macular edema.
In our numerical experiments, we have performed the classification of eye fundus of people affected by severe diabetic retinopathy (positive bags, Figure 2) against those ones of healthy people (negative bags, Figure 3).
Regarding the segmentation process, we have used a procedure similar to that one described in [29, 30]. The resolution of each image has been reduced to 128 × 128 pixels, and the pixels have been grouped into square subregions, called "blobs." This has allowed us to represent each image as a bag, where each blob corresponds to an instance of the bag. For each instance (blob), we have considered the average and the variance of the RGB (red, green, blue) intensities of the blob extracted from the eye and the differences between the average and the variance of the RGB and the same corresponding quantities computed for the adjacent blobs (up, down, left, right).
Since we have not considered the blobs along the frame of each image, we have come out with a data set where each image is a bag characterized by sixteen instances and 30 features for each instance.
In order to consider in our tests different sizes of the testing and training sets, we have used three
different validation protocols: the 5-fold cross-validation (5-CV), the 10-fold cross-validation
(10-CV) and the Leave-One-Out (LOO) validation. As for the optimal computation of the tuning
parameter characterizing model (
The average testing results we have obtained (in terms of accuracy, sensitivity, specificity, F-score and CPU time) are reported in Tables 2, 3 and 4, comparing against Algorithm MIL-RL [14], which is another MIL technique preliminarily used in [33] for diabetic retinopathy images classification. For the sake of completeness, we also report the results obtained by running the standard SVM technique, for which we have used the linear and the Gaussian kernels (columns SVM and SVM-RBF, respectively). For each table and for each type of performance metric, the best results are reported in bold.
Tables 2, 3 and 4, show that algorithm mi-SPSVM exhibits the best performance not only in terms of accuracy, but also in terms of sensitivity and F-score, which are the most significant parameters in medical diagnosis since they express the capability of the classifier to correctly individuate positive patients. Moreover, in terms of CPU time, even if the winner appears the SVM with RBF kernel, it also evident that mi-SPSVM is much faster than MIL-RL.
In this work, we have presented some numerical results obtained from the classification of severe diabetic retinopathy fundus images against normal ones images, by means of a MIL algorithm. These results appear promising, especially if we consider that no preprocessing phase has been performed. Moreover, the adopted MIL technique appears appealing also in terms of computational efcfiiency since the separation hyperplane is always obtained in less than one second.
Future research could consist in appropriately preprocessing the images and in considering additional features to be exploited in the classification process. The obtained classification performances can also give important results using ensemble learning configurations. In this sense, what is presented in this work is preliminary to the creation of hybrid solutions, based on the use of DL approaches for the preprocessing and segmentation steps of fundus images and on the adoption of advanced ML algorithms for the classification phase. Image-based diagnostic approaches are cross-cutting and lend themselves too future fields of application like the ones related to the prediction of pathologies for plants [34] or oriented towards specific tasks on forecasting non-renewable energy production [35]. [5] W. Yun, U. Acharya, Y. Venkatesh, C. Chee, L. Min, E. Ng, Identification of different stages of diabetic retinopathy using retinal optical images, Information sciences 178 (2008) 106–121. [6] E. Imani, H.-R. Pourreza, T. Banaee, Fully automated diabetic retinopathy screening using morphological component analysis, Computerized medical imaging and graphics 43 (2015) 78–88. [7] H. Yazid, H. Arof, H. Isa, Automated identification of exudates and optic disc based on inverse surface thresholding, Journal of medical systems 36 (2012) 1997–2004. [8] D. Ting, C. Y.-L. Cheung, G. Lim, G. Tan, N. Quang, A. Gan, H. Hamzah, R. Garcia-Franco, I. San Yeo, S. Lee, et al., Development and validation of a deep learning system for diabetic retinopathy and related eye diseases using retinal images from multiethnic populations with diabetes, Jama 318 (2017) 2211–2223. [9] L. Caroprese, E. Vocaturo, E. Zumpano, Argumentation approaches for explanaible ai in medical informatics, Intelligent Systems with Applications 16 (2022) 200109. [10] G. Rani, A. Misra, V. Dhaka, E. Zumpano, E. Vocaturo, Spatial feature and resolution maximization GAN for bone suppression in chest radiographs, Comput. Methods Programs Biomed. 224 (2022) 107024. [11] G. Rani, A. Misra, V. Dhaka, D. Buddhi, R. Sharma, E. Zumpano, E. Vocaturo, A multimodal bone suppression, lung segmentation, and classification approach for accurate covid19 detection using chest radiographs, Intelligent Systems with Applications 16 (2022) 200148. [12] G. Rani, P. Thakkar, A. Verma, V. Mehta, R. Chavan, V. Dhaka, R. Sharma, E. Vocaturo, E. Zumpano, Kub-unet: Segmentation of organs of urinary system from a KUB x-ray image, Comput. Methods Programs Biomed. 224 (2022) 107031. [13] A. Fuduli, M. Gaudioso, W. Khalaf, E. Vocaturo, A heuristic approach for multiple instance learning by linear separation, Soft Comput. 26 (2022) 3361–3368. [14] A. Astorino, A. Fuduli, M. Gaudioso, A Lagrangian relaxation approach for binary multiple instance classification, IEEE Transactions on Neural Networks and Learning Systems 30 (2019) 2662 – 2671. [15] E. Vocaturo, E. Zumpano, G. Giallombardo, G. Miglionico, DC-SMIL: a multiple instance learning solution via spherical separation for automated detection of displastyc nevi, in: B. C. Desai, W. Cho (Eds.), IDEAS 2020: 24th International Database Engineering & Applications Symposium, Seoul, Republic of Korea, August 12-14, 2020, ACM, 2020, pp. 4:1–4:9. [16] X. Dong, Z. Yu, W. Cao, Y. Shi, Q. Ma, A survey on ensemble learning, Frontiers of
Computer Science 14 (2020) 241–258. [17] Z.-H. Zhou, Ensemble learning, Springer, 2021. [18] F. Herrera, S. Ventura, R. Bello, C. Cornelis, A. Zafra, D. Sánchez-Tarragó, S. Vluymans, Multiple instance learning: Foundations and algorithms, Springer International Publishing, 2016. [19] G. Quellec, G. Cazuguel, B. Cochener, M. Lamard, Multiple-instance learning for medical image and video analysis, IEEE Reviews in Biomedical Engineering 10 (2017) 213–234. [20] J. Amores, Multiple instance classification: Review, taxonomy and comparative study,
Artificial Intelligence 201 (2013) 81–105.