After an overview on most relevant methods for image classification, we focus on a recently proposed Multiple Instance Learning (MIL) approach, suitable for image processing applications and based on a mixed integer nonlinear optimization problem. In particular, the algorithm has been preliminarily applied to a set of color images, with the aim to identify images containing some specific color pattern, and successively to a medical dataset, containing photos of melanoma and common nevi. Since the results appear promising, this technique could be at the basis of computer vision systems that act as a filter mechanism to support physicians in detecting melanomas cancer.
Nowadays, continuous development of digital technologies and ICT solutions implies a great availability of digital images in various fields, such as scientific, medical and industrial. Use of this information requires navigation, searching, indexing, classification and retrieval. Image classification is aimed at classifying images according to their visual contents: a particular application is in the medical field, where medical images classification can support the diagnosis of some diseases.
Interesting algorithms to evaluate image similarities have been proposed in [
In this paper, we face image classification by means of a Multiple Instance Learning
approach [
The objective of the Multiple Instance Learning (MIL) is to discriminate among sets of
points: the sets to be classified are called bags, while the points inside the bags are
named instances. In the image classification, an image is represented by a bag, whereas
the sub-regions forming the images are the instances. In a MIL problem, during the
learning phase only the labels of the bags are known, whereas the labels of the instances
are unknown. This is different from a classical supervised classification approach,
where the label of each point is known. In this work, we focus on the binary MIL
classification, whose objective is to discriminate between positive and negative images.
Two different approaches are mainly possible: one is in the instance-space and the other
one is in the bag-space. In the instance-space, the learning process looks only at the
characteristics of each individual instance, without considering the more global
characteristics of the entire bag. Vice-versa, in the bag-space case, the learning phase is
more global, because each bag is treated as a whole entity: the classification consists in
separating directly the positive bags from the negative ones. There exists also a
compromise between these two approaches. It consists in representing each bag by means
of some of its instances containing maximal similarities and it is referred as the
embedding-space approach [
points (the instances) inside the bags.
In this section we describe the heuristics optimization MIL algorithm proposed in [
positive bags and negative ones. The two sets are therefore defined respectively as: + = { 1+, … , + } and − = { 1−, … , −}. Let p be the total number of 1, … ,
and it is equal to -1 for each negative bag − = 1, … , .
The ℎ instance, = (1, … , ) is represented by a vector ∈ ℝ and its class label ∈ {−1,1} is supposed to be unknown. Differently from the class label of the instances, the class label of each bag is known: it is equal to +1 for each positive bag + = We hypothesize to work under the MIL standard assumption: we suppose that a bag is positive if it contains at least a positive instance, while it is negative if it contains only negative instances. The objective is to determine a hyperplane: (, ) ≜
{ ℝ | + = 0 } separating the two classes of bags, where ∈ ℝ is the normal to the hyperplane and ∈ ℝ
is the bias. Since a negative bag is defined containing only negative instances,
only the class label ∈ {−1,1} of the instances inside the positive bags are actually
unknown. If in addition we want to minimize a measure of the classification error of all
the instances inside the bags, the optimization model [
+ )}. { ∈ {−1,1}, =1 ∈ + + ∑
∑ {0, 1 + ( + ∑
∑ {0, 1 −
where
∑
Moreover, if variables were fixed, we would obtain a classical supervised SVM
model [
) () { ∈ {−1,1}, where ≥ 0 is the vector of the Lagrangian multipliers in ℝ . Any optimal solution of (), denoted by ((), (), ()
), provides a lower bound for problem , that is ≤ ∗for any λ ≥ 0. The Lagrangian dual problem is defined as:
{ ≜ ≥0
( )=
∈ {−1,1},
≥0
+ ∈
,,
it is possible to obtain an optimal solution to by using any dual ascent
is nondifferentiable, the use of nonsmooth optimization
with, of course,
solving
method. Since problem
uate the objective function
face problem
techniques is in order. We adopt the subgradient method [
)is fixed, it is possible to solve it, with respect to , by inspection.
In case BCD provides an infeasible solution to the original problem, such solution may
be easily modified to obtain feasibility, giving, in this way, an upper bound (the
incumbent) on the optimal solution of P. A simplified scheme of the MIL algorithm is
reported in Figure 1; further details can be found in [
In the first numerical preliminary experiment (see [
We have performed a segmentation process by means of some image processing standard Matlab routines. In particular, given a bitmap image, this image is read by the “imread” Matlab routine, which provides a 128x128 matrix, the indexed image, each element corresponding to a pixel and containing a triplet which represents the RGB (red, green, blue) scale. In fact, the first value of each triplet represents the red pixel intensity, the second one represents the green pixel intensity and the last one represents the blue pixel intensity. Once the indexed image has been generated, the successive step consists in converting each indexed image (and the corresponding color map) into an RGB image by means of the “ind2rgb” Matlab subroutine. After that, we have proceeded by grouping the pixels in square sub-regions of appropriate dimension: each sub-region forms the so called “blob”. For each blob, we have computed the average of the RGB intensities and the differences between this value and the same quantity calculated for the adjacent blobs (up, down, left, right).
We recall that a crucial issue in a Support Vector Machine type approach is the choice of parameter C, which generally is performed among a grid of possible values. We report the results obtained fixing C = 10, since in correspondence to this value the classification algorithm has exhibited the best performance. In discriminating between positive and negative color images, we have considered the following two cases. In the first one we have generated, for ten times, a testing set by choosing ten different images (five positive and five negative), so that each time the remaining ninety ones (forty-five positive and forty-five negative) have constituted the training set.
In the table reported in Figure 2, for each trial, we report the training and the testing correctness percentage, respectively, with the average for both of them in the last row.
With Yellow
Without Yellow
In the second case, we have performed a leave-one-out cross validation, obtaining
84.07% and 83% as training and testing correctness, respectively. We have noted that,
among the seventeen failures in the testing set, fifteen are positive images and two are
negative images. Considering the fifteen misclassified positive images it is worth
noting that, apart two images, all of them contain a very small yellow colored region.
In the second type of numerical experiment, the MIL classification algorithm proposed
in [
In our experiments, none of the features resulting from the manual analysis was used in the automatic classification process. We have only considered negative the images of common nevi and positive the images of melanomas. The dataset considered consists of 40 images of melanoma and 40 images of common nevi. To contain the number of instances, we first reduced the image resolution to 128x128 pixels and subsequently we have considered square blobs of 32x32 pixel size. We have adopted a be-level 10-fold cross-validation, listing the corresponding results in Figure 3, where we report, for each fold, the testing correctness, the testing sensitivity, the testing specificity and the CPU time. The results are very promising, even because they have been obtained from photos with a small resolution (128x128 pixels): we note in fact that in three cases (folds n. 2, 4 e 7) the algorithm has been able to correctly classify all the images in the testing set. We conclude this section by noting that the healthy image with the green box (see Figure 3), characterized by a lot of hair, is the unique misclassified image belonging to the testing set of fold n. 10. This suggests that better results could be obtained by using preprocessing techniques aimed at eliminating the presence of possible noises.
Melanomas
Common Nevi
In this paper, we have presented an application of a Multiple Instance Learning
approach on color and melanoma images, obtaining promising results, both in terms of
classification accuracy and CPU times. MIL paradigm fits very well on image
classifications because it is able to detect global information (bags) by working at the local
level (instance-level). Future research could consist in designing more sophisticated
segmentation techniques, aiming to improve the classification performance. Moreover,
it would be useful also to test the performance of the proposed algorithm considering
preprocessing steps for the optimization of medical images [