This paper describes methods and results archived by our research group at the Cross Language Evaluation Forum (CLEF'2009). In our work we concentrated on the medical image retrieval task. All the attention was given to the retrieval based on nine visual topics only. No textual information was considered. Our goal was to develop and comparatively assess image descriptors for content based retrieval on the large medical image database. In addition to results of official relevance judgment, time- and computational efficiency of the algorithms has also been of our interest. An approach based on generalized co-occurrence matrices was used. The following elementary image characteristics have been utilized: signal intensity, absolute value of the intensity gradient and the angle between the gradient vectors in the pixel pairs.
2.1
Generalized co-occurrence matrices [
As both variables take continuous values, the construction of such a histogram will require their quantization. Depending on the range of relative distances considered and the number of bins used, this step could be quite computationally intensive. However, it is not necessary to consider a large range of distances. Depending on the task we want to perform, we may restrict ourselves to only a small range of. If this distance value is small, the histogram will contain local information concerning the described shape or texture. If the value is large, global information is conveyed.
Taking into account the extensive experience on the medical image analysis, classification and recognition using these descriptors, the following elementary image characteristics have been utilized: • brightness (signal intensity); • absolute value of the intensity gradient; • the angle between the gradient vectors in the pixel pairs.
As it was shown experimentally [
A multidimensional co-occurrence matrix of a general form that combines all of above characteristics and relationships looks as follows:
W = kw (I(i), I(k), G(i), G(k), A(i, k), d(i, k))k , where i = (xi, yi), k = (xk, yk) is an arbitrarily pair of pixels, d(i, k) is the distance between them, I(i), I(k) is the intensity of the pixels, G(i), G(k) is the absolute values of brightness gradients,
A(i, k) is the angle between the directions of the gradients.
The color images were converted into gray images using:
I = 0.3 × R + 0.59 × G + 0.11 × B 2.2
The histograms/co-occurrence matrices that represent different images have to be compared and their difference be expressed by a single number if possible. This can be achieved with the help of a metric defined to measure the ”distance” between two histograms.
As a similarity measure of the classes (topics), the following distances were tested: • canberra metrics • manhattan metrics • and absolute value of correlation coefficient n dcanberra(X, Y ) = X |xi − yi| , i=1 xi + yi
n dmanhattan(X, Y ) = X |xi − yi|
i=1 dcorrelation(X, Y ) =
E(XY ) − E(X)E(Y ) pE(X2) − E2(X) pE(Y 2) − E2(Y ) where E is the expected value operator, X = (x1, x2, . . . , xn) and Y = (y1, y2, . . . , yn) are feature vectors.
Some visual topics could contain several images as a query, so we tried two types of distance measures to the group of images: dcentroid(X, Y ) = mean(xi)
dmin(X, Y ) = min(xi). and minimum distances
Additionally, color control was used during the retrieval: if the query contained color/gray images, corresponding images were moved at the top of the results. 3
Initially, due to descriptors’ normalization, original images were scaled down: • proportional rescaling to 320 pixels width; • rescaling to 150 × 150 pixels; • rescaling to 300 × 300 pixels.
After that different types of descriptors were calculated and stored into binary files (Table 1). In our experiments we have chosen 5–7 bins for every dimension of the co-occurrence matrix. Average time for generation one image descriptor was about half of a second.
Various types of descriptors and metrics in combination with image rescaling, produced 42 runs.
In our opinion, the best retrieval results on such a common database were achieved with IIGGAD descriptors and minimum distance on manhattan metrics (Fig. 2). Thus only two runs were submitted (Table 2)
As it was our first experience at this competition, we erroneously did not run mixed and semantic topics. Thus, only 9 out of 25 topics were submitted. According to a general number of relevant results our two runs can by ranked 3rd and 6th respectively out of 13 visual runs.
Generalized co-occurrence matrices are intended to find visually similar images. As some topics can by semantically the same but visually absolutely different, employment of textual information has more potential. Despite all this, generalized co-occurrence matrices showed very promising results as similarity measures of medical images.
This work was partially supported by the ISTC grant B-1682.