In the framework of ImageClef 2013 [1], plant identification task [2], we have submitted three runs. For the first run named Mica Run 1, we employ GIST descriptor with k-nearest neighbor (kNN) for all sub-categories. Concerning Mica Run 2, we observe that global descriptors such as color histogram and texture are able to distinguish classes of two sub-categories that are flower and entire. For the others sub-categories, we still employ GIST descriptor and kNN. Based on our work for leaf identification, for the third run (named MICA run 3), we have proposed to apply our method for leaf images for both SheetasBackground and Natural background. For the remaining subcategories, we used the same method as the two first runs. Concerning our method for leaf images, we firstly apply Un-sharp Masking (USM) on SheetAsBackground images. Then, we extract Speeded-Up Robust Features (SURF). Finally, we used Bag-of-Words (BoW) for calculating feature vector of each image and Support Vector Machine (SVM) for training the model of each class in training dataset and for predicting class id of new images. In this paper, we describe in detail the algorithms used in our runs.
Results of variety of state of the art scene recognition algorithms [
To capture remarkable/considering of a scene, Oliva et al in [
- 12 δxx22 + y 2 h( x, y) = e δ y2 e- j 2π (u0 x+v0 y )
The parameters (δ x ,δ y ) are the standard deviation of the Gaussian envelope along vertical and horizontal directions; ( u0 , v0 ) refers to spatial central frequency of Gabor filters. The configuration of Gabor filters contains 4 spatial scales and 8 directions. At each scale (δ x ,δ y ), by passing the image I(x,y) through a Gabor filter h(x,y), we obtain all those components in the image that have their energies concentrated near the spatial frequency point ( u0 , v0 ). Therefore, the gist vector is calculated using energy spectrum of 32 responses. We calculated averaging over each grid of 16 x 16 pixels on each response.
Totally, a GIST feature vector is reduced to 512 dimensions. After feature extraction procedure, K-Nearest neighbor (K-NN) classifier is selected for classification. Given a testing image, we found K cases in the training set that have minimum distance between the gist vectors of the input images and those of the training set. A decision of the label of testing image was based on majority vote of the K label found. The fact that no general rule for selected appropriate dissimilarity measures (Minkowsky, Kullback-Leibler, Intersection..). In this work, we select Euclidian distance that is usually realized in the context of image retrieval. In our run, the value of K is 32.
Concerning the second run, for two sub-categories (flower and stem), we apply
global descriptors that are color histogram combining with color moment and texture.
These features are described in [
3. Plant identification method of MICA run 3
For this run, based on the obtained result of our work on leaf identification, we apply our method for leaf images for both SheetasBackground and Natural background. This method is shown in Fig. 1. a. Preprocessing
First of all, segmentation methods are usually used for leaf shape recognition, while our search focuses on local features such as leaf veins and textures. Secondly, while segmentation methods work well with most uniformed background image data, they may not work well with complex background image. Thus, using segmentation methods in our work may constraint the possibility of further system development.
Instead of using segmentation methods for image preprocessing, we decided to apply image normalization and Unsharp Mask (USM) algorithms for grayscaleconverted image. These preprocessing algorithms help enhance the detail of our input image as well as improve system performance.
The preprocessing procedure will be taken in three main steps: • Grayscale conversion: Convert the original image to grayscale image, which is a matrix of pixel intensities.
• Image normalization: Change the range of pixel intensity values in order to enhance the grayscale image.
• Unsharp masking: Sharpen the image local details to help feature extraction more accurate.
b. Grayscale Conversion
Grayscale images (also known as intensity level image) are digital image in which
the value of each pixel is a single sample, that is carries only intensity information.
Grayscale images are commonly called black-and-white image and exclusively
composed of shades of gray, varying from weakest intensity (black) to strongest
intensity (white) [
A common strategy to convert image to grayscale is to match the luminance of the grayscale image to the luminance of the color image. Usually, the luminance component of the grayscale image in the YUV and YIQ models used in PAL and NTSC is calculated by:
R, G and B in the above equation represent Red, Green and Blue channels respectively. The coefficients represent human perception of colors, in particular that humans are more sensitive to green and least sensitive to blue.
c. Image normalization
Image normalization, also known as dynamic range expansion [
The main purpose of dynamic range expansion is to bring the image in to a range that is more familiar to normal sense. Its motivation is often to achieve consistency in dynamic range of images to avoid mental distraction or fatigue. In our research, image normalization algorithm helps balance the intensity level distribution of our image. Thus, normalized image has better contrast which results in a clearer representation of local leaf characteristics.
Normalization transforms an n-dimensional grayscale image:
With intensity values in the new range . The linear normalization of a grayscale digital image is performed according to the formula:
In our research, we desire to have all input images intensity levels to be normalized from 0 to 255. The normalization is applied directly to the grayscale image produced from grayscale conversion process above.
d. Unsharp masking (USM)
Unsharp masking (USM) is a digital image processing technique which sharpens the image to amplify the local details. An unsharp mask cannot create additional detail, but it can greatly enhance the appearance of detail by increasing small-scale acutance.
Digital unsharp masking can be done by combining two images: The original image is called negative image and the blur version of the original image which called positive image.
First of all, we make the positive image by applying Gaussian blur filter to a clone version of the normalized grayscale image produced from the last step. As our original leaf image is a normalized grayscale image, we apply the equation of Gaussian filter in one dimension matrix using the following Gaussian function:
The blurred version of the original is then subtracted away from the original to detect the presence of edges, creating the unsharp mask (effectively a high-pass filter). Contrast is then selectively increased along these edges using this mask leaving behind a sharper final image. An unsharp mask improves sharpness by increasing acutance, although resolution remains the same. In short, there are two steps in unsharp masking: • Step 1: Detect Edges and Create Mask • Step 2: Increase Contrast at Edges The process of unsharp masking is described below: e. Feature Extraction
After processing all input images, the image matrices will then be passed through SURF feature detector and descriptor extractor, which is the local feature extraction approach. We prefer local feature extraction approach over shape feature extraction because it gives us the advantage of recognizing and classifying complex background leaf images. In our research, we take full advantage of the implemented OpenCV methods, which are configurable and bug-free.
In the first step of feature extraction, SURF feature detector received preprocessed image as input. It attempted to detect points of interest (keypoints) over the whole image using Fast Hessian detection algorithm. In the first run of our system, the detector operated as it with no parameterization, meaning that the minimum Hessian threshold, number of octaves and number of octave layers are default. However, in later experiment, we tried to modify the parameters in order to challenge the system at different running conditions.
If no keypoints were found in an input image, the system will give a warning message, ignore the error image and continue the extraction process.
Below figure illustrates the keypoints founded by SURF feature detector. In practice, we expect to have the maximum number of keypoints to be 2000, in order to make our final software runs smoothly on any computer with average memory capacity.
f. Feature Dimensionality Redution
In combination with SURF feature extraction method, we use BOW model to reduce the dimensionality of our computed SURF descriptors. In our research, we use a slightly different version BOW model from OpenCV library which is composed of two main components: BOW k-means trainer and BOW image descriptor extractor.
As discussed above, computed SURF descriptors of each image will be stored in
the machine memory as a descriptor matrix. This matrix will then become input for
BOW k-means trainer in order to cluster the histograms of descriptors which have
similar characteristics into separate visual words. BOW k-means trainer takes the
predefined dictionary size to be parameter K for k-means++ algorithm introduced in [
After clustering the histograms of descriptors into different visual words using
BOW trainer, we saved the clustered dictionary into local machine storage as XML
file. The dictionary produced by BOW trainer is actually a matrix of floating point
number in which the number of columns is the size of SURF descriptor, and the
number of rows is the size of BOW dictionary. Below figure shows a sample
dictionary data.
keypoint. This process involves using Fast Library for Approximate Nearest
Neighbors (FLANN) matcher [
g. Classification
The classification layer of our system takes the feature set from the last layer as its input. In our research, we prefer using Support Vector Machine (SVM) as our supervised learning method. Similar to other supervised learning approaches, depend on the type of input data, SVM has two main functions which are training and testing. OpenCV library has already provided all needed functions for SVM training and testing, thus we made full use of the library for our system benefits.
SVM classifier takes two main parameters for classification: • SVM type: Type of SVM formulation. In our research, we define this parameter as C_SVC or C-Support Vector Classification. It allows imperfect separation of classes for n-class classification with penalty multiplier C for outliers. • Kernel type: Type of SVM kernel. Here we use Radial Basis Function (RBF) kernel for our classification which has the equation ular kernel type for SVM. . This is the most pop
Other support parameters are selected automatically by the system library function which has less impact on the classification performance.
SVM training time depends mostly on the size of the input data. When experimenting with Flavia dataset, we noticed that our training time range from 1 to 2 hours depends mostly on the size of BOW dictionary.
III.
Concerning SheetAsBackground, our third run has obtained the highest score among three runs (0.314) while the first and the second have the low value of score (0.09). This means that the method based on SURF, BOW and SVM is robust for SheetAsBackground category.
With the Natural Background, MICA run 2 has a greater value of score than MICA run 1 and MICA run 3. The main reason is that the global descriptor used in MICA run 2 is effective for the Flower category. The obtained score for Leaf category for MICA run 3 is relatively high. This prove that the method based on SURF, BOW and SVM is robust not only for leaf with SheetAsBackground but also for leaf with natural background.
The results of our runs show that GIST is robust for scene classification but it is not relevant descriptor for plant identification.
Table 1: Obtained results of our runs with natural background