ImageCLEF 2012 has a challenge based on leaf analysis for plant identi cation. This paper reports the method proposed by IFSC/USP team in the participation of this task. We try to explore several attributes (i.e. shape, location and texture) to make a system more accurate. The achieved results are promising and show as a principal outcome the power of texture on leaf analysis.
Plants identi cation has become an important and challenging research area since it is estimated that approximately one half of world plant species is still not cataloged. Among such unidenti ed species one may nd, for instance, the healing of a disease or a plant that can cooperate in the equilibrium of the ecosystem around it. Despite the importance of studies related to the description and categorization of plants, this is still a di cult task for a botanist once this specialist still has a limited amount of information about the vegetal. Furthermore, among the information which may be collected, the most relevant for the botanist analysis are owers and fruits. However, it turns out that in most cases these elements are observed only in speci c periods of the year. This is a complicated issue given that the observation may not be possible when these characteristics are noticeable.
A solution for this impasse is the use of the plant leaf. This structure uses to be observed the whole year and can be collected in a straightforward manner. Nevertheless, most leaves lack more distinguishable attributes for a visual analysis. Thus, image analysis based on computational tools is a worthwhile approach in order to help the botanist or even provide by itself a reliable outcome for the classi cation task.
In this context, ImageCLEF is a world campaign to encourage the development of novel strategies for the description and identi cation of objects, in this case, plant leaves, based on computational/mathematical techniques applied over digital images.
As the group of this work has a signi cative background on computer vision
techniques applied to plant identi cation [
This work is composed by 6 sections, including this introduction. The following section describes brie y the materials and methods employed in the experiments. The following one shows the experiments setup. The fourth section shows obtained results over the training data. The fth one exhibits the results for the test data set while the last section presents the conclusions of the results. 2 2.1
The experiments are performed over Pl@antLeaves dataset [
Each image has an xml le associated that contain the date, type (single leaf, single dead leaf or foliage), name of the author and GPS coordinates of the observation among other data.
The full database is split into training and testing dataset. The train dataset has 8422 images (4870 scans, 1819 scan-like photos, 1733 natural photos) and de test dataset have 3150 images (1760 scans, 907 scan-like photos, 483 natural photos). 2.2
For scan and scan-like photos in both test and train datasets, the Otsu's method
[
Semi-automatic: rst, the photo is automatically segmented by the wellknown mean shift segmentation method. Then, the user roughly indicates the location of the leaf and the background by using some strokes (markers). The regions marked by the user guide the merging process, which gradually labels each non-marker region as either leaf or background based on the histogram similarity.
Automatic: the k-means algorithm is applied to cluster the pixels from the photo into two groups based on the RGB color. Thus, pixels with similar color are in the same group (e.g. green pixels which may be leaves). To decide which group contains the leaf pixels, we calculate the mean RGB value of the central region of the image. The centroid closest to the mean RGB value is chosen as the leaf pixels.
After the segmentation step, we apply a contour detection method to extract the contour of the leaf. We do not bother to treat open or imperfect contours, because the method of shape analysis is robust to such problems. 2.3
In order to explore several aspects of the leaf we use di erent kinds of methods.
Each method returns a feature vector that is used together in the nal classi er.
{ Complex Network: proposed by [
The imageCLEF task allows the submission of 3 di erent approaches trying to recognize the leaves. In this way, we con gure 3 runs as: 1. IFSC USP run1: The training classi er is done with all features and we simply concatenate them in order to obtain a single features vector (237 features). Additionally we used all samples for training (scan, like scan and natural photos). 2. IFSC USP run2: this run is similar to IFSC USP run1 and the main di erence is that only scan and like-scan images are used on training procedure. The reason is try to avoid the in uence of bad-segmented leaves on training set. 3. IFSC USP run3: For this run, we try a fully automated procedure. For natural photos segmentation we use a simple clustering described above. So, we use only Gabor and GPS features, due to the impossibility of measuring shape information of this cluster-segmented images. 4
In order to evaluate the accuracy of methods after submitting the results, we use a 10-fold cross-validation over training data set. The employed classi er is a Linear Discriminant Analysis (LDA), also known as Fisher discriminant.
In a preliminary analysis, we investigate the accuracy of each method in an independent way. The Table 1 shows this result. We notice that VFD provided the best result in this case. This success is justi ed by the richness intrinsic to the texture of leaves, which contain complex patterns with a high potential to describe accurately the plant.
In a subsequent analysis we try to concatenate theses features in order to evaluate if each method explores di erent attributes of leaf data and images. The result is the exact con guration of the run IFSC USP run1 presented above. As expected, we have a very good result showed by Figure 1:
We also comprobe that features which have not presented signi cative success when used alone showed an expressive contribution when concatenated with other features.
Additionally, the Figure 1 shows as recognition rate increases if we consider the classes with more a posteriori probabilities. If we are looking only at the class with more higher probabilities, we get a 80:24% as success rate. As expected, the accuracy increases if we consider a higher number of the possible classes. For example, we reach 98:17% of accuracy if we look for the 9 most likely species and 99:09 with 18 species with higher a posteriori probabilities. 5
We have submitted three runs to the plant identi cation task. The rst two runs are human assisted while the third one is fully automatic. Table 2 presents the classi cation score obtained for each run and each image type (scans, scan-like photos and photos). We can see that the best average result was obtained by the rst run. For the scan and scan-like, the results of the rst run show similar results to the second run. For the photos, however, the result of the rst run was superior to the other two runs. This result demonstrates that it is important to train the classi er using features extracted from the photos. It is worth to mention that the result of the rst run for the photos was the best of all runs including other participants.
Finally, we conclude that using di erent types of features (e.g. shape, texture and GPS) improves the classi cation score in all image types. Moreover, the human assisted segmentation is a good way to improve the discrimination of natural photos.
Run name
No. of descriptors scan scan-like photos Avg IFSC USP run1 IFSC USP run2 IFSC USP run3 Although we do not have the best results among all participants, we made some good ndings about plant identi cation using leaves.
As the main point, we have is the power of texture analysis in leaf discrimination. We employ only simple methods and, besides the lack of standardization of the images, especially images of free natural photos, the texture analysis works ne.
The results of texture methods are best than shapes approaches. In our
previous works [
Otherwise, the synergy from the investigation of various aspects seems to be the most promising ways, with good prospects to make a good system of leaf identi cation with the use of all variables.