=Paper=
{{Paper
|id=Vol-1498/HAICTA_2015_paper28
|storemode=property
|title=Plant Disease Diagnosis Based on Image Processing, Appropriate for Mobile Phone Implementation
|pdfUrl=https://ceur-ws.org/Vol-1498/HAICTA_2015_paper28.pdf
|volume=Vol-1498
|dblpUrl=https://dblp.org/rec/conf/haicta/Petrellis15
}}
==Plant Disease Diagnosis Based on Image Processing, Appropriate for Mobile Phone Implementation==
https://ceur-ws.org/Vol-1498/HAICTA_2015_paper28.pdf
Plant Disease Diagnosis Based on Image Processing,
Appropriate for Mobile Phone Implementation
Nikos Petrellis1
1
Department of Computer Science and Engineering, Technology Educational Institute of
Thessaly, Greece, e-mail: npetrellis@teilar.gr
Abstract. A steady plant monitoring is necessary to control the spread of a
disease but its cost may be high and as a result, the producers often skip critical
preventive procedures to keep the production cost low. Although, official
disease recognition is a responsibility of professional agriculturists, low cost
observation and computational assisted diagnosis can effectively help in the
recognition of a plant disease in its early stages. The most important symptoms
of a disease such as lesions in the leaves, fruits, stems, etc, are visible. The
features (color, area, number of spots) of these lesions can form significant
decision criteria supplemented by other more expensive molecular analyses
and tests that can follow. An image processing technique capable of
recognizing the plant lesion features is described in this paper. The low
complexity of this technique can allow its implementation on mobile phones.
The achieved accuracy is higher than 90% according to the experimental
results.
Keywords: plant disease, lesions, image processing, agricultural production.
1 Introduction
Plant diseases can increase the cost of agricultural production and may extend to total
economic disaster of a producer if not cured appropriately at early stages. The
producers need to monitor their crops and detect the first symptoms in order to
prevent the spread of a plant disease, with low cost and save the major part of the
production. Hiring professional agriculturists may not be affordable especially in
remote isolated geographic regions. Machine vision can offer an alternative solution
in plant monitoring and such an approach may anyway be controlled by a
professional to offer his services with lower cost. Of course, there are several
additional tests that have to be performed in order to confirm a specific disease but
image processing can give a first clue on what really happens at the field.
Before focusing on the existing image processing techniques, the features of
molecular tests are reviewed (Sankaran et al, 2010). Molecular test sensitivity
depends on the minimum amount of microorganism that can be detected. For
example, bacteria detection can range from 10 to 106 colony forming units per mL
(Lopez et al, 2003). A popular molecular diagnosis method is the ELISA that is
238
based on the use of a microbial protein associated with the plant disease. This protein
is injected into an animal that produces antibodies that are extracted and used for
antigen detection with fluorescence dyes and enzymes. PCR is another popular
technique based on DNA analysis (Shaad and Frederick, 2002). Molecular tests
require expensive equipment and samples may need to be transported to the premises
where the tests can be performed, although portable low cost equipment has been
recently presented capable of performing tests like PCR (Spathis et al, 2014).
The spectroscopic and imaging are non-destructive low cost techniques that can be
used for plant disease diagnosis based on its symptoms. Spectroscopic techniques
can also identify water stress levels and nutrient deficiency, measure the fruit quality
after the harvest, etc. Spectroscopic techniques include fluorescence or multispectral
imaging (Chaerle et al., 2007), infrared spectroscopy (Purcell et al, 2009), etc.
Reviews of image processing techniques in visual light for plant disease detection
can be found in (Barbedo, 2013), (Camargo and Smith, 2009) and (Kulkarni and
Patil, 2012). In (Kulkarni and Patil, 2012) an image segmentation takes place in the
CIE L*a*b color scale, then a Gabor filter is used to generate the input of a neural
network that achieves a disease recognition with a 91% accuracy. Other classification
techniques take into consideration the shape, the texture, fractal dimensions,
lacunarity, dispersion, grey levels, grey histogram discrimination and the Fourier
descriptor.
Most of the image processing and spectroscopic techniques require the analysis to
be performed by specialized equipment and software packages. In this paper, we
focus on a low complexity image processing technique that can be implemented and
installed on a mobile phone. The image processing technique described here, is
developed in the framework of a plant disease recognition system that is under
development. This system operates in multiple levels ranging from a single
standalone mobile phone, to a mobile phone communicating with a cloud or database
and cooperating with the portable DNA analysis equipment for complementary PCR-
like tests (Spathis et al, 2014).
Although the color features are also important in the process of plant disease
recognition we focus on three parameters of the lesions that can appear at the leaves,
the stem, or the fruit of a plant: (a) the number of spots, (b) their area and (c) their
gray level. The measurement of these three features can give a first indication on the
condition of the plant. The proposed system called henceforth Spot Recognition
System (SRS) can be easily extended to generate the Red-Green-Blue (RGB)
features of the spots or their CIE L*a*b color scale as will be described in the
following sections although this feature is not experimentally tested in the present
work.
Having installed the software implementation of the described image processing
technique on a mobile phone, the producer would be able to take pictures of plant
parts with lesions, immediately analyze the photos and take any further action needed
to confirm the potential disease and apply the recommended therapy.
In this paper we apply the proposed technique to tangerine tree leaves with lesions
and measure the accuracy in the spot feature recognition. Such measurements could
have been used to discriminate between sooty mold (fungal growth), citrus canker,
scab, etc that can have affected a citrus tree. Experimental results show that the
measurement of the number of spots, their gray level and area can be achieved with
239
higher than 90% accuracy. The proposed technique is a low complexity algorithm
that does not rely on expensive or complicated image processing tools and thus can
be easily implemented in Java or C to create an appropriate mobile phone
application. The results of the proposed image processing technique can be used for
example by a neural network or by a more deterministic rule-based decision system.
The plant disease recognition framework where the proposed image processing
technique has been developed is described in Section 2. The implementation details
of the SRS are given in Section 3 and experimental results are presented in Section 4.
2 Plant disease recognition framework
Cellular network internet
Plant part
under analysis ca Mobile phone
with plant
mera disease rec.
application Antenna
USB, IrDA,
Bluetooth
Direct
connection Cloud or Plant
disease database
Sensor Controller
(for molecular tests
like PCR)
Fig. 1. The plant disease recognition framework under development.
The plant disease recognition framework under development is shown in Fig. 1. Its
basic functionality operates on a single mobile phone equipped with a color camera
with reasonable resolution. The mobile phone should be capable of connecting to the
internet if a more detailed disease database or cloud has to be accessed. This may be
required if the number of plants/diseases that can be examined is too high and the
recognition rules, patterns and other data cannot be stored locally on the phone.
Moreover, the storage of the pictures taken by the phone and the analysis results to
the remote cloud or database can make easier their access by professional
agriculturists.
The producer can use his phone with the installed plant disease recognition
application when he wants to check the condition of the plants in his crops. He can
take pictures of plant parts with lesions (e.g., leaves, stem, fruit) and run the plant
disease recognition application on the pictures taken. The SRS of the plant disease
recognition application extracts the lesion features like number of spots, grey level,
area and these results are used by the decision module of the application that will
extract a conclusion on the condition of the plant. As already mentioned the decision
240
module can be a neural network. If a more advanced decision module has to be used
operating on a diversity of disease recognition rules and data, the output of the SRS
can be simply sent through GPRS or the internet to an external database or cloud. If
the phone is moved close to the computer where this external cloud or database is
installed then, the SRS results can be transferred through a different communication
method like Bluetooth, WiFi, IrDA, etc. The plant disease recognition application
may also need additional information that can be retrieved from the telephone itself
like for example temperature and moisture conditions (e.g., these can be retrieved by
the internet after the localization of the user geographical position). Additional
statistical information can be given by the user through a questionnaire shown to him
by the application.
In a more advanced setup, the mobile phone can cooperate with a DNA analysis
module like the biosensor readout circuit described in (Spathis et al, 2014) that has
been developed for the Corallia/LabOnChip project. The communication with this
module can be performed in a wired or wireless manner (USB, WiFi, Bluetooth, etc).
3 The developed Spot Recognition System
Mobile
phone Camera
L
D φ
L
Fig. 2. Estimation of the leaf dimensions.
The Spot Recognition System (SRS) assumes that the picture of the plant part has
been captured from as known distance D as shown in Fig. 2. The awareness of the
distance D can be guaranteed if the user is ordered to take the picture from a roughly
known distance e.g., related to his hand or to adjust the distance so that the plant part
that will be captured fits the photograph. If the camera angle φ is also known then,
the half leaf length is estimated by:
L=D•tan(φ). (1)
241
If the length L is known and it corresponds to P pixels then, the constant S can be
used to estimate a distance from any other number of pixels:
L (2)
S=
P
For example, if two points in the horizontal axis, correspond to a real distance L’
and the number of intermediate pixels is P’ their distance L’ is
P' (3)
L'= L
P
If the two points do not reside on the same axis, the same method can be used to
estimate their distance and P’ is the number of pixels between the two points in the
diagonal line connecting them. In 2D, if the dimensions of the covered area are Lx, Ly
in the x and y axis, corresponding to Px and Py pixels, then each pixel occupies an
area Ap estimated as:
1 Lx Ly (4)
Ap =
S 2 Px Py
A spot of any shape consisting with Pi pixels will correspond to an area: Pi• Ap.
The next issue is how the spots and their features (dimensions, position, color or
grey level features) will be recognized using a simple algorithm. First of all, the plant
part should be separated by its background. This can be performed by a segmentation
procedure that nevertheless is based on complicated operations or a dedicated image
processing libraries. A simple method used in this work is to assume that the
background is much brighter than the plant color. This can be easily reassured if for
example a leaf is captured with a white sheet of paper as its background. The user’s
hand can also be used as a background in most cases if the plant part has a darker
color.
Although, the three color components of an image (Red, Green, Blue) can be
easily handled to extract detailed image features we focus on a grey scale
characterization of the image components and the spot number, coordinates, area and
darkness are the outputs of the SRS. Thus, the captured image is initially converted
into an inverted grey image and the original pixels with high brightness (below a
threshold Tw in the inverted grey image) are ignored since they are assumed to belong
to the background. An average grey level Ag is estimated by the rest of the pixels that
are assumed to belong to the plant part. Then, the image matrix is scanned to locate
the pixels i with a grey level Gi such as:
(5)
Gi − Ag > Th
242
(a)
(b)
(c)
Fig. 3. Original photograph (a), in inverted grey level (b) and the visualization of BW1 with
Th=115 (c).
If the difference between the grey level of the pixel and the average is higher than
the threshold Th the specific pixel is assumed to belong to a spot (lesion) and a 0-1
matrix BW1 (with the same dimensions as the original image) is constructed with 1’s
at the positions of pixels belonging to spots as shown in Fig. 3. The BW1 is scanned
again to group neighboring pixels belonging to the same spot. The resulting matrix
BW2 has an integer spot identity at the position of each pixel or 0 if the pixel does
not belong to a spot. The BW2 matrix is constructed using the following algorithm: a)
the rows are scanned from left to right and neighboring pixels are assigned with the
same identity, b) if the previous pixel on the left of the current one does not belong to
a spot, the already visited neighboring pixels at the row above are checked and if one
or more of these has been assigned to a spot identity, this identity is also used for the
current pixel, c) the BW2 matrix is scanned iteratively merging spot identities if
neighboring spots are found with different identities until no change is detected. A
filtering can also be applied discarding spots consisting of very few pixels (less than
MinArea) because either they are noise or they are too small to be considered.
From the matrix BW2 all of the desired features can be easily available: a) the
maximum spot identity is the number of spots, b) area covered by the spots is
243
estimated using the sum of the pixels belonging to spots (see equation (4)), an
interesting parameter is the fraction of the plant part that is occupied by spots, c) the
average grey level of each spot, d) the coordinates of each spot and its dimensions.
More advanced information can also be extracted if the coordinates of each spot
are used to visit the original colored image and extract the texture of a spot,
information like CIE L*a*b (Kulkarni and Patil, 2012), etc.
4 Experimental Evaluation
In this section the SRS method described in the previous section is applied to the
pictures of Fig. 3a and Fig. 4a. These images show tangerine leaves with dark spots
that may indicate for example the fungus Capnodium oleae or CTV among other
diseases. The number of spots and the area they occupy on the leaf are significant
inputs for the decision module of the mobile disease recognition application
described in Section 2. The SRS output for the photographs of Fig. 3 and 4 are listed
in Table 1. The grey level is not displayed since it cannot be compared to a reference
grey level. The best results were retrieved when setting Th equal to 115. The
parameter MinArea of Table 1 that is set to 4 represents the least number of pixels
required to take into consideration a spot.
Table 1. SRS Measurement Result (MinArea=4, Th=115).
Photo Spots Area Spot Error Area Error
Fig. 3 68 2.1% -5.88% +8%
Fig. 4 65 1% -10% -16%
The negative sign in the errors of Table 1 indicates that the estimated number of
spots or area is smaller than the real ones. The errors concerning the number of spots
and the estimated area are inversely proportional. This means that when changing the
parameters Th and MinArea to improve one error the other gets worse. Using the
image of Fig. 4 and setting MinArea=2, the number of spots recognized is higher
since smaller spots will also be taken into consideration. If Th is also increased (e.g.,
set to 120), then spots with higher contrast are used and those with lower contrast
compared to the leaf background are ignored. Consequently, the parameters Th and
MinArea balance somehow each other. Setting Th to a very high or a low value may
significantly reduce or increase the estimated spot area respectively. Using
MinArea=2 and Th=120 with Fig. 4 leads to the detection of 87 spots (error +20%)
but the estimated spot area is only 0.2%. The conducted experiments show that the
values 2 and 115 selected for the parameters MinArea and Th respectively lead to the
highest estimation accuracy.
Although the spot recognition method has been applied in two indicative leave
images, a predictable accuracy can be obtained in other plant case studies if the spot
density and brightness is similar.
244
(a)
(b)
(c)
Fig. 4. Original photograph (a), in inverted grey level (b) and the visualization of BW1 with
Th=115 (c).
5 Conclusions
An image processing technique that can be easily implemented on smart phones,
capable of recognizing plant lesion features has been presented. The preliminary
measurement results in the recognition of the number of spots and their area on plant
leaves showed accuracy higher than 90%.
In future work the color features of the recognized spots will also be taken into
consideration for safer plant disease diagnosis and the presented algorithm will be
implemented on smart phones and tested under outdoor conditions.
Acknowledgments. This work is protected by the provisional patent 1008484,
published by the Greek Patent Office, May 12, 2015.
245
References
1. Barbedo, G.C.A. (2013) Digital image processing techniques for detecting
quantifying and classifying plant diseases, Springer Plus, 2:660.
2. Carmago, A. and Smith, J.S. (2009) Image pattern classification for the
identification of disease causing agents in plants, Computers and Electronics in
Agriculture, 66(2009), p. 121-125.
3. Chaerle, L., Lenk, S., Hagenbeek, D., Buschmann, C., Van Der Straeten, D.
(2007) Multicolor fluorescence imaging for early detection of the hypersensitive
reaction to tobacco mosaic virus, Journal of Plant Physiology, 164(3), p. 253-262.
4. Kulkarni, A. and Patil, A. (2012) Applying image processing technique to detect
plant diseases, International Journal of Modern Engineering Research, 2(5), p.
3361-3364.
5. Lopez, M.M., Bertolini, E., Olmos, A., Caruso, P., Gorris, M.T., Llop, P.,
Penyalver, R., Cambra, M. (2003) Innovative tools for detection of plant
pathogenic viruses and bacteria, International Microbiology, 6, p. 233-243.
6. Purcell, D.E., O’ Shea, M.G., Johnson, R.A., Kokot, S. (2009) Near infrared
spectroscopy for the prediction of disease rating for Fiji leaf gall in sugarcane
clones, Applied Spectroscopy, 63(4), p. 450-457.
7. Sankaran, S., Mishra, A., Eshani, R. and Davis, C. (2010) A review of advanced
techniques for detecting plant diseases. Computers and Electronics in
Agriculture, 72.
8. Schaad, N.W. and Frederick, R.D. (2002) Real time PCR and its application for
rapid plant disease diagnostics, Canadian Journal of Plant Pathology, 24(3), p.
250-258.
9. Spathis, C., Georgakopoulou, K., Petrellis, N. and Birbas, A. (2014) Integrated
microelectronic capacitive readout subsystem for lab-on-a-chip applications, IOP
Measurement Science and Technology, 25, 055702.
246