The intensification and expansion in cultivation of olives have contributed to the significant spread of verticillium wilt, which is the most important fungal problem afecting those trees. Recent studies confirm that practices such as the use of innovative natural minerals (Zeoshell ZF1) and the application of beneficial microorganisms (Micosat F BS WP), restore health in infected trees However, for their eficient implementation the above methodologies require the marking of trees in the early stages of infestation; a task that is impractical with traditional means (manual labor) but also very dificult as early stages are dificult to perceive with the naked eye. In this paper we present the results of the MyOliveGroveCoach project which uses multispectral imaging from unmanned aerial vehicles to develop an olive grove monitoring system that is able to a) collect large amount of data that is particularly important in relation to the evolution of tree infestation b) quickly detect the problem, using innovative signal processing methods, multispectral imaging and computer vision, in combination with machine learning techniques, providing accurate spatial identification of afected trees c) guide the farmer / agronomist when required, with a communication and decision-making support system, with appropriate interventions and providing maps of quantitative and qualitative characteristics of the grove.
Olive cultivation in Greece is widespread. Olive groves occupy an area of 7.16 million acres, numbering about 130 million olive trees. This represents a large percentage of the total agricultural land and a very large percentage of agricultural land that would be, considering territorial characteristics such as low fertility and sloping, dificult or impossible to exploit from other crops.
Verticillium wilt is the biggest fungal problem of olive cultivation.
It contributes to serious reduction in olive productivity, plant capital
destruction and soil degradation. Verticillium wilt causes a gradual
malfunction and eventually a complete blockage of the vessels of
the tree, in part or in whole, interrupting the movement of water
from the roots to the leaves, resulting in interruption of the water
supply in the afected part of the tree. This reduction in water supply
leads to nutritional deficiencies and even starvation of the leaves.
Before the complete blockage and total necrosis of the afected
tissue associated with the part of the root that has been infected,
there precedes a stage of temporary water stress, a reversible stress,
which is due to the closure of the mouths of the afected plant
tissue [
In this stage of temporary stress, a deregulation is caused in the process of photosynthesis which results in a slight light-green discoloration of the leaves; a discoloration that is very subtle and very dificult to detect with the naked eye.
Thermal and multispectral surveying has shown high
correlations of leaf’s spectral characteristics to the degree of infestation, as
measured in the 11 point scale of Table 1 [
The main goal of MyOGC is the development of an intelligent system that will monitor olive groves and support farmers in the detection and treatment of Verticillium, using multispectral sensors and spectrophotometers. With MyOGC it will be possible to a) collect important data on the progress of tree infestation b) quickly detect the problem, using innovative signal processing methods, multispectral imaging and computer vision, in combination with machine learning techniques, providing accurate spatial identification of afected trees c) guide the farmer / agronomist when required, with a communication and decision-making support system, with appropriate interventions and providing maps of quantitative and qualitative characteristics of the grove.
Remote sensing of agricultural crops that facilitate the timely prediction of plant infestation by diseases has developed rapidly. Both in Greece and abroad, companies have been set up to provide services that monitor and support farmers and their fields. Figure 1 presents platforms available to farmers and producers that ofer remote monitoring of their fields and collection of agricultural data by remote sensing. Figure 2 presents a comparison of some systems and data capturing sensors that are on the market, available as commercial solutions for monitoring vegetation and crops.
Apart from commercial applications targeting farmers and other
specialists of the field, there is significant research interest in using
remote sensing data [
Specific applications include computer vision algorithms
targeting productivity monitoring through tree counting / tree crown
delineation [
In the detection and delineation of individual tree crowns, deep
learning and machine learning approaches [
There is also significant research going on the use of visible
and infrared spectroscopy for disease detection in plants in a fast,
non-destructive, and cost-efective manner. The visible and infrared
portions of the electromagnetic spectrum provide the maximum
information on the physiological stress levels in the plants, even
before the symptoms can be perceived from the human eye.
Different studies have been conducted for disease detection in plants
using this technology [
Lastly, on sourcing data for remote sensing, there exist a variety
of active, passive and mixed sources like GEOSATs, LIDARs and,
more recently, UAVs [
MyOGC integrated system provides an overall automation solution for detecting the Verticillium wilt from aerial multi-spectral images.
The basic user requirements for the MyOGC platform is to support diferent ways of data insertions, manually from the user or direct from the camera located to the drone. Thus, it combines Cloud and Edge Computing technologies, ensuring a highly eficient and scalable high-demanding data processing system and the execution of adapted AI prediction models in an embedded platform in user’s edge devices. The functional module of MyOGC platform is depicted in Figure 4.
MyOGC system consists of four main sub systems: a) The Core has coordination role, it provides the interfaces to the users and edge devices and it accepts and schedules data processes requests for execution in the other subsystems; b) The Data Storage combines a classical RDBMS and File System to store metadata, multi-spectral images and calculated results; c) Containers Execution Engine initiates containers which execute a specific data processing task during a data processing pipeline; and d) the Drone which hosts the Edge device, a Coral Edge TPU device from Google, deployed for executing region of interest detection and classification tasks.
In the Core subsystem, the Process Orchestrator is the module that receives input data and requests for processing. Such requests can be either the process of multi-spectral images of an olive field and the prediction of the spread of the disease on it, or use the stored data in order to train the AI prediction models (both cloud and embedded). According to the request, it selects the appropriate analysis workflow, it calculates the required resources and proceeds to create the execution plan. The plan contains the data processes microservices that must be used and a workflow that defines the execution order of the analysis tasks. The Process Orchestrator, coordinates and monitors the analysis workflow initiating each step and passing the intermediate results between the tasks.
The two interfaces of the Core subsystem are a Graphical User Interface and a HTTP-based Application Programming Interface (API). The GUI is the point of interaction of the users with the system. It is implemented using Python Django Framework and Angular Js library for the frontend. The user can define fields, upload new multispectral images of a field, and ask for their processing while the results are depicted in a GIS-based interactive map. The HTTP API is mainly used for the interoperability between the cloud platform and the edge device, the embedded installed in the drone. The HTTP API uses the GET, POST methods for allowing the invocation of methods that support various tasks such as image uploading, new trained IA models downloading, image process execution, prediction upload, etc.
The Data Storage, as mentioned before, is the centralized subsystem, responsible to securely store all the data of the MyOGC integrated system. A RDBMS is used, the proposed approach utilises
Figure 1: Platforms available to farmers and producers for remote monitoring of fields.
PostgresSQL, for storing users and fields metadata, pre-processed data, devices connection info, prediction results, etc. On the other hand, the filesystem is used to save binary files such as the input and processed images and the AI trained prediction models.
The Containers Execution Environment takes the advantage
of the virtual containers’ technology providing on demand data
process functionalities in a cloud infrastructure. Each container is
independent of computational resources and provides a specific
data analysis task in the notion of the microservices architectural
model [
Finally, the Drone sub-system, aims to bring the intelligence provided by the AI prediction models near to the user’s main device. In MyOGC, the drone with a multi-spectral camera, is used to capture the aerial image datasets. These datasets contain overlapping images that can be merged to create a reflectance map, which is a mosaic of the area of interest where each of pixels in the image represents the actual reflectance of the imaged object used for plant health analysis and the detection of the Verticillium wilt in olive trees. The classic procedure is to upload the images in the MyOGC platform for further processing and algorithmic analysis.
The MyOGC system provides and additional feature. An embedded board with GPU capabilities is installed with the camera in the drone. A compact version of the AI prediction models is installed in the embedded, which is able to perform the data process analysis on the spot. The results are sent to the MyOGC platform for presentation to the user. 4
Plant leaves contain information which is highly associated to their health. Optical leaf properties such as reflectance and transmittance are useful in remote sensing techniques for disease detection. They allow early detection, well before they can be perceived by the human eye, in a non-invasive manner.
In assessing a plant’s health, the most basic and common metric
used is the reflection of vegetation, i.e the ratio of the reflected
radiation to the incident radiation. An assumption is made that the
reflection of vegetation at a certain electromagnetic wavelength,
or spectral reflectivity, depends on the properties of the vegetation
due to factors such as the type of each plant, its water content,
its chlorophyll content and its morphology [
NDVI =
NIR − RED
NIR + RED
NDVI values range from -1 to 1 with the values closest to 1 corresponding to healthier and denser vegetation. NDVI can be calculated using reflexivity or non-physical measurements for the wave bands.
Another example where a vegetation index can provide biophysical information is the Green Exceedance Index (GEI), which is calculated using the red, blue and green wavelength bands. Research for GEI showed that the measured gross primary product in a deciduous forest was significantly correlated with GEI. Thus, a specialized vegetation index can be used as a substitute for measurable biophysical variables that are important when evaluating the phenology of a particular site or plant.
Calculation of vegetation indices is usually done on a pixel-bypixel basis and is, therefore, very sensitive to even slight image distortions. In order to calculate the real changes in biochemical and physiological parameters of vegetation, collected multispectral data have to be geometrically and radiometrically aligned, calibrated and corrected, so as to ensure that the pixels in two images represent the same soil characteristics and the same soil point. Thus, a crucial part of MyOGC is the correct design and implementation of appropriate geometric transformations and spatial-temporal image filters which include, characteristically, algorithms for image registration and alignment, image stitching, creation of orthomosaic with photogrammetry techniques, spectral and luminosity corrections and noise filtering. Classical computer vision techniques are, in most cases, adequate for the implementation of the aforementioned processes.
Another class of processing algorithms relates to the removal of image noise due to data acquisition and enhancing the distinction between the objects under detection (i.e. tree crowns) and the background (i.e. shaded area).
The next stage in processing of the multispectral data concerns
the extraction of useful macroscopic characteristics of the grove, in
an individual tree basis. A key part of this process is the detection
of individual olive trees and the delineation of their crown. This is
achieved by using state of the art classical computer vision
techniques [
A second method is used for the same purpose but targeting a diferent platform, an embeddable device tuned for running ML applications. This device can be mounted on the UAV and connects to the multispectral sensors, allowing real time processing of the captured multispectral images. To make possible the on-the-fly processing of incomplete and noisy data, we use a Convolutional Neural Network (CNN), a class of NN that is ideal for tasks involving image segmentation and classification, trained on ground truth data that where automatically generated from classically processed multispectral images. The main trade-ofs between the simple computer vision and the CNN methods are on implementation complexity, accuracy and eficiency. On one hand, the CV approach is much simpler to implement, shows high and consistent accuracy but is not eficient enough and therefore not a good choice for embedded devices. The CNN approach, on the other hand, is significantly more complex and requires much more work to get it to satisfactory results; furthermore, the accuracy of segmentation is not as consistent as in the CV case and the CNN may need some fine-tuning and readjustment between runs or between fields. The deciding advantage, though, of the CNN method is that it gives very good results when deploying data from fewer or even one band, eliminating the preprocessing overhead, and making the method suitable for low power and low memory embedded platforms, especially on ML-tuned devices that further enhance the eficiency benefits of the method. 5
MyOGC system consists of two basic platforms: a) the cloud platform that contains the most of the MyOGC subsystems and b) the edge platform which is an embedded board (Coral’s Dev Board) capable to execute complex AI and image processing techniques. The role and interconnection between them is depicted in the Section 2 of the current article.
Cloud platforms GUI is the main access point for the users to the MyOGC system. It provides the basic authorisation and authentication mechanism and the forms for managing the fields related meta-data, such as location, photography sessions, owner, prediction results.
Regarding the last, in order to demonstrate the condition of the ifelds to their respective farmers, the platform generated multiple colored layers which are presented as overlays on the original map of the field. When the end-user decides to spectate a field, the platform redirects to a specific interactive map screen where the preprocessed orthomosaic with three basic colors (red, yellow, green) is presented. Green represents healthy trees without phytopathological stress signs, yellow represents stress which is quantified by reduced photosynthetic activity of the afected plant’s canopy and therefore possible onset of disease symptoms and finally red indicates sick trees and/or ground. The end-user can zoom-in and out the map, in order to preview every single tree on the map, with high detail.
For the map representation, Google’s Leaflet library was utilized with Google Map’s satellite image tiles. The overlay is a preprocessed orthomosaic that was constructed with open source photogrammetry software (OpenSFM and GDAL libraries), ensuring the maintenance of the spectral reflectance accuracy (reflectance map) and the exact geographical coordinates of the original multispectral images. Consequently, the image is rendered with a level of transparency, and the map is initialized based on the orthomosaic’s coordinates. In this manner, only the farmers’ fields which can be stretched with map zooms, are visualized (Figure 5).
The edge platform used in MyOGC is the “Dev Board” by Coral. It is a development board for prototyping on-device ML products. The device’s Edge-TPU is ideal for running embedded ML applications. In this project a dev-board is employed on the drone in order to assist and assess the data collection procedure in real time, bypassing the need for the cpu-intensive and time consuming step (uploading images to the server and processing), at least for preliminary data analysis. More specifically, algorithms are run on the dev-board that delineate the olive trees and provide preliminary info for their health status. 0 400 500 600
700 800 Wavelength (nm) 900 1000 1100 MyOGC uses two main sources of data: a) data from direct relfectance measurements on leaves, collected from fields and used as samples for training the assessment- and prediction- algorithms, and b) data from aerial surveying with multispectral cameras.
Olive leaves’ reflectance measurements are performed in certain bands of the electromagnetic spectrum, mainly in the visible and near-infrared wavelengths. A typical reflectance spectrum of a healthy plant is shown in Figure 6. The reflectance of healthy leaves is usually low in the visible spectrum (400–700 nm) due to the significant absorbance from chlorophyll. Healthy plants have high chlorophyll concentration since this substance is crucial for photosynthesis, allowing plants to absorb light energy. Chlorophyll reflects the green portion of the spectrum, producing the characteristic green color of the leaves. Healthy leaves reflect strongly in the near-infrared spectrum, as absorbance of infrared light would cause overheat and consequently damage of the plant tissue.
However, when a plant dies, the process of photosynthesis slows down, chlorophyll content is reduced, allowing other pigments to appear. These pigments reflect light on wavelengths which are perceived as yellow or orange by the human eye. A diseased plant’s leaves absorb infrared light while they reflect the visible portion of the spectrum; the plant gradually dries up and eventually dies. It has been observed that the efect of a disease on a plant changes its leaf reflectance in a specific manner. Consequently, reflectance change of plant leaves is correlated to certain diseases. Remote sensing techniques combined with vis/near-infrared spectroscopy are capable of diagnosing diseases at an early stage without observable indications, by simply measuring the reflectance of a plant leaf.
When light incidents on a plant leaf, two types of reflectance are observed, specular and difuse. Specular reflectance takes place in the plant epidermis—air interface. Specular reflectance does not contain useful information for the health of a plant as the reflected light does not penetrate the interior tissue of the leaf and therefore has not interacted with biochemical constituents (such as chlorophyll, carotenoids etc.).In contrast, light collected by difuse reflectance has interacted with the mesophyll, the inner part of the leaf, where multiple processes of scattering and absorption of light by its biochemical constituents takes place. Therefore, the light from difuse reflectance contains information about the biochemistry of the leaf: difuse reflectance plays an important role in determining the health status of a plant, while specular reflectance acts as noise.
The difuse reflectance component of a leaf is usually measured using a spectrophotometer and an integrating sphere. The difused reflectance component is difused inside the integrating sphere, while the specular reflectance component exits to the outside of the integrating sphere.
In the scope of MyOGC, leaf reflectance measurements are performed using Lambda 35 UV/Vis Spectrophotometer along with an integrating sphere and Spectralon as the reflectance standard.
Leaf samples are collected from olive trees infected with verticillium wilt at diferent stages over a long period of time starting from March up to June at 15-day intervals. Five leaf samples are usually collected from randomly selected branches of each tree. Each olive leaf is mounted in a special sample holder provided by the spectrophotometer’s manufacturer. The sample holder is placed at the exit port of the integrating sphere. A light source of wavelength range of 190 nm to 1100 nm is at the entrance port of the integrating sphere. We collect leaf reflectance spectra from about 400 nm to 1100 nm, calculate the mean reflectance for each tree,and perform a first- and a second-order derivative analysis. Due to the high sensitivity of derivative analysis to noise, a Savitzky – Golay iflter is applied for smoothing the data with a polynomial order of 4 and a frame length of 17.
The first and second-order derivative analysis provides information for the reflectance slope in the red-edge position. The slope in the red-edge is highly associated with the chlorophyll content of the leaf. If the slope of a leaf reflectance spectrum is low, then the leaf has a low chlorophyll content. This means that leaf is infected with a disease or it slowly dies. Peaks in the second-order derivative are correlated to certain issues such as nitrogen deficiency.
The aerial multispectral images are collected using a Pix4d Parrot Sequoia camera mounted on a C0 class drone. The Parrot camera is a multispectral camera capturing images on the four characteristic bands: green (550nm), red (660nm), red edge (735nm) and nearinfrared (790nm). Figure 7 visualizes a typical drone flight pattern, at a height of 70m. A sample of the collected images is presented in Figure 8.
An early processing stage takes place on the dev board mounted on the drone, providing some real-time preliminary analysis of the olive grove. Notably, this first analysis includes visualization of olive trees crowns (Figure 9) and vegetation indices. 6.1
The efectiveness of deep learning algorithms significantly relies on the proper acquisition and manual annotation of a large amount of good quality data. In many cases, limitations occur that have to do with the lack of expert knowledge for data labeling, dificulties in capturing large quantities of data with suficient variety, or even the ability to capture good quality data volumes might be extremely expensive and under privacy restrictions. In such cases, the lack of real-world data can be tackled by generating synthetic data that share the same basic characteristics with the real ones.
The use of synthetic data can be twofold. For example, synthetic data can be initially used to train a deep learning model with the intention to use them on real-world data, or even train generative models that refine synthetic data for making them more suitable for training. In addition, synthetic data can be used to increase real-world datasets, or even be generated from existing data using generative models, in order to produce a hybrid dataset able to efectively cover the data distribution that is not adequately represented in the real dataset and, therefore, alleviate dataset bias.
In this line of research and due to lack of high volumes of proper olive tree data in diferent environmental conditions, generation of synthetic data is investigated here with the use of Blender tool. Blender is an open source software for creating 3D environments, able to run on any operating system and having the ability to write scripts and addons in Python programming language. In our case, scripting was used in the Blender environment for generating multiple olive trees with great variability. From a single leaf and the use of specific textures of the tree branches, trunks and the soil, a close-to-real synthetic tree as well as a number of synthetic trees were created, using the sequential approach shown in the block diagram of Figure 10.
Initially, the appropriate textures needed for the olive tree creation (healthy/ill leaves, branches, trunk), as well as the position of the soil of the trees were gathered (Figure 11 (a,b)). The 3D model of the leaf was then produced (Figure 11 (c)), followed by the creation of the branch by replicating the created leaf model, or combining multiple leaf models (Figure 11 (d-f)).
Using the created branches and combining them with the olive tree trunk texture, an olive tree can be created. By replicating the same methodology a random number of trees can be positioned onto the given soil, as shown in Figure 12. 7
Monitoring vegetation using drones can provide important data for the assessment of the condition of crops. However, it is vital that data collection with today’s media be done as carefully as possible, as it will be the basis for future studies of Precision Agriculture and ecological monitoring. Despite the plug-and-play nature of the latest generation of multispectral sensors, such as Parrot Sequoia and MicaSense RedEdge, a number of factors require careful consideration if the goal is to collect high quality data that are comparable between sensors, geographically and over time.
MyOliveGroveCoach has developed and is implementing a standard workflow for processing agricultural multispectral data, taking into account the technical aspects and challenges of multispectral sensors, flight planning, weather and sun conditions, as well as aspects of geographic positioning.
By using multispectral imaging from UAVs and employing innovative signal processing methods in combination with machine learning techniques, MyOGC ofers an olive grove monitoring system that is useful in the early detection and prediction of verticillium wilt spread, and provides a platform that helps the farmer asses the condition of their fields through maps of important characteristics of the grove and guides the agronomist through a communication and decision-making support system.
MyOliveGroveCoach (MIS 5040498) is implemented under the Action for the Strategic Development on the Research and Technological Sector, co-financed by national funds through the Operational Programme of Western Greece 2014-2020 and European Union funds (European Regional Development Fund).