The efficiency of farming is increasingly dependent on precision farming. This is due to significant competition, the emergence of new pests and bacteria that spoil the crop, environmental problems, and many other factors from which products lose their value. One of such factors is the timeliness of harvesting. This is especially true in greenhouse complexes, where harvesting occurs regardless of the season, regularly after the ripening of products. The ripening of tomatoes is quite unpredictable, so it is necessary to identify the ripening process to harvest in time. Machine vision can solve this problem by highlighting a separate spectrum of color, which is characteristic of already-ripe tomatoes. Therefore, the article proposes a method for identifying the processes of tomato ripening using image processing methods based on color detectors. The OpenCV library was used for software implementation. A Rasbberry Pi unicameral computer was used to solve this problem.
Methods of automated image analysis are manifestations of artificial intelligence, and the area of their use can be attributed to machine learning, which allows you to implement object recognition tasks by various criteria. Such criteria include the size of the object under study, its color and shape, and particular points in the image as a whole.
It is logical that the need to solve such problems increasingly appears in agriculture. This is due to fierce competition between agribusiness entities, increasing food requirements, and many other needs that increase the need to use artificial intelligence to process images.
However, the availability of machine vision technologies for small farmers does not always exist, especially when, due to lack of funds, they are forced to set up their own automation of their own production. Therefore, unresolved issues related to small farmers make such tasks extremely difficult.
Another problem that can be solved by introducing precision farming using machine vision technology can be considered an increase in the human need for food. Thus, the world's ever-growing population needs to increase food production by 70% over the next 40 years to feed everyone. Representatives of the Food and Agriculture Organization of the United Nations voiced this opinion. And while climate change will increase yields in some regions, it will also bring new challenges to growing healthy crops. Therefore, to meet such demand, it is necessary to increase the efficiency of agriculture in terms of resources and time, and, not surprisingly, farmers turn to assistive technologies. EMAIL: kvashuk11@ukr.net (D.
Kvashuk);
2022 Copyright for this paper by its authors.
All this can be met in terms of precision farming, which is increasing, in the face of increasing consumption of food resources on the planet, makes itself felt. This is combined with the uncertainty of risks and threats, creating what is repeatedly mentioned in the scientific literature.
Today, the urgent need of farmers is to identify the various processes of pre-ripening of plants, in order to harvest in time and without losses.
As an example of solving this problem, we take the fruits of tomatoes, which farmers grow in their greenhouses, regardless of the season. Timely identification of the sowing processes of such crops is extremely important because the quality of the obtained products depends on their solution.
Due to the fact that today there are many approaches to recognizing tomatoes, determining their characteristics by color and shape, simple and accessible to farmers methodologies are quite difficult to find.
Thus, today, the most common methods can be considered machine learning using neural networks,
various algorithms for finding features in images, color detectors, special points, as well as other
methods of analysis [
Determining tomatoes in the greenhouse by color will require only matrix calculations, which will result in an acceptable result because the primary criterion for growth is the color, not even size.
Illumination and the presence of noise may be separate obstacles to such identification.
In this case, the identification of the maturation process requires the selection of a single spectrum
by color in the image [
To do this, the process of photo fixation must be organized so that the size of the image does not differ, and the angle of the photo recorder was unchanged. The lighting during photo fixation should not be variable.
Such requirements can be implemented using a simple movable platform, which can move on a stretched wire (Fig. 1). This solution will be the most optimal in greenhouse conditions.
Thus, it is possible to diagnose the right time to harvest tomatoes, which will allow farmers to respond in time to the yield.
Spectral analysis of images during the identification of different fruits is widely used in many areas.
Thus, a significant part of the work is aimed at identifying the individual contours of the fruit [
In [
However, although the identification of fruits by color has a number of advantages, compared with, for example, methods of identifying fruits by specific points, or by contours, there are certain problems associated with the noise formed as a result of shadows or changes in the light.
Therefore, a number of filters are used to solve such problems, such as the Otsu method, which is
often used to reduce noise by low-pass filters [
Due to the wide popularity of methods of spectral analysis of images [
To highlight the color saturation, you often use the RGB model for identification purposes. This model contains pixels of at least 3 colors with different wavelengths. Such as red, blue, and green (Fig. 2), but for the convenience of working with the color spectrum, usually use the HSV model, where the characteristics of the spectrum also reflect the saturation, which is more in line with human perception of color.
Thus, HSV (English Hue, Saturation, Value - tone, saturation, value) is a color model in which the color coordinates are: Hue – color tone in the range of 0-360; Saturation – saturation, in the range of 0-100 or 0-1; Value – the value of color (brightness). Set from 0-100, or 0-1. (Fig. 3).
The selection of individual color spectra of the image can be done using the library for pattern recognition OpenCv, which includes the possibilities of software implementation of both models. For HSV, the range of shades in this library will be [0,179], the range of saturation - [0,255], and the range of values - [0,255]. If you convert the image of a tomato bush from the format RGB y HSV, we obtain the following images (Fig. 4).
RGB
HSV
Such, you can use image saturation elements.
To identify the fruit of the tomato, you can select the cream pixels by color using this library.
Creating a mask that characterizes only the fruit. To reduce noise, you can use a Gaussian filter [
To determine the spectrum that corresponds to the color of the tomato, you can select its range, but the surface of this fruit has a glossy base, so there is a reflection, which is the cause of certain errors.
For example, the image of a tomato was taken, the range of its characteristics by color was established and as a result, incomplete identification was obtained. Fig. 5 shows the following inaccuracies.
But despite some error, you can count the pixel IDs of the selected color and set the criteria for the growth of tomatoes by calculating the selected color spectrum at different times, following certain requirements for lighting and angle of the photo-recorder.
To verify the effectiveness of the method of identifying the growth of tomatoes by color spectrum, we obtain a sample of images (Fig. 5). To reduce noise, which is presented in the form of glare in the images, apply a Gaussian filter to the selected spectrum. Thus, blurring is required in order to reduce unnecessary noise. Gais blur is a typical image blur filter that uses a normal distribution to calculate the conversion applied to each pixel of the image. The Gaussian distribution equation in N dimensions has the form:
Using the GaussianBlur function of the OpenCv library, which implements Gaussian blur, we can get a more optimal version of the selection of tomatoes (Fig. 6). In order to determine the number of selected objects, the set color range of tomatoes was turned into the white peak of the village, and others into black. This is done using the threshold function, which returns an image where all pixels that are darker (less) 127 are replaced by 0, and anything brighter (more) 127 is replaced by 255. a b c d e 1 2 3 79258 68482 80003 4 40742 51518 39997 Figure 6: The process of identifying tomatoes by color
Thus, the image shows that the number of tomatoes can not be estimated, because there are no clear criteria in the images. Still, at the same time, by color, it is possible to study the ripening process.
In Fig. 7 the number of dots that characterize the selected color and black (line 3 - black, 4 - white) is presented. From this we can represent the proportion of the selected color in the images (Fig. 7). 50,00 40,00 30,00 % 20,00 10,00 0,00 a2 b2 c2 d2 e2 Thus, to identify the fruits of tomatoes, you can use the following sequence of steps: obtaining the original raster image from the photo-capture (.jpg or other sim-ilar format); color filtering; smoothing with a Gaussian filter and converting the selected area to black and white; counting the pixels that characterize the selection of tomatoes (white) count other pixels (black).
Detection of changes in the images of tomatoes can be implemented on the basis of a movable mechanism (Fig. 1), which is controlled by a single-board computer RaspberryPi. To establish the places of photography, on such a device you can install an infrared sensor that will respond to specially selected places (Fig. 8).
Standard infrared distance sensors allow you to respond to approximations at a distance of up to 30 cm. This is sufficient to identify the stopping point of a moving device.
The implementation of the software algorithm can be performed based on Python interpreter and libraries: sqlite3, for database operation, OpenCv, for image processing, RPi, for work with RaspberryPi COM port, which will be used to provide control signal to drive the transport platform.
The algorithm of operation of the movable photo-clamp is presented in Fig. 9.
In this case, the start, stop and input functions can be implemented using a single-chamber RspberryPi computer. Approbation of this algorithm can be implemented with the following sample of photographs, where there is a moderate increase in the number of red tomatoes (Fig. 10). After receiving data on the number of color-defined points and storing them in a database, you can work with them. a b c d e f
Thus, the growth is identified by increasing the red color in the images, which can be visualized as the ratio of the obtained points to their total number (Fig. 11), where line 2 is the total number of points in the image, line 3 is the number of identified points.
60 50 40 %30 20 10 0 1 2 3
4 Maturation periods 5 6
The color of tomatoes is the most crucial factor in their growth. Using traditional color filters and the provided parameters of the color range, it is possible to alter the selected color in photographs. Due to the lack of complicated computations, this method is simple to apply. It is sufficient to utilize a single-chamber computer and a mobile photo clamp, which can record changes in the picture at a certain time, to detect the ripening of tomatoes. Due to the error due to the formation of light concentration, the resulting growth dynamics indicates the feasibility of using this method. However, this requires a number of conditions, namely: detailed images must be taken, using a mobile photo-fixer, during photofixation, the light must be the same as in the previous photo-fixation, the angle and location of photofixation should be non-variable.