Spectral-Spatial Analysis of Data of Images of Plantings for Identification of Stresses of Technological Character

Abstract

Methods of spectral-spatial analysis are promising for the identification of technological stresses. The most common solution for interpreting the causes of stress is the use of machine learning technologies, namely neural networks. As at technological stresses in particular at chemical poisoning of crops, there can be various options of the coloring of the affected plants the possibility of providing a sufficient amount of initial data for training of neural networks is doubtful. An alternative is graph analysis of the distribution of stress areas on the field map. Given the urgency of the problem for promising technologies of precision agriculture, the work aimed to develop a spectral-spatial method of monitoring technological stresses, namely the algorithm and software for its. Experimental studies of the manifestation of technological stresses on winter crops on the example of wheat and rapeseed were conducted during 2018-2020 in production fields using universal cameras in the visible range and special multispectral Slantrange systems. For remote monitor, the state of winter crops, an algorithm for identifying technological stresses was developed, which is implemented in the developed software in Python for spectral-spatial analysis of stress index maps. It has been experimentally confirmed in the production fields that the use of the developed software allows identifying the contours of areas of plants with stresses of technological nature based on stress index distribution maps.

Keywords

• 1 UAVs
• winter crops
• vegetation indices
• stresses
• herbicides

1. Introduction

The prospects for agricultural production management based on objective remote monitoring data were obvious both at the state level and for agricultural enterprises. Accordingly, research was carried out to develop various theories and methods for obtaining information about vegetation. Under uncertainty, M. Lotfi et al. (2009) in [ 1 ] proposed computer data processing systems for satellite data filtering and machine learning technology for object recognition. That is, in the spectral-spatial analysis, the field of the field as a whole was not considered as the object of research. This approach is used in particular in aviation for the implementation of orientation in the use of electronic warfare as shown in the work of S. Shvorov and others (2018) in [ 2 ]. Regarding agricultural production, Xianlong Zhang and others (2019) in [ 3 ] proposed the division of spectral-spatial monitoring methods into 2 conditional categories. The first category uses the spectral characteristics of terrestrial objects and then obtains vegetation information by comparing the difference with the results of spectral monitoring. An example of such monitoring is the identification of trees in densely populated cities based on satellite images shown in S.W. Myint et al. (2013) in [ 4 ].

The second category is based on a combination of external knowledge such as decision tree for image classification shown in Andrea S. Laliberte et al. (2007) in [ 5 ], neural networks, and wavelet transforms described in Mitch Bryson and others (2010) in [ 6 ]. This promising method has not been widely used in satellite monitoring because the combination of time delay and low resolution leads to unacceptably large errors, which was shown in the article by Passang Dorji et al. (2017) in [ 7 ]. UAV monitoring is devoid of these shortcomings and accordingly, this method can be implemented on a new technological basis. Thus, in the work of J. Senthilnath et all (2017) [ 8 ], it was possible to successfully identify weeds in crops by fixing plants in automatically determining technological tracks.

Wavelet analysis methods do not require the division of the image into blocks, because the required localization properties are already embedded in the wavelet system. Accordingly, it is possible to filter out a significant number of errors inherent in pixel analysis methods. The method of wavelet analysis for the identification of affected areas due to technological stresses, namely the prolonged action of herbicides was shown in the work of M. Dolia and others (2019) in [ 9 ]. The proposed solution proved its effectiveness when, as a result of a dosing error in a part of the field, a higher dose of herbicides was applied. Since the application was made by appropriate ground equipment, the authors in the analysis of the map image focused on the search for linear functions. The authors noted some difficulties in the established systems when choosing thresholds. The complexity of this controversial issue was confirmed by Yu-Hsuan Tu et al. (2020) [ 10 ] where it was the limit values that were recommended to be studied at higher resolutions. Due to this specificity of the method, the analysis will be effective for the affected crops on a large scale, which significantly limits its effectiveness. Large-scale impressions can be easily identified by satellite technology or ground-based monitoring, but small areas will be difficult to detect. Crop management technologies need to be adapted to respond to such problem areas, as weakened plants are easily affected by pests and can become a breeding ground for them.

A possible technology for the analysis of spectral-spatial distribution is artificial neural networks which, due to the rapid development of multi-core processors, have become available to farmers. There is a positive experience of using neural networks for various monitoring tasks which, if necessary, can be adapted to monitor technological stresses. Section 1 shows that in the initial stages of the growing season, the dimensions of plants may indicate their stress. Neural networks for estimating plant height during rice lodging are shown in Ming-Der Yang et al. (2020) [ 11 ]. According to the provided results, it was possible to detect rice lodging with acceptable accuracy based on images from universal cameras in the visible range, but the calculations were performed using cloud services, which is difficult to implement in our country. Autonomous work of neural networks is shown in the work of Wojciech Gruszczyński and others (2019) [12] to identify grass among general vegetation. When analyzing the image was segmented into parts and carried out training of the network on the distribution of the cloud of points. This approach is promising for the identification of low-growing grass because only one manifestation is considered, but under technological stress, there may be more. In principle, for neural networks, there can be several options for identifying objects. They can be used in particular to determine the state of rice yield at the stage of ripening, as shown in Qi Yang and others (2019) [13], or the state of mineral nutrition described in V. Lysenko and others (2017) [14]. Spatial distribution was also considered in Yan Pang et al. (2020) [15] to calculate the number of plants in a ridge. All these works are combined by a limited number of classification options and a large sample of source data for neural network training. In this case, in contrast to the vegetation indices, which focus on pixel-by-pixel analysis, the training of neural networks was based on crop areas obtaining more accurate results.

As at technological stresses in particular at chemical poisoning of crops, there can be various options of the coloring of the affected plants the possibility of providing a sufficient amount of initial data for training of neural networks is doubtful.

There are no ready-made software solutions for analyzing the distribution of stress areas on the field map to identify the nature of stress. Given the urgency of the problem for promising technologies of precision agriculture, the work aimed to develop a spectral-spatial method of monitoring technological stresses, namely the algorithm and software for its implementation.

2. The state of the issue

2.1. Identification of direction of movement of technological equipment

Stressful conditions of crops of technological character are caused by human actions which are realized by the use of the ground technological equipment. The identification of equipment directions was considered in Junfeng Gao et al. (2018) [16] regarding the detection of weeds in row crops, where all plants between rows were considered weeds. In Carlos Henrique Wachholz de Souza et al. (2017) [ 17 ], sugar cane rows were identified to estimate row gaps. In both cases, the rows were considered to be the arrangement of plants in a row, because this is how ground equipment moves. However, in agricultural practices, the directions of ground equipment movement should change from year to year, and, accordingly, the distribution of stress areas may differ from the direction of crop rows. Accordingly, the identification of stresses can be based on the assessment of the contour of the stress section, which for technological stresses must have the correct geometric shape inherent exclusively in artificial objects. In particular, in the case of chemical poisoning of plants, the boundary between affected and healthy crops will be directly linear.

2.2. Choice

environment of the software

Assessing the nature of stress for crops is an urgent task to be solved both by agronomists directly in the fields and by relevant specialists using cloud services. Accordingly, for the versatility of the operating system used, it is advisable to use a cross-platform programming language such as Python, which is adapted to the fate of large data processing and machine learning. In the work of Emad Ebeid and others (2018) [ 18 ], devoted to the review of flight controllers and flight control of UAVs, the prospects of the Python language for these tasks were emphasized primarily due to the use of technical means from different manufacturers on different operating systems. 2.3.

Experimental research

Experimental studies of the manifestation of technological stresses on winter crops on the example of wheat and rapeseed were conducted during 2018-2020 in production fields. Photography was performed using: in 2018-2019, to monitor the stationary experiment and fragments of production fields - hexacopter based on multi-rotor platform CD600 with a set of specialized sensor equipment in the digital action camera GoPro HERO4, in 2019-2020 multispectral system 3p, mounted on a DJI Matrice 600 hexacopter, which allowed to obtain orthophotos of industrial fields. It is the spectralspatial analysis of the obtained orthophoto plan that allowed us to establish the dependences on the basis of which the identification of technological stresses is carried out.

Technological stresses on winter oilseed rape can be detected by means of leaf diagnostics because in September-November there is an abnormal color of the lower leaves, which is easy to establish both by ground visual assessment and research using UAVs. For winter wheat, such manifestations suitable for reliable identification from the UAV platform on an industrial scale (height from 60 meters) could not be detected. In ground-based monitoring, it was noted that plants have a characteristic deformation of the leaves, which can be a characteristic criterion for identifying the nature of stress. Affected areas inside the field are more dangerous for industrial fields, which are difficult, often impossible to visually detect by ground monitoring means. This situation is extremely dangerous, as areas with weakened plants appear in the field, which is more susceptible to pests and can become centers for the spread of the latter. Accordingly, it is advisable to develop a technology that will identify stress areas regardless of their location. This was taken into account when choosing the experimental production field. 2.4.

Analytical research

Laboratory studies accompanied all stages of plantation monitoring. A sampling of plants and soil was performed on the day of monitoring or within two days thereafter. Soil samples were taken from a layer of 0-25 cm, prepared for analysis according to DSTU ISO 11464: 2007. Agrochemical analysis was performed in scientific and research laboratories of the Department of Agrochemistry and Plant Product Quality, Ukrainian Laboratory of Agricultural Products Quality, in compliance with accepted methods and techniques.

3. Algorithm for identification of

technological stresses, its software implementation, and results of experimental data processing

3.1. Select the source data format

To process spectral monitoring data, the Slantrange sensor system has its own Slantview software, which allows you to save the received maps in several data formats, namely Shapefile, KMZ, GeoTiff. The shape format contains attributive information of geometric objects and is designed primarily to create tasks for ground equipment. KMZ files are 3D data in Google Earth and represent a map of the distribution of vegetation indices on satellite images. According to the results of experimental studies on the recognition of the values of vegetation indices, it was found that the data was distorted during the overlay of the images - the recorded colors were missing in the palette for the specified vegetation indices. Probable explanation in image correction for overlay on the satellite image to facilitate visual perception by the user. By comparing the data for the distribution points of the distribution map from the working window of the SlantView program, it was found that for the GeoTiff format color distortion and, consequently, the values of vegetation indices do not occur. Unlike the KMZ format, the file does not have positioning labels, but when you save the map, the program retains the scaling, and, accordingly, when using landmarks, the calculation of positioning is quite possible. In view of the above, the GeoTiff format was adopted for analysis.

3.2. Data processing

3.2.1. Evaluation of the contour of the map

To manage the harvest, farms, regardless of weather conditions, need maps of the distribution of vegetation indices in many production fields available on the farm. Based on these circumstances, the Slantrange sensor complex was created to survey up to 10,000 ha/day, which can be provided on aircraft platforms. Since the average area of production fields in the plains of Ukraine is 70-100 hectares, it is desirable to survey several fields at once during one flight. In the analysis of a particular field, it is necessary to determine its boundaries. The Python-supported OpenCV library contains ready-made procedures for finding the contours of graphical objects that can be used in this case. An example of the result of card processing is shown in Figure 1.

It should be noted that the forest-steppe zone of Ukraine is characterized by strong forest belts, the leaf cover of which, as well as the shadow from them are also fixed by the system. From the available experience, maintenance of forest belts and their renewal is not carried out regularly and there are many cases when tree crowns completely cover the road surface due to which significant errors are possible in determining the contours of crops. Since the field boundaries are stable, to analyze the presence of technological stresses on the maps of the distribution of vegetation indices stored in the Geotiff format, it is advisable to enter them manually, using certain reference points.

3.2.2. Estimation of the orientation of the sections of the field caused by stresses of technological character

With the identification of crop rows, the direction of crop rows was stable, but this is not a prerequisite for technological stresses. Thus, Figure 1 (a) shows the presence of a green band on the left and top, which for this index Green Chlorophyll index corresponds to healthier crops than those with yellow. This condition may be due to the best condition of mineral nutrition at the field boundaries because it is there that the equipment slows down, turns, adjusts the operation of nozzles, augers, and more. The width of such a layer, as a rule, does not exceed the radius of reversal of ground equipment, which can be taken into account when analyzing the distribution of stress areas

The distribution in the field of stress areas caused by phytotoxic action (aftereffect) of herbicides, as well as violation of the seeding rate, is related to the direction of technological tracks, the organization of which meets certain rules. This is due to the fact that the introduction of chemical reagents or seeds during sowing is not carried out in an arbitrary manner, namely in compliance with the laid technological tracks. The directions of technological tracks in one field can change from year to year to maintain soil fertility, but their number usually does not exceed 2, in some cases 3 directions. Soil loosening can be carried out in any order, but technological stresses cannot be caused by this operation. Determining the direction of technological tracks has certain prerequisites, so mechanics when planning work are interested in the maximum length of the runs. Accordingly, in the absence of data from technological maps for the implementation of mechanical tillage, the orientation of the experimental field should be carried out along the maximum length of the field.

Figure 2 presents a map of the distribution of stress areas for winter wheat crops where chemical poisoning of winter wheat crops as a result of the after-effects of herbicides from the predecessor crop was recorded.

The specificity of SlantView software data processing is the observance of the north-south geographical orientation. As a result, to reduce the amount of data that does not belong to the field under study, it is necessary to change the orientation from geographical to local reference to the dimensions of the field. Due to the change in the orientation of the image, the number of pixels of the image obtained from the GeoTIFF file decreased from 1100 × 1660 to 245 × 1521, ie the amount of data decreased almost 5 times.

Figure 3 shows the interface of the developed program in python to identify stressful areas of technological nature.

3.2.3. Convert data from color format to numeric view

Figure 4 shows the map window palette, which is used to encode data and save them in tiff format according to the method presented in S. Shvorov et all (2020) [ 19 ]. Since the NDVI indices for plants change in the range 0… 1 for visualization in the 8-bit color model, the index values were multiplied by 255 (Fig. 4)

3.2.4. Image segmentation

To assess the presence of stress of a technological nature, the field image was divided into separate sections. The size of the plot was determined based on the resolution of the distribution map and the standard nomenclature of ground equipment available on the farm. The size was 13 × 13 pixels (6.5 × 6.5 m).

3.2.5. Calculation parameters of distribution

The GaussAmp equation was used to approximate the experimental data on the color intensity when color-coding the values of the intensity of the GreenNDVI index. Determined the value of the average value. Figure 5 shows the image of the program interface when indicating the intermediate results for statistical processing of the distribution of index values on the map segment.

3.2.6. Filtering of the received data

Previous studies have found that stress should be determined by the magnitude of the standard deviation. For filtration, a limit value was set at which the stress status was set for the plots. 3.2.7. Graph search

analysis for in-depth

The stressful state of plantations is caused by chemical poisoning of plants or their thickening due to non-compliance with production technology when moving ground equipment. Accordingly, stress areas will form bands. The DFS (Depth-first search) method was used to identify such stress areas. That is, single manifestations of plant stress due to differences from the total mass of the water supply regime, etc. are not taken into account. The results obtained are presented in Figure 6.

The developed software passed a production test, during which its accuracy and selectivity were confirmed.

4. Conclusions

High-resolution maps of high-resolution stress indices can be considered as a separate object of study on the interpretation of the causes of the stress of complex biological objects, such as winter crops. For remote monitoring of the state of winter crops, an algorithm for the identification of technological stresses has been developed on the basis of the spectral-spatial analysis of the nature of the location of stress areas. The algorithm is implemented in the developed software for spectral-spatial analysis of stress index maps to identify stress areas due to technological factors.

It has been experimentally confirmed in the production fields that the use of the developed software allows identifying the contours of areas of plants with stresses of a technological nature on the basis of stress index distribution maps.

5. References

References

[1] M. Lotfi ( 2009 ) Combining wavelet transforms and neural networks for image classification / M. Lotfi , A. Solimani , A. Dargazany , H. Afzal , M.Bandarabadi //. System Theory, Ssst 2009 . Southeastern Symposium on . IEEE . pp. 44 - 48 . https://doi.org/10.1109/SSST. 2009 . 4806819 .

[2] S. Shvorov ( 2018 ) UAV Navigation and Management System Based on the Spectral Portrait of Terrain / S .Shvorov, D. Komarchuk , N. Pasichnyk , О.Opryshko, Y. Gunchenko , S.Kuznichenko // 2018 IEEE 5th Int. Conf. on Methods and Systems of Navigation and Motion Control (MSNMC) , - Proceedings. рр. 68-71 , http://dx.doi.org/10.1109/MSNMC. 2018 . 85 76304.

[3] Xianlong Zhang ( 2019 ) New research methods for vegetation information extraction based on visible light remote sensing images from an unmanned aerial vehicle (UAV) / Xianlong Zhang , Fei Zhang, Yaxiao Qi, Laifei Deng, Xiaolong Wang, Shengtian Yang // International Journal of Applied Earth Observation and Geoinformation , Vol. 78 , рр. 215 - 226 , https://doi.org/10.1016/j.jag. 2019 . 01 .001.

[4] S. W. Myint ( 2013 ) Per-pixel vs. Objectbased classification of urban land cover extractio n using high spatial resolution imagery / S.W .Myint, P. Gober , A. Brazel , S. Grossman-Clarke , Q . Weng // Remote Sens. Environ . Vol. 115 ( 5 ), рр. 1145 - 1161 , https://doi.org/10.1016/J.RSE. 2010 . 12 .017.

[5] Andrea S. Laliberte ( 2007 ) Combining Decision Trees with Hierarchical Objectoriented Image Analysis for Mapping Arid Rangelands / Andrea S . Laliberte, Ed L.Fredrickson, Albert Rango // Photogrammetric Engineering & Remote Sensing , Vol. 73 ( 2 ), pp. 197 - 207 ( 11 ), https://doi.org/10.14358/PERS.73.2.197.

[6] Mitch Bryson ( 2010 ) Airborne vision‐based mapping and classification of large farmland environments / Mitch Bryson , Alistair Reid, Fabio Ramos, Salah Sukkarieh // Special Issue: Visual Mapping and Navigation Outdoors , Vol. 27 ( 5 ), рр. 632 - 655 , https://doi.org/10.1002/rob.20343.

[7] Passang Dorji ( 2017 ) Impact of the spatial resolution of satellite remote sensing sensors in the quantification of total suspended sediment concentration: A case study in turbid waters of Northern Western Australia / Passang Dorji , Peter Fearns // PLoS ONE, Vol. 12 ( 4 ), e0175042 . https://doi.org/10.1371/journal.pone. 017504 2 .

[8] J. Senthilnath ( 2017 ) Application of UAV imaging platform for vegetation analysis based on spectral-spatial methods / J.Senthilnath, M. Kandukuri , A. Dokania , K.N. Ramesh // Computers and Electronics in Agriculture. Vol. 140 , pp. 8 - 24 ; https://doi.org/10.1016/j.compag. 2017 . 05 .02 7.

[9] M. Dolia ( 2019 ) Information Technology for Remote Evaluation of after Effects of Residues of Herbicides on Winter Crop Rape / M. Dolia , V. Lysenko , N. P asichnyk, O.Opryshko , D. Komarchuk , V. Miroshnyk , T. Lendiel , A .Martsyfei // 2019 3rd International Conference on Advanced Information and Communications Technologies , AICT 2019 - Proceedings рр. 469-473 . https://doi.org/10.1109/AIACT. 2019 . 88478 50.

[10] Yu-Hsuan Tu ( 2020 ) Optimizing drone flight planning for measuring horticultural tree crop structure / Yu-Hsuan Tu , Stuart Phinn, Kasper Johansen, Andrew Robson, Dan Wu // ISPRS Journal of Photogrammetry and Remote Sensing , Vol. 160 , рр. 83 - 96 , https://doi.org/10.1016/j.isprsjprs. 2019 . 12 .0 06 .

[11] Ming-Der Yang ( 2020 ) Adaptive autonomous UAV scouting for rice lodging assessment using edge computing with deep

[17] Carlos Henrique Wachholz de Souza ( 2017 ) Mapping skips in sugarcane fields using object-based analysis of unmanned aerial vehicle (UAV) images / Carlos Henrique Wachholz de Souza, Rubens Augusto Camargo Lamparelli, Jansle Vieira Rocha, Paulo Sergio Graziano Magalhães // Computers and Electronics in Agriculture, Vol. 143 , рр. 49 - 56 , http://dx.doi.org/10.1016/j.compag. 2017 . 10 . 006.

[18] Emad Ebeid ( 2018 ) A Survey of OpenSource UAV Flight Controllers and Flight Simulators / Emad Ebeid, Martin Skriver, Kristian Husum Terkildsen, Kjeld Jensen, Ulrik Pagh Schultz // Micro- processors and Microsystems, https://doi.org/10.1016/j.micpro. 2018 . 05 .00 2.

[19] S. Shvorov ( 2020 ) The method of determining the amount of yield based on the results of remote sensing obtained using UAV on the example of wheat / S. Shvorov , V. Lysenko , N. Pasichnyk , Y. Rosamakha , А. Rudenskyi, V. Lukin , A . Martsyfei // 2020 IEEE 15th International Conference on Advanced Trends in Radioelectronics, Telecommunications and Computer Engineering (TCSET), рр . 245 - 248 . http://dx.doi.org/10.1109/PICST47496. 2019 .9061238Wang, Xin , Tapani Ahonen , and Jari Nurmi . "Applying CDMA technique to network-on-chip." IEEE transactions on very large scale integration (VLSI) systems 15 .10 ( 2007 ): 1091 - 1100 .