Agricultural parcel localization on satellite images using U-Net-based neural network

Abstract

-This work considers the problem of automatic delineation of agricultural parcels on satellite images based on truecolor images and NDVI vegetation index maps from Sentinel-2 satellites (10 m ground sampling distance). The problem is solved using a U-Net-based convolutional neural network. We consider problem formulation either as parcel mask or boundary detection; the multiclass (simultaneous) training did not prove to be effective. The approach looks promising and applicable for automated land mapping for agricultural monitoring systems.

Keywords

• U-Net
• convolutional neural network
• precision agriculture
• automated mapping

I. INTRODUCTION

In this paper, we consider the problem of automating the mapping of agricultural fields using satellite data of 10meter spatial resolution. This task is relevant both in cadastral accounting and in agricultural monitoring. High-precision manual mapping of parcels is a labor-intensive process, and knowledge about the boundaries of parcels is an essential element for solving other tasks of agricultural monitoring, in particular, evaluating various indicators of productivity and land condition when using precision farming approaches.

There are many works on the related problem of crop classification [ 1 ]–[ 3 ]. The problem of field mapping automation is not so studied, although there are some works [ 4 ], [ 5 ].

Some works, including [ 4 ], [ 5 ], use high-resolution data, which makes the problem easier. For example, high-resolution 80-cm allow to detect single trees [ 6 ]. However, lowerresolution data is more widely available thanks to research programs such as Sentinel-2 [ 7 ]. It provides regularly updated (about two times per week) multispectral satellite imagery in resolutions from 10 to 60 meters per pixel, depending on the spectral channel.

Determining the most suitable spectral ranges for mapping is not a trivial task. This paper uses the fact that the so-called vegetation indices-images calculated from images in different wavelength ranges [ 8 ], [ 9 ] - have long been known and widely used to solve problems of agricultural monitoring.

In this paper, in addition to the image in the visible range, we investigate the use of the NDVI (Normalized Difference Vegetation Index) as input data for automated mapping. It is a normalized relative vegetation index, which is useful in crop monitoring problems [ 10 ]. NDVI is calculated from the ratio of the observed intensity of red (RED) and near-infrared (NIR) channels:

N DV I =

N I R RED N I R + RED

Currently, algorithms based on training in artificial neural networks, in particular using convolutional layers, are widely used in various problems of image analysis [ 4 ], [ 5 ], [ 11 ], [ 12 ]. A particular case of full-convolution architecture, showing good quality in segmentation problems, is the U-Net family of neural networks [ 13 ], which implements a multiscale approach to image analysis. We can also note one of the universal segmentation algorithms MaskRCNN [ 14 ], in which object detection is accompanied by further pixel-by-pixel segmentation using convolutional layers. This algorithm allows one to detect individual objects and their exact boundaries, including when their images intersect. However, this more complicated method is not required for defining masks or parcel boundaries when objects do not intersect, and there is no problem with partial obscuration.

II. METHOD

We used a neural network approach to automatically detect parcels. In this approach we model detection with a function fw : X ! Y . The fw function maps multi-channel images x 2 X = [0; 1]h w c to single-channel images y 2 Y = [0; 1]h w. The pixels yi;j 2 [0; 1] of such an image contain confidence estimates that the i; j pixel in the source image refers to a parcel (or a parcel boundary in case of boundary detection). The type of fw function depends on the task to be solved and defines the architecture of an artificial neural network.

A sequential network is a simple example of an artificial neural network. Such a network consists of inputs x = h0 and a few trainable functions (layers) hi = fi(hi 1); i = f1; :::; ng, applied sequentially. The last layer output y = hn is treated as the neural network output. For the image processing tasks, the convolutional layers are usually used. In this case, the input and the output of the layer are images, i.e., 3d tables (tensors).

0

X dx;dy;cf;ct

wdix;dy;cf;ct hix+1dx;y+dy;cf A ; where ReLU (x) = max(x; 0). The idea of such filters is to locally process different image parts in the same way. This also allows to simultaneously optimize weights w on different parts of the image. Convolutions have typically small kernel size, usually 3 3, (bigger ones are reducible to several such ones). To make the output size to be equal with the input size, an input image is usually zeros-padded.

Function fw parameters (or weights) w are automatically tuned on the collected dataset fxi; yig with the expertly marked position of the parcels (and their borders). The w is changed so to minimize the loss function (or empirical 1 risk) L(w) = Pi l(fw(xi); yi), where l is the two-class cross entropy.

l(y; y^) = ( log(y) log(1

; y^ = 1 y) ; y^ = 0:

U-Net-like architectures [ 13 ] are commonly used in problems of image segmentation and pixel-wise classification. Neural networks of this type have an encoder-decoder architecture with several layers of different resolutions. Also, such architecture is not sequential and has shortcut connections. UNet-like networks have several advantages, including: a sufficiently large receptive field, which allows a neural network to make each pixel decision based on a relatively wide spatial neighborhood; a computational efficiency due to the multiscale approach. The network of this architecture works with a small number of weights in the original resolution image, then with a large number of weights in the image of a much lower resolution and, in the end, combines the low-resolution results with highresolution data from the original image for more accurate pixel prediction. In this paper, we used a neural network with 32 filters on the first layer and the smallest network scale of 1=8 (see Fig. II).

A. Training

A dataset consisting of 122 4-channel images of 22 areas of the Earth surface for the period 04/05/2018 – 11/11/2018 from the Sentinel-2 satellite imagery archive was prepared. The size of each image is 1030 1030 pixels, imagery resolution is 10 m/pixel. The first 3 channels of each image are visible colors (T CI in Sentinel-2 nomenclature), and as the fourth channel, the N DV I vegetation index map is used (calculated by bands 4 and 8 of the original multispectral image), see Fig. 1ab.

Fields and similar structures were manually marked on each full-color image in the form of polygonal contours, resulting in 400 average objects being marked on all images. The field boundary mask was constructed as follows: the field mask was morphologically dilatated with a square window of 10 10, after which the points included in the dilatated mask but not in the original ones were considered as the boundary points (see Fig. 1cd).

The dataset was divided into training and test parts, consisting of 17 regions with 94 images total and 5 sections with 28 images, respectively.

The network has been trained for a 25 epochs with 500 batches. Each batch contains 32 random 128 128 patches, cropped randomly from the original images. The used optimizer is Adam [ 15 ] — one of stochastic gradient descent methods. We used a loss function consisting of a cross entropy and l2 — regularization with 0:0001 coefficient to prevent overfitting.

We use pixel-wise classification metrics. The most common one is the AUC-ROC metric. To calculate it we consider all possible thresholds to obtain the all possible classifiers T ci;j = [yi;j > T ]; T 2 R: These classifiers’ parameters on the TPR and FPR coordinates plot the so-called ROC curve. The area under this curve is the desired value. However, this metric may be low informative if the classes are highly unbalanced. In our case, the number of background non-borders pixels is much bigger than the number of object pixels (borders). To reduce this effect, we also calculate the AUC-PR metric built similarly on the values of precision and recall instead of TPR and FPR.

We did a few computational experiments to study dependence by the input channels: TCI, NDVI, TCI+NDVI. Also, we did a few experiments to check the multiclass method, in which the neural network learns to predict parcels and its boundaries simultaneously. In each case, the experiment was repeated three times to estimate not only the value but its standard deviation also.

For comparison, the value of metrics for a random classifier is given. For boundaries & parcels prediction, different boundary (b) and parcels (p) results are presented. The results are listed in Tab. I.

The presented results show that a neural network is significantly higher than the random algorithm, which has AU C ROC and AU C P R equals to 0:5 / 0:4 for the fields and to 0:5 / 0:06 for the boundaries. The results of the multitask training are not significantly different from the usual ones. The quality of border detection by AUC-PR metrics is much lower than that of AUC-ROC, because of a much lower share of boundary pixels compared to the share of parcels pixels. The results of the network look strongly correlated with the correct answer (see Fig. 1de), which is probably the most important thing in this task: both the metrics themselves and the ground truth values are not very reliable, as it is difficult to check whether it is a field boundary at a given point or it is another visually similar structure. We see that this approach is applicable to the construction of agricultural monitoring systems and the appropriate optimization for a particular application scenario. In the case of a larger dataset, there are no obstacles to obtain good results.

IV. CONCLUSION

The paper considers the problem of determining the location of agricultural parcels using multispectral satellite images. The input images contain visible range imagery, NDVI vegetation index maps, or both. The output ground truth annotation contains parcels, boundaries, or both.

The results demonstrated the applicability of the U-Net network architecture for this task with at least 0:7 AUC-ROC metric value for the parcels and boundaries. The comparison of different variants of the training task setting showed that using vegetation index maps may be useful in this task. However, in the absence of infrared images using only the image in the visible range shows the comparable result. Multiclass training did not show any advantages.

In general, the obtained results show the perspective of UNet architecture neural networks application for solving the tasks of large-scale automated agricultural monitoring using freely available satellite data of medium spatial resolution (10 m/px in the considered case of Sentinel-2). Further development of this work can be the construction of more relevant metrics (and, accordingly, loss functions) for the task, using more multispectral information and historical images of the same parcels to improve the accuracy and relevance of recognition results.

References

[1] S. M. Borzov , M. A. Guryanov , and O. I. Potaturkin , “ Study of the classification efficiency of difficult-to-distinguish vegetation types using hyperspectral data,” Computer Optics , vol. 43 , no. 3 , pp. 464 - 473 , 2019 .

[2] S. A. Bibikov , N. L. Kazanskii , and V. A. Fursov , “ Vegetation type recognition in hyperspectral images using a conjugacy indicator,” Computer Optics , vol. 42 , no. 5 , pp. 846 - 854 , 2018 .

[3] A. A. Varlamova , A. Y. Denisova , and V. V. Sergeev , “ Earth remote sensing data processing for obtaining vegetation types maps ,” Computer Optics , vol. 42 , no. 5 , pp. 864 - 876 , 2018 .

[4] G. M. Musyoka , “ Automatic delineation of small holder agricultural field boundaries using fully convolutional networks , ” Master's thesis , University of Twente, Enschede, The Netherlands, 2018 .

[5] M. M. Alemu , “ Automated farm field delineation and crop row detection from satellite images,” Master's thesis , University of Twente, Enschede, The Netherlands, 2016 .

[6] A. Y. Denisova , A. A. Egorova , V. V. Sergeev , and L. M. Kavelenova , “ Requirements for multispectral remote sensing data used for the detection of arable land colonization by tree and shrubbery vegetation ,” Computer Optics , vol. 43 , no. 5 , pp. 846 - 856 , 2019 .

[7] M. Drusch , U. Del Bello , S. Carlier , O. Colin , V. Fernandez , F. Gascon , B. Hoersch , C. Isola , P. Laberinti , P. Martimort et al., “Sentinel-2: Esa's optical high-resolution mission for gmes operational services , ” Remote sensing of Environment , vol. 120 , pp. 25 - 36 , 2012 .

[8] N. Pettorelli , J. O. Vik , A. Mysterud , J.-M. Gaillard , C. J. Tucker , and N. C. Stenseth , “ Using the satellite-derived ndvi to assess ecological responses to environmental change,” Trends in ecology & evolution , vol. 20 , no. 9 , pp. 503 - 510 , 2005 .

[9] X. Jinru and B. Su , “ Significant remote sensing vegetation indices: A review of developments and applications , ” Journal of Sensors , vol. 2017 , pp. 1 - 17 , 2017 .

[10] M. Boori , K. Choudhary , and A. Kupriyanov , “ Crop growth monitoring through sentinel and landsat data based ndvi time-series,” Computer Optics , vol. 44 , no. 3 , pp. 409 - 419 , 2020 .

[11] J. Davidse , “ Semi-automatic detection of field boundaries from highresolution satellite imagery . ” Master's thesis , Wageningen University, The Netherlands, 2015 .

[12] A. Garcia-Pedrero , C. Gonzalo-Mart´ın, and M. Lillo-Saavedra, “A machine learning approach for agricultural parcel delineation through agglomerative segmentation ,” International journal of remote sensing , vol. 38 , no. 7 , pp. 1809 - 1819 , 2017 .

[13] O. Ronneberger , P. Fischer , and T. Brox , “ U-net: Convolutional networks for biomedical image segmentation,” International Conference on Medical image computing and computer-assisted intervention , Springer, pp. 234 - 241 , 2015 .

[14] K. He , G. Gkioxari, P. Dolla´r, and R. Girshick , “ Mask r-cnn, ” Proceedings of the IEEE international conference on computer vision , pp. 2961 - 2969 , 2017 .

[15] D. P. Kingma and J. Ba , “ Adam: A method for stochastic optimization , ” arXiv preprint arXiv:1412.6980 , 2014 .