The paper presents the results of the first version algorithm development for sea ice binary classification by SAR images. To create training, test, and validation samples, we have used 81 images acquired with the Sentinel-1 radar for the area of the Pechora Sea (the south-western part of the Barents Sea) for the 2019-2020 ice period. We conducted the preprocessing procedure for each image aimed at better image quality, noise removal, including gradient noise, and geospatial reference. The marking images was carried out semi-automatically using the K-means clustering algorithm. The result of clustering is a bitmap file with a class number assigned to each pixel. The raster was then vectorized and the expert manually divided the resulting vector polygons into water and ice classes. Validation images were monitored using a set of metrics with the following average result achieved: 0.86 (Jaccard), 0.14 (Binary Crossentropy), 0.90 (Precision), 0.95 (Recall). Expert analysis of binary classification errors has shown that they are typical for the periods when ice is being actively formed or destructed, which results in alternating small areas of ice and open water offshore.
The economic development of the Arctic Region requires increased attention to maintaining safe
production operations and maritime logistics. The operations on active offshore oil carbon development
and the expansion of the Northern Sea Route are complicated by unfavorable ice conditions which
significantly impede ship traffic in the frozen water area. This requires safe maritime logistics. For the
operational analysis of the current ice situation in the Arctic seas, remote sensing data from space are
mainly used because of their large one-time coverage and high survey frequency. However, due to
frequent cloudy sky and polar night, it is often impossible to use conventional optical images, which
makes conditions for the use of SAR (or radar - another term) images more complex to be interpreted
[
SAR images have their own features related to data acquisition technology. These include sidelooking geometry which causes geometric distortions in images as well as the presence of speckle noise.
2021 Copyright for this paper by its authors.
It is also important to account for how signal reflection depends on surface characteristics and
conditions at the time of the acquisition, particularly when studying seas and oceans [
Sea ice is a rather complex natural object for SAR image interpretation. Even with expert analysis,
an error in ice interpretation can be quite high, especially during the period of ice formation and
destruction. This is because at certain life stages, age-specific forms of ice can be displayed on images
in the same way as open water areas [
Since sea ice drifts in a very dynamic way, data processing critically depends on the speed of analysis
results on ice edge position in the water area (in particular, during active navigation). Traditional
methods mainly rely on expert analysis when a sea ice specialist draws a water/ice boundary as a result
of visual interpretation. Sometimes this takes several hours, which can be critical for safe navigation.
To increase the speed of SAR image processing, the global academic community has been developing
algorithms for the automatic interpretation of sea ice based on radar images with the use of neural
network technologies [
Neural networks (NNs) and machine-learning algorithms have been used to interpret space optical
images for quite a long time [
The main areas for the use of neural network technologies and machine-learning algorithms include
detection of water/land boundary (ice edge) [
This paper presents the experience in the binary classification of radar images for identification of ice field boundaries with due account for particular features of ice cover formation and destruction based on the open data from the Sentinel-1 radar.
As source data we have used Sentinel-1 images available in the public domain (https://scihub.copernicus.eu/dhus/#/home), HH-polarized in the Extra-Wide Swath Mode (Level-1 GRDM) and with a spatial resolution of 40 m. The key study area was the area covered by the Pechora Sea (the south-western part of the Barents Sea) where there was a fairly high intensity of vessel traffic and dynamic conditions for ice formation.
The algorithms for binary classification of sea ice by radar images were developed in several stages: Development of the SAR images dataset for the 2019–2020 ice season; SAR image pre-processing; SAR image semi-automated marking; Training the neural network (NN); Quality analysis of the results obtained.
At stage I, we analyzed the data on ice concentration in the Pechora Sea during the 2019–2020 ice period according to the data presented on the website of the University of Bremen, Germany (https://seaice.uni-bremen.de/databrowser/). It was established that the 2019–2020 ice period had taken place from October 26, 2019, to June 1, 2020. For the concerned period, we obtained 150 Sentinel-1 images at intervals of 1–2 to 5–7 days.
During the 2019–2020 ice season, there are four stages with indistinct time boundaries: 1. Ice cover formation from late October to late November 2019. 2. Ice cover stabilization with periods of its destruction during air temperature increases to nearzero ones from the end of November 2019 to mid-January 2020.
Relatively stable and cohesive ice cover during the period of maximum fall in temperature from mid-January to mid-February 2020.
Alternating ice cover stabilization and destruction periods caused by rises/drops in temperature from mid-February to early April 2020.
Ice cover destruction due to the general trend for air temperature increase from the beginning of April 2020 to the end of the ice period.
A preliminary analysis of the collected SAR images has shown a complex and highly variable ice situation in the Pechora Sea, which is a complicating factor for the construction of SAR classification system with the use of NN.
At stage II, we performed preliminary processing of radar images. Each image was preprocessed with the free ESA SNAP software (http://step.esa.int) using the graph shown in the figure below (Figure 1). The first stages in the processing graph (in Figure 1 these are ThermalNoiseRemoval, Apply Orbit File, Calibration, LinearToFromdB) include standard SAR images preprocessing operations which are required to remove technical noise, update information on satellite position at the time of the survey, calibrate and convert to physical quantities (decibels).
To make geometry corrections in the BandMath block, the following formula was used (from the paper by [16]):
[dB] = [dB] + 1(33 − θ0) (1) where b1 is the coefficient obtained from the regression analysis of the incidence angle—dispersion factor dependence for images in the study area, θ0 is the incidence angle, σ° is the backscattering coefficient.
At the Speckle Filter stage, we used the IDAN filter with such parameters as Number of looks = 1 and Adaptive neighbor size = 50 to remove speckle noise (image graininess which typical for SAR images). In the Undersample block we used the following parameters: Sub-sampling, 1, 1. To convert to geographic coordinates, at the Ellipsoid Correction GG stage we applied the following parameters: Billinear interpolation, WGS84 (DD). In the Convert Datatype block, we converted the data type of image pixels to uint8 with the Scaling = linear parameter (between 95% clipped histogram) and replaced nodata values with 255. The preprocessing was carried out on the Yandex.Cloud server. The resulting image was saved in the Geotiff format in the Geographic projection WGS 84, EPSG 4326.
An example of SAR image before and after preprocessing is shown in Figure 2. One can see that the performed operations managed to eliminate the gradient noise (the right edge of the image is highlighted) which would have significantly distorted the results of automatic binary classification.
Further preprocessing was carried out using the tools of the GDAL, Numpy and OpenCV libraries of the Python programming language. It included reprojection into UTM 40 N WGS 84, EPSG CODE 32640, land image removal based on vector layer (mask) with sea/land boundary. The results of this stage make the source data for neural network operation.
At stage III, the images were marked. A marking sample consisted of 58 images evenly distributed over the ice season. Additionally, 23 images were selected to test the model's operation. These images were also evenly distributed over the ice season and combined all ice features on RIs.
The marking was carried out using an automated method. The image obtained after preprocessing was bilaterally filtered with the use of a filter implemented in the OpenCV library with a neighborhood size of 7, inter-pixel distance of 15, and a color difference of 15.
Then we used the K-means algorithm to create an image clustered into 7 classes and vectorized this image in the Shapefile format. Further, we selected manually the areas which fell within the Clean Water/Nilas and Ice classes. These areas were combined into a single coverage for each class. Based on image edges, the data-deprived region was labeled as a separate class. As a result, we created a vector image file in the .shp format which completely covered the entire image with an attribute table having the Class field with values of 0 for No Data, 1 for the Pure Water/Nilas class, and 2 for the Ice class. An example of an image after preprocessing and clustering as well as the marking result is shown in Figure 3.
Based on the operating results we generated a dataset consisting of 81 fully processed SAR images, with deleted land areas and SAR images parts being beyond the Pechora Sea border. The dataset included 58 images with the appropriate marking which were intended to be used as training and validation samples as well as 23 images for testing samples without the appropriate marking. Among 58 marked SAR images, 12 were selected to be used as a validation sample. Thus, the SAR images were distributed as follows: 46 for training, 12 for validation, 23 for test samples.
To select the most optimal NN architecture at stage IV, we conducted a set of experiments that covered two basic architectures, which were most effective for image segmentation purposes: UNet; FPN. and several different encoder architectures, including: SEResNeXt50; EfficientNetb7.
The experimental results were evaluated using a set of metrics for NN predictions on validation images as well as visually for predictions on validation and test images. Based on the evaluation results, we selected the UNet architecture with the EfficientNetb7 encoder.
NN training was performed with the use of the following hardware and software: NVidia GeForce GTX 1060Ti video card; Ubuntu 18.4 operating system; Python 3.7 programming language; Segmentation Models PyTorch and TensorFlow 2.0 libraries.
In the process of training, we used 256x256 pixel regions (crops) randomly selected and prepared for each image—a total of 8,200 crops for training and 2,400 crops for validation.
The training of the NN with the selected architecture was conducted for 20 epochs, which took 7.2 hours. For training, we used a combined loss function, including the Jaccard index and binary crossentropy. An additional quality control was carried out according to the Jaccard index. The training results for 8,200 + 2,400 crops are shown in Figure 4.
Let us consider the obtained results using the most complex images for the period of ice formation, stable position, and destruction.
On the SAR image dated November 2, 2019 (Figure 5), there are prevailing errors in ice recognition as water; the errors are concentrated in a heterogeneous mass of newly-formed ice in the Pechora Bay. The wavy sea surface is correctly defined as water.
On the Sentinel-1 image dated November 19, 2019 (Figure 6), there are common errors in the areas of low-concentrated ice and they are characterized as false recognition of water as ice.
On the SAR image dated January 25, 2020, there are few errors; incorrect recognition of water as ice prevails (Figure 7). Geographically, they are concentrated in the ice massif broken by currents to the west of Lake Vaygach as well as in Khaypudrskaya and Perevoznaya bays.
On the Sentinel-1 image dated March 7, 2020 (Figure 8), the problem areas are concentrated in the coastal part of the image where errors of type 1 and type 2 occurred, along the fuzzy edge of the ice massif (errors of type 1) as well as in the water area to the northwest of Lake Vaygach, similar in appearance to the coastal areas in the southern part of the radar image.
On the SAR image dated April 6, 2020, type 2 errors prevail (Figure 9). The NN recognizes the water areas as ice in the large waterway to the east of Lake Vaygach, in coastal openings, and in newlyformed ice mass broken by currents in the outer part of the ice massif.
Thus, one can see that the errors are mostly associated with particular features for displaying different stages of sea ice on radar images as well as with combined open water areas and ice fields, individual ice floes, and initial forms of ice. Figures 5–9 show the most complex cases of sea ice display on SAR images which can be often incorrectly interpreted by experts. Most of the remaining image array represents simpler situations when the ice edge is clearly separated from the water surface.
The results obtained in this paper make it possible to conclude that radar images can be very accurately and reliably processed using neural network technologies to detect ice cover edge in the water area.
Such a natural object as sea ice, which is difficult for automatic interpretation, requires more complex algorithms which will determine on input the season when the image was obtained and typical ice phases with due account for the current hydrometeorological situation and the physical and geographical features of the water area. The maximum accuracy of binary classification is only achievable when all the factors determining ice display on SAR images are included in the model
[15] F. S. Hass, A. Jokar Deep Learning for Detecting and Classifying Ocean Objects: Application of YoloV3 for Iceberg–Ship Discrimination, ISPRS International Journal of Geo-Information 9(12) (2020) 758–772. [16] M. Mäkynen, J. Karvonen Incidence Angle Dependence of First-Year Sea Ice Backscattering Coefficient in Sentinel-1 SAR Imagery Over the Kara Sea, IEEE Transactions on Geoscience and Remote Sensing 55(11) (2017) 6170–6181. doi: 10.1109/TGRS.2017.2721981.