Interpreting morphological features of the seabed is a labor-intensive task for marine geologists especially when it concerns extensive portions of seabed. By applying Machine Learning (ML) techniques from the field of computer vision, it is possible to significantly streamline this process, speeding it up considerably. In this paper we present a model capable of automatically categorizing seabed features, identifying diferent morphological elements, such as submarine canyons, escarpments, canyon headwalls and mass movements. This model will serve as the basis for new tools to assist geologists as well as stakeholders dealing with management of coastal or ofshore areas in their work, providing them with an eficient support for seabed analysis and characterization.
seabed interpretation can be considered as a special case of image segmentation [3], where images are replaced One important task for marine geologist is the identifica- with a map of seabed features and the labeling assocition of the morphological characteristics of the seabed. ated to each pixel corresponds to the morphological feaUnderwater morphological features, such as canyons, es- ture of the seabed in the specific latitude and longitude. carpments, are components of the seabed environment. Hence, standard image segmentation techniques can be These elements are typically shaped by geological pro- employed and adapted for this task. In this paper we cesses, including erosion, sedimentation, and tectonic present the first results of using a U-Net [4] architecture activity, over extended periods. Detecting their occur- for seabed classification trained using the data obtained rence and characteristics play a crucial role in assessing via the MaGIC project (Marine Geohazards along the Italmarine hazards or when placing communication cables at ian Coasts) [5, 6], which produced maps of the Italian seabed [1, 2]. However, detection and mapping of these coasts of Central and South Italy, Sicily, Sardinia, and underwater morphological elements require specialized Liguria in a five-year time frame starting from 2007. domain expertise, such as marine geologist. Despite this The paper is structured as follows: in Section 2 the process being very time-consuming for scientists, nowa- current state if the art in image segmentation and the days there is still no automated method to detect and main groups of existing techniques are presented. In classify the various elements of the underwater environ- Section 3 the available data are presented and the specific ment. task to be solved is further detailed. The architecture
Hence, any support to help the automatic identifica- of the network and the training process are detailed in tion of these features via machine learning (ML) would Section 4. The results are then presented in Section 5. produce a significant benefit for marine geologists, mak- Finally, in Section 6 we present the planned research ing the entire process smoother and less time-consuming. directions.
The overarching aim of this project is to develop a model capable of autonomously classifying seabed features, and thus assisting geologists in their analysis. This model 2. Image Segmentation will be designed to identify various morphological elements present in seabed data, like submarine canyons, Image segmentation is a computer vision technique that escarpments, mass movements. To do so, the task of involve partitioning a digital image into multiple segments or regions, separating meaningful objects or strucItal-IA 2024: 4th National Conference on Artificial Intelligence, orga- tures within an image from the background or other obnized by CINI, May 29-30, 2024, Naples, Italy jects, i.e., it can be considered a classification problem at * Corresponding author. pixel-level. The segmentation process divides images into † These authors contributed equally. diferent regions based on certain characteristics such as $ udilaudo@ogs.it (U. Di Laudo); sceramicola@ogs.it color, intensity, texture, or other features. In particular, (S. Ceramicola); lmanzoni@units.it (L. Manzoni) with the advent of powerful ML techniques, the features (L. 0M00a0n-z0o0n0i2)-1318-1272 (S. Ceramicola); 0000-0001-6312-7728 are learned by the data instead of being handcrafted by © 2024 Copyright for this paper by its authors. Use permitted under Creative Commons License experts.
Attribution 4.0 International (CC BY 4.0).
According to [3], there exists three groups of image segmentation: to derive via standard GIS tools two additional features that are considered useful by domain experts: the slope and the profile curvature of the seabed, that are respec• Semantic segmentation: Semantic segmentation tively the first and the second derivative of the depth. involves classifying each pixel in an image into a The corresponding data can then be interpreted as 2 specific category or class. ifelds with three features/channels (i.e., it can be directly • Instance segmentation: it extends semantic seg- interpreted/visualized as an image), each one describing mentation by not only classifying each pixel into a specific feature of the same seabed area. The data are categories but also distinguishing between difer- organized in a first image of about 2800 × 2400 pixels, ent object instances of the same category. while the second of 3000 × 3800 pixels. Each of then • Panoptic segmentation: it aims to unify semantic is cut into smaller square windows of length 100 × 100 segmentation and instance segmentation into a pixels for the training process, for a total of more than single framework. It divides an image into seman- 2100 squares. tically meaningful regions and assigns a unique label to each region, regardless of whether it cor- 3.2. Ground Truth responds to an object instance or a background category.
a human interpretations of the seabed structure i.e., a
The main traditional image segmentation techniques labels map of the same dimensions of the input 2D fields include several methods like thresholding, histograms, indicating the positions of all the elements present in watersheds, region-growing and clustering based seg- that region. In particular, the labeling is done by drawing mentation [7]. lines of diferent types over the depth maps. The diferent
As stated above, with the advent of deep learning, new types of lines corresponds to diferent classes which are techniques for image segmentation were developed [7], 97 in the original data. Due to the large number of classes with the main distinction usually being the architecture and the fact that many of them were only represented by of the neural network used. One of the first architecture a small number of samples, a first pre-processing step was used for semantic segmentation was a Fully Convolu- done by reducing the number of classes to 15. These 15 tional Network (FCN) proposed by Long et al. in 2015 [8]. classes corresponds to grouping of the original 97 classes That network includes only convolutional layers and is with the partitioning done according to the morphologiable to take an image of a certain size as input and re- cal and geological similarity of the seabed features and turns a segmentation map of the same dimensions. Other with the help of marine geologists. Hence, the output of deep learning-based models for segmentation was pro- the model has to be a 2 field in which each coordinate posed following the encoder-decoder architecture: Badri- (each pixel) belongs to one of the 16 classes (15 related narayanan et al. proposed the SegNet [9]. to the diferent morphological elements plus 1 for the
Inspired by the FCN and the encoder-decoder archi- background). tecture, V-Net [10] and U-Net [4] were proposed, initially mainly for medical and biomedical purposes. Over the years, several modification of the U-Net architecture were 4. Architecture and Training performed to adapt it to diferent kind of images. For example Zhou et al [11] proposed a nested U-Net architec- In this work we perform semantic segmentation using ture. Furthermore Cicek [12] build a U-Net architecture a U-Net architecture [19]. The network architecture is for 3 images. Nowadays U-Net are also used in other comprised of three main elements 1: ifelds, e.g., road segmentation [ 13], face detection [14], and autonomous driving [15].
The data are provided by the Italian MaGIC project are GIS data specifying the depth of the shallow coastal regions in Italy [16, 17, 18]. Starting from depth data it is possible • A contracting path that reduce the spatial di
mensions of the input image; • An expansive path that increase the spatial di
mensions; • Skip connection between corresponding layers
in the contracting and expansive part.
The specific architecture used in this study is presented in Figure 1, with the parameters of the diferent layers presented in Table 1. The contracting path has the typical structure of a Convolutional Neural Network (CNN), which in our case it consist in five layers. The first has three input channels (corresponding to the three input 512 512 I/8
Then there is the expansive path composed by five layers in which deconvolution are performed in order to upscale the image. The last layer has 16 output channels like the total number of classes of seabed’s elements. The output values are logit, which, if necessary, can then be transformed in a probability distribution over the diferent classes via softmax. features) and performs two convolutions followed by problem that is related to the intersection between the batch normalization and ReLU activation function. As prediction and the target images. stated above, the input data have a size of 100 × 100 We partitioned the dataset into an 80% − 20% split pixels. The remaining layers of this part has the same for training and testing, respectively and used a batch structure with the addition of a max pooling operation at size of 128. We employed the Adam optimizer with a the beginning. learning rate set to 0.001 and the training continued for 60 epochs.
Table 1 Notice that the actual classification is not done directly Convolutional and deconvolutional layers structure by assigning the most probable class, but by assigning a threshold for the logit value for all the 15 classes corre
Type Kernel size Padding Stride sponding to actual morphological features of the seabed. Convolution 3 1 1 If one of them is above the threshold then the assigned Deconvolution 2 0 2 classes is the one corresponding to it, even if the background (i.e., no feature present) has a higher logit value and would have been the most probable. This is done because of the strong imbalance of the dataset (the “no feature” class is present for more than 98% of the pixels) and, thus, the model would prefer selecting that class.
As stated before, the network is trained with 100 × 100 pixels images, cut from the total features map. Hence, this network take a 100 × 100 3-channels image as input and returns a 100 × 100 16-channels tensor. In this framework for each pixel we have 16 number, each one related to a specific class. The class related to the channel with the bigger number is associated to that pixel.
The loss between the output of the U-Net and the transformed labels map is computed by using a traditional Cross-Entropy loss in addition to the Dice loss [20]. The last one is a particular loss usually used in segmentation
In Figure 2 two labels maps are presented. They correspond to the first region in which the total dataset was split: in the left one there are the human interpretations of the underwater environment provided by the MaGIC project that map every relevant element in the area; the right one is the map reconstructed by the network after the training. The the white squares denote the test set that represent the 20% of the total part of the region.
In Table 2 a comparison between the frequencies of the labels of the reconstructed map and the human interpretations (the ground truth) is given. The frequencies excludes the background pixels, i.e., where no feature is present.
Recall that due to the predominance of one of “no features” class over the others, the model tends to favour this class, as typical for highly imbalanced datasets. To show that the use of threshold actually helps in improving the classification we compared the class frequencies with and without a threshold. In Table 3 frequencies of the classes of the ground truth are compared with those of the output map with no threshold and with ℎℎ = − 1.5 considering now also the background (label 0). Notice that the frequency of the pixels of the background decrease with adding the threshold and also the other frequencies are more similar to those of the ground truth.
The model trained with the data representing the region shown in Figure 2, that are only a part of the entire dataset, is used on another region to test the model. The results are shown in Figure 3: the image to the left is ground truth while the image to right is the reconstructed map with no threshold used. We can easily notice that a predominance of the background is present. In the Figure 4 results with diferent values of threshold acting on the logit of the non-zero classes are present. The reduction of the threshold value correspond to an increase in the presence of pixels belonging to non-zero labels. Clearly, the correct choice of a suitable threshold ment of the seabed in an automatic way. The current is essential for obtaining a good labeling of the seabed. results shows that, by using a U-net it is possible to pro
One important aspect to notice is that, for the proposed duce good results from a qualitative point of view, as results, the actual loss is not a good indicator of the use- long as thresholding is used in the output of the network. fulness of the results. In fact, it represents only an proxy Future goals are to improve the performance of the exof the real usefulness of the proposed labeling, since label- isting model in order to obtain more precise results. In ing by experts is itself a subjective and noisy act. Thus, particular, a new quantitative measure to encode the exobtaining a feature that is shifted by a small amount pert knowledge should be devised in order to speed-up might be immaterial (the original lines were themselves the evaluation of the model (and, possibly, as a loss funcdrawn by hand), like getting a non-continuous line (the tion for the training). Additionally, a future project will expert interpretation would be that there should be a be to construct a model capable not only of identifying “connection” between two features). Hence, most of the and mapping morphological elements of the seabed but evaluation is still qualitative and based on discussion also detecting potentially dangerous zones within it, i.e., with experts, that are particularly interested in the ability geohazards. of the model to produce the correct “general shape” of the features, more than some specific details.
identify and recognize the relevant morphological ele[1] F. Chiocci, D. Ridente, Regional-scale seafloor mapping and geohazard assessment. The experience from the Italian Project MaGIC (Marine Geohazards along the Italian Coasts), Marine Geophysical Research, 2011. doi:https://doi.org/10.100 LGRS.2018.2802944.
7/s11001-011-9120-6. [14] K. Luu, C. Zhu, C. Bhagavatula, T. H. N. Le, M. Sav[2] F. L. Chiocci, A. Cattaneo, R. Urgeles, Seafloor map- vides, A Deep learning approach to joint face detecping for geohazard assessment: state of the art, tion and segmentation, 2016. doi:10.1007/978-3 Marine Geophysical Research, 2011. doi:https: -319-25958-1_1.
//doi.org/10.1007/s11001-011-9139-8. [15] Çağrı Kaymak, A. Uçar, A brief survey and an ap[3] A. Kirillov, K. He, R. Girshick, C. Rother, P. Dollar, plication of semantic image segmentation for auPanoptic segmentation, volume 2019-June, 2019. tonomous driving, volume 136, 2019. doi:10.100 doi:10.1109/CVPR.2019.00963. 7/978-3-030-11479-4_9. [4] O. Ronneberger, P. Fischer, T. Brox, U-net: Convolu- [16] S. Ceramicola, F. Fanucci, C. Corselli, E. Colizza, tional networks for biomedical image segmentation, D. Morelli, A. Cova, A. Savini, D. Praeg, M. Zecchin, volume 9351, 2015. doi:10.1007/978-3-319-2 A. Caburlotto, O. Candoni, D. Civile, M. Coste, 4574-4_28. D. Cotterle, S. Critelli, A. Cuppari, M. Deponte, [5] Progetto MaGIC - Marine Geohazards along the R. Dominici, E. Forlin, E. Gordini, C. Tessarolo, Italian Coasts, 2007-2012. URL: https://www.protez F. Marchese, F. Muto, S. Palamara, R. Riccardo, ionecivile.gov.it/en/approfondimento/progetto-m L. Facchin, R. Romeo, Calabria Ionica (Tavola 8, agic-marine-geohazards-along-italian-coasts-0/, Fogli 35-39), in: C. FL, B. F, C. S, G. F, O. P. (Eds) accessed on 2024-04-15. (Eds.), Atlante dei lineamenti di pericolosità geolog[6] S. Ceramicola, D. Praeg, M. Coste, E. Forlin, A. Cova, ica dei mari italiani - Risultati del progetto MaGIC, E. Colizza, S. Critelli, Submarine Mass-Movements CNR Edizioni, 2021, pp. 174–195.
Along the Slopes of the Active Ionian Continen- [17] S. Ceramicola, M. R. Senatore, A. Cova, A. Meo, tal Margins and Their Consequences for Marine M. Zecchin, D. Praeg, C. Diego, C. Salvatore, Geohazards (Mediterranean Sea), Springer Interna- A. Caburlotto, D. Civile, M. Coste, R. Dominici, tional Publishing, Cham, 2014, pp. 295–306. doi:10 E. Forlin, F. Muto, A. Bosman, C. Francesco Latino, .1007/978-3-319-00972-8_26. E. Lai, D. Casalbore, E. Morelli, O. Candoni, E. Gor[7] S. Minaee, Y. Boykov, F. Porikli, A. Plaza, N. Kehtar- dini, D. Michele, R. Riccardo, L. Facchin, R. Romeo, navaz, D. Terzopoulos, Image segmentation using Golfo di Taranto (Tavola 9, Fogli 40-46), in: C. FL, deep learning: A survey, IEEE Transactions on Pat- B. F, C. S, G. F, O. P. (compilers) (Eds.), Atlante dei tern Analysis and Machine Intelligence 44 (2022) lineamenti di pericolosità geologica dei mari ital3523–3542. doi:10.1109/TPAMI.2021.3059968. iani - Risultati del progetto MaGIC, CNR Edizioni, [8] J. Long, E. Shelhamer, T. Darrell, Fully convolu- 2021, pp. 196–225.
tional networks for semantic segmentation, volume [18] C. F.L., F. Budillon, S. Ceramicola, F. Gamberi, 07-12-June-2015, 2015. doi:10.1109/CVPR.2015. P. Orrù (Eds.), Atlante dei lineamenti di pericolosità 7298965. geologica dei mari italiani-Risultati del progetto [9] V. Badrinarayanan, A. Kendall, R. Cipolla, Segnet: MaGIC, CNR Edition, 2021.
A deep convolutional encoder-decoder architecture [19] A. Milesi, Pytorch-UNet: Pytorch implementafor image segmentation, IEEE Transactions on Pat- tion of the u-net for image semantic segmentatern Analysis and Machine Intelligence 39 (2017) tion with high quality images, 2024. URL: https: 2481–2495. doi:10.1109/TPAMI.2016.2644615. //github.com/milesial/Pytorch-UNet. [10] F. Milletari, N. Navab, S. A. Ahmadi, V-net: Fully [20] R. Zhao, B. Qian, X. Zhang, Y. Li, R. Wei, Y. Liu, convolutional neural networks for volumetric med- Y. Pan, Rethinking dice loss for medical image ical image segmentation, 2016. doi:10.1109/3DV. segmentation, volume 2020-November, 2020. doi:10 2016.79. .1109/ICDM50108.2020.00094. [11] Z. Zhou, M. M. R. Siddiquee, N. Tajbakhsh, J. Liang,
Unet++: A nested u-net architecture for medical image segmentation, volume 11045 LNCS, 2018.
doi:10.1007/978-3-030-00889-5_1. [12] Özgün Çiçek, A. Abdulkadir, S. S. Lienkamp, T. Brox,
O. Ronneberger, 3d u-net: Learning dense volumetric segmentation from sparse annotation, volume 9901 LNCS, 2016. doi:10.1007/978-3-319-467 23-8_49. [13] Z. Zhang, Q. Liu, Y. Wang, Road extraction by deep residual u-net, IEEE Geoscience and Remote Sensing Letters 15 (2018) 749–753. doi:10.1109/