Monitoring of Coastal and Marine Environments

Abstract

Human and natural processes continuously threaten coastal and marine ecosystems. Human being has, since the last century, undermined the coastal equilibrium, accentuating or sometimes triggering irreversible erosive phenomena and causing salt wedge intrusion in areas where agricultural production represents a relevant inducement for the local economy. Also critical is the overexploitation of sandy shores, with the consequent alteration of the beach environment, and sometimes the disfigurement of the maritime territory, with the consequent loss of landscape and economic value. In particular, the uncontrolled release of large quantities of plastic material into the environment is increasingly threatening our seas and their marine living organisms. Aiming at addressing the challenges from the above context, the CVPR and CI&SS Labs, both afiliated with the interdisciplinary Lab Neptun-IA of the Department of Science and Technology at the University of Napoli Parthenope, started working on several tasks related to beach and undersea litter detection and recognition through Artificial Intelligence and Computer Vison-based techniques. Undersea and Beach litter, Litter recognition, Object detection, Instance segmentation, Aerial and marine drone, Smart rover, Recent reports [1], [2] emphasize how the human be- and economic value. In particular, the uncontrolled reItal-IA 2023: 3rd National Conference on Artificial Intelligence, orgaCEUR

-

1. Introduction ing has, since the last century, undermined the coastal equilibrium, accentuating or sometimes triggering irreversible erosive phenomena and causing salt wedge intrusion in areas where agricultural production represents a relevant inducement for the local economy. Moreover, in recent years, several international reports by the IPCC1 have emphasized the importance of developing economic models that are less dependent on fossil fuels. The latter is primarily responsible for rising temperatures on a global scale, which in turn are responsible for rising sea levels. In the near future, entire coastal belts may be permanently invaded by the sea [ 3 ]. In addition to these processes of physical imbalance, there are, unfortunately, those related to the overexploitation of sandy nized by CINI, May 29–31, 2023, Pisa, Italy ∗Corresponding author. CEUR htp:/ceur-ws.org ISN1613-073 carried out jointly by the Computer Vision and Pattern

Recognition Laboratory (CVPRL) “Alfredo Petrosino”, the

Computational Intelligence & Smart Systems Laboratory

(CI&SS), the High-Performance Scientific Computing

Laboratory (HPSC) and the Neptun-IA Interdisciplinary

ifcial Intelligence (AI) techniques, in particular, Machine shores, with the consequent alteration of the beach en- following protocols for the management and sustainable

Learning (ML), Deep Learning (DL) Computational In- ing 10 as the desirable drone flight above ground. The telligence (CI), and Computer Vision (CV) are applied to proposed methodology represents a benchmark for the the field of interest for the detection of anthropogenic definition of a standardized procedure for the indirect debris released in coastal and marine environments using evaluation and monitoring of the coastal environmenaerial and underwater drones. These activities follow the tal status. Besides allowing the investigation of large numerous guidelines established by MedECC (Mediter- areas with limited human efort, the proposed system ranean Experts on Climate and Environmental Change), enables the evaluation of the beach litter spatial distribuwith the support of the European Community, and aim tion and magnitude, providing useful information for the at protecting those marine and coastal ecosystems where assessment of tailored beach quality indices. Since modthe rate of pollution is increasing. The research involves els for instance segmentation require many annotated the implementation of several tasks, such as the process- images to obtain significant results, and the annotation ing of aerial and submarine images; the development process, although supported by software tools for labelof object recognition techniques; the development of ing, is extremely time-consuming, a new approach based optimization strategies for garbage collection; the devel- on HyperGraph Convolutional Networks is developed for opment of techniques for drone guidance; Virtual Real- a semantic segmentation Weakly-supervised (HyperGCNity (VR) reconstruction of real scenes captured by video WSS) [ 7 ]. Specifically, HyperGCN-WSS constructs spatial cameras. The tools have a minimal impact on the en- and k-Nearest Neighbor (k-NN) graphs from the images vironment and can be used in marine protected areas in the dataset to generate the hypergraphs. It then trains and underwater archaeological parks. The activities are a convolutional network architecture with specialized coordinated in the CVPRL “Alfredo Petrosino” CI&SS hypergraphs (HyperGCN) using some weak signal. The labs, which are respectively the Parthenope node of the outputs of the HyperGCN are called pseudo-labels, which CINI Artificial and Intelligent Systems (AIIS) lab and the are later used to train a fully convolutional network for CINI BIG Data node, for the development and implemen- semantic segmentation. The advantage of such a model tation of the AI algorithms. The HPSC lab, CINI HPC is accurate semantic segmentation with small training node, develops the HPC architectures required for the AI data sets. algorithms. The Neptun-IA lab provides the necessary Beach litter recognition is a collaboration with the expertise for coastal monitoring issues and the synergy Department of Earth and Geoenvironmental Sciences, of activities between AI and the environmental domain. University of Bari Aldo Moro, which provided the images for analysis, whereas HyperGCN-WSS is a collaboration with the MIA Laboratory of the University of La Rochelle, 3. Task descriptions France.

Furthermore, the use of CI based methodologies (i.e., 3.1. Beach litter recognition deep learning and multi-objective optimization through Beach litter monitoring [ 6 ] programs play a key role in genetic algorithms) for predicting marine debris trajecestablishing efective management measures to preserve tories by UAV and for optimal path recovery for an authe ecological, scenic, and economic value of the coastal tonomous vehicle was investigated [ 8 ]. For this purpose, areas. In this study, an innovative analysis system is pro- realistic data generated by an oceanographic model (e.g. posed for the automatic identification of beach debris on Lagrangian and particle drift models) on semi-submerged aerial-photogrammetric images acquired by unmanned bottles were studied. The methodology allows obtaining aerial vehicles (UAV) at diferent elevations. The work- the exact location of the marine debris over time and lfow (Fig. 1) is based on a Mask-RCNN model, here actu- then developing a recovery strategy to optimize the time ally used for instance segmentation tasks (Fig. 1). Test and distance of the automated catamaran that will be cases were conducted along the Adriatic sector of the responsible for the recovery of the marine debris. Apulia region (Italy), where the beaches have remarkable economic importance, attracting national and interna- 3.2. Underwater litter detection tional tourists, and ecological values, hosting species of high ecological value and protected areas. The images were acquired at two coastal sites, Torre Guaceto, a marine protected area of Apulia located on the Adriatic coast of Upper Salento, and Torre Canne, a marine site located a few tens of kilometers from Brindisi, which falls within the Regional Natural Park of the Dune Costiere, from Torre Canne to Torre San Leonardo. The results of the tests carried out in this study allowed for definThis activity aims to study, develop and apply image processing (IP), DP and CI methods for the detection of underwater debris using drones. Recent research in the iMTG (Innovative Marine Technology for Geology & Archaeology), CI&SS and Neptun-IA laboratories has aimed at developing a system capable of detecting and recognizing seafloor objects using the ARGO drone (Fig. 2). ARGO is a geophysical information-gathering drone equipped with several onboard cameras and a device (i.e.

Raspberry PI) containing the object recognition module. currently no established scientific research in this direcThe limited computational capacity of the hardware and tion involving the development and use of drones capathe need for real-time response imply the design of a ble of eficiently exploring shallow-water marine coastal model that optimizes and reduces computational and environments. Moreover, the marine drone that will be memory requirements. used in the research (ARGO) is an open project prototype,

ArgonautAI [ 9 ], is a containerized distributed process- optimized to carry out non-invasive, high-resolution ining platform for autonomous surface vehicles. The Arg- direct surveys in very shallow water areas, with almost onautAI architecture uses a cluster of single-board com- no environmental impact, allowing it to be also used in puters with diverse and diferent characteristics (com- marine protected areas and underwater archaeological puting power, CUDA GPUs, FPGAs, GPIOs, PWMs, spe- parks. cialized I/O), orchestrated using Kubernetes and a cus- Due to the lack of large datasets for underwater obtomized programming interface. The proposed solu- ject recognition tasks, a synthetic dataset of underwater tion introduces two diferent types of containers, the scenes with optical flow labels [ 14 ] has been created former managing the vehicle’s instruments (e.g., posi- to demonstrate the benefits of training a specific deep tion, attitude, environment, depth), data storage, ship-to- neural model for optical flow estimation in the considshore communication, etc., and the latter hosting mission- ered environment. Experimental comparisons between a specific software components. The proposed platform general-purpose deep neural model and the same model has been applied to AI-based marine debris detection specifically trained with the newly proposed dataset have using a hierarchical computer vision approach on hetero- confirmed an increase in the accuracy of the final estimageneous onboard computing resources. tion.

For the detection of environmental objects a system based on a Deep Neural Networks [ 10 ] has been designed 3.3. Development of digital twins of ifsorbathseedmoanritnheeASRinGgOle dSrhoonteM.TuhlteiBporxopDoesteedctaorrchmitoedcteul,rae aerial and maritime drones particular class of CNNs that combines localization and The goal of this activity is to create digital models (digital classification using a single deep neural network, thus twins) of aerial and marine drones that can be used to test limiting the explosion of the computational complexity and understand how they behave under varying weather of the network. conditions, environmental complexity, onboard sensors,

The expected results of the research are the produc- and navigation algorithms using virtual space and simution of a tool capable of innovating and automating the lation. A first prototype digital twin of the ARGO marprocess of detection and removal of solid waste in marine itime drone has already been built (Fig. 2) using the aws environments employing “explainable” decision-making robotics tool (https://aws.amazon.com/it/robomaker/). systems based on approximate reasoning [ 11, 12 ] and The digital duplicate created is a three-dimensional repdata integration methodologies [ 13 ]. The degree of in- resentation of the drone that contains all the information novation of the proposed research is high, as there is of the physical drone, including its mechanics, geomea)

b) try, sensors (Gps, side scan sonar, echosounder, camera, p900 forward scan sonar), actuators, and related data and software. The goal is to take advantage of this enabling technology both for experimental activities such as incremental testing of adding sensors to optimize navigation in the unstructured and open environment and for data collection; and for activities to predict possible anomalies, failures, and identification of unanticipated risks in the design phase of the physical prototype, this at the cost of large savings in time and cost for direct physical model creation and a risk reduction.

4. Projects

• PAS (PAESAGGI ARCHEOLOGICI SOMMERSI

DELLA CAMPANIA), progetto MISE. • Computational Intelligence Methods for Digital

Health, GNCS. • HPC-Based navigation system for Marine Litter hunting, FF4EUROHPC . • Tecniche di Machine Learning e di Soft Computing per l’elaborazione di dati MultiVARIATI (SOFTMULAN), Dipartimento di Scienze e Tecnologie, Università degli Studi di Napoli Parthenope. • Progetto Parco Archeologico Urbano di Napoli (PAUN), PON 03PE 00164, Rete Intelligente dei Parchi Archeologici (RIPA - PAUN). • Erasmus+ “Framework for Gamified Programming Education” (FGPE). • Euro-HPC H2020 “Adaptive multi-tier intelligent data manager for Exascale” (ADMIRE). • SE4I (Smart Energy Eficiency & Environment for Industry), PON, area di specializzazione: Fabbrica Intelligente. • Convenzione con Unlimited Technology srl nell’ambito del progetto H2020 “Piattaforma Logistica Integrata 4.0 - P.L.I 4.0”. • Convenzione di ricerca DIST-Università di Napoli Parthenope e Dipartimento di Scienze della Terra e Geoambientali dell’Università degli Studi di Bari “Aldo Moro”, “Monitoraggio dell’Ambiente Costiero e della Marine & Beach Litter Attraverso Metodi di Indagine Diretti, Indiretti e di Machine Learning”.

Acknowledgments

The work was supported by the fundamental contributions of the following people:

• Ciro Giuseppe De Vita and Gennaro Mellone, students of the PhD Course in ”Environmental Phenomena and Risks” cycle XXXVII, Università di Napoli Parthenope, Italy; • Vincenzo Mariano Scarrica, student of the Italian National PhD Course in Artificial Intelligence Agrifood and Environment, cycle XXXVII, Università di Napoli Federico II and Università di Napoli Parthenope, Italy; • Angelo Casolaro, student of the Italian National PhD Course in Artificial Intelligence - Agrifood and Environment, cycle XXXVIII, Università di Napoli Federico II and Università di Napoli Parthenope, Italy;

References

[1] M. D. V. L. Bonora , D. Carboni (Ed.), Eighth International Symposium on Monitoring of Mediterranean Coastal Areas. Problems and Measurement Techniques, Proceeding and Report , Firenze University Press, 2020 . doi:1 0 . 3 6 2 5 3 / 9 7 8 - 8 8 - 5 5 1 8 - 1 4 7 - 1 .

[2] MATTM-Regioni , Linee Guida per la Difesa della Costa dai fenomeni di Erosione e dagli efetti dei Cambiamenti climatici , Technical Report , Documento elaborato dal Tavolo Nazionale sull'Erosione Costiera MATTM-Regioni con il coordinamento tecnico di ISPRA, 2018 .

[3] M. Davidson , Forecasting coastal evolution on timescales of days to decades , Coastal Engineering 168 ( 2021 ).

[4] W. Cramer , J. Guiot , M. Fader , J. Garrabou , J.-P. Gattuso , A. Iglesias , M. lange, P. Lionello, M. Llasat , S. Paz , et al, Climate change and interconnected risks to sustainable development in the mediterranean , Nature Climate Change 8 ( 2018 ) 972 - 980 .

[5] A. Addamo , P. Laroche , G. Hanke, Top marine beach litter items in europe, 2017. doi:1 0 . 2 7 6 0 / 4 9 6 7 1 7 .

[6] V. Scarrica , P. Aucelli , C. Cagnazzo , A. Casolaro , P. Fiore , M. L. Salandra , A. Rizzo , G. Scardino, G. Scicchitano , A. Staiano , A novel beach litter analysis system based on uav images and convolutional neural networks , Ecological Informatics 72 ( 2022 ).

[7] J. H. Giraldo , V. Scarrica , A. Staiano , F. Camastra , T. Bouwmans, Hypergraph convolutional networks for weakly-supervised semantic segmentation , in: 2022 IEEE International Conference on Image Processing (ICIP) , 2022 , pp. 16 - 20 .

[8] V. Bevilacqua , A. D. Marino , A. Ciaramella , A. A. Biancardi , G. Budillon, P. de Ruggiero, E. D. Volpe , L. Gifuni , D. Mascolo , S. Pierini , E. Zambianchi, Computational intelligence for marine litter recovery semantic segmentation , in: Smart Innovation, Systems and Technologies, Applications of Artificial Intelligence and Neural Systems to Data Science , in press.

[9] G. Mellone , C. G. D. Vita , D. D. Sánchez-Gallegos , D. D. Luccio , G. Mattei , F. Peluso , P. P. C. Aucelli , A. Ciaramella , R. Montella , A containerized distributed processing platform for autonomous surface vehicles: preliminary results for marine litter detection , in: IEEE CPS Proceedings of PDP 2023 (Parallel, Distributed, and Network-Based Processing) , in press.

[10] A. Ciaramella , F. Perrotta , G. Pappone, P. Aucelli , P. F. , G. Mattei, Environment object detection for marine argo drone by deep learning 1266 ( 2021 ).

[11] A. Ciaramella , C. Mencar , S. Montes , S. Rovetta (Eds.), Advanced Fuzzy Realtional Neural Network, CEUR Workshop Proceedings , 2021 .

[12] F. Camastra , et al., A fuzzy decision system for genetically modified plant environmental risk assessment using mamdani inference , Expert Systems with Applications 42 ( 2015 ) 1710 - 1716 .

[13] A. Ciaramella , D. Nardone , A. Staiano , Data integration by fuzzy similarity-based hierarchical clustering , BMC Bioinformatics 21 ( 2020 ).

[14] A. Ferone , M. Lazzaro , V. M. Scarrica , A. Ciaramella , A. Staiano , A synthetic dataset for learning optical lfow in underwater environment , in: Applications of Artificial Intelligence and Neural Systems to Data Science, Smart Innovation, Systems and Technologies , in press.