This paper introduces a novel AI-driven platform that integrates satellite imagery, unmanned aerial vehicle data, and Automatic Identification System signals into an analytics pipeline for hazardous object detection in maritime and coastal environments. The system leverages YOLO11 for object detection and a knowledge graph based on Blue Brain Nexus to achieve semantic interoperability. Results demonstrate the ability of the developed information technology to detect maritime debris, oil spills, and vessel activity, while enabling adaptive route planning and decision support. This approach provides a scalable framework for emergency response and environmental monitoring, aligning with current advances in Artificial Intelligence, machine learning, and applied modeling in information technologies.
The frequency of natural disasters, maritime accidents, and climate-related hazards has increased
significantly in recent years [
This paper presents a multimodal AI platform that merges satellite imagery, UAV data, and Automatic Identification System (AIS) signals into a unified knowledge graph, enabling near-realtime hazardous object detection and decision support. The goal of this paper is to demonstrate the feasibility of multimodal fusion for maritime monitoring, present a knowledge-driven architecture for semantic interoperability, and validate the system.
Research on AI for remote sensing has expanded rapidly [
Relevant data, offered by monitoring services, can be classified by source:
Copernicus Marine Service (CMEMS): provides comprehensive sea surface temperature, salinity, and currents data, along with marine ecosystem data and sea ice concentration and extent measurements.
European Marine Observation and Data Network (EMODnet): offers bathymetry, seabed habitats, and human activities information, complemented by oceanographic data including tides and currents, as well as geological, biological, and chemical datasets.
National Oceanic and Atmospheric Administration (NOAA): contributes ocean temperatures, salinity, and currents data, marine debris monitoring and habitat mapping services, and weather and sea surface data collected from buoys and drones.
Ocean Observatories Initiative (OOI): delivers ocean temperature, salinity, and chemical properties data, seafloor imaging and topography information, and biological observations.
To collect necessary data, MariNeXt service [
This study presents the development and validation of a UAV-based object detection system for maritime surveillance applications. While the broader multimodal data fusion framework encompasses satellite imagery (Sentinel-1 SAR, Sentinel-2), Automatic Identification System (AIS) data, and knowledge graph integration, this paper specifically focuses on the UAV component utilizing YOLOv11 object detection algorithms.
The proposed information technology implements a five-component modular pipeline architecture (Figure 1). The data ingestion layer aggregates heterogeneous sources including Copernicus Sentinel Hub API, UAV-mounted sensors, and AIS data streams. Data preprocessing is performed through the QueryOptima™ platform for harmonization and normalization. A Blue Brain Nexuspowered knowledge graph manages semantic data storage and interoperability. The analytics layer integrates YOLOv11 object detection with graph-based reasoning, while the visualization component provides real-time dashboards and geospatial interfaces for end-user interaction.
A Minimum Viable Product (MVP) of the system was developed and tested during the Case Study: 8th CASSINI Hackathon (November 2024) at the POLE Product Design Center, Ukraine.
Sentinel data Drone data
Hazard detection
Visualization
External API Sources https://api.myshiptracking.com/ Ship detection Using YOLO11 Pollutant detection Using MariNeXt
Risk
Assessment 1. Predict high-risk areas: Piracy, pollution. 2. Generate actionable: insights for operators.
The core contribution of this work centers on the development of a specialized UAV-based
maritime object detection system employing the YOLOv11 architecture. The detection model was
trained exclusively on the High-Resolution Ship Collections 2016 Multi-Scale (HRSC2016-MS)
maritime dataset [
Dataset Characteristics and Preprocessing. The HRSC2016-MS dataset comprises 1,680 high-resolution optical remote sensing images containing 7,655 annotated ship instances. The dataset exhibits comprehensive environmental diversity, encompassing maritime scenes across multiple operational conditions: sea and coastal environments, diurnal and nocturnal imaging scenarios, and varied meteorological conditions including clear and cloudy weather patterns. The dataset’s multi-scale nature provides images with varying resolutions and aspect ratios, essential for training robust detection algorithms capable of identifying vessels across different scales and perspectives.
YOLOv11 Model Training and Fine-tuning. The YOLOv11 object detection architecture was selected for its demonstrated superiority in real-time object detection tasks and computational efficiency suitable for UAV deployment scenarios. The model was trained end-to-end on the HRSC2016-MS dataset, which was split into 60% for training, 20% for testing and 20% for validation, using transfer learning from pre-trained weights, with specific hyperparameter optimization for maritime object detection. Fine-tuning procedures incorporated UAV-specific imagery to enhance detection performance in coastal operational environments. The training process employed data augmentation techniques including geometric transformations, photometric adjustments, and multi-scale training to improve model generalization across diverse maritime conditions. Model convergence was monitored through validation metrics including precision, recall, and mean Average Precision (mAP) at multiple Intersection over Union (IoU) thresholds.
While this paper focuses on the UAV detection component, the broader system integrates with semantic knowledge representation through Blue Brain Nexus, enabling cross-modal data fusion and reasoning capabilities. The knowledge graph architecture facilitates semantic interoperability between UAV detection results, satellite imagery analysis, and AIS tracking data, supporting comprehensive maritimesituational awareness and risk assessment applications.
The system was validated through three primary use cases: (1) detection and classification of oil spills, debris, and environmental anomalies using MariNeXt; (2) ship detection using Sentinel-1 SAR imagery (trained on SAR-Ship-Dataset2) split into 60% for training, 20% for testing and 20% for validation that was cross-validated with UAV imagery; and (3) prediction of vessel routes with anomaly detection using AIS data.
The comparison highlights that the YOLO11 model, fine-tuned with UAV imagery, demonstrates reliable ship detection in coastal environments. While some false positives occur, the overall bounding box alignment with labeled data confirms high precision (Figure 2).
(a) Figure 2: Comparison of labeled (a) and predicted (b) ships. (b)
The Figure 3 illustrates the evolution of validation metrics during model training. The initial drop in precision is an artifact of the learning rate scheduler and optimizer state at the beginning of the training. Precision rapidly stabilizes around 0.9, while Recall converges near 0.75. The mean Average Precision at Intersection over Union threshold 0.5 reaches approximately 0.85, and the stricter metric mAP@[0.5:0.95] converges to 0.66. These results indicate consistent model improvement across epochs and confirm robust generalization for maritime object detection.
All losses decrease steadily, with the most significant reduction observed during the first 20 epochs, after which the curves gradually converge and stabilize. The absence of divergence between training
and validation losses indicates that the model does not suffer from overfitting and generalizes well to unseen data (Figure 4).
Therefore, the main results of the information technology validation can be summarized as follows:
Detection Accuracy: The fine-tuned YOLO11 model achieved a precision of 90% and a recall of 75%, demonstrating robust performance in identifying maritime objects with UAVenhanced data.
Processing Latency: The analytics pipeline maintained an average processing time of under 5 seconds per frame, ensuring near-real-time performance suitable for time-critical applications.
Data Integration: The platform successfully integrated multimodal data sources (satellite, UAV, and AIS) into a unified knowledge graph using Blue Brain Nexus, enabling effective semantic linking and querying.
The MVP was awarded third place at the national level of the 8th CASSINI Hackathon competition.
The validation of the proposed information technology confirmed its strengths, including the provision of analytics with low latency, semantic interoperability through a knowledge graph, and a scalable pipeline applicable to multiple domains such as maritime, coastal, and disaster response. However, a few challenges were identified. These include limitations in data access, such as restricted AIS feeds and cloud dependency, the high computational demand of near-real-time pipelines, and the limited availability of labeled datasets for training YOLO11 on maritime hazards. Future work should focus on expanding the platform’s capabilities beyond maritime safety to address other disaster types like floods and wildfires. Furthermore, important future developments include integration with European Data Spaces initiatives and ensuring compliance with FAIR data principles3 for open science.
The proposed information technology demonstrates the feasibility of an AI-driven multimodal data fusion system for maritime safety and environmental monitoring. By combining satellite, UAV, and AIS data within a knowledge graph architecture, the system provides near-real-time hazardous object detection and decision support. The fine-tuned YOLO11 model achieved a precision of 90% and a recall of 75%, with a mAP@[0.5:0.95] of 0.66, demonstrating robust performance. Data owners will benefit from the developed information technology by maximizing the value of their data assets to provide high-quality insights for maritime security.
The ship detection model from Sentinel-1 SAR data performs exceptionally well achieving 93% precision and 92.5% recall. This demonstrates high accuracy in identifying ships with minimal false positives, critical for cross validation with AIS third party API data.
The results from the CASSINI Hackathon validate its effectiveness and highlight the potential of such approaches in broader applications, including emergency response, sustainable maritime operations, and environmental protection.
This work is based on the DroneSight project, which was awarded third place in the Ukrainian national selection of the 8th CASSINI Hackathon: EU Space for Defence and Security4. The authors gratefully acknowledge the CASSINI Hackathon, an initiative of the European Union, for providing the framework and premium access to satellite imagery that were instrumental to this research. Rework-Space LLC acknowledges support from the European Innovation Council through the Seeds of Bravery (UASEEDs) project.
During the preparation of this work, the authors used GPT-4 and Grammarly in order to: Grammar and spelling check. After using these tools/services, the authors reviewed and edited the content as needed and take full responsibility for the publication’s content.