With an increase in weather-related calamities, the importance of effective disaster management has never been more critical. Leveraging cutting-edge technologies has become a necessity in this area. The TEMA project, supported by the European Union, aims to enhance the handling of natural disasters through the automation of detailed semantic 3D mapping and forecasting of disaster progression. This project will amalgamate a wide variety of extreme data sources for analysis and will create a comprehensive, pioneering platform for managing natural disasters. TEMA's emphasis will be on real-time semantic extraction from diverse data types and sources, which will be used to create a continuously updated 3D map of the disaster zone, complete with semantic annotations. This will provide personnel with the ability to visualize the disaster area and assess various response strategies through simulations.
from
multiple disparate sources, including smart drone and on-site sensors, remote sensing data, topographic and weather data/predictions, and geosocial media content. TEMA's focus will be on the extreme nature of this data, characterized by diverse resolution and quality, large volume and frequency of updates, different spatiotemporal resolutions and acquisition rates, real-time requirements, and multilingualism. It aims to create a revolutionary NDM platform that will extract semantics from multiple heterogeneous data forms and sources in real-time. It will construct a semantically annotated 3D map of the disaster zone, predict the course of the disaster, and enhance communication between service providers and end users. This will be accomplished through automated process initiation and response recommendations. Semantic analysis computations will be distributed along the edge-to-cloud continuum in a federated fashion to minimize latency. Trustworthy and transparent extreme data analytics will be conducted by significantly advancing AI and XAI methodologies. The continuously updated 3D map and disaster progression predictions will serve as the foundation for an advanced interactive Extended Reality (XR) interface. Here, the current situation will be visualized, and various response strategies will be dynamically assessed through simulation by NDM personnel. The
2020 Copyright for this paper by its authors. innovative, scalable, and efficient TEMA platform will provide precise NDM support, based on extreme data analytics.
TEMA is supported by a pan-European consortium of 19 partners who are experts in all relevant subfields, such as data analytics, AI/machine learning, remote sensing/Earth Observation, federated and cloud computing, fire/flood modeling, geovisual analytics, AR, and NDM. These partners, who hail from leading research laboratories with substantial international R&D experience and from various parts of Europe, are well-equipped to take on this challenge.
Trustworthy AI. Project TEMA will advance Explainable AI (XAI) by integrating innovative methods applicable to extreme data situations. These methods will cater to both general multimodal analysis from diverse sources and specific vision-based scenarios. TEMA will also incorporate cuttingedge Out-of-distribution (OOD) detection algorithms and geometric training regularizers for strengthening DNN robustness. Thus, TEMA is set to enhance XAI for multimodal analysis and bolster AI sturdiness.
Real-time Semantic Visual Analysis. Considering the current state of real-time visual analysis for extreme data during emergencies, TEMA aims to elevate the standard by incorporating innovative and rapid DNN-based real-time semantic visual analysis methods tailored for the project's use-cases. TEMA also aspires to tackle the problem of data scarcity during DNN training. In order to enhance the adaptability of trained DNNs across different imaging sensors and acquisition platforms under diverse atmospheric conditions and scene properties, TEMA will provide domain adaptation strategies. This will improve DNN generalization across different geographical areas when examining satellite data, and will ensure better 3D smoke reconstructions in real-time and advanced DNN-based visual privacy preservation methods.
Geosocial Media, News, and Text Analysis. While DNNs are at the forefront of sentiment analysis, accuracy can significantly decrease when dealing with complex texts in social media posts. TEMA will integrate swift semantic social media/news post analysis methods, capable of accurate sentiment analysis in intricate text, correctly identify the topic while considering accompanying images, and assign a relevance score to the post/article.
Federated Analytics. The TEMA solution will facilitate smart management of federated data for NDM. Its analytics will operate on a new edge-to-cloud continuum, allowing dynamic and transparent distribution of AI/DNN inference workload for large affected areas. This continuum will analyze data from varied sources in real-time.
Near-real Time Phenomenon Modeling. TEMA will expedite existing forest fire and flood modeling engines to near-real-time using strategies such as parallelization and ghost cells. It aims to enhance data collection from a diverse range of sources/modalities and continuously incorporate the latest fused information. The information fed to the modeling engines will be derived from a broad range of georeferenced data sources which will be fused swiftly and in near-real-time.
Decision Support for Remote Sensing. TEMA will surpass current restrictions of using remotesensing aids for operational and decision-making processes. It will offer an innovative decision support service relying on fully automated processing of public WWW data and TEMA modeling outputs. This service will notify human operators about impending emergencies faster and less labor-intensively than current methods.
Response Planning and Recommendations for Optimal Sensor Placement. To enhance semantic mapping and situational awareness, TEMA will provide near-real-time recommendations for mission/path planning for supported drones and ground units. It will also facilitate interactive exploration of contingent response alternatives in this map.
XR based Interactive Visualization. TEMA aims to boost the usage of Digital Twins in NDM and overcome the limitations of current geovisual analytics. This will be achieved by combining a georeferenced Digital Twin and a geospatial map constructed on-the-fly and in real-time during the emergency, resulting in a 3D area map with semantic annotations, predictions, decision proposals, and recommendations derived automatically from the other TEMA components. It will ensure an optimal user experience for the human operator through real-time interactive visualization of the annotated, multi-view, high-resolution 3D map via an AR interface and a complementary desktop GUI.