Raster is a common data format in satellite image processing. A raster allows us to model geographic phenomena as a regular surface in which each cell (or pixel) is associated with a phenomenon value. Many rasters can be provided for the same geographic area, for the same phenomenon at di erent dates or di erent phenomena; they can be compared, combined, used to generate a new one, etc. A recurrent issue however is to transfer data from pixels to features to qualify territorial units, which requires complex aggregation processes. This paper addresses this issue thanks to a semantic data integration process based on spatial and temporal properties. We propose i) a modular and generic ontology used for the homogeneous representation of data qualifying a geographical area of interest; and ii) a Semantic Extraction, Transformation, and Load (ETL) process that relies on the ontology and data extracted from rasters and that maps the aggregated data to the corresponding areas. We evaluate our approach in terms of the (i) adaptability of the proposed model and pipeline to accommodate di erent use cases, (ii) added value of the generated datasets in helping decision making, and (iii) approach scalability.
? Supported by H2020 grant for the CANDELA project from an H2020 EC research ?? Copyright ©2021 for this paper by its authors. Use permitted under Creative Commons License Attribution 4.0 International (CC BY 4.0).
One common data format in satellite image processing is raster. A raster
models geographic phenomena as a regular surface in which each cell (or pixel)
is associated with an indicator or a phenomenal value according to a prede ned
codi cation or classi cation. Such representations may be automatically built
by di erent types of algorithms including machine learning. Several rasters can
be provided for the same geographic area to monitor the same phenomenon
at di erent dates or di erent phenomena; they can be compared, combined,
used to generate a new one [
This paper addresses the integration of data calculated from rasters as a way of qualifying geographic areas of interest, based on their spatio-temporal properties. These areas of interest are usually represented as geospatial features in a vector format. We are interested in studying (i) the kind of ontology required to support knowledge extraction from EO data and to describe homogeneously di erent analysis results provided by the rasters; (ii) how to make accessible and usable rich EO data and (iii) how to enable the data traceability to enhance user con dence and data exploitation. The contributions of this paper are as follows: (i) a generic vocabulary that allows the semantic and homogeneous description of spatio-temporal data to qualify prede ned areas (ii) a con gurable semantic ETL process based on the proposed model (iii) an EO eco-system that allows for exploiting Sentinel images.
The rest of this paper is organized as follows. Section 1 presents the semantic model for integrating geographic areas and data extracted from rasters, through their spatial and temporal dimensions. Section 2 details the semantic data integration process. Section 3 discusses the experimental evaluation of our approach. The main related work is discussed in section 4 and nally, section 5 ends the paper and presents the perspectives for future work. 1
We propose a semantic model to represent data from rasters that provide observed properties of a geographic area of interest (land parcel, administrative unit, forest, etc.) along with their spatio-temporal dimensions. This model also allows keeping track of integrated data and image metadata. To this end, it relies on a generic and modular ontology that extends vocabularies available as OGC standards such as GeoSPARQL, and W3C recommendations such as OWL-Time1, SOSA2, DCAT3, and PROV-O4.
The model is composed of several sub-models describing the various kinds of data without the need of instantiating the whole model. These sub-models are a) a territorial observation model (for representing geospatial units and their 1 https://www.w3.org/TR/owl-time/ 2 https://www.w3.org/TR/vocab-ssn/ 3 https://www.w3.org/TR/vocab-dcat-2/ 4 https://www.w3.org/TR/prov-o/ associated observations); b) an EO model (for representing Sentinel image metadata); and c) an EO analysis model (for representing raster results produced by image processing). Due to the lack of space, only a) is depicted.
Territorial observation model (tom): The tom model (Figure 1) takes as input any output of EO analysis activities as long as it comes in raster format. A collection of observations (represented as tom:GeoFeatureObservationCollection )5 is composed of several observations (tom:GeoFeatureObservation), observing a given property (sosa:ObservableProperty ) on a given territorial . Territories, presenting a footprint on Earth, are represented by the tom:GeoFeature class that specializes the sosa:FeatureOfInterest and geo:Feature classes. They belong to a type (tom:GeoFeatureType), such as an administrative unit (village, county), land register parcel, agricultural parcel, forest, or geospatial data grid (such as Sentinel tiles). Each observation result in the percent value (sosa:hasSimpleResult ) covered of the property among all observed properties in the collection to which it belongs; for example, 40% of Mixed Forest and 60% of Coniferous Forest. The prov-o:Entity class allows keeping trace (using the prov-o:wasDerivedFrom property) of the raster le used to create the collections of observations on the one hand and of the vector le used to create the territorial units on the other hand (Cf the eoam module described below).
Sentinel images metadata (eom): The eom (EO model) is the part of the ontology dedicated to representing metadata of Sentinel images. It mainly describes the product (an image) as a result of a sosa:Observation. Each observation is associated with a temporal information, i.e., the date of capture (sosa:phenomenomTime) and a spatial information, i.e., the area being cap5 The sosa:ObservationCollection class is an extension proposed in a current working draft (https://www.w3.org/TR/vocab-ssn-ext/) tured (either by the geometry property of the eom:Product or through the sosa:hasFeatureOfInterest relation).
EO analysis model (eoam): Sentinel images are consumed in di erent kinds of analyses, such as machine learning algorithms that identify changes between two images. The eoam model (EO Analysis Model) provides information about the results of these activities since they are or will be consumed by the semantic integration process. Thanks to the PROV-O vocabulary, it is possible to know which Sentinel images have been used as input of a process (eoam:EOAnalysis ), and which agent (prov-o:wasAssociatedWith ) realized it. DCAT is used to catalog both the raster datasets and vector datasets. In this way, both raster les (eoam:RasterFile) and vector les (eoam:GeoFeatureFile) are considered as distributions of these datasets. A raster distribution is detailed with temporal coverage information (i.e. the dct:temporal and dct:Location properties provided by DCAT), but also the spatial resolution (dcat:spatialResolutionInMeter ). A vector distribution describes the main attributes of the les, such as the le size (dcat:byteSize), le format (dct:format ), or used CRS (dct:conformsTo). 2
The EO data analysis pipeline is depicted in Figure 2. This pipeline is composed of three main tasks: { Satellite images processing and analysis: This task consists of processing and analyzing satellite images coming from a DIAS (Data and Information Access Services). The possible analysis could be from the simple one as NDVI calculation to the sophisticated ones like change detection on time series or land cover annotation. The task generates raster les as a result. { Semantic data integration: The task extracts data from the generated raster les using vector les that contain the territorial units to be observed. Vector sources could come from open data repositories or our semantic database through Semantic search. { Semantic search: The task aims to analyze the integrated data by querying the semantic database. The SPARQL query results can be used to perform once again the rst two tasks for further analysis, either as parameters for guiding the process or as input data. The results can also be used by specialized GIS applications for detailed analyses of the whole integrated data.
The semantic data integration process can be divided into several steps, as in an ETL process. The main steps of the process are described in the following. Data extraction from rasters and vector les This step aims to extract and structure data according to di erent purposes. The process requires two sources: a vector and a raster. First, metadata are extracted. Next, the vector le is processed to extract information about territorial units of a given type. They will populate the tom:GeoFeature. The raster is also processed to qualify each territorial unit contained in the vector le. Properties of interest (e.g. mean values) are extracted through pixel aggregation or spatial masks are created to eliminate undesired areas. Currently, we distinguish two types of raster, depending on the type of their pixel values: { Categorical rasters (as for land cover): in this kind of rasters, since a pixel value represents a class (vineyard, for example), additional information is needed to decode it. For example, the value 15 is decoded as a vineyard in a CESBIO land cover raster; { Continuous rasters (as for NDVI or change indicators): a pixel value will be automatically classi ed into a level (or class) such as Very low, Low, Middle, High, or Very high.
To qualify a territory unit from a raster, either new properties (e.g., mean values) are extracted through pixel aggregation, or spatial masks are created to eliminate undesired areas.
Data transformation This step aims at transforming the processed data into
the semantic one. Templates that de ne the mappings between the extracted
data structure (in JSON) and the ontologies are used as a basis in this process.
They are usually handwritten. While di erent data translation tools exist, such
as D2RQ6, Ultrawrap7, Morph8, Ontop9, TripleGeo10 or GeoTriples11, we chose
to adapt the mapping template and processing mechanism described in [
Data load The nal step consists of importing the RDF les into the triplestore, following a materialization approach. The advantage of such an approach is to facilitate future processing, analysis, or reasoning on the materialized RDF data. 6 http://d2rq.org/ 7 https://capsenta.com/ultrawrap/ 8 http://mayor2.dia. .upm.es/oeg-upm/index.php/en/technologies/315-morph-rdb/ 9 http://ontop.inf.unibz.it/ 10 https://github.com/GeoKnow/TripleGeo 11 http://geotriples.di.uoa.gr/
3.1
Two use cases of the CANDELA project have been chosen for demonstration: { Vineyard use case: The objective of the use case is to retrieve changes in vineyards that were damaged by natural hazards such as frost or hail. The area of study is located in the Aquitaine region, in France. The vineyards of the area were reported heavily damaged by frost on the 20th of April 2017. { Urban expansion use case: The use case aims at studying the changes related to urban expansion in agricultural areas. We study changes in villages around Bordeaux city, one of the largest cities in France, and it is surrounded by agricultural areas, between 2017 and 2020.
Both use cases share a common set of raster types: { Change indicator: Change indicators, representing the probability of changes (between 0 and 1) of pixels of two Sentinel images, are obtained by executing tools from partners of the project. { NDVI: NDVI information is obtained by processing near-infrared and red sensors of Sentinel images. The output values are between -1 and 1. Since the values between -1 and 0 represent the elements composed of water, these values are set to 0 so that the rasters only contain values between 0 and 1. { Land cover: The datasets provide land cover information given an area on Earth. The CESBIO land cover datasets12 are used for our use cases. They cover the French territory with a spatial resolution of 10m2.
Regarding the territorial units (vector data), land register data is used for the rst use case while the second consumes administrative unit data: { Land register: Land register data is available from the French government data website13 in GeoJSON format or shape les. { Administrative unit data: Information of villages inside an area of interest can be obtained from the French government website14. The datasets are available in shape les and are updated yearly. Administrative units are linked to the INSEE RDF database15. 3.2
The genericity of the model is proved by the facts that: (i) The model treats in the same manner whatever the raster datasets, and their versions, as long as they exist in the right format; (ii) Di erent classi cations can be used to observe the same type of EO property; (iii) The system can consume whatever the vector source describes any territorial divisions. 12 http://osr-CESBIO.ups-tlse.fr/ oso/ 13 https://cadastre.data.gouv.fr/datasets/cadastre-etalab 14 https://www.data.gouv.fr/en/datasets/decoupage-administratif-communalfrancais-issu-d-openstreetmap 15 https://rdf.insee.fr/
Since the components of the pipeline are organized as services (python functions and docker images), the users can customize the pipeline by choosing and chaining up as many services as they like through Jupyter notebooks of the platform. { Vineyard use case: (i) we rst obtain all parcels of the Saint Emilion village from the land register data and CESBIO land cover raster for 2017. Semantic data integration is next used to integrate land cover and parcel information. (ii) Vineyard parcels inside the villages are retrieved via Semantic search. (iii) Appropriated Sentinel-2 images are used for NDVI calculation. (iv) These images are also used for change detection. (v) The generated rasters from the 3rd and 4th steps along with the vector from the 2nd step are integrated into the semantic database. (vi) Finally, semantic search can be used for analyzing all integrated information related to the vineyard of interest. { Urban expansion use case: (i) We rst select adapted Sentinel images and execute NDVI calculation. (ii) These images are also used for change detection. (iii) We obtain vector data of all villages of the Gironde department. Semantic data integration is next launched using the raster generated from the previous step and the obtained vector les. iv) Finally, we perform a semantic search to analyze the integrated data related to the villages of interest.
The semantic data integration is con gurable with a set of parameters that can be provided as raster metadata or as function parameters. Users can also provide custom thresholds for class classi cation since the results of image processing and analysis may highly depend on the scenarios.
We evaluate the result obtained using the pipeline presented in 3.3, knowing that the results highly depend on the precision of the algorithms provided by our partners.
{ Vineyard use case: Two Sentinel-2 images collected on the T30TYQ tile are used for change detection and NDVI computation; they are respectively dated 2017/04/19 and 2017/04/29: the choice of these images is based on the fact that they have very low cloud cover (0% and 15%) and the interval between these observations covers the period of study.
Figure 3 represents an overview of the change levels detected and the degradation of NDVI The Very low change level is eliminated since it's not very relevant. The NDVI degradation indicator represents the total percentage of degradation of ve NDVI levels. We also eliminated parcels having the NDVI degradation below 20% . Finally, there are 858 parcels detected as having changed, 756 parcels detected as having NDVI degradation above 20%, and 510 parcels detected in both cases. { Urban expansion use case: For NDVI calculation and change detection, it is recommended to collect images in the same period and in summer to limit the cloud cover and the in uence of vegetation growth. So, two Sentinel-2 images were collected on 2nd August 2017 and 6th August 2020 and have 0% of cloud cover. Figure 4 (right) represents an overview of the change levels detected and the NDVI levels degraded between these two dates, together with the source Sentinel images (left). We can observe that: (i) the change levels detected match quite well with the degraded NDVI due to urbanization; (ii) the more the village approaches the city, the more it is changed. The next analysis could be comparing change, NDVI and land cover information at parcel level for particular villages.
In terms of triplestore, we rst opted for Strabon16 for its many advantages.
It has a good overall performance thanks to particular optimization techniques
that allow spatial operations to take advantage of PostGIS functionality instead
of relying on external libraries [
Di erent semantic ETL proposals have addressed the transformation and
integration of (open) EO data to LOD. In [
Raster data can be represented following two approaches: either by treating the
entire raster grid as coverage or by providing procedures to extract vector objects
from the raster matrix. The rst approach relies on constructing a semantic
representation of the raster pixels so that each pixel attributes (geometry, values)
are maintained. In [
The second approach consists of extracting entities from rasters and
representing them as ontological features. These entities are sets of raster elements
(i.e. sets of pixels) that meet a certain context-dependent de nition. In [
This paper presented an approach for the integration of data calculated from rasters as a way of qualifying territorial units, based on their spatio-temporal features. We proposed a modular and generic ontology for semantically and homogeneously describing spatio-temporal data that qualify prede ned areas, together with the provenance of all sources of data. This ontology is generic enough for describing data that can be calculated from any source that conforms to a raster format. We de ned a con gurable semantic ETL process guided by this ontology. The process extracts data from rasters and links observations to territorial units through their spatio-temporal dimensions. This process produces a semantic database that can be exploited for di erent purposes. We illustrated the approach by integrating various raster datasets for two use cases of a joint project. As future work, we plan to exploit big data scenarios with the management of Natura 2000 areas19. We also consider extending the proposed approach to deal with data coming from CSV, especially, weather observation data. Finally, we would like to publish our knowledge base as linked open data for scienti c purposes. 19 https://www.eea.europa.eu/data-and-maps/data/natura-11