This article presents ST-IIM (Spatio-Temporal Image Information Mining) - an ontology to model the spatio-temporal changes during a flood disaster event. It is integrated with the image ontology to represent the various characteristics of Remote Sensing (RS) images. The goal is to query, detect and monitor disaster affected areas from multi-temporal images by automatic reasoning. The spatial and temporal aspects were modeled using DL-safe rules based on qualitative topological relationships and Allen's interval algebra respectively. This enables to draw inferences about the temporal changes and evolution of the Land Use/Cover (LU/LC) classes during the disaster event for rapid disaster response and recovery activities.
Natural disasters such as, flood, earthquake have deep impact on human life. Such natural events cannot be prevented, but their impact can be mitigated and the loss of human life can be minimized by preparing an action plan for the entire disaster management cycle.
Remote Sensing (RS) data, captured by various Earth Observation (EO) satellites plays a very important role in a disaster event, as it provides a synoptic view of the affected area. During a disaster event, a series of RS images obtained from various RS platforms can be processed and analyzed, and the information such as flood affected area (during response phase), and Land Use/Cover (LU/LC) change over the flood duration, flood recession pattern (during recovery phase) can be extracted. However, the low-level features obtained from RS imagery are unsuitable to capture the highlevel semantic concept such as spatial patterns, spatio-temporal evolution of LU/LC classes. Hence, an ontology is useful to conceptualize the domain knowledge, and to reduce the semantic gap between low-level image features and high level spatiotemporal semantics.
This paper describes the ontology development (ST-IIM) to model the spatiotemporal changes specifically for flood disaster situation.
The ontology presented in this paper encodes the formal knowledge about the
qualitative spatial relationships between objects in an image as well as their temporal
behavior in order to model, retrieve and detect the spatio-temporal changes. The Basic
Formal Ontology (BFO2) is imported as a base ontology and the domain knowledge
(LU/LC classes) and spatial and spatio-temporal knowledge is built upon it. The
ontology (ST-IIM) is developed in Web Ontology Language (OWL-DL) and is openly
available3. The developed ontology is able to do reasoning about the subsumption
relationships among LU/LC classes, reasoning about spatial relations and
spatiotemporal changes during dynamic events such as flood disaster. The similar work has
been presented in [
To understand the spatial configuration, the spatial relationships among image regions
need to be formally defined and encoded in an ontology. The Region Connection
Calculus (RCC8) defines a set of eight jointly exhaustive and pair wise disjoint (JEPD)
topological relationships (see Figure 1 (a)) [
Also in this work, image regions are assumed to be represented by their Minimum
Bounding Rectangles (MBRs) (see Figure 1(b)). The OWL classes, Region and
BoundingBox are defined in an ontology and connected via hasBBox object property.
The MBR representation of a region divides the space into 25 partitions (4 points, 12
lines and 9 region partitions) [
The time ontology published by W3C, available on-line4 is imported in ST-IIM (reusability) as a base ontology, as it has the taxonomy of the time classes already defined. However, this basic ontology is not capable of performing reasoning over temporal intervals. For example, if events and timestamps of those events are given, this ontology will not be able to do automatic reasoning to retrieve image regions based on their temporal interval relations. Hence, SWRL rules were developed to address this issue. In this ontology, the major classes Instant, DateTimeInterval etc. are defined to capture the temporal event's instant (e.g. flood event) and the valid time duration of that event (e.g. flood duration).
In ST-IIM the Allen's temporal algebra [
During flood disaster, in response phase, the pre-flood RS imagery can be used along post-flood RS images to identify the submerged areas and underlying LU/LC classes such as, flooded roads, flooded cropland, flooded built-up. During the recovery phase (post flood disaster), a series of multi-temporal RS images captured during and after the flood, and along with the few pre-flood images of same area can be analyzed together to get more insight about the temporal behavior of flood. For example, LU/LC submergence pattern, flood recession pattern etc (as shown in Figure 2(b)). Such a information mining during recovery phase, provides very important insights for the future flood mitigation and preparedness plans.
In this work, the class DateTimeInterval is considered as an important class, which is used to denote a temporal interval during which an image region has been observed with a specific LU/LC class. The object property hasObservedLULCFirstTime and hasObservedLULCLastTime are defined as subPropertyOf hasBeginning and hasEnd respectively. These properties have a class DateTimeInterval as domain, whereas a class Instant as range. This interval is obtained by tracking an image object with a particular LU/LC class in a temporal stack of RS images. Hence, a class DateTimeInterval in ST-IIM holds the information about a persistence of an image object with a particular LU/LC class along with its temporal interval. To achieve this, two object properties hasFirstDefiningRegionOfInterval and hasLastDefiningRegionOfInterval are defined to describe the regions which are associated with the interval. These properties have range class LULCRegion, which will indicate the LU/LC observed over an interval. It may happen that, the LU/LC of a region in an image persists for some temporal interval and later changed to another class forming another temporal interval. Such temporal interval instances can be identified and connected via an object property hasPreviousTemporalNeighborhood, which connects current and previous temporal intervals of a region. 3.1. Example Scenario: Detecting the spatio-temporal interactions between a road and river during a flood event Figure 3 shows an example of some temporal intervals which are assumed to be observed during a flood event. As RS images are obtained at discrete time intervals, the temporal intervals shown for different LULC classes are discrete. The example shows, the temporal intervals of two image objects River and Road. After the flood has occurred the object road has disappeared and submerged by the spread of flooded rivera region with a new class called StagnatedFloodWater is evolved (refer Figure 2(b)). The subsequent intervals shows that the River object has started receding before the Road object resurfaces. Here, an interval I3 is connected with intervals I1, I2 via hasPreviousTemporalNeighborhood object property. The SWRL rules are developed and integrated into ST-IIM ontology to infer that, the Road and River regions in interval I1, I2 isSubmergedBy a region in an interval I3. Moreover, the topological relations among MBRs of interval defining region's can be used to find whether the region is partially or completely flooded (refer Table 1 (rules 7-9)). Representing a temporal interval as described above enables the use of Allen's temporal algebra for comparing two intervals and do reasoning5 over them. This is useful to find whether one interval is overlapping other or finishes with other. In the case of flood disaster, comparing two intervals of FloodAffectedBuilding at two different parts of the flood affected area can give the insight about the recession pattern of the flood. Similarly, the post flood evolution of the submerged roads in an affected area can be obtained by reasoning over these interval relations using rules defined in this ontology. The ST-IIM ontology to model spatio-temporal changes during a flood disaster event is presented. Although flood disaster is chosen for demonstration, the formalism is applicable to other spatio-temporal domains, where there is a need to understand the dynamically changing events. The presented ontology can be further extended to model Allen's temporal interval relations between spatial relations of image objects to understand the evolution of spatial configuration over time. 5Pellet- OWL-DL reasoner is used to validate rules and developed ontology (http://clarkparsia.com/pellet)