Spatiotemporal = Spatial × Temporal Yunpiao Bai1,∗, †, Michael Grüninger1, † 1 Department of Mechanical and Industrial Engineering, University of Toronto, Toronto, Ontario, Canada Abstract There has been a constant debate about how to integrate spatial and temporal representation together. Three-dimensionalists believe that objects only have spatial dimensions; thus, they take space and time as two separate domains, so-called ”three-plus-one” dimensional approach. Controversially, four- dimensionalists argue that objects extend on time just as they extend on space, in which space and time is one primitive domain. In this research, we attempt to harmonize both of these views and interlink the two;meanwhile we justify our choices through a set of motivating scenarios. We also present our axiomatization of spatiotemporal regions and their mereotopology using the product order of spatial and temporal mereotopologies. Keywords spatiotemporal, mereotopology, ontology, first-order logic 1. Introduction Real-world processes involve time and space in nature, and almost every knowledge-based system needs to represent spatial or temporal knowledge. There is considerable foundational work on qualitative representation and reasoning about space [1, 2], as well as relationships between physical objects and spatial regions [3, 4]. Meanwhile, temporal formalisms have also been studied extensively together with actions and events [5, 6]. To reason about a spatial object, it is not only essential to represent its location and spatial relations to other objects in time but also changes in the spatial aspects over time, such as motion. Thus, a practical concern of spatial and temporal reasoning is to deal with the historical and spatial changes of objects and hence the emerging need for developing spatiotemporal hybrids. Studies over the past decades have established a solid foundation of formal ontologies of both space and time. However, there is a continuing debate about how time should be incorporated into space. There are often two distinguished views of space and time integration [7]. The first one takes space and time as two separate domains, so called “three-plus-one” dimensional approach. In this view, the world is described as a sequence of snapshots, and the existence of an object is described as an entity existing at a given time has a spatial location. The second view recognizes time and space as primitive and captures continuous changes and processes as fluents, also called four-dimensional approach. The state of the world is represented as an entity exists at a slice of spacetime. This approach always couples spatial with temporal regions when The Eighth Joint Ontology Workshops (JOWO’22), August 15-19, 2022, Jönköping University, Sweden ∗ Corresponding author. † These authors contributed equally. Envelope-Open wbai@mie.utoronto.ca (Y. Bai); gruninger@mie.utoronto.ca (M. Grüninger) © 2022 Copyright for this paper by its authors. Use permitted under Creative Commons License Attribution 4.0 International (CC BY 4.0). CEUR Workshop Proceedings http://ceur-ws.org ISSN 1613-0073 CEUR Workshop Proceedings (CEUR-WS.org) describing the fact of existence. Both approaches provide valuable basis for linking space and time, and many researchers have proposed the theoretical models and ontologies of space-time based on these two views. Yet the generalisability of much published research on spatiotemporal representations does not justify their ontological choices, and even though they did, there is still little attempt to provide logic formalisms in these studies. This research assesses existing approaches to spatiotemporal knowledge representations and presents our design and axiomatization of a new spatiotemporal ontology. We advocate both three-plus-one and four-dimensional models, and our proposed ontology interlinks the two. We develop a set of use case scenarios and competency questions to capture the requirements of proposed ontology as well as evaluating existing spatiotemporal ontologies. Our ontology is compatible with the mereotopology of spatial regions based on Region Connection Calculus (RCC) and regular topology and the mereotopology of time intervals in compliance of Allen’s Algebra. We also provide a formalism of the link between these two mereotopologies using the product order of the two. This paper begins by motivating scenarios and competency questions of our spatiotemporal ontology in Section 2. In Section 3, we review some philosophical backgrounds of treating space-time and some existing work on spatiotemporal modelling. In section 4, we review the fundamental ontological choices for spatiotemporal entities, and thereby based on our ontological commitments, we present our axiomatizations of the proposed spatiotemporal ontology at the end1 . 2. Motivating scenarios To capture the requisites of the proposed ontology and identify the insufficiency of existing spatiotemporal work, we develop a set of scenarios that could potentially apply spatiotemporal ontologies. It should be noted that we are only interested in the spatial and temporal aspects of the physical world where objects locate and phenomenon occur. Scenario 1: Toronto’s boundary remained unchanged until 1880 and then followed a series of expansions. The most recent occurrence of amalgamation of Toronto was in 1998, merging the six previous municipalities that made up Etobicoke, Scarborough, York, East York, North York, and the City of Toronto, into a new singular City of Toronto. Scenario 2: The Toronto Santa Claus Parade starts at noon, and the performers march from Christie Pits along Bloor Street West, south on Avenue Road/Queen’s Park Crescent/University Avenue to Front Street West, and east along Front Street to St. Lawrence Market. Scenario 3: Alice leaves her house in Riverdale and crosses the Don Valley to Leaside. She purchases plumbing supplies at Canadian Tire (825 Eglinton Avenue East) and bathroom tiles at the Home Depot that is nearby on Wicksteed Avenue. Scenario 4: A forest fire begins on the side of a mountain, grows to cover the entire valley adjacent to the mountain, and is then extinguished by firefighters. These scenarios lead to the following competency questions: 1 The full version of this paper, containing proofs for all results, can be found at http://stl.mie.utoronto.ca/publications/ spatiotemporal.pdf 1. What are the relationships among events, space and time (e.g., parade at St. Lawrence Market and the amalgamation of Toronto in 1998)? 2. Are there distinct types of relationships between an event and space-time and an object and space-time respectively? If so, how do these relationships relate to the mereotopologies of objects, events, space and time? 3. Motion and spatial change: when an entity moves or has spatial change, what exactly changes? In all the scenarios above, it is almost inevitable to associate an event to a location and time. For example, we cannot discuss the amalgamation of Toronto without taking about when it happened and what the old and new spatial boundaries are, nor can we describe a parade and a wildfire without their locations. Therefore, it is quite natural to ask the question about what relations an event holds to a spatial region and a temporal interval. Scenario 2 and 3 motivate [CQ2] – once we identify how events are related to space and time [CQ1], we would like to know whether the spatial and temporal relations an event has are the same as what an object has. Is Alice at Canadian Tire the same relation as her purchase at Canadian tire? No two physical objects can occupy the same spatial region at the same time, but it is possible for compresent events occupy the same space-time region [8, 9]. For example, a globe can change its color and rotate at the same time. Therefore, the location of an object is different from the location of an event. There are distinct mereotopologies for different physical objects, events, spatial regions and temporal regions and certainly connections and mappings among these mereotopologies. Regarding [CQ3], each of these scenarios involves some kind of spatial change, and to summarize them, we can see that the spatial change of an object usually comes from two aspects: • An object changes its shape or boundary over time, such as the expansion of the boundary of City of Toronto. • An object changes its positions but maintaining its shape, in the case moving objects. There are also complicated cases that involves both types of changes, for example the growth and movement of a wildfire. In these examples, a spatial region does not change, but the relations between objects and spatial regions change. The identification of spatial changes leads to the requirements for a spatiotemporal ontology to be useful: • The ability to represent change in time of spatial relations; for example, City of Toronto contains Etobicoke after the amalgamation of Toronto in 1998. What was the relations between the boundaries of City of Toronto and Etobicoke before 1998? • The ability to represent the continuity of spatial change; for example, the parade marchers move along the planned routes. The aforementioned scenarios and competency questions are used throughout the paper to evaluate the existing work on spatiotemporal modelling and address the needs for a spatiotem- poral ontology. 3. Related work 3.1. Philosophical background: Three and four-dimensionalism There is a long history of studying space and time in philosophy. There are mainly two major theories about how objects are related to time: endurantism (three-dimensionalism) and perdurantism (four-dimensionalism) [10]. Three-dimensionalism claims that any object persists by being wholly present at which it exists, referred to as endurant or continuant. It is from a view that objects only have spatial dimensions and move through time. Controversially, four-dimensionlists believe that all objects persist over time and have temporal parts, called perdurant or occurrent. The sum of all temporal parts of an object is referred to as a space-time worm. They argue that objects extend on time just as they extend on space. Therefore, at any time we see an object, we only see part of it. From a physics view, three-dimensional approach appears to be correct most of the time in Galilean and Newtonian physics, which are perceived in a common-sense way by most people. On the other hand, the temporal part and spacetime view is supported by Einstein’s theory of special relativity and Minkowski spacetime [11]. In more recent research, four-dimensional approach is more appealing in terms of addressing the criterion of identity and explaining some puzzle cases of object identification, such as The Ship of Theseus. In this paper we will not discuss the arguments between these two views. Instead, we would like to present the ontological choices with respect to each theory. 3.2. Conceptual spatiotemporal model Spatiotemporal data has been a foundation of most location-based services. Thus, a key aspect of database management is modelling both spatial and temporal dimensions of data. The main approaches to integrating time information into spatial data can be summarized as follows [12, 13]: • Timestamping objects: time is treated as an attribute of objects. Examples are Simple Time-Stamping models, Object-Relationship (O-R) Models and Spatio-Temporal Object- Oriented (O-O) Data Models. These models focus on capturing the change of objects or phenomena. • Timestamping events: events and processes are tagged with timestamps. This approach is adopted by Event-oriented models, where events happening with an object are major entities and recorded in a chain-like structure based on the temporal order. • Timestamping objects and events: an object’s static, changing and ceased states are all attributed by time information, and an object’s history is represented by a sequence of states. This approach is adopted by The History Graph model, and aims to describe a limited extent in time and space for objects. • Time as a standalone entity: applied by The Three Domain model, where semantics (objects to any human concepts), space and time are three separate domains, and there are links between each of the two domains to represent processes and phenomena. For example, the land owner is a semantic entity that is linked to a land parcel (spatial object), with changes to the parcel associated with dates (temporal object). • Time as an integral part of entities: the concept is carried out by moving object data models. A moving point or region is an abstract data type that have both spatial and temporal dimensions (similar to a four-dimensional object), and this primitive data type is an attribute of a physical entity such as parcels, or pedestrians. Most existing modelling approaches are able to handle both discrete and continuous time structure using a linear time axis, but this is inadequate for reasoning about future events. Complex time structure such as branching needs to be incorporated for predicting changes. Moreover, each of these approaches is designed to capture only one aspect of change, such as the geometry change, movement, or occurring event, but not all of them. Therefore, there is a need for a spatiotemporal ontology that can harmonize different tools and designs for spatiotemporal data models. 3.3. Formalisms for spatiotemporal ontology Qualitative representations of both space and time have been well established, and there also exists a wide spectrum of attempts to construct spatiotemporal hybrids. In correspondence with three dimensionalism, a number of researchers have explored methods for temporalizing spatial relations. For example, Cui, Cohn and Randell used a function space (𝑥, 𝑡) to represent the space occupied by object 𝑥 at time 𝑡 in the formalism of Region Connect Calculus (RCC) [14]. Subsequently, Galton combined RCC with his modification of Allen’s Temporal logic to represent continuous motion [15], and Wolter and Zakharyaschev provided a logic of temporalizing RCC-8 for qualitative representation and reasoning about spatial regions in time [16]. Alternatively, Muller developed a mereotopological theory of spatiotemporal objects, in which space-time is one primitive domain [17]. The work lay the foundations for representation of four-dimensional entities. Similarly, Hazarika and Cohn investigated notions of spatiotemporal continuity together with the possible transitions of RCC-8 relations [18], where regions in space are considered temporally extended. However, these spatiotemporal logics do not comply with Allen’s algebra of convex time intervals, which is widely acknowledged in many upper ontologies. In spite of arguments between three and four-dimensionalists, there are some attempts to harmonize both theories. Grenon and Smith proposed two distinct types of entities and their relations: ’SNAP’ and ’SPAN’, with respect to continuants and occurrents [7]. The study also provided a framework to unify two ontologies. Basic Formal Ontology (BFO) [22] applied SNAP and SPAN concepts and formalize two classes of entities as continuant and occurrent, each holding its own parthood relations. BFO recognizes that a spatiotemporal region is an occurrent entity that can be occupied by other occurrent entities, such as processes. However, it fails to provide a complete mereotopology of spatiotemporal regions. Meanwhile, BFO temporalized most relationships among entities in order to interconnect continuants and occurrents, which is redundant in some cases. 4. Ontological choices Our study focuses on building spatiotemporal hybrids, but we do not plan to create new ontolo- gies for space and time. The foundations of our spatiotemporal ontology are the mereotopology of spatial regions and the mereotopology of time intervals. This section discusses some onto- logical choices we need to make prior to formal axiomatization. 4.1. Three-dimensional and four-dimensional entities One important choice to make is the adoption of three-dimensionalism and four-dimensionalism. Indeed, all objects persist over time; however, from a commonsense perspective, it is not obvious to accept that a physical object has temporal parts, and we only see part of a physical object because we do not see its whole existence (from the creation of the object to its destruction). It is interesting to note that such an approach is not directly reflected in natural language expressions about physical objects. On the other hand, the theory of four-dimensionalism can be applied perfectly to entities like events and processes. Events extend naturally through time and are always bound to a temporal region. Therefore, we accept that physical objects are three dimensional objects (continuant or endurant), meaning that they can exist wholly at a single moment, and they do not have any temporal parts. Entities in time, such as events, processes, etc, are four-dimensional objects (occurrent or perdurant), and they have temporal parts corresponding to timepoints or time intervals. Since we adopt both three dimensional and four-dimensional theories, a natural question to ask concerns the relationship between three-dimensional and four-dimensional entities. In some upper ontologies such as BFO and Tupper, a participation relation links three dimensional objects (physical objects) to four dimensional objects (events). However, in this paper, we only focus on the spatiotemporal region and its mapping to space and time. We do not cover concepts of spatiotemporal regions nor the relationships between a spatiotemporal and objects or events. 4.2. Occupying relation Substantivalists believe that entities are located at regions of space or spacetime, as opposed to the supersubstantival view in which located entities are identical to their locations. Following substantivalism, this study distinguishes a physical object and the region it occupies. Occupation is the relationship between an object and its located regions. The motivation is that the basic properties of a spatial region is quite different from a physical object and it is less acceptable to say a region is a physical body from a linguistic view [3]. As discussed in the motivating scenarios, a physical object occupying a spatial region is a different type of relation from an event occupying a spatial /space-time region, but there is still an analogy between the two. An event should also be distinct from the space-time region it occupies, and events and space-time maintain their own mereotopologies. 4.3. Treatment of space, time, and spacetime A key consideration of spatiotemporal representation is the choice of space and time as two separate domains or space-time as a primitive entity. The paper takes a common-sense approach to address this issue. It is true that an object’s existence is related to time, but it is unnecessary to always temporalize the relation between an object and its location, especially when an object is not involved in an event. It has no significance to always tie an object’s location with time if an object’s location and spatial relations to other objects never change during its existence. When we ask the question where is/was an object, it already implicitly indicates an occurrence of some spatial change (an event). Meanwhile, it is natural to accept that space and time as independent domains since human beings can only observe snapshots of an object. For example, humans can only see the snapshot position instead of a swept volume during the movement of an object. Furthermore, there is no explicit expression in natural language to represent spatial and temporal relations as a whole, compared to spatial and temporal prepositions in general linguistics, such as on, at, in, etc. Therefore, we take space and time as two separate domains when we deal with three-dimensional objects. On the other hand, space-time is considered as one primitive for events. Four-dimensional objects (events) are always associated with both spatial and temporal extent. An event extends on time just as a physical object extends on space, and it is essential to link space to an event that physical objects participating in. Meanwhile, even though humans can only observe snapshots of an object’s movement, motion is often assumed as a continuous change, and to capture such continuity in our ontology is a big motivation to have space-time as one primitive domain and develop a mereotopology for spatiotemporal regions 5. The Spatiotemporal Mereotopology The design of the spatiotemporal mereotopology is driven by semantic requirements extracted from the motivating scenarios and competency questions: 1. Spatiotemporal regions are disjoint from spatial regions and time intervals. 2. Each class of entities has its own mereotopology (pluralism). 3. The mereotopology of spatiotemporal regions is the product of the mereotopologies of spatial regions and time intervals. An analogous situation can be seen in the development of time ontologies, in which there are two disjoint classes of temporal entities. There are ontologies in which only timepoints exist2 . and there are ontologies in which only time intervals exist3 . Rather than debate about which ontological commitment is the right the one, combined time ontologies4 include both timepoints and time intervals, while axiomatizing the relationships between the two sorts of temporal entities. In particular, the ordering over time intervals is isomorphic a subordering of the product of the ordering over timepoints. The mereology over time intervals is definable using the ordering over timepoints. We can use the combined time axioms to reason about this mereology, or we can identify the mereology of time intervals that is faithfully interpreted by the combined time axioms (which is synonymous with different theories in the ℍ𝑝𝑒𝑟𝑖𝑜𝑑𝑠 Hierarchy of COLORE, depending on the exact ordering on timepoints). In the mereotopology of spatiotemporal regions, we allow spatial regions, time intervals, and spatiotemporal regions as three mutually disjoint entities. The ontology axiomatizes the relationships between the distinct mereotopologies specified on these entities, namely, that the mereotopology of spatiotemporal regions is the product of the mereotopologies of spatial regions 2 colore.oor.net/timepoints/ 3 colore.oor.net/periods 4 colore.oor.net/combined_time/ and time intervals. We can use these axioms to reason about the spatiotemporal mereotopology, or we can identify the mereotopology of spatiotemporal regions that is faithfully interpreted by the combined axioms. 5.1. Mereographs The ontological commitment to mereotopological pluralism means that we need to specify sepa- rate mereotopologies for time intervals, spatial regions, and spatiotemporal regions. We follow the work of [19] for the approach to mereotopology in which both parthood and connection are primitive relations. Mereotopologies are represented by the amalgamation of partial orderings and graphs with loops: Definition 1. ℙ ⊕ 𝔾 = ⟨𝑉 , E, ≤⟩ is a mereograph iff 1. ℙ = ⟨𝑉 , ≤⟩ such that ℙ ∈ 𝔐𝑝𝑎𝑟𝑡𝑖𝑎𝑙_𝑜𝑟𝑑𝑒𝑟𝑖𝑛𝑔 ; 2. 𝔾 = ⟨𝑉 , E⟩ such that 𝔾 ∈ 𝔐𝑔𝑟𝑎𝑝ℎ_𝑙𝑜𝑜𝑝𝑠 ; 3. 𝑈 ℙ (𝑁 𝔾 (x)) ⊆ 𝑁 𝔾 (x), for each x ∈ 𝑉. 𝔐𝑚𝑒𝑟𝑒𝑜𝑔𝑟𝑎𝑝ℎ denotes the class of mereographs. 5.2. Products of Structures The primary challenge in the design of the ontology is to find a class of mathematical structures that can be used to represent a mapping between the mereotopologies of time intervals and spatial regions and the mereotopology of spatiotemporal regions. Definition 2. A bijective-tripartite incidence structure is a tuple 𝕀 = ⟨𝑃, 𝐿, 𝑄, I⟩ such that 1. 𝑃 ∩ 𝐿 = ∅; 𝑃 ∩ 𝑄 = ∅; 𝑄 ∩ 𝐿 = ∅ 2. I ⊆ (𝑃 × 𝐿) ∪ (𝐿 × 𝑃) ∪ 𝐼 𝑑; 3. I− = I; 4. for each pair l ∈ 𝐿, q ∈ 𝑄, ∣ 𝑁 𝕀 (l) ∩ 𝑁 𝕀 (q) ∩ 𝑃 ∣= 1 5. for each p ∈ 𝑃, ∣ 𝑁 𝕀 (p) ∩ 𝐿 ∣= 1 ∣ 𝑁 𝕀 (q) ∩ 𝑄 ∣= 1 6. for each l ∈ 𝐿, 𝑁 𝕀 [𝑁 𝕀 [l]] ∩ 𝑃 = 𝑁 𝕀 [l] ∩ 𝑃 7. for each q ∈ 𝑄, 𝑁 𝕀 [𝑁 𝕀 [q]] ∩ 𝑃 = 𝑁 𝕀 [q] ∩ 𝑃 𝔐𝑏𝑖𝑗𝑒𝑐𝑡𝑖𝑣𝑒_𝑡𝑟𝑖𝑝𝑎𝑟𝑡𝑖𝑡𝑒 denotes the class of bijective-tripartite incidence structures. The idea behind bijective-tripartite incidence structures is that spatial regions, time intervals, and spatiotemporal regions are represented by the three disjoint sets 𝑃, 𝐿, and 𝑄 respectively5 , and a spatiotemporal regions is incident to a spatial region and time intervals in the incidence structure iff the spatiotemporal region corresponds the pair in the product order and graph prod- uct. However, to guarantee that incidence represents a mapping, we must enforce the condition that each pair of elements of 𝐿 and 𝑄 is incident with a unique element of 𝑃. Condition (4) in Definition 2 enforces the existence of such an element, and Condition (5) enforces uniqueness. Theorem 1. ⟨𝑃, 𝐿, 𝑄, I⟩ ∈ 𝔐𝑏𝑖𝑗𝑒𝑐𝑡𝑖𝑣𝑒_𝑡𝑟𝑖𝑝𝑎𝑟𝑡𝑖𝑡𝑒 iff there exists a bijection 𝜇 ∶ 𝐿 × 𝑄 → 𝑃 such that x, y ∈ 𝑁 𝕀 [𝜇(x, y)] 5.3. Products of Orderings We can amalgamate the bijective tripartite incidence structure 𝕀 to capture the condition that the partial ordering 𝕊 of spatiotemporal regions is isomorphic with the product of the partial ordering (ℙ for spatial regions and the partial ordering ℚ for time intervals: Definition 3. A product multigeometry is a structure ℙ ⊕ ℚ ⊕ 𝕀 ⊕ 𝕊 such that 1. ℙ, ℚ, 𝕊 ∈ 𝔐𝑝𝑎𝑟𝑡𝑖𝑎𝑙_𝑜𝑟𝑑𝑒𝑟𝑖𝑛𝑔 ; 2. 𝕀 ∈ 𝔐𝑏𝑖𝑗𝑒𝑐𝑡𝑖𝑣𝑒_𝑡𝑟𝑖𝑝𝑎𝑟𝑡𝑖𝑡𝑒 ; 3. for each x ∈ 𝐿, y ∈ 𝑄, 𝑈 𝕊 [𝑁 𝕀 (x) ∩ 𝑁 𝕀 (y) ∩ 𝑃] = 𝑁 𝕀 (𝑈 ℙ [x]) ∩ 𝑁 𝕀 (𝑈 ℚ [y]) ∩ 𝑃 The class of product multigeometries is denoted by 𝔐𝑝𝑟𝑜𝑑𝑢𝑐𝑡_𝑚𝑢𝑙𝑡𝑖𝑔 . Theorem 2. There is a surjection 𝜑 ∶ 𝔐𝑝𝑟𝑜𝑑𝑢𝑐𝑡_𝑚𝑢𝑙𝑡𝑖𝑔 → 𝔐𝑝𝑎𝑟𝑡𝑖𝑎𝑙_𝑜𝑟𝑑𝑒𝑟𝑖𝑛𝑔 such that 1. 𝜑(ℙ ⊕ ℚ ⊕ 𝕀 ⊕ 𝕊) = 𝕊; 2. there is an isomorphism 𝜇 ∶ ℙ × ℚ → 𝕊 such that 𝜇(x, y) = 𝑁 𝕀 (x) ∩ 𝑁 𝕀 (y) ∩ 𝑃 5.4. Graph Products Since mereographs are the amalgamation of partial orderings and graphs, we also need to represent the notion of graph products. We can amalgamate the bijective tripartite incidence structure 𝕀 to capture the condition that the graph ℍ of spatiotemporal regions is isomorphic with the product of the graph (𝔾 for spatial regions and the graph ℍ for time intervals: Definition 4. A product subgraph is a structure 𝔾 ⊕ ℂ ⊕ 𝕀 ⊕ ℍ such that 1. 𝔾, ℂ, ℍ ∈ 𝔐𝑝𝑎𝑟𝑡𝑖𝑎𝑙_𝑜𝑟𝑑𝑒𝑟𝑖𝑛𝑔 ; 5 Within incidence structures, the elements of 𝑃 are referred to as points, the elements of 𝐿 are referred to as lines, and the elements of 𝑄 are referred to as planes. 2. 𝕀 ∈ 𝔐𝑏𝑖𝑗𝑒𝑐𝑡𝑖𝑣𝑒_𝑡𝑟𝑖𝑝𝑎𝑟𝑡𝑖𝑡𝑒 ; 3. for each x ∈ 𝐿, y ∈ 𝑄, 𝑁 ℍ [𝑁 𝕀 (x) ∩ 𝑁 𝕀 (y) ∩ 𝑃] = 𝑁 𝕀 (𝑁 𝔾 [x]) ∩ 𝑁 𝕀 (𝑁 ℂ [y]) ∩ 𝑃 The class of product subgraphs is denoted by 𝔐𝑝𝑟𝑜𝑑𝑢𝑐𝑡_𝑠𝑢𝑏𝑔𝑟𝑎𝑝ℎ . Theorem 3. There is a surjection 𝜑 ∶ 𝔐𝑝𝑟𝑜𝑑𝑢𝑐𝑡_𝑠𝑢𝑏𝑔𝑟𝑎𝑝ℎ → 𝔐𝑝𝑎𝑟𝑡𝑖𝑎𝑙_𝑜𝑟𝑑𝑒𝑟𝑖𝑛𝑔 such that 1. 𝜑(𝔾 ⊕ ℂ ⊕ 𝕀 ⊕ ℍ) = ℍ; 2. there is an isomorphism 𝜇 ∶ 𝔾 × ℂ → ℍ such that 𝜇(x, y) = 𝑁 𝕀 (x) ∩ 𝑁 𝕀 (y) ∩ 𝑃 5.5. Axiomatization Since mereographs are the amalgamation of partial orderings and graphs, we similarly need to amalgamate the structures for product orderings and graph products: Definition 5. ℚ ⊕ ℂ ⊕ ℙ ⊕ 𝔾 ⊕ 𝕀 ⊕ 𝕊 ⊕ ℍ is an spatiotemporal structure iff 1. ℚ ⊕ ℂ, ℙ ⊕ 𝔾, 𝕊 ⊕ ℍ ∈ 𝔐𝑚𝑒𝑟𝑒𝑜𝑔𝑟𝑎𝑝ℎ ; 2. ℙ ⊕ ℚ ⊕ 𝕀 ⊕ 𝕊 ∈ 𝔐𝑝𝑟𝑜𝑑𝑢𝑐𝑡_𝑚𝑢𝑙𝑡𝑖𝑔 ; 3. 𝔾 ⊕ ℂ ⊕ 𝕀 ⊕ ℍ ∈ 𝔐𝑝𝑟𝑜𝑑𝑢𝑐𝑡_𝑠𝑢𝑏𝑔𝑟𝑎𝑝ℎ . 𝔐𝑠𝑝𝑎𝑡𝑖𝑜𝑡𝑒𝑚𝑝𝑜𝑟𝑎𝑙_𝑚𝑡 is the class of spatiotemporal structures. The following result shows us that the sentences in Figure 5.5 axiomatize the class of spa- tiotemporal structures: Theorem 4. There exists a bijection 𝜑 ∶ 𝑀𝑜𝑑(𝑇𝑠𝑝𝑎𝑡𝑖𝑜𝑡𝑒𝑚𝑝𝑜𝑟𝑎𝑙_𝑚𝑡 ) → 𝔐𝑠𝑝𝑎𝑡𝑖𝑜𝑡𝑒𝑚𝑝𝑜𝑟𝑎𝑙_𝑚𝑡 such that 𝜑(⟨𝑀, spatiotemporal_region, spatiotemporal_part, spatiotemporal_C, region, part, C, timeinterval, temporal_part, meets⟩) = ℚ ⊕ ℂ ⊕ ℙ ⊕ 𝔾 ⊕ 𝕀 ⊕ 𝕊 ⊕ ℍ iff 1. 𝑀 = 𝑉1 ∪ 𝑉2 ∪ 𝑉3 ; 2. ⟨x, y⟩ ∈ spatiotemporal_C iff (x, y) ∈ E3 ; 3. ⟨x, y⟩ ∈ spatiotemporal_part iff x ⊑ y; 4. ⟨x, y⟩ ∈ C iff (x, y) ∈ E1 ; 5. ⟨x, y⟩ ∈ part iff x ⪯ y; 6. ⟨x, y⟩ ∈ meets iff (x, y) ∈ E2 ; 7. ⟨x, y⟩ ∈ temporal_part iff x ≤ y; 8. ⟨x, y⟩ ∈ spatial_extent iff (x, y) ∈ I and x ∈ 𝑉3 , y ∈ 𝑉1 ; 9. ⟨x, y⟩ ∈ temporal_extent iff (x, y) ∈ I and x ∈ 𝑉3 , y ∈ 𝑉2 ; ∀𝑥 𝑠𝑝𝑎𝑡𝑖𝑜𝑡𝑒𝑚𝑝𝑜𝑟𝑎𝑙_𝑟𝑒𝑔𝑖𝑜𝑛(𝑥) ⊃ ¬𝑟𝑒𝑔𝑖𝑜𝑛(𝑥) (1) ∀𝑥 𝑠𝑝𝑎𝑡𝑖𝑜𝑡𝑒𝑚𝑝𝑜𝑟𝑎𝑙_𝑟𝑒𝑔𝑖𝑜𝑛(𝑥) ⊃ ¬𝑡𝑖𝑚𝑒𝑖𝑛𝑡𝑒𝑟𝑣𝑎𝑙(𝑥) (2) ∀𝑥 𝑟𝑒𝑔𝑖𝑜𝑛(𝑥) ⊃ ¬𝑡𝑖𝑚𝑒𝑖𝑛𝑡𝑒𝑟𝑣𝑎𝑙(𝑥) (3) ∀𝑝 𝑠𝑝𝑎𝑡𝑖𝑜𝑡𝑒𝑚𝑝𝑜𝑟𝑎𝑙_𝑟𝑒𝑔𝑖𝑜𝑛(𝑝) ⊃ (∃𝑙 𝑟𝑒𝑔𝑖𝑜𝑛(𝑙) ∧ 𝑠𝑝𝑎𝑡𝑖𝑎𝑙_𝑒𝑥𝑡𝑒𝑛𝑡(𝑝, 𝑙)) (4) ∀𝑝, 𝑙1 , 𝑙2 𝑠𝑝𝑎𝑡𝑖𝑜𝑡𝑒𝑚𝑝𝑜𝑟𝑎𝑙_𝑟𝑒𝑔𝑖𝑜𝑛(𝑝) ∧ 𝑟𝑒𝑔𝑖𝑜𝑛(𝑙1 ) ∧ 𝑟𝑒𝑔𝑖𝑜𝑛(𝑙2 ) ∧𝑠𝑝𝑎𝑡𝑖𝑎𝑙_𝑒𝑥𝑡𝑒𝑛𝑡(𝑝, 𝑙2 ) ∧ 𝑠𝑝𝑎𝑡𝑖𝑎𝑙_𝑒𝑥𝑡𝑒𝑛𝑡(𝑝, 𝑙1 ) ⊃ (𝑙1 = 𝑙2 ) (5) ∀𝑝 𝑠𝑝𝑎𝑡𝑖𝑜𝑡𝑒𝑚𝑝𝑜𝑟𝑎𝑙_𝑟𝑒𝑔𝑖𝑜𝑛(𝑝) ⊃ (∃𝑞 𝑡𝑖𝑚𝑒𝑖𝑛𝑡𝑒𝑟𝑣𝑎𝑙(𝑞) ∧ 𝑡𝑒𝑚𝑝𝑜𝑟𝑎𝑙_𝑒𝑥𝑡𝑒𝑛𝑡(𝑝, 𝑞)) (6) ∀𝑝, 𝑞1 , 𝑞2 𝑠𝑝𝑎𝑡𝑖𝑜𝑡𝑒𝑚𝑝𝑜𝑟𝑎𝑙_𝑟𝑒𝑔𝑖𝑜𝑛(𝑝) ∧ 𝑡𝑖𝑚𝑒𝑖𝑛𝑡𝑒𝑟𝑣𝑎𝑙(𝑞1 ) ∧ 𝑡𝑖𝑚𝑒𝑖𝑛𝑡𝑒𝑟𝑣𝑎𝑙(𝑞2 ) ∧𝑡𝑒𝑚𝑝𝑜𝑟𝑎𝑙_𝑒𝑥𝑡𝑒𝑛𝑡(𝑝, 𝑞1 ) ∧ 𝑡𝑒𝑚𝑝𝑜𝑟𝑎𝑙_𝑒𝑥𝑡𝑒𝑛𝑡(𝑝, 𝑞2 ) ⊃ 𝑞1 = 𝑞2 ) (7) ∀𝑙, 𝑞 𝑟𝑒𝑔𝑖𝑜𝑛(𝑙) ∧ 𝑡𝑖𝑚𝑒𝑖𝑛𝑡𝑒𝑟𝑣𝑎𝑙(𝑞) ⊃ (∃𝑝 𝑠𝑝𝑎𝑡𝑖𝑜𝑡𝑒𝑚𝑝𝑜𝑟𝑎𝑙_𝑟𝑒𝑔𝑖𝑜𝑛(𝑝) ∧𝑠𝑝𝑎𝑡𝑖𝑎𝑙_𝑒𝑥𝑡𝑒𝑛𝑡(𝑝, 𝑙) ∧ 𝑡𝑒𝑚𝑝𝑜𝑟𝑎𝑙_𝑒𝑥𝑡𝑒𝑛𝑡(𝑝, 𝑞)) (8) ∀𝑙, 𝑞, 𝑝1 , 𝑝2 𝑠𝑝𝑎𝑡𝑖𝑜𝑡𝑒𝑚𝑝𝑜𝑟𝑎𝑙_𝑟𝑒𝑔𝑖𝑜𝑛(𝑝1 ) ∧ 𝑠𝑝𝑎𝑡𝑖𝑜𝑡𝑒𝑚𝑝𝑜𝑟𝑎𝑙_𝑟𝑒𝑔𝑖𝑜𝑛(𝑝2 ) ∧𝑟𝑒𝑔𝑖𝑜𝑛(𝑙) ∧ 𝑡𝑖𝑚𝑒𝑖𝑛𝑡𝑒𝑟𝑣𝑎𝑙(𝑞) ∧ 𝑠𝑝𝑎𝑡𝑖𝑎𝑙_𝑒𝑥𝑡𝑒𝑛𝑡(𝑝1 , 𝑙) ∧ 𝑡𝑒𝑚𝑝𝑜𝑟𝑎𝑙_𝑒𝑥𝑡𝑒𝑛𝑡(𝑝1 , 𝑞) ∧𝑠𝑝𝑎𝑡𝑖𝑎𝑙_𝑒𝑥𝑡𝑒𝑛𝑡(𝑝2 , 𝑙) ∧ 𝑡𝑒𝑚𝑝𝑜𝑟𝑎𝑙_𝑒𝑥𝑡𝑒𝑛𝑡(𝑝2 , 𝑞) ⊃ (𝑝1 = 𝑝2 ) (9) ∀𝑥, 𝑦 𝑠𝑡_𝑝𝑎𝑟𝑡(𝑥, 𝑦) ⊃ 𝑠𝑝𝑎𝑡𝑖𝑜𝑡𝑒𝑚𝑝𝑜𝑟𝑎𝑙_𝑟𝑒𝑔𝑖𝑜𝑛(𝑥) ∧ 𝑠𝑝𝑎𝑡𝑖𝑜𝑡𝑒𝑚𝑝𝑜𝑟𝑎𝑙_𝑟𝑒𝑔𝑖𝑜𝑛(𝑦) (10) ∀𝑥, 𝑦 𝑝𝑎𝑟𝑡(𝑥, 𝑦) ⊃ 𝑟𝑒𝑔𝑖𝑜𝑛(𝑥) ∧ 𝑟𝑒𝑔𝑖𝑜𝑛(𝑦) (11) ∀𝑥, 𝑦 𝑡𝑒𝑚𝑝𝑜𝑟𝑎𝑙_𝑝𝑎𝑟𝑡(𝑥, 𝑦) ⊃ 𝑡𝑖𝑚𝑒𝑖𝑛𝑡𝑒𝑟𝑣𝑎𝑙(𝑥) ∧ 𝑡𝑖𝑚𝑒𝑖𝑛𝑡𝑒𝑟𝑣𝑎𝑙(𝑦) (12) ∀𝑝1 , 𝑝2 , 𝑙1 , 𝑙2 , 𝑞1 , 𝑞2 𝑠𝑝𝑎𝑡𝑖𝑜𝑡𝑒𝑚𝑝𝑜𝑟𝑎𝑙_𝑟𝑒𝑔𝑖𝑜𝑛(𝑝1 ) ∧ 𝑠𝑝𝑎𝑡𝑖𝑜𝑡𝑒𝑚𝑝𝑜𝑟𝑎𝑙_𝑟𝑒𝑔𝑖𝑜𝑛(𝑝2 ) ∧𝑟𝑒𝑔𝑖𝑜𝑛(𝑞1 ) ∧ 𝑟𝑒𝑔𝑖𝑜𝑛(𝑞2 ) ∧ 𝑠𝑝𝑎𝑡𝑖𝑎𝑙_𝑒𝑥𝑡𝑒𝑛𝑡(𝑝1 , 𝑞1 ) ∧ 𝑠𝑝𝑎𝑡𝑖𝑎𝑙_𝑒𝑥𝑡𝑒𝑛𝑡(𝑝2 , 𝑞2 ) ∧𝑡𝑖𝑚𝑒𝑖𝑛𝑡𝑒𝑟𝑣𝑎𝑙(𝑙1 ) ∧ 𝑡𝑖𝑚𝑒𝑖𝑛𝑡𝑒𝑟𝑣𝑎𝑙(𝑙2 ) ∧ 𝑡𝑒𝑚𝑝𝑜𝑟𝑎𝑙_𝑒𝑥𝑡𝑒𝑛𝑡(𝑝1 , 𝑙1 ) ∧ 𝑡𝑒𝑚𝑝𝑜𝑟𝑎𝑙_𝑒𝑥𝑡𝑒𝑛𝑡(𝑝2 , 𝑙2 ) ⊃ (𝑠𝑡_𝑝𝑎𝑟𝑡(𝑝1 , 𝑝2 ) ≡ (𝑡𝑒𝑚𝑝𝑜𝑟𝑎𝑙_𝑝𝑎𝑟𝑡(𝑙1 , 𝑙2 ) ∧ 𝑝𝑎𝑟𝑡(𝑞1 , 𝑞2 ))) (13) ∀𝑝1 , 𝑝2 , 𝑙1 , 𝑙2 , 𝑞1 , 𝑞2 𝑠𝑝𝑎𝑡𝑖𝑜𝑡𝑒𝑚𝑝𝑜𝑟𝑎𝑙_𝑟𝑒𝑔𝑖𝑜𝑛(𝑝1 ) ∧ 𝑠𝑝𝑎𝑡𝑖𝑜𝑡𝑒𝑚𝑝𝑜𝑟𝑎𝑙_𝑟𝑒𝑔𝑖𝑜𝑛(𝑝2 ) ∧𝑟𝑒𝑔𝑖𝑜𝑛(𝑞1 ) ∧ 𝑟𝑒𝑔𝑖𝑜𝑛(𝑞2 ) ∧ 𝑠𝑝𝑎𝑡𝑖𝑎𝑙_𝑒𝑥𝑡𝑒𝑛𝑡(𝑝1 , 𝑞1 ) ∧ 𝑠𝑝𝑎𝑡𝑖𝑎𝑙_𝑒𝑥𝑡𝑒𝑛𝑡(𝑝2 , 𝑞2 ) ∧𝑡𝑖𝑚𝑒𝑖𝑛𝑡𝑒𝑟𝑣𝑎𝑙(𝑙1 ) ∧ 𝑡𝑖𝑚𝑒𝑖𝑛𝑡𝑒𝑟𝑣𝑎𝑙(𝑙2 ) ∧ 𝑡𝑒𝑚𝑝𝑜𝑟𝑎𝑙_𝑒𝑥𝑡𝑒𝑛𝑡(𝑝1 , 𝑙1 ) ∧ 𝑡𝑒𝑚𝑝𝑜𝑟𝑎𝑙_𝑒𝑥𝑡𝑒𝑛𝑡(𝑝2 , 𝑙2 ) ⊃ (𝑠𝑡_𝐶(𝑝1 , 𝑝2 ) ≡ (𝑚𝑒𝑒𝑡𝑠(𝑙1 , 𝑙2 ) ∧ 𝐶(𝑞1 , 𝑞2 ))) (14) Figure 1: 𝑇𝑠𝑝𝑎𝑡𝑖𝑜𝑡𝑒𝑚𝑝𝑜𝑟𝑎𝑙_𝑚𝑡 : Axiomatization of spatiotemporal structures, together with the axioms of 𝑇𝑚𝑡 for the mereotopology of spatial regions and 𝑇𝑖𝑛𝑡𝑒𝑟𝑣𝑎𝑙_𝑚𝑒𝑒𝑡𝑖𝑛𝑔 for the mereotopology of time intervals. The axioms of 𝑇𝑠𝑝𝑎𝑡𝑖𝑜𝑡𝑒𝑚𝑝𝑜𝑟𝑎𝑙 are independent of the specific axiomatization of the spatial regions and time intervals. No additional ontological commitments are imposed other than the condition that the classes are mutually disjoint. As a result, the axioms are compatible with a wide range of existing upper ontologies such as TUpper [4] and SUMO [20]. Furthermore, they are also compatible with the formalisms typically used for spatial reasoning (RCC8) and temporal reasoning (Allen’s Interval Algebra). RCC8 has been shown [19] to be logically synonymous with the simplest mereotopology 𝑇𝑚𝑡 6 while Allen’s Interval Algebra is synonymous with the mereotopology of convex intervals of a linear ordering [21]. 6 colore.oor.net/combined_mereotopology/mt.clif 6. Conclusions This study set out to provide a new approach to the axiomatization of a spatiotemporal mereotopology based on the philosophical stance of space-time, which is endurantism (three- dimensionalism) and perdurantism (four-dimenalism). We have identified multiple motivating scenarios to justify our ontological commitments on the three and four-dimentional entities and the treatment of spatial, temporal and spatiotemporal regions. We recognize that physical objects are three dimensional entities that only occupies spatial regions, and events and pro- cesses are four-dimensional, taking occupancy on spatiotemporal regions. Thus, we achieve a harmonization between three and four-dimensional theories in a common-sense way. The spatiotemporal ontology proposed in this paper is based on the mereotopology of RCC and Allen’s time algebra, and this research provides a new set of axioms of the spatiotemporal region and its relationships to spatial and temporal regions, using the product order of the two mereotopologies. A natural progression of this work is to provide a rigorous formalization of relationships among events, spatial, temporal and spatiotemporal regions, which have been discussed in our motivating scenarios. One question we do not discuss in this paper is that by using Allen’s algebra of convex time intervals, we add constraints on the sum operation of spatiotemporal regions, which could possibly limit the representation of our ontology. We are intended to address this issue in the following research on linking physical objects, events, space, time and spatiotemporal regions. We plan to provide two sequels to this paper. The first one is to explore the occupation relationship between events and spatiotemporal regions. The second one will be focused on how the location of events is related to the location of objects, as well as the application of our spatiotemporal ontology to different use cases, such as semantic trajectories. A. Notation Definition 6. Suppose ℙ ∈ 𝔐𝑝𝑎𝑟𝑡𝑖𝑎𝑙_𝑜𝑟𝑑𝑒𝑟𝑖𝑛𝑔 such that ℙ = ⟨𝑉 , ⪯⟩. The upper set for x in ℙ, denoted by 𝑈 ℙ (x), is 𝑈 ℙ (x) = {y ∶ x ≤ y} 𝑈 ℙ (𝑋 ) = ⋃ 𝑈 (x) x∈𝑋 ℒℙ = ⟨𝑉 , 𝐸⟩ is the lower bound graph for ℙ: (x, y) ∈ 𝐸 𝐿ℙ [x] ∩ 𝐿ℙ [y] ≠ ∅ Definition 7. Suppose 𝔾 ∈ 𝔐𝑔𝑟𝑎𝑝ℎ_𝑙𝑜𝑜𝑝𝑠 , such that 𝔾 = ⟨𝑉 , E⟩. The neighbourhood of x in 𝔾, denoted by 𝑁 𝔾 (x), is 𝑁 𝔾 (x) = {y ∶ (x, y) ∈ E} 𝑁 𝔾 (𝑋 ) = ⋃ 𝑁 𝔾 (x) x∈𝑋 Definition 8. Suppose 𝕀 ∈ 𝔐𝑏𝑖𝑗𝑒𝑐𝑡𝑖𝑣𝑒_𝑡𝑟𝑖𝑝𝑎𝑟𝑡𝑖𝑡𝑒 , such that 𝕀 = ⟨𝑃, 𝐿, 𝑄, I⟩. The neighbourhood of x in 𝕀, denoted by 𝑁 𝕀 (x), is 𝑁 𝕀 (x) = {y ∶ (x, y) ∈ I} 𝑁 𝕀 (𝑋 ) = ⋃ 𝑁 𝕀 (x) x∈𝑋 References [1] M. Egenhofer, R. Franzosa, Point-set topological spatial relations, International Journal of Geographical Information Systems 5 (1991) 161–174. [2] D. A. Randell, Z. Cui, A. Cohn, A spatial logic based on regions and connection, in: Proceedings of the Third International Conference Principles of Knowledge Representation and Reasoning, Morgan Kaufmann, San Mateo, California, 1992, pp. 165–176. [3] B. Aameri, M. Grüninger, Location Ontologies based on Mereotopological Pluralism, Applied Ontology 15 (2020) 135–184. [4] M. Grüninger, Y. Ru, J. Thai, TUpper: A Top Level Ontology within Standards, Applied Ontology 17 (2022) 143–165. [5] J. Allen, Maintaining Knowledge about Temporal Intervals, Communications ACM 26 (1983) 832–843. [6] A. Pani, Bhattacharajee, Temporal Representation and Reasoning in Artificial Intelligence: A Review, Mathematical and Computer Modelling 34 (2001) 55–80. [7] P. Grenon, B. Smith, SNAP and SPAN: Towrds Dynamic Spatial Ontology, Spatial Cognition and Computation 4 (2004) 69–104. [8] A. Quinton, Objects and Events, Mind 88 (1979) 197–214. [9] P. Hacker, Events and Objects in Space and Time, Mind 91 (1982) 1–19. [10] T. Sider, Four-Dimensionalism, Oxford University Press, 2001. [11] S. Hales, T. Johnson, Endurantism, Perdurantism and Special Relativity, Philosophical Quarterly 53 (2003) 524–539. [12] N. Pelekis, B. Theodoulidis, I. Kopanakis, Y. Theodoridis, Literature Review of Spatio- temporal Database Models, Knowledge Engineering Review 19 (2004) 235–274. [13] R. Nandal, Spatio-temporal Database and its Models: A Review, IOSR-JCE 11 (2013) 91–100. [14] Z. Cui, A. Cohn, D. Randell, Qualitative simulation based on a logical formalism of space and time, in: Proceedings of the AAAI-92, AAAI Press, 1992, pp. 679–684. [15] A. Galton, Towards an integrated logic of space, time and motion, in: IJCAI, 1993, pp. 1550–1555. [16] F. Wolter, M. Zakharyaschev, Spatio-temporal representation and reasoning based on rcc-8, in: Proceedings of the Seventh Conference of Knowledge Representation and Reasoning, 2000, pp. 3–14. [17] P. Muller, Topological spatio–temporal reasoning and representation, Computational Intelligence 18 (2002) 420–450. [18] S. Hazarika, A. Cohn, Qualitative spatio-temporal continuity, in: Conference on Spatial Information Theory, 2001, pp. 92–107. [19] M. Grüninger, B. Aameri, A new perspective on the mereotopology of RCC8, in: Proceeding of the 13th International Conference on Spatial Information Theory (COSIT 2017), Leibniz International Proceedings in Informatics, 2017, pp. 2:1–2:13. [20] I. Niles, A. Pease, Towards a standard upper ontology, in: FOIS ’01: Proceedings of the international conference on Formal Ontology in Information Systems, ACM, 2001, pp. 2–9. [21] M. Grüninger, Z. Li, The time ontology allen’s interval algebra, in: Proceedings of 24th International Symposium on Temporal Representation and Reasoning, 2017, pp. 1–16.