<!DOCTYPE article PUBLIC "-//NLM//DTD JATS (Z39.96) Journal Archiving and Interchange DTD v1.0 20120330//EN" "JATS-archivearticle1.dtd">
<article xmlns:xlink="http://www.w3.org/1999/xlink">
  <front>
    <journal-meta>
      <issn pub-type="ppub">1613-0073</issn>
    </journal-meta>
    <article-meta>
      <title-group>
        <article-title>Advanced Temporal Reasoning for Intelligent Trafic Monitoring</article-title>
      </title-group>
      <contrib-group>
        <contrib contrib-type="author">
          <string-name>Anees ul Mehdi</string-name>
          <email>anees.ulmehdi@de.bosch.com</email>
          <xref ref-type="aff" rid="aff1">1</xref>
        </contrib>
        <contrib contrib-type="author">
          <string-name>Alessandro Oltramari</string-name>
          <email>alessandro.oltramari@us.bosch.com</email>
          <xref ref-type="aff" rid="aff0">0</xref>
        </contrib>
        <contrib contrib-type="editor">
          <string-name>Ontologies, Knowledge Graphs, Temporal Reasoning, Trafic Monitoring</string-name>
        </contrib>
        <aff id="aff0">
          <label>0</label>
          <institution>Bosch Center for Artificial Intelligence</institution>
          ,
          <addr-line>Pittsburgh</addr-line>
          ,
          <country country="US">USA</country>
        </aff>
        <aff id="aff1">
          <label>1</label>
          <institution>Corporate Research and Central of Artificial Intelligence</institution>
          ,
          <addr-line>Renningen</addr-line>
          ,
          <country country="DE">Germany</country>
        </aff>
      </contrib-group>
      <pub-date>
        <year>2023</year>
      </pub-date>
      <abstract>
        <p>Times series, namely data characterized by temporal information, are a common feature of industrial use cases, especially when considering sensor-based technology. The work presented in this paper focuses on trafic time series collected from stationary cameras, and preprocessed through machine learning algorithms. We introduce the notion of trafic scene, which is central to an ontology of the trafic domain and instrumental to instantiate a knowledge graph for such domain. We then define the formal syntax and semantics of a new language for querying knowledge graphs that is capable of handling temporal information beyond SPARQL expressivity. Finally, we illustrate examples of advanced temporal queries for intelligent trafic monitoring applications.</p>
      </abstract>
    </article-meta>
  </front>
  <body>
    <sec id="sec-1">
      <title>-</title>
      <p>CEUR
ceur-ws.org</p>
    </sec>
    <sec id="sec-2">
      <title>1. Introduction</title>
      <p>In this work, we aim to address the limitations of current knowledge graph (KG) approaches
in representing and reasoning over dynamic information. In particular, we designed a query</p>
    </sec>
    <sec id="sec-3">
      <title>2. Problem</title>
    </sec>
    <sec id="sec-4">
      <title>Description and Setup</title>
      <p>Query 11 shows a KG excerpt related to trafic objects and their properties. In particular, Line 2
and 3 are asserting that there are two entities of type Car, whereas Line 4 and 5 are describing
their speeds (the implicit unit of measure adopted is miles per hour ).</p>
      <p>The instant at which a given information holds can be easily represented in RDF. As an
example, we can state that ”car 1 has a speed of 22.0 on July 27, 2021 at 14:21”. One way of
https://www.bosch.com/research/about-bosch-research/our-research-experts/alessandro-oltramari/
© 2023 Copyright for this paper by its authors. Use permitted under Creative Commons License Attribution 4.0 International (CC BY 4.0).</p>
      <sec id="sec-4-1">
        <title>Query 1: Virtualization query example</title>
        <p>Query 2: Querying information with time
1 SELECT ?car
2 WHERE
3 {
4 tmo:car_1 rdf:type tmo:Car .
5 tmo:car_1 tmo:associatedWith ?x .
6 ?x tmo:hasSpeed ?speed .
7 ?x tmo:time ?time .
8 FILTER(?speed&gt;22.0)
9 FILTER(?time&lt;”2021-06-27T09:00:00” &amp;&amp; ?time&gt;”2021-06-27T21:00:00”)
10 }
representing this information in RDF is to use reification 2, whereas tmo:associatedWith is
just an auxiliary predicate and _:x is called a blank node. Blank nodes are auxiliary nodes for
connecting diferent entities in a KG.</p>
        <p>tmo:car_1 tmo:associatedWith _:x .
_:x tmo:hasSpeed 22.0 .</p>
        <p>_:x tmo:time ”2021-06-27T14:21:00”^^xsd:dateTime
Alternatively, we can use RDF-star3 and represent the same information as following:
&lt;&lt;tmo:car_1 tmo:hasSpeed 22.0&gt;&gt;</p>
        <p>tmo:time ”2021-06-27T14:21:00”^^xsd:dateTime</p>
        <p>Regardless of the way we represent temporal information, the question is how to efectively
query such information and extract salient dynamic properties from it. When treating time
like any other property in RDF, we can use standard query language such as SPARQL. For
example, Query 2 asks for all cars with speed greater than 20.0 between 9:00AM to 9:00PM on
July 27, 2021. This is a usual interpretation of time. Filters like the one shown in Line 9, allow to
query for data that satisfy certain restrictions on the associated time point. In other words, the
way we can filter time-associated data is to compare the time points using the usual operators
like , &lt;, =, ! = etc. But how to conceptualize and generalize events from time points is left to
the human. In case of trafic situations, sometimes we are interested in relating information
at diferent time points and beyond the simple semantics of the aforementioned comparison
operators. As an example, suppose we want to check if there is a car idling until a police patrol
car arrives at the scene. Such query cannot be formalized in SPARQL, as it would require some
sort of a while-loop in order to traverse time-indexed KG triples unless certain conditions are
2https://www.w3.org/wiki/RdfReification. Note that we are not using the usual reification approach of associating a
triple to a blank node.
3https://w3c.github.io/rdf-star/cg-spec/2021-07-01.html
satisfied (appearance of a police vehicle in this example). Query languages like SPARQL lack
such a capability. Even though some queries about information over time could be expressed in
SPARQL, such queries usually end up to be too complex and prone to error. In this work, our
goal is to devise a query language to address these issues. We call the language temporal query
language (TQL). TQL basically allows us to query about information while constraining them
over time. A detailed description will be provided in upcoming sections. For this language, we
take certain assumptions about the notion of time in the target knowledge graph, as described
below.</p>
        <sec id="sec-4-1-1">
          <title>2.1. KG Requirements</title>
          <p>The query language we present in this work takes the following assumptions about the target
knowledge graph.</p>
          <p>
            Universality of Schema The schema or the ontology part of a given KG graph is assumed to
be universal. i.e., we do not allow time to be associated with ontological axioms. In other words,
the schema is true at all time points i.e., the axioms or knowledge about a particular domain
are not time dependent. Further, we want to avoid computational issues like undecidability of
query answering [
            <xref ref-type="bibr" rid="ref1">1</xref>
            ].
          </p>
          <p>Temporal Data Not all data need to be temporal. But the parts of the data with temporal
information need to be associated with some notion of time. In the previous section, we
have seen example how a timestamp can be associated to a triple using the usual datatype
xsd:dateTime. Nevertheless, this is not a hard requirement in our case. All we need is some
way of ordering information. Further, we need this ordering to be linear i.e., give a time point 
in this order, there is at most one successor  ′ time point of  . A KG  thus can be perceived
as a union of   ,   1, … ,    where   represents the schema part of  along with the triples
which are not associated to some time and hence contain no temporal information, whereas   
represents the set of triples which are associated with time point   for 1 ≤  ≤  . Without loss
of generality, we associate time to a set of triples onward.</p>
        </sec>
      </sec>
    </sec>
    <sec id="sec-5">
      <title>3. Use-case</title>
      <p>In this work, we apply TQL to improve the analysis of trafic video feeds: the goal is to infer
complex behavior from object classes, positions, trajectories recognized by state-of-the-art
machine vision systems running on top of stationary cameras. A trafic monitoring ontology
has been created, to represent the semantic content or scene of each video frame, and to generate
a knowledge graph with the annotated data.</p>
      <sec id="sec-5-1">
        <title>3.1. Trafic Monitoring Ontology</title>
        <p>Videos collected from diferent cameras are analyzed frame by frame. For each frame we get
information about the objects and their characteristics in it. We describe this information in a
KG using diferent classes and relationships. Note that each frame is recorded at a particular</p>
        <p>Query 3: Triples describing an observation about a car in a frame
1 tmo:atomicScene1 tmo:composedOf tmo:position1.
2 tso:TimeStamp ”2021-03-03T07:30:44”^^xsd:dateTime;
3 tmo:position1 rdf:type tso:Position;
4 tmo::hasParticipant tmo:car1;
5 tmo::hasProperty ?speedProperty1.
6 ?speedProperty1 rdf:type tso:Speed;
7 tmo:hasValue 20.0.</p>
        <p>time point. For our use-case, we consider these time points when dealing with dynamic aspect
of the domain. The notion of time will be explained in Section 5.</p>
        <p>Figure 1 shows a snippet of the trafic monotoring ontology (TMO). We next go through the
classes and relations in TMO.</p>
        <p>AtomicScene and Observation The class AtomicScene is a composition of the class
Observation. The elements of Observation represent diferent types of observations made in a
given frame. In our case, the only observations we have are of type Position, which describes a
trafic object using some spatial coordinates 4. However, as an example, if there is a car observed
in a frame, triples describing all the relevant information about the car are shown in Query 3.
As shown on line 4, Position (Observation) is used to accumulate information about a trafic
object (tmo:car1 in this case). What we are describing is that in the given frame we make an
observation (of type Position) in which we have a participant (tmo:car1), a property (tmo:Speed)
for which we have a value of 20.0. Finally, note that that Line 3 associates a single time point
to tmo:atomicScene1. We could actually associate the same time to every triple in Query 3.
But since all the triples carry the same time point, we can group them using an instance of
AtomicScene.</p>
        <p>ComplexScene A complex scene is a composition of atomic scenes. Which atomic scenes
constitute a complex scene depends on the pertinent interpretations derived by the use case
(see Event) or external constraints.
4Including geo-spatial coordinates, bounding boxes and Cartesian coordinates, etc. Note that Figure 1 is only
showing a partial view of the ontology, focused on temporal aspects.
Event Previously we mentioned that complex scenes are composition of atomic scenes. We
do not enforce any condition on which atomic scene can be composed into a complex scene. In
practice however, we are not interested in random aggregates of atomic scenes. To this end,
we use the class Event to represent scenes in a meaningful manner. Events thus can be seen
as semantic tags we put on scenes (atomic or complex). Figure 3.1 outlines this idea. As an
example, a scene can be interpreted as a congestion in, say, a village, while it can be interpreted
as a rush hour in a city. Pictorially, we see in Figure 3.1 the relationship between atomic scenes,
complex scenes and events. Diferent interpretations (events) can be associated to the same
complex scene.</p>
        <p>After presenting an overview of the relevant part of our KG, we next describe the need for
a temporal query language. In the next section, we will see why standard querying is not
suficient in use-cases like trafic monitoring where the data is inherently dynamic and thus
involves time as an aspect.</p>
      </sec>
    </sec>
    <sec id="sec-6">
      <title>4. Query Languages Limitations</title>
      <p>In this section, we describe why standard query languages like SPARQL are not suficient when
dealing with data, such as from trafic monitoring domain, which involves dynamic aspects. This
will advocate the idea of devising a more expressive query language which we will introduce in
Section 5. To this end, we present the following example scenarios.</p>
      <p>Query 4: query checking for accelerating car</p>
      <p>Accelerating car: Suppose we are interested in finding out if 1 ∶ a video (from a trafic
camera) contains an accelerating car. For simplicity we imagine that there is only one car (and
the same one) in each of the video frames. Suppose there is a video with 20 frames and that
these frames are represented as atomic scenes:  1, … ,  20 in a KG  . Let  1, … ,  20 be the time
point associated with  1, … ,  20 respectively in the order  1 &lt;  2 &lt; ⋯ &lt;  19 &lt;  20. Note that
each atomic scene   can be represented in  in the same fashion as in in Query 3. We, thus, are
only interested in the speed property of the position for each of the atomic scenes. The given
video is a candidate answer to our query if we observe that the speed of the car is increasing
as we go from atomic scene  1 all the way to  20. Retrieving such video from our knoweldge
graph can be formalized as a SPARQL query even though it is temporal in nature. All we need
to do is to compare the speed corresponding to the position in the atomic scene5 (frame) at time
  to speed corresponding to the position in the atomic scene at  +1 where 1 ≤  &lt; 20 . That
is, we compare ever consecutive pair of scenes for the speed value. Query 4 shows a snippet
of such a query. In general, such queries can be very long and non-trivial. As in our example
query, we see that we have to get speed for all the 20 atomic scenes.</p>
      <p>Note that not all queries dealing with temporal data can be translated into a SPARQL query
(or at least to a single SPARQL query). We call the queries that can be translated into some
equivalent SPARQL representation, simple temporal queries; we refer to the other queries as
complex temporal queries. The next example illustrates the latter type.</p>
      <p>Congestion ending in an accident: Suppose we are interested in videos where we
encounter trafic congestion ending up in an accident. Let’s assume by congestion we mean that
the cars are not moving at all for at least 10 time points6. Further, we identify an accident if two</p>
      <sec id="sec-6-1">
        <title>5Frames are represented as instances of the class AtomicScene</title>
        <p>6We can assume any number of seconds to be one time point.
cars share the same geo-spatial space. In other words, they have overlapping bounding boxes.
A video is an answer to this query if (i) there are atomic scenes   0,   1, …    with time points
 1,  2, … ,   respectively such that all cars have the same position at least in atomic scenes   
for 0 ≤  ≤ 10 , (ii) no two cars have overlapping position in none of the atomic scenes    for
0 ≤  ≤ 10 , and (iii) there is a time point   with 10 &lt;  ≤  such that there are at least two cars
with overlapping positions in the atomic scene    . Further, none of the car changes position
nor two car’s position overlaps in the scenes    for 10 &lt;  &lt;  . Note how Condition (i) and (ii)
check for a congestion (10 time points) and not happening of an accident. Whereas, Condition
(iii) checks for the earliest time point where we encounter an accident. Unlike Condition (i)
and (ii), the last condition cannot be translated into a single SPARQL query as we do not know
prior how many frames we need to check. This requires some sort of a while-loop which keeps
running till the condition is satisfied.</p>
        <p>The goal of this work is to answer queries as shown in the above examples. Note that even
though the accelerating car example (simple temporal query) can be translated into a SPARQL
query, but such queries are large and complex, and thus are prone to error. Next we present a
query language which allows expressing simple as well as complex temporal queries.</p>
      </sec>
    </sec>
    <sec id="sec-7">
      <title>5. Towards a Formal Query Language</title>
      <p>In this section, we define the formal syntax and semantics for the temporal query language we
mentioned in the previous section. We first start with some basic notions.</p>
      <sec id="sec-7-1">
        <title>5.1. Preliminaries</title>
        <p>
          All the knowledge graphs we consider here are assumed to be OWL based knowledge graphs7
which roots in Description Logics (DLs) [
          <xref ref-type="bibr" rid="ref3">3</xref>
          ]. We now present some preliminary definitions.
Definition 5.1 (Assertional Triples). A class assertion triple is a triple of the following form
[individual] rdf:type [class].
        </p>
        <sec id="sec-7-1-1">
          <title>A data property assertion triple is a triple of the following form</title>
          <p>[individual] [data property] [data value].</p>
          <p>An object property assertion triple is a triple of the following form</p>
          <p>[individual] [object property] [individual].</p>
          <p>
            Based on these notions, we next define what we mean by a temporal knowledge graph.
Definition 5.2 (Temporal Knowledge Graph). Let  be a non-empty set and &lt; be a strict partial
order on  . Further, let  be a set of triples. Then, a temporal knowledge graph (or TKG in
7OWL is an acronym for Web Ontology Language a W3C standard for describing ontologies [
            <xref ref-type="bibr" rid="ref2">2</xref>
            ]. Here we assume
basic familarity to RDF, RDFS and OWL.
short) based on ( , &lt;) and  is a knowledge graph if there is a set  of assertional RDF triples
with  ⊆ 
such that
where   ∈  for 1 ≤  ≤  . We represent the knowledge graph by  (,&lt;) .
          </p>
          <p>The intuition behind this definition is that the set of all assertional triples in a temporal
knowledge graph can be partitioned into subsets    based on some order given by &lt;. In
practice, we can take  to be a set of xsd:dateTime literals and &lt; is such that   &lt;   if   is
before   for two distinct xsd:dateTime times   and   in  . Note that by definition, we do not
require  to be a set of xsd:dateTime always. All we need is to have a strict partial order &lt; on 
. Hence,  can be set of integers or even strings so far we can establish &lt; amongst its elements8.
To this end, we define a partial function time ∶  ↦ 
that for any triple  ∈    we have time( ) =   .</p>
          <p>
            Definition 5.2 describes our notion of time. In fact, by time we mean just an order as required
by &lt; in the definition of the temporal knowledge graphs. We difer with the usual notion of
time (or the natural time) represented by xsd:dateTime. Our definition doesn’t restrict us to
xsd:dateTime only. In fact, ( , &lt;) is the only requirement in our definition i.e., all we need
is some way of ordering. At first glance, these requirements seem to be a bit confusing for
use-cases in practice. However, a knowledge graph where assertional triples can be grouped
based on some order is a temporal knowledge graph according to our definition. We will see
later why these requirements sufice when we define temporal operators. From now onward,
by time we mean an element from some set on which a strict partial order is defined.
called the time association function such
Representation of Temporal Knowledge Graph:
The representation of time can be a
part of the temporal knowledge graph itself. To elaborate this, suppose we have a temporal
knowledge graph  (,&lt;) and  is the subset of all assertional triples in  with
 =

⋃  
=1
second representation.
then the two possible representations of time in a temporal knowledge graph are either
associating each triple in    is to   using techniques like reification [
            <xref ref-type="bibr" rid="ref4">4</xref>
            ] or grouping all the
triples in    in a set and the set is associated to   . Without loss of generality we assume the
          </p>
        </sec>
      </sec>
      <sec id="sec-7-2">
        <title>5.2. Temporal Query Language (TQL)</title>
        <p>
          As mentioned in the previous section, we restrict the association of time to class assertions, data
property assertions and object property assertions only. This restriction allows us to simplify
TQL without compromising its expressivity. Further, we avoid issues with computational
decidability, which otherwise would be highly probable when dealing with such languages [
          <xref ref-type="bibr" rid="ref5">5</xref>
          ].
8This ordering can be implicit as well in the sense that we can relate diferent sets of assertional triples using, say, a
property next or successor etc.
        </p>
        <p>
          Before defining the syntax of TQL, we define some further notions. Here we assume some
familiarity with SPARQL query language. We refer the interested reader to [
          <xref ref-type="bibr" rid="ref4">4</xref>
          ]. In SPARQL,
basic graph patterns (BGPs) are the building blocks of the language. A BGP is an RDF triple
where variables are allowed at the position of subject, predicate as well as object. Similar to
this, we have the following definitions.
        </p>
        <p>Definition 5.3 (Assertional Basic Graph Pattern). An assertional basic graph pattern or (ABGP
in short) is an assertional RDF triple of one of the following forms:
• [individual |variable] rdf:type [class].
• [individual] [data property] [data value |variable].</p>
        <p>• [individual |variable] [object property] [individual |variable].</p>
        <p>By definition, every assertional triple is an assertional basic graph pattern (without variables)
as well as a standard basic graph pattern. ABGPs simply restricts the occurrence of variables at
certain positions compared to the standard ones. Hence, each ABGP can be queried against a
knowledge using any SPARQL engine.</p>
        <p>Definition 5.4 (ABGP Solution). Let  be an ABGP with var( ) representing the set of all
variables occuring in  . Then a solution for  in given a knowledge graph  is a partial function
 from var( ) to RDF terms in  such that the assertional triple obtained by simultaneously
replacing each variable  ∈ var( ) by () , can be found in  i.e., () ∈  .</p>
        <p>The notion of solutions can be extended to set of AGBPs in an obvious way: for a set of
ABGPs Γ, we say  is a solution for Γ if and only if  is a solution for each  ∈ Γ .</p>
        <p>Note that as shown in ’accelerating car’ example in Section 4, a temporal query checks
whether a certain relation holds over time. In other words, we want certain facts to hold when
traversing from one time point to another. We call such a requirement as a temporal constraint.
Hence, a temporal constraint is a condition that need to be satisfied between sets of ABGPs
representing factual information where each set corresponds to a time point. In fact, constraints
can be applied to the variables occurring in ABGPs due to the fact that variables can represent
diferent values (RDF terms) in a temporal KG at diferent time points. Hence, a class ABGP can
be constrained at the subject position only whereas data and obejct ABGPs can be constrained
at both subject as well as object positions. We will see later how the notion of ABGP solutions
can be applied to ABGPs with constrained variable.</p>
        <p>In practice, we can have diferent temporal constraints depending on the required conditions.
Variable constrained under constraint ℭ is a variable which need to satisfy the conditions
enforced by  . ℭ(?) represents a variable ? constrained under ℭ.</p>
        <p>Constrained ABGP is an ABGP such that at least one variable is constrained under exactly
one constraint. If the variable is constrained under ℭ, we say the ABGP is constrained under
ℭ. A constrained ABGP is written as an ABGP except that we write ℭ() for the constrained
variable  . For example, the following is a constrained ABGP where ?
is constrained
under rigidity (defined below).</p>
        <p>tmo:car_1 tmo:position ℜ(?position).</p>
        <p>We define two temporal constraints of interest that we use in this work. To this end, let
 (,&lt;) be a temporal KG9 with  =  ∪</p>
        <p>where
and   ∈  for 1 ≤  ≤  . Then,
Definition 5.5.</p>
        <p>(Rigidity) The Rigidity constraint is represented by ℜ and requires the values
of the variables not to change when moving from one time point to another. Formally, we
say rigidity is existentially satisfied for a constrained ABGP  in the temporal knowledge graph
 (,&lt;) for time  to  ′ if and only if there is a solution  for  in  ∪ 
 and a solution  ′ for 
in  ∪ 
 ′ such that for each  ∈ var( ) constrained under rigidity we have that () = 
′() .</p>
        <p>We represent existential rigidity by ℜ .</p>
        <p />
        <p>We say rigidity is universally satisfied for  in the temporal knowledge graph  (,&lt;) for time 
to  ′ if and only if for every solution  for  in  ∪ 
 , there is a solution  ′ for  in  ∪ 
 ′ such
that for each  ∈ var( ) constrained under rigidity we have that () = 
universal rigidity by ℜ .
′() . We represent</p>
        <p>Note that rigidity requires both mappings  and  ′ to be solutions for  with the additional
requirement that the constrained variable (under rigidity) in  is mapped to the same value by
both mappings. The intuition behind rigidity constraint is that when moving from time  to  ′,
the value for the constrained variable does not change and interprets the triple rigidly for the
variables constrained under rigidity. Sometimes we intend to allow for minor changes in the
values of variables. We define the so-called likelihood constraint. Note that such a constraint
makes sense in case of data property assertional triples as we can enforce likelihood on the
values of the data property only. The amount of change for data values can be measured using
some threshold function i.e., the satisfaction of likelihood constraint relies on certain threshold.
We would like to mention here that the definition of threshold functions as well as threshold
values can be defined as per use-case. This adds a great flexibility in controlling change over
time.</p>
        <p>Definition 5.6 (Likelihood). Let  be a data property ABGPs and let D be the range of the data
property in  . In other words, the object position in  is either a variable or a value from D.
Further let  ∈ var( ) be the variable at object position. Let  be a threshold function on D i.e.,
if and only if there is a solution  for  in  ∪ 
 ∶ D × D ↦ {true, false}. We say likelihood is satisfied for  in  (,&lt;)
 and a solution  ′ for  in  ∪</p>
        <p>′ such that
given  for time  to  ′
 ((),</p>
        <p>′() ) = true. In other words,  is mapped to two values, say,  1 and  2 by  and  ′
respectively such that  ( 1,  2) yields true.
9Here  represents the non-temporal part of the knowledge graph. This will be our assumption onward.</p>
        <p>From now onward we assume there is always a given threshold function when we consider
constrained ABGP where the variable is constrained under likelihood, unless stated otherwise.
Note that the notion of universality and existentiality does not make sense in case of likelihood
constrained.</p>
        <p>Example of Constrained ABGPs: Consider the following ABGP:
Here we suppose that the following threshold function is given for ( ?speed)
 ∶</p>
        <p>tmo:car_1 tmo:Speed ( ?speed).
 (,  ′) = {
true
false
if  ′ −  ≤ 1.5
otherwise
Let  and  ′ be solutions for  in  ∪   and  ∪   ′ respectively such that ( ?speed) = 15
and  ′(?speed) = 18. We thus see that likelihood (with the given threshold function) is satisfied
for  in  (,&lt;) for  to  ′ as both  and  ′ are solutions as well as  (18, 17) = true since
18 − 15 = 3 ≤ 1.5.</p>
        <p>The notion of temporal constraint allows us to enforce certain conditions to be met by
solutions when moving over time in a knowledge graph. In other words, temporal constraints
are ways to monitor how information in knowledge alters over time. However, temporal
operators allow us to traverse over time points. There can be diferent temporal operators.
While temporal constraints enforces conditions that need to be satisfied over time, the temporal
operators govern what time points need to be considered when applying these constraints.
5.2.1. Syntax
Standard temporal logic allows for diferent temporal operators. We, however, we define two
temporal operators namely, N and U for defining TQL.</p>
        <p>Definition 5.7 (Syntax of TQL). Let  (,&lt;) be a temporal knowledge graph with  =  ∪ 
where  =   1 ∪   2 ∪ ⋯ ∪    . Let  be a set of ABGPs and  ℭ be a set of constrained ABGPs.
Then,</p>
        <sec id="sec-7-2-1">
          <title>I. any subset of  is a temporal query.</title>
          <p>II. if Λ ⊆  , Γ ⊆  ∪  ℭ such that no variable in Γ is constrained under more than one
constraint and Λ′ is non-empty set with Λ′ ⊆  , then
• Λ N (Γ) for  ≥ 1 is a temporal query called N temporal query.
• Λ.ΓUΛ′ is a temporal query called U query.</p>
          <p>• ΛN (Γ)UΛ′ is a temporal query where  ≥ 1 . We call such query as N U query.</p>
          <p>Note how constrained ABGPs are used along with the temporal operators. For
simplicity, we restrict to non-recursive occurrences of the temporal operators in temporal queries.
Nevertheless, the language can easily be extended to cover more complex queries.
5.2.2. Semantics
In Definition 5.3, we have mentioned that assertional basic graph patterns (ABGPs) are by very
definition basic graph patterns as in SPARQL. Also, in Definition 5.4, we have seen the notion of
ABGP solution which again is what we have in standard SPARQL. Note that, similar to SPARQL,
a set of ABGPs can be seen as a query. Thus, for a given set of ABGPs Φ and a knowledge graph
 , a solution for Φ in  is simply answer to Φ. Based on this, we next define the notion of TQL
query answers.</p>
          <p>Definition 5.8. Suppose  (,&lt;) is a temporal KG with  =  ∪ where  =   1 ∪  2 ∪⋯∪  
and   ∈  for 1 ≤  ≤  . Further, Let  be a set of ABGPs and  ℭ be a set of constrained ABGPs.
An answer/solution to  in  (,&lt;) is defined as following:
•  = { 1, … ,   } ⊆ 
a mapping  from var( ) to RDF terms is a solution or an answer to  in  (,&lt;) if and
only if (  ) is a match in  (,&lt;) for 1 ≤  ≤  . We say  is a solution or an answer for 
in  (,&lt;) . Note that in this case, we have no constraint nor time to consider as the query
is just a set of ABGPs. It, thus can be seen as a SPARQL query.
•  = Λ N (Γ) for Λ ⊆  , Γ ⊆  ∪  ℭ for  ≥ 1
there are time points  0, …   and mappings  0, … ,   such that:  0 is a solution for Λ in
 ∪   0,   is a solution for Γ in  ∪    for 0 ≤  ≤  , and the constraints in  ∈ Γ are
satisfied for  in  (,&lt;) for   to  +1 where 0 ≤  &lt;  .</p>
          <p>The solution to  for  is obtained by combining all above   while replacing in constrained
variable  by  () i.e., if  is a non-constrained variable in  and  ↦   () is in   then it
is replaced by  () ↦   () in  . In simple words, we create copies of the non-constrained
variables for each time point.
• Λ.ΓUΛ′ for Λ ⊆  , Γ ⊆  ∪  ℭ and a non-emtpy set Λ′ ∈ 
there are time points  0,  1, … and mappings  0,  1,  , and there exists  ≥ 0 such that:
 0 is a solution for Λ in  ∪  0,   is a solution for Γ in  ∪   for 0 ≤  &lt;  , each constraint
in constrained ABGP  ∈ Γ is satisfied for  in  (,&lt;) for   to  +1 for 0 ≤  &lt;  − 1 , there
is no solution for Λ′ in  ∪    for 0 ≤  &lt;  , and   is a solution for Λ′ in  ∪    .
• ΛN (Γ)UΛ′ for Λ ⊆  , Γ ⊆  ∪  ℭ and a non-emtpy set Λ′ ∈ 
there are time points  0, … ,   ,  +1 , … and mappings  0, … ,   ,  +1 , … such that:  0 is a
solution for Λ in  ∪   0,   is a solution for Γ in  ∪    for 0 ≤  ≤  , the constraints in
 ∈ Γ are satisfied for  in  (,&lt;) for   to  +1 where 0 ≤  &lt;  , there is a time point  ≥ 
such that   is a solution for Γ in  ∪    for  ≤  &lt;  , the constraints in  ∈ Γ are satisfied
for  in  (,&lt;) for   to  +1 where  ≤  &lt;  − 1 , there is no solution  ′ for Λ′ in  ∪   
for  ≤  ≤  − 1 , and   is a solution for Λ′ in  ∪    .</p>
          <p>Note that the temporal operator N allows us to iterate over a fix number of time points 
and check whether a temporal constraint is satisfied or not. Meanwhile, the temporal operator
U checks the validity of a temporal constraint for as long as certain condition is satisfied. In
our case, this condition corresponds to the origination of some new facts represented by the set
Λ′. Let’s consider an example of a temporal query from trafic monitoring domain.</p>
          <p>U

.</p>
          <p>An Accelerating Car Ending in an Accident
Suppose a car encounters an accident if it’s
bounding box overlaps with that of another car. Then, the query can be formalized as
{?scene rdf:type tmo:Scene.</p>
          <p>ℜ (?car) rdf:type tmo:Car.</p>
          <p>?car tmo:hasSpeed ( ?speed).}
{?car tmo:hasPosition ?position.</p>
          <p>car2 rdf:type tmo:car.
?car2 tmo:hasPosition ?position2.</p>
          <p>?position tmo:overlaps ?position2.}
We assume that the threshold function for ( ?speed) is as:  (,  ′
) = true if  ′ &gt;  and
 (, 
′
) = true otherwise. Note the likelihood constraint with threshold function  on the
variable ?speed makes sure that the value of ?speed increases as we move over time. This way
we capture the notion of acceleration for the car. Further, the above query is of the form Λ.ΓUΛ′
with Λ = ∅. Further, ℜ (?car) constraints the existence of a car over all time points unless there
is a time point  where we we find a second car with position of the first car overlapping that of
the second one in the knowledge graph  ∪</p>
        </sec>
      </sec>
    </sec>
    <sec id="sec-8">
      <title>6. Related Work</title>
      <p>
        Incorporating time into RDF has been investigated in prior work [
        <xref ref-type="bibr" rid="ref10 ref11 ref6 ref7 ref8 ref9">6, 7, 8, 9, 10, 11</xref>
        ]. By and
large, these studies difer in aspects like treatment of the notion of time and expressivity of the
query language. Contributions like [
        <xref ref-type="bibr" rid="ref6">6</xref>
        ] extend the idea of TSQL2 [12] in traditional databases to
RDF. The treatment of time there is diferent from the usual notion of time in classical temporal
logic like LTL [13]. Nevertheless, works like [
        <xref ref-type="bibr" rid="ref11">11</xref>
        ] extends SPARQL with constructs provided
in LTL. A thorough comparison of our approach to the state-of-the-art is beyond the scope
of this paper. However, the treatment of time in our work exhibits some similarities to the
one presented in [
        <xref ref-type="bibr" rid="ref6">6</xref>
        ]. What distinguishes TQL is that it considers only one dimension for time,
as our definition of temporal KG is based on a linear order (T, &lt;) unlike N-dimensional time
 =
      </p>
      <p>
        1 ×  2 × ⋯ ×   in [
        <xref ref-type="bibr" rid="ref6 ref7">6, 7</xref>
        ], where the focus is ontology versioning. When it comes to our
query language, our work is somewhat similar to SPARQL-LTL presented in [
        <xref ref-type="bibr" rid="ref11">11</xref>
        ]. However, the
focus in [
        <xref ref-type="bibr" rid="ref11">11</xref>
        ] when it comes to the notion of time is more on versioning of RDF data. Hence, the
defined semantics consider diferent named graphs where each graph represents one version
of the data. As pointed out by the authors, it is not a limitation per se. We believe this is
comparable to our definition of temporal knowledge graph (Definition
      </p>
      <sec id="sec-8-1">
        <title>5.2), where without loss</title>
        <p>of generality, we assume a set of triples to be associated with a time point.</p>
        <p>Besides all similarities to the existing approaches, the notion of temporal constraint, to the
best of our knowledge, adds new expressivity to SPARQL extended with the temporal operators.
Even though such constraints are translated into SPARQL checks, the succinctness they add to
the query language is of great advantage, as it helps to reduce complex queries to simpler and
more compact ones (as shown in the examples in the trafic monitoring domain).</p>
      </sec>
    </sec>
    <sec id="sec-9">
      <title>7. Conclusion and Future Work</title>
      <p>We have discussed the benefit of temporal queries in knowledge graphs. We then presented
a formal query language for formalizing temporal queries. We called it as Temporal Query
Language (TQL). We have precisely defined syntax and semantics of TQL, where we considered
diferent types of temporal queries. We have seen how temporal constraints adds further
expressivity to the query language. We have already a first implementation of a query engine
for answering temporal queries. A description of the engine, along with a report on experiments
and evaluation conducted, is beyond the scope of this paper. In future work, we would like to
analyze what other types of temporal queries can be of interest for diferent use-cases and, in
parallel, developing an advanced query editor for TQL.
of the 3rd Workshop on Managing the Evolution and Preservation of the Data Web
(MEPDaW 2017) and the 4th Workshop on Linked Data Quality (LDQ 2017) co-located
with 14th European Semantic Web Conference (ESWC 2017), Portorož, Slovenia, May
28th-29th, 2017, CEUR-WS.org, 2017.
[12] J. Cliford, C. E. Dyreson, R. T. Snodgrass, T. Isakowitz, C. S. Jensen, ”now”, in: R. T.</p>
      <p>Snodgrass (Ed.), The TSQL2 Temporal Query Language, Kluwer, 1995, pp. 383–392.
[13] F. Kröger, S. Merz, Basic Propositional Linear Temporal Logic, 2008.</p>
    </sec>
  </body>
  <back>
    <ref-list>
      <ref id="ref1">
        <mixed-citation>
          [1]
          <string-name>
            <given-names>C.</given-names>
            <surname>Lutz</surname>
          </string-name>
          ,
          <string-name>
            <given-names>F.</given-names>
            <surname>Wolter</surname>
          </string-name>
          ,
          <string-name>
            <given-names>M.</given-names>
            <surname>Zakharyaschev</surname>
          </string-name>
          ,
          <article-title>Temporal description logics: A survey</article-title>
          ,
          <source>in: Proceedings of the Fifteenth International Symposium on Temporal Representation and Reasoning</source>
          , IEEE Computer Society Press,
          <year>2008</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref2">
        <mixed-citation>
          [2]
          <string-name>
            <given-names>S.</given-names>
            <surname>Bechhofer</surname>
          </string-name>
          ,
          <string-name>
            <surname>F. van Harmelen</surname>
          </string-name>
          ,
          <string-name>
            <given-names>J.</given-names>
            <surname>Hendler</surname>
          </string-name>
          ,
          <string-name>
            <given-names>I.</given-names>
            <surname>Horrocks</surname>
          </string-name>
          ,
          <string-name>
            <given-names>D.</given-names>
            <surname>McGuinness</surname>
          </string-name>
          ,
          <string-name>
            <given-names>P.</given-names>
            <surname>Patel-Schneijder</surname>
          </string-name>
          ,
          <string-name>
            <given-names>L. A.</given-names>
            <surname>Stein</surname>
          </string-name>
          ,
          <source>Technical Report</source>
          ,
          <year>2004</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref3">
        <mixed-citation>
          [3]
          <string-name>
            <given-names>F.</given-names>
            <surname>Baader</surname>
          </string-name>
          , I. Horrocks,
          <string-name>
            <given-names>C.</given-names>
            <surname>Lutz</surname>
          </string-name>
          ,
          <string-name>
            <given-names>U.</given-names>
            <surname>Sattler</surname>
          </string-name>
          , An Introduction to Description Logic, Cambridge University Press,
          <year>2017</year>
          .
          <source>doi:1 0 . 1 0</source>
          <volume>1 7 / 9 7 8 1 1 3 9 0 2 5 3 5 5 .</volume>
        </mixed-citation>
      </ref>
      <ref id="ref4">
        <mixed-citation>
          [4]
          <string-name>
            <given-names>P.</given-names>
            <surname>Hitzler</surname>
          </string-name>
          ,
          <string-name>
            <given-names>M.</given-names>
            <surname>Krtzsch</surname>
          </string-name>
          ,
          <string-name>
            <given-names>S.</given-names>
            <surname>Rudolph</surname>
          </string-name>
          , Foundations of Semantic Web Technologies, 1st ed.,
          <source>Chapman</source>
          and Hall/CRC,
          <year>2009</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref5">
        <mixed-citation>
          [5]
          <string-name>
            <given-names>C.</given-names>
            <surname>Lutz</surname>
          </string-name>
          ,
          <string-name>
            <given-names>F.</given-names>
            <surname>Wolter</surname>
          </string-name>
          ,
          <string-name>
            <given-names>M.</given-names>
            <surname>Zakharyaschev</surname>
          </string-name>
          ,
          <article-title>Temporal description logics: A survey</article-title>
          ,
          <source>in: Proceedings of the Fifteenth International Symposium on Temporal Representation and Reasoning</source>
          ,
          <year>2008</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref6">
        <mixed-citation>
          [6]
          <string-name>
            <given-names>F.</given-names>
            <surname>Grandi</surname>
          </string-name>
          ,
          <string-name>
            <surname>T-SPARQL</surname>
          </string-name>
          :
          <article-title>A tsql2-like temporal query language for RDF</article-title>
          ,
          <source>in: Local Proceedings of the Fourteenth East-European Conference on Advances in Databases and Information Systems</source>
          , Novi Sad, Serbia,
          <source>September 20-24</source>
          ,
          <year>2010</year>
          , CEUR Workshop Proceedings,
          <year>2010</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref7">
        <mixed-citation>
          [7]
          <string-name>
            <given-names>F.</given-names>
            <surname>Grandi</surname>
          </string-name>
          <article-title>, Multi-temporal RDF ontology versioning</article-title>
          ,
          <source>in: Proceedings of the 3rd International Workshop on Ontology Dynamics</source>
          ,
          <source>(IWOD</source>
          <year>2009</year>
          ) ,
          <article-title>collocated with the 8th International Semantic Web Conference ( ISWC-</article-title>
          <year>2009</year>
          ), Washington DC, USA, October
          <volume>26</volume>
          ,
          <year>2009</year>
          , CEUR Workshop Proceedings,
          <year>2009</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref8">
        <mixed-citation>
          [8]
          <string-name>
            <given-names>J.</given-names>
            <surname>Tappolet</surname>
          </string-name>
          ,
          <string-name>
            <surname>A</surname>
          </string-name>
          . Bernstein,
          <article-title>Applied temporal RDF: eficient temporal querying of RDF data with SPARQL</article-title>
          ,
          <source>in: The Semantic Web: Research and Applications, 6th European Semantic Web Conference, ESWC</source>
          <year>2009</year>
          , Heraklion, Crete, Greece, May 31-June 4,
          <year>2009</year>
          , Proceedings,
          <year>2009</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref9">
        <mixed-citation>
          [9]
          <string-name>
            <given-names>A.</given-names>
            <surname>Pugliese</surname>
          </string-name>
          ,
          <string-name>
            <given-names>O.</given-names>
            <surname>Udrea</surname>
          </string-name>
          ,
          <string-name>
            <given-names>V. S.</given-names>
            <surname>Subrahmanian</surname>
          </string-name>
          ,
          <article-title>Scaling RDF with time</article-title>
          ,
          <source>in: Proceedings of the 17th International Conference on World Wide Web, WWW</source>
          <year>2008</year>
          , Beijing, China,
          <source>April 21-25</source>
          ,
          <year>2008</year>
          ,
          <year>2008</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref10">
        <mixed-citation>
          [10]
          <string-name>
            <given-names>C.</given-names>
            <surname>Gutierrez</surname>
          </string-name>
          ,
          <string-name>
            <given-names>C. A.</given-names>
            <surname>Hurtado</surname>
          </string-name>
          ,
          <string-name>
            <given-names>A. A.</given-names>
            <surname>Vaisman</surname>
          </string-name>
          ,
          <article-title>Introducing time into RDF</article-title>
          ,
          <source>IEEE Trans. Knowl. Data Eng</source>
          . (
          <year>2007</year>
          ).
        </mixed-citation>
      </ref>
      <ref id="ref11">
        <mixed-citation>
          [11]
          <string-name>
            <given-names>M. W.</given-names>
            <surname>Chekol</surname>
          </string-name>
          ,
          <string-name>
            <given-names>V.</given-names>
            <surname>Fionda</surname>
          </string-name>
          ,
          <string-name>
            <surname>G.</surname>
          </string-name>
          <article-title>Pirrò, Time travel queries in RDF archives</article-title>
          , in: Joint proceedings
        </mixed-citation>
      </ref>
    </ref-list>
  </back>
</article>