Reactive decision making on heterogeneous streaming data is gaining importance in a wide range of situations, e.g., in the electricity management domain for reacting to anomaly consumption or in oil and gas extraction sites to detect dangerous situations. Modern cities represent a relevant scenario for reactive decision making because of the vast number of stakeholders willing to bene t from the growing availability of streams of data from various sources. In the state-of-the-art, this problem is addressed through ad-hoc solutions that t only a speci c scenario. In this paper, I report on the models and technical implementations, which I propose to enable reactive analysis of a variety of spatio-temporal data, and on their evaluation in real-world scenarios to prove their adequacy.
In an increasing number of situations, a decision must be reactive1 and must be based on a variety of streaming data. In the electricity management domain, a reactive anomaly detection system for the consumption data is useful to avoid network problems. In the oil and gas extraction sites, the analyses of sensors' readings from the wells are vital for reactive detection of dangerous situations.
The urban environment is particularly relevant when talking about
reactive decision. In modern cities, a dense network of interactions between people
and the environment produces a great amount of spatio-temporal fast
evolving data [
In the mid 2000s', the growing use of location-based social networks via mobile devices, improved the ability to capture the people's interests, habits, and
the planned action useless.
preferences in a privacy-preserving manner and enabled innovative scenarios. It
became possible to create an accurate and up-to-date representation of reality
(a.k.a. Digital footprint or Digital re ection or Digital twin) exploiting either
social media or mobile phones data, i.e. Call Data Records (CDR). For instance,
analyzing social media Cho et. al. [
However, better decisions can result from the analyses of multiple data sources simultaneously. The growing availability of new urban data sources (e.g. IoT, WIFI logs) stimulated the research of a conceptual model to manage data variety in a comprehensive way. The current interest is for solutions that fuse streaming heterogeneous data to enable reactive decisions. 2
Before starting my PhD, I investigated for three years the modeling and the
analysis of streaming data from social media [3, 7{9]. I approached the problem
with Stream Reasoning [
Re ecting on the obtained results, I identi ed two main ndings: (i) when
dealing with data stream, a continuous ingestion mechanism avoids data losses,
but continuous analysis is not always needed; an analysis can be reactive even if
postponed. (ii) Ontologies are an adequate knowledge representation technique
for modeling data characterized by high variety. In the previous works I counted
on two assumptions: (A) adequate ontologies to model a domain are available,
or they can be obtained with minimal e ort by extending existing ones. Indeed,
SMA[
In my PhD, aiming to continuously and reactively analyze a variety of
spatiotemporal data, I re ected on the nding and on the assumptions of my previous
work. Social media data is semi-structured: only time and space information is
presented in a structured way; the content is unstructured, e.g. free texts or
images. On the contrary, IoT data, WI-FI logs, CDRs are structured. While the
integration of semi-structured data is generally based on the content analysis
(e.g. named entity recognition and linking), the integration of structured data
requires other methods, e.g., Ontology Based Data Integration (OBDI) [
In approaching my PhD keeping working on Stream Reasoning, I needed to check if the assumptions of my previous work still hold. Assumption A does not hold in this extended scenario, so a rst problem emerges:
Rp.1 De ning a conceptual model to represent a variety of streaming data.
Moreover, Assumption B holds only to a limited extent, i.e. for social media data. Therefore, I need to face two problems :
Rp.2 De ning a streaming computational model to enable analysis on a variety of data.
Rp.3 De ning appropriate technical instantiations of the computational model in Rp.2.
Last, but not least, to verify and validate the solutions proposed to solve the problems above, I need to:
Rp.4 Assess, in real world scenarios, the feasibility and the e ectiveness of the instantiations developed addressing Rp.3 using the models developed in solving Rp.1 and Rp.2. 3
Concerning Rp.1, visual analytics is a common approach to support reactive
decision making, but there was a gap between low-level time-varying geo-located
data and the high-level needs of visual analytics. Vocabularies to publish the
lowlevel data exist, e.g., geosparql vocabulary5, event ontology6 or time ontology7,
but the high-level part, to enable visual analytics, was missing. Social Pixel [
The transient nature of streaming information often requires to treat it di
erently from persistent data. Data streams are often consumed on the y by
continuous queries. Such a paradigmatic change was investigated by the Database
community [
processing and OBDI to enable the layout of complex query networks that continuously analyze social media. But, as I already mentioned in Section 2, I based my works on Assumptions A and B, that don't hold in all the scenarios.
With regards to Rp.3 and to Rp.4, I assessed the work done in
benchmarking [
I developed my research question with the Macro, Mezzo and Micro method [
At Macro level I focused on relevancy and formulated the question: Is it possible to support reactive decisions by managing data characterized by velocity and variety without forgetting volume?
At Mezzo level, I focused the attention on a question for which I could nd a viable solution. I concentrate my e ort on spatio-temporal streaming data, I focused on the ndings of my previous work and I characterized the way to support reactive decisions, i.e. visually make sense of data. So, the Mezzo level question is: Is it possible to visually make sense of a variety of spatio-temporal streaming data by enabling continuous ingestion and reactive analysis?
Finally, at Micro level, I formalized a question that can be evaluated. I concentrate my e ort on the streaming urban data and I specify a way to exploit the visual analytics instrument to support reactive decision making, i.e. nd emerging patterns and data dynamics. As a result, my research question is: Is it possible to continuously ingest and reactively analyses a variety of streaming urban data in order to visualize emerging patterns and their dynamics?
In answering to the Micro level question, I'm directly contributing to answer the Mezzo level question, and, indirectly, to cast some light on the Macro level question. 5
Inspired by OBDI methods, I approached the research problems in a modular way by relaxing, in parallel, the two original assumptions presented in Section 2. This modularity re ects the research problems structures and allows me performing a continuous evaluation.
On the one hand, relaxing Assumption A, I approached the creation of a
conceptual model in the form of an ontology by following the Methontology [
On the other hand, relaxing Assumption B, I planned the development of
a computational model to enable continuous ingestion, wrangling and reactive
analysis of heterogeneous data streams. I planned to implement such a
computational model using di erent technologies, i.e. single-threaded and distributed,
in order to prove its adequacy in di erent work conditions. To nalize the work
I planned the evaluation of the cited implementations against already existing
system (SLD) and one against the other. In particular, inspired by COST [
The modular approach, during the development and the evaluations phases,
allowed me planning an overall evaluation. I planned to put at work a complete
system, composed by an implementation of the computational model that
exploits the conceptual model, in di erent scenario and to evaluate it: (i) in terms
of guessability [
In order to answer my research questions, I formulated a set of hypotheses that I used to operationalize my work, w.r.t. the four problems in Section 2.
Hp.1 A conceptual model containing concepts from the image processing domain can represent spatio-temporal data in an extendable and coherent way with a minimal encoding bias and a minimal ontological commitment.
Hp.2 A streaming computational model that defers as long as possible the data transformation is less complex, in terms of time and space, than a computational model that cast data into RDF at ingestion time. Hp.3 A single-threaded implementation of the streaming computational model from Hp.2 that uses the conceptual model from Hp.1 can be more coste ective than a distributed implementation of the same model while guaranteeing the reactiveness of the system Hp.4 An implementation from Hp.3 can create a bridge between data analytics and data visualization that enhances the comprehension of a variety of spatio-temporal data and, at the same time, is reactive. 7
To validate Hp.1, I created the FraPPE ontology. Figure 1(a) o ers a graphical
overview of the FraPPE concepts. The abstractions in the FraPPE ontology [
EB
EA A Places n
Events (a) n+1
Frames made of 4 Pixels B
Grid made of 4 cel s
S⟨τ⟩ Stream
S2I⟨τ⟩ I2S⟨τ⟩ (b)
I⟨τ⟩
Inst.
Collection
I2I⟨τ,τ′⟩
data model is the discretization of space and time in atomic units. The
representation of the geographical space is mediated by a Grid of Cell s that contain
Places. Frame, Pixel s and Event s are the time-varying representation (i.e. taken
every given interval of time) of, respectively, Grid, Cells and Places. FraPPE
was developed using Methontology [
In parallel, I investigated a streaming computational model to enable access and analysis of a variety of streaming data. The main idea behind this part of the work is to combine my previous ndings (see Section 2) with the intuition that, often, data transformation can be deferred (as stated in Hypothesis Hp.2). For example, if we need to lter a stream of JSON items in a rst stage of a long query network, the execution of a path query with JSONiq8, before transforming the data in RDF, is for sure faster than transforming the data in RDF and then executing a graph pattern matching.
Figure 1(b) shows the three proposed classes of operators inspired by
CQL [
As a rst implementation of the computational model, I developed Natron:
a direct improvement of SLD that maintains the single threaded nature of the
original platform. I empirically evaluated the performance of Natron against
SLD and validated Hypothesis Hp.2 by proving that a deferred data
transformation, namely Lazy Transformation principle, can improve the performance
of a stream processing framework [
implementations operate on data in its original format as long as they can, and
they transform it only if it is really needed. I evaluated the cost-e ectiveness
of the distributed implementation against Natron and I demonstrated that the
single-threaded implementation can outperform the distributed one [
In order to validate Hp.4, Natron and FraPPE were then put at work and evaluated in real-world scenario [28{30]. Those works demonstrate the validity of the whole infrastructure in various scenarios facing heterogeneous streaming data. The guessability and the reactiveness of the visual analytics instruments enabled by the system were evaluated by tens of real-world users via questionnaires and interviews. 8
During my PhD, I collected positive evidences that a system such as Natron (based on a streaming computational model and on the Lazy Transformation principle), and a conceptual model such as FraPPE (containing concepts inspired by image processing) represents an adequate solution to enable visual analytics of heterogeneous streaming urban data in a reactive way. Unfortunately, so far, the evaluation was conducted only exploiting the multiple implementations of the two proposed models. This approach poses limits to the positive evaluation of Hypothesis Hp.2. I now need to perform a formal evaluation of Hypothesis Hp.2. In the remaining part of my PhD, I intend to de ne a formal algebra for the computational model in order to estimate the time and space complexity of the operators and to de ne cost models that can be exploited to automatically optimize query networks designed by users with a limited know-how on the internals of my implementations.
Acknowledgments. I worked under the supervision of Prof. E. Della Valle.