=Paper=
{{Paper
|id=Vol-2622/paper10
|storemode=property
|title=An Edge Centric Middleware for City Surveillance Platform (CityPro)
|pdfUrl=https://ceur-ws.org/Vol-2622/paper10.pdf
|volume=Vol-2622
|authors=Mariam Hakim,Ibrahim Tawbe,Mohamed Dbouk
|dblpUrl=https://dblp.org/rec/conf/bdcsintell/HakimTD19
}}
==An Edge Centric Middleware for City Surveillance Platform (CityPro)==
https://ceur-ws.org/Vol-2622/paper10.pdf
An Edge Centric Middleware for City
Surveillance Platform (CityPro)
Mariam Hakim Ibrahim Tawbe Mohamed Dbouk
Faculty of Sciences - I Faculty of Sciences - I Faculty of Sciences - I
Lebanese University Lebanese University Lebanese University
Beirut, Lebanon Beirut, Lebanon Beirut, Lebanon
mariamhakim181@gmail.com ibrahim.tawbe@gmail.com mdbouk@ul.edu.lb
Abstract—Smarter Cities provide better triggers these risks is traditionally monitored by
management for city services, reporting problems, and different governmental agencies; the weather
predicting future issues to enhance city protection. The monitoring agency differs from local police forces. On
need for a robust and unified middleware platform in the other hand, these risks put citizens in danger and
Smart Cities is not yet a solved problem. Traditional coordinating the emergency procedures is critical to
middleware that evolved in enterprise environments lower the losses. Furthermore, detecting and dealing
can’t handle smart cities complexities. Therefore, a with an emergency is not enough, there is a need to
platform -CityPro- that integrates multiple existing predict and act before things happen. That’s why
systems into a collaborative environment to protect the
models are built and run to simulate real-world
city is proposed. It highlights the concept of
scenarios using machine learning and artificial
“connectors” between different city systems and a
central “core” system. In this article we provide a new intelligence. In addition to the models, early alarms
middleware (architecture and framework) - called Edge sometime come from the correlation between different
Centric Middleware- that adapts CityPro's architecture data sources. The data produced by these sources may
to connect existing city systems to the core system. The become gigantic over time; such data can be
suggested solution tries to solve issues such as merging considered as a valuable mine of information.
heterogeneous systems, event processing, real-time data, Therefore, a technique must be adapted to access this
scalability, availability, and interoperability, while data wisely in order to benefit the whole city.
exploring edge computing capabilities. We also provided
an implementation for this middleware to serve as an
These necessities nourished the idea of an
open-source generic framework that can be extended integrated platform of a collaborative surveillance
and customized. system, called CityPro [1].This system is intended to
protect and monitor people and public infrastructures.
Keywords—Smart Cities; Business Processing; Edge It is expected to:
Computing; Business Intelligence; Big Data; Smart Data
x Operate within live-mode by using the city
digital infrastructures
I. INTRODUCTION AND PROBLEM POSITION
x Combine and inter-operate heterogeneous pre-
In a typical modern city, multiple computerized existing operational systems
standalone systems exist, e.g. banks, hotels, and
hospitals. All of these systems work separately which
limits their powers to a specific business domain (e.g. A. CityPro; an overview
bank), specific area, etc. The amount of challenges a Ref [1] presents CityPro; a collaborative platform
city faces is increasing daily. Smart City paradigms for city protection tries to standardize the relation
approach these challenges by using the city’s between different systems with a centralized,
resources efficiently to optimize services and supervised control and data repository architecture
managements such as traffic control, electricity, public shown in Fig. 1. It defines data providers as domain-
safety, etc. It is worth to note that tackling these issues specific independent (stand-alone) systems that
using technological advancements is not for “modern” coexist within the considered territory such as police
cities only. Cities in developed countries can and departments, fire stations, banks, etc. In the context of
should take measures following Smart City paradigm CityPro these systems are data producers. They are
to solve challenges. Smart City is not a one-shot accessed through “collaboration-links” or
solution; developed countries can gradually build up “connectors”. CityPro defines a connector as
their Smart City eco-system. “Dedicated links that are materializing the
collaborative inter-relationships between CityPro
Public safety is not considered a luxurious service; components. They mainly consist of ETL like
it’s an essential challenge for any city. Cities are dedicated data exchange automated protocols based on
facing variety of risks such as natural disasters, ‘adapter pattern’.” CityPro delivered the general
terrorists’ attacks, crimes, vehicle accidents, etc. What architecture of the system, while we still need to
Copyright © 2019 for this paper by its authors. Use permitted under
Creative Commons License Attribution 4.0 International (CC BY 4.0).
64
investigate deeper into providing a standard and layer). In addition most data providers hold big data
uniform access to these heterogeneous distributed volumes. In its architecture the core system which
systems with minimum effort at the data provider side. holds the federated data repository is responsible for
analysis and decision making.
This article is dedicated to finding a feasible
solution that uses edge computing concepts to
optimize data transfer and delegate some core system
operations and computations to near-source
components especially in the case of real-time alerts,
while maintaining the security when moving and
accessing remote data from other parties. These
operations should scale accordingly and should be
done reliably where the system should guarantee high
availability and fault tolerance.
Fig. 1. CityPro's General Architecture
In the later sections of this paper, we are going to
To address the need for city surveillance, CityPro introduce an architecture that adopts edge computing
defines two data flows: periodically where data concepts and incorporate techniques to access
continually arrives from data providers and on- distributed heterogeneous information systems. By
demand where the central system asks providers for creating a middle layer between the Core System and
instant detailed data. We need to define the standard the data providers such as banks, police, airports,
protocols for this data exchange. customs, etc. The proposed architecture is expected to
enhance the interoperability within the system itself
Highly distributed systems of this kind may and the city systems. It is also expected to maintain a
produce gigantic amounts of data. A pure central high availability and create a scalable cache for
system might struggle in handling all this data flow messages from various providers.
and delivering value, especially in the case of near
real-time alerts. That’s why it’s tempting to use the In this paper, Section II covers the previous works
distributed computation environment along the and technologies. The work and solution we are
process of detecting anomalies and preparing the data adapting for the smart repository in section III. Section
for analysis. IV presents a basic implementation and evaluation of
the work. Finally, Section V concludes and states
Privacy and security are always an issue in any some future works.
collaborative and network-based solution. In today's
world the approach to this issue is a mix between II. TECHNOLOGIES AND STATE OF THE ART
political or governmental policies and technical
implementations. From the technical side we should Traditional middlewares aren't fit to play the role
take these issues into consideration from early stages, of CityPro’s connectors. Additional criteria and
starting from the design of the system. functionalities are required. Therefore, reviewing
traditional middlewares in the edge computing era
Applying our concept to CityPro, provides a opens a door for more options and facilities.
middleware between data providers such as (banks,
customs, hospitals…), the existing information A. Edge and Fog Computing
systems, and the “Core” system of the city. Although
we started by challenges raised by CityPro, we Ref [2] refers to edge computing as the enabling
designed and implemented the middleware to be technology that performs computation at the edge of
generic, opensource, and customizable so it can be the network, on downstream data on behalf of cloud
applicable in many scenarios. services and upstream data on behalf of IoT services.
Another technical trend is Fog Computing a term
B. Problem position and proposed approach created by Cisco. While many interchange it with edge
computing, Cisco states that “Fog computing is a
CityPro intends to combine and inter-operate (in a standard that defines how edge computing should
supervised mode), heterogeneous pre-existing work” [3].
operational systems; e.g., banks, hospitals,
cellular/landline phone management engines, police- B. Middleware in the Context of Cities &
stations, video surveillance networks, etc. The Surveillance
proposed architecture should provide insights to
protect the city and some events should trigger real- Before edge computing and smarty city era,
time alerts. Yet we are faced with major challenges. “Embedded Middleware on Distributed Smart
The data providers are heterogeneous distributed Cameras” [4] designed and implemented a middleware
systems, where each provider has implemented its for distributed embedded image processing on a
own technological stack (Hardware-OS-Database-App network of smart cameras. They embedded the
65
middleware in the camera device. In our case CityPro, opensource software” which is causing inter-
instead of the camera or device there is a complete operability issues and limits the collaboration among
information system – the data provider. Unlike the researchers. They applied the micro-service paradigm
embedded middleware used in [4] where they to provide a modular and scalable middleware.
interchange data between cameras, in CityPro data InterSCity microservices architecture is shown in
partners don’t interact with each other. Furthermore, in Figure 4. The abstraction is through “city resource” a
CityPro an embedded middleware can’t handle big- logical concept that resembles a physical entity such
data, scalability, availability, and other processing as cars, traffic lights, etc.
tasks. That is why we evolved the concept of
embedding to an edge device. Each resource has attributes and functions to
provide data and receive commands. For
communication protocols, microservices use
1) Civitas synchronous HTTP Rest API and asynchronous
Ref [5] highlighted that traditional middleware message bus using RabbitMQ. Their work adopts
technologies are designed for enterprise environments many open source projects such as PostgreSQL and
and can’t address issues of heterogeneity and Redis. In CityPro context the distributed existing
scalability in a Smart City. They connected different information systems are more data producers that
entities such as citizens, governmental institutions, and service providers. Furthermore, to some extent,
companies to the “Civitas” platform which is middlewares and micro-services solve different
considered as the core of the IT infrastructure in the problems. Micro-services architecture takes the whole
Smart City. The connection is through a device called application (Smart City) and de-couples it into
“Civitas Plug”, these devices can be smartphones, independent services.
company servers, residential gateways…
C. Accessing Distributed Heterogeneous
Another key point in the Civitas platform is the Database Systems
“Core Nodes”. They are servers that host different
kind of services to the city entities. They consider that Today's computing and analysis techniques are
heterogeneity and inter-operability is solved by the tempting to integrate different Information Systems to
distributed object-oriented middleware, where each provide additional insights. On the other hand, many
entity is an object with a set of defined standard international enterprises are providing services that
interfaces. It also supports event-based communication span different subjects in different locations. In
through publish-subscribe pattern. CityPro project which tries to maximize the benefit of
the distributed operating systems, there is a clear case
for this challenge.
Ref [7] highlights the need to “combine and
analyze the distributed data along with contextual
factors”. The authors list the current solutions and
technologies in the cloud computing infrastructure
stack (Azure, Amazon, private stacks) in three main
domains: managing distributed clusters, distributed
data processing models (such as MapReduce), and the
data management service across datacenters which
“integrated different cloud data storage services by
providing a transparent interface” such as Simple
Cloud API , PDC@KTH’s proxy service, Open Grid
Fig. 2. Civitas Platform [5] Services Architecture Data Access and Integration
OGSA-DAI).
We share the idea of the plug device, but this
device can’t alone solve the availability and scalability
issue, that’s why we added a broker layer between our 1) MUSYOP
“edge device” and the “core system”. Moreover, we Ref [8] provided “a federated approach - a
needed a deeper study of the inner-design and inner- mediator server - that allows users to query access to
workings of the plug device. multiple heterogeneous data sources” for relational
databases, Triplestore, NoSQL databases, and XML
with a management layer using SPARQL and
2) InterSCity mapping different databases to RDF.
Ref [6] noted that there is no agreed middleware
platform for SmartCities' platforms. They listed three
factors for this challenge: security and privacy 2) Apache Spark
policies, the lack of scientific and practical validation, One more interesting solution for accessing
and the “extensive use of development of non- different datasets is Apache Spark. Apache Spark is a
66
“unified analytics engine for large-scale data Custom drivers for different database engines will use
processing” [9]. The unified part is baked into the these interfaces to support those functionalities.
Spark SQL module. Spark SQL Layer is built on top Currently drivers for most database engines (MySQL,
of two interfaces Data Frame API and Data Source MongoDB, HBase, Cassandra, HDFS ...) are already
API which supports schema understanding, reading implemented and ready for production use.
data with filters, and writing custom aggregations.
There are two important sides of this topic:
distribution and heterogeneity. For the heterogeneity E. Complex Event Processing (CEP)
part there are two trends: use ontology-based solutions With various systems and sensors generating and
or build custom interface layer. It’s also important to sending data, there is a need to detect interesting
note that whatever the integration solution. SQL is the patterns along the data streams. CEP paradigm has an
preferred language to query these distributed datasets. opposing concept to regular databases. Instead of
The SQL layer is used for the unified access with the executing a query on a dataset, the data is executed on
added benefit that it can easily integrate with upper a well-defined query. This technology has been
layer tools and technologies such as BI tools. Often deployed and heavily used in the financial sector
suggested solutions tackle the whole process from especially for fraud detection.
accessing the data to integrating it, but my focus is on Ref [15] introduced the concept back in 1998. It
providing the interface to different database systems was a hot topic again in the research community in
and the integration part will be solved in later phases 2006-2009 where it was discussed in the context of
of CityPro. big-data and adding machine learning for prediction of
events. Many commercial and open-source CEP
D. Message Broker systems are available such as Apache Flink [16],
A Message Oriented Middleware (MOM) is Siddhi.io [17], and Esper [18].
responsible for sending and receiving data
encapsulated in messages between different III. PROPOSED SOLUTION; AN EDGE CENTRIC
distributed systems. A MOM can be with a broker or MIDDLEWARE FOR CITYPRO
broker-less. TIBCO Inc. [10] defines a message broker
as a discrete service that provides data marshaling, The "Edge Centric Middleware" consists of two
routing, persistence, and delivery to all appropriate parts: edge devices distributed along with the data
consumers. providers' systems and a broker between edge devices
and the "City Core System". We delegate some
We will highlight different technologies and computation tasks to the distributed edge devices
researches with respect to important features we are which provide a uniform and standard interface to the
interested in persistent cache, high availability and variety of existing systems. The edge devices follow
fault tolerance, scalability, with added value features the concept of black box to tackle privacy and security
such as message formats optimizations for binary concerns. While the broker handles the scalability and
messages and compression. First we consider current availability of message queues from a distributed
production-grade technologies. The state of art in network of edge devices.
persistence is to use a journaling file system write-
ahead commit log backed by operating system page A. General Architecture
cache this is implemented in Apache Kafka [11] and
Apache ActiveMQ Artemis [12] or delegate this to a There are two main parts for the Edge Centric
database that uses this implementation. For high Middleware: the edge device and the broker as shown
availability and fault tolerance, a replication set of 3 in Fig. 3.
brokers is recommended for production. Scalability is The Edge Device - It’s a software and hardware
done horizontally by adding more nodes as brokers, package deployed at the edge network of the data
but a consensus and management service is needed to provider and connected to the existing system via
keep track of nodes and data index in the cluster. One computer network communications. It can scale up
of the well-known and heavily used software that from a simple computer board such as Raspberry-Pi
implements this functionality is Apache Zookeeper [19] to a rack of servers. The main roles of the edge
[13] which itself can be replicated. Ref [14] introduced device are:
“EQS: an Elastic and Scalable Message Queue for the
Cloud”. They discuss automatic scaling and load x Provide an interface to access different
balancing in message queues to optimize the databases at rest at the provider side according
throughput along systems by layering additional to a schema contract required by the
components for monitoring, rules, and scaling government agency
management. x Consume live data from the provider side
according to a schema contract
67
x Provide the required computation resources to x Detect and propagate real-time alerts specified
host and execute data summary and ETL-like by a defined list of triggers
operations
x Enabler for the confidentiality and integrity of
x Send batches of data according to the the data and business rules in question
configured time interval and schema contract
Fig. 3. Proposed Solution General Architecture
The Broker - It is between the network of realizing the above criteria, where traditional message
distributed edge devices and the core system. Only queues support producer and consumer, some brokers
our certified edge devices can connect to the broker, also support a logic- processing endpoint such as
this enhances the privacy and quality of data flowing Apache Kafka's [11] Processing API which can be
through the middleware. From the broker point of used for data integration at this stage before
view edge devices are data producers and the “Core consuming the data.
System” is a data consumer. The broker roles are:
B. Edge Device Software Framework
x A message queue that supports high volume
and speed of data The edge device framework is a software &
hardware package. In this section, we will describe the
x Support publish-subscribe pattern software stack of the edge device (Fig. 4). We
x Support high availability in case of systems decomposed the framework into subcomponents with
failures and high traffic decoupled functionalities. We used a file-based
configuration for some settings and standard
x Support horizontal scalability to handle communication protocols for inter-component
existing and new systems communications and for the outer interfaces whether
with the data provider or the core system.
While we invested more on the edge device part to
propose a new framework, we opted to rely on
existing technologies for the broker side. In addition to
Fig. 4. Edge Device Inner Components
68
Schema Contract: States the required (selected) fields Admin Module: Receives commands and direct
and fields’ types from the data provider. queries from the core system and replies with the result.
Live Data Module: Consumes live data from the
provider and validates the raw data according to the C. Data Flow in the Edge Centric Middleware
schema contract. Edge Centric Middleware supports three data flow
modes:
Complex Event Processing Module: Works on a
stream of data and filter events that match the required x A defined event pattern can trigger sending data
query. The query uses standard language SQL. Any near real-time to the core system
matching result should be sent to the broker immediately.
x The core system requests detailed data on a
Detailed Data Module: Provides an interface to query specific subject. The request is fulfilled by the
heterogeneous databases and files. It also generates edge device which sends back a reply message.
dynamic reports of the results using the reports templates.
This module provides a unified standard query interface x Data is collected from the provider, prepared, and
language SQL. then sent in batches to the core system
Data Storage: Stores temp data between batch
intervals. It’s optimized for high-performance sequential 1) Live Dataflow
operations. This is considered as the regular periodic scenario (Fig.
5) that is always tracking familiar patterns from within the
Batch Module: Queries the cache according to the data to register any possible anomalies.
defined time interval and sends them to the broker.
Fig. 5. Live Data Flow
1. Data is streamed from the data provider live over a forwarded at real-time to the broker via network
network connection to the live data module which connection. CEP publishes to a specific broker topic
accepts data at a specific open TCP port. to avoid real-time alerts delays.
2. Live Data Module uses the schema contract to 5. The batch module runs at custom time intervals,
validate and apply light computation on the incoming collects cached data, and sends them in a batch to the
data whether it’s as simple as attribute selection or broker via a network connection
ETL-like operations.
3. Live Data Module forwards processed data to the 2) On-Demand Data Flow
Complex Event Processing (CEP) module and to the This scenario (Fig. 6) is initiated whenever the
cache storage in parallel at the same time via internal platform requires immediate and detailed data, especially
memory. in the cases of alerts. In this case the platform directly
4. In the CEP module the stream of data is executed on connects to the provider's edge without the need of an
the event pattern query. Upon any match, the event is intermediate broker.
Fig. 6. On-Demand Data Flow
69
1. CityPro Core initiates this process by sending a as a data provider system. At the provider side we used
request, which is a detailed query about a specific Microsoft SQL Server [21] as the database solution. To
subject, to the admin module which awaits establish live data from the telecom data provider system
connections and commands. to the edge framework we implemented a .NET service
2. The admin module initiates a new instance of the that wraps an MS SQL Server feature called “query
detailed data module with the proper report notifications”. “Query Notifications” are best defined and
parameters such as the exact datastore query and the documented as “query notifications that allow applications
report template. to be notified when data changes. This feature is
particularly useful for applications that provide a cache of
3. The detailed data module has the capability of
information from a database.” [22] Using query
querying different types of databases whether SQL or
notifications our .NET service streams any new data
NoSQL. After querying the provider’s database at inserted in SQL Server to our edge framework. This stream
rest and getting the result, the detailed data module is serialized using Apache Avro [23].
will build the report and forward it to the admin
module via inter-process communication. Edge Framework on RaspberryPi - The software stack
4. The admin module will send back the report to the for the edge is a cross-platform so we don’t have a
CityPro Core. problem in selecting the hosting operating system. For the
hardware, the edge software can scale from a small
computer board to a rack of enterprise servers according to
3) Provider Live Data Source
the load at each data provider. This makes it more efficient
Two modes operate while getting live data from the
in any budget planning. For testing, we deployed it on a
provider. To achieve this, we studied two paradigms at the
RaspberryPi [19] board (Fig. 7) with the following
abstract level.
specifications:
First Paradigm (Two-Tier Systems): We consider the
database as the source of “live” data. So, we must detect x Model 3 B+
and forward any data changes at the database level and x 1 GB RAM
forward the changes to a live data stream.
x 32GB Storage
Second Paradigm (Three-Tier Systems): We can
stream data live from the business logic layer or we can x Quad-core 64-bit processor clocked at 1.4GHz.
use the database layer in a similar way to the first case.
x 300Mbps Ethernet
IV. IMPLEMENTATION AND EVALUATION
To test the “Edge Centric Middleware” the inner
workings and how the data flows we considered the case of
Telecom Call Detail Records (CDR). We generated live
phone calls and used Microsoft SQL Server as a database
solution at the provider side. Our edge part of the
middleware was deployed on a RaspberryPi [19] board and
connected to the simulated database. The edge device
detected alerting patterns and sent them to the broker, in
addition to sending batches of data. We configured a Kafka
broker with two topics and validated the data flow.
Furthermore, we simulated a city core system panel to test Fig. 7. RaspberryPi 3 B+
the admin channel.
Deploying and running the edge software framework
A. Telecom Test Case on limited resources such as the RaspberryPi board proved
Preparing a DataSet - Due to privacy concerns, there the performance and the work that is done to optimize the
are no real data sets for telecom CDR. So, we generated computing footprint.
random CDR records. The CDR schema includes ID, Kafka Broker - The broker of the “Edge Centric
CALLING_NUM, CALLED_NUM, START_TIME, Middleware,” was tested with Apache Kafka [11]. For the
END_TIME, CALL_TYPE, CHARGE, and telecom (CDR) data case, we created two topics: one for
CALL_RESULT normal data batches, and one for alerts. The batch
We wrote a NodeJS [20] script to randomize the values component in the edge framework published messages to
while keeping the numbers in Lebanese format. This script the normal topic while the CEP module published
keeps running emulating current phone calls that are taking messages to the alert topic. This will guarantee fast
place right now. delivery for alerts and then the infrastructure supporting
each topic can be scaled and optimized accordingly.
Database Engine and Notification Service - After
generating call detail records (CDR) we consider Telecom Some of the implementations are shown in Fig. 8.
70
Fig. 8. Components Implementation: a) Report Template - b) Schema Contract - c) CEP Module
The software implementation is based on standard and
B. Assessment and evaluation opensource technologies. Developing within an
Although we didn't implement much security features, opensource community helps in boosting the pace of
we consider our solution as a security and privacy enabler. solving issues and adding features.
For example, we deployed our framework on a separated
and dedicated hardware where we can add physical ACKNOWLEDGMENT
tampering detection. In addition, only our certified edge
devices can send messages to the broker. Furthermore, This work is supported and funded by the Department
encryption on the network layer and on the device cache of research at the Lebanese University, Lebanon.
storage can be added.
REFERENCES
V. CONCLUSION, OBSERVATION AND FUTURE WORK [1] M. Dbouk, M. Mcheick and I. Sbeity, "CityPro: city-surveillance
collaborative platform," Int. J. Big Data Intelligence, vol. 4, no. 3,
Smart cities tackle development obstacles and improve 2017.
the quality of life for citizens. CityPro system focuses on [2] W. Shi, J. Cao and Q. Zhang, "Edge Computing: Vision and
city protection by supporting the collaboration of existing Challenges," IEEE Internet of Things Journal, vol. 3, no. 5, 2016.
operational systems. The need for a robust and standard [3] Cisco, "Edge computing vs. fog computing: Definitions and
middleware is critical for any smart city platform. enterprise uses," [Online]. Available:
https://www.cisco.com/c/en/us/solutions/enterprise-networks/edge-
However, due to challenges such as continuous big computing.html.
data streams, heterogeneous systems, security, and privacy, [4] B. Rinner, M. Jovanovic and M. Quaritsch, "Embedded
in-addition to non-opensource software solutions there is a Middleware on Distributed Smart Cameras," in 2007 IEEE
lack of such a middleware. This article proposes a new International Conference on Acoustics, Speech and Signal
architecture for a smart city middleware using emerging Processing - ICASSP '07, 2007.
edge computing trends and provides an open-source [5] F. J. Villanueva, M. J. Santofimia, D. Villa, J. Barba and J. C.
López, "Civitas: The Smart City Middleware, from Sensors to Big
implementation for the proposed framework. Data," in 2013 Seventh International Conference on Innovative
The "Edge Centric Middleware" consists of two parts: Mobile and Internet Services in Ubiquitous Computing, 2013.
edge devices distributed along with the data providers' [6] A. d. M. D. Esposte, F. Kon, F. M. Costa and N. Lago, "InterSCity:
A Scalable Microservice-based Open Source Platform for," in 6th
systems and a broker between edge devices and the "City International Conference on Smart Cities and Green ICT, 2017.
Core System". We delegate some computation tasks to the [7] L. Wang and R. Ranjan, "Processing Distributed Internet of Things
distributed edge devices which provide a uniform and Data in Clouds," IEEE Cloud Computing, vol. 2, no. 1, 2015.
standard interface to the variety of existing systems. The [8] Z. Liu, F. Cretton, A. L. Calvé, N. Glassey, A. Cotting and F.
edge devices follow the concept of black box to tackle Chapuis, "MUSYOP: Towards a Query Optimization for
privacy and security concerns. While the broker handles Heterogeneous Distributed Database," in International conference
the scalability and availability of message queues from a on Computing Technology and Information Management, Dubai,
2014.
distributed network of edge devices.
[9] Apache Spark, [Online]. Available: https://spark.apache.org/.
More metrics and experimental validations are needed. [10] TIBCO Inc., "What is a Message Broker?," [Online]. Available:
We had very limited time to develop a PoC (Proof of https://www.tibco.com/reference-center/what-is-a-message-broker.
Concept) for this research. More experimentation should [11] "Apache Kafka," [Online]. Available: https://kafka.apache.org/.
stress test big data flows.
71
[12] "Apache ActiveMQ Artemis," [Online]. Available: [17] "Siddhi," [Online]. Available: https://siddhi.io/.
https://activemq.apache.org/components/artemis/. [18] "Esper," [Online]. Available: http://www.espertech.com/esper/.
[13] A. Zookeeper. [Online]. Available: https://zookeeper.apache.org/. [19] "RaspberryPi," [Online]. Available: https://www.raspberrypi.org.
[14] N.-L. Tran, S. Skhiri and E. Zim´nyi, "EQS: An Elastic and [20] "NodeJS," [Online]. Available: https://nodejs.org/en/.
Scalable Message Queue for the Cloud," in 2011 IEEE Third
[21] "Microsoft SQL Server," [Online]. Available:
International Conference on Cloud Computing Technology and
https://www.microsoft.com/en-us/sql-server/.
Science, 2011.
[22] Microsoft, "Query Notifications in SQL Server," [Online].
[15] D. Luckham and B. Frasca, "Complex Event Processing in
Distributed Systems," Stanford University, 1998. Available: https://docs.microsoft.com/en-
us/dotnet/framework/data/adonet/sql/query-notifications-in-sql-
[16] "Apache Flink," [Online]. Available: server.
https://flink.apache.org/news/2016/04/06/cep-monitoring.html.
[23] "Apache Avro," [Online]. Available: https://avro.apache.org/.
72