An Edge Centric Middleware for City Surveillance Platform (CityPro) Mariam Hakim Ibrahim Tawbe Mohamed Dbouk Faculty of Sciences - I Faculty of Sciences - I Faculty of Sciences - I Lebanese University Lebanese University Lebanese University Beirut, Lebanon Beirut, Lebanon Beirut, Lebanon mariamhakim181@gmail.com ibrahim.tawbe@gmail.com mdbouk@ul.edu.lb Abstract—Smarter Cities provide better triggers these risks is traditionally monitored by management for city services, reporting problems, and different governmental agencies; the weather predicting future issues to enhance city protection. The monitoring agency differs from local police forces. On need for a robust and unified middleware platform in the other hand, these risks put citizens in danger and Smart Cities is not yet a solved problem. Traditional coordinating the emergency procedures is critical to middleware that evolved in enterprise environments lower the losses. Furthermore, detecting and dealing can’t handle smart cities complexities. Therefore, a with an emergency is not enough, there is a need to platform -CityPro- that integrates multiple existing predict and act before things happen. That’s why systems into a collaborative environment to protect the models are built and run to simulate real-world city is proposed. It highlights the concept of scenarios using machine learning and artificial “connectors” between different city systems and a central “core” system. In this article we provide a new intelligence. In addition to the models, early alarms middleware (architecture and framework) - called Edge sometime come from the correlation between different Centric Middleware- that adapts CityPro's architecture data sources. The data produced by these sources may to connect existing city systems to the core system. The become gigantic over time; such data can be suggested solution tries to solve issues such as merging considered as a valuable mine of information. heterogeneous systems, event processing, real-time data, Therefore, a technique must be adapted to access this scalability, availability, and interoperability, while data wisely in order to benefit the whole city. exploring edge computing capabilities. We also provided an implementation for this middleware to serve as an These necessities nourished the idea of an open-source generic framework that can be extended integrated platform of a collaborative surveillance and customized. system, called CityPro [1].This system is intended to protect and monitor people and public infrastructures. Keywords—Smart Cities; Business Processing; Edge It is expected to: Computing; Business Intelligence; Big Data; Smart Data x Operate within live-mode by using the city digital infrastructures I. INTRODUCTION AND PROBLEM POSITION x Combine and inter-operate heterogeneous pre- In a typical modern city, multiple computerized existing operational systems standalone systems exist, e.g. banks, hotels, and hospitals. All of these systems work separately which limits their powers to a specific business domain (e.g. A. CityPro; an overview bank), specific area, etc. The amount of challenges a Ref [1] presents CityPro; a collaborative platform city faces is increasing daily. Smart City paradigms for city protection tries to standardize the relation approach these challenges by using the city’s between different systems with a centralized, resources efficiently to optimize services and supervised control and data repository architecture managements such as traffic control, electricity, public shown in Fig. 1. It defines data providers as domain- safety, etc. It is worth to note that tackling these issues specific independent (stand-alone) systems that using technological advancements is not for “modern” coexist within the considered territory such as police cities only. Cities in developed countries can and departments, fire stations, banks, etc. In the context of should take measures following Smart City paradigm CityPro these systems are data producers. They are to solve challenges. Smart City is not a one-shot accessed through “collaboration-links” or solution; developed countries can gradually build up “connectors”. CityPro defines a connector as their Smart City eco-system. “Dedicated links that are materializing the collaborative inter-relationships between CityPro Public safety is not considered a luxurious service; components. They mainly consist of ETL like it’s an essential challenge for any city. Cities are dedicated data exchange automated protocols based on facing variety of risks such as natural disasters, ‘adapter pattern’.” CityPro delivered the general terrorists’ attacks, crimes, vehicle accidents, etc. What architecture of the system, while we still need to Copyright © 2019 for this paper by its authors. Use permitted under Creative Commons License Attribution 4.0 International (CC BY 4.0). 64 investigate deeper into providing a standard and layer). In addition most data providers hold big data uniform access to these heterogeneous distributed volumes. In its architecture the core system which systems with minimum effort at the data provider side. holds the federated data repository is responsible for analysis and decision making. This article is dedicated to finding a feasible solution that uses edge computing concepts to optimize data transfer and delegate some core system operations and computations to near-source components especially in the case of real-time alerts, while maintaining the security when moving and accessing remote data from other parties. These operations should scale accordingly and should be done reliably where the system should guarantee high availability and fault tolerance. Fig. 1. CityPro's General Architecture In the later sections of this paper, we are going to To address the need for city surveillance, CityPro introduce an architecture that adopts edge computing defines two data flows: periodically where data concepts and incorporate techniques to access continually arrives from data providers and on- distributed heterogeneous information systems. By demand where the central system asks providers for creating a middle layer between the Core System and instant detailed data. We need to define the standard the data providers such as banks, police, airports, protocols for this data exchange. customs, etc. The proposed architecture is expected to enhance the interoperability within the system itself Highly distributed systems of this kind may and the city systems. It is also expected to maintain a produce gigantic amounts of data. A pure central high availability and create a scalable cache for system might struggle in handling all this data flow messages from various providers. and delivering value, especially in the case of near real-time alerts. That’s why it’s tempting to use the In this paper, Section II covers the previous works distributed computation environment along the and technologies. The work and solution we are process of detecting anomalies and preparing the data adapting for the smart repository in section III. Section for analysis. IV presents a basic implementation and evaluation of the work. Finally, Section V concludes and states Privacy and security are always an issue in any some future works. collaborative and network-based solution. In today's world the approach to this issue is a mix between II. TECHNOLOGIES AND STATE OF THE ART political or governmental policies and technical implementations. From the technical side we should Traditional middlewares aren't fit to play the role take these issues into consideration from early stages, of CityPro’s connectors. Additional criteria and starting from the design of the system. functionalities are required. Therefore, reviewing traditional middlewares in the edge computing era Applying our concept to CityPro, provides a opens a door for more options and facilities. middleware between data providers such as (banks, customs, hospitals…), the existing information A. Edge and Fog Computing systems, and the “Core” system of the city. Although we started by challenges raised by CityPro, we Ref [2] refers to edge computing as the enabling designed and implemented the middleware to be technology that performs computation at the edge of generic, opensource, and customizable so it can be the network, on downstream data on behalf of cloud applicable in many scenarios. services and upstream data on behalf of IoT services. Another technical trend is Fog Computing a term B. Problem position and proposed approach created by Cisco. While many interchange it with edge computing, Cisco states that “Fog computing is a CityPro intends to combine and inter-operate (in a standard that defines how edge computing should supervised mode), heterogeneous pre-existing work” [3]. operational systems; e.g., banks, hospitals, cellular/landline phone management engines, police- B. Middleware in the Context of Cities & stations, video surveillance networks, etc. The Surveillance proposed architecture should provide insights to protect the city and some events should trigger real- Before edge computing and smarty city era, time alerts. Yet we are faced with major challenges. “Embedded Middleware on Distributed Smart The data providers are heterogeneous distributed Cameras” [4] designed and implemented a middleware systems, where each provider has implemented its for distributed embedded image processing on a own technological stack (Hardware-OS-Database-App network of smart cameras. They embedded the 65 middleware in the camera device. In our case CityPro, opensource software” which is causing inter- instead of the camera or device there is a complete operability issues and limits the collaboration among information system – the data provider. Unlike the researchers. They applied the micro-service paradigm embedded middleware used in [4] where they to provide a modular and scalable middleware. interchange data between cameras, in CityPro data InterSCity microservices architecture is shown in partners don’t interact with each other. Furthermore, in Figure 4. The abstraction is through “city resource” a CityPro an embedded middleware can’t handle big- logical concept that resembles a physical entity such data, scalability, availability, and other processing as cars, traffic lights, etc. tasks. That is why we evolved the concept of embedding to an edge device. Each resource has attributes and functions to provide data and receive commands. For communication protocols, microservices use 1) Civitas synchronous HTTP Rest API and asynchronous Ref [5] highlighted that traditional middleware message bus using RabbitMQ. Their work adopts technologies are designed for enterprise environments many open source projects such as PostgreSQL and and can’t address issues of heterogeneity and Redis. In CityPro context the distributed existing scalability in a Smart City. They connected different information systems are more data producers that entities such as citizens, governmental institutions, and service providers. Furthermore, to some extent, companies to the “Civitas” platform which is middlewares and micro-services solve different considered as the core of the IT infrastructure in the problems. Micro-services architecture takes the whole Smart City. The connection is through a device called application (Smart City) and de-couples it into “Civitas Plug”, these devices can be smartphones, independent services. company servers, residential gateways… C. Accessing Distributed Heterogeneous Another key point in the Civitas platform is the Database Systems “Core Nodes”. They are servers that host different kind of services to the city entities. They consider that Today's computing and analysis techniques are heterogeneity and inter-operability is solved by the tempting to integrate different Information Systems to distributed object-oriented middleware, where each provide additional insights. On the other hand, many entity is an object with a set of defined standard international enterprises are providing services that interfaces. It also supports event-based communication span different subjects in different locations. In through publish-subscribe pattern. CityPro project which tries to maximize the benefit of the distributed operating systems, there is a clear case for this challenge. Ref [7] highlights the need to “combine and analyze the distributed data along with contextual factors”. The authors list the current solutions and technologies in the cloud computing infrastructure stack (Azure, Amazon, private stacks) in three main domains: managing distributed clusters, distributed data processing models (such as MapReduce), and the data management service across datacenters which “integrated different cloud data storage services by providing a transparent interface” such as Simple Cloud API , PDC@KTH’s proxy service, Open Grid Fig. 2. Civitas Platform ‎[5] Services Architecture Data Access and Integration OGSA-DAI). We share the idea of the plug device, but this device can’t alone solve the availability and scalability issue, that’s why we added a broker layer between our 1) MUSYOP “edge device” and the “core system”. Moreover, we Ref [8] provided “a federated approach - a needed a deeper study of the inner-design and inner- mediator server - that allows users to query access to workings of the plug device. multiple heterogeneous data sources” for relational databases, Triplestore, NoSQL databases, and XML with a management layer using SPARQL and 2) InterSCity mapping different databases to RDF. Ref [6] noted that there is no agreed middleware platform for SmartCities' platforms. They listed three factors for this challenge: security and privacy 2) Apache Spark policies, the lack of scientific and practical validation, One more interesting solution for accessing and the “extensive use of development of non- different datasets is Apache Spark. Apache Spark is a 66 “unified analytics engine for large-scale data Custom drivers for different database engines will use processing” [9]. The unified part is baked into the these interfaces to support those functionalities. Spark SQL module. Spark SQL Layer is built on top Currently drivers for most database engines (MySQL, of two interfaces Data Frame API and Data Source MongoDB, HBase, Cassandra, HDFS ...) are already API which supports schema understanding, reading implemented and ready for production use. data with filters, and writing custom aggregations. There are two important sides of this topic: distribution and heterogeneity. For the heterogeneity E. Complex Event Processing (CEP) part there are two trends: use ontology-based solutions With various systems and sensors generating and or build custom interface layer. It’s also important to sending data, there is a need to detect interesting note that whatever the integration solution. SQL is the patterns along the data streams. CEP paradigm has an preferred language to query these distributed datasets. opposing concept to regular databases. Instead of The SQL layer is used for the unified access with the executing a query on a dataset, the data is executed on added benefit that it can easily integrate with upper a well-defined query. This technology has been layer tools and technologies such as BI tools. Often deployed and heavily used in the financial sector suggested solutions tackle the whole process from especially for fraud detection. accessing the data to integrating it, but my focus is on Ref [15] introduced the concept back in 1998. It providing the interface to different database systems was a hot topic again in the research community in and the integration part will be solved in later phases 2006-2009 where it was discussed in the context of of CityPro. big-data and adding machine learning for prediction of events. Many commercial and open-source CEP D. Message Broker systems are available such as Apache Flink [16], A Message Oriented Middleware (MOM) is Siddhi.io [17], and Esper [18]. responsible for sending and receiving data encapsulated in messages between different III. PROPOSED SOLUTION; AN EDGE CENTRIC distributed systems. A MOM can be with a broker or MIDDLEWARE FOR CITYPRO broker-less. TIBCO Inc. [10] defines a message broker as a discrete service that provides data marshaling, The "Edge Centric Middleware" consists of two routing, persistence, and delivery to all appropriate parts: edge devices distributed along with the data consumers. providers' systems and a broker between edge devices and the "City Core System". We delegate some We will highlight different technologies and computation tasks to the distributed edge devices researches with respect to important features we are which provide a uniform and standard interface to the interested in persistent cache, high availability and variety of existing systems. The edge devices follow fault tolerance, scalability, with added value features the concept of black box to tackle privacy and security such as message formats optimizations for binary concerns. While the broker handles the scalability and messages and compression. First we consider current availability of message queues from a distributed production-grade technologies. The state of art in network of edge devices. persistence is to use a journaling file system write- ahead commit log backed by operating system page A. General Architecture cache this is implemented in Apache Kafka [11] and Apache ActiveMQ Artemis [12] or delegate this to a There are two main parts for the Edge Centric database that uses this implementation. For high Middleware: the edge device and the broker as shown availability and fault tolerance, a replication set of 3 in Fig. 3. brokers is recommended for production. Scalability is The Edge Device - It’s a software and hardware done horizontally by adding more nodes as brokers, package deployed at the edge network of the data but a consensus and management service is needed to provider and connected to the existing system via keep track of nodes and data index in the cluster. One computer network communications. It can scale up of the well-known and heavily used software that from a simple computer board such as Raspberry-Pi implements this functionality is Apache Zookeeper [19] to a rack of servers. The main roles of the edge [13] which itself can be replicated. Ref [14] introduced device are: “EQS: an Elastic and Scalable Message Queue for the Cloud”. They discuss automatic scaling and load x Provide an interface to access different balancing in message queues to optimize the databases at rest at the provider side according throughput along systems by layering additional to a schema contract required by the components for monitoring, rules, and scaling government agency management. x Consume live data from the provider side according to a schema contract 67 x Provide the required computation resources to x Detect and propagate real-time alerts specified host and execute data summary and ETL-like by a defined list of triggers operations x Enabler for the confidentiality and integrity of x Send batches of data according to the the data and business rules in question configured time interval and schema contract Fig. 3. Proposed Solution General Architecture The Broker - It is between the network of realizing the above criteria, where traditional message distributed edge devices and the core system. Only queues support producer and consumer, some brokers our certified edge devices can connect to the broker, also support a logic- processing endpoint such as this enhances the privacy and quality of data flowing Apache Kafka's [11] Processing API which can be through the middleware. From the broker point of used for data integration at this stage before view edge devices are data producers and the “Core consuming the data. System” is a data consumer. The broker roles are: B. Edge Device Software Framework x A message queue that supports high volume and speed of data The edge device framework is a software & hardware package. In this section, we will describe the x Support publish-subscribe pattern software stack of the edge device (Fig. 4). We x Support high availability in case of systems decomposed the framework into subcomponents with failures and high traffic decoupled functionalities. We used a file-based configuration for some settings and standard x Support horizontal scalability to handle communication protocols for inter-component existing and new systems communications and for the outer interfaces whether with the data provider or the core system. While we invested more on the edge device part to propose a new framework, we opted to rely on existing technologies for the broker side. In addition to Fig. 4. Edge Device Inner Components 68 Schema Contract: States the required (selected) fields Admin Module: Receives commands and direct and fields’ types from the data provider. queries from the core system and replies with the result. Live Data Module: Consumes live data from the provider and validates the raw data according to the C. Data Flow in the Edge Centric Middleware schema contract. Edge Centric Middleware supports three data flow modes: Complex Event Processing Module: Works on a stream of data and filter events that match the required x A defined event pattern can trigger sending data query. The query uses standard language SQL. Any near real-time to the core system matching result should be sent to the broker immediately. x The core system requests detailed data on a Detailed Data Module: Provides an interface to query specific subject. The request is fulfilled by the heterogeneous databases and files. It also generates edge device which sends back a reply message. dynamic reports of the results using the reports templates. This module provides a unified standard query interface x Data is collected from the provider, prepared, and language SQL. then sent in batches to the core system Data Storage: Stores temp data between batch intervals. It’s optimized for high-performance sequential 1) Live Dataflow operations. This is considered as the regular periodic scenario (Fig. 5) that is always tracking familiar patterns from within the Batch Module: Queries the cache according to the data to register any possible anomalies. defined time interval and sends them to the broker. Fig. 5. Live Data Flow 1. Data is streamed from the data provider live over a forwarded at real-time to the broker via network network connection to the live data module which connection. CEP publishes to a specific broker topic accepts data at a specific open TCP port. to avoid real-time alerts delays. 2. Live Data Module uses the schema contract to 5. The batch module runs at custom time intervals, validate and apply light computation on the incoming collects cached data, and sends them in a batch to the data whether it’s as simple as attribute selection or broker via a network connection ETL-like operations. 3. Live Data Module forwards processed data to the 2) On-Demand Data Flow Complex Event Processing (CEP) module and to the This scenario (Fig. 6) is initiated whenever the cache storage in parallel at the same time via internal platform requires immediate and detailed data, especially memory. in the cases of alerts. In this case the platform directly 4. In the CEP module the stream of data is executed on connects to the provider's edge without the need of an the event pattern query. Upon any match, the event is intermediate broker. Fig. 6. On-Demand Data Flow 69 1. CityPro Core initiates this process by sending a as a data provider system. At the provider side we used request, which is a detailed query about a specific Microsoft SQL Server [21] as the database solution. To subject, to the admin module which awaits establish live data from the telecom data provider system connections and commands. to the edge framework we implemented a .NET service 2. The admin module initiates a new instance of the that wraps an MS SQL Server feature called “query detailed data module with the proper report notifications”. “Query Notifications” are best defined and parameters such as the exact datastore query and the documented as “query notifications that allow applications report template. to be notified when data changes. This feature is particularly useful for applications that provide a cache of 3. The detailed data module has the capability of information from a database.” [22] Using query querying different types of databases whether SQL or notifications our .NET service streams any new data NoSQL. After querying the provider’s database at inserted in SQL Server to our edge framework. This stream rest and getting the result, the detailed data module is serialized using Apache Avro [23]. will build the report and forward it to the admin module via inter-process communication. Edge Framework on RaspberryPi - The software stack 4. The admin module will send back the report to the for the edge is a cross-platform so we don’t have a CityPro Core. problem in selecting the hosting operating system. For the hardware, the edge software can scale from a small computer board to a rack of enterprise servers according to 3) Provider Live Data Source the load at each data provider. This makes it more efficient Two modes operate while getting live data from the in any budget planning. For testing, we deployed it on a provider. To achieve this, we studied two paradigms at the RaspberryPi [19] board (Fig. 7) with the following abstract level. specifications: First Paradigm (Two-Tier Systems): We consider the database as the source of “live” data. So, we must detect x Model 3 B+ and forward any data changes at the database level and x 1 GB RAM forward the changes to a live data stream. x 32GB Storage Second Paradigm (Three-Tier Systems): We can stream data live from the business logic layer or we can x Quad-core 64-bit processor clocked at 1.4GHz. use the database layer in a similar way to the first case. x 300Mbps Ethernet IV. IMPLEMENTATION AND EVALUATION To test the “Edge Centric Middleware” the inner workings and how the data flows we considered the case of Telecom Call Detail Records (CDR). We generated live phone calls and used Microsoft SQL Server as a database solution at the provider side. Our edge part of the middleware was deployed on a RaspberryPi [19] board and connected to the simulated database. The edge device detected alerting patterns and sent them to the broker, in addition to sending batches of data. We configured a Kafka broker with two topics and validated the data flow. Furthermore, we simulated a city core system panel to test Fig. 7. RaspberryPi 3 B+ the admin channel. Deploying and running the edge software framework A. Telecom Test Case on limited resources such as the RaspberryPi board proved Preparing a DataSet - Due to privacy concerns, there the performance and the work that is done to optimize the are no real data sets for telecom CDR. So, we generated computing footprint. random CDR records. The CDR schema includes ID, Kafka Broker - The broker of the “Edge Centric CALLING_NUM, CALLED_NUM, START_TIME, Middleware,” was tested with Apache Kafka [11]. For the END_TIME, CALL_TYPE, CHARGE, and telecom (CDR) data case, we created two topics: one for CALL_RESULT normal data batches, and one for alerts. The batch We wrote a NodeJS [20] script to randomize the values component in the edge framework published messages to while keeping the numbers in Lebanese format. This script the normal topic while the CEP module published keeps running emulating current phone calls that are taking messages to the alert topic. This will guarantee fast place right now. delivery for alerts and then the infrastructure supporting each topic can be scaled and optimized accordingly. Database Engine and Notification Service - After generating call detail records (CDR) we consider Telecom Some of the implementations are shown in Fig. 8. 70 Fig. 8. Components Implementation: a) Report Template - b) Schema Contract - c) CEP Module The software implementation is based on standard and B. Assessment and evaluation opensource technologies. Developing within an Although we didn't implement much security features, opensource community helps in boosting the pace of we consider our solution as a security and privacy enabler. solving issues and adding features. For example, we deployed our framework on a separated and dedicated hardware where we can add physical ACKNOWLEDGMENT tampering detection. In addition, only our certified edge devices can send messages to the broker. Furthermore, This work is supported and funded by the Department encryption on the network layer and on the device cache of research at the Lebanese University, Lebanon. storage can be added. REFERENCES V. CONCLUSION, OBSERVATION AND FUTURE WORK [1] M. Dbouk, M. Mcheick and I. Sbeity, "CityPro: city-surveillance collaborative platform," Int. J. Big Data Intelligence, vol. 4, no. 3, Smart cities tackle development obstacles and improve 2017. the quality of life for citizens. CityPro system focuses on [2] W. Shi, J. Cao and Q. Zhang, "Edge Computing: Vision and city protection by supporting the collaboration of existing Challenges," IEEE Internet of Things Journal, vol. 3, no. 5, 2016. operational systems. The need for a robust and standard [3] Cisco, "Edge computing vs. fog computing: Definitions and middleware is critical for any smart city platform. enterprise uses," [Online]. Available: https://www.cisco.com/c/en/us/solutions/enterprise-networks/edge- However, due to challenges such as continuous big computing.html. data streams, heterogeneous systems, security, and privacy, [4] B. Rinner, M. Jovanovic and M. Quaritsch, "Embedded in-addition to non-opensource software solutions there is a Middleware on Distributed Smart Cameras," in 2007 IEEE lack of such a middleware. This article proposes a new International Conference on Acoustics, Speech and Signal architecture for a smart city middleware using emerging Processing - ICASSP '07, 2007. edge computing trends and provides an open-source [5] F. J. Villanueva, M. J. Santofimia, D. Villa, J. Barba and J. C. López, "Civitas: The Smart City Middleware, from Sensors to Big implementation for the proposed framework. Data," in 2013 Seventh International Conference on Innovative The "Edge Centric Middleware" consists of two parts: Mobile and Internet Services in Ubiquitous Computing, 2013. edge devices distributed along with the data providers' [6] A. d. M. D. Esposte, F. Kon, F. M. Costa and N. Lago, "InterSCity: A Scalable Microservice-based Open Source Platform for," in 6th systems and a broker between edge devices and the "City International Conference on Smart Cities and Green ICT, 2017. Core System". We delegate some computation tasks to the [7] L. Wang and R. Ranjan, "Processing Distributed Internet of Things distributed edge devices which provide a uniform and Data in Clouds," IEEE Cloud Computing, vol. 2, no. 1, 2015. standard interface to the variety of existing systems. The [8] Z. Liu, F. Cretton, A. L. Calvé, N. Glassey, A. Cotting and F. edge devices follow the concept of black box to tackle Chapuis, "MUSYOP: Towards a Query Optimization for privacy and security concerns. While the broker handles Heterogeneous Distributed Database," in International conference the scalability and availability of message queues from a on Computing Technology and Information Management, Dubai, 2014. distributed network of edge devices. [9] Apache Spark, [Online]. Available: https://spark.apache.org/. More metrics and experimental validations are needed. [10] TIBCO Inc., "What is a Message Broker?," [Online]. Available: We had very limited time to develop a PoC (Proof of https://www.tibco.com/reference-center/what-is-a-message-broker. Concept) for this research. More experimentation should [11] "Apache Kafka," [Online]. Available: https://kafka.apache.org/. stress test big data flows. 71 [12] "Apache ActiveMQ Artemis," [Online]. Available: [17] "Siddhi," [Online]. Available: https://siddhi.io/. https://activemq.apache.org/components/artemis/. [18] "Esper," [Online]. Available: http://www.espertech.com/esper/. [13] A. Zookeeper. [Online]. Available: https://zookeeper.apache.org/. [19] "RaspberryPi," [Online]. Available: https://www.raspberrypi.org. [14] N.-L. Tran, S. Skhiri and E. Zim´nyi, "EQS: An Elastic and [20] "NodeJS," [Online]. Available: https://nodejs.org/en/. Scalable Message Queue for the Cloud," in 2011 IEEE Third [21] "Microsoft SQL Server," [Online]. Available: International Conference on Cloud Computing Technology and https://www.microsoft.com/en-us/sql-server/. Science, 2011. [22] Microsoft, "Query Notifications in SQL Server," [Online]. [15] D. Luckham and B. Frasca, "Complex Event Processing in Distributed Systems," Stanford University, 1998. Available: https://docs.microsoft.com/en- us/dotnet/framework/data/adonet/sql/query-notifications-in-sql- [16] "Apache Flink," [Online]. Available: server. https://flink.apache.org/news/2016/04/06/cep-monitoring.html. [23] "Apache Avro," [Online]. Available: https://avro.apache.org/. 72