-Applications that make use of Internet of Things (IoT) capture an enormous amount of raw data from sensors and actuators, which is frequently transmitted towards the cloud data centres for processing and analysis. However, due to varying and unpredictable data generation rates and network latency, sending the data towards a cloud data centre can lead to a performance bottleneck. With the emergence of Fog and Edge computing hosted microservices, data processing could be moved towards the network edge. We propose a novel Pareto-based approach that makes use of a multi-criteria bin packing optimisation for efficient and optimal distributed deployment of microservices - along edge, fog/cloudlet and cloud tiers. This optimisation takes account of non-functional requirements, such as operational cost, compute resource utilisation, service availability, response time, latency and similar. The results show that the present approach provides an optimal and sustainable consumption of compute resources and improves Quality of Service of the application during its runtime. The approach can also be integrated into software engineering workbenches for the creation and deployment of cloud-native applications, enabling partitioning of an application across the multiple infrastructure tiers outlined above.
The number of Internet of Things (IoT) devices that change
their physical location has grown significantly in the last few
years. According to estimates by Forbes [
Real time applications, in areas such as disaster warning systems, real time video analysis and deep learning models related to object, speech and text recognition have time varying demands on compute resources (based on the generation and availability of data). Sending and processing such data in a Copyright © 2021 for this paper by its authors. Use permitted under Creative Commons License Attribution 4.0 International (CC BY 4.0). cloud data centre presents bottlenecks due to high latency, transmission costs, and privacy issues. Recently, with the emergence of Fog and Edge computing the trend has become to move the services, data and processing power towards the network edge for processing and analysis.
A novel approach is presented for budget- and performance-efficient deployment of a data pipeline based on microservices along Edge-Cloudlet/Fog-Cloud ecosystem. A traffic management use case is considered, which makes use of deep learning models for object recognition. The proposed Pareto-based multi-criteria bin packing optimisation approach considers a variety of non-functional requirements that can be divided into mandatory (hard constraints) and desirable (soft constraints), and returns optimal non-dominant solutions as a trade-off among conflicting objectives. These trade-offs include operational cost and network latency for efficient microservice deployment. Our key contributions are: (1) novel approach for supporting optimal and efficient distributed microservice placement across Edge-Fog/Cloudlet-Cloud ecosystem and (2) enhanced QoS of the overall cloud-native application based on data processing pipeline realised through microservices.
The rest of this paper is organised as follows: Section II discusses related work, followed by Section III where we introduce the motivating application scenario related to traffic management. Section IV proposes an Edge-Fog-Cloud architecture that is suitable for the distributed deployment of a data analysis pipeline. In Section V we formulate the problem as a multi-criteria bin packing optimisation approach. In Sections VI and VII we discuss experimental setup and provide results and evaluation of the approach. We reveal our future research directions and conclude the paper with Section IX.
II.
Recently, big data [
Moreover, processing video streams in the clouds and at
the edge of the network has also gained importance and
popularity recently, due to possibility for use of increasing
availability of elastic computing environments that support
real-time resource allocation and provisioning [
III.
A traffic management scenario is used to illustrate the
benefits of using both edge and cloud resources. A traffic
management system needs to acquire data from fixed field
sensors and autonomous vehicles in real time. A variety
of fixed sensors are placed along roads and highways as
road side units for detecting traffic conditions, such as traffic
flow and congestion monitoring, vehicle, density and incident
detection, overspeeding etc. Additionally, autonomous vehicles
communicate with one another and with road side units and
transmit their information on speed and location for forecasting
potential congestion and to estimate travel time [
In this scenario, data sent from IoT devices to cloudbased systems must be processed and analysed in near realtime. Sending the data to the clouds means high latency and consequently longer response time to end users. On the other hand, processing data at the network edge and fog nodes offers low latency and consequently faster response to traffic events and therefore better application QoS.
The traffic management system implies the following components: (i) Input sensors are installed at fixed distance from one another on roads and collect and send data to a fog device for analysis; (ii) Global Display Actuators receive up-to-date responses from controllers or router fog devices and present parameter values such as lane closure signs or average vehicle speed, vehicle route displays and similar; (iii) Controller Fog device is responsible for compute, storage and network resources; it receives input from sensors and sends updates to the actuator; (iv) Router Fog device is a router linked to the controller fog device and enables multiple communication channels; moreover, it is also capable of support computational analysis; (v) Cloud data center is responsible for maintaining the entire application state. The workflow pipeline showing interaction between these different components is provided in Figure 1.
Global Display
Actuator Sensor (PPL) Detect Density
Triggers Module Global Display Actuator
Updates Display
Updates Global State
Controller Fog Device
Client Module Controller Fog Device
Client Module
Router Fog Device Traffic Density
Module y nsit Detecte D ffic ule Tra Mod gers ity s g n
Tri De UpdatesDisplaydatesalState RoDMDTueoertaenvdfrsiuficiFlcteeyog
UpGlob
Cloud Data center Connector Module Fig. 1: Traffic Management scenario: Integrating Fog and Cloud.
IV. EDGE-FOG-CLOUD SYSTEM ARCHITECTURE Our proposed system architecture consists of three tiers: edge, fog or cloudlet nodes and cloud data centres, as illustrated in Figure 2. At the edge of the network, fixed and dynamically positioned IoT devices generate data that is transmitted over a public network. The generated raw data intransit, passes through fog or cloudlet nodes and continues to the cloud data centres. Fog nodes or cloudlets (to support mobility) are positioned between the network edge and the cloud.
Initial data pre-processing can be performed on the edge and fog nodes, however more complex tasks such as object detection and recognition (which can have greater demands on computing resources) should consequently be processed in cloud data centres. Therefore, we propose the distribution of the data pipeline functions, such as (i) data collection, cleaning and data pre-processing and, on the other hand, (ii) object detection and recognition through deep learning models to be executed across edge, fog and cloud infrastructure respectively as illustrated in Figure 2. We treat each functionality of the data analysis pipeline as a microservice that collectively represent the overall application.
data pre-processing
Shallow learning Deep learning video decoding object detection object recognition Edge-Fog-Cloud-based Big data pipeline Fig. 2: Data analysis pipeline.
Cameras Sensors data collection video capturing V.
The multi-criteria, bin packing optimisation method was designed to provide a Pareto-based trade-off analysis for an efficient deployment of microservices along Edge-FogCloud environment based on NFRs. As a result, the method returns a reduced number of options, Pareto non-dominant solutions which present the optimal infrastructure deployment options for running services. The implemented method fits into software engineering life-cycle of cloud-native applications, particularly as part of the service provisioning stage, where the deployment configuration is proposed, based on steps illustrated in Figure 3. In the following subsections all phases that are part of the method are introduced in details. iton d isam tehom it eop ikng iv c tc pa jboe ibn
During the creation of cloud-native applications, a
developer decides which non-functional requirements (NFR) belong
to hard and soft constraints. Hard constraints are mandatory
due to their essential influence on service operation, therefore
they must be satisfied during runtime. On the other hand, soft
constrains are not mandatory and tend to improve QoS. NFRs
can be understood as parameters with categorical, continuous
and ordinal values [
2) QoS-aware resource optimisation based on soft
constraints: As the hard constraints are satisfied, for the
QoSaware optimisation we have utilised multi-criteria
optimisation for fine tuning the optimal choice of suitable instances
based on soft constraints. However, soft constraints can be
mutually conflicting, whereby altering one attribute can have
a detrimental effect on the others. For example, increasing
service availability means providing system redundancy and
consequently leads to higher operational costs. On the contrary,
reducing operational cost means limited options for renting
computational resources which can cause execution overheads
[
Our approach uses the concepts of domination to consider
multiple conflicting NFRs. For a 2 A dominates over a0 2 A if
a0 is greater than a in relation to all objective functions, while
a0 has worse value for at least one solution. A solution a0 2 A
is non-dominated, if there are no other solutions a 2 A that
dominate over a0. A set of solutions P 0 2 P is called Pareto
optimal set if there are no other solutions in P that dominate
any solution in P 0. The set of all Pareto optimal solutions is
known as Pareto front. The Pareto front is an efficient tool for
supporting decision making. It narrows down the search space,
and provides insights for efficient exploitation of the space of
non-dominated solutions [
The deployment of workflow pipeline functionalities across Edge-Fog-Cloud environment is not a trivial procedure. We consider both coupled and interdependent (micro)services that exchange data. The deployment plan for the entire application could be generated in succession for one microservice after another, similarly, as the functionalities are executed in the pipeline. We propose 2 main phases as part of our method.
Phase 1: For the first workflow pipeline functionality (e.g. data-processing) as a microservice our approach proposes deployment plan. Based on it’s initial requirements on compute resources and developer’s desire on soft constraints (V-B) the approach returns a Pareto front, presenting optimal infrastructure options at fog nodes or cloud data centres. The developer makes a decision by selecting one Pareto point.
Phase 2: After proposing a plan for the deployment of the first workflow pipeline functionality, we consider the second and every next functionality (e.g. object detection and recognition) as a separated functionality yet dependent and linked to the first or previous functionality. For the generation of additional workflow pipeline functionality, our approach considers the selected Pareto point from phase 1 and generates A Pareto front. Each solution defines infrastructure pairs for the deployment of the second functionality whereas the approach considers latency from every fog node and cloud data centre in relation to the chosen node (selected Pareto point) in the phase 1. The advantage of this procedure is to narrow down the search space if we consider relationship and dependency among infrastructures in distributed fog nodes and clod data centres based on latency and geo-location. The two-phase process is depicted in Figure 4.
We have simulated the traffic management scenario by
running benchmark tests on 15 different Amazon’s Elastic
Compute Cloud (EC2) regions worldwide1 including edge
locations and data centres offered by EC2 [
The Client module collects the data from sensors (PPL), such as information on traffic and lane state and stores the data into database microservice. On each EC2 region, we have configured a test t2.micro instance and deployed the database microservice. The query requests were send from every test t2.micro instance to all test instances on various regions whereby we have measured round trip time (RTT) or latency of the query response. Additionally, to extend the experiments and consider a variety of other constraints, such as infrastructure-based metrics (CPU and memory utilisation) and operational cost of an instance we have configured several EC2 instances with different amount of allocated compute resources and on-demand operational cost. We have sent query requests from every test instance to all other test instances in all regions and measured the round trip time (RTT) of the query request and response. The collection of the monitoring data took more than a month whereby queries were sent every hour. For simplification of our experiments, we assumed that the RTT from specific instance at some region to the instances at the same region should be the same since it does not correlate 1The following data centres have been utilised: N. Virginia, Ohio, N. California, Oregon, Mumbai, Seoul, Singapore, Sydney, Tokyo, Montreal, Frankfurt, Ireland, London, Paris, Sa˜o Paulo. with other compute properties of the instances, such as the amount of CPU and memory utilisation.
We then created a simple data-preprocessing microservice with initial constraints on CPU and memory utilisation and user constraints on latency and operational cost. Our main goal was to find the optimal infrastructure – a Client or Connector Module for the deployment of data pre-processing microservice based on hard (CPU memory utilisation) and soft constraints (latency, budget).
VII.
For our particular problem of Pareto front construction we
have utilised the JMetal multi-objective optimization
framework [
For the sake of the evaluation, we have created a random deployment plan system by choosing random solutions and compared its performance against our approach. We have generated deployment plan 50 times whereas we still embedded basic logic into random deployment plan system in order to avoid making it too dummy by considering more amount of allocated compute resources as the service needs to avoid under provisioning. The summary of the evaluation is illustrated in Figure 6. The charts present Pareto-based and randomly chosen solutions (instances) according to the Fig. 6: Line charts depict pareto and randomly chosen solutions (VM instances). The horizontal line illustrates constraint. 0.05
CPU and memory constraint. For the two constraints (CPU, memory), we assumed that the service operates well with 1 CPU core and 4 GB of memory. Nevertheless, our newly developed approach beats the random deployment plan system by (1) allocating as minimum amount of computing resources needed under the consideration of hard constraints and by returning the optimal solutions according to two conflicting objectives. On the contrary, the system for random instance choice considers instances with sufficient amount of compute resources but misses optimal allocation of compute resources and chooses the instances with much more resources which means over provisioning with higher operational costs and unsustainable consumption of compute resources.
The presented approach can be used for the efficient distributed deployment of big data pipeline functionalities along edge, fog and cloud tiers with consideration of non-functional requirements. Supporting job placement and submission to edge resources enables extension to existing Science Gateways, which often make use of cloud-based resources for execution. Supporting close-to-the-user enactment of jobs and data placement enables approximate analysis and what-if investigations to be carried out at lower latency, compared to execution on a remote data centre.
A Science Gateway can also make use of the proposed approach to deploy services and support data migration in an automated manner. As a gateway can provide unified access to a number of different types of cyberinfrastructure and application libraries, the presented approach enables edge and fog devices to also be integrated in a seamless manner within such an environment. The optimisation approach outlined in this work enables dynamic: (i) deployment of services at a cloud data centre and at edge resources concurrently, based on requirements that have been identified by a user via a Webbased interface; (ii) auto-scaling of services across edge and cloud resources based on constraints that have been identified by a user.
The optimisation approach identified in this work can be
used to schedule and execute tasks to a particular timeline and
to specific resources. In the centrepiece is a Gateway where a
user submits tasks e.g. (data pipeline functionalities) and the
system (with the present approach integrated) automatically
identifies the number of instances and resources needed to
execute the tasks. The user would be able to manage tasks and
resources through a user-friendly interactive web-based portal
for automated job submission, such as [
With the emergence of Internet of Things (IoT) that periodically generate a massive amount of raw data, the need for processing and analysing of this data has become a challenge. Additionally, IoT applications that consist of workflow pipeline can benefit promptly from data processing near network edge due to low latency and preserved data privacy. On the other hand, large-scale cloud data centres offer high compute resources, scalability and availability. Due to the pipeline-based nature of IoT applications they are suitable for the distribution across edge, fog/cloudlet, cloud tiers.
We treated workflow pipeline functionalities as linked microservices that can be deployed one after another in the pipeline manner and jointly compose a fully working application. As an example, we provided the description of traffic management application scenario that illustrates, how workflow pipeline can be modelled and deployed to edge-fogcloud infrastructure.
As the core of our work, we presented a Pareto-based
Multi-criteria bin packing optimisation approach suitable for
assuring a sufficient amount of mandatory compute resources
and budget- and performance-efficient trade-off analysis of
conflicting Non-functional Requirements. The experiments
were run on Amazon’s Elastic Compute Cloud (EC2) platform,
particularly on 15 edge and data centre regions worldwide. The
result of our approach are Pareto front non-dominant solutions
that present an optimal infrastructure (e.g. instance) within
fog nodes or cloud data centres for optimal deployment of
the data-processing microservice with set initial constraints.
Pareto front provides insights for easier exploitation of the
search space and allows certain properties of particular interest
to be easily explored. The method was evaluated in terms of
operational budget and network latency benefits against the
algorithm for random infrastructure choice. The results show
that the present approach indeed returns optimal solutions
and consequently the efficient deployment strategy plan. We
estimate that the use of the present approach will encourage
sustainable consumption of on-demand compute resources and
provide enhanced Quality of Service of the overall application.
Additionally, the present approach can be integrated into
software engineering workbenches for the creation and deployment
of cloud-native applications [
Our future research directions will be focused on orchestration, such as how to map the proposed deployment plan as part of the overall application workflow to the OASIS Topology and Orchestration Specification for Cloud (TOSCA) and dynamically update TOSCA. We will explore how data learning models can be executed in parallel and consequently horizontally and vertically distributed across edge, fog and cloud infrastructure.
The project has received funding from the European Union’s Horizon 2020 Research and Innovation Programme under grant agreement No. 815141 (DECENTER: Decentralised technologies for orchestrated Cloud-to-Edge intelligence).