=Paper=
{{Paper
|id=Vol-2330/paper3
|storemode=property
|title=Towards sustainable ecosystems for cloud functions
|pdfUrl=https://ceur-ws.org/Vol-2330/paper3.pdf
|volume=Vol-2330
|authors=Yessica Bogado-Sarubbi,Walter Benitez-Davalos,Josef Spillner,Fabio Lopez-Pires
|dblpUrl=https://dblp.org/rec/conf/ucc/Bogado-SarubbiB18
}}
==Towards sustainable ecosystems for cloud functions==
https://ceur-ws.org/Vol-2330/paper3.pdf
Towards sustainable ecosystems for cloud functions
Yessica Bogado-Sarubbi, Walter Benitez-Davalos, and Fabio Lopez-Pires
Information and Communication Technology Center
Itaipu Technological Park
Hernandarias, Paraguay
Email: {yessica.bogado,walter.benitez,fabio.lopez}@pti.org.py
Josef Spillner
Service Prototyping Lab
Zurich University of Applied Sciences
Winterthur, Switzerland
Email: josef.spillner@zhaw.ch
Abstract According to the early visions, rich ecosystems
would emerge based on centralised exchanges such as
The main technologies around modern cloud registries implementing the Universal Description, Dis-
development and deployment paradigms such as covery and Integration (UDDI) specification including
Function-as-a-Service (FaaS) environments follow a the Universal Business Registry (UBR) (Saini 2016).
typical technology life-cycle. Starting with basic code Yet, on a large scale, the ideas mostly failed. In-
installation and execution environments, they unfold stead, several technology-specific exchanges, often de-
into a complete ecosystem with rich collaborative centralised, have emerged in recent years. After several
development and market enablement tools. In this iterations across web technologies, heavy-weight and
paper, we analyse the growth of such ecosystems, light-weight cloud technologies and more recently fog
reveal causes of hindrances in previous service-oriented computing approaches, a few of them have manifested
approaches, and present a vision of how an ecosystem as useful practical tooling. A brief overview on rele-
with sustainable operation could look like both in gen- vant technology-specific exchanges is presented next in
eral and specifically for cloud functions. We present Table 1.
Function Hub, a partial prototypical implementation
to gain first knowledge about the potential operational Technology Exchange
ecosystem behaviour. Source code GitHub
Java libraries Maven
Keywords: Cloud Applications, Serverless Com- Containers Docker Hub
puting, Service Ecosystems, Tangible Microservices, Web services Programmable Web
Marketplaces. Cloud services Open Service Broker API
Table 1: Overview on technology-specific exchanges.
1 Problem statement
Hence, the initial ideas increasingly become viable
In the late 1990s and early 2000s, the idea of albeit often in limited form, focusing on pragmatic pro-
marketplaces to exchange software development arte- ductivity gains especially for rapid prototyping of com-
facts arose in the research communities for Component- posite applications and services.
Based Software Engineering (CBSE) and later Service- In the context of previously described ecosystems,
Oriented Software Engineering (SOSE). emerging computing paradigms such as Function as
18
a Service (FaaS) (Baldini et al. 2017) taking into ac-(Yu 2011). Among his key observations is the slow-
count serverless computing architectures and imple- down effect which occurs when moving up the stack.
mentations with cloud functions, relevant challenges The dependent products grow slower with logarithmic
still remain open as the ones presented as follows. relation compared to the independent product. From
a consumer perspective, Petsas et al. reveal in the
• Who operates such exchanges (e.g. brokers or context of mobile application ecosystems that down-
marketplaces) in a sustainable way? load statistics do not follow a Zipf distribution (Petsas
• If the operation of these exchanges is not cen- et al. 2013). In these systems, there are moreover few
tralised and not linked to concrete business mod- alternative system software choices, leading to a com-
els, how can it be sustained? parably huge offering of applications.
• Which future exchanges will emerge for new tech- 2.3 Obstacles and hindrances
nologies such as cloud functions in seemingly
serverless computing environments? A key concern is the stakeholder structure behind the
operating entities of ecosystem elements. Multiple
• How these future exchanges may look like?
popular exchanges are operated by a single commercial
player. In case of bankruptcy or change of business
Starting by answering the last question, this paper
model, the exchange may vanish or fade into obscu-
presents our position by outlining an ecosystem vi-
rity quickly. The ownership structure also distorts the
sion for exchanging and collaboratively developing soft-
analysis of the economic viability. Docker Hub, for in-
ware with cloud functions. Additionally, a comparison
stance, is operated by Docker, Inc. but does not itself
of the conceptual ecosystem against existing ones is
generate direct revenue. Instead, it is cross-financed by
presented, and main reasons about the sustainability
other company operations whose growth is in turn sup-
aspects are included to give answers to the remaining
ported by the dominance of the exchange on the mar-
three questions.
ket. Another issue is the concentration of providers in
ecosystems. For example, images on Docker Hub are
2 Ecosystem analysis prone to several security issues. This situation also af-
fects the officially maintained images from which the
For conceptualisation and contextualisation, an issues propagate into derivatives. Any security vul-
analysis of the actual software ecosystem environment nerability that could be exploited has the potential
is needed. In this section a definition is presented as to affect vast shares of development and deployment
well as an analysis of ecosystems current growth and workflows in critical applications as a consequence (Shu
obstacles. et al. 2017).
In summary, main identified obstacles and hindrances
2.1 Ecosystem Definition could be single commercial owners and concentration
of providers in ecosystems.
In this paper an ecosystem is defined to be an open
system consisting of a set of technical elements which
enable growth by attracting contributions from par- 3 Value proposition and position
ticipants. Among the elements are service-oriented
elements (marketplaces, hubs, brokers), as well as This paper propose to strive towards sustainable
platforms and further provider and consumer tools. ecosystems for heterogeneous application development
artefacts which can be customised for arbitrary do-
2.2 Growth of ecosystems mains including cloud and mobile applications and
so forth. The sustainability will be achieved by de-
Ecosystems can be measured primarily by looking centralisation and abstraction. Arguing that by
at provider and consumer metrics which cover both a suitable combination of decentralised ecosystem el-
the absolute volume and the growth factors. Yu ements with built-in abstraction capabilities, a long-
observes the co-evolution of ecosystems on a wider scale term growth and sustainable operation can be achieved
from a provider perspective involving hardware, sys- even in volatile environments with changing technolo-
tem software and software-implemented applications gies and market forces.
19
4 Conceptual elements perspective
Users This section covers detailed design and architecture of
the core elements within the proposed ecosystem.
Decentralization 4.1 Marketplaces
Functions Hub Interface Decentralized
repositories The idea of a marketplace is to create an environ-
ment where developers could interact with the plat-
form ecosystem in a way that allows them to create,
share and trade tools, enabling users to deploy, scale
Abstraction
and create functions more easily and efficiently.
Abstraction tools
In that sense, industry already has some marketplaces
from other software ecosystems. For example, GitHub
has the GitHub Marketplace, a marketplace where
Figure 1: General view of the proposed ecosystem. third-party companies create and commercialise in-
tegration tools that allow users to work more effi-
ciently with their source code. Additionally, Docker
The considered arguments are as follows. has DockerStore, a place where third-party companies
offer plugins and certified containers to docker users.
This allows them to access enterprise solutions and
• Decentralisation: In recent years, several appli- create industry-ready applications using the Docker
cations on the Internet have become centralised ecosystem. Amazon Web Services (AWS) Marketplace
conglomerate services. Yet over longer periods of from Amazon, allows both users and third-party com-
time, they may not prevail (DeLegge & Wangler panies to trade tools that uses the AWS ecosystem. At
2017). A decentralised ecosystem guarantees that the time of this writing, only Amazon has a first pro-
in a worst case even when individual market par- totype of a serverless ecosystem with some still basic
ticipants vanish, the system will continue to func- features1 .
tion in reduced form. From the above mentioned example marketplaces,
some relevant characteristic that the marketplace
should have can be specified:
• Abstraction: The right level of abstraction is
important. While existing ecosystems have only • Enable third-party companies and/or users to cre-
become successful for technologically specialised ate tools that interact with the ecosystem.
artefacts, there is a large spectrum between these
and fully generic approaches which can be ex- • Enable third-party companies and/or users to
ploited. This exploitation can be partially auto- commercialise their add-on tools.
mated by converting formats and protocols.
• A framework that allows users to collaborate with
each other within the ecosystem.
To include these characteristics in a serverless
ecosystem, the following approach has been establish • Integration with other software platforms (Docker,
as shown in Figure 1. For decentralisation, a set of de- GitHub, AWS, among others).
centralised repositories that allows users to download,
test and use cloud functions directly. These reposito- One characteristic that stands out of these software
ries will be connected through a marketplace, where platforms is marketplace centralisation. This could
developers will be able to share, deploy and even com- lead to potential software disruptions in case of change
mercialise cloud functions. For abstraction, tools to of policies or departure of mayor stakeholders.
convert and deploy functions will be used, allowing de-
1
ployment on a diverse set of cloud providers. https://aws.amazon.com/serverless
20
4.2 Converters Microsoft like C#, F# , JavaScript or PHP. Never-
theless, the field of the deployment of functions does
Even deployment-ready applications have to deal with
not belong only to private platforms. In this context,
syntax associated with each cloud platform. For in-
one of the most popular open source cloud platform
stance, every serverless cloud platform has its own Ap-
is Apache OpenWhisk, that execute functions in re-
plication Program Interface (API), methods and syn-
sponse to events, supporting programming languages
tax to deploy functions, generating a vendor lock-in
like JavaScript/Node.js, Swift, Python, Java and even
issue, where users have to create functions for specific
implementing Docker logic.
cloud providers instead of a general one.
A feasible solution is to automatically add wrappers
through a converter, allowing this way developers to 4.5 Interaction and serverless ecosystem
create code according to its real requirements, without In serverless architectures, cloud providers have com-
worrying with the code necessary to make it run on a plete management over the environment in which func-
specific cloud environment. With the advent of mo- tions run. These creates high-level abstract environ-
bile edge computing, this could be a very interesting ments for the users, where they should not worry about
approach to federate cloud providers. deployment or maintenance and expects it to be fault-
tolerant with auto-scaling features (Baldini et al. 2017).
4.3 Deployers This architecture is basically a event-driven computa-
tion pattern that promotes loosely couple services and
To accomplish a total integration with the industrial
ensures a trigger function execution (Stigler & Stigler
cloud ecosystem, a flexible tool that allows users to de-
2018).
ploy their functions on multiple cloud environments is
The interaction between users, interfaces and reposi-
needed. Most of the tools used in industrial cloud en-
tories is given in a way that presents a continuous ex-
vironments are associated to its own platforms. Ama-
changing of functions and allows the growth of a mar-
zon Cloud Formation, Google Cloud Deployment Man-
ketplace. The interface creates the link between all
ager and Azure Resource Manager are examples of lo-
repositories and users. Because each repository is per-
cal frameworks that manage the set of resources for its
sonal, every user could create a new repository and has
own specific platform.
the option to share theirs or store functions from other
For cloud multi-tenancy, for example, allowing migra-
users. As a way of allowing abstraction, every function
tion from one cloud provider to another is a key fea-
is created as simple as possible, to after that, establish
ture. An example of a tool that currently applies this
a conversion to match a cloud provider specific require-
concept is the serverless framework2 , an open source
ment.
Command Line Interface (CLI) for building serverless
architectures and event-driven applications, using spe-
4.6 Related ecosystems
cific software resources for deployment of functions in
a wide set of cloud platforms. At the time of this writing, a new serverless ecosystem
for sharing functions in AWS Lambda is the AWS
4.4 Execution environments Serverless Application Repository. This repository al-
lows users of AWS Lambda to create, share and use
To easily create and deploy functions, an execution en-
functions on that specific environment. The main
vironment has to be set in place. Each cloud provider
drawback of this new ecosystem is the vendor lock-in
focuses its environment in accordance to an aimed de-
issue, because all components are associated with this
veloper group or their specific infrastructure. For ex-
specific provider, resulting in a centralised non-abstract
ample, AWS Lambda supports major programming
ecosystem. Additionally, it only allows registration for
languages such as Java, JavaScript, Python, C# and
users with an associated bank account (credit or debit
lately Go. Each of these languages with their most
card), difficulting students to access this repository and
popular runtimes like Node.js, Java 8, Python 2 and
associated functions.
3, but they add extra features allowing users to inter-
In contrast, the new approach considered for the pro-
act with the infrastructure. On the other hand, Azure
posed ecosystem, let users create their own repository
Functions supports major development languages for
to share, test and deploy functions. Remembering that
2
https://github.com/serverless/serverless functions are created as simple as possible, they could
21
be converted and deployed in different cloud environ- can search a function by name or specifying other
ments, making it easy to create functions associated attributes, having the testing option on the same
with a specific product without having any dependen- platform and then download functions according
cies over a certain provider. to its requirements. For the interconnection be-
tween HTTP and XMPP protocols, a message bro-
ker named MW 2 was developed.
5 Proof of concept and implemen-
tation • Decentralisation: In this work, decentralisation
is achieved through repositories to store functions
For a Proof of Concept (PoC), several ecosystem ele- and serving as an active server for users. A soft-
ments were implemented to gain insights and knowl- ware named Snafu (Swiss Army Knife of Serve-
edge about their actual behaviour and effectiveness. less Computing), a FaaS host process, was used
For an implementation, it was focused on an event- for this task. Snafu was chosen because it ful-
driven application as a subset of the entire cloud ap- fils two main characteristics, as an execution en-
plication space, due to their alignment with stateless vironment for functions (active mode) and as a
cloud function processing. Furthermore, the use of function repository (passive mode). As Function
decentralised open messaging infrastructure was ex- Hub does not pretend to use an unique centralised
plored. Figure 2 shows the proposed ecosystem to storage, it communicate with other repositories
achieve a Function Hub that could help to cre- through XMPP. For this reason, each provider has
ate a decentralised environments for the serverless a XMPP Client account associated to its message
ecosystem. broker MW 1 and to its associated XMPP Server
for sending data required by users. The use of
5.1 Function marketplaces message brokers allows interconnection between
HTTP and XMPP protocols, because both Func-
The initial prototype its based on the core idea that tion Hub and Snafu use HTTP as a main protocol
defines what and how elements are trade within it. In connector. The implementation of the message
this case, an ecosystem that allows free exchange of brokers (MW 1 and MW 2 ) was made through
functions between users and generates the required en- flask and nbxmpp libraries in Python.
vironment for a serverless market to proliferate.
For this purpose a Function Hub3 was designed con- • Abstraction: The abstraction sector is composed
sidering the following characteristics: by a basic converter that add all the basic wrap-
pers needed to deploy the function, according to
• Decentralisation, by using an Extensible Mes- most of the main cloud providers and uses the
saging and Presence Protocol (XMPP) for com- serverless framework (Collins 2015) as a deploy-
munications, without centralised storage. ment tool.
• Abstraction, by using Snafu 4 which allows man-
aging cloud functions across provider convention 5.2 Function converters
boundaries.
As an early prototype6 , a converter of Python functions
Additionally, the complete Function Hub ecosystem was developed to add wrappers for different modules
is composed by four main components: that the file could have. It works according to these
steps:
• Users: Users that access the Function Hub In-
• It consumes and checks if the function has some
terface5 to share and get functions according to
run-time code to avoid security issues when im-
its needs.
porting.
• Function Hub Interface: It is a website based
• After checking, depending of the choice of the con-
on AngularJS which runs on a NginX server. Users
verter user, it dismantles different modules of the
3
https://github.com/serviceprototypinglab/functionshub file and adds wrappers with the function for each
4
https://github.com/serviceprototypinglab/snafu
5 6
https://github.com/YessicaBogado/FunHub https://github.com/walter-bd/faas-converter
22
User 1 Encabezado User N
Repository 1
HTTP
Web Browser Web Browser MW 1 Fun_A
HTTP R
HTTP R Fun_B
User
Repository 2
Web User Interface
HTTP
R MW 2 XMPP Server 1 HTTP
MW 1 Fun_C
Web Server Fun_D
Function Hub Interface XMPP Server K
Repository M
Converter
HTTP
MW 1 Fun_E
Cloud Provider
Fun_F
Deploy
Abstraction Decentralization
Abstraction tools Decentralized repositories
Figure 2: Proposed ecosystem implementation flowchart.
one on different files or it add the wrapper of one facilitate the deployment on the most popular cloud
of this module at the end of a copy of the file. providers. Furthermore, a composeless7 service is being
developed to deploy functions on serverless ecosystems
• It has option to add wrappers for providers like or in a container, according to particular needs.
AW S, OpenW hisk, F ission, OV H and Azure.
• The user can choose wherever to add the wrappers 5.4 Function execution environments
on different files for each provider or to add it all
The execution environment of the ecosystem is pro-
together in one file but, it has to manually check vided by the FaaS host process Snaf u, and it’s sup-
for some collisions with other wrappers added. port different programming languages like C, Java,
This tool will allow users of the environment to easily JavaScript and Python. Snafu is deployed on an
create standard function to deploy in different cloud Alpine-based docker image, where it has the following
providers. run-time environments Python 3.5, Python 2.7, open-
jdk8, nodejs and for compiling purpose gcc and g + +
for C programs.
5.3 Function deployers
For deploying functions into the ecosystem, a new func- 5.5 Type of users
tionality was needed to be added. This will give users
the option to upload their functions from their reposi- In the proposed ecosystem, the interaction is based in
tories to the Function Hub ecosystem; hence, for this four kind of users:
purpose a Snafu server is used.
• Regular users: They use the ecosystem to look,
To deploy functions from the Function Hub to a
download, test or upload functions to allowed
private cloud provider is intended to use the server-
7
less framework, because this have already options to https://github.com/serviceprototypinglab/composeless
23
repositories. To make this possible, a web interface
is available, showing the user all the repositories Rabbah, R., Slominski, A. et al. (2017), Serverless
and its respective functions. computing: Current trends and open problems, in
‘Research Advances in Cloud Computing’, Springer,
• Exclusive repository user: For companies that pp. 1–20.
want to create exclusive functions to their prod- URL: https://doi.org/10.1007/978-981-10-5026-
ucts associated with their own infrastructure. 81
Function Hub would provide a place where they
can share the use of this function, allowing them Collins, A. (2015), ‘Serverless framework’, GitHub.
to decide what to share and what to sell. URL: https://github.com/serverless/serverless
• Passive repository user: In this case, collabo- DeLegge, A. & Wangler, H. (2017), ‘Is this the end for
rative users or companies could create a repository facebook? A mathematical analysis’, Applied Math-
so that other users could use it to store its func- ematics and Computation 305, 364–380.
tions. URL: https://doi.org/10.1016/j.amc.2017.02.014
• Active repository user: It would work as a Pas- Petsas, T., Papadogiannakis, A., Polychronakis, M.,
sive Repository, but also will allow regular users Markatos, E. P. & Karagiannis, T. (2013), Rise
to test the functions or execute it there. of the planet of the apps: a systematic study of
the mobile app ecosystem, in ‘Proceedings of the
6 Conclusions and future directions 2013 Internet Measurement Conference, IMC 2013,
Barcelona, Spain, October 23-25, 2013’, pp. 277–290.
The rapid growth of serverless computing creates a URL: http://doi.acm.org/10.1145/2504730.2504749
need for an ecosystem in order to bring users necessary
tools for a fast and cheap deployment of their software. Saini, A. (2016), An extension to UDDI for the dis-
On that point, Amazon took the first steps, providing covery of user driven web services, in ‘Distributed
to their users a serverless repository with basic func- Computing and Internet Technology - 12th Inter-
tionalities. Also, new open source serverless initiatives, national Conference, ICDCIT 2016, Bhubaneswar,
like OpenWhisk and Fission, gives developers freedom India, January 15-18, 2016, Proceedings’, pp. 92–96.
to create new tools. URL: https://doi.org/10.1007/978-3-319-28034-
However, and as it was showed in this paper, it is also 9 11
needed properties like decentralisation and abstraction
Shu, R., Gu, X. & Enck, W. (2017), A study of
that allows them to create applications that interact
security vulnerabilities on docker hub, in ‘Proceed-
with a diverse cloud ecosystem and take advantage of
ings of the Seventh ACM on Conference on Data
this diversity according to their needs, without worry-
and Application Security and Privacy, CODASPY
ing when individual market participant vanish. Point-
2017, Scottsdale, AZ, USA, March 22-24, 2017’,
ing on that direction, this paper presented a vision
pp. 269–280.
for how ecosystems with a decentralised Function Hub
URL: http://doi.acm.org/10.1145/3029806.3029832
may give to developers a way to share their functions,
thinking not only on a specific cloud provider but also
Stigler, M. & Stigler, M. (2018), Beginning Serverless
considering application itself instead. Further discus-
Computing, Springer.
sion, research and develop is required with the objec-
URL: https://doi.org/10.1007/978-1-4842-3084-
tive of give such ecosystem essential properties like sus-
81
tainability, scalability and reliability associated with a
wider adoption. Yu, L. (2011), ‘Coevolution of information ecosystems:
a study of the statistical relations among the growth
References rates of hardware, system software, and application
software’, ACM SIGSOFT Software Engineering
Baldini, I., Castro, P., Chang, K., Cheng, P., Fink, Notes 36(6), 1–5.
S., Ishakian, V., Mitchell, N., Muthusamy, V., URL: http://doi.acm.org/10.1145/2047414.2047435
24