COOP - automatiC validatiOn of evOlving microservice comPositions Olga Groh, Harun Baraki, Alexander Jahl, and Kurt Geihs Distributed Systems Group University of Kassel Kassel, Germany {groh, baraki, jahl, geihs}@vs.uni-kassel.de which still have to be maintained. In case of a model change, the old versions need to be adapted as well Abstract to adhere to updates. With an increasing number of services, the complexity of version management rises Current change management solutions apply drastically [Bya13]. This is especially true for projects versionized interfaces to ensure coherent evo- with short development cycles. In other words, scala- lution in service environments. This leads to bility of change management is not fostered. several drawbacks such as an increasing num- ber of service versions and time-consuming We present in this paper our framework COOP that testing of backward compatibilities in case of provides management for semantic, syntactic and pro- drastic changes. In this paper, we present an tocol changes. Syntactic changes are changes in the approach with continuous integration that au- interface structure while semantic changes consider tomatically validates interfaces for compati- changes in the meaning and context, e. g., of a return bility. Developers are relieved from keeping value or a parameter. The latter requires an ontology track of multiple versions of different services. that is referenced by the interface. This guarantees that dependent parties co- By employing the declarative logic programming evolve accordingly to the interface changes. language ASP (Answer Set Programming) [GK14] in- Our solution is based on the automatic gener- stead of OWL (Web Ontology Language) [Bec09], not ation of declarative logic programs for validat- only changes with respect to the interface are possible, ing compatibilities and extracting changes. but also changes regarding the ontology definition. In contrast to existing change management frame- 1 Introduction works, such as [FS14] and [RP12], COOP is not bound to a specific protocol. Depending on the applied pro- After the first mentioning of the term Microservice in tocol and the used programming language, the frame- 2011 [LF14], the new style of architecture emerged work generates the client stubs accordingly. quickly. Instead of creating single layered systems, the development shifted towards a composition of Mi- Furthermore, COOP provides a notification mech- croservices, connected through well-defined interfaces. anism for informing service providers whether prior Microservices are developed independently from versions have any active clients. This prevents inter- each other and, thus, allow separate development cy- ruptions and failures during runtime and reduces the cles. However, this may cause conflicts due to ser- resource consumption since unused versions are iden- vice changes. A typical measure is to apply service tified and deleted safely. versioning. This requires to preserve all previous ver- The paper is organized as follows. COOP and its sions and leads to an increasing number of services, components are presented in Section 2. The underly- ing architecture comprises a central repository and the Copyright c 2019 for this paper by its authors. Use permitted three main modules Service Description, Client Inte- under Creative Commons License Attribution 4.0 International (CC BY 4.0). gration, and Change Detection. Section 3 describes an In: Anne Etien (eds.): Proceedings of the 12th Seminar on Ad- example workflow for our framework. Relevant related vanced Techniques Tools for Software Evolution, Bolzano, Italy, works are summarized in Section 4. Finally, the main July 8-10 2019, published at http://ceur-ws.org findings of this paper are concluded in Section 5. 1 2 Architecture of COOP 2.1 Service Description Module COOP is composed of a central repository and the The Service Description module is platform-specific as three modules Service Description, Client Integration, it is responsible for extracting essential information re- and Change Detection. While the Service Description garding the syntactic, semantic and protocol aspects module extracts interface details to capture the seman- from a given service implementation. Thereafter, it tic and syntactic facets of the service, the Client In- creates an intermediate, comprehensive and platform- tegration module utilizes the description through the independent service description. Therefore, the mod- central repository to generate a client stub accord- ule scans the service implementation and searches for ingly and to check for new versions regularly. In case interfaces with specific annotations. This means in of an interface change, the Client Integration module particular that the developer has to follow certain im- triggers the Change Detection module to identify in- plementation guidelines in order to flag the respective compatibilities. By applying COOP, developers gain interfaces and to enrich them with additional semantic a threefold benefit. Firstly, service can be enriched information. with semantic information by annotating the respec- As a clarifying example, the COOP Service Descrip- tive methods in the implementation. The annotation tion module for Java shall be considered here. By is resolved automatically by means of incooperated on- means of the build tools Maven or Gradle it gets inte- tologies. Secondly, ontologies and interfaces are repre- grated into the build lifecycle of a service. The module sented in ASP, which is partcularly siuted for dynamic searches for typical annotations for service interfaces environments. In contrast to OWL, ASP ontologies al- in Java such as @RestController (Spring Framework1 ), low adaptations of existing knowledge during runtime @Path (Java EE2 ), @WebService (Java EE2 ) and oth- by employing defaults and externals [GK14]. Thirdly, ers. The annotated interface is then analyzed further COOP checks automatically for semantic and syntac- for additional details. Assuming that a @RestCon- tic of different versions during development time. troller annotation is present, the Service Description While the Service Description module has to be in- module looks out for annotations such as @GetMap- tegrated in the service implementation, the Client In- ping and @PostMapping. The attributes of these an- tegration and Change Detection modules are included notations reveal how a service method is accessed. In in the client application. However, all modules are particular, this includes the used protocol, the path, available on both, the server and the client side, as and the data format. Further technical details relate to both may consume or provide further services. It is the syntax of parameters and return values. For both, thus applicable for Microservice compositions. The the protocol and the syntax information, the Service framework is depicted in Figure 1. The following sec- Description module applies existing tools to transform tions dive into each component and explain the inter- the technical details to a JSON representation. If the play between them. interface contains complex data types, the module will map them as well. In case of a REST service, the gen- Repository erated JSON section adheres to the OpenAPI spec- ification3 . By using a widespread specification, the Client Integration module, in turn, is enabled to gen- Version ASP JSON erate a client stub in different programming languages Framework Registration Programs Description by third-party tools. While the previous annotations were required by the respective framework to configure and provide the Service Change Client service, semantic annotations are specific to COOP. Description Detection Integration For each identified and extracted method, the Service Description module expects semantic information with respect to the parameters, the return value, and the method itself. These information are added by the Service Client developer by means of the @SemanticContext anno- tation. The semantic values used in the annotation have to be terms that are defined by the developer or another party in an ASP (Answer Set Program- Provider Developer ming) dictionary, which is also called an ASP program 1 https://spring.io/ Figure 1: Framework Overview 2 https://javaee.github.io/javaee-spec/javadocs/ 3 https://github.com/OAI/OpenAPI-Specification 2 [GK14, BET11]. In contrast to OWL (Web Ontology available version of the service description, which is Language) [Bec09], ASP programs are non-monotonic likewise translated into ASP. and, thus, can be extended and updated at runtime without discarding the current knowledge base. Fur- Listing 1: ASP example thermore, they support unique names, which is essen- tial for deriving ASP code from the interface descrip- 1 service ( holidayService ). 2 protocol ( holidayService , rest ) . tion with our Change Detection module in Section 2.3. 3 path ( h o l i d a y S e r v i c e , ” / a p i / h o l i d a y s ” ) . Besides that, closed-world reasoning and defaults are 4 method ( h o l i d a y S e r v i c e , g e t H o l i d a y s ) . supported. The latter allows to assume standard val- 5 context ( getHolidays , holiday ) . ues, if certain details are currently missing. For exam- 6 r e t u r n ( g e t H o l i d a y s , ” l i s t ” ) . 7 returnContext ( getHolidays , date ) . ple, we could express that temperature is measured in 8 p a r a m e t e r ( g e t H o l i d a y s , country , s t r i n g ) . Celsius by default, but that Fahrenheit and Kelvin are 9 c o n t e x t ( g e t H o l i d a y s , country , countryName ) . allowed as well. 10 p a r a m e t e r ( g e t H o l i d a y s , year , i n t ) . 11 c o n t e x t ( g e t H o l i d a y s , year , d a t e ) . The developer associates, preferably, each method, parameter and return type with a semantic annota- tion. However, defaults in the ASP program may sup- Since the JSON description contains syntactic, se- port the developer. A method annotated with @Se- mantic and protocol information, the generated ASP manticContext(”getWeather”) might assume the coun- program allows to identify changes in any of these di- try code as default parameter and a temperature as mensions. Syntactic information cover the data types response. of parameters and return values. Incorporating only The Service Description module adds the semantic the syntax, may lead to a successful build process, annotations and a reference to the ASP program to the but omits crucial information with respect to the con- JSON description of the service and submits it to the text. For example, the HolidayService in Listing 2 repository. The repository creates an entry in the ver- accepts the parameter country with the full country sion registry and provides then the JSON description name as value. After an update, the parameter ex- to other developers. pects a two character country code as defined in ISO 3166 ALPHA-2. The value ”Germany” would change to ”DE”. While the syntactic representation remains 2.2 Client Integration Module the same, i.e. a String, the semantic annotation would A developer on client side will employ the Client In- be changed accordingly by the developer to @Se- tegration module to obtain the JSON description of manticContext(”iso3166ALPHA2”). The context in the requested service. Subsequently, the module gen- Line 9 (Listing 1) would automatically be transformed erates a client stub in the needed language. The stub to context(getHolidays,country,iso3166ALPHA2). is integrated in the project and used for development. Regarding the protocol, the Client Integration mod- Developers neither need to worry about the structure ule generates the client stub. A change in the protocol of the service used nor do they need to deal with the would be catched automatically by generating a new protocol. Here again, developers should integrate the stub. An alternative protocol would neither affect the module in the build lifecycle of their project. After syntax nor the semantic and, thus, would relieve de- each built, the module triggers a check against the velopers from taking any further action. newest version of the service registered at the Ver- sion Registration of the repository. If a new version is available, the Change Detection module is executed 2.4 Repository to determine the deltas between the used and the most current version and whether any incompatibilities ex- The repository stores received ASP programs and ist. JSON descriptions. Besides, it provides the Ver- sion Registration. The Service Description module 2.3 Change Detection Module announces new versions to the Version Registration, which, in turn, creates an entry to keep track of the The Change Detection module operates on the client number of registered clients per service version. A side and is triggered whenever a new version of an developer may trigger through the Client Integration involved service is available. module a registration for the used version. Whenever The JSON description will be translated into an no more clients are associated to a previous version, ASP program. For example, the JSON structure in the Version Registration informs the respective ser- Listing 2 is translated to the ASP program in List- vice provider. This allows the provider to delete the ing 1. This program will be compared with the locally unused version safely. 3 3 Exemplary Workflow with the parameter state of type String. This change is reflected subsequently in the respective JSON descrip- A Microservice environment typically consists of sev- tion in the repository. Since the Client Integration eral projects. Therefore, our solution operates on the module of OrganizerService registered for version V1 complete composition of services. In the following, we of HolidayService, both versions of HolidayService are provide an example scenario that contains the two sep- kept. arate Microservices HolidayService and OrganizerSer- vice. HolidayService provides the dates and names of Listing 2: JSON example holidays that are leveraged by the OrganizerService to 1 {” s e r v i c e ” : { enter them in a calendar. 2 ”name” : ” H o l i d a y S e r v i c e ” , The development cycles of both Microservices are 3 ” p r o t o c o l ” : ”REST” , depicted in Figure 2. After a first version of Holiday- 4 ” methods ” : [ 5 { ” path ” : ”/ a p i / h o l i d a y s ” , Service is developed and deployed, COOP generates a 6 ”name” : ” g e t H o l i d a y s ” , JSON description of the Microservice, which is kept in 7 ” context ” : ” holiday ” , a repository. Listing 2 shows an excerpt for Holiday- 8 ” re turnType ” : ” l i s t ” , 9 ” returnTypeContext ” : ” date ” , Service V1. This includes, inter alia, the used proto- 10 ” parameters ” : [ col, the provided interface and semantic information 11 { ”name” : ” c o u n t r y ” , about the method and the communicated parameters 12 ” type ” : ” s t r i n g ” , and return values. 13 ” c o n t e x t ” : ” countryName ” } , 14 { ”name” : ” y e a r ” , HolidayService 15 ” type ” : ” i n t ” , generate register 16 ” context ” : ” date ” } ] } ] } } develop Start JSON first version service description of service On client side, the developer of the OrganizerSer- register vice is informed about the update of HolidayService. update second change version of JSON service COOP informs the developer about the type of change description service and whether the old client stub is compatible to Hol- idayService V2. In case of a compatible change, the yes keep old delete old developer can switch directly to HolidayService V2 by old End service service version still generating a new client stub through the JSON de- in use? no scription from the repository. In parallel, the usage of HolidayService V1 is deregistered. Accordingly, the developers of HolidayService are informed about no OrganizerService registered clients and, hence, may delete V1 safely. get JSON generate register Assuming that the change is incompatible, Holiday- Start usage of description client stub Service V1 would be kept as long as the client is reg- service istered for. The developers of OrganizerService first deregister generate yes have to adapt before switching to V2. Again, the dele- old service new client tion of V1 can be executed when no further clients are client stub compa- registered. register tible? adapt no End usage of changes new service 4 Related Work Providing automated tools for analyzing service be- Figure 2: Development Cycle haviour and its changes during the application lifecycle is essential for both change analysis and change man- agement. These tools may detect semantic, syntactic, At the OrganizerService, the Client Integration and protocol changes and inform dependent clients in module of COOP fetches the JSON description to gen- case of significant changes. This section reviews major erate the client stub for HolidayService V1. The stub approaches of service change detection. is automatically included in the dependencies and can Fokaefs et al. present in [FMT+ 11] VTracker, a be employed by the developer. At the same time, the tool for analysing WSDL interfaces. The application Client Integration module registers the usage of the identifies differences between two WSDL service de- current version of HolidayService. scriptions. In particular, the authors created an inter- By using the provided JSON description of Holiday- mediate XML representation to reduce the verbosity Service, OrganizerService can detect changes of Holi- of the WSDL specification. In [FS14], Fokaefs et al. dayService. Let us assume, HolidayService is extended propose the WSDarwin tool. The tool detects and ex- 4 tracts structural changes in the specification of a ser- Acknowledgment vice by identifiying changes between different versions This work is supported by the German Research Foun- of WSDL descriptions. Unlike VTracker, the approach dation (DFG) under the project PROSECCO, grant identifies differences between any pair of XML-based number 5534111. The authors would like to thank the documents by comparing all elements with each other. DFG for supporting their participation in worldwide This leads to the computation of the mapping based research networks. on their structural similarity. However, WSDarwin is a maintenance support tool that operates exclusively at development time and did not consider semantic References service changes. [Bec09] Sean Bechhofer. Owl: Web ontology lan- Similarly, Romano et al. [RP12] present in their guage. In Encyclopedia of Database Sys- work an application called WSDLDiff that recognises tems, pages 2008–2009. Springer, 2009. more fine-grained changes in WSDL descriptions, by comparing the subsequent versions of WSDL specifi- [BET11] Gerhard Brewka, Thomas Eiter, and cations. This approach takes into account the syntax Miroslaw Truszczyński. Answer set pro- of the WSDL file and the schema file XSD that is used gramming at a glance. Communications to define the data types of the WSDL interface and of the ACM, 54(12):92–103, 2011. enable the developer to analyse how a particular Web [Bya13] Brandon Byars. Enterprise integration service evolves. using rest. http://martinfowler. In [SAR+ 16, NKKM04], the authors compare the com/articles/enterpriseREST.html# semantic service description of subsequent versions to versioning, Accessed: May 2018, 2013. extract differences between them. The client develop- ers are relieved due to the automatic extraction of the [FMT+ 11] Marios Fokaefs, Rimon Mikhaiel, Nikolaos semantic description of the service implementation. Tsantalis, Eleni Stroulia, and Alex Lau. The presented framework assesses semantic change in An empirical study on web service evolu- terms of semantic concept drift using text and struc- tion. In 2011 IEEE International Confer- tural similarity methods to provide valuable insights. ence on Web Services, pages 49–56. IEEE, We provide therefore the Service Description Module. 2011. Tran et al. discuss in [TBK+ 16] an agent-based no- [FS14] Marios Fokaefs and Eleni Stroulia. Wsdar- tification mechanism that focuses on coordinated co- win: Studying the evolution of web service evolution within resource-constrained environments. systems. In Advanced Web Services, pages It detects and informs dependent clients automati- 199–223. Springer, 2014. cally about service changes by means of the analysis of ontology-based service descriptions. The paper mainly [GK14] Michael Gelfond and Yulia Kahl. Knowl- emphasizes the need for a notification mechanism, but edge representation, reasoning, and the does not dive into change detection and management. design of intelligent agents: The answer- set programming approach. Cambridge University Press, 2014. 5 Conclusion [LF14] James Lewis and Martin Fowler. Mi- In this paper, we present our COOP framework. croservices - a definition of this new archi- COOP automatically validates service compositions tectural term. https://martinfowler. with respect to their syntactic, semantic, and proto- com/articles/microservices.html# col compatibility. Therefore, it tracks service changes footnote-etymology, Accessed: May on behalf of the client and supports the version man- 2018, 2014. agement of service providers. Furthermore, it enables [NKKM04] Natalya F Noy, Sandhya Kunnatur, that dependent parties co-evolve in case of a service Michel Klein, and Mark A Musen. Track- change. ing changes during ontology evolution. In Currently, COOP comprises of the Service Descrip- International Semantic Web Conference, tion and Client Integration modules, while the first pages 259–273. Springer, 2004. prototype implementation of the Change Detection and Repository are still ongoing. Although the frame- [RP12] Daniele Romano and Martin Pinzger. An- work is tailored for Microservices, the mechanism can alyzing the evolution of web services us- be applied to service-oriented architectures in general. ing fine-grained changes. In 2012 IEEE 5 19th International Conference on Web Services, pages 392–399. IEEE, 2012. [SAR+ 16] Thanos G Stavropoulos, Stelios An- dreadis, Marina Riga, Efstratios Kon- topoulos, Panagiotis Mitzias, and Ioannis Kompatsiaris. A framework for measuring semantic drift in ontologies. In SEMAN- TiCS (Posters, Demos, SuCCESS), 2016. [TBK+ 16] Huu Tam Tran, Harun Baraki, Ramaprasad Kuppili, Amir Taherko- rdi, and Kurt Geihs. A notification management architecture for service co-evolution in the internet of things. In 2016 IEEE 10th International Symposium on the Maintenance and Evolution of Service-Oriented and Cloud-Based Envi- ronments (MESOCA), pages 9–15. IEEE, 2016. 6