With the substantial growth of cloud computing over the past decade, microservice architectures have gained significant popularity and have become a prevalent choice for designing cloud-based applications. Microservices based applications are distributed and each service can run on a diferent machine. Due to its distributed nature, one of the key challenges when designing applications is the mechanism by which services communicate with each other. There are several approaches for implementing inter process communication (IPC) in microservices; each comes with diferent advantages and trade-ofs. While theoretical and informal comparisons exist between them, this paper has taken an experimental approach to compare and contrast the popular forms of IPC communications. Several load test scenarios have been executed to obtain quantitative data related to performance eficiency, and availability of each method. The evaluation of the experiment indicates that, although there is no universal IPC solution that can be applied in all cases, the asynchronous pattern ofers various advantages over its synchronous rival.
1. Introduction
enormous attention and gained popularity from the industry. They helped large organisation such as Amazon and Netflix to serve millions of requests per minutes [1]. Microservice architecture is a style of developing software as a collection of independent services.
Each service is running on its own process that is independent from other processes and can be deployed separately from other services [2]. Designing a software based on microservices involves answering questions and overcoming technical challenges that often do not exist in monolithic architecture, like inter process communication (IPC) [3], service discovery [4], decomposition strategy [5], or managing ACID transactions [6].
Despite the growth and importance of microservices in industry, there has not been suficient research on microservices, partly due to lacking a benchmark system that reflects the characteristics of industrial microservice systems [7]. IPC is one of the important challenges of microservice architectures [8]. In monolithic based systems, components can call each other at the language-level while in microservices each component is running on its own process and possibly on a diferent machine than other services. The choice of IPC mechanism is an important architectural decision which can impact the software’s non-functional requirements [8].
As of today, there are no concrete explanations or any standardized approach that can help to decide the right IPC method when designing microservice based applications. Due to this reason, there is an abundant confusion around the question of when to use which method and what are the trade-ofs for choosing that method. Deciding between a synchronous and asynchronous approach is an important decision to take in regards to how services collaborate with each other [9].
There are two questions this paper is working towards answering: 1. From performance eficiency standpoint, what are the implications for utilizing available synchronous and asynchronous methods for implementing IPC in microservice architectures? 2. How does the IPC method choice impact avail
ability of the system?
The motive behind selecting the performance eficiency, and availability as the two criteria for this research is that the choice of IPC method directly impacts these two non-functional requirements in a microservices based system, while other non-functional requirements such as security [10] and maintainability [11] can span over few other areas and goes beyond IPC. Being able to measure these qualities in the system are critical in order to achieve an eficient management of any software system [12, 13]. Moreover, the chosen quality attributes are among the top priorities for most modern applications [14, 15].
In this work, we describe a systematic approach for selecting IPC method when designing microservices based software. The remainder of this paper focuses on state of the art in identifying diferent IPC models in section 2. Next, in section 3, we discuss the development of the prototypes built for the purpose of discovering the relationship between each IPC method and its impact on performance eficiency and availability.
We then run a test against each prototype to investigate its outcome and discuss previous work conducted in this domain. Finally, we draw a conclusion in section 6. 2. State of the Art each service exposes a set of endpoints to enable the interactions with other microservices and exchange of information between them. The server interacts directly with client through its interface also known as Web API. • gRPC: gRPC is an open source high performance RPC framework designed an developed by Google. Remote procedure call (RPC) is a mechanism used in many distributed applications to facilitate inter process communication.
RPC was first implemented by Birrell and Nelson [18] and it has been regarded as a protocol that enables a message exchange between two process with characteristics of low overhead, simplicity and transparency [19]. By default, when a client sends a request to a server it halt the process and waits for the results to be returned. RPC is therefore considered as synchronous form of communication [20]. Figure 1 presents the operational process between client and server in gRPC. In this model, the client implements the same method as its corresponding server through local objects also known as stubs. 2.2. Asynchronous Communication When designing IPC mechanism, there are two type The asynchronous form of communication can be imof interaction style to choose from: synchronous and plemented in microservices when services exchange asynchronous, which we will shortly introduce next. messages with each other through a message broker. In this form of interaction, the message broker acts as 2.1. Synchronous Communication an intermediary between services to coordinate the request and responses [8]. One of the fundamental Synchronous communication is often regarded as re- diferences in asynchronous communication as comquest/response interaction style. One microservice pared to the synchronous mode is that in asynchromakes a request to another service and waits for the nous communication the client no longer makes a diservices to process the result and send a response back. rect call to the server and expect an immediate answer. In this style, it is common that the requester blocks its Instead, other services subscribe to the same broker to operation while waiting for a response from the re- pick-up the available requests and process them furmote server. Representational state transfer (REST) ther before placing them back to the message queue. application programming interfaces (API) [16] and Figure 2 provides an example of the asynchronous gRPC1 are the most common framework for imple- pattern. In this sample, when a new order is created, menting Synchronous form of communication in mi- the customer service publishes a request to the brocroservices [8]. ker with some metadata such as customer id, customer email address, etc. Other services such as loyalty, post, and email service subscribe to that broker and take the request from there without having to communicate with Customer service directly. • REST API: REST is an architectural style that is commonly used for designing APIs for modern web services [17]. In a system that uses REST API for its IPC communication, each service typically has its own web-server up and running on a specific port such as 8080 or 443, and 3. Implementation • Product Information Service: This microservice is responsible for fetching the primary metadata associated with the requested product. Information such as product name, price, description, color, and image are stored in this microservice database.
we have designed and developed a set of microservices for an e-commerce scenario. In this scenario, the goal is to simulate fetching all the information required All the microservices have been developed using to display a product page of an e-commerce website. NodeJS2. A non-relational database system, MonA client requests a product page to be displayed on goDB3, has been used as the database solution for all his/her device and behind the scenes the following mi- the microservices except for the service responsible for croservices work together to serve that request: providing shipment information. Due to the nature of data required by shipping service, the shipping service uses MYSQL4. Docker5 has been utilized to containerize all the microservices. In order to run the test system, the services have been deployed to Microsoft Azure Kubernetes Cluster Service6. Table 1 shows the • Customer Shopping Cart Service: This microservices is responsible for fetching the existing items in the customer’s shopping cart in order to display them to the customer. • Product Review Service: This microservice is responsible for fetching the customer reviews 2https://nodejs.org/en/ 3https://www.mongodb.com/ 4https://www.mysql.com/ 5https://www.docker.com/ 6https://azure.microsoft.com/sv-se/services/kubernetesservice/ Table 1 to understand how each IPC method reacts diferently Kubernetes Cluster specification of the test system. when the concurrent requests and trafic to the system Instance Type Azure DS2-v2 increase or decrease. vCPU 2 Throughput is calculated by the total number of reMemory 7 GiB quests and responses the method managed to make Storage 8 GiB, SSD, 6400 IOPS within the specified duration of 180 seconds; the Kubernetes Version 1.14.8 higher the number, the higher the throughput and the Node Count 3 better it is.
The results are presented in figure 3. The data indicates that gRPC has outperformed REST API, and hardware specification of the testing system used for RabbitMQ in the first case with 50 users by being this research. able to process 43 requests higher than REST API, and
In the synchronous mode both REST API, and gRPC 147 requests more than RabbitMQ; this signifies that have an identical architecture; in both methods, there synchronous form of communication can ofer higher is a direct communication between API Gateway7 and throughput than the asynchronous method in the siteach microservice. Each microservice acts as server, uation when the load to the system is relatively low. and the API Gateway acts as a client of those server. Meanwhile, the result of the first case also reveals The key diference between REST API and gRPC is the that synchronous form of communication can process underlying communication protocol as well as the for- requests slightly faster than asynchronous form and, mat of the messages they exchange. gRPC has adopted therefore, has lower latency when the number of conprotocol bufer 8 as its proprietary message format, current threads10 in the system is low. while the REST API uses JSON [22] format to exchange The second case has double the number of virtual data. users as compared to the first one. Increasing the num
The asynchronous architecture uses RabbitMQ as ber of virtual users causes the number of concurrent message broker. In this pattern, the communication threads in the system to grow and results in longer between API gateway and other services does not take processing time. The same data imply that gRPC has place directly, rather it goes through a mediator also the highest throughput by processing a higher numknown as message queue. In both synchronous and ber of requests compared to RabbitMQ and REST API; asynchronous methods, the API Gateway is the entry however, the gap between gRPC and RabbitMQ is now point to the system, which receives a request with spe- more narrowed than in the first case. In this test, gRPC cific product id from client’s device such mobile app or managed to score the best average response time than web browser over HTTPS protocol. The gateway then REST API and RabbitMQ by 200 milliseconds. The communicates back and forth with each microservice processing time between REST API and RabbitMQ are depending on the IPC method the system uses. equal to each other; however, RabbitMQ managed to process extra 25 requests than its synchronous rival.
The number of virtual users in the third case has 4. Results and Evaluation increased four times as compared to the first case. The outcome of the third testing experiment im4.1. Performance Eficiency plies considerable diference between synchronous Three test cases have been designed and executed us- versus asynchronous form of communication both in ing Apache JMeter9. All test cases aim to measure throughput and latency when the number of paralthe throughput of each IPC method. Throughput is lel requests increases. In this test, asynchronous form an essential attribute for calculating performance efi- of communication using RabbitMQ has outperformed ciency. In all three test experiments, the test duration the other two methods by being able to process a was 180 seconds, while the number of concurrent vir- total of 4480 requests within the given period while tual users that continuously send requests to the sys- gRPC managed to process 132 requests lower than tem and wait for response has been varied. The motive RabbitMQ, and REST API processed 146 less requests behind having test duration as a constant variable and than its asynchronous rival. What makes the asynchnumber of virtual users as the controlled variable is ronous pattern to operate better in the third test case is that, in asynchronous form the performance decline take place more gradually while in the synchronous
10Each virtual user occupies one thread in the system. pattern the performance begins to drop radically as and the number of requests/responses were not been soon as the load to the system intensifies. tracked since they do not contribute to determining the availability of the method. The first case ran with 4.2. Availability 200 virtual users, the second with 300 virtual users, and third with 400 virtual users. Without having a There are variety of parameters that can afect avail- high number of parallel users measuring availability ability of a system –even hardware components can becomes more challenging as the system remains opplay a role in determining the availability rate of a sys- erational for a significantly longer duration. tem. For this measurement, all the parameters out- Figure 4 provides a summary of the conducted tests. side IPC has been ignored. The availability of each During the first test, it took about seven minutes for IPC method has been calculated by using the following the services to become unavailable using RabbitMQ, equation [23]: while gRPC went down after about five minutes, and the REST API took approximately four and a half = , minutes. These numbers were then dropped in each + method in the subsequent tests as the number of parwith MTTF standing for "Mean Time to Failure," and allel requests were doubled. After the services became MTTR for "Mean Time to Recovery." MTTF represents unavailable, the Kubernetes cluster has been manuthe duration that the system is expected to last in op- ally restarted. From that moment, both gRPC and eration before failure occurs. In contrast, MTTR rep- REST API took about 20 seconds only to become availresents the duration the system requires to return to able again, while RabbitMQ took ten extra seconds. operation after a failure has occurred. The higher the The main reason behind RabbitMQ taking longer than MTTR, the longer it takes for the system to recover synchronous form to return back to operation is the from a failure, which consequently reduces the avail- fact that it has an extra component known as a mesability of the system. sage broker that requires to be refreshed and establish
Based on this formula, three other tests were exe- a new connection with each service. From this expercuted using Apache Jmeter against all the three difer- iment, it is possible to infer that an asynchronous apent IPC methods to discover which one ofers higher proach ofers higher availability than its synchronous availability. Unlike the previous test cases that had a opponents. ifxed duration, these test cases had no specific dura- Consequently, if microservices use a synchronous tion. They ran as long as the services became unavail- based communication both client and server must be able due to the high number of requests coming to the responsive at all time, otherwise the request will fail system. Further, in this test, the average response time, after a specific duration depending on the configuration. In contrast, a temporary outage of the server in threats to validity. First, we performed our experian asynchronous setting causes minimal to no impact ments just with single technologies as representatives to the consumer, since the consumer is loosely coupled for certain principles (synchronous vs. asynchrowith the server. The requests can stay in the message nous). Therefore, our results can just indicate certain queue and be processed at the later timing when the advantages of these principles. Second, we simulated server is back to operation. The asynchronous pattern no complete system but just a small part of a bigger ofers capabilities that can help the system to improve system, e.g., there is no communication between the its availability and resiliency from outage. It allows microservices during our requests. However, this continuous operation even if there is a failure in one of ensures that we are not testing other efects, but the system’s components without compromising the only the interaction between the gateway and the availability of the entire system. microservices. Third, we were using technologies that are highly configurable, thus a completly diferent 4.3. Discussion and Threats to Validity configuration could lead to other results. However, as we just changed configurations where necessary, we assume that others can reproduce our results, especially as they are in accordance with our theoretical expectations.
In addition to the two non-functional requirements that have been evaluated throughout this work, it is important to take into account the functional requirements for which microservices are being developed for. It is essential to distinguish whether the scenario requires an immediate response back from services or 5. Related Work not. To elaborate further on this, in the proof of concept scenario that was built during this work, display- Suficient work has been done to benchmark the pering a product page for an e-commerce was simulated. formance of microservices, and compare and contrast In this scenario, the client sends a request to load the it with other architectures such as service oriented arproduct page and expects an immediate result back. chitecture (SOA) [24], or with the monolithic architecThe result of the request can either be the product page ture [25, 26, 27, 28, 29]. or an error that indicating the request was failed. The Ueda et al. [30] conducted research at IBM that key point in this scenario is that the client expects an aimed to design an infrastructure that is optimized for immediate result. In such scenarios the synchronous running microservice architectures. The team built form of communication can be more suitable as these two versions of the sample application. One based on scenarios cannot take advantage of the features that monolithic and the other based on microservices. The an asynchronous form can ofer. team discovered a significant performance overhead Furthermore, our research incorporates some and higher hardware resource consumption in the microservices version of the application as compared to Further, the authors argue that microservice archimonolithic one. The paper has marked poor design tectures lead to a higher availability as the new system of process communication in microservice architec- is broken down into several components and decoutures as one of the significant performance degrada- pled from each other, which makes it possible to loadtions, and, therefore, unleashed the potential for fur- balance individual services as needed. This was parther research and improvements in this topic. The pa- ticularly not possible in the legacy monolithic based per has also pointed out that network virtualization system. At the same time, the new architecture oftechniques, which are often used in a microservice ar- fers higher reliability and can better cope with failchitectures, is another non-negligible reason behind ures. This is due to the fact that in the new system the the performance gap of monolithic versus microser- communication relies on a message-broker that can be vice architecture. The paper, however, has not pre- configured to ensure all messages get delivered evenscribed any specific solution or suggestion as to how tually. to overcome these challenges but rather pointed out the potential future work for it.
Fernandes et al. [
AMQP performs far better than REST API as it has a better mechanism for data loss prevention, better References message organization, and utilize lower hardware resources. [1] J. Thönes, Microservices, IEEE software 32 (2015)
In contrast to Fernandes et al., check we in our work 116–116. the behavior of the systems with diferent loads. We [2] D. Namiot, M. Sneps-Sneppe, On micro-services recognize that synchronous approaches perform good architecture, International Journal of Open Inwith low loads while asynchronous approaches scale formation Technologies 2 (2014) 24–27. better at higher loads. [3] L. L. Peterson, N. C. Buchholz, R. D. Schlicht
Meanwhile, Dragoni et al. [28] have conducted a ing, Preserving and using context information migration for a real-world mission-critical case study in interprocess communication, ACM Trans. in the banking industry by transforming a monolithic Comput. Syst. 7 (1989) 217–246. doi:10.1145/ software into a microservice architecture. They ob- 65000.65001. served how availability and reliability of the system [4] S. Haselböck, R. Weinreich, G. Buchgeher, Decichanged as a result of the new architecture. The so- sion guidance models for microservices: service lution consists of decomposing several large compo- discovery and fault tolerance, in: Proceedings of nents to which some of them requires to communicate the Fifth European Conference on the Engineerwith third-party services. The services in the new ar- ing of Computer-Based Systems, 2017, pp. 1–10. chitecture use message-based asynchronous commu- [5] J. Fritzsch, J. Bogner, A. Zimmermann, S. Wagnication as its IPC model to exchange data with each ner, From monolith to microservices: A clasother. The authors believe that aiming to have a simple sification of refactoring approaches, in: J.-M. and decouple integration between services and follow- Bruel, M. Mazzara, B. Meyer (Eds.), Software Ening principle to handler failure will eventually lead to gineering Aspects of Continuous Development higher reliability in microservice architecture. and New Paradigms of Software Production and Systems (TOCS) 2 (1984) 39–59.
Deployment, Springer International Publishing, [19] J.-K. Lee, A group management system analCham, 2019, pp. 128–141. ysis of grpc protocol for distributed network [6] C. K. Rudrabhatla, Comparison of event choreog- management systems, in: SMC’98 Conference raphy and orchestration techniques in microser- Proceedings. 1998 IEEE International Conference vice architecture, Int J Adv Comput Sci Appl 9 on Systems, Man, and Cybernetics (Cat. No. (2018) 18–22. 98CH36218), volume 3, IEEE, 1998, pp. 2507– [7] X. Zhou, X. Peng, T. Xie, J. Sun, C. Xu, C. Ji, 2512.
W. Zhao, Poster: Benchmarking microservice [20] R. A. Olsson, A. W. Keen, Remote Procedure systems for software engineering research, in: Call, Springer US, Boston, MA, 2004, pp. 91–105. 2018 IEEE/ACM 40th International Conference doi:10.1007/1-4020-8086-7_8. on Software Engineering: Companion (ICSE- [21] S. G. Du, J. W. Lee, K. Kim, Proposal of grpc as Companion), IEEE, 2018, pp. 323–324. a new northbound api for application layer com[8] C. Richardson, Microservices patterns: with ex- munication eficiency in sdn, in: Proceedings of amples in Java, Manning Publications, 2019. the 12th International Conference on Ubiquitous [9] S. Newman, Building microservices : designing Information Management and Communication, ifne-grained systems, first edition.. ed., 2015. 2018, pp. 1–6. [10] P. Johnson, D. Gorton, R. Lagerström, M. Ekst- [22] C. Severance, Discovering javascript object noedt, Time between vulnerability disclosures: A tation, Computer 45 (2012) 6–8. measure of software product vulnerability, Com- [23] P. Johnson, R. Lagerström, M. Ekstedt, M. Österputers & Security 62 (2016) 278–295. lind, It management with enterprise architecture, [11] R. Lagerström, P. Johnson, M. Ekstedt, Architec- KTH, Stockholm (2014).
ture analysis of enterprise systems modifiability: [24] T. Erl, Service-oriented architecture: concepts, a metamodel for software change cost estima- technology, and design, Pearson Education India, tion, Software quality journal 18 (2010) 437–468. 1900. [12] P. Närman, P. Johnson, R. Lagerström, U. Franke, [25] T. Cerny, M. J. Donahoo, J. Pechanec, DisM. Ekstedt, Data collection prioritization for sys- ambiguation and comparison of soa, microsertem quality analysis, Electronic Notes in Theo- vices and self-contained systems, in: Proceedretical Computer Science 233 (2009) 29–42. ings of the International Conference on Research [13] M. Ekstedt, U. Franke, P. Johnson, R. Lagerström, in Adaptive and Convergent Systems, RACS T. Sommestad, J. Ullberg, M. Buschle, A tool ’17, Association for Computing Machinery, New for enterprise architecture analysis of maintain- York, NY, USA, 2017, p. 228–235. doi:10.1145/ ability, in: 2009 13th European Conference on 3129676.3129682.
Software Maintenance and Reengineering, IEEE, [26] D. Taibi, V. Lenarduzzi, C. Pahl, Processes, mo2009, pp. 327–328. tivations, and issues for migrating to microser[14] U. Franke, M. Ekstedt, R. Lagerström, J. Saat, vices architectures: An empirical investigation, R. Winter, Trends in enterprise architecture IEEE Cloud Computing 4 (2017) 22–32. practice–a survey, in: International Workshop [27] R. Chen, S. Li, Z. Li, From monolith to microseron Trends in Enterprise Architecture Research, vices: A dataflow-driven approach, in: 2017 Springer, 2010, pp. 16–29. 24th Asia-Pacific Software Engineering Confer[15] P. Johnson, R. Lagerström, P. Närman, M. Simon- ence (APSEC), 2017, pp. 466–475. sson, Extended influence diagrams for system [28] N. Dragoni, S. Dustdar, S. T. Larsen, M. Mazzara, quality analysis, Journal of Software 2 (2007) 30– Microservices: Migration of a mission critical 42. system, arXiv preprint arXiv:1704.04173 (2017). [16] R. T. Fielding, R. N. Taylor, Architectural styles [29] Z. Kozhirbayev, R. O. Sinnott, A performance and the design of network-based software ar- comparison of container-based technologies for chitectures, volume 7, University of California, the cloud, Future Generation Computer Systems Irvine Irvine, 2000. 68 (2017) 175 – 182. doi:10.1016/j.future. [17] M. Masse, REST API Design Rulebook: Design- 2016.08.025.
ing Consistent RESTful Web Service Interfaces, " [30] T. Ueda, T. Nakaike, M. Ohara, Workload charO’Reilly Media, Inc.", 2011. acterization for microservices, in: 2016 IEEE in[18] A. D. Birrell, B. J. Nelson, Implementing remote ternational symposium on workload characteriprocedure calls, ACM Transactions on Computer zation (IISWC), IEEE, 2016, pp. 1–10.