Performance analysis of distributed systems with tiered software architecture has popularly entailed mean response time as the commonly used metric. It must be noted however that as a metric, response-time percentile is of greater importance since it is more desirable to reduce the variability of a system's response time, rather than minimizing the mean response time. It is a fact that analytical approximations for response time distribution do exist. However these analytical solutions capture only the steady-state behaviour (long-run behaviour) of the system. On the other hand, today's tiered cloud-based systems are so complex that they never reach steady-state. Consequently, analyzing their transient behaviour (short-term behaviour) becomes far more important than analyzing their steady-state behaviour. Regardless, it is a difficult task to accomplish transient analysis analytically due to the enormous state space of such systems. In this work, we analyze the transient behaviour of a 3-tier cloud-based system using discrete event simulation. We model the system as an open queueing network and estimate the response time distribution through the simulation. The results show that in a 3-tier system, a configuration with large number of virtual machines (VMs) does not necessarily perform better than a configuration with smaller number of VMs. The results further show that different system configurations containing the same number of VMs yield different performance depending on the replication level of software components running in different tiers. We demonstrate that our model can serve as part of a decision support system associated with dynamic VM provisioning. Our model can be used to determine whether a given number of VMs can meet the desired service-level objectives (SLOs) specified in terms of response time percentile.
Increasingly, tiered web applications are getting deployed in clouds since cloud computing allows for dynamic scaling of Copyright © 2016 for the individual papers by the papers' authors. Copying permitted for private and academic purposes. This volume is published and copyrighted by its editors. computational resources as required on a pay-per-use basis. This relieves the application service providers from buying and maintaining data centers thereby reducing the operational cost. However, such deployment poses challenges for automated performance management of these applications. This is because the management system now needs to decide on the amount of computational resources (VMs) to be acquired or released dynamically for a change in system workload while ensuring that the SLOs are not violated. One way to make such a decision is to use system performance models repeatedly to evaluate various what-if scenarios [3].
System performance models can be used to predict different performance measures. The commonly used performance measure has been the mean response time (RT). However, Broadwell [5] has justified that response-time percentile as a metric is of greater importance than mean RT since it is more desirable to reduce the variability of a system’s response time, rather than minimizing the mean response time.
In this work, we have used an open queueing network as our system performance model because we assume that the cloudbased systems have large number of users and the users are transient in their use of websites. Consequently, the web application might behave more like an open system as suggested by Harchol-Balter [9].
For open queueing networks, computing the exact response time distribution analytically is difficult since it may have to deal with infinite number of system states. Several approximations to response time distribution do exist though [1, 4, 6, 10, 11]. However these analytical solutions are for long-run or steady-state behaviour of the system.
Cloud-based web systems are so complex and dynamic that they never reach steady-state. Consequently, analyzing their transient behaviour becomes far more important than analyzing their steady-state behaviour [2]. In this work, we are interested in analyzing the transient behavior of the system. In spite of the importance of transient analysis, it is a difficult task to achieve it analytically due to the enormous state space of these systems. We therefore resort to discrete event simulation for our analysis. The goal of this paper is to develop a simulation model to analyze transient behavior of a 3-tier cloud-based web system. Our model predicts the response time distribution for a given system workload. We model the system as an open queueing network with only feed-forward arcs. The system workload is represented by the arrival rate, i.e. the number of job arrivals per unit time. Here, we assume that the software server at each tier is replicated into one or more copies and each copy runs on a separate virtual machine (VM). Thus, the queueing network consists of a variable number of VMs in three tiers. Although our simulation model may be computationally expensive as compared to an analytical counterpart, it is more general in terms of service time and interarrival time distributions.
For our hypothetical 3-tier system, over- or under-utilization of VMs could occur if an application service provider didn’t purchase enough number of VMs from the cloud provider for different tiers of the system. For example, when the number of VMs purchased is too few to handle a given workload, most of the requests will not be processed within the required response time threshold. On the contrary, if too many VMs were purchased to handle relatively fewer number of requests, the VMs will be under-utilized and this could lead to wastage of computational resources. Hence, the challenge is to find a configuration for the system consisting of the appropriate number of VMs for each stage to process the incoming requests that will ensure that a required response-time percentile is within a given threshold. The key contribution of our work is twofold. First our work presents a model—not only to predict mean response time but also to predict the response time distribution. Our model is general enough to accommodate non-Markovian inter-arrival and servicetime distributions. Second, our work demonstrates how our model can serve as part of a decision support system to find the appropriate configuration that would ensure that a given SLO (in terms of response-time percentile) is met.
The rest of the paper is organized as follows. Section 2 describes the 3-tier software architecture. Section 3 describes the open queueing network performance model for this architecture. Section 4 analyzes the queueing model and discusses the results. Finally section 5 concludes the work.
Figure 1 shows the software architecture of our hypothetical 3-tier cloud-based system. We analyze this architecture in this work. users and their web browsers servers (DB servers) run in the third tier (tier-3). The users access the application at the web servers. We assume that at any given tier, one or more VMs can be provisioned, each running a single instance of a server relevant to that tier. For our modeling purposes, we assume that the workload is equally distributed among the servers at any given tier. We indicate this in Figure 1 using the phrase "balanced load".
We assume that a service request will be processed exactly once (in a server) at each tier. After completion of processing at the third tier, the response is returned to the user. We further assume that a request incurs a waiting time in the server’s queue before being processed, if the server is busy. The request then incurs a service time for getting processed in the server.
The request is first sent to a Web server for processing. If the Web server is busy then the request needs to wait in the server’s queue before getting processed.
The request is then redirected to an App server present in the second tier. If the App server is busy then the request needs to wait in the server’s queue before being processed.
Next, the request is redirected to a DB server present in the third tier. As before, the request waits in the server’s queue if the server is busy. Once the processing of the request is finished at the DB server, the response is sent back to the user.
The 3-tier software architecture of Figure 1 is modeled as an open queueing network (see Figure 2). In Figure 2, each layer of queueing stations represents the collection of servers (each server running on its own VM) supporting execution of requests at each tier. We assume that the replicas of servers in a given tier have identical service time distribution and that the arrivals are split uniformly among them. Let denote the arrival rate of user requests at tier 1. If we have 3 server replicas in tier-1, then the arrival rate at each of that replica will be /3. We assume that 1 is the service rate of each Web server replica at tier-1, 2 denotes the service rate of each App server replica at tier-2, and 3 denotes the service rate of each DB Server replica at tier-3.
As shown in Figure 2, the response time of a request is the time between the arrival of the request at a tier-1 server to the completion of the request at a tier-3 server. This time includes the waiting times at the queues of the relevant servers at different tiers and the service times of those servers.
Let RTi denote the response time of the i-th request. We assume that the SLO is specified in terms of response time percentile. An example SLO is “The response time should be less than or equal to 0.3 seconds with probability 0.95”. This means that 95% of the requests should complete within 0.3 seconds. Here, 0.3 seconds is the response time threshold. We denote this threshold by .
We have simulated the open queueing network shown in Figure 2 using a discrete event simulation framework called SimPy—a Python based framework.
Let N denote the total number of requests completed in one simulation run. During every simulation run, we record the response time of each request RTi. At the end of each simulation run, we compute the number of requests whose response time is less than or equal to the threshold . For request i, Let = 1 if { RTi ≤ } = 0 otherwise Workshop on Software Architectures for Adaptive Autonomous Systems (SAAAS 2016) - colocated with ISEC 2016, Goa, India, Feb 18, 2016 42 Tier-1 Tier-2
In this section, we analyze the queueing network of Figure 2 for
different configurations of VMs. We denote a VM configuration
as (C1, C2, C3) where C1, C2, C3 denote the number of VMs in
tier1, tier-2 and tier-3 respectively. We assume that a VM in a tier
runs a server replica relevant to that tier. In sub-section 4.1, we
take an example VM configuration (
To illustrate the predictions of response time distribution and
percentiles, we consider the configuration (
A plot of the five resulting response time distributions is shown in
Figure 3 for the configuration (
Subsequently, it will not be able to meet the SLO any further.
Figure 4 summarizes some important statistics about the five
response time distributions. It shows the mean response time and
three different response time percentiles (90th, 95th and 99th) with
the passage of time. We find that during a time period of 5
Workshop on Software Architectures for Adaptive Autonomous Systems (SAAAS 2016) - colocated with ISEC 2016, Goa, India, Feb 18, 2016
43
minutes, the configuration (
Since we are allowed a maximum of 3 VMs per tier, there could be 27 potential VM configurations that could be analyzed. We analyze the response time per request for processing 1000 requests using the model parameter values provided in Table 1.
We undertake this analysis for the response time threshold range of 0.1 to 0.7 seconds. We consider only those configurations for which the probability of meeting a given threshold (in the range 0.1 to 0.7 secs) is 0.55 or higher. We find that out of the 27 different configurations, only 8 of them meet this requirement.
Figure 5 shows the response time distribution for the eight
configurations (
In this section, we illustrate how our model is used to evaluate different what-if scenarios to decide for a VM configuration that would meet a specified SLO.
We are aware that the cloud computing paradigm allows for the dynamic scaling of computational resources as required on a payper-use basis. Let us assume that our cost budget will allow us to buy a maximum of 3 VMs for each tier. We further assume that every VM costs the same amount of dollars. Therefore, instead of a static configuration, our aim is to change the configuration
Table 2 summarizes some important statistics about the response
time distributions of eight VM configurations. It shows the mean
response time and 95th and 99th response time percentiles. This
table demonstrates that percentile measures are of greater
importance than mean values. As an example, let us consider an
SLO specified in terms of mean RT as “The mean RT should be
below 0.1 seconds”. Eyeballing the Table 2, we find that two
configurations (
Next we demonstrate that a configuration with large number of
VMs does not necessarily perform better than a configuration with
smaller number of VMs. Let us consider two VM configurations:
(
With the assumed model parameter values of Table 1, this figure
shows that the configuration (
We have developed a simulation model to analyze transient behavior of a 3-tier cloud-based web system. Our model not only predicts mean response time but also the response time percentiles. Our model is general enough to accommodate nonMarkovian inter-arrival and service-time distributions. We have demonstrated how our model can serve as part of a decision support for VM planning process. Given all the VM plans satisfying SLO requirements, we acknowledge that it is not a straight forward task to figure out the optimal plan. We recommend that research be undertaken to investigate whether our model can be used jointly with an optimization engine to select the best VM plan.
We would like to thank NSERC Canada for their financial support through NSERC Discovery Grant of Olivia Das.
Workshop on Software Architectures for Adaptive Autonomous Systems (SAAAS 2016) - colocated with ISEC 2016, Goa, India, Feb 18, 2016 45