Many applications today are deployed on the cloud and require their owners to decide how many resources (e.g., CPUs, memory) to allocate, known as provisioning. Due to the complexities of understanding variations in load, many applications are either under-provisioned or overprovisioned. We developed an automated resource configuration recommender system but found that application owners were hesitant to trust an automated system that may impact their applications' performance. Towards increasing trust, we built an interactive dashboard that allowed them to understand their resource usage, review the automated system's recommendation, and control when the recommendations are applied. We iteratively designed and piloted the system with owners of twenty cloud applications and discuss seven design needs for designing dashboards for automated resource management systems.
________________________________________________________ Workshop proceedings Automation Experience across Domains In conjunction with CHI'20, April 26th, 2020, Honolulu, HI, USA Copyright © 2020 for this paper by its authors. Use permitted under Creative Commons License Attribution 4.0 International (CC BY 4.0). Website: http://everyday-automation.tech-experience.at
automation; resource allocation; dashboard
•Human-centered computing ! Interactive systems and tools;
Performance of an application executing in the public cloud invariably depends on its provisioned resources. Underprovisioning can result in performance degradation and costly application-level agreement (SLA) violations, while over-provisioning leads to low resource utilization and wasted money. Designing cloud applications such that they can deliver on resource efficiency without performance degradation is key to their success. However, deciding these resource requirements is not straightforward for application owners. Cloud applications undergo striking variations in load and application owners don’t always have tools to understand how their resources are being used over time. To alleviate these pains, we developed an automated resource configuration recommendation algorithm that provides recommendations for right-sized resource provisioning. However, during piloting, we found that application owners were hesitant to make changes to their applications’ configuration due to a lack of understanding of how their allocated resources are being used and distrust that the automated recommendations would not degrade their applications’ performance. Based on previous literature in the trust and automation domain, we decided to develop an interactive dashboard for application owners to build trust in the recommendations.
In this paper, we (1) distill seven design needs for an interactive dashboard for application owners through iterative prototyping and expert reviews, and (2) present our final system for automated cloud resource management(AutoCRM).
Previous work to aid right-provisioning falls into reactive and
predictive approaches. Reactive approaches, such as
autoscaling with predefined heuristics, are often used to adapt
to changes in load [
Much work has been done surrounding trust in automated
systems, including determining trust [
The initial recommender was piloted with six teams owning twenty applications total. Each team saw CPU and memory line charts showing allocated resources versus actual usage, as well as the recommended allocation over time. Based on feedback, we iteratively developed a prototype in close collaboration with engineering teams and project managers who were responsible for collecting resource usage, identifying pilot teams, and overall coordination. The development phase took about six months. We then synthesized the feedback into seven design needs. s U P
C AutoCRM Backend b
(1) Add automation in stages. Because no similar system
existed in a previously fully manual process, it was a large
shift in application owners’ workflows which resulted in a
lack of trust. Developing the system slowly in stages
ensured accuracy of the tool and increased users’ confidence.
For example, though possible, a fully automated version
of the system was not immediately deployed. Instead, a
mixed-initiative approach where users can manually apply
recommendations is used.
(2) Perform formative analysis to model user needs.
Because users were primarily concerned with not interrupting
application performance, it was essential to be conscious
about the recommendations and provide appropriate error
margins, since the real future demands of the application
are unknown [
Here we describe the final version of the system in two parts: (1) the recommender backend and (2) the dashboard UI.
The main components of AutoCRM (Fig. 1) are an ARIMAenabled forecaster, a simulator, and a controller that emFigure 2 (cont’d): The sidebar displays the recommendations with cost and efficiency benefits. The line charts display historically allocated (red) and used (grey) CPUs and memory, so users can understand their applications’ resource needs.
The simulated CPU allocation based on the recommended configuration is also shown on the CPU chart (green). ploys a carefully designed optimization function to arrive at efficient application resource sizing values. events) for an application. The configuration that minimizes the cost function is recommended.
Forecaster The Forecaster predicts future usage of an
application applying the ARIMA model [
Controller The controller receives the predicted resource usage values (Fig 1c) and compares two sets of resource configurations for their utility in terms of utilization and overheads using a cost function. The cost function is a weighted sum of resource wastage and overheads (number of scaling
The UI (Fig. 2) was designed to display the recommendations with cost and efficiency benefits and help application owners understand and compare their prior resource utilization against the simulated utilization.
Recommendation Display The sidebar displays the recommended configurations and projected utilization and cost savings. We provide both “Before” and “After” measurements so application owners can directly gauge any costand resource-saving benefits.
User Controls Many application owners have multiple applications, so a dropdown menu allows them to choose the application of interest. Its repository name and region are shown to the right. The date picker allows users to choose from pre-defined date ranges (last week, last month, last three months, last six months) or select a custom date range. Most importantly, users can manually Apply recommendations or Undo the last application.
Utilization Charts As discussed, the most important aspect was for users to examine their utilization history. Thus, we provide CPU and memory charts that display historical allocated versus actual usage for each resource. Next, it was important for users to see how the resources would scale given their utilization for a particular period, so in the CPU chart, the simulator results are also shown, which made users more comfortable in reducing allocation without sacrificing performance or u ptime. Users can pan and zoom on the charts to inspect the data closer.
In this paper, we present design needs and a system for automated resource management. We developed a backend recommender and an accompanying dashboard UI to increase trust in the UI and share our most important lessons learned that can benefit the c ommunity. Future work includes deploying the system to the company at large and evaluating the adoption rates for the recommendations, as well as fully automating the system in the long-term.
We would like to thank Israel Derdik, Travis Borovatz, and Chandler Allphin for their valuable feedback and support during designing, deploying, and evaluating the system.