Eco-Efficient Cloud Resource Monitoring and Analysis An approach for saving energy and reducing the carbon footprint of cloud infrastructures Axel Tenschert, Pavel Skvortsov, Michael Gienger HLRS – High Performance Computing Center University of Stuttgart Stuttgart, Germany {tenschert, skvortsov, gienger}@hlrs.de Abstract—This paper presents an approach for monitoring server being connected to Zabbix client agents installed on cloud computing resources by using customized monitoring each physical node. Additionally, a data mining analysis is metrics. In addition, based on the monitoring results, this performed on the monitored data set. approach includes a data mining analysis for making The developed and implemented concept for monitoring assumptions regarding consumed power of infrastructure and virtual machines (VMs) aiming at a reduced carbon footprint. cloud resources makes use of the European funded BonFIRE Index Terms—Cloud computing, Monitoring, Data Mining, [4] project that was extended by the ECO2Clouds [5] project. Eco-efficiency, Carbon footprint ECO2Clouds adapts and extends the already provided monitoring capabilities by the providers EPCC, INRIA and I. INTRODUCTION HLRS with the aim to reduce the power consumption and improve the carbon footprint through monitoring and analyzing This work presents an enhanced monitoring approach required data. One major step of the overall approach is to considering the infrastructure of providers’ sites and the virtual elaborate the power consumption of VMs. machines (VMs) hosted on physical nodes. It considers the This paper is structured as follows: (1) Introduction, (2) GAMES [1] and the OPTIMIS [2] projects, that were both Monitoring and Metrics, (3) Data Mining Analysis and (4) developing methods for energy-aware resource usage. As the Conclusions. monitoring infrastructure (figure 1) is based on standardized software, customization is the goal for ensuring the relevant II. MONITORING AND METRICS values to reduce the power consumption and in addition, the The ECO2Clouds monitoring system is a layered approach eco-efficiency of the infrastructure providers and the overall distinguishing mainly between the physical infrastructure and carbon footprint. the virtualization layer. Additionally, applications can be monitored as well but as this information is highly dependent on the application, this paper won’t target the specific application monitoring. For those three layers, a Zabbix client agent is gathering monitoring information and sends it to the Zabbix server in order to provide a monitoring information repository to assist in optimization decision making. To enable power measurements for all three layers, power distribution units are attached to the physical servers. The monitored parameters are derived as defined by customized monitoring metrics based on standard programming languages like Python, Ruby or even Bash. A. The infrastructure and virtualization layer The infrastructure layer is represented by the physical infrastructure of a cloud provider, including physical nodes, storages or network devices. Thus, the developed monitoring Figure 1: The monitoring infrastructure approach needs to measure parameters of those physical components in order to enable a data analysis based on the The presented monitoring approach demonstrates a current state of the infrastructure for making assumptions monitoring system making use of a Zabbix [3] monitoring regarding the overall system behavior. Besides the Copyright © papers' authors. Copying permitted only for private and academic purposes. This volume is published and copyrighted by its editors. infrastructure layer, especially the virtualization layer is a As already stated, the applications inside a VM can also be crucial part as more or less the virtualization defines a monitored in particular. But due to the application character, completely separated environment for applications. Thus, it is the metrics are highly specialized and cannot be regarded in a essentially important to monitor those two layers in a fine generic fashion. grained manner. Furthermore, for providing accurate C. The monitoring metrics information for an optimized power consumption, both layers have to be linked together by the creation of calculated metric This subsection will describe some of the most important as presented in figure 2. metrics to improve the energy consumption as well as the carbon footprint for the infrastructure providers as well as for the users. For improving the carbon footprint of an infrastructure provider, metrics like “Carbon Usage Effectiveness of the Provider”, “Green Efficiency Coefficient of the Electrical Supplier” or the “Site Infrastructure Efficiency” are considered. They are dealing with the amount of energy consumed and in particular, on which basis the requested energy was generated. As presented in the Data Mining section, there are big differences in the amount of CO2 created to power the Fig. 2: The layered monitoring approach machines at different days or even times. For infrastructure It presents the structure of the layered approach by providers, the mandatory metrics are global: monitoring a presenting the virtual machines (VMs), the infrastructure and single host is not sufficient. In order to understand the the power distribution units (PDUs) for measuring the power infrastructure load, metrics like “Service Availability” as well consumption of physical nodes and showing the connection of as “Storage and CPU as well as Memory Utilization” are also those three via Zabbix to the three layers. To perform all of interest and have to be considered. necessary calculations being defined in the set of monitoring The power consumption of VMs depends on the size of a metrics the Zabbix server (infrastructure) and the aggregator VM defined through the used memory, the data I/O identified (Zabbix at VM level) need to be able to communicate to each through the send and receive activities, the disk activity other. identified through the read and write operations and the Coming back to the infrastructure layer, it refers to the consumed CPU seconds. Thus, the infrastructure and the characteristics of the sites and the resources available in each of virtualization layer are involved to derive the information for them. Thus a set of metrics was defined for measuring relevant this important metric (figure 3). infrastructure parameters and calculate them for enabling data analysis. B. The virtualization layer The virtualization layer contains the underlying hypervisor including the virtual machines (VMs) hosting the applications (or part of applications) that are used to manage the cloud infrastructure (e.g., the monitoring system) or that are offered to the users (e.g., user specific business applications). In order to create separated environments, the virtualization layer can be Fig. 3: Power consumption per VM freely configured: the amount of CPU, memory and disk space can be aligned to the used application hosted inside. Finally, the power information of a VM can be correlated Metrics at the virtualization layer aim to characterize the with the “Green Efficiency Coefficient of the Electrical VMs on which the applications are running. They are analyzed Supplier” resulting in additional metrics. But the results are to evaluate if the current deployment can be further improved impressive: the calculation of the virtual machine power and thus optimized. The analysis of the VM energy consumption is possible and even more, the produced carbon consumption aims to understand how the energy consumed by can be observed as well. Especially for distributed computing the host is distributed among the deployed VMs. Moreover, and data centers, there are differences for the location and new metrics were defined inspired by the data center metrics timeslot hosting the virtual instances. (infrastructure) proposed in the last years, especially by The Green Grid Consortium (GreenGrid). The idea is to redefine III. DATA MINING ANALYSIS the classical infrastructural metrics, like PUE and Data Centre The monitoring infrastructure described before is producing Energy Productivity (DCeP), at virtualization level to measure large amounts of data in a short period of time. This data is the impact of the application tasks in terms of energy being collected into Accounting DB by the Accounting consumption and carbon emissions. Service, which runs on a separate host. During our experiments, we have observed that the size of the collected Copyright © papers' authors. Copying permitted only for private and academic purposes. This volume is published and copyrighted by its editors. metrics data varies between 10 and 20 Mb per day. As the D. Data Analysis result, the fast-growing SQL database of the Accounting On the DM Storage side, the metrics data analysis is being Service host may shortly reach the storage capacity limits of performed. The aim of statistical data analysis is to find the Accounting node. This would trigger the automatic deletion correlation patterns, which may help to define the deployment of the old data by Zabbix. In order to avoid the loss of strategies depending, for instance, on the time of the day. historical raw data and to allow for its statistical analysis, we One example a pattern of the power consumption and have developed the ECO2Clouds Data Mining service (DM emissions depending on the time of the day is shown in Fig. 5. Service). Next, we present architecture, concepts and analysis 75# principles of the DM Service in more detail. 70# A. Architecture 65# DM Service consists of two major components (figure 4). 60# Week#day#average# The first component is running on the Accounting Service to 55# Weekend#day#average# perform transfer of non-reduced metrics data to a remote data 50# storage (DM Storage), and to generate a reduced data set. 45# The second component is DM Storage, which gathers the 40# non-reduced metrics data and performs statistical analysis over 1:0 # 2:0 # 3:0 # 4:0 # 5:0 # 6:0 # 7:0 # 8:0 # 9:0 # 10 0# 11 # 12 0# 13 0# 14 0# 15 0# 16 0# 17 0# 18 0# 19 0# 20 0# 21 0# 22 0# 23 0# # 0 0 0 0 0 0 0 0 0 :00 :00 0:0 :0 :0 :0 :0 :0 :0 :0 :0 :0 :0 :0 :0 them, e.g., correlation analysis over a large enough portion of data. The resulting stable parameters of this analysis are Fig. 5: Emissions during the week and weekend days [6] (see ECO2Clouds inserted into the Accounting DB as a separate table. project publications of WP3 for detailed analysis methods) Results transfer IV. CONCLUSIONS Transfer manager Data analysis service The presented monitoring architecture uses customized monitoring metrics and enables a data analysis providing result DM Storage Reduction service data regarding a cloud system behavior. The focus of the Accounting DB approach is the monitoring and analysis of data related to the Fig. 4: Data Mining Architecture used energy mix and consumed power on infrastructure and virtualization level. B. Concept Thus, it becomes possible to align the VM scheduling to the The Data Mining workflow consists of the following steps. workload of the physical infrastructure by considering the (1) Triggered daily, the Accounting Service inserts the new power consumption and the carbon footprint through knowing non-reduced data into the database of DM Storage. (2) about the energy mix. Herewith, costs for consumed power and Triggered daily, the Accounting Service performs the CO2 emissions are reduced for a cloud infrastructure. reduction of the metrics data; the results of data reduction are Future work will be using the presented monitoring inserted into a separate database on the Accounting VM. (3) infrastructure and the implemented metrics as basis for running The initial non-reduced data are deleted – thus, each time the experiments and supporting the deployment of use cases. Accounting Service has to perform operations over a metrics ACKNOWLEDGMENT data set which was produced only during the last 24 hours The resulting loss of data at the Accounting DB does not This work has been supported by the ECO2Clouds project affect the Scheduler functionality, because it does not use the (http://eco2clouds.eu/) and has been partly funded by the historical data for physical hosts, while for experiments we European Commission’s IST activity of the 7th Framework only reduce the data of finished (non-active) experiments only. Programme under contract number 318048. This paper On the DM Storage side, the statistical analysis of the data expresses the opinions of the authors and not necessarily those is performed. The results are inserted into Accounting DB. of the European Commission. The European Commission is not liable for any use that may be made of the information C. Implementation contained in this paper. On the Accounting VM, we install a bash script which performs the following operations: (1) Creates a temporary REFERENCES SQL-dump of the current non-reduced “e2c_collector” [1] M. Bertoncini, B. Pernici, I. Salomie, S. Wesner: GAMES: database. (2) Imports the generated SQL-dump into DM Green Active Management of Energy in IT Service Storage by inserting the rows incrementally to the previously Centres. CAiSE Forum 2010: 238-252. stored there data. (3) Removes the temporary SQL-dump from [2] G. Katsaros, J. Subirats, J. O. Fitó, J. Guitart, P. Gilet, D. the Accounting VM. (4) Calls the reduction service Espling, A service framework for energy-aware implemented as a Java program, which performs the data monitoring and VM management in Clouds, Future reduction and inserts its results into a separate database on the Generation Computer Systems, Vol. 29, 8, October 2013. same VM. [3] The Zabbix website documentation, https://www.zabbix.com, Last visited: 27.06.2014. Copyright © papers' authors. Copying permitted only for private and academic purposes. This volume is published and copyrighted by its editors. [4] The BonFIRE Project Website, http://www.bonfire- project.eu/, Last visited: 27.06.2014. [5] The ECO2Clouds Project Website, http://eco2clouds.eu/, Last visited: 27.06.2014. [6] C. Cappiello, P. Melià, B. Pernici, P. Plebani, M. Vitali. Sustainable choices for cloud applications: a focus on CO2 emissions. Proceedings of the ICT4S, 2014. Copyright © papers' authors. Copying permitted only for private and academic purposes. This volume is published and copyrighted by its editors.