=Paper=
{{Paper
|id=Vol-2815/CERC2020_paper24
|storemode=property
|title=Operational Management of Data Centers Energy Efficiency by Dynamic Optimization -Based on a Vector Autoregressive Model- Reinforcement Learning(VAR-RL) Approach
|pdfUrl=https://ceur-ws.org/Vol-2815/CERC2020_paper24.pdf
|volume=Vol-2815
|authors=Yang Hai,Truong Van Nguyen,Youngseok Choi,Mohammed Bahja,Habin Lee
|dblpUrl=https://dblp.org/rec/conf/cerc/HaiNCBL20
}}
==Operational Management of Data Centers Energy Efficiency by Dynamic Optimization -Based on a Vector Autoregressive Model- Reinforcement Learning(VAR-RL) Approach==
https://ceur-ws.org/Vol-2815/CERC2020_paper24.pdf
Business and Society
Operational Management of Data Centers Energy
Efficiency by dynamic optimization -Based on a Vector
Autoregressive Model- Reinforcement Learning(VAR-
RL) approach
Yang Hai1[0000-0003-4656-1594] ,Truong Van Nguyen2[0000-0001-9380-5710], Youngseok Choi3,
Mohammed Bahja4[0000-0002-2138-1784], and Habin Lee5[0000-0003-0071-4874]
1,2,3,5
Brunel University London, Middlesex UB8 3PH, UK
4 University of Birmingham, Edgbaston, Birmingham B15 2TT, UK
Yang.Hai@Brunel.ac.uk
TruongVan.Nguyen@Brunel.ac.uk
Youngseok.Choi@Brunel.ac.uk
M.Bahja@Bham.ac.uk
Habin.Lee@Brunel.ac.uk
Abstract. With the increasing demands of digital computing, Data Centers (DCs)
have become a leading scheme for global energy issues. Major efforts that can be
observed for DC energy efficacy solutions are focusing on relatively problematic
infrastructure designs. Nevertheless, we emphasised the managerial strategies of
using the existing facilities to achieve energy efficiency through active interven-
tion. It is believed that there exists a trade-off between the cooling devices and
IT devices. Accordingly, the Vector Autoregressive Model- Reinforcement
Learning(VAR-RL) approach will be proposed as a combination of traditional
multivariate time series modeling technique and the artificial intelligence tech-
nique which allows us to predict and adjust the prediction of an error would help
to explore the complex dynamic interrelationships between the two types of de-
vices. Moreover, an optimization decision support system will also be conducted
subsequently to optimize Power Usage Effectiveness (PUE) by controlling the
combination of Air Conditioners (ACs). The proposed VAR-RL approach would
not only increase the forecasting accuracy but also would adapt to the environ-
ment changes dynamically, this would give a better foundation for the DC energy
efficiency optimization. The data we adopted is the real-time data from a DC
located in Turkey. Consequently, the novel of this study would save the DC en-
ergy consumption tremendously.
Keywords: DC, Energy consumption, optimisation.
1 Introduction
DCs are energy-intensive industries and they are taking 1-1.5% of global electricity
usage every year [1]. There are two main units of DCs that are supplied the energy, IT,
Copyright © 2020 for this paper by its authors.
Use permitted under Creative Commons License 371 CERC 2020
Attribution 4.0 International (CC BY 4.0).
Business and Society
2
and cooling units. Widely cited studies and conservatively evaluation demonstrated the
fact that there are around 40% of energy has been taken by the cooling system in a
typical air-cooling DC [2]. IT equipment, along with its respective power supply, cre-
ates heat when being in use and will raise the surrounding ambient temperature. In con-
sequence, the equipment is likely to fail if the temperature becomes too high. Previous
to 2004, IT companies were too fixated upon the performance of their equipment and
adjusted the equipment’s environment with only reliability in mind, with little
weighting to energy costs. Nowadays the temperature range has been wider to 18 −
27℃ (ASHRAE 2016)[3]. However, there are over 90% of DCs still keeping a constant
temperature which means that the DCs are over-cooled and energy inefficiency [4].
Since most of the energy consumption lies in cooling, efforts have been made on
reducing the cooling energy consumption in the DCs. It has been found that configura-
tion design predominantly affects on energy consumption. As long as there is a design
revolution, the application still requires a long time. As a result, traditional air-cooling
will still domain the DC cooling system in the next few decades [5]. Therefore, we are
seeking DC cooling efficiency solutions from a different aspect. As we mentioned ear-
lier, cooling devices are the biggest energy consumers in the DC however there is no
guideline on how to make the optimal usage of them. The common operations in DCs
are still following traditional rules by turning off some certain number of ACs to save
energy in winter. But to our knowledge, there is no sophisticated analysis so far to guide
the optimal use of ACs combination, which gives us ideas to fill up this gap.
According to the relationship of energy consumption units both IT and Air condi-
tioning devices, a lower temperature will increase the energy supplied, inversely, will
reduce the energy consumption of IT devices due to an increasing computing effi-
ciency. Therefore, smart operation management on DCs to find out the optimal solution
on temperature control to minimize the energy consumption without affecting the per-
formance of IT devices and meeting the service-level agreement has been investigated.
Therefore, this has become our motivation for this study. The question can be modeled
as a Linear Programming (LP) problem. However, applying the LP method to the prob-
lem requires understanding the complex interactions among many variables within the
DC. To solve and simplify this issue, we adopt VAR model to identify such complex
interactions. It has been taken to account for common features of the industry big data
Changes rapidly in the structures. Changes in server workload, outside environment,
device locations or human intervention all can be reasons that lead to structural breaks
of the series. Numerous empirical studies put attention on post-event detection, which
wildly used for economic or business analysis, while rarely of them are looking at this
issue in a real-time. Instead, we expect the model would able to autonomous evaluate
itself and correct the mistake once it notices it. In this study, we adopt RL approach for
dynamic real-time adjustment of VAR model. It would take the responsibility to detect
the structural break and trigger the parameter re-estimated system. With the proposed
VAR-RL approach, the subsequent optimization problem can be solved with the con-
sideration of the changes in the environment in real-time.
Our contributions to the literature including the following: (1) Give the DC energy-
efficient solution without changing DC configurations. (2) A dynamic simulation sys-
CERC 2020 372
Business and Society
3
tem based on VAR-RL approach has been made, this will provide an efficient and ac-
curate forecast for the complex environment with the adjustment of structural changes
in the data. (3) Real-time optimization will be conducted based on the simulation result
and future optimization also can be made by the forecasted data set. (4) This study will
also arouse the environmental awareness of energy saving.
2 Literatures review
We have searched the empirical studies on DC effective cooling strategy of simulation-
based optimization. Among the ten results, eight of them adopted the Computational
Fluid Dynamic (CFD) while and the rest of them used Data-driven Models (DDM)
model and other configuration design simulation tools for airflow simulation.
Most of them are from a pure configuration design and layout aspect, ie. [6] on sen-
sors placement strategy and [8] on air aisle and racks layouts [7]. There are only [8, 9,
10] among the results that looking at the optimal temperature solutions, however [9] it
doesn’t take the trade-off relationships between cooling and IT into considerations and
[8] is an equation-based simulation that looking at system network control. Also [10]
studied the combination of water and airflow in Indirect Adiabatic Cooling (IAC) DC.
Numerous studies that use the First principle (FP) in terms of DC objectives largely
rely on pre-defined algorithms. However, in practical, there are a variety of unknown
relationships that cannot be acquired from physical principles. Data-Driven Models
(DDMs) avoid this problem by adopting experimental data to train a system. There is
study compares temperature prediction performance of four different types of DDMs
including Artificial Neural Networks (ANN), Support Vector Regression (SVR),
Gaussian Process Regression (GPR) as well as Proper Orthogonal Decomposition
(POD) in a DC, the training data is given by CFD simulation and the result demon-
strated that most of them can give a relatively accurate prediction however only ANN
could handle multiple output points in one model. Because of the unknown features of
the system and multi-dimensional problem need to be solved in one model, so that it
requires a large volume of data to feed in the model and moreover, all these types of
models are facing similar difficulties which are computational expensive practical cases
and relatively time-consuming [11].
We conservatively conclude that our VAR-RL approach would be the first study that
further extended Linear Regression (LR) based RL to analyze complex industrial envi-
ronment, then apply the simulation result to real-time optimization in industrial practi-
cal case. Due to the limited resources, we reviewed similar studies that used the similar
method for different problems. RL approach has been used for an auto-select different
combination of data streams to feed to the parameters-fixed LRs and practical applica-
tion on typhoon rainfall prediction shows a better performance than traditional LRs
[12]. More comprehensively, a Multi-Agent reinforcement learning (P-MARL) on pre-
dicting the future environment which allows the agents to adapt to the changes off-line
by the combination of ANN and Autoregressive Integrated Moving Average (ARIMA)
models, this joint approach also increased the prediction accuracy of the agents [13].
Moreover, efforts have been made on adapting RL to the side of the sensor to reduce
373 CERC 2020
Business and Society
4
the energy transmission cost in the wireless sensor networks for signal prediction [14].
These approaches provided evidences that with RL adjustment, the prediction accuracy
would be largely increased and gives the model more flexibility by self-learning during
prediction without any training data which would require large storage and costs for
computing.
Our VAR-RL approach would take advantage of these empirical studies, moreover,
we will adapt this into DC industrial practice: (1) Our model parameters will be dynam-
ically changed according to the feedback from the learning process that would allow
our model to adapt to the environment changes. (2) We will propose a time-series linear
regression model that will not only consider the previous one step (MACOV) but the
whole period which has an effect on the present. (3) RL tool will be plugin to determine
whether the model should be reused or rebuilt. (4) We will also avoid using the tech-
nique which has a black-box property such as Artificial Neuron Network (ANN) be-
cause it hides the interrelationships into the black-box procedure which limited the in-
terpretative of the model. (4) Also, as we expect to perform a light, fast, and efficient
model to adapt to the rapid industrial practice. we will avoid the use of techniques that
requires huge size of training data and local storage. (5) We will not change any DC
configurations including the sensors, only managerial strategies will be applied to the
DC energy-saving practice.
3 Methodology.
3.1 VAR model- A fundamental simulator
Based on the field study in TUKSAT DC target IT room, we identified the main factors
that participated in the IT room computing environment. We are going to include ceil-
ing sensors temperatures, Server rack inlet and outlet temperatures, air conditioner out-
flow temperatures as well as PDU values of the servers as our endogenous variables.
To our knowledge, IT room objectives are mutually affected by each. As the graph
shows below, we can infer that each variable can affect the others in two directions
(direct or indirect), shown as a circulation (Fig.1). Statistical test (Granger causality
test) results also confirmed the underlined inferences. Therefore, we briefly include all
the related variables into one VAR model.
Fig. 1. The DC variables and relationships
Here we present how VAR modelling the above dynamic. VAR assumes all the var-
iables to be endogenous and explain those endogenous variables one by one by all their
CERC 2020 374
Business and Society
5
past values. This allows us to use the estimated model to predict the future values of
variables. A 𝑝𝑡ℎ order 𝑉𝐴𝑅(𝑝) can be represented as:
𝑋𝑡 = ∑𝑘𝑖=1 П𝑖 𝑋𝑡−𝑖 + 𝐶 + 𝑢𝑡 (1)
Assume we have 𝑁 variables, then 𝑋𝑡 is the 𝑁 × 1 order time series vector, 𝐶 is the
𝑁 × 1 order constant vector, the П𝑖 is the 𝑁 × 𝑁 order parameter matrix, 𝑢𝑡 is the
𝑁 × 1 order random error vector. We can extend the above equation to the matrix for-
mula as following (The lag length will be selected by the combination of "AIC", "HQ",
"SC", "FPE" criteria):
(2)
As VAR model is a dynamic forecasting model. We can use it to simulate the DC
environment as well as forecast the future values of each variable. After we get the
model parameters by real-time estimation, we will feed the data that cover the lag length
and forecast the future value of each variable. To make it clear, here we summarize the
procedure to train a VAR model and use it as a simulator to forecast DC environment
in a flowchart (See Fig.2).
Fig. 2. The structure of the simulator
We first downloaded data from the historical Application Programming Interface
(API) (Step 1). After data processing (Step 2), we use this data set to train a VAR model
(Step 3). Then download historical data again (Step 4), after processing the data for
another time (Step 5), we extract the most recent data which cover our lags, then input
to the VAR model. The simulator will carry out iterations until reach to the requested
forecasting length.
375 CERC 2020
Business and Society
6
3.2 RL -A dynamic environment adaptor
As we mentioned earlier, the data estimated in DC has a rapidly changing feature, there-
fore our fundamental simulator VAR may not apply for some special cases: ie., sud-
denly changing load by holidays or online exams, temperatures changes, or other hu-
man interventions. Therefore, we need to adjust our model to ensure the prediction ac-
curacy and be able to detect the environment changes and adapt itself to the changing
world. Reinforcement learning as an environment adaptor to VAR model will be intro-
duced in this section. The process is shown in the following graph (Fig.3).
Fig. 3. The environment adaptor based on RL
With every prediction, we will have an evaluation of the accuracy. And we will give a
reward (or punishment) to each prediction. The accumulated reward would be:
𝑅𝑐𝑢𝑚 = ∑𝑛𝑔=1 𝑅𝑡+𝑔 (3)
Where 𝑅𝑐𝑢𝑚 is the total cumulated reward values, 𝑅 is the reward for each forecast eval-
uation. With the number of time steps increasing, the difficulty level to predict would
be increasing too, to make it fair enough for the judgement, we assign the weight to
each reward, and the weight of the reward would be decreasing over the time.
𝑅𝑐𝑢𝑚 = ∑∞ 𝑘
𝑘=0 𝛾 𝑅𝑡+𝑘+1 , 𝛾 ∈ [0,1) (4)
Where, 𝛾 is the weight.
Time windows will be plugin at every seasonality changing point. And the system will
trigger RL to evaluate the prediction result. In this case, the prediction result from the
fundamental simulator VAR will be evaluated by the error rate. A reward will be given
to each evaluation. When the accumulated reward value reaches to a certain boundary,
the environment changes will be detected. Then the RL adaptor will trigger the alarm
then the VAR model parameters and features will be rebuilt.
3.3 LP approach- An energy efficiency optimizer
Empirical evidences show that increasing the AC setpoint by a single degree can result
in 4-5% energy cost savings; and increasing the setpoint by 10 degrees, which is also a
realistic number, can result in savings of over 40% [15]. Although this sounds straight-
forward and simple, considering the complex nature of DC assets, it is hardly the case.
Increasing the AC setpoints blindly can jeopardize the health of servers and other hard-
ware, as existing hot spots may become even hotter and higher hot aisle temperature
CERC 2020 376
Business and Society
7
may activate server fans and offset efficiency gains. Therefore, a rigorous plan for op-
timizing the AC temperature setpoint is critical to increasing the energy efficiency of
the DC. Particularly, we aim to optimize the energy efficiency in the DC by determining
the optimal combination of the supplied temperature of AC units, while taking into
consideration the dynamic nature of IT power consumption, as well as satisfying the
temperature constraints.
We will use the following notations:
𝑆 = {𝑆1 , 𝑆2 , … , 𝑆𝑚 } denotes the Server rack number 1 to 𝑚
𝐶 = {𝐶1 , 𝐶2 , … , 𝐶𝑙 } is a set of ACs unit number 1 to 𝑙.
𝑠𝑢𝑝
𝑇𝑥 (𝑡): the temperature supply of the 𝑥 𝑡ℎ AC unit at time 𝑡.
𝑇𝑗𝑖𝑛 (𝑡): the inlet temperature of the 𝑗𝑡ℎ server rack at time 𝑡.
𝑇𝑗𝑜𝑢𝑡 (𝑡): the outlet temperature of the 𝑗𝑡ℎ server rack at time 𝑡.
𝑇 𝑟𝑜𝑜𝑚 (𝑡): the room temperature at time 𝑡.
𝐶𝑜𝑚𝑝
𝑃𝑗 (𝑡): the computational power (PDU) for the 𝑗𝑡ℎ server rack at time 𝑡 .
𝑃𝑥𝐶𝑜𝑜𝑙 (𝑡): the computational power for the 𝑥 𝑡ℎ AC unit at time 𝑡.
𝐸[𝑊𝑗 (𝑡)]: the estimated workload (CPU usage) for the 𝑗𝑡ℎ server rack at time 𝑡 .
𝐶𝑜𝑃𝑥 : the coefficient of performance.
𝐶𝑇𝐼𝑗𝑥 is the thermal correlation index.
The IT consumption
. Assumingly, the IT power consumption would not only be influenced by the compu-
tational workload, but also will be affected by the working temperature because the
temperature will affect its working performance. Hence, at any time 𝑡, the total IT com-
puting power of server rack 𝑆𝑗 is the function of power spent on executing IT jobs and
the rack inlet temperature.
𝐶𝑜𝑚𝑝
𝑃𝑗 (𝑡) = 𝛼𝑗 𝐸[𝑊𝑗 (𝑡)] + 𝛽𝑗 𝑇𝑗𝑖𝑛 (𝑡) (5)
Where 𝛼𝑗 and 𝛽𝑗 are weight coefficients.
The cooling consumption
. Based on [16,17,18], the cooling cost of AC device 𝐶𝑥 ∈ 𝐶 can be presented as:
𝐶𝑜𝑚𝑝
𝐶𝑇𝐼𝑥𝑗 ∑𝑚
𝑗=1 𝑃𝑗 (𝑡)
𝑃𝑥𝐶𝑜𝑜𝑙 (𝑡) = 𝑠𝑢𝑝 (6)
𝐶𝑜𝑃𝑥 (𝑇𝑥 (𝑡))
Where 𝐶𝑜𝑃𝑥 is the performance coefficient, shown as the ratio of the amount of heat
the AC device 𝐶𝑥 needs to remove to the energy it needs to consume to perform the
removal. 𝐶𝑜𝑃𝑥 indicates the efficiency of the AC device, and is typically a non-linear,
𝑠𝑢𝑝
increasing function of the supplied cold air temperature, 𝑇𝑥 (𝑡). It means that operat-
ing the AC system at a higher temperature is saving energy, as providing colder air
requires the AC to work harder and consume more energy to remove heat. Hence, we
can minimise 𝑃𝑥𝐶𝑜𝑜𝑙 (𝑡) by maximise the allowable supplied cold air temperature,
𝑠𝑢𝑝
𝑇𝑥 (𝑡) that satisfies the constraint of redline thresholds. The simulation approach will
377 CERC 2020
Business and Society
8
𝑠𝑢𝑝
also be used to get the function of 𝐶𝑜𝑃𝑥 (𝑇𝑥 (𝑡)). 𝐶𝑇𝐼𝑗𝑥 is the thermal correlation in-
∆𝑇𝑗𝑖𝑛
dex, 𝐶𝑇𝐼𝑗𝑥 = 𝑠𝑢𝑝 , which represents the influence of each AC unit 𝐶𝑥 on inlet temper-
∆𝑇𝑥
ature of server rack 𝑆𝑗 . As defined in Eq. (6), it quantifies the response of the server
𝑠𝑢𝑝
𝑆𝑗 ’s inlet temperature 𝑇𝑗𝑖𝑛 to a step-change in the supply temperature 𝑇𝑥 of 𝐶𝑥 .
𝐶𝑇𝐼𝑗𝑥 is a static metric, which is stable with time but based on the physical configura-
tion of the DC. Hence, we use the simulation approach to get the value of 𝐶𝑇𝐼𝑗𝑥 . The
detailed explanation of this metric can be seen in [17,18].
Thermal modelling
. According to the law of energy conservation, almost all the computing power con-
sumed by a server is transformed into heat, hence the relationship between the power
consumption and inlet/outlet temperature of server rack 𝑆𝑗 can be presented as:
𝐶𝑜𝑚𝑝
𝑇𝑗𝑜𝑢𝑡 (𝑡) = 𝑇𝑗𝑖𝑛 (𝑡) + 𝐾𝑗 𝑃𝑗 (𝑡)
(7)
Where 𝐾𝑗 = 𝑝𝑓𝑗 𝑐 is the thermal-physical term. This can be estimated by our data ob-
tained in DC.
Typically, the server’s inlet temperature (𝑇𝑗𝑖𝑛 ) tends to be higher than the AC’s sup-
𝑠𝑢𝑝
plied air temperature (𝑇𝑥 ) due to the phenomenon so-called heat recirculation where
the hot air from the server and the supplied cool air from the AC are mixed then recir-
culats in the room. Based on the energy conservation as described in Eq. (8) and the
assumption of the fixed airflow pattern in the computer room, prior studies (eg. [19])
characterise this phenomenon with a heat distribution matrix 𝐴 = {𝑎𝑗𝑜 } where 𝑎𝑗𝑜 is
the temperature increase at the inlet of server rack 𝑀𝑗 due to the heat emitted at the
outlet of the server rack 𝑀𝑜 . Here we adjust this to matrix 𝐴 = {𝑎𝑗 } denotes the heat
increased at the inlet of server rack 𝑆𝑗 caused by the heat recirculated inside the rack
due to computation. Hence, the inlet temperature of a server rack 𝑆𝑗 comes from the
combination of the supplied cold air from the AC and hot air recirculated inside the
rack. This relationship can be written as:
𝑠𝑢𝑝
𝑇𝑗𝑖𝑛 (𝑡) = ∑𝑙𝑥=1 𝑐𝑗𝑥 𝐶𝑇𝐼𝑗𝑥 𝑇𝑥 (𝑡) + 𝑑𝑗 𝑃𝑗𝐶𝑜𝑚𝑝 (𝑡) (8)
Where 𝑐𝑗𝑥 is a binary variable which equals to 1 if the AC unit 𝐶𝑥 is assigned to supply
cold air to the rack slot of server rack 𝑆𝑗 , and 0 otherwise. As the DC layout is fixed,
the value of 𝑐𝑗𝑥 will be given by VAR estimation.
𝑠𝑢𝑝
With equations (5) and (8), we can transfer 𝑇𝑗𝑖𝑛 (𝑡) to the function of 𝑇𝑥 (𝑡).
𝑠𝑢𝑝
∑𝑙𝑥=1 𝑐𝑗𝑥 𝐶𝑇𝐼𝑗𝑥 𝑇𝑥 (𝑡)+𝛼𝑗 𝐸[𝑊𝑗 (𝑡)]
𝑇𝑗𝑖𝑛 (𝑡) = (9)
1−𝛽𝑗
Optimization solution
CERC 2020 378
Business and Society
9
. PUE value is a measurement of the power utilization efficiency of DCs that is
adopted internationally. It is the ratio of total power consumed by the DCs to the power
consumed by the IT load.
𝑃𝑡𝑜𝑡𝑎𝑙 (𝑡) 𝑥 𝑃𝐶𝑜𝑜𝑙 (𝑡)
𝑃𝑈𝐸 = 𝐶𝑜𝑚𝑝 = 1 + 𝐶𝑜𝑚𝑝 (10)
𝑃𝑗 (𝑡) 𝑃𝑗 (𝑡)
The closer the PUE value is to 1, the higher the greenness of a DC.
𝑃𝑥𝐶𝑜𝑜𝑙 (𝑡)
Let 𝐺 ∗ = 𝐶𝑜𝑚𝑝 (11)
𝑃𝑗 (𝑡)
Then to optimize 𝑃𝑈𝐸 is equal to the question to minimize 𝐺 ∗ .
Let denote [𝑡1 , 𝑡2 ] be the interval of interest. Based on the Eq. (5, 6) above, our objec-
tive function can be written as:
𝐶𝑜𝑚𝑝
𝐶𝑇𝐼𝑥𝑗 ∑𝑚
𝑗=1 𝑃𝑗
(𝑡)
∑𝑙𝑥=1(∑𝑚
𝑗=1( 𝑠𝑢𝑝 ))
𝑃𝐶𝑜𝑜𝑙 𝑡 ∑𝑙 𝑃𝐶𝑜𝑜𝑙 (𝑡) 𝑡 𝐶𝑜𝑃𝑥 (𝑇𝑥 (𝑡))
𝑀𝑖𝑛 𝐺 ∗ = 2 𝑥=1 𝑥
𝐶𝑜𝑚𝑝 = ∫𝑡 ( 𝑚 𝐶𝑜𝑚𝑝 ) 𝑑𝑡 = ∫𝑡 2 𝑑𝑡 =
𝑃 1 ∑𝑗=1 𝑃𝑗 (𝑡) 1 ∑𝑚 𝑖𝑛
𝑗=1(𝛼𝑗 𝐸[𝑊𝑗 (𝑡)]+𝛽𝑗 𝑇𝑗 (𝑡))
( )
𝐶𝑇𝐼𝑥𝑗 (∑𝑚 𝑖𝑛
𝑖=1(𝛼𝑗 𝐸[𝑊𝑗 (𝑡)]+𝛽𝑗 𝑇𝑗 (𝑡))
∑𝑙𝑥=1(∑𝑚
𝑗=1 ( 𝑠𝑢𝑝 ))
𝑡2 𝐶𝑜𝑃𝑥 (𝑇𝑥 (𝑡))
∫𝑡 𝑑𝑡 (12)
1 ∑𝑚 𝑖𝑛
𝑗=1(𝛼𝑗 𝐸[𝑊𝑗 (𝑡)]+𝛽𝑗 𝑇𝑗 (𝑡))
( )
Align Eq.(12) with Eq.(9), we can transfer the objective function to the function of the
𝑠𝑢𝑝
cooling temperature combinations 𝑇𝑥 (𝑡). Hence, to minimise 𝐺 ∗ , we can optimise
𝑠𝑢𝑝
𝑇𝑥 (𝑡), which is also the decision variable in this model.
Nevertheless, the adjustment of the supplied cold air temperature is subject to the con-
straint that the inlet temperatures of all server racks are below the redline temperature
threshold specified by the device manufacturers (i.e. typically below 25oC). Hence,
based on Eq. (8)(5), the constraint of redline threshold (𝑇 𝑟𝑒𝑑1 ) can be presented as:
𝑠𝑢𝑝 𝐶𝑜𝑚𝑝
𝑇𝑗𝑖𝑛 (𝑡) = ∑𝑙𝑥=1 𝑐𝑗𝑥 𝐶𝑇𝐼𝑗𝑥 𝑇𝑥 (𝑡) + 𝑑𝑗 𝑃𝑗 (𝑡) = ∑𝑙𝑥=1 𝑐𝑗𝑥 𝐶𝑇𝐼𝑗𝑥 𝑇𝑥𝑠𝑢𝑝 (𝑡) +
𝑑𝑗 . (𝛼𝑗 𝐸[𝑊𝑗 (𝑡)] + 𝛽𝑗 𝑇𝑗𝑖𝑛 (𝑡) ≤ 𝑇 𝑟𝑒𝑑1 (13)
Also, we will restrict the room temperature to be within the allowance (18~27 oC ac-
cording to AHREA 2016). As we define the room temperature as the function of ACs
temperatures and the Server racks outlet temperatures, based on Eq.(7)(5), the temper-
ature with thresholds (𝑇 𝑟𝑒𝑑2 ) and (𝑇 𝑟𝑒𝑑3 ) will be represented as:
𝑠𝑢𝑝
𝑇 𝑟𝑒𝑑2 ≤ 𝑇 𝑟𝑜𝑜𝑚 (𝑡) = ∑𝑙𝑥=1 ℎ𝑥 𝑇𝑥 (𝑡) + ∑𝑚 𝑗=1 𝑔𝑗 𝑇𝑗
𝑜𝑢𝑡
(𝑡) = ∑𝑙𝑥=1 ℎ𝑥 𝑇𝑥𝑠𝑢𝑝 (𝑡) +
𝐶𝑜𝑚𝑝
∑𝑚 𝑖𝑛
𝑗=1 𝑔𝑗 (𝑇𝑗 (𝑡) + 𝐾𝑗 𝑃𝑗 (𝑡)) = ∑𝑙𝑥=1 ℎ𝑥 𝑇𝑥𝑠𝑢𝑝 (𝑡) + ∑𝑚 𝑖𝑛
𝑗=1 𝑔𝑗 (𝑇𝑗 (𝑡) +
379 CERC 2020
Business and Society
10
𝑠𝑢𝑝
𝐾𝑗 (𝛼𝑗 𝐸[𝑊𝑗 (𝑡)] + 𝛽𝑗 𝑇𝑗𝑖𝑛 (𝑡)) = ∑𝑙𝑥=1 ℎ𝑥 𝑇𝑥 (𝑡) + ∑𝑚 𝑖𝑛
𝑗=1((𝑔𝑗 + 𝑔𝑗 𝛽𝑗 )𝑇𝑗 (𝑡) +
𝑔𝑗 𝐾𝑗 𝛼𝑗 𝐸[𝑊𝑗 (𝑡)]) ≤ 𝑇 𝑟𝑒𝑑3 (14)
Moreover, for each AC, there are thresholds (𝑇 𝑟𝑒𝑑4 ) and (𝑇 𝑟𝑒𝑑5 ), which will restrict
the AC temperatures to be within the range (0 − 30℃).
𝑠𝑢𝑝
𝑇 𝑟𝑒𝑑4 ≤ 𝑇𝑥 (𝑡) ≤ 𝑇 𝑟𝑒𝑑5 (15)
Similarly, by aligning with Eq. (9), we can transfer the constraints functions Eq. (13,14)
𝑠𝑢𝑝
to the function of 𝑇𝑥 (𝑡).
In short, the operational problem that we will address at the first stage is to optimize
the objective function in Eq. (12) by determining the optimal supplied cold temperature
of the AC devices, given the constraints in Eq. (13, 14, 15). The optimization would
start once we estimate the DC may perform inefficiently by VAR-RL forecasting, and
target temperature combination will be set at 𝑡1 − 𝛥𝑡 periods on the timeline (Where
𝛥𝑡. is also defined by VAR-RL).
4 Future Studies
Future studies will be made on forecasting verification and model adjustment. An ap-
plication UI (User Interface) will be applied for the DC managers to make sustainable
DC management. Field trial studies will also be conducted subsequently, we will mod-
ify our models and further studies will be done accordingly.
References
1. A. Andrae and T. Edler: On Global Electricity Usage of Communication Technology:
Trends to 2030. Challenges, 6(1). 117–157, (2015).
2. A. Capozzoli and G. Primiceri: Cooling systems in data centers: state of art and emerging
technologies. Energy Procedia, 83, 484–493, (2015).
3. ASHRAE: Data Center Power Equipment Thermal Guidelines and Best Practices. pp.1–60,
(2016).
4. M.K. Stansberry: JulianUptime Institute 2012. Data Center Industry Survey, Uptime Insti-
tute (2013).
5. Lu, H., Zhang, Z., Yang, L: A review on airflow distribution and management in data center.
Energy & Buildings, 179, 264–277, (2018).
6. Wang, X., Xing, G., Chen, J., Lin, C. X., Chen, Y. Intelligent sensor placement for hot server
detection in data centers. IEEE Transactions on Parallel and Distributed Systems, 24(8),
1577–1588, (2013).
7. Volk, E., Rathgeb, D., Oleksiak, A., Volk, E., Rathgeb, D, Rathgeb, D., Oleksiak, A.: CoolE-
mAll-optimising cooling efficiency in data centres. Compute Sci Res Dev, 29, 253–26,
(2014).
CERC 2020 380
Business and Society
11
8. Fu, Y., Zuo, W., Wetter, M., VanGilder, J. W., Han, X., Plamondon, D.: Equation-Based
object-oriented modeling and simulation for data center Cooling: A case study. Energy and
Buildings. 186, 108–125, (2019).
9. Song, M., Chen, K., & Wang, J.: Numerical Study on the Optimized Control of CRACs in
a Data Center Based on a Fast Temperature-Predicting Model. Journal of Energy Engineer-
ing, 143(5), 1–8, (2017).
10. A. Beghi, L. Cecchinato, G. Dalla, and M. Lionello: Modelling and control of a free cooling
system for Data Centers. Energy Procedia, vol. 140, 447–457, (2017).
11. J. Athavale, M. Yoda, and Y. Joshi: Comparison of data driven modeling approaches for
temperature prediction in data centers. Int. J. Heat Mass Transf., vol. 135, 1039–1052,
(2019).
12. S. Y. Lin: Reinforcement learning-based prediction approach for distributed Dynamic Data-
Driven Application Systems.Inf. Technol. Manag., 16(4). 313–326, (2015).
13. A. Marinescu, I. Dusparic, and S. Clarke: Prediction-based multi-agent reinforcement learn-
ing in inherently non-stationary environments, ACM Trans. Auton. Adapt. Syst., 12(2),
2017.
14. H. Nazaktabar, K. Badie, and M. N. Ahmadabadi: RLSP: a signal prediction algorithm for
energy conservation in wireless sensor networks. Wireless Networks, 23(3), 919–933, 2017.
15. DCR Homepage, https://datacenterresources.com/articles/increasing-crac-set-points/. last
accessed 2020/07/11
16. Q. Tang, S. K. S. Gupta, and G. Varsamopoulos: Energy-efficient thermal-aware task sched-
uling for homogeneous high-performance computing data centers: A cyber-physical ap-
proach. IEEE Trans. Parallel Distrib. Syst., 19 (11),1458–1472, (2008).
17. R. Zhou and Z. Wang: Modeling and control for cooling management of data centers with
hot aisle containment. HP Lab. Tech. Rep., 82, (2011).
18. A. Beghi, L. Cecchinato, G. Dalla, and M. Lionello: Modelling and control of a free cooling
system for Data Centers. Energy Procedia, 140, pp.447–457, (2017).
19. H. Sun, P. Stolf, J. M. Pierson, and G. Da Costa: Energy-efficient and thermal-aware re-
source management for heterogeneous datacenters. Sustain. Comput. Informatics Syst., 4(4),
292–306, 2014.
381 CERC 2020