University ITMO (ifmo.ru) is developing the cloud of geographically distributed data centers under OpenStack. The article describes the deployment cycle for the developed multi-agent system on physical and virtual machines using the developed software product repository. The cloud system of distributed data centers assumes the existence of a number of subsystems such as independent agents. An agent is aimed at implementing the creation, management and provision of certain services with a defined SLA. At the same time, it is necessary to perform both the initial deployment of the system and to provide the means that perform the automated modification of the hardware or software infrastructure, that is, the complete agent life cycle. The approaches suggested in the article allow us to use the minimum number of pre-installed components for the purpose of deploying and managing an array of physical and virtual entities of data centers, including deployment and configuring using Salt and monitoring the infrastructure using Zabbix.
Cloud computing paradigm has drastically changed the way in which we consume resources. Approaches to pay for the use and standardization of resources through virtualization lead to the need to use automation tools to reduce the cost of performing routine operations. At the same time, there is the task of deploying and monitoring a set of standardized entities with different parameters of service quality for a variety of private and public cloud clients.
In the previous paper [
The article is structured as follows: Section 2 describes goals and objectives of automated service deployment in data centers; Section 3 describes proposed approaches for automated deployment of services; Section 4 describes monitoring approaches of deployed services.
Automation of deployment follows the formal definition of the sequence of actions required to
configure and run virtual entities. At the same time, the input parameters are not only the available
cloud hardware resources, but also the requirements for the quality of customer service. Deployment
can be initiated by the virtualization management service upon request from the client interaction
system [
Main benefits of automated deployment are: 1. Repeatability – every deployment deploys the same with various parameters in configuration; 2. Fewer errors – removing manual steps and passing the testing procedure reduces error; 3. Work on what matters – developers can spend time working on new features, not on fixes for manual deployments; 4. Lower costs – fewer errors, fewer human hours needed for deployment.
The deployment of cloud infrastructure and services requires a number of preparatory procedures. Therefore, a series of operations are performed to ensure the possibility of using automation: 1. Preparing repository accessibility; 2. Prepare DNS and DHCP; 3. Preparing of system images using separate repositories for different types of services such as storage, computing, networking, agents; 4. Preparing of monitoring and DB facilities and messages queue service;
The further deployment process is aimed at using information from this list of services in order to determine whether it belongs to a particular service variant and initiate the launch of the Salt deployment scripts.
In order to use a VM for further deployment of virtual entities, it is necessary to create basic images. These images are used to create VMs running in OpenStack, by obtaining the necessary configuration of hardware resources, network environment and domain names. The preparation process includes the following steps: 1. Use separate project repositories for different types of services; 2. Deployment of base system (Naulinux distribution based on Scientific Linux); 3. Automating the installation of packages after deployment for agents using custom kickstart; 4. Installing Salt Minion packages and define required naming and address of Salt
Master.
Using these prepared images when deploying OpenStack virtual machines, you can verify that the installed packages from a specific repository are present, the correct configuration of the Salt system, and proceed to further configuration.
The project assumes the use of CEPH clusters as a storage system and provides users with
access to block devices, or to the file system using NextCloud [
A feature of our approach is the deployment of CEPH clusters for each client, which provides increased reliability of the storage. In this case, the virtual machines for cluster deployment and configuration are created using the OpenStack platform. The dedicated CEPH cluster connected to the Cinder OpenStack module is used as a backend for block devices of virtual machines. That is, block devices used as object storage devices (OSDs) for creating client storage are created in the CEPH cluster, which has its own replication. Figure 1 represent deployment procedure. Storage pool in the picture is the dedicated CEPH cluster; memory and CPU pools are provided by OpenStack platform for VM creation. The approach of using CEPH in CEPH is aimed at solving the following tasks: 1. Providing increased reliability for user data due to an additional level of data replication; 2. Creating clusters of various sizes using block devices of a smaller size than the capacity of physical disks; 3. Improving the system’s flexibility to change by using the level of abstraction of network interaction and hardware resources of the OpenStack platform.
A number of measurements aimed at assessing the performance of deployed clusters: we
measure the speed of sequential and random data reading and writing using the rados bench utility [
So, a certain degree of degradation of the data exchange rate due to the use of nesting is visible, but this decrease in performance is within, for example, the percentage of losses for virtualization (10-15%).
Virtual objects created during the automated deployment process need to be monitored for checking resource availability, utilization and for billing. Salt Stack is used to connect the created objects to the monitoring subsystem. With creation of virtual objects, such as CEPH cluster, or user`s gateway for data with Nextcloud instance, automated procedure add that entity into Zabbix.
Another task is to solve the problems that arise in the system. In general, the system may fail one of the servers and you need to determine which of the virtual objects were running on it before the error message appears. Modules have been developed that search for such information and assess the degradation of the system as a whole.
In the article, approaches of preparation and automated deployment are described. We use described methods in our project that aims on development management systems of geographically distributed Data Centers. The project is in active debugging stage now. The system is planned to be in preproduction stage first half of 2019.
The research has been carried out with the financial support of the Ministry of Education and Science of the Russian Federation under grant agreement № 03.G25.31.0229 “Development of new technological components for management systems of geographically distributed Data Centers”.