University ITMO (ifmo.ru) is developing the cloud of geographically distributed data centers under Openstack. The term “geographically distributed” in our proposal means data centers (DC) located in different places far from each other at least by many tens of kilometers. Authors follow the conception of “dark” DC, i.e. the DC has to perform normal operation without permanent maintainers even with minor problems (single machine or a number of disk drives went down). In such “dark” DC the staff might visit DC if required to fix appeared problems. Authors describe thoughts and experiments with service reliability for cloud of DCs under Openstack.
The main cloud services to be provided in Data Centers (DCs) include Computing (VMs),
Storage, Data transfer channels. Here it is considered several globally dispersed DCs which are
functioning like united cloud of DCs. There are several advantages for such the architecture [
All DCs are running under Openstack (ocata) [
Obviously DCs need the data links in between each other. In the project it is distinguished
two types of virtual channels: regular channels (encrypted) for data transfer and control channels
(with OpenVPN). Also the cloud needs distributed storage which is implemented with CEPH [
Machine (VM) on any DC in the cloud; If DC went down where the user uses Openstack and user’s VM was running in the same DC, then user can log into another Openstack instance in different DC in the cloud and start new user’s VM instance. Presumably all intermediate results from broken DC are available in distributed cloud storage.
In the table 1 several options shown when one DC in the cloud went down. From the table can be seen that even with cloud consisting of two DCs the reliability to get the computing service for user is significantly higher to compare with one DC.
1 2 3 4
User was logged into Openstack instance in broken DC and user’s VM was running in the same DC.
User was logged into Openstack instance in broken DC but user’s VM is running on different DC.
User was logged into Openstack instance in running DC but user’s VM was running in broken DC One of the DCs became broken but user is logged in Openstack instance in different DC and user’s VM is running in running DC.
It is assumed that user’s VM is dead but intermediate results are saved in distributed storage and available in different DCs. User might start up new VM instance in different DC and use saved earlier data from distributed storage.
No affection to user and user’s VM.
To increase computing service reliability it is used at least one independent Openstack
deployment in each DC. To make united cloud with decentralized administration [
As a storage subsystem, the CEPH platform is chosen to provide long-term distributed storage. It is configured and deployed the CEPH storage cluster to geographically distributed servers using the Internet line with nominal capacity 1 Gbit. Servers are divided into two groups (two sites) which are far from each other at a distance of about 40 Km. In other words it has been configured the testbed for geographically distributed storage. The only disadvantage of this configuration is the relatively low speed of replication. At the same time this testbed is quite enough for reliable archiving purposes for relatively long time (years). Clients might use the virtual storage when part of data are located on one group of storage servers and another part is in another group in the manner shown on the fig.1. All the data transfers between remote sites are performed over network tunnel with OpenVPN.
CEPH
Virtual Storage DC-1 DC-2
In the proposed testbed (fig.2), we focused on providing fault tolerance for computing. Components of the OpenStack do use common MySql database to store information about project settings, created virtual entities, provide message delivery. In order to do that database available on both DCs it were created MySql cluster in Master-Master mode. To make possible control interconnection by message quesues between sites it was created RabbitMQ. Both keystone database content are available in each DCs, and therefore you can make changes from any working DC.
With number of regions more than two, the cloud configuration with Pacemaker and HAProxy might give more advantages. Pacemaker can make automation of Active/Passive cluster availability. HAProxy can be used in this case not only for load balancing, but also to ensure routing of packets.
As feature development the reliable data links between DCs are required. It is assumed that in between DCs more than one data link is to be used. The data links are planned to be configured with failover option with SDN approach.
To guarantee strong encryption of transferred data in this project is planned use Quantum
Key Distribution [
This work was financially supported by the Ministry of Education and Science of Russian Federation (project No. 02.G25.31.0229).
It was described a range of tests on developed testbed when one DC is down to verify the reliability of computing services. It is shown that it is possible to increase cloud computing service reliability under specially configured Openstack in the cloud of geographically distributed data centers significantly.