=Paper=
{{Paper
|id=None
|storemode=property
|title=Cloud Computing for Teaching and Learning MPI with Improved Network Communications
|pdfUrl=https://ceur-ws.org/Vol-945/paper5.pdf
|volume=Vol-945
|dblpUrl=https://dblp.org/rec/conf/ltec/Gomez-FolgarGPZ12
}}
==Cloud Computing for Teaching and Learning MPI with Improved Network Communications==
https://ceur-ws.org/Vol-945/paper5.pdf
1st International Workshop on Cloud Education Environments (WCLOUD 2012)
Cloud computing for teaching and learning MPI with improved network
communications
F. Gomez-Folgar, R. Valin, A. Garcia-Loureiro and T.F. Pena I. Zablah
Centro de Investigación en Tecnoloxı́as da Información (CITIUS) Sistema de Difusión de Radio y Televisión
University of Santiago de Compostela Universidad Nacional Autónoma de Honduras
Santiago de Compostela, Spain Tegucigalpa, Honduras
(fernando.gomez.folgar, raul.valin, antonio.garcia.loureiro, tf.pena)@usc.es mrzablah@unah.tv
Abstract—Nowadays, the teaching-learning processes are due to the fact that cloud users usually employ virtualized
being supported by the development of new technologies. resources. Hypervisors provide the necessary abstraction
During the recent past, technologies such as email, chat, au- layer and isolation in the same way as a sandbox. As a
dioconferencing, videoconferencing and webconferencing were
incorporated as new tools in the teaching-learning process. result, virtualized learning environments allow us to use the
Currently, another step is being walked with the development computational power of the compute nodes without the need
and the popularization of cloud technologies that are arousing of changing the physical host configuration, reducing the
great interest in educational environments. There has been systems administration effort and isolating the physical host
an actively development of cloud platforms with the release configuration from the student’s virtualized environment.
of several open-source solutions to build private, public and
hybrid clouds such as OpenNebula, Eucalyptus, OpenStack Furthermore, they allow users installing different guests
and CloudStack. Each of them has unique features that are operating systems and testing software that can coexist under
not found in the others. the same physical hosts without compromising or modifying
In the most basic cloud service model, Infrastructure as a its configuration.
Service, it is possible to provide computational resources as In this work, two different scenarios of a cloud infrastruc-
virtual machines. In Computer Science this model offers to
teachers and students the possibility of managing virtual infras- ture based on CloudStack for MPI learning are introduced:
tructures in which system administration and programming a virtual cluster as a MPI execution environment and an
languages practices can be performed without compromising improved virtual cluster whose MPI communication latency
the configuration of the underlying physical compute nodes. was reduced. This paper is organized as follows. In section II
In order to test a cloud infrastructure as a tool for learning
the architecture and characteristics of the CloudStack plat-
MPI, two different scenarios were evaluated in this work using
CloudStack: a virtual cluster as a MPI execution environment, form are presented. Section III describes the two teaching
and an improved virtual cluster whose MPI communication scenarios deployed under CloudStack for learning MPI. In
latency was improved. The results of this study are presented the first one a basic deployment is described, whereas the
in this paper. second one describes a scenario with improved performance.
Keywords-cloud; CloudStack; OpenMPI; Open-MX; Section IV describes the benchmarks to evaluate the perfor-
mance of both scenarios. The results obtained are presented
I. I NTRODUCTION in section V. Finally, the conclusions of this paper are drawn
Cloud technologies [1]–[6] are arousing great interest in in section VI.
educational environments as well as in business compa-
nies [7], and they are emerging as new tools that can be II. C LOUD S TACK
employed to support teaching-learning processes in a similar CloudStack is an open-source cloud management plat-
way that, in the past, technologies such as email, chat, audio- form, owned by Citrix, whose software architecture is
conferencing, videoconferencing, webconferencing, virtual shown in Fig. 1. It is composed by five types of compo-
classrooms, and collaboration suites were incorporated to nents: Compute Nodes (CNs), Clusters, Pods, Availability
support these processes. As a result, there is an increasing Zones, and a Management Server. The Compute Nodes are
number of open-source solutions to build private, public hypervisor-enabled hosts that have installed and configured
and hybrid clouds. Some of the most popular platforms the CloudStack agent. These hosts are the basic physical
are OpenNebula [8], Eucalyptus [9], OpenStack [10] and block that allow us scaling the platform. Additional hosts
CloudStack [11]. All of them have unique features that are can be added at any time to increase the provided capacity
not found on the others. for guest Virtual Machines (VMs). The hosts are not visible
In the teaching-learning processes, clouds, under Infras- to the end users, therefore, they can not determine which
tructure as a Service (IaaS) model, could be very useful hosts have been assigned to them to execute their guest
22
1st International Workshop on Cloud Education Environments (WCLOUD 2012)
VMs. A Cluster is a collection of CNs that share the the cloud. We have observed very interesting options like
same hypervisor type and have also access to the same the ability to define highly available VMs. They are kept
Primary Storage system. The Primary Storage stores the root operational by CloudStack without user or administrator
filesystem of guest VMs. Clusters are not visible to end intervention at all. Another interesting option for educational
users and represent the second level of scaling. A Pod is a environments is the installation of an operating system using
collection of clusters. It represents the third level of physical a standard ISO image. Its installation can be accomplished
scaling in the CloudStack platform. As clusters, Pods are not through the web interface without the need of using addi-
visible to end users. The Availability Zone is a collection of tional tools. This feature is very interesting because teachers
Pods and a Secondary Storage that stores predefined VM or students can install their own VMs without the need
templates and ISO images. It represents the fourth level of of using predefined VM templates. Furthermore, teachers
physical scaling. The Availability Zones are visible to the and students can create templates of their VMs that can be
end user who must select one of those to start a VM for private, only visible for the users of a specific account, or
the first time. The Management Server manages the entire public, visible for all users.
cloud.
III. T EACHING AND LEARNING MPI WITH C LOUD S TACK
Message Passing Interface (MPI) is a language-
independent communications protocol that has become a
de facto standard for communication among processes that
implements a parallel program using the message-passing
model. Distributed memory supercomputer clusters often
offers the use of MPI to their users.
The main goal of this article is to show how a cloud
infrastructure can be used to teach MPI but following, at the
same time, the Constructivism theory that allows students
construct their own knowledge by means of their personal
experience and interpretations. The role of the teacher is
to be a help in the understanding, improving the learning
quality and fostering the knowledge construction.
Students can deploy a safe infrastructure under Cloud-
Stack to learn the complete process including the installation
of a VM, the configuration of the operating system, the
installation of the MPI environment and the related develop-
ment tools. Under this infrastructure, students must be able
Figure 1. CloudStack architecture.
to carry out the performance analysis of their applications
and testing and implementing different approaches to release
CloudStack supports three user roles: root administrator, a MPI solution for a given computational problem.
domain administrator and not privileges users. The root In order to test the CloudStack cloud infrastructure as a
administrator can manage the entire cloud. Domain admin- teaching-learning tool for MPI programming paradigm two
istrators can perform the administrative operations for users scenarios were prepared and deployed. The first one is a
who belong to that domain and do not have visibility into basic testing setup scenario deployed as proof of concept,
the physical CNs. The not privileged users can manage their and the last one constituted an improvement from the first
own VMs. one, with the purpose of getting better performance. Both
The hypervisors supported by CloudStack are KVM [12], of them are described in the next subsections.
Citrix XenServer [13] and VMware vSphere [14]. The purpose of these scenarios is to prepare the students
This cloud platform can be managed completely through for solving problems in complex environments. The Cloud
the Web management server. It also provides a RESTFUL technology will help to achieve this objective and thanks to
API access to all its features. CloudStack also provides the CloudStack Web interface, the teacher can assist the stu-
CloudBridge, which is a server process that runs in com- dents easily and review their progress, focusing the attention
panion to CloudStack and provides an Amazon EC2 [15] on the most relevant topics. CloudStack provides flexibility
compatible API to access to CloudStack using existing EC2- to the teaching-learning process providing independence of
compatible tools. CloudBridge translates the EC2 API calls time and space. The students can perform their activities
to the CloudStack’s native API. without the need of being present in the computer laboratory.
One of the most notable characteristics in CloudStack is The cloud infrastructure used is based on CloudStack
the Web interface that provides a complete management of 2.2.14 employing commodity hardware. CNs are Intel Core
23
1st International Workshop on Cloud Education Environments (WCLOUD 2012)
i5 nodes with 8 GB of RAM, employing CentOS 6.3 64 bits The deployed nodes are VMs with a 10 GB hard disk,
as operating system, and KVM as the CloudStack managed each one has one core CPU and 1 GB of RAM under
hypervisor. The NFS server, acting as CloudStack Primary CentOS 6.3 operating system. The nodes mount the head
Storage, is a Core 2 DUO 6600 @ 2.40 GHz, with 4 GB of shared directories. A 1 Gb Ethernet network interconnection
RAM, 500 GB hard disk (7200 RPM SATA) and CentOS is being shared by the deployed virtual machines.
6.3 64 bit. The interconnection network of this cloud is an The deployed virtual cluster employs OpenMPI 1.6 [16]
Ethernet Gigabit Network with a MTU of 1500 bytes. as MPI implementation. OpenMPI is an open source MPI-
2 [17] implementation developed by a consortium composed
A. Basic scenario by research, academic and industry partners. Its features
In order to test the CloudStack cloud infrastructure as include full MPI-2 standards conformance, thread safety
a learning tool for the MPI programming paradigm, it is and concurrency, dynamic process spawning, network and
necessary to deploy a virtual cluster. A virtual cluster can process fault tolerance, network heterogeneity support, and
be defined as a cluster composed by Virtual Machines (VMs) run-time instrumentation, among others.
where the parallel applications are executed.
B. Improved scenario
In computer science, a cluster is a group of interconnected
computers that work together, and which can be viewed as Due to the high latency of MPI communications over
a single system. Typically, as shown in Fig. 2, two types of Ethernet networks using TCP, the performance obtained is
components can be part of a cluster attending the way that limited. However, this latency can be reduced using Open-
they are used: head and nodes. The head, or master, is the MX [18].
computer where the users connect. The nodes are intended as Open-MX is a high-performance implementation of the
computational resources that will be employed to run user Myrinet Express message-passing stack over generic Eth-
applications. Typically, users do not have direct access to ernet networks. It implements the capabilities of the MX
nodes so they cannot log in. Users will launch applications firmware running in Myri-10G NICs as a driver in the Linux
from the head that will be executed on the nodes. Each kernel. For legacy applications, a user-space library exposes
computer that compounds the cluster runs its own instance the MX interface to legacy applications. Open-MX supports
of an operating system. Linux on any architecture and works at least on Linux
kernels equal or greater than 2.6.15 version. It works on
all Ethernet hardware that the Linux kernel supports and all
connected peers, or compute nodes, must be on the same
LAN. Therefore, any router can not be between them but
switches. Open-MX is compatible with the IP traffic and
can perfectly coexist on the same network and drivers. To
setup Open-MX to be used by OpenMPI, it is necessary
to take into account that OpenMPI must be compiled and
installed enabling the Open-MX support.
The purpose of this scenario is to make students aware of
the importance of the analysis of computer performance.
The virtual cluster employed in this setup has the same
configuration as described previously but MPI communica-
tions are held by Open-MX, avoiding the overhead of TCP
for communicating MPI processes.
IV. B ENCHMARKS DESCRIPTION
In order to test the scenarios described previously, and
Figure 2. Cluster architecture. making students aware of the importance of performance
evaluation, three types of applications were executed: Intel
The virtual cluster that students must deploy in Cloud- MPI Benchmarks [19], the HEAT MPI [20] example, and
Stack is composed by a VM configured as the head and the Gadget-2 [21] application. The first one, the Intel MPI
two VMs configured as nodes. The deployed head is a VM Benchmarks 3.2.3 (IMB), provides a concise set of elemen-
employing a 10 GB hard disk, one core, 1 GB of RAM tary MPI benchmark kernels. It has several program param-
and CentOS 6.3 operating system. It also serves the home eters such as message lengths or selection of communicators
directory to the nodes that will compound the virtual cluster to run a specific benchmark. IMB also provides a standard
employing Network File System (NFS) as a distributed file and an optional configuration. If standard mode is used, all
system protocol. parameters mentioned previously are fixed and must not be
24
1st International Workshop on Cloud Education Environments (WCLOUD 2012)
changed. The mode selected to test the virtual infrastructure munication model implemented with the standardized MPI
is the standard ones. In this mode, message lengths varies communication interface. Gadget-2 computes gravitational
from 0, 1, 2, 4, 8, 16 to 4194304 bytes. forces with a hierarchical tree algorithm and represents
The current version of IMB, contains different classes fluids by means of smoothed particle hydrodynamics (SPH).
of benchmarks: Single Transfer, Parallel Transfer and Col- Gadget-2 can be used for studies of isolated systems, or
lective. The Single Transfer benchmarks are PingPong in simulations that include the cosmological expansion of
and PingPing. The Parallel Transfer benchmarks are space, with or without periodic boundary conditions in both
Exchange and Sendrecv. The collective benchmarks cases. In these types of simulations, Gadget-2 follows the
are Bcast, Allgather, Allgatherv, Alltoall, evolution of a self-gravitating collisionless N-body system,
Alltoallv, Reduce, Reduce_scatter, Allreduce and allows gas dynamics to be optionally included.
and Barrier. PingPong is used for measuring startup
and thoughput of a single message send between two pro- V. R ESULTS
cesses. PingPing measures also the startup and throughput
of single messages with the difference that messages are This section shows the results for the three applications
obstructed by oncoming messages. Sendrecv is based on described in IV. They were obtained for the basic scenario,
MPI_Sendrecv and each process sends to its right and using TCP for communicating MPI processes, and for the
receives from its left neighbour in a chain. Exchange improved scenario, using Open-MX for communicating MPI
is a communication pattern often used in grid splitting processes.
algorithms, in which the group of processes is seen as In the first place, we are going to show the obtained
a periodic chain, and each process exchanges data with results for both TCP and Open-MX of the Intel MPI
both left and right neighbours in the chain. Reduce is Benchmarks. The results obtained for both the PingPong
the benchmark for the MPI_reduce function. It reduces and the PingPing single transfer benchmarks are depicted
a vector of length L float items employing the MPI_SUM in Fig. 3. Notice that in all these figures the X-axis is
operation. Reduce_scatter is the benchmark for the in logarithmic scale. For the PingPong benchmark, the
MPI_Reduce_scatter function that reduces a vector of latency is reduced by around a 30% when Open-MX is
length L float items employing the MPI_SUM operation. In used for communication in comparison to TCP, such as is
the scatter stage, the L items are split as evenly as possible. shown in the square marks of the figure. The PingPing
Allreduce is the benchmark for the MPI_Allreduce benchmark also gets its latency reduced, even in a bigger
function that reduces a vector of length L float items employ- quantity (around a 36%) than PingPong, when Open-MX
ing the MPI_SUM operation. Allgather is the benchmark is used.
for the MPI_Allgather function in which every process
sends r bytes and receives a number of bytes that is equal
to r multiplied by the number of processes. Allgatherv
is the benchmark for the MPI_Allgatherv function that
shows whether MPI produces overhead due to the more
complicated situation as compared to MPI_Allgather.
Alltoall is the benchmark for the MPI_Alltoall
function in which every process inputs a number of bytes
equal to r multiplied by the number of processes (r for each
process) and receives a number of bytes equal to r multi-
plied by the number of processes (r from each process).
Alltoallv is the benchmark for the MPI_Alltoallv
function. Bcast is the benchmark for MPI_Bcast in
which the root process broadcast r bytes to all. In this
benchmark the root process of the operation is changed
cyclically.
In the second place, we test the performance of the virtual
cluster employing John Burkardt’s HEAT MPI [20], which Figure 3. PingPong and PingPing single transfer benchmarks.
is a C implementation of the 1D time Dependent Heat
Equation employing a form of domain decomposition. The results obtained for both the Exchange and the
In the third place, we test the infrastructure employing Sendrecv parallel transfer benchmarks are depicted in
the Gadget-2 software [21]. Gadget-2 is a freely available Fig. 4. For the Exchange benchmark, the latency is re-
code for cosmological N-body/SPH simulations on parallel duced by around 45% when Open-MX is used in comparison
computers with distributed memory. It uses an explicit com- to TCP, such as is shown in the square marks of the figure.
25
1st International Workshop on Cloud Education Environments (WCLOUD 2012)
Figure 4. Exchange and Sendrecv parallel transfer benchmarks. Figure 6. Alltoall and Alltoallv collective transfer benchmarks.
The Sendrecv benchmark also gets its latency reduced in Open-MX is used for communications.
35% when Open-MX is used for communications. The results obtained for both the Reduce and the
The results obtained for both the Allgather and the Reduce_scatter collective benchmarks are depicted in
Allgatherv collective benchmarks are depicted in Fig. 5. Fig. 7. For the Reduce benchmark, the latency is reduced
For the Allgather benchmark, the latency is reduced around 35% when Open-MX is used in comparison to
around 33% when Open-MX is used in comparison to TCP, such as is shown in the square marks of the figure.
TCP, such as is shown in the square marks of the figure. The Reduce_scatter benchmark also gets its latency
The Allgatherv benchmark also gets its latency reduced reduced around 32% when Open-MX is used for communi-
around 31% when Open-MX is used for communications. cations.
Figure 7. Reduce and Reduce scatter collective transfer benchmarks.
Figure 5. Allgather and Allgatherv collective transfer benchmarks.
The results obtained for both the Alltoall and the In the second place, for the HEAT MPI example, the per-
Alltoallv collective benchmarks are depicted in Fig. 6. formance is measured employing the computational elapsed
For the Alltoall benchmark, the latency is reduced time. When TCP is used to communicate MPI processes,
around 35% when Open-MX is used in comparison to employing two virtual nodes deployed under CloudStack,
TCP, such as is shown in the square marks of the figure. the elapsed time obtained was 22.708 milliseconds. When
The Alltoallv benchmark also gets its latency reduced Open-MX is used, the elapsed time obtained was 15.878
around 35% in a similar way as described previously when milliseconds (a 30% better than TCP).
26
1st International Workshop on Cloud Education Environments (WCLOUD 2012)
Finally, as a counter-example, Gadget-2 does not get a bet- R EFERENCES
ter performance when Open-MX is used for communicating [1] C. Babcock, Management Strategies For The Cloud Revolu-
MPI processes. This shows that the obtained improvements tion, USA, 2010.
depend on a large extent of the problem.
[2] T. Mather, S. Kumaraswamy, and S. Latif, Cloud Security and
Privacy. Sebastopol: O’REILLY, 2009.
VI. C ONCLUSIONS
[3] B. Chee and C. Franklin, Cloud Computing. Technologies and
The development of cloud technologies are arousing great Strategies of the Ubiquitous Data Center. Boca Raton: CRC
interest in educational environments. Cloud technologies Press, 2010.
are emerging as new tools to support teaching-learning
processes in a similar way that, in the past, technologies [4] R. Krutz and R. Vines, Cloud Security. Indianapolis: Wiley
Publishing, 2010.
such as email, chat, audioconferencing, videoconferencing,
webconferencing, virtual classrooms or collaboration suites [5] J. Rittinghouse and J. Ransome, Cloud Computing: Imple-
were incorporated to support these processes. Two character- mentation, Management, and Security. Boca Raton: CRC
istics of clouds are the use of virtualization technologies and Press, 2010.
the isolation between virtualized resources and the physical
[6] A. Velte, T. Velte, and R. Elsenpeter, Cloud Computing: A
infrastructure. These characteristics convert this type of Practical Approach. USA: McGrawHill, 2010.
infrastructures into a very useful tool in teaching environ-
ments where teachers and students can perform experiments, [7] K. Stanoevska-Slabeva, T. Wozniak, and S. Ristol, Grid and
avoiding compromising the configuration of the underlying Cloud Computing: A Business Perspective on Technology and
physical infrastructure and reducing the effort of system ad- Applications. Germany: Springer, 2010.
ministration tasks. As we have seen, teaching MPI on clouds, [8] OpenNebula Project Leads. http://opennebula.org
following the Constructivism theory, can be performed with
the purpose of preparing the students for problem solving in [9] Eucalyptus. http://open.eucalyptus.com
complex environments. Therefore, two different scenarios,
using commodity hardware and commodity interconnection [10] OpenStack. http://www.openstack.org
networks, were deployed under CloudStack using KVM as [11] CloudStack. http://www.cloud.com
hypervisor. The first one constitutes a virtual cluster for
executing MPI applications. A virtual cluster can be defined [12] KVM. http://www.linux-kvm.org
as a cluster composed by virtual machines. The second one
is an improved virtual cluster using Open-MX to get better [13] Xen. http://xen.org
latency of the MPI communications to make students aware [14] VMware. http://www.vmware.com
of the importance of performing the computer performance
analysis. Both virtual MPI infrastructures were tested em- [15] Amazon Elastic Compute Cloud EC2.
ploying the Intel MPI benchmarks, the HEAT MPI example, http://aws.amazon.com/ec2
and the Gadget-2 software. The obtained results show that
[16] Open MPI. http://www.open-mpi.org
executing MPI applications over the cloud were suitable
and the latency of the MPI communications was reduced [17] MPIForum. http://www.mpi-forum.org
around a 30% using Open-MX in comparison to TCP. The
elapsed time obtained for the HEAT MPI example is also [18] OpenMX. http://open-mx.gforge.inria.fr
around a 30% better than TCP. However, depending on the
[19] Intel MPI Benchmarks: Users Guide and Methodology De-
implementation of the applications that will be executed scription, Germany, 2006.
using MPI, there are cases in which the improvements on the
latency are not observed, as happened with Gadget-2. As it [20] MPI examples. http://people.sc.fsu.edu/ jburkardt/c src/mpi/mpi.html
was shown, the experiments described can be performed by
teachers or students as system administration, programming [21] Gadget. http://www.mpa-garching.mpg.de/gadget
languages and performance measurement practices.
ACKNOWLEDGMENT
This work has been supported by FEDER funds and
Xunta de Galicia under project 09TIC001CT, contracts
2010/28, and by Spanish Government (MCYT) under project
TEC2010-17320.
27