SUFFER: A Cloud System for Teaching Labs in Distance Education Francisco Javier Rodrı́guez Lera Miguel Ángel Conde David Fernandez Gonzalez 0000-0002-8400-7079 0000-0001-5881-7775 dferng@unileon.es Francisco J Rodrı́guez Sedano Ángel Manuel Guerrero-Higueras Camino Fernández Llamas 0000-0001-5909-1566 am.guerrero@unileon.es 0000-0002-8705-4786 Ingenierı́as Mecánica, Informática y Aeroespacial Universidad de León León, Spain Abstract—During a pandemic, redesigning the education However, for the most part of these tools are only used system to improve access to the available platforms at the as a support for regular classes, or as a way to maintain universities originally designed for research and local use is that same regular method in the cloud. For example: AVIP, necessary. The emergence of cloud computing services has solved many of the performance problems. This paper presents the Google Meet and Microsoft Teams to teach; Moodle to characteristics of a cloud e-learning tool. In order to do so, the access the contents and test the students; tutoring sessions concept of cloud computing is analyzed, and the architecture of through different forums or video call tools, etc. Meanwhile, the cloud computing platform used in different courses of the cooperation tools are very limited, specially because, even Computing Engineering degree are described. This platform was though students have multiple platforms they can use, these used in a proof of concept in the 2019-2020 academic year. The evaluation results, both from teachers and students that have are all hard to evaluate, and sometimes not even accessible used it, show positive and promising results. to the teacher [2]. Not to comment that in these cases it is Index Terms—Remote labs, online teaching, cloud computing not easy to find tools that facilitate the correct feedback in application academic contexts [3]. Having this type of tools is key in any environment, but it is even more so in the actual pandemic I. I NTRODUCTION situation [4]. Even if the methodology is the main element For years, cloud computing mechanisms have been a key needed to handle the current situation, providing access to element in the digital age. Different solutions to online tools that allow cooperation between students that are flexible learning have been incorporated to the educational context. and scalable is absolutely necessary [5], [6]. The goal has always been to improve the academic experience This report is presenting one of these tools named and to allow it to take place properly in spite of the physical SUFFER, that follows a computation model in the cloud and temporal restrictions [1]. An online learning platform in to offer different content to satisfy the current demands in the cloud must be able to dynamically scale on demand, to teaching and research, working even in complex or demanding offer the applications needed by the students, to be easily environments, like labs. The remainder of this paper is personalized to fit the particular needs of each student and structured as follows: first the different possibilities for online to be easy to maintain. With this criteria, its incorporation teaching and which ones SUFFER offers will be commented; into the academic institutions would be easier. next an experiment carried out as a concept test will be The current online learning platforms carry a high initial presented and commented; lastly a series of conclusions will cost in infrastructure and existing software applications be provided. or in ad-hoc developments. The academic institutions are under financial restrictions, and as a result there is a II. P RACTICAL LABS FOR CLOUD TEACHING shortage in human resources to operate, update and effectively In recent years, teaching and learning platforms have administrate the existing infrastructure. Faced with this become a natural tool for educational institutions. They situation, the adoption of cloud computing can help the support both on site and on line teaching. There is a huge institution to reduce costs in infrastructure, software and amount of platforms offering long-life teaching, employees human resources. This way, a school can rent these services training, academic courses, etc. Some of the most popular only when they are needed, or at least tailor them to the current platforms are Moodle, Blackboard, Sakai, etc. They provide needs. a collaboration space for teachers and learners as well as a This work has been partially funded by Ministerio de Ciencia, Innovación way for them to contact [7]. Prior to the identification of the y Universidades through grant RTI2018-100683-B-I00. elements that a virtual cloud teaching platform should have, Copyright © 2020 for this paper by its authors. Use permitted under Creative Commons License Attribution 4.0 International (CC BY 4.0). Fig. 1. Platform for cloud teaching. Teacher levels of participation it is necessary to identify the tasks that students and teachers equipment, teachers can not use the lab tools and the lab perform in classical teaching. On the one hand, the student is being used by no one. In some situations, a plausible has to take notes, make exams, send comments, perform alternative to solve all these issues together could be using individual and group assignments, access contents, etc. On the cloud for teaching. the other hand, the teacher needs to manage the contents to be offered, tools to prepare tasks and exams, and communication B. Cloud lab and evaluation systems. These tools should offer students and According to the proposals gathered from literature and teachers a way to perform education tasks following several the bidirectional needs expressed by teachers and learners, models, such as, master class, seminars, practical sessions, it is necessary to establish the set of services to be offered tutoring, exams, reviews, etc. One of the most difficult parts, by a cloud system and how to link them to the traditional regarding virtualization, are practical labs because of their teaching/learning processes. In this context, once the elements practical and tangible nature. This section shows a summary of a usual cloud service structure have been identified [9], of which are the implications of this kind of labs and how their application in learning environments will be evaluated. they are moved to the cloud. The more common cloud scenarios are presented in Fig 1. Their description is as follows: A. Traditional lab 1) Infrastructure as a service (IAAS): this model offers Our staring point is practical teaching in a traditional the user the elements needed to perform system tasks classroom or lab. This teaching takes place in an institutional (processing, storage, networks, GPU, etc) in order to environment where students, teachers and physical tools such deploy and run arbitrary software. When transferred as PCs, pipettes, or plastic material, share the same place. to the educational context, the teacher does no longer Somehow, teachers and learners interact face to face in order manage the infrastructure, as in the real lab, does not to fulfil that knowledge transmission that takes place through have physical access to the computer, but does have direct experimenting of all the actors involved. absolute control of the logical configuration of the Although the lab is the key tool for the practical part system: operating system and applications. of many courses, it presents several issues as Rosado 2) Container as a service (CaaS): this mechanism shows in his work [8]: a) Normally, the equipment used relies on the concept of virtualization through is especially expensive, making it difficult to perform and containers. This concept is based on deploying repeat the experiments outside the lab; b) Many labs are ”preconfigured” systems avoiding the need of installing overcrowded or underequipped due to budget constraints; c) the corresponding software infrastructure. When applied Lab classes usually require a direct supervision and feedback to the educational context. This approach implies from the teacher and an individual set of material for every deploying basic prefabricated solutions offered by student, which result in overcrowded environments, as already manufacturers and adapting them to the classroom mentioned; d) Not always are the students ready to work with needs. specific devices and techniques, risking expensive equipment. 3) Platform as a service (PaaS): this model does not allow Besides all these issues, there is a new scenario, due to the the user to manage or control the underlying cloud pandemic situation, that can prevent onsite teaching to take infrastructure but it offers complete control over the place. In this case, students do not have access to the lab applications deployed and their associated data. For the teacher, this means that the working environment can include different data sources, or even can install new plugins and software initially not available. 4) Software as a service (SaaS): this model is based on the opportunity to offer the consumer the use of an application. The idea is to offer a multi-platform application to be used through another application of through the web browser. In this case, the teacher uses a preconfigured tool that offers a variety of elements to perform and visualize the work being done, such as Google Docs or Moodle, but with the limitation of not being able to install new extra applications. 5) Functionality as a service (FaaS): this mechanisme is more enclosed than all the previous ones, as the possibilities are limited to specific functionalities. An example of this kind of service can be observed in Fig. 2. Suffer Framework the Virtual Programming Laboratory of the Moodle platform, that is used when required. One of the main characteristics of FaaS is its resemblance to the A. Generator of Platforms serverless solutions, that is, they are only active when The main objective of SUFFER is to adopt an approach they are needed, but not always. oriented towards services on cloud to provide the technological These scenarios have already been tested in different resources needed by the teacher and, thereby, by the student proposals for the educational context. For instance, Rosado [8] too. This way, the teacher can focus completely on the specific had already presented a work for teaching Physics in the cloud knowledge to be transmitted and the student can focus on in 2005, and Orozco [10] did it for Chemistry. Java applets getting it. were used in the first work to simulate a lab environment For the teacher that wants himself to deploy the software although the grading system or teacher support are unclear. needed for his class, a pool of containers will be offered with In the second work, a self-assessment system is proposed just an operating system installed. In these containers, the including true/false and autocomplete questions starting from teacher will have access to installing and removing everything a sequence of documents available on line. In the field of needed. This way, not only the code of applications should computer science, works related to tools such as VPL (Virtual be loaded, but also all technical requirements for coding, Programming Lab) can be found [11]. VPL is a software compiling and managing the applications to be used. The tool that allows managing programming assignments inside approach used to solve this scenario is mainly based on Moodle. However, this solution is limited to one specific containers, and it is therefor associated to a CaaS scenario. functionality, which is testing code, excluding part of the For the teacher that decides to use an approach based on infrastructure available in a lab, such as robots. predefined containers for his classes, an advanced environment will be provided with software already installed. This option prevents the teacher from the management of the middleware III. A RCHITECTURE and minimizes configuration problems. This way, the teacher relies on the technologies supplied by the provider, and deploys the applications and the corresponding data sets Taking into account the conditions described above, This together with the platform. This case is considered as PaaS work proposes SUFFER as a tool to move labs to the solution. cloud. SUFFER lays out the possibility of deploying labs The third option considered in SUFFER framework is the with different characteristics based on CaaS, PaaS and SaaS SaaS solution. In this case, the whole infrastructure needed attending teachers’ needs (see Fig 1). SUFFER can offer a for the lab is offered to the teacher, that is, students will be complete infrastructure simulating a real PC (infrastructure as working with a closed platform where the addition of new a service). It would allow offering practical labs for courses content should be minimal. Although modifying the platform such as Operating Systems or Embedded Systems. But due by adding new material can be blocked, we consider that to the huge amount of possibilities that this approach can offering the possibility to include new elements that could offer, it is necessary to narrow down and monitor many more improve the class is the right choice for a successful practical elements in the machine, both regarding maintenance and session. cibersecurity. For this reason, the more general perspective of the project has been postponed for future iterations. Figure 2 B. Monitoring System shows graphically the elements included in SUFFER in the Beyond the technological infrastructure of the lab, it is context of robotic labs. necessary to offer monitoring tools in order to reduce the inactivity time of the three main actors, that is, the tool, the IV. T ESTING SUFFER teacher and the student. The tool should minimize the no In order to validate the framework developed, an experiment productivity periods of time or the possible system crashes. has taken place as a proof of concept. The two goals of this The amount of money employed in hiring an infrastructure test are: on the cloud can not be lost because of unavailability of the 1) Using SUFFER as a robotic virtual environment as a service. The goal is to provide distributed resources capable replacement of the traditional robotic lab. of being used in an optimal use. 2) Quantitative analysis of the performance of both teachers Teachers should also have monitoring tools for the progress and students. and evaluation of their classes so that a positive or negative Teachers and students do not need to know about the evaluation of students’ performance can be offered at the end functioning rules of the platform. The requirements for them of each session. For instance, figure 3 shows the state of four are: 1) it is necessary for them to have a web browser; 2) it different sessions of students: two of them have not entered is desirable for them to have at least some basic knowledge yet, and the other two are already working on their assignment. about ubuntu or any other similar system; 3) the teacher can When used together with a videoconference tool, SUFFER choose the way to access the platform from the information allows the teacher to interact directly with the student desktop received by an email sent by the system. The next step is for in order to solve or fix a problem. the teacher to guide the session. When it is finished, he notifies Last, it is necessary to know the state and the use that the us and we gather the information about the class and delete student is doing of the platform. This is not only important the machines. because of reasons related to providing feedback to the teacher, but also to guarantee that the user is not performing any illicit A. Structure of the experiment action by means of the platform. For this reason, SUFFER Using SUFFER is proposed to different postgraduate carries out a soft monitoring of the system with the network courses at University and, therefore, to the master and monitoring tool CICFlowMeter [12]. In addition, it records a doctorate students enrolled in those courses. set of data logs with timestamp that provide information about The courses involved in the experiment were: Virtual when and in which conditions the interaction with the system Reality from the Master of Science in Computer Science takes place during the practice session. This information will (4 students); Robotics applied to aging biosanitary sciences allow the teacher to predict the behaviour of the student during from the Master of Science of Healthy Aging and Quality of the class. Life (Inter-university studies) (15 students), both in University of León. In addition, SUFFER was also used in a training course in which a practical lab about reinforcement learning in robotics was included. This course was devoted to teachers of different areas and departments of the university. From this point, every teacher manages the set of machines freely and notifies any issue than may prevent the practical session to take place. B. Methodology Fig. 3. Example of the teacher’s window to supervise the students In order to give a practical lesson, we assume that the teacher knows the subject to be taught. This way, the sequence These three lines of supervision allow the improvement to follow with the teacher is always the same one: of the resources of the cloud system and the services 1) The information about the machines requested for the offered to every teacher. At the same time, they easy the class is received. optimization of the mechanisms used to properly provision 2) In some cases also the list of tools needed is provided. the resources offered at any time. This supervision allows also 3) The class takes place. the observation of possible limitations of the system and the 4) Information about the behaviour of the students in class proposal of mechanisms to solve them. is offered to the teacher. In the current state of the research, besides the direct 5) The performance of the machines used for the class is supervision of the students by the teacher, SUFFER offers evaluated from the technological point of view. a monitoring system and also a mechanism to launch events associated to files and applications. Their source are mainly the C. Results log files provided by the applications and the user interaction The results have been obtained from the three experiments with the terminal. This information turns out to be of great where SUFFER was used. In each experiment, the platform importance for lab activities related to computer science in was used in sessions that took from one to three hours. particular. This monitoring can take place one time or through The data obtained from these sessions were used for the historic data. In both cases, the analysis of this data is of platform evaluation. Table I shows the results collected from great value to analyze how students solve their practical the questionnaires uses. This information is also available from assignments. the following link https://forms.gle/yKXF2u3U1J75Le7z7. TABLE I R ESULTS OF THE QUESTIONNAIRE SHOWING ADVANTAGES AND DISADVANTAGES , P REVIOUS EXPERIENCE WORKING WITH CLOUD APPLICATIONS (E XP.: Y ES -1, N O -0). T EACHER ’ S CONTROL (TC: Y ES -1, N O -0). T IME TO GET USED TO IT (T G U WHERE L IKERT: 1-5). R ECOMMENDATION OF THE SYSTEM TO BE USED IN CLASS (R:M AY- BE -2, Y ES -1, N O -0). Advantages Disadvantages Exp TC TgU R A1 The interface. It is just like your own Linux None observed. 1 1 1 1 PC. No effort needed to learn how to use a new interface. A2 Having all the software installed already None observed. 1 1 1 1 available. A3 Very easy to use technology. All the None observed. 0 1 1 1 Fig. 4. Example of one of the desktop of a student using the platform. contents for the course are already installed and configured, which eases the class progress. It all worked fast and stable. I1 A whole environment in the cloud ready to Someone has to give you access to 0 1 1 2 be used the platform V. D ISCUSSION I2 You do not have to worry about installing None by now. 0 1 1 1 the software you need to make your practical assignments. A. Traditional lab vs Cloud lab I3 It allows me to share my desktop with my You can not install any new tool. 1 1 1 1 teacher and other students. The main goal of this work is to determine whether I4 ”Abstraction of the operating system. Be sure that all the students do have exactly The possible resources. limitation of 1 1 1 2 virtual laboratories can be used in environments with special P1 the same versions of libraries and tools.” Normalization of the student’s environment. The access in the web browser 1 1 1 1 computational restrictions such as robotics field. In order Accessing through a web browser helps the teacher to solve problems of each student seems not to be safe. Having access to the GPU may help for to achieve this goal, the approach proposed in [13] will P2 individually. Very intuitive tool to learn. Flexibility, monitoring, possibility of view workshops of machine learning. It does not include a 1 1 2 1 be followed. According to the authors, the use of these the student’s desktop and provide feedback to each of them. videoconference tool and an external one has to be used. laboratories allows to guarantee that the students acquire the P3 Students do not have to install anything and teachers can watch the machine of every Sometimes, for some student whose connection is not very 0 1 1 1 required competences. Sometimes, this process implies the use student good, the system shows a slow performance. of tools that are not always available or that have a temporary availability. This issue about availability has been a sound one specially a technology acceptance one. In both cases, it would be in the first sessions. For all teachers, students and researchers interesting to study the differences between the different roles show a very positive opinion in general about having a and their specific interests. cloud robotics lab already available with all the tools needed to start working pre-configured. Regarding advantages and B. Cibersecurity and Privacy disadvantages found by the different actors involved in the In this section, several points suggested by the work experiment of using SUFFER some issues have to be pointed of Alshwaier et al. [14] have been considered. During out. The first one are the doubts about the resources needed the experiments, students have shown some reservations for a practical class to take place. The issue about limited when using tools not supported by multinational technology resources requires farther research. For instance, a ROS enterprises because, to their understanding, they would (Robot Operating System) using the Gazebo simulator requires guarantee their privacy. That is why some students prefer to something like 4 processors working at 100% of an Intel(R) spend several hours configuring their personal machines in Xeon(R) CPU E5-2640 v4 @ 2.40GHz machine when the order to fulfil their practical assignments. To our knowledge, student is fulfilling some of the exercises, although the RAM this is more a perception than a fact. The tool implemented consumption is kept under 2GB. Authors are now working in University of León stored all the data in servers located on stress tests of the system in order to assess the maximum in Spain, guaranteeing the European GDPR (General Data number of students for a given provision of resources in the Protection Regulation). Even more, we are working on cloud. providing mechanisms to guarantee other extra aspects related Beyond these opinions, and regarding other aspects to data security. evaluated of the framework performance, three specific factors have to be brought to your attention: 1) Most of the persons VI. C ONCLUSIONS participating in the experiment had previous experience with This work presents a proposal to move the virtual classes cloud applications (60%), and this fact could somehow bias in technological laboratories to a cloud system and, as a the results obtained but at the same time shows the kind of result, the SUFFER framework has been defined. Although actual student, teacher and researcher who use cloud tools on the idea of the framework was initially considered for robotics a daily basis both for education and also professionally; 2) laboratories, it can be applied to any kind of technological The possibility offered to the teacher of remotely controlling laboratories. It has proofed to be specially useful in contexts students’ desktop has been found useful by 100% of the where on site teaching is not possible or the resources are participants; 3) The time needed to get used to the system limited. has been generally a very short one. The study presents a qualitative analysis of the different Nonetheless, after this first experiment, it would be types of users of the framework, that is, students, teachers appropriate to carry out a standardized usability test or and researchers. Their evaluations show positive results that support the continuity of the development of the platform and its use in different types of laboratories. The next step in this research is focused on formalizing several practical sessions with a higher number of students in order to observe, analyze and try to improve the monitoring system. This system should offer the teacher valuable information about the students taking part in the practical sessions. R EFERENCES [1] F. J. Garcı́a-Peñalvo, M. Johnson, G. R. Alves, M. Minović, and M. Ángel Conde-González, “Informal learning recognition through a cloud ecosystem,” Future Generation Computer Systems, vol. 32, pp. 282 – 294, 2014, special Section: The Management of Cloud Systems, Special Section: Cyber-Physical Society and Special Section: Special Issue on Exploiting Semantic Technologies with Particularization on Linked Data over Grid and Cloud Architectures. [Online]. Available: http://www.sciencedirect.com/science/article/pii/S0167739X13001696 [2] M. Á. Conde, F. J. Rodrı́guez-Sedano, F. J. Rodrı́guez-Lera, A. Gutiérrez-Fernández, and Á. M. Guerrero-Higueras, “Analyzing students’ whatsapp messages to evaluate the individual acquisition of teamwork competence,” in Learning and Collaboration Technologies. Ubiquitous and Virtual Environments for Learning and Collaboration, P. Zaphiris and A. Ioannou, Eds. Cham: Springer International Publishing, 2019, pp. 26–36. [3] D. Boud and E. Molloy, Feedback in higher and professional education: understanding it and doing it well. Routledge, 2013. [4] F. J. Garcı́a-Peñalvo, V. Abella-Garcı́a, A. Corell, and M. Grande, “La evaluación online en la educación superior en tiempos de la covid-19,” Education in the Knowledge Society, vol. 21, no. art 12, 2020. [5] F. J. Garcı́a-Peñalvo, “Modelo de referencia para la enseñanza no presencial en universidades presenciales,” Campus Virtuales, vol. 9, no. 1, pp. 41–56, 2020. [6] A. Garcı́a-Holgado, F. J. Garcı́a-Peñalvo, A. Vázquez-Ingelmo, N. Burgos, D., N. F., Padilla Zea, C. Higuera, D. Hvarchilkova, A. Teixeira, U. D. Ehlers, and J. Brunton, “Handbook of successful open teaching practices,” European Union, Tech. Rep., 2020. [7] P. Avgeriou, A. Papasalouros, S. Retalis, and M. Skordalakis, “Towards a pattern language for learning management systems,” Educational Technology & Society, vol. 6, no. 2, pp. 11–24, 2003. [8] L. Rosado and J. Herreros, “Nuevas aportaciones didácticas de los laboratorios virtuales y remotos en la enseñanza de la fı́sica,” Recent Research Developments in Learning Technologies, vol. 1, 2005. [9] S. Kächele, C. Spann, F. J. Hauck, and J. Domaschka, “Beyond iaas and paas: An extended cloud taxonomy for computation, storage and networking,” in 2013 IEEE/ACM 6th International Conference on Utility and Cloud Computing. IEEE, 2013, pp. 75–82. [10] C. Orozco, N. González Delgado, A. Pérez Serrano, B. Caballero, J. Martı́n, and V. Ramos, “Elaboración de material para presentación on-line de prácticas de laboratorio de quı́mica aplicada,” Libro de Resúmenes II Jornadas de Innovación Educativa. Resumen EB13 106, pp. 80–88, 2007. [11] J. C. Rodrı́guez del Pino, E. Rubio Royo, and Z. J. Hernández Figueroa, “Vpl: laboratorio virtual de programación para moodle,” in XVI Jornadas de Enseñanza Universitaria de la Informática. Universidade de Santiago de Compostela. Escola Técnica Superior d’Enxeñarı́a, 2010, pp. 429–435. [12] C. I. F. C. Security, “Cicids 2017,” http://www.unb.ca/cic/datasets/flowmeter.html, 2017. [13] A. Zaldı́var-Colado, “Laboratorios reales versus laboratorios virtuales en las carreras de ciencias de la computación,” IE Revista de investigación educativa de la REDIECH, vol. 10, no. 18, pp. 9–22, 2019. [14] A. Alshwaier, A. Youssef, and A. Emam, “A new trend for e-learning in ksa using educational clouds,” Advanced Computing, vol. 3, no. 1, p. 81, 2012.