This work presents an exploratory analysis about the students' behavior in the UNED-COMA MOOCs in order to detect and minimize drop-outs in MOOC courses. To achieve this, the following research questions are answered: a) what is the relationship between videos and students?; and b) what is the relationship between forums and students?. A set of indicators about students' behavior in MOOCs have also been extracted from this study, and a set of hypothesis have been analyzed. In particular, the amount and duration of videos do not a ect the students' outcomes, according to the statistical analysis taken with our datasets, and possible drop-outs within the course. Some courses must include speci c videos for a broader range of ages and country culture. In addition to this. the students' activity in courses does in uence in dropouts. As the period of the course advances, faculty should encourage students to participate in the interaction tools and visualize multimedia resources.
Massive Open Online courses (MOOCs) have nowadays become an excellent
platform to teach a great number of students at the same time with a distance
methodology. As a consequence, a big amount of data is generated from students,
interactions with the tools provided by MOOCs [
A producer is a student who interacts actively with the platform, but a consumer
does not interact with the platform, she/he only uses the learning resources in an
Copyright © 2017 for the individual papers by the papers' authors. Copying permitted for private and academic purposes. This volume is published and copyrighted by its editors.
isolated way [
On the other hand, the Spanish University for Distance Education, UNED,
has started the UNED-COMA (Curso Online Masivo Abierto; in English,
UNEDMOOC) [
A very preliminar work focused on only languages courses in the OpenMOOC
platform of UNED-COMA was presented in the LAK 2014 conference [
The OpenMOOC platform hosts a set of videos and discussion forums for each course, as main interactive available resources for students, being the principal interest of our study. Courses in OpenMOOC consist of units built by a set of knowledge pills. Pills are short videos linked to supplementary material (like documents, links, or exercises) and questions with their respective answers.
Pills are classi ed in normal, homework, and exams. The di erence among them is the time to study and the possibility or not to examine the answers before its deadline. Each course has associated an intelligent discussion forum, where students and faculty can discuss and collaborate on a unit/knowledge pill.
As for the tracking process, badges are automatically awarded to participants, recognising their contributions to the learning community. Other two kinds of accreditation are possible in OpenMOOC: 1) Credential: validation of having successfully nished and passed the course; and b) UNED-COMA certi cate: a formal face-to-face exam in a associated centre of UNED.
With respect to the OpenMOOC structure, most of the courses are hosted as
centralized MOOCs (xMOOCs). A small set of courses encourages self-learning
and creating learning communities, sharing of intellectual work and open access
to materials. We therefore need to consider the speci c characteristics of MOOCs
to see how quality can be described, assured and developed. For instance, a
taxonomy of eight types of MOOC was proposed in [
Apart from pedagogy, the courses in OpenMOOC can be classi ed in
accordance with ten dimensions identi ed by Margaryan et al. [
The rest of this work consists of four additional sections. First, a brief description of the methodology used in this work is given. Then, the most relevant details about the data pre-processing and contextualization are presented. Next section presents the research questions and hypothesis, and discusses the results obtained from the datasets. Finally, the primary conclusions and future work of this research are detailed. 2
To carry out the analysis of the resulting research questions and hypothesis
formulated in this work, the visual e-learning analytics process (VeLA) proposed
by [
The VeLA process taken here has been followed as detailed next. First of all, our dataset has been transformed in order to apply analytic models, particularly, an Exploratory Data Analysis (EDA), by including some statistical analysis. This initial dataset was composed by two di erent databases. On one hand, a PostgreSQL database contains all the static data related to OpenMOOC: information related to users, and courses structure and data platform. Additionally, a MySQL database has given support to the Q&A forums within our courses. Our OpenMOOC platform also uses a third database, a NoSQL MongoDB, to store activity data. At the time of this work, this database can not be accessed for its analysis.
In parallel with data analysis, we have mapped a portion of the data to visualize relevant patterns within the course resources, which could be relevant for studying our research questions below. The generated results have provided feedback to post-processing the dataset and to repeat the process in order to discover new knowledge.
Next sections provided with details about the datasets, research questions and hypothesis, and the results obtained in this work. 3
Within the OpenMOOC platform, there are hosted 38 di erent courses and 99 course instances due to several repetitions of the courses during several years. The courses are classi ed in the following categories selected by the UNEDCOMA administrators: { General purpose (G). { Languages (L). { Science and Technology (ST). { Social Service (SS). { Laws (LW). { Humanities (H). { Economic and Business (EB). { Psychology (P).
Due to the complex nature of some of the courses, it has been very di cult to classify them into one speci c category. Therefore, some of them are associated with two categories. The number of registered students is near 280.000. In Table 2, the percentage of students per category group is represented. The OpenMOOC platform does not provide many personal information related to the students such as age or gender. In order to perform this analysis within the VeLA process, an automated classi cation to detect user gender from his/her name has been implemented. We can stablish that there is a strong interest in courses related to languages; which includes English, Spanish, and German. It is also remarkable the great female interest in courses of the OpenMOOC platform.
If we pay attention towards the inscription of new students and the permanence of the students in the OpenMOOC platform represented at Table 3, there was an important interest in OpenMOOCs during 2013, but this initial enthusiasm dropped during 2014 and 2015 years. The lack of new students and the short permanence of the students in the platform are also two relevant issues that should be analysed carefully. 4
In this section two research questions are detailed and discussed exhaustively. These ones focus on the students' patterns when visualizing videos hosted in the studied OpenMOOC courses, and their interaction within the social communication tools o ered to them. In our particular case, the debate forums have been analyzed. A number of indicators belonging to OpenMOOC courses are also given along this section, in order to employ LA techniques when studying students' behaviors. 4.1
RQ1 - What is the relationship between videos and students?
As stated above, the most relevant elements within OpenMOOC courses are
videos [
Table 4 shows the geographical distribution, in terms of age and sex, of visitors for the videos hosted in the principal video channels employed by the OpenMOOC platform. This information can only be gathered when the user visualizes a video, while logged with a Gmail account. This is not a problem, since almost 70.000 students use it to access the OpenMOOC platform, according to the data stored in the database. They represent
As observed in Table 4, the percentage of visits are shown for each origin country of videos, and divided by the age in ranges and sex (male or female) of student visitors. On one hand, we would like to highlight the great impact of our courses has in American countries of Spanish tongue. Age range is also important in order to detect the most convenient public of our courses, and to study how to improve courses to attract them to the rest of visitors. For instance, visitors between 25 and 34 years old are the most interested in our courses in Peru. As for the particular sex of visitors, although the volume of students enrolled in courses is higher for women, men are more common as video visitors. This can become a contradiction, it is probably that women watch videos without a Gmail account session.
Figure 1 shows the course activity in terms of number of video visits with the same order scheduled in the OpenMOOC courses. From the 38 courses hosted in the OpenMOOC platform, each course contains a particular number of videos, and it can di er among courses (in this case, from 7 videos the shorter one to 191 videos the longer one). The number of visits for each video in the courses have been normalized to make possible a comparative study. The colour of each point in Figure 1 is proportional to the number of video visits. A dark green color indicates a big amount of visits, whereas a clearer point indicated a lower number of visits. In our particular case, in most of the courses we can nd an prede ned pattern, which consists on a series of videos with a few visits followed by a videos very visited. This particular video usually contains guidelines amount the course assessment, so it gets more visits.
Taking into account the previous information several hypothesis have been formulated: { Hypothesis 1: shorter videos are better for the students' outcomes. { Hypothesis 2: courses with fewer videos are more successful that those ones with a great number of videos.
We are going to study these hypothesis by means of a set of data variables or indicators. Now, a set of candidate variables (indicators) related to the success or failure of students in the OpenMOOC platform are given: { Number of videos per course (jvideosj): it is the amount of videos that each student has to watch to gather the minimum required knowledge. { Mean duration of videos per course (duration ): it is the mean duration in seconds of available videos in the course. { Mean amount of visits per video in the course (viewcount ): Each time a student watches a video, this YouTube counter increases in one unit. { Mean amount of likes per video in the course (likecount ); Each time a student likes a video, this YouTube counter increases in one unit.
Variable Mean Max. Min. jvideosj 81,08 45,82 229 7 -0,012 viewcount 37568,79 39838,23 146970,51 354,87 0,094 likecount 30,40 114,13 601,91 0,23 -0,005 duration 163,55 107,68 559,23 4 0,01 p-value 0,91 0,40 0,64 0,91
All the candidate variables obtained in this video category are not linked to
students' behavior. This evidence is supported by statistical analysis when we
perform the Spearman's rank correlation test [
RQ2 - What is the relationship between forums and students?
The debate forums in the OpenMOOC platform are represented as Q&A, in a similar manner as StackOver ow platform. In this sense, students can start questions (question), answer to existing questions (answer ), or enforce some previous answer (comment ).Figure 2 represents the activity volume in forums associated to each OpenMOOC course. From the analyzed data, the most popular Q&A activity has been answer, and there are a few comments. We would also like to highlight that some courses have not had any activity in it. This fact is very signi cant and, faculty should encourage his/her students to participate in the platform in order to enrich the learning process. The most active courses in forums are also the ones related to languages with a high di erence of messages.
Figure 3 shows the daily activities when the debate forums are analyzed per course. The periods of more activity is when a course starts (they start at di erent periods of time). In particular, we can observe that December and March are the months with a higher activity, the dates coincides with the starting of language courses.
Fig. 3. Daily activity of debate forums for the OpenMOOC courses.
According to debate forums, the candidate indicators to become relevant indicators of students' behavior are the following ones: { Number of activities (jpostsj): it is the number of total post messages for each course. { Number of questions (jquestionsj): it is the number of started posts for each course. { Number of answers (janswersj): it is the number of answered posts for each course. { Number of comments (jcommentsj): it is the number of comments for each course. { Percentage of the number of students (pauthors): it is the percentage of the total of the number of students of the course that takes part of the forum's activities. { Average number of activities per user (activitiesuser): it is the mean activity of users for each course. { Average number of activities per day (postperday): it is the mean number of posts per day for each course.
The main hypothesis that we can formulate over forum data is that the activity is directly related to the success of the course. We can formulate as follows: { Hypothesis 3: a elevated number of activities in the forums of the course is related to the student success within the course.
According to the evidence obtained by means of the Spearman's rank
correlation test [
We focus our attention towards the more active students, we can not correlate
the success of the students that not take an active part in the forums. Although
there are works that have been analyze this type of students [
The main objective of this work has been looking for the reasons why a big amount of students drop-outs in MOOCs courses and studying the information o ered by these types of courses. The UNED-COMA courses have been chosen for this study. In this sense, the VeLA methodology has been adapted for our purposes in order to prepare the data to make an exploratory data analysis,
Variable Mean Max. Min. p-value jpostsj 1226,82 2845,02 10144 1 0,66 0,0006 jquestionsj 207,43 474,42 1744 1 0,48 0,021 janswersj 862,26 1994,44 7150 0 0,70 0,0001 jcommentsj 157,13 381,93 1316 0 0,58 0,004 pauthors 8,49 10,83 32,89 0,03 0,29 0,19 activitiesuser 3,28 1,55 6,05 1 0,366 0,086 postperday 0,27 0,52 1,83 0,001 0,63 0,0013 which includes analyze the data statistically, so extracting new knowledge from our MOOC courses.
The lack of one of the databases from the platform, that contains data related to the student's activities, has been a great impact in our results. This database could allow us to correlate the results of the partial evaluations with the performance of the students inside the course and the videos. So, the conclusions of this analysis could be improved.
In our case, the OpenMOOC platform has been employed, which focuses on multimedia resources and interaction tools provided to students during their period of learning. Two main questions have been analyzed in this work: a) what is the relationship between videos and students?; and b) what is the relationship between forums and students?. Many courses must include additional videos for a broader range of ages, and thinking the destination country culture. As the period of the course advances, faculty should also encourage students to participate in debate forums, use the multimedia resources; that is, maintaining the student alive in the course.
A set of hypothesis have also been proposed and discussed from the detected candidate variables (indicators) from datasets. This indicators are about students' behavior in MOOCs given both from the point of view of videos and debate forums. For instance, we have concluded in a statistical way that the duration and amount of videos in an OpenMOOC platform do not a ect the students' outcomes (that is, possible drop-outs in the course), according our datasets, although their activity does. This way, these indicators could be useful as basis for further studies in any MOOCs platforms.
The authors are specially grateful to UNED-COMA for facilitating the access to the needed data for this research study. Also, authors would like to acknowledge the support of the European research project ERC-2015-STG-679528 POSTDATA, and the local project (2014I/PPRO/031) from UNED and Banco Santander; and the Region of Madrid for the support of E-Madrid Network of Excellence (S2013-ICE2715). The authors also acknowledge the support of SNOLA, o cially recognized Thematic Network of Excellence (TIN2015-71669-REDT) by the Spanish Ministry of Economy and Competitiveness. 14. Siemens, G., Baker, R.S.J.d.: Learning analytics and educational data mining: Towards communication and collaboration. In: Proceedings of the 2Nd International Conference on Learning Analytics and Knowledge. pp. 252{254. LAK '12, ACM, New York, NY, USA (2012), http://doi.acm.org/10.1145/2330601.2330661c