=Paper=
{{Paper
|id=Vol-1599/9WAPLA_2015
|storemode=property
|title=Combining Learning Analytics and the Flipped Classroom in a MOOC of Maths
|pdfUrl=https://ceur-ws.org/Vol-1599/9WAPLA_2015.pdf
|volume=Vol-1599
|authors=Diego Redondo Martínez,Pedro J. Muñoz-Merino,José A. Ruipérez-Valiente,Carlos Delgado Kloos,Héctor J. Pijeira Díaz,Javier Santofimia Ruiz
|dblpUrl=https://dblp.org/rec/conf/ectel/MartinezMVKDR15
}}
==Combining Learning Analytics and the Flipped Classroom in a MOOC of Maths==
https://ceur-ws.org/Vol-1599/9WAPLA_2015.pdf
Trends in Digital Education:
Selected papers from EC-TEL 2015 Workshops CHANGEE, WAPLA, and HybridEd
Combining Learning Analytics and the Flipped
Classroom in a MOOC of maths
Diego Redondo Martíneza, Pedro J. Muñoz-Merinob, José A. Ruipérez-Valienteb,c,
Carlos Delgado Kloosb, Héctor J. Pijeira Díazb, Javier Santofimia Ruizb
a
CEPA Sierra Norte de Torrelaguna, Madrid
b
Universidad Carlos III de Madrid, Avenida Universidad 30, 28911 Leganés (Madrid) Spain
c
IMDEA Networks Institute, Av. del Mar Mediterráneo 22, 28918 Leganés (Madrid) Spain
diego@cepasierranorte.es, {pedmume@it, jruipere@it,
cdk@it, 100075697@alumnos, 100060449@alumnos}.uc3m.es
Abstract. This paper describes the configuration, setup and initial analysis ex-
amples of an experience for introducing learning analytics in a MOOC of maths
for adult high school education, using the flipped the classroom methodology.
An overview of the MOOC of maths is provided, as well as an overview of the
ANALYSE learning analytics tool which is used to analyze the learning pro-
cess. We describe how the learning analytics tool can be useful in this setup and
methodology and we illustrate specific examples of conclusions in the MOOC
of maths.
Keywords: learning analytics, MOOCs, high school education, flipped class-
room
1 Introduction
Learning analytics is a useful technique to track and intervene in the learning pro-
cess in order to improve it. Massive Open Online Courses (MOOCs) have emerged as
a new paradigm that usually has an intensive use of videos and interactive exercises.
The type of rich users’ interactions in MOOCs makes learning analytics a very useful
technique in these scenarios. However, current support of learning analytics in
MOOC platforms is at an early stage. We have extended the learning analytics sup-
port for the open edX platform [1] with a new visual analytic module named
ANALYSE (Add-on of learNing AnaLYtics Support for open Edx), which takes ideas
from the learning analytics support of the Khan Academy platform and a new exten-
sion we implemented in it [2].
Emerging and useful educational technologies such as learning analytics are usual-
ly designed, tested and experienced in research contexts. But there is a need of ex-
tending these educational advancements to the whole society. In this direction, we
aimed at applying the ANALYSE learning analytics tool as well as the open edX
platform in a MOOC of maths in the CEPA Sierra Norte de Torrelaguna High School
71
Trends in Digital Education:
Selected papers from EC-TEL 2015 Workshops CHANGEE, WAPLA, and HybridEd
Center (in the north of the Madrid Region). This center provides high school educa-
tion for adults. The profile of the participants of this center is quite different from
traditional high school students as there is a difference of age and old generation are
e.g. not as used as new ones with technologies.
In this document, we give an overview of ANALYSE, the design of a MOOC for
maths for high school education for adults or the methodology used in high school
education for using the MOOC in combination with traditional class sessions and
ANALYSE. In addition, we provide some initial examples of how to interpret some
data of ANALYSE in this MOOC of maths for high school education for adults in
order to improve the learning process.
2 ANALYSE
The analytics visualizations that are provided by Open edX by default are very
limited, therefore we have developed and installed ANALYSE1 (Add-on of learNing
AnaLYtics Support for open Edx). This add-on extends the learning analytics support
of Open edX with 12 new visualizations related to exercises, videos and course activi-
ty. These new indicators are obtained by processing the logs of Open edX and trans-
forming them into useful information that can be used by the different stakeholders.
The process of computing the different indicators is done by ANALYSE as a back-
ground job in regular intervals, which is transparent for the user; the indicators are
updated in regular intervals of time, instead of being calculated each time that a visu-
alization is requested.
ANAYSE provides of a different interface for instructors and students. Students
can access the information related to their own activity only, whereas instructors can
access the information related to the full aggregate of the class but also each one of
the students taking the course individually. This can help students to gain self-
awareness about how they are progressing in the platform and improve their motiva-
tion. Additionally, this dashboard helps instructors to keep track of all the students in
the class to know who is doing what and who is having problems. They can also use
this tool to detect problems in resources such as videos who are not transferring
knowledge properly or problems with an elevate complexity compared to the rest of
the course. The different visualizations of ANALYSE are grouped in the learning
analytics dashboard in the following sections:
• Exercise visualizations are related to the activity of students with the dif-
ferent problems of a course. There are three different visualizations
which are the next:
o Course Summary: Provides an overview of the percentage of
students who have achieved proficiency, passed, failed or not
accessed each category of resource
o Students Grades: Provides the grade of each item of the course
1
http://www.it.uc3m.es/pedmume/ANALYSE/
72
Trends in Digital Education:
Selected papers from EC-TEL 2015 Workshops CHANGEE, WAPLA, and HybridEd
o Problem Time Distribution: Time invested in each problem of
the course
• Video visualizations are related about the interaction of students with the
course videos. There are four different visualizations:
o Video Time Watched: It shows the progress done in each of the
videos from 0 to 100 by the students
o Repetitions of Video Intervals: This visualization provides in-
formation about what segments of time of each video have been
seen more times
o Video Time Distribution: Shows the time invested in each one of
the videos separately
o Video Event Distribution: This visualization displays the posi-
tion in the video of the different video events triggered by stu-
dents when watching the video (pause, stop, play, seek…)
• Course visualizations are related to both course and exercise activity at
the same time and other things:
o Problem and Video Progression: This visualization shows the
progress over time in both exercises and videos that students
have done in a stepped area chart.
o Daily Time on Problems and Videos: This visualization shows
the time invested each day in exercises and videos separately.
o Course Accesses: This visualization shows the number of ac-
cesses to each chapter of the course and their respective subsec-
tions and units.
o Chapter Time: It shows the time elapsed in each chapter of the
course and it divides this time in graded and ungraded.
o Student Time Schedule: This visualization shows the amount of
time invested in three different time intervals (morning, after-
noon and night)
3 The Design of a MOOC for Maths
Open edX is one of the most widespread MOOC platforms due to the great quality
of the software but also because is one of the few that has been released as an open
source application that can be used for free. Open edX is composed of two main ap-
plications which are the Learning Management System (LMS) and the Course Man-
agement System (CMS) which is denominated as Studio. On one hand, the role of
Studio is to provide of a friendly interface for instructors where they can build the
course and create the different elements of each section of the course, set the grading
policy, configure the different course deadlines or the timing in which each section of
the course need to be released. Studio application allows organizing courses in sec-
tions, sub-sections and units. The contents can be diverse such as videos, HTML con-
tent or a big selection of assessment activities. Assessment activities include from the
simple ones like checkboxes, multiple choice, or different types of input field prob-
73
Trends in Digital Education:
Selected papers from EC-TEL 2015 Workshops CHANGEE, WAPLA, and HybridEd
lems, to those more advanced like the evaluation of programing expressions, drag and
drop or peer assessment. On the other hand the role of the LMS is to display for each
one of the courses the contents according to how the course has been configured in
Studio. Students and instructors can navigate the course contents, use the different
elements such as exercises and videos and interact socially with other peers and
course instructors through the use of forums.
Figure 1. Example of the video for "Quadratic equations" in the LMS interface
The topics that have been covered and designed in the course have been divided in-
to three main sections. Additionally, an introductory section about how to use the
platform and the contents was included. The main sections of the course are the next:
• Introduction: Introductory section so that students know how to interact
with the contents, and other information regarding the MOOC.
• Units of measurement: This section describes the different basic units of
measurement and their use for mass, length, capacity, volume, time and
surface area.
• Algebra: This section starts describing monomials and how to operate
with them, next it explains first grade and also quadratic equations by de-
scribing how to resolve different particular cases with exercises and vide-
os.
• Geometry: The last section describes the concept of angles in polygons,
then it explains triangles and the Pythagorean theorem. Finally the calcu-
lation of areas for the different geometric shapes is explained.
74
Trends in Digital Education:
Selected papers from EC-TEL 2015 Workshops CHANGEE, WAPLA, and HybridEd
Each one of the three main topics of the course is distributed with different sub-
sections, where each sub-section contains a unit that can be either HTML content, an
exercise or a video. One of the decision design aspects that we took when building the
course, was to only put one element (HTML content, exercises or videos) in each one
of the units. The objective for this design is to be able to obtain individual analytics
about the time spent in each course element separately, and that would have not been
possible if we put several elements in the same unit. Additionally, the general idea is
to provide short videos and after then some exercises to check if the students have
learned the concept; this design also helps students not to get bored as it is more dy-
namic than other traditional methodologies where all the contents are delivered first
and then there is an exam that includes everything.
There are a total number of 31 different problems. Most of the problems have a
value of one point, but there are some others which are longer that score more be-
cause they require several answers instead of only one. Additionally a total number of
28 videos were recorded, edited and added to the MOOC. Videos are short, most of
them within the range of 2 to 4 minutes and they explain a small piece of content of
the course. Similarly, exercises evaluate small pieces of content that are closely relat-
ed to the explanation at the previous videos. The main types of exercises that have
been used are checkboxes, multiple choice and input field problems.
4 The methodology and the role of learning analytics
The MOOC of maths has been available for everyone interested in learning these
concepts of maths, but not only for the students of the CEPA Sierra Norte de Torrela-
guna. The MOOC is still available (activities in Spanish language) at:
http://ela.gast.it.uc3m.es/
The students of the CEPA Sierra Norte de Torrelaguna were encouraged to use the
different videos and exercises of the MOOC. Educational materials (videos and exer-
cises) were released in sequence but not all at once, so that the educational activities
related to a face to face class session were available a few days before the face to face
class session. Students had to make the educational activities (watch the videos, solv-
er the exercises) before the correspondent face to face session. The face to face ses-
sions were devoted to reinforce the concepts, solve questions, etc. This is a flipped
classroom [3] methodology.
The role and importance of the learning analytics methodologies and the
ANALYSE module relied on the following points:
- Self-regulated learning: Students were able to see their statistics, data, etc. They
receive personalized feedback through visualizations so they can reflect about their
own learning and they can take actions on the MOOC of maths to improve their learn-
ing experience.
- Augmented information for teachers for the flipped the classroom methodology.
Teachers can know general tendencies about the students, know the specific problems
75
Trends in Digital Education:
Selected papers from EC-TEL 2015 Workshops CHANGEE, WAPLA, and HybridEd
of each student, etc. before the face to face lesson, so teachers can adapt the face to
face lesson to be more effective based on the data provided by ANALYSE.
- Evaluation of materials, students and the learning process. The ANALYSE in-
formation permit the evaluation of material, students and the learning process. Based
on these evaluations, different actions can be taken to improve the course for future
editions.
5 Use of ANALYSE in the classroom
ANALYSE can be used by both students and instructors with different objectives
in the classroom. Students can track their own progress in the Open edX courses with
visualizations that represent their own learning process. Therefore this feedback can
help them maintain motivation towards achieving their goals and discover more about
how they actually learn. They can know if they are progressing as expected and meet-
ing the requirements that can be established in order to surpass a course. It also shows
interest in self-regulated tools that can be related to the fact that they really want to
learn the contents of the course. On the other hands, instructors can use ANALYSE in
the classroom with several objectives. They can be tracking students individually, so
that they can have time to intervene before one of them drops the course with no pos-
sibility of resuming it back. They can check for students who are struggling in certain
areas of knowledge, i.e. a section of the course with deals with a specific topic, so that
they are able to contact the students with problems and help them reinforce this prob-
lematic section of the course. Also they can try to detect problems in the educational
resources for example exercise which hardness is about the rest of the course prob-
lems or they have been badly design attending to the concepts transmitted in the vide-
os, and also videos who are not very well recorded and students are having problems
to understand the concepts that are being delivered by the instructor in the recorded
lesson. In order to illustrate some of these possibilities in a real experience, we pro-
pose here a couple of example obtained from the real data obtained from the maths
MOOC which provides an idea about how to interpret some of the visualizations
available through ANALYSE.
In the first case we are going to check about a concrete student denominated as
“student A” to preserve their anonymity. What we are trying to represent here is the
natural flow that an instructor could follow in order to understand what is happening
with student A. First we check figure 2 for Students Grades visualization which
shows us the grade from 0 to 100 in each one of the sections of the course. Then we
can see that the scores in Units of measurement (100) and also Geometry (86) are
good, and the student obtained good grades, however for the Algebra section (44) the
student failed. So, the instructor might want to know more about this situation. A
possibility is that student A failed Algebra because he/she did not put enough effort,
another option is because he/she is really struggling with this topic. Therefore, in
order to deepen into this matter we check figure 3 for Course Accesses visualization
where we can check that the number of times that the student A has accessed Algebra
section is 10, which is not a big difference compared to Units of measure and Geome-
76
Trends in Digital Education:
Selected papers from EC-TEL 2015 Workshops CHANGEE, WAPLA, and HybridEd
try which are 12 and 17 respectively. Next we can check in figure 4 the Chapter Time
visualization where we can see that student A has devoted 64 minutes to Algebra
section, whereas he/she devoted to Units of measurement and Geometry, 48 and 81
minutes respectively (remember that student A completed these two other sections
with high scores). Therefore we can see that despite student A has made an effort on
Algebra section, he/she did not manage to surpass the problems. Therefore the in-
structor could conclude that it is probable that student A is actually having issues with
Algebra section in order to intervene and help the student with the most appropriate
method.
The second example that we want to propose here is related to find problematic
parts of a course. Our first step as an instructor was to check by the use of Students
Grades visualization the average grade of each one of the sub-sections on the course
(each section has several sub-sections). After this analysis we found out that the sub-
section with the lowest average score is one called “Area calculation” which explains
how to calculate the area of different shapes and belongs to the Geometry section.
Therefore, it would be interesting to check this more in depth in order to detect if
there are problems with this specific sub-section of the course. This sub-section is
composed of 6 videos explaining the area calculation method of different shapes with
their respective problems. The second thing we can check is the Video Time Watched
visualization in figure 5. We notice that the last 6 videos which are the ones that be-
long to the “Area calculation” sub-section are some of the most watched of the entire
course. We can check this detail by comparing the light blue bar with the ones of the
rest of the course. Additionally, the Course Accesses visualization revealed that the
“Area calculation” sub-section had 67 accesses by the entire class, which is actually
the highest number of accesses of all the course sub-sections. Therefore we can con-
clude that despite students have watched a lot of videos and also accessed this con-
crete sub-section many times, which means that they have put a considerable effort in
this sub-section, their grades in the “Area calculation” sub-section are still the lowest
ones in the entire course. Thus it is possible that there are some issues with the con-
tents of this sub-section, such as an elevate complexity of the topic, or too much con-
tent that might indicate that a revision and further analyses of the sub-section contents
would be advisable.
Figure 2. Students Grades visualization for student A.
77
Trends in Digital Education:
Selected papers from EC-TEL 2015 Workshops CHANGEE, WAPLA, and HybridEd
Figure 3. Course Accesses visualization for student A.
Figure 4. Chapter Time visualization for student A.
Figure 5. Video Time Watched visualization of the aggregate of all the students in the
class.
78
Trends in Digital Education:
Selected papers from EC-TEL 2015 Workshops CHANGEE, WAPLA, and HybridEd
6 Conclusions
This paper shows a way for combining a MOOC of maths, a flipped classroom meth-
odology, with a learning analytics module we developed in the context of adult high
school education. We show a general overview of how to take advantage of learning
analytics in this context and some initial examples of use of the ANALYSE module in
this MOOC. More analysis and use of visualizations are required for a complete eval-
uation of the course, students and materials.
Acknowledgements
This work has been supported by the “eMadrid” project (Regional Government of
Madrid) under grant S2013/ICE-2715, the EEE project (Spanish Ministry of Science
and Innovation, "Plan Nacional de I+D+I) under grant TIN2011-28308-C03-01, and
the mapaTIC project (Spanish Ministry of Education, Culture and Sports).
References
1. Santofimia Ruiz, J., Pijeira Díaz, H.J., Ruipérez-Valiente, J.A., Muñoz-Merino, P.J., Del-
gado Kloos, C.: Towards the Development of a Learning Analytics extension in Open
edX. TEEM conference (2014), Salamanca
2. Ruipérez-Valiente, J. A., Muñoz-Merino, P.J., Leony, D., Delgado Kloos, C.: ALAS-KA:
A learning analytics extension for better understanding the learning process in the Khan
Academy platform. Computers in Human Behavior. 47, 139–148 (2015).
3. Bergmann, J. & Sams, A.: Flip Your Classroom. ISTE/ASCD (2012).
79