It is widely known that interaction, as well as communication, are very important parts of successful online courses. These features are considered crucial because they help to improve students' attention in a very signi cant way. In this publication, the authors present an innovative application, which adds di erent forms of interactivity to learning videos within MOOCs such as multiple-choice questions or the possibility to communicate with the teacher. Furthermore, Learning Analytics using exploratory examination and visualizations have been applied to unveil learners' patterns and behaviors as well as investigate the e ectiveness of the application. Based upon the quantitative and qualitative observations, our study determined common practices behind dropping out using videos indicator and suggested enhancements to increase the performance of the application as well as learners' attention.
It is a common knowledge that interaction, as well as the communication, are
very important in uencing factors of students' attention. This indicates that
di erent possibilities of interaction should be o ered at a MOOC 1 in all possible
directions. So it is vital to o er some communication channels like e-mail or a
discussion forum, and in addition it is suggested that a form of interaction with
the content of the course itself is available [
The attention is considered as the most crucial resource for human learning
[
2 http://imoox.at/wbtmaster/startseite en/maker.html (last accessed Jannuary 2016)
Copyright © 2016 for the individual papers by the papers' authors. Copying permitted only for private and academic purposes. This volume is published and copyrighted by its editors.
SE@VBL 2016 workshop at LAK’16, April 26, 2016, Edinburgh, Scotland and is attended by both, school-teachers as well as people who educate children in non-school settings. It is scheduled in seven weeks with at least one video per week. A detailed analysis of the activity of the attendees at the videos is presented by Section 4.
Finally, this work aims to show how learning analytics could be done to monitor the activity of the students within videos of a MOOC. In other words, the research goal of this publication could be summarized to "using an interactive video platform to support students' attention and to analyze their participation". 2
In comparison to the approach shown by Section 3 there are other services providing similar features.
First, there is the possibility to use the built-in features of Youtube3 (e.g. text annotations or polls) itself. However, the di erent methods of analysis are very limited. A further tool is Zaption4 which provides various forms of interactive content for videos (e.g. multiple-choice questions) at planned positions as well as a rich set of analysis possibilities. Unfortunately, it shows the position of the interactions in the timeline of the video. This means that the users are able to jump from interaction to interaction without really watching the video. In comparison to that, a tool named EdTed5 also o ers the possibility to enrich a video with questions. However, the questions are not bound to a position in the video and furthermore, they could be accessed every time during the video.
The real-world pendant of interactive learning videos is known as ARS 6 ,
which enables the lecturer to present questions to students during the lecture in
a standard classroom situation [
To provide interactive learning videos a web-based information system called
LIVE 7 rst introduced by [
The rst ones are normal users who could be seen as students. They are only allowed to watch the videos and to participate to the interactions. Figure 1 shows a screenshot of a playing video which is currently paused and overlaid by an interaction (1). To resume playing, it is required to respond to the interaction which means that the displayed multiple-choice question has to be answered in this example. Furthermore, it can be seen that there are some other control elements on the right side of the videos (2). They could be used to invoke interactions manually. For instance, it is possible to ask a question to the teacher. [12]
In comparison to that, the users of the second group are equipped with teacher privileges. They are additionally able to embed interactions in the videos as well as to view di erent forms of analysis. To add an interaction, the teacher has to select its position within the video by using a preview of it or by entering the position. With a dialog, the teacher can embed multiple-choice questions or text-based questions in the video and furthermore, it is possible to add an image to a question. [12]
The analysis consists of several parts. At rst, there is a list of all students who watched the video and for each student in this list it is shown how much of the videos they watched. In addition, a chart shows the number of users (green) and views (red) across the timeline of the video (see Figure 2). This chart could be used to identify the most interesting part of the video. Furthermore, it is possible to access a detailed analysis of each student. It shows the timeline of the video and marks each watched part of it with a bar (see Figure 3). If such a bar is hovered with the mouse-pointer, additional information is displayed. This consists of the time of the joining and the leaving of this watched timespan in relative as well as the absolute values. [12]
In comparison to these forms of analysis related to the watching of the students, there is also a detailed statistic about the answers to the embedded questions. This means that for the multiple-choice questions, the answers of the students as well as their correctness is displayed. For the text-based questions, LIVE displays answers of the students and the teacher has to evaluate them manually because text-based answers are impossible to check automatically. [12]
The third group of users are researchers. They are able to download di erent forms of analysis as raw data. This means that they can select a video and obtain the data as a spreadsheet (CSV 8).
Finally, the following list aims to give a summarizing overview of the features
of LIVE [
9 short for Completely Automated Public Turing Test to Tell Computers and Humans Apart a calculation of an attention level to measure the attention of the students { raw data download for researchers 4
This section presents a very detailed analysis of the videos of the MOOC as well as of the multiple-choice questions. For that, the data provided by LIVE is evaluated using visualizations and exploratory analysis.
First, the delay of response to the questions provided by LIVE in the MOOC videos during the seven weeks is demonstrated by two gures. Figure 4 visualizes a box plot. The x-axis records MOOC videos during the period of the course, while the y-axis shows students' delay of response in seconds. This period was limited to 60 seconds. Students are categorized to certi ed students, who nished the course successfully and applied for a certi cate, and non-certi ed students. In this gure, we tried to study the di erence in behavior between both categories. In some of the weeks, certi ed students took more time to answer the questions such as in week 4 and week 7. For instance, certi ed students median in week 4 was 15 seconds, while the median for the non-certi ed students was 13 seconds. Furthermore, there was 3 seconds di erence in the median between certi ed and non-certi ed students in week 7. Additionally, the median in week 1 and week 5 are typically the same with an insigni cant variation between the rst and the third quartiles.
In comparison to that, Figure 5 visualizes a violin plot. The x-axis indicates students' status. This visualization summarizes the students' status and the delay of response time to the multiple-choice questions in all of the MOOC videos. The thickness of the blue violin shape is slightly wider than the red one in the (8-13) seconds range, which indicates the more time needed to answer the questions. In addition to that, the non-certi ed violin shape holds more outliers attributes than the certi ed division. It is believed from the previous two observations, that certi ed students took less time in answering the questions in general. This case can be explained as the questions were easy to answer if the student were paying enough attention to the video lectures.
Figure 6 displays the timespan division in percentage and the timing of the rst multiple-choice question represented as a vertical dashed line. Using this visualization, we can infer the relevance timing of the rst question to describe the drop rate during videos. The questions were programmed to pop up after 5% of any MOOC video. Students may watch the rst few seconds and make skips or drop out after that [13], and this can be seen in the plot where students are dropping in the early 15% of the videos. To grab the attention of the students and maintain a wise attrition rate, the multiple-choice questions were intended to be shown randomly in the high drop rate scope. Further, week 6 was tested to check the postponed question e ect on the retention rate. The data in the gure also shows that students do not drop out a learning video in the range between 20%-80%, unless they replay it on that period and spend time on a particular segment to understand a complex content. The promising outcomes are seen with a stable attrition rate in the last four weeks when students are o ered an interactive content during the video indoctrinate process.
In Figure 7, the data is displayed in order to trace the video drop ratio of each second in every video. The x-axis displays the percentage of videos. The colored points specify the video name and the watchers count. While the black shadowed points indicate number of views. For instance, it is obvious that the data in the rst three weeks shows more views per user which can be explained as an initial interest of the rst online course weeks. On the other hand, the views nearly equaled the number of users from week 4 to the last week. Another interesting observation is the slow drop rate during the videos in all of the weeks despite the high drop in the last 2-3% of every video. A clari cation of such attitude is due to the closing trailer of every video which most students jump over it.
Due to the independency of the examined MOOC, each video of this course
does not rely on the previous one. The activity of every video varies in every
week. For this reason, Figure 8 shows activity of the total number of stop and
play actions in the MOOC videos. The blue points denote the certi ed students
while the orange ones denote the non-certi ed students. In fact, the rst three
weeks re ect proper enthusiastic count of actions. We realized that there was a
distinct activity by the non-certi ed students in week 5. A reasonable clari
cation is because of the interesting topic of that week which was about 3D-Printing.
However, their engagement becomes much less in the last two weeks, as this was
proven in other MOOCs' videos analysis [
Catching the attention of learners in online videos of MOOCs is an intriguing argument across learning analytics discussions. With this publication, the usage of an interactive video platform presenting videos of a MOOC is shown. It points out the main functionalities of this platform as well as the participation and the activity of the students. Additionally, we demonstrated an evaluation of this system in order to examine its performance and describe the behavior of students. Finally, the results show that the main concept behind latching on the students' attention becomes attainable through evaluating the questions' content and the interactions timing. 12. Wachtler, J., Ebner, M.: Support of video-based lectures with interactionsimplementation of a rst prototype. In: World Conference on Educational Multimedia, Hypermedia and Telecommunications. vol. 2014, pp. 582{591 (2014) 13. Wachtler, J., Ebner, M.: Impacts of interactions in learning-videos: A subjective and objective analysis. In: EdMedia: World Conference on Educational Media and Technology. vol. 2015, pp. 1642{1650 (2015)