Massive Open Online Courses (MOOCs) have been a hype in technology enhanced learning systems the last couple of years. The promises behind MOOCs stand on delivering free and open education to the public, as well as training a large criterion of students. However, MOOCs clashes severely with students dropout which by then forced educationalists to deeply think of MOOCs effectivity from all angles. As a result, the authors of this paper propose a pedagogical idea that strongly depends on injecting the online learning (MOOC) with face-to-face sessions to refresh the students minds as well as integrating them in the real learning process. The authors after that analyze the results of their experiment using Learning Analytics. The outcomes have shown a new record of certification ratio (35.4%), an improvement of student interaction in the MOOC platform, and a manifest in social interaction in the MOOC discussion forum.
Massive Open Online Courses, shortly MOOCs, are still on the move and on top in
any discussions in the research area of Technology Enhanced Learning. Nevertheless,
we can look back now for more than 8 years now, when George Siemens and Stephen
Downes started their first trials on open global online courses [
In this publication, we address the research question what can we learn from analyzing learners’ data of a MOOC that is supported by the didactical approach of Inverse Blended Learning?
This paper is organized as follows: Section 2 shows the research design of the Inverse Blended Learning. Section 3 describes the structure of the studied MOOC, while section 4 reveals how did we collect, analyze and interpret students data using the iMooX Learning Analytics software. Finally, section 5 discusses and concludes the findings. 2
The research design of this article follows the case study design concept. We implemented a MOOC, which is based on the concept of Inverse Blended Learning and ran it for 6 weeks. Overall, we can summarize four different phases (lasting more than one year) and are pointed out as the following: • Concept phase (I): The concept of the MOOC as well as marketing strategies and first trials have been done. • Preparing phase (II): The content of the MOOC was prepared and first instructions to face-to-face trainers were given. • Execution phase (III): The MOOC becomes online and is being launched. • Completion phase (IV): Final work was done (e. g. emails to learners who didn’t get it in time) as well as final analyses of the evaluation To measure the results respectively and answer the scientific questions concerning the concept of Inverse Blended Learning, different kind of data collection was carried out. There was an initial questionnaire at the beginning of the course, and a final evaluation form to all successful learners, interviews with trainers and learners as well as tracking learners’ data on the iMooX server.
Tracking students on the Austrian MOOC platform has been done to help decision makers like researchers and instructors to make proper interpretation of students bulk data. The Learning Analytics tool was developed based on four stages [9]. In brief, the first stage includes the data generation, and this is assuredly done by the students who produce huge clickstreams on the platform. The second stage is the data collection. A dedicated web server was enabled to track students and collect their left traces such as their login information, quiz inputs, downloads,…etc. Following that, the third stage involves processing the data and tidying them up from any noisy and unwanted stacks. Finally, the fourth stage is the interpretation and the optimization phase. Usually, the last stage could be translated into interventions, decisions, amendments in the instructional design or a set of recommendation for future offered courses. 3
The offered course was called “Erwachsenenbildungs-MOOC” (short EBmooc); translation: adult education professionals MOOC) that addressed adult learning professionals as a main target group in the German speaking area. The content of the course was about digital tools, which can assist adults through their daily business or their daily life. The MOOC started on the 6th of March 2017 and lasted for six weeks. In details, the MOOC was offered with videos and some additional learning resources on the following topics: • Week 1: Learning with MOOCs • Week 2: Digital Tools for daily work • Week 3: Social media in adult education • Week 4: Blended Learning and Technology Enhanced Learning • Week 5: Open Educational Resources • Week 6: Online consulting In addition, four webinars were given to deepen the learning contents and answering questions.
As mentioned before, the course followed the Inverse Blended Learning concept. This concept reverses the original concept of Blended Learning, where face-to-face education is interrupted by online elements up to 30-50% of the course duration. Typically, a course starts with face-to-face meetings, followed by online elements, face-to-face elements etc. Inverse Blended Learning turns this idea completely, by bringing a complete online course back to face-to-face situation. Therefore online elements are interrupted by real life situations or local regularly training supporting the online course (see Fig. 1). Similar to Christensen et al. [12] Inverse Blended Learning can be seen as one explicit scenario within their Blended Learning Matrix.
The summary of our experiment is as the following: more than 40 trainers supported the MOOC with local trainings. In other words, learners had the chance to meet other learners in those offered groups for discussions or information exchange. On average, local groups were offered on a weekly basis and most of them were located in typical adult education training centers spread over the German speaking area. At some places, learners had to pay a small fee and there was one completely online group, too. The number of participants varied from 2-12 learners. The corresponding trainers got an additional train-the-trainer package for about 2 months before the launch of the MOOC. An introduction to the concept of Inverse-Blended-Learning was given about the content of the provided MOOC. The trainers got the possibility to access the MOOC one month earlier so that they could start their own preparations. 4
In this section, we provide details and analysis of the studied MOOC. In order to do so, we have enabled the iMooX Learning Analytics software to track students over the MOOC platform. By following the four stages in section 2 of this study, we provide outcomes. Starting up from the launch of the EBmooc till the last offered day, the Learning Analytics tool has provided us with over 200,000 data records of student logs. We have cleaned the records from any unnecessary data bulks, since the data in the log files was unstructured, duplicated and not regularly formatted. We processed the information after that using the R software (https://www.r-project.org), version 3.40.
The Learning Analytics tool translates low-level collected data into useful information. The tool follows vital stages to prepare the data for analysis. 1. The first stage incorporates when the data are generated by the students. The records are then saved on a web-server for further analysis in a later step. 2. The second stage involves filtering the stored scripts in the web server and thereafter distilling main activities such as the number of tried quizzes, the number of discussion forum reads and posts as well as the total number of logins. 3. The third stage incorporates storing the filtered data in another protected server for the analysis stage.
After having the data prepared, it was one of the first steps to look at the general description of the studied MOOC participants (see Table 1). In the initial analysis of counting the enrolment types, we defined four types of students:
1. Registrants: The total number of registration in the MOOC.
Active: and those are the students who were more involved in the MOOC, we defined this cohort members by those who log in, did some activities like attending a quiz, making a couple of forum reads or writing a post/comment. Passive: and those are the student who just signed up in the MOOC and did very minimal activities like a single login.
Certified: the students who successfully completed all the requirements and passed the quizzes of the MOOC and applied for a certificate at the end.
Registrants Active Passive Certified (type I/type II) The summary in Table 1 showed that there were 3,064 registrants in the MOOC. The number of inactive students was 817 students. However, the active students were 2,247. The students who were handed certificates were 1,083 students, which mean a ratio of 35.35% of the total registrants. Whilst if we took into consideration our special typecast, the certification ratio is 48.20% with reference to the total number of active students in the EBmooc.
Next, we tried to sum up the total number of the available activities and interaction that have been mined by the Learning Analytics software in the MOOC. Table 2 depicts four types of MOOC interactions. The values in the table below are shown based on 95% of the data. Given that, we tried to avoid interruptions and negative impact by the outliers.
The first activity that we tracked is the logins, which indicate the log in frequency that each user has done during the active days of the MOOC. In our case study, the total number of logins is 18,812 with an average of 7.17 per student and the standard deviation is (σ=6.95).
logins forums reads forums posts quiz attempts
The second activity that was mined is the discussion forum events. What has been interesting is the extreme number of forum scans (reads) that exceeded any prior provided MOOC in iMooX. Arithmetically, there have been 66,661 discussion forum reads done by the students. The mean value for this activity is 29.21 and a noticeable high standard deviation of (σ=79.06). On the other side, we looked at the discussion forum comments and posts that were done by the students. Table 2 shows that there has been 885 forum posts. These posts can take the format of comments, replies, and new threads. The average number of forums posts is 4.58 while the standard deviation is (σ=9.97).
Last but not least, we investigated the performed quiz attempts by all the registrants in the MOOC. With over 35,000 quiz attempts, the students showed a vast level of fluctuation in dealing with the quizzes. With an average value of 17.32 quiz trials per student and a standard deviation of (σ=12.74), we move to our next stage of analysis.
Provided that our reference for success in this MOOC is the students who are certified, it was wise to examine the certified students interaction sequences. As a result, we filtered our data upon this request and present the following table (see Table 3).
Login F. reads F. posts Quizzes Certified students activity and interaction summary are provided in Table 3 showing the minimum value, first quartile, median, mean, third quartile, the maximum value and the standard deviation. The first quartile can be defined as the middle number between the minimum value and the median in the data set. The third quartile is the middle point between the median and the maximum value.
In the first place, the login facts display that the average number of logins of the certified students during the six weeks was of 12.31, a median of 11 and σ=6.602. We also noticed some quiz miners in the MOOC similar to several publications provided before [7,9]. This can be tracked by those who have less than the minimum expected logins (6 times, 1 per week). Next, the forum reads demonstrated a quite high number of activities the certified students performed during this course. The table shows that the average number of reads was 43.49 per student. On the other hand, there were very active social participants such as those who read over 1,000 reads. The standard deviation of 95.102 is quite high because of the high value of some students who read intensively. On the other way around, the discussion forum posts show a minimum thread input of 1 post, a maximum of 90, a mean of 5.12 and a standard deviation of (σ=11.592).
The quizzes part displays that the certified students did an average of 25.96 quiz attempts during the six weeks long of the MOOC. The maximum was 47, the median was close to the mean, while the standard deviation was (σ=10.09).
Provided that social interactivity has been fundamental in this MOOC (see section 2), we have used the log data to analyze the forum reads and posts on each week for all the enrolled students (see Fig. 2 and Fig. 3).
The social interactivity in the EBmooc holds great potential and was seen as a key difference in comparison to previous offered MOOCs on the iMooX platform. Given that most social interactivity like reading in the forums usually is dropped up to the half in the mid-range of offered MOOCs, the EBmooc reading was unalike. As seen in Fig.2, There have been over 8000 reads in the forum in the first week, followed by increased reads of around 8,800 reads in the second week. The third week holds a stable frequency of reads similar to the first week. Nevertheless, the fifth week recorded the lowest number of reads dropping from 8,000 to 4,300 reads. While the general direction is declining, the last week of the MOOC recorded the largest number of reads. The sixth week has over 9000 numbers of reads.
Fig.3 depicts the number of posts in the EBmooc digital forum. We noticed that there were two peak points in the figure. These peaks are in the first and in the second week, respectively. The first peak point was on the 15th.March.2017 with 100 posts, and the second peak point was on the 20th.March.2017 with 70 posts. 5
This publication has introduced the Inverse Blended Learning concept in short. Unlike normal xMOOCs, the new concept of this research has been carried out on one MOOC offered on the Austrian MOOC Platform (iMooX) with great potential behind. Inverse Blended Learning focuses on integrating online learners into live faceto-face sessions like local round tables, central meetings, webinars, and a high level of preparation such as advertisement, training, and distributing printed materials. The secret outcome word is (“uniqueness”). This experiment was unique with several results such as the high level of online interaction, high certification rate, and a remarkable ratio of active students. By using Learning Analytics and tracking the successful (certified) students, we were able to identify an action sequence. Remarkably, certified students logged in, in average, 12 times, read 43 times in the forums and posted 5 times. This indicates that online learners impelled by the Inverse Blended Learning concept has driven students to stay active during the weekly online sessions. The impact of the Inverse Blended Learning was relevant based on our observations in Fig.2 and Fig.3:
Fig. 2 and Fig. 3 pointed clearly out that the concept of Inverse Blended Learning induced a very high commitment by the participants. Based on the regularly offline meetings, learners are motivated to do their tasks and they also liked to discuss their experiences with peers and trainers.
Fig. 3 displays the high number of posts even in the second week. This was the week of the “digital tools for adult education”. The participants’ learners were able to try them (digital tools, …etc.) on themselves. Many learners expressed their willingness of using different applications in the posts.
The concept of Inverse Blended Learning was following the idea to enhance social engagement duo to learning is highly social process. It seems that the combination of off- and online was a perfect mix.
On the other side, even the trainers reported back that learners were motivated and engaged in their trainings, deep discussions on different experiences often took longer than the planned weekly hours.
A very interesting fact is also that the offline trainings differed arbitrarily. One training even happened online in a webinar room. 6
The concept of Inverse Blended Learning worked with high success. Not only the certification ratio was significant higher, many learners expressed their satisfaction and showed great enthusiasm to attend another MOOC in the near future. The learning experiences have led to even receiving many emails from those who did not finish the MOOC. They expressed that they would love to do the course in the next run more intensively. Nevertheless we have to point out that, of course, the preparation effort is very time consuming, but it seems to be worth it! 7
The Inverse Blended Learning can be limited with a large number of students involved. Given that even splitting the participants into multiple cohorts may provide a temporal solution for organizing this teaching methodology, the cost of workshops and training is considered as one barrier. Furthermore organizational efforts increasing arbitrarily.
We express our gratitude to the Austrian Federal Ministry of Education for the support and funding of the MOOC project. 6. Daniel, J.: Making Sense of MOOCs: Musings in a Maze of Myth, Paradox and Possibility. In: Journal of Interactive Media in Education, 3 (2012), http://wwwjime.open.ac.uk/jime/article/viewArticle/2012-18/html, last accessed 2017/08/07 7. Khalil, M., Ebner, M.: When Learning Analytics Meets MOOCs - a Review on iMooX Case Studies. Innovations for Community Services: 16th International Conference, I4CS 2016, Vienna, Austria, June 27-29, 2016, Revised Selected Papers. G. Fahrnberger, G.
Eichler and C. Erfurth. Cham, Springer International Publishing: 3-19 (2016). 8. Ebner, M., Lorenz, A., Lackner, E., Kopp, M., Kumar, S., Sch ön, S., Wittke, A.: How OER enhance MOOCs – A Perspective from German-speaking Europe. In: Open Education: from OERs to MOOCs. Jemni, M., Kinshuk, Khribi, M. K. (eds.). Springer. Lecture Notes in Educational Technology. pp. 205-220 (2016) 9. Khalil, M., Ebner, M.: What Massive Open Online Course (MOOC) Stakeholders Can Learn from Learning Analytics? In: Spector, M., Lockee, B., Childress, M. (Ed.), Learning, Design, and Technology: An International Compendium of Theory, Research, Practice, and Policy, Springer International Publishing, pp. 1-30 (2016) 10. Ebner, M., Scerbakov, A., Kopp, M.: All About MOOCs. Digital Medien in Arbeits- und
Lernumgebungen. Jost, P, K ünz, A (Ed.). S. 148-155. Pabst, Lengrich (2015) 11. Ebner, M., Schön, S., Käfmüller, K.: Inverse Blended Learning bei „Gratis Online Lernen“ - über den Versuch, einen Online-Kurs für viele in die Lebenswelt von EinsteigerInnen zu integrieren. In: Digitale Medien und Interdisziplinarität. Nistor, N. & Schirlitz, S. (Hrsg).
Waxmann, Medien in der Wissenschaft Bd 68. pp. 197-206 (2015) 12. Christensen, C. M., Johnson, C. W., Horn, M. B.: Disrupting Class: How Disruptive Innovation will Change the Way the World Learns (1 ed.). Mcgraw-Hill (2008)