Interaction is essential to promote learner engagement in MOOCs. However, the main MOOC platforms have important limitations regarding the interactivity of the built-in tools they provide. Consequently, teachers struggle to design MOOCs that foster learner interaction with the types of educational resources available. This paper presents a successful example of integration of an existing and interactive external tool in three MOOCs on computer science deployed in the edX platform. This tool is Codeboard, a web-based development environment which allows editing, compiling and executing code in different programming languages directly from the browser. This tool is integrated in edX through the well-known interoperability standard IMS LTI, which is supported by both the edX platform and Codeboard. After running the three MOOCs with 112 Codeboard activities included, results show an extensive use of this type of activity and a very positive evaluation on the usefulness of Codeboard by the learners.
MOOCs (Massive Open Online Courses) must be designed in such a way that they
are engaging to learners [
One of the ways that can help to increase learner engagement in MOOCs is the
promotion of interaction, both with the course materials [
However, most MOOCs replicate the traditional, and harshly criticized teaching
model of information delivery [
Nevertheless, when designing a MOOC, it is possible to consider the integration of
third-party external tools as part of the course, to generate a more interactive
environment. To this end, custom integration developments can be implemented, although
it is also possible to make use of standards, if supported by both the tool to integrate
and the MOOC platform [
This paper presents the integration through the IMS LTI standard of an existing
external tool, Codeboard (a web-based development environment) [
The remainder of this paper continues with a brief overview of the three MOOCs in which Codeboard was integrated. The following section explains how activities designed in Codeboard were integrated in these three MOOCs. Next, results taken from learners’ opinions and logs are presented and discussed. The paper finishes with conclusions and some future lines of work. 2
Teachers from the Departments of Telematics Engineering and Computer Science at
Universidad Carlos III de Madrid (UC3M) started in 2015 the design and
development of a series of three five-week MOOCs dedicated to introducing programming
using Java as the driving language. These three MOOCs were released under the
name “Introduction to Programming with Java” [
These three MOOCs were offered in English (videos also contain subtitles in
Spanish), and got more than 300,000 enrollees overall. Learners expected workload is
between 5 and 10 hours per week. Videos were an important part of the learning
materials, although these MOOCs were designed to include numerous interactive
activities, both using edX built-in tools, and externals tools, such as Codeboard. Each week
contained four learning sequences, whose core were videos and interactive activities,
a laboratory, which intended to propose a transversal development project for each
MOOC, and summative activities, which were used to calculate learners’ grade in the
course. A detailed description of the design of the course can be seen in [
The integration of Codeboard, more specifically the integration of activities in which learners must modify, compile and run a given Java code using Codeboard, has been possible because both Codeboard and edX support the IMS LTI standard. No specific developments were needed to integrate Codeboard activities in the MOOCs on “Introduction to Programming with Java”. However, some configuration steps were needed in edX before learners could use Codeboard. First, LTI had to be enabled in the advanced configuration settings of the MOOC in edX, adding two values for parameters key and secret (provided by Codeboard staff), to identify requests between the platform and the tool. Then, LTI activities were created where appropriate in the course, given them a name and a URL; each Codeboard activity created has a unique URL, which had to be copied and pasted in the corresponding LTI activity in edX.
The integration of Codeboard in edX is seamless for learners (see Figure 1), as Codeboard activities are shown within the learning sequence. Each learning sequence contains videos, exercises and others (under which category Codeboard activities fall), as it can be seen in the three types of icons of the top bar in Figure 1. Each Codeboard activity has an introductory text explaining the activity; this text is created with a built-in HTML component in edX; Codeboard activity is located below the text. Learners can modify the code, compile and run it as many times as they wish. However, every time learners refresh the page they go to the initial state of the activity (the default code provided by the teachers). Learners can, however, create an account in Codeboard, and if they are logged in, they will be able to save their progress.
Codeboard activities aim to practice and are not mandatory to pass the course.
Fig. 1. Seamless integration of a Codeboard activity in a learning sequence in edX.
Codeboard activities are distributed throughout the five weeks of each MOOC. Particularly, in each week, they can be found in any of the four learning sequences, or in the laboratory, which is a special additional learning sequence in which learners work in a project which is transversal to the MOOC, has incremental difficulty, and focuses on the specific concepts addressed in that week. Table 1 shows the distribution of Codeboard activities in the three MOOCs, indicating the number of Codeboard activities in the four learning sequences that make up each week and in the laboratory.
Several aspects are worth noting looking at Table 1. First, there are many
Codeboard activities throughout the three MOOCs. This was an intentional design decision
to foster learner interaction. Only week 5 of MOOC 2 does not include any
Codeboard activity. The reason is that this week is dedicated to introducing ethical issues
in software development, thus including other types of activities. Second, apart from
Codeboard, other external tools were used in these MOOCs, some of them seamlessly
integrated in edX, such as Blockly [
Total
TOTAL
Learners who reached the end of any of the three MOOCs had the possibility of answering an optional questionnaire in which they were asked about different aspects of that specific MOOC. Regarding Codeboard, learners were asked, in a scale from 1 to 5, about the usefulness of Codeboard activities to check their learning process (5 – very useful), and about the difficulty level of Codeboard activities (5 – very difficult). Figures 2 and 3 show the results for each of the three MOOCs. Regarding the usefulness of Codeboard activities, MOOC 1 obtained an average of 4.12 (std. 1.024, N=1075), MOOC 2 obtained an average of 4.04 (std. 1.004, N=137), and MOOC 3 obtained an average of 4.6 (std. 0.699, N=10). Regarding the difficulty of Codeboard activities, MOOC 1 obtained an average of 3.42 (std. 0.995, N=1075), MOOC 2 obtained an average of 3.29 (std. 0.925, N=137), MOOC 3 obtained an average of 4.1 (std. 0.738, N=10). Fig. 3. Difficulty of Codeboard activities in MOOC 1 (top) (N=1075), MOOC 2 (middle) (N=137), MOOC 3 (bottom) (N=10), from 1 (very easy) to 5 (very difficult) (Y-axis) and number of answers (X-axis)..
In these results, it stands out the decreasing sample size in the three MOOCs (an order of magnitude less in each subsequent MOOC). The explanation to this relies on several facts. First, these are the answers accumulated in all the editions a MOOC has been running (three runs for MOOC 1, two runs for MOOC 2, and one run for MOOC 3). Second, the number of enrolments significantly decreases comparing MOOC 1 and MOOC 2 and comparing MOOC 2 and MOOC 3. Third, a low number of learners (roughly a 5% of enrollees) reaches the end of this kind of courses. And, finally, filling in the questionnaire was an optional activity.
The effect on learners of integrating this tool in the MOOCs to foster interaction is positive, since the results indicate that learners consider Codeboard activities useful. Moreover, the level of difficulty of Codeboard activities, in general, is not a factor which contributes to learner frustration (which may happen if difficulty is very high), but is moderately challenging to engage learners.
Example of use of Codeboard activities Codeboard is also especially useful for MOOC teachers, allowing them to analyze in detail the interaction of learners with each Codeboard activity. The internal analytics provided by Codeboard shows the number of compilations and runs of code for each Codeboard activity, and their distribution over time. This allows MOOC teachers to detect if learners get stuck (i.e. Codeboard shows a high number of compilations but few executions of code), and take corrective actions, such as providing complementary explanations in the forums, or in the text that precedes the code.
Figure 4 shows an example of the information provided by Codeboard for one of the activities of MOOC 1. It represents the number of compilations (60,559) and executions (51,714) over time, with the three runs of MOOC 1 clearly framed; there is some activity between runs, as enrollees could keep working in the course, but no more new enrollments were allowed. The code provided by the teachers is simple and compiles correctly printing the resulting output on screen. No major problems are detected in this exercise due to the balanced relationship between number of compilations and runs, which is detailed in Figure 4 (bottom) for the first run of MOOC 1.
Fig. 4. Example of Codeboard activity from MOOC 1: full time span of the three runs (top) marked with red boxes, and time span of the first run (bottom).
Figure 5 shows another example in which an unbalanced relationship is seen between number of compilations (1748) and executions (848) (more than twice as many compilations). This example belongs to a Codeboard activity of the first week of the MOOC 3. Teachers should therefore provide additional clues to help to resolve the exercise, as an important number of learners got stuck and were not able to advance. This paper presented the integration of the Codeboard tool, through the IMS LTI standard, into three MOOCs on “Introduction to programming with Java”, deployed in the edX platform. The integration process was simple, since this standard is supported by both the Codeboard tool and the edX platform, and only high-level configuration adjustments were required. The aim of this integration was to foster learner interaction in an eminently practical MOOC, and where the MOOC platform did not provide purpose-specific tools for the subject taught. The fostering of learner interaction has as its ultimate goal increasing the overall engagement with the course. For this, 112 Codeboard activities were specifically created; learners positively evaluated these activities. All in all, MOOC teachers should explore how best to foster learner interaction and take advantage of existing external tools that can be integrated in the MOOC platform with this purpose.
Acknowledgements. This work has been co-funded by the Erasmus+ Programme of the European Union, projects MOOC-Maker (561533-EPP-1-2015-1-ES-EPPKA2-CBHE-JP), SHEILA (562080-EPP-1-2015-BE-EPPKA3-PI-FORWARD), COMPETEN-SEA (574212-EPP-1-20161- NL-EPPKA2-CBHE-JP), and COMPASS (2015-1-EL01-KA203-014033), by the Madrid Regional Government, through the eMadrid Excellence Network (S2013/ICE-2715), and by the Spanish Ministry of Economy and Competitiveness, project RESET (TIN2014-53199-C3-1-R). We especially thank Hans-Christian Estler and Martin Nordio for allowing us to use Codeboard in our MOOCs selflessly; this work could not have been done without their support.