=Paper=
{{Paper
|id=Vol-1346/paper9
|storemode=property
|title=The MOOC Hype: Can We Ignore It? Reflections on the Current Use of Massive Open Online Courses in Software Modeling Education
|pdfUrl=https://ceur-ws.org/Vol-1346/edusymp2014_paper_9.pdf
|volume=Vol-1346
|dblpUrl=https://dblp.org/rec/conf/models/StikkolorumDZBG14
}}
==The MOOC Hype: Can We Ignore It? Reflections on the Current Use of Massive Open Online Courses in Software Modeling Education==
https://ceur-ws.org/Vol-1346/edusymp2014_paper_9.pdf
The MOOC Hype: Can We Ignore It?
Reflections on the Current Use of Massive Open Online
Courses in Software Modeling Education
Dave R. Stikkolorum1 , Birgit Demuth2 , Vadim Zaytsev3 , Frédéric Boulanger4 ,
and Jeff Gray5
1
LIACS Leiden University, The Netherlands
d.r.stikkolorum@liacs.leidenuniv.nl
2
Technische Universität Dresden, Germany
Birgit.Demuth@tu-dresden.de
3
Universiteit van Amsterdam, The Netherlands
vadim@grammarware.net
4
CentraleSupélec - Département Informatique, France
frederic.boulanger@centralesupelec.fr
5
Department of Computer Science, University of Alabama, USA
gray@cs.ua.edu
Abstract. At the end of the 2014 MODELS Educators Symposium, a
panel discussed the topic of the use of MOOCs (Massive Open Online
Courses) in model-driven engineering education with the audience. Cur-
rently, there are no MOOCs that target software modeling. The panel
argued if it would be worthwhile to investigate the idea of using MOOCs
for modeling courses. Participants had different positions about the ad-
vantages and disadvantages of MOOC use in software engineering and
modeling. This paper summarizes our positions on the current use of
MOOCs in the teaching of software modeling and on its future use. We
identified the need for a MOOC targeted at modeling to evaluate its
benefits in the future. We agree about the challenges we face, such as
time and costs, when developing qualitative MOOCs. We see possibili-
ties to integrate MOOCs into existing lectures or into flipped-classroom
settings. A key focus should be on intensive interaction between students
and lecturers.
Keywords: online education, software modeling, MOOC, Massive On-
line Open Course, e-learning
1 Introduction
Since 2005, the International Conference on Model Driven Engineering Lan-
guages and Systems (MODELS) organizes the Educators Symposium. During
this symposium, lecturers from all over the world discuss education topics re-
lated to model-driven engineering (MDE). These discussions considered new ed-
ucational insights, current problems and challenges for the future. In 20146 , the
6
http://models2014.webs.upv.es/educatorsym.htm
subject of discussion was the use of Massive Open Online Courses (MOOCs) in
our educational practice. The discussion about MOOCs was driven by a panel
that was organized at the end of the symposium. To start the discussion the
following abstract (first paragraph) of the panel introduction was published:
The current MOOC hype is already the second incarnation of the idea that
electronic media and the internet can help to decrease the cost of education.
While the mediocre outcome of massively funding research on e-learning (as it
was called during the first wave washing over us at the turn of the century)
has caused considerable disappointment, the idea of online courses seems to have
wintered the depression that followed, and has recently resurged under the guise of
democratizing higher education. Not surprisingly, the key insight of the first flood,
that learning profits from teaching and that teaching means human interaction
involving an actual teacher, is currently being rediscovered.
The critical point of view of the introductory speaker, Friedrich Steimann,
caused the panel and audience to discuss different aspects of the current use of
MOOCs in software modeling courses.
In this paper the organizers, panelists and audience have summarized their
views on the current application of MOOCs and give suggestions for future use.
Their view is summarized in the form of positions and therefore this paper does
not claim to give the ultimate status quo or the overall evaluation of MOOCs in
software engineering education.
In Section 2, we list questions that the panelists had about MOOCs and
explain the context of each question. In Sections 3 to 7, we present our different
positions. In Section 8, we conclude and propose future work.
2 MOOCs
The interest for popular MOOC initiatives such as Coursera, Udacity, or the
MOOCs from universities like MIT and Stanford has grown enormously. These
initiatives offer MOOCs on programming-related subjects such as programming
in C++/Java, algorithms. To our knowledge, no specific courses on software
modeling are offered. Do we want to see them in the (near) future? And, if so,
what do we have to consider?
Although some observed 2012 to be ‘The Year of The Mooc’ [11] because of
growing interest and popularity, there are many others who have questioned “Is
software engineering ready for MOOCs?” [2]. Dasarathy et. al. discuss different
topics in “The past, present, and future of MOOCs and their relevance to soft-
ware engineering” [3]. In 2014 the topic of MOOCs still gives us enough reason
to discuss and/or criticize. Based on what is emphasised in the literature, in this
paper the positions focus on one or more of the following questions that help us
to decide whether developing a MOOC is worthwhile:
Will a MOOC save costs? While some argue that MOOCs will be used for
cost reduction, Schmidt et al. [14] show the effort that has to be put in the
development process of a MOOC and mention “It took us two solid months of
filming to produce 80+ individual videos that ran for 20 hours. In contrast, 40
hours are spent lecturing in a conventional semester-long Vanderbilt class”.
How is it possible to grade students’ assignments? A massive approach
needs a different way of grading assignments. Two methods are mentioned often
in literature: peer review and automated grading. Tillmann et. al. [17] explored
the second one in their game based software engineering MOOC.
How to maintain the quality of a course? As indicated by the panel intro-
duction, MOOCs may give universities the idea they could replace the lecturers
and thereby decrease the communication between students and lecturers. Fox et
al. [6] argue MOOCs can even improve students’ engagement (along with other
subjects connected to MOOCs) by introducing SPOCs (Small Private Online
Courses).
The positions in this paper continue the discussion above and aim to reflect
on our current use of MOOCs in the software modeling context and explore
possible future use.
3 We should distinguish between kinds of MOOCs and
leverage MOOC content and design principles for
flipped classroom courses (Vadim Zaytsev)
It is widely known that MOOCs or any other automated content delivery/ con-
sumption system, opens many doors for learning analytics. It should be pointed
out that some statistics work differently for the MOOCs: for instance, such
courses usually score an exceptionally high satisfaction extent of around 80–
90% among the students — this is at least partly due to the evaluation being
done among the students who actually succeed in finalising the course. Filtering
occurs on each stage. Consider an example: 17000 registered, 7000 watched a
lecture video, 2000 watched all lectures, 1000 did some homework, 500 took the
exam [12]. Among students who start being engaged in some way beyond simply
expressing interest, a success rate of 20–45% is typically considered acceptable.
Increasing active teacher involvement does not significantly improve the rates
either [18]. For normal presence courses such percentages are usually consider-
ably higher, which means that less dissatisfied students drop out before the final
evaluation. Hence, pro-MOOC arguments based on their successful evaluation,
should never be considered out of their context.
One of the recent trends in distance learning and massive online courses is
recognizing the distinction between xMOOC and cMOOC: the former are, in
Keith Devlin’s words, “textbooks on steroids”, basically e-learning platforms for
delivering knowledge and distributing learning activities over space and time,
while the latter are collaborative efforts involving teaching staff and an in-
terested community. xMOOCs gravitate toward concrete decisions, leading to
frameworks such as Coursera, MiriadaX, Udemy, Udacity, Edx. They are easy
to assess, compare and ultimately draw a conclusion whether they should be
ignored, adopted or feared, based on the compatibility level of desired learning
objectives/activities with the functionality offered by the framework. cMOOCs,
on the other hand, are harder to measure and to compare, since they are built
around interaction among students, not between students and content.
cMOOCs in general are built on top of several well-known concepts such as
“autonomous learning” [9] (creating an environment that encourages learning),
“self-determination” [13] (rewarding learning activities in a continuous tangible
way) and “computer-supported collaborative learning” [8] (cooperative form of
education when subjects learn from one another while assisted by automated
tools). Social and web 2.0 technologies make it rather unproblematic to set up
and the existing research on gamification helps to properly design learning ac-
tivities. Supporting yet extracurricular activities such as teachers’ blogging (in
addition to compulsory lecture videos), social exposure of results (whose solu-
tion gets the most likes?) and discussion boards generally receive positive eval-
uation [7] and have been shown to improve students’ results [5]. What is also
interesting: students who have already completed such a MOOC, express in-
terest in maintaining the community beyond the course [7] and actually stay
subscribed to the community at least for some time [5]. (This is not that sur-
prising and corresponds to the experience we have had with informal activities
and social network maintenance of learning activities). What is also important,
the most active contributors of such online communities exhibit approximately
the same statistical traits as their less active peers (so they do not alienate the
silent majority and do not turn to trolling) and globally produce high quality
content [10].
In either kind of MOOCs, students often demand other assessment methods
besides traditional testing, such as detailed feedback and peer reviewing [7].
Some of these requests are impossible to fulfil due to the massive nature of the
course, but for the rest of us it means that the setup can be reused for the flipped
classroom paradigm: it is not massive, usually under 100 students per course, it
relies on autonomous consumption of content, it profits from online community
support, but at the same time can utilise instructors’ time to answer questions,
provide feedback and further adapt to students’ demands. Indeed, leveraging
MOOC content for the flipped classroom has been pointed out earlier as one of
the options for traditional educators [15].
4 MOOC tools can be used to improve traditional
teaching by motivating students to learn by themselves
and by making a better use of the time spent in
classrooms (Frédéric Boulanger)
MOOCs are often considered only as tools for delivering online education to
large virtual classes. However, it is still difficult to assess their efficiency for this
task and compare them fairly to more classical classroom settings. Another po-
tential use for the technologies that were developed for MOOCs is to improve
traditional teaching by reducing the time spent in classrooms, raising the moti-
vation of students, and taking advantage of cooperative work of students. The
ideas exposed here come from my experience as a professor in a French “grande
école” (engineering school with a competitive entrance examination) in the field
of Computer Science.
Reducing the Time Spent in Classrooms Many educational organizations have
shrinking budgets and may try to save money by increasing the number of stu-
dents per teacher. However, it is also possible to reduce the number of hours spent
in classrooms while maintaining a good supervision ratio. In many courses, there
are parts that can be self-learned by students and that are a pain to teach in
classrooms. For instance, learning the basics of a programming language is bet-
ter done by reading a book and running examples and exercises on a computer
than by listening to a teacher or looking passively at slides. However, simply
asking students to read a book and make exercises before attending a lecture
may lead to failure because of a lack of motivation. Using online material and
making the students work on a project that they can achieve by learning from
the online material works better, and it prepares the students to learn more
advanced topics during the lecture. It also relieves the teacher from describing
boring details about the syntax of a language or the basics of some scientific or
technical field.
The technologies associated with MOOCs can be of great help for this kind of
online teaching because they allow for interactive content, and even personalised
content according to the results of each student. The difficult part is to find the
right sequence of classroom sessions and online work. A first lecture is necessary
to explain how the class works and what is expected from the students. Some
feedback by a human teacher is also necessary to keep the students motivated
and help them to avoid getting stuck in their progression. However, this feedback
can be reduced by encouraging cooperative work among students.
Motivation and Cooperative Work Working on a project can be a good moti-
vation for self-learning, and the availability of online material and immediate
feedback to the progress of the students can drive this motivation. It can be im-
proved further by encouraging cooperative work among students. I often notice
that, during practical lab sessions, students help each other to achieve the goal
of the session. It can happen because I am already busy with another group,
or just because those who understood some point are willing to explain it to
others, and also because students sometimes prefer to learn from other students
than from a professor. When such clusters of students gather in front of a screen
and start discussing a topic, I keep an ear on what they say in order to fix
any possible misunderstanding, but I generally do not have to interfere in their
discussion because they find their path to a correct answer, sometimes through
detours that I would not have thought of.
Using online material with problems to solve and immediate feedback about
the correctness or the quality of the solution can therefore be made more efficient
if the students are encouraged to work together. One possibility is to organize
their work in sessions where they meet during predefined time slots. It seems that
having to report on their progress to an educator is also a source of motivation.
Advantages and drawbacks The main advantage of using MOOC-like online ma-
terial associated to projects for self-learning is that when students come to the
next classroom, they already know the basics of the field and have been exposed
to issues that make them more sensitive to what you will explain during the
lecture. Other advantages are less boring lectures, both for the students and the
teacher, and the increased motivation that comes from achieving different goals
during the projects and exercises. Competition between groups of students to
develop the best algorithm or the most efficient solution to a problem can also
drive motivation.
There are minor drawbacks, the main one being the extra amount of work
required to prepare the online material. You cannot just put the equivalent of
a text book on a web site. You have to prepare a series of exercises that lead
students toward some knowledge of the field you teach, and some projects to
make them use what they learned to solve exciting problems. For this, you need
to discriminate automatically between correct and wrong answers, and detect
early when they take a path that will prevent them from succeeding in their
project. Another issue is to detect the loss of motivation. Contrary to MOOCs
that can drop 90% of the students between the initial subscription and the final
exam, I have to take all my students to the end of the journey, with an expected
rate of failure of less than 10% at the exam. Tutoring by a real educator seems
to be necessary to avoid students getting lost and coming (or not coming) to the
lecture without mastering the assumed prerequisites.
Conclusion One of the main challenges I am facing when teaching algorithms,
programming, system modeling, programming languages semantics or computer
architecture is to make students interested in the topic I have to teach. Focusing
on practical sessions proved to be efficient, but students tend to be demotivated
by the initial learning phase where they have to get familiar with the basic
notions on which the rest of the course relies to explore more advanced and
interesting topics. Using online resources which allow them to find the required
information to make exercises and solve problems, and providing them with tool-
kits for evaluating their results, allows the students to learn at their own pace,
with a more focused attention. MOOC technologies can therefore be used to
replace some lectures by self-training sessions, and to keep classroom time for
more advanced topics to which the students have been made receptive through
their own experiments. Encouraging cooperative work among students by orga-
nizing scheduled work sessions is also a great way of taking advantage of their
spontaneous tendency to share the knowledge they have just acquired. However,
at least in my context, interaction with an educator is necessary to keep students
motivated and to show them how advanced concepts can be useful for solving
some of the issues they met during the practical sessions. Preparing online ma-
terial and the tool-kits used for practical sessions also requires more work than
preparing slides for a regular lecture, but spending less time in classroom and
having more interested students may be worth it.
5 We should experiment with multiple forms and levels
of MOOCs to understand how to best teach modeling
in the future (Birgit Demuth)
The MOOC hype is discussed controversially in Germany. There are only a few
professors and universities who support and drive the development of MOOCs.
There are several reasons for this observation. Academicians are concerned about
the necessary related drastic change in education. Related issues are high launch
costs, the underlying business model and privacy of student data. Furthermore,
teachers wonder how they are able to ensure the educational quality of their
courses and to automatically grade the performance of students. The academic
position to MOOCs was studied and summarized in the Dagstuhl Manifesto
2014 [4]. Eight partially provocative positions were identified and explored by
the international workshop participants. The included state of the art report
confirms my observation that most of the MOOCs both on commercial and
non-commercial MOOC platforms focus on the “end of high school - first year
of university” level. For example, one can find quite a few MOOCs on basic
programming principles. But to the best of our knowledge, there is no MOOC
that teaches object-oriented modeling concepts in the form of an xMOOC or
cMOOC (explained in Section 3). Most generally, modeling skills are typically
not the subject of the first year of a computer science curriculum. However, we
see another reason for the scarcity of modeling MOOCs: modeling is a higher
thinking skill. In the framework of the revised Bloom’s taxonomy of educational
objectives [1] the verb model means to create something and therewith can be
classified as an activity of the highest level of cognitive processes. Besides the
ability to abstract complex facts, modeling requires social skills such as com-
munication with different stakeholders and conflict resolution skills. A modeling
MOOC has to rely on very high social interactivity to produce positive outcomes.
This primary challenge is strengthened when one considers that future stu-
dents belong to the digital natives for whom it is normal to use electronic ed-
ucational technology. We should experiment with multiple forms and levels of
MOOCs to gain the experience on how to teach modeling the best in the future.
MOOCs could help to manage courses with large-scale enrollments. But before
we start, we have to establish the organizational requirements. A meaningful
alternative way may be to practice the flipped classroom concept, in particular
if we are able to supervise groups of students in face-to-face meetings. Then, the
face-to-face events can be dedicated to the collaborative work in modeling.
6 Use MOOCs for engagement, motivation and research
(Dave Stikkolorum)
In my opinion, MOOCs have been proven to work quite well. In The Netherlands,
the Open University7 uses MOOCs on a large scale. In fact, it is their selling
point. Leiden University uses MOOCs in their education and research8 . The
experience with MOOCs is on both student and lecturers side. Also, different
joint projects like ‘Netwerk Open Hogeschool-Informatica’9 are using MOOCs
or MOOC-like environments to support their long distance learning. These or-
ganisations and initiatives use blended learning and flipped classrooms.
In my own experience I used MOOCs for students that wanted to broaden
their knowledge and skills further than our program (software engineering) could
offer, such as the “From Nand to Tetris Course”10 - building a general-purpose
computer system from the ground up. For a course in Game Development mixed
with Simulation, students are referred to Udacity’s “Intro to Artificial Intelli-
gence”. In the latter situation, the MOOC is more used in a blended situation
than ‘stand alone’.
What students need is a challenging and motivating environment that stim-
ulates learning. MOOCs can enrich students’ way of studying by offering them
flexibility (learning whenever and whatever they want) and exercises. MOOCs
can also offer modules that are not part of the students’ standard programs and
give them the possibility to study subjects to broaden their knowledge.
In our research group (LIACS, Leiden University and Chalmers/Gothenburg
University) we explore students motivation and reasoning about software design.
We see possibilities in using MOOCs in combination with game elements, as we
see for example in [17]. We already started to upscale our “Art of Software
Design” game [16] to an online game 11 in which competition elements are easily
thinkable.
A huge benefit for research is to actually use the data that students generate
in a MOOC for analysis on their studying and/or learning behaviour. Often we
have to deal with the fact that we only can observe a small amount of students
from a particular student population. MOOCs can give us opportunities to study
larger amounts.
In the future, MOOCs can be used as ‘add-on’ for modeling courses or stu-
dents’ programs. However, we agree there are challenges in the level of effort
needed to create a MOOC. We are aware that developing online systems with
trustful grading and helpful feedback is a time consuming activity.
7
http://www.ou.nl
8
http://leidenuniv.onlinelearninglab.org/
9
http://www.noh-i.nl
10
http://www.nand2tetris.org/
11
http://editor.models-db.com
7 The key challenge will be in supporting participant
engagement while exploring shared modeling activities,
where the specific details of the modeling context
intersect with the currently available infrastructure
supporting MOOC development (Jeff Gray)
How can we engage a large community of learners and support their sharing of
modeling experiences using current modeling tools and existing MOOC infras-
tructure?
Several lessons were learned from designing and teaching a 6-week MOOC
that served the training needs of high school teachers across the United States.
The CSP4HS 12 course was supported by a Google CS4HS (Computer Science
for High School) grant and focused on introducing a new high school course
called Computer Science Principles, which is being piloted by the College Board
in the US. This MOOC had over 1,200 registered participants, of which about
50% played an active part in the community, and only about 10% completed
all of the course requirements. The course had participants from 45 US states,
as well as representation from six countries. The CSP4HS course was built with
several different tools that enabled both asynchronous and synchronous partici-
pation: the core lessons were hosted on Google’s freely available Course Builder
platform; asynchronous interaction with the course participants was facilitated
by a Piazza discussion forum; synchronous coordination of participant interac-
tion was enabled by Hangouts on Air (for large broadcasts) and Hangouts (for
group office hours). The course was rather low-budget, costing under $25,000
with support from one graduate student, four undergraduates (two who were
Film majors), and four content experts who were also high school teachers. To
support the course, over 95 videos were produced (over 30 hours of video, which
took over 100 hours to record).
A key lesson learned from the CSP4HS MOOC was the need for deeply en-
gaged interaction among the participants, through both synchronous and asyn-
chronous approaches, to develop a Community of Practice among the partici-
pants. This observation is also supported by recent research on MOOC effec-
tiveness. Many new studies are being reported in the literature about MOOC
practices that are most effective; in fact, there is a new ACM conference in the
area focused on “Learning at Scale” – where learning scientists meet computer
scientists13 . One specific practice that has been studied recently is interactive
engagement pedagogy, which can be described as “where students interact fre-
quently in small groups to grapple with concepts and questions.”14 As evidenced
by several studies of MIT edX courses that used interactive engagement with
online tools, it is possible for multiple types of learners with different motivations
12
Computer Science Principles CS4HS: http://csp-cs4hs.appspot.com
13
Learning at Scale, http://learningatscale.acm.org/
14
David Chandler, “Study: Online classes really do work,” MIT News Office, http:
//newsoffice.mit.edu/2014/study-shows-online-courses-effective-0924
and background preparation to achieve results similar to peers taking the same
course face-to-face14 .
The benefit of engaged interaction among MOOC participants seems rather
intuitive, but the way that this would be implemented in MOOCS for a mod-
eling course is not clear. In traditional modeling courses that are face-to-face,
there may be a need for more feedback to students as they explore some ac-
tivity using a specific modeling tool. Providing feedback to students using a
visual modeling tool may pose new challenges to MOOC instructors (who may
be even in different parts of the world), such as how the instructor views the
content of the student’s modeling work (screenshots or live streaming of the
student’s screen?), how the instructor offers feedback on a student’s work, and
the ways that students remotely interact with other participants in the course.
The idea of modeling through experience, rather than imitation, can be help-
ful to students in a modeling course that is face-to-face. It is not clear how a
small group of students can be engaged together to explore a modeling experi-
ence while physically separated on a MOOC. For a truly effective experience,
the modeling tools used in a course may need to support team-based project
collaboration to enable the type of engagement that is suggested by effective
MOOC practice. It is also not clear how the modeling tool experience can be
integrated into the underlying infrastructure supporting a MOOC. In essence,
modeling tool support is often lagging behind tools for teaching programming –
there are often challenges when teaching modeling in a face-to-face context that
may be compounded when a modeling course is moved to a MOOC, given the
desire to support online engagement among participants.
The traditional activities associated with MOOC construction (e.g., prepa-
ration of videos and online lecture outlines, creating a web-based asynchronous
discussion forum) do not seem to be the challenging part for online modeling
courses. The key challenge will be in supporting participant engagement while
exploring shared modeling activities, where the specific details of the modeling
context (e.g., nuances of a particular modeling tool used in the course) intersect
with the currently available infrastructure supporting MOOC development.
8 Conclusion and Future Work
Although the panel’s introductory speaker warns us of the negative effects a
MOOC could have, our positions make clear we cannot ignore the MOOC hype.
In contrary, we clearly see possibilities for the application of MOOCs in future
curricula. We identified opportunities for more ‘in depth’ lectures while MOOCs
are used for preparation on one side and exercising (modeling through experi-
ence) on the other side.
There is no doubt interaction between the lecturer and the students has a
key value. We even see possibilities to increase students’ engagement by using
MOOCs in flipped classroom constructions and blended learning scenarios.
From the positive experience in software engineering targeted MOOCs, it
seems possible to develop a MOOC course on modeling to investigate and evalu-
ate if our educational practices benefit from the use of MOOCs, especially with
concern for the level of support for collaborative learning.
Acknowledgements
We would like to thank all panelists and the audience for their participation in
the discussion, and in particular Frieddrich Steinmann, for his ‘spicy’ introduc-
tion to start the fire!
In Memory of Robert France
We sincerely regret that Robert France is no longer with us. He passed away
while preparing this paper. We want to thank Robert France, not only for his
contribution to the MOOC panel, but for all his contributions to the Educa-
tor Symposium over the years. He was given a Special Lifetime award by the
MODELS conference in early 2015.
References
1. Anderson, L.E., Krathwohl, D.E., Airasian, P., Cruikshank, K., Mayer, R., Pin-
trich, P., Raths, J., Wittrock, M.: A taxonomy for learning, teaching, and assessing:
A revision of Bloom’s Taxonomy of Educational Objectives (Complete edition).
Longman, New York (2001)
2. Ardis, M.A., Henderson, P.B.: Software engineering education (seed): is software
engineering ready for moocs? ACM SIGSOFT Software Engineering Notes 37(5),
14–14 (2012)
3. Dasarathy, B., Sullivan, K., Schmidt, D.C., Fisher, D.H., Porter, A.: The past,
present, and future of moocs and their relevance to software engineering. In: Pro-
ceedings of the on Future of Software Engineering. pp. 212–224. ACM (2014)
4. Dillenbourg, P., Fox, A., Kirchner, C., Mitchell, J., Wirsing, M.: Massive Open
Online Courses: Current State and Perspectives (Dagstuhl Perspectives Workshop
14112). Dagstuhl Manifestos 4(1), 1–27 (2014), http://drops.dagstuhl.de/opus/
volltexte/2014/4786
5. Fidalgo-Blanco, A., Sein-Echaluce, M.L., Garcı́a-Peñalvo, F.J., Escaño, J.E.: Im-
proving the MOOC Learning Outcomes Throughout Informal Learning Activities.
In: Proceedings of the Second International Conference on Technological Ecosys-
tems for Enhancing Multiculturality. pp. 611–617. TEEM’14, ACM (2014)
6. Fox, A.: From moocs to spocs. Communications of the ACM 56(12), 38–40 (2013)
7. Garcı́a-Peñalvo, F.J., Hermo, V.F., Blanco, A.F., Sein-Echaluce, M.: Applied Ed-
ucational Innovation MOOC: Learners’ Experience and Valorization of Strengths
and Weaknesses. In: Proceedings of the Second International Conference on Tech-
nological Ecosystems for Enhancing Multiculturality. pp. 139–145. TEEM’14, ACM
(2014)
8. Henri, F.: Computer Conferencing and Content Analysis. In: Kaye, A.R. (ed.) Col-
laborative Learning Through Computer Conferencing, NATO ASI Series, vol. 90,
pp. 117–136. Springer (1992)
9. Holec, H.: Autonomy and Foreign Language Learning. Pergamon Press (1981)
10. Huang, J., Dasgupta, A., Ghosh, A., Manning, J., Sanders, M.: Superposter Behav-
ior in MOOC Forums. In: Proceedings of the First ACM Conference on Learning
at Scale Conference. pp. 117–126. L@S’14, ACM (2014)
11. Pappano, L.: The year of the mooc. The New York Times 2(12), 2012 (2012)
12. Rutenbar, R.A.: The First EDA MOOC: Teaching Design Automation to Planet
Earth. In: Proceedings of the The 51st Annual Design Automation Conference on
Design Automation Conference. pp. 213:1–213:6. DAC’14, ACM (2014)
13. Ryan, R.M., Deci, E.L.: Intrinsic and Extrinsic Motivations: Classic Definitions
and New Directions. Contemporary Educational Psychology 25(1), 54–67 (2000)
14. Schmidt, D.C., McCormick, Z.: Producing and delivering a coursera mooc on
pattern-oriented software architecture for concurrent and networked software. In:
Proceedings of the 2013 companion publication for conference on Systems, pro-
gramming, & applications: software for humanity. pp. 167–176. ACM (2013)
15. Siever, W.: Leveraging MOOCs. Journal of Computing Sciences in Colleges 30(2),
125–131 (Dec 2014)
16. Stikkolorum, D.R., Chaudron, M.R., de Bruin, O.: The art of software design, a
video game for learning software design principles. Gamification Contest MODELS
(2012)
17. Tillmann, N., De Halleux, J., Xie, T., Gulwani, S., Bishop, J.: Teaching and learn-
ing programming and software engineering via interactive gaming. In: Software
Engineering (ICSE), 2013 35th International Conference on. pp. 1117–1126. IEEE
(2013)
18. Tomkin, J.H., Charlevoix, D.: Do Professors Matter?: Using an A/B Test to Eval-
uate the Impact of Instructor Involvement on MOOC Student Outcomes. In: Pro-
ceedings of the First ACM Conference on Learning at Scale Conference. pp. 71–78.
L@S’14, ACM (2014)