=Paper=
{{Paper
|id=Vol-1967/FLMOOCS_Paper3
|storemode=property
|title=Discussion analytics: identifying conversations and social learners in FutureLearn MOOCs
|pdfUrl=https://ceur-ws.org/Vol-1967/FLMOOCS_Paper3.pdf
|volume=Vol-1967
|authors=Shi Min Chua,Caroline Tagg,Mike Sharples,Bart Rienties
}}
==Discussion analytics: identifying conversations and social learners in FutureLearn MOOCs==
https://ceur-ws.org/Vol-1967/FLMOOCS_Paper3.pdf
Discussion Analytics: Identifying Conversations and
Social Learners in FutureLearn MOOCs
Shi Min Chua1*, Caroline Tagg2, Mike Sharples1, Bart Rienties1
1
Institute of Educational Technology
2
Faculty of Wellbeing, Education and Language Studies
The Open University, Milton Keynes, UK
shimin.chua@ou.ac.uk, caroline.tagg@ou.ac.uk,
mike.sharples@ou.ac.uk, bart.rienties@ou.ac.uk
ABSTRACT. Discussion among learners in MOOCs has been hailed as benefi-
cial for social constructive learning. To understand the pedagogical value of
MOOC discussion forums, several researchers have utilized content analysis
techniques to associate individual postings with differing levels of cognitive ac-
tivity. However, this analysis typically ignores the turn taking among discussion
postings, such as learners responding to others’ replies to their posts, learners
receiving no reply for their posts, or learners just posting without conversing with
others. This information is particularly important in understanding patterns of
conversations that occur in MOOCs, and learners’ commenting behaviors. There-
fore, in this paper, we categorize comments in a FutureLearn MOOC based on
their nature (post vs. reply to others’ post), classify learners based on their con-
tributions for each type of posting, and identify conversations based on the types
of comments composing them. This categorization quantifies the dynamics of
conversations in the discussion activities, allowing monitoring of on-going dis-
cussion activities in FutureLearn and further analysis on identified conversations,
social learners, and types of comments with an unusual number in a course step.
Keywords: MOOCs; Computer-mediated Collaborative Learning; Learning
Analytics; FutureLearn
1 Introduction
Discussion forums in Massive Open Online Courses (MOOCs) have attracted research
interest since the beginning of MOOCs, particularly in the LAK community [1, 2]. This
could be due to two reasons. Firstly, enormous text data are easily available for analysis,
either by manual coding, text mining or natural language processing (e.g., [1–4]). The
general findings from these content analyses are that postings in MOOC discussion
forums indicate different levels of cognitive thinking. For example, Kellogg and col-
leagues [4] found that, in the two MOOCs for school teachers on digital learning and
mathematics learning they analyzed, 2 to 3% of the discussion postings achieve the
FutureLearn data: what we currently have, what we are learning and how it is demonstrating learning in
MOOCs. Workshop at the 7th International Learning Analytics and Knowledge Conference. Simon Fraser
University, Vancouver, Canada, 13-17 March 2017, p. 36-62.
Copyright © 2017 for the individual papers by the papers' authors. Copying permitted for private and acade-
mic purposes. This volume is published and copyrighted by its editors.
37
highest phase of knowledge construction. Secondly, discussion among learners and ed-
ucators in MOOCs is an important element of social constructive learning because it
allows learners with varied experience and expertise from around the world to interact
with each other [5]. Yet, this apparent advantage of discussion in MOOCs has been
undermined by concerns about educators and learners being overwhelmed by the sheer
number of postings, lack of focus on what is being discussed, lack of “appropriate”
comments or responses (likes or replies) from educators and peers [1, 6, 7], and lack of
in-depth discussion or recurrent interaction [8–10]. These drawbacks warrant further
research to improve the discussion experiences of MOOC learners. To provide a basis
for future learning analytics and qualitative research on discussion activities in Future-
Learn, a relatively new MOOC platform that has not received as much research cover-
age as EdX or Coursera in LAK community, we propose an approach to categorize
learners’ discussion postings and their posting behaviours based on the discussion
structure afforded by the FutureLearn platform. As will be discussed next, discussion
activities in FutureLearn is different from the discussion forums used in other MOOC
platforms, so a categorization approach tailored to its unique discussion function is
needed for analytics and other research purposes [11]. After introducing FutureLearn,
previous research on MOOC discussion will be reviewed before the proposed catego-
rization is presented. We then explore how this categorization could be used in both
quantitative and qualitative analysis to study the conversational interaction and dis-
course in a FutureLearn MOOC, and how educators and course designers could use the
analytics for discussion monitoring and course revision.
2 FutureLearn
In FutureLearn, a discussion area is present in each course step, except in steps for
quizzes and exercises. Learners are encouraged to share their experience, contribute
their reflection, discuss issues raised in the course step, and interact with others in the
discussion area right below or beside the course content [5, 9]. The focus of the discus-
sion is dictated by the discussion prompt or the course content in that step, thus creating
a shared attention for social learning among learners. This discussion function is dif-
ferent from the centralized discussion forum used in other MOOC platforms such as
EdX and Coursera [1, 12], which is independent of the course step. The “discussion in
context” approach taken by FutureLearn may be able to overcome the problem of lack
of focus in MOOC discussion, one of the problems mentioned in the introduction.
This “discussion in context” approach also allows educators to design each step of
the learning journey to support learners’ conversations with themselves and others by
building on their previous experience and existing knowledge while going through the
course, in accordance with Laurillard’s Conversational Framework [13, 14].The con-
versational framework operationalizes learning as an iterative process between reflect-
ing within oneself and conversing with educators and others, while also interacting with
the outside world. The process starts with learners’ initial description of concepts.
Through interaction with content, activities, or peers, learners adapt their initial under-
standing and expand their knowledge. Based on this framework, learning happens
38
through the whole process, not only relying on discussion with others and feedback,
but also involving reflective conversation within learners themselves during the pro-
cess. Similarly, clarification of concepts and sharing experience are as important as
evaluating and debating with each other.
According to Laurillard [14], for the learning process of conversation with oneself
and others to be successful, good learning design is needed. Different questions,
prompts, course media may be designed to lead learners through a journey of initial
reflection with concepts, interaction with content, others or practice environments, and
finally synthesis or critical thinking of what has been learned. Under this framework, a
discussion function that is attached to each stage of learning is needed, instead of a
centralized discussion forum that relies on learners’ initiative to raise topics or ques-
tions. Therefore, the FutureLearn “discussion in context” approach may be warranted
to achieve the cycle of Laurillard’s conversational framework, and can be a suitable
testbed to examine how course step design may invoke different kinds of discussion
postings during a learning journey, which is one of the research question considered in
this paper.
3 Previous Research
3.1 Content Analysis
Several frameworks have been used in previous content analysis research (e.g., [3, 4,
15]) to categorize discussion postings in MOOCs and other small scale online discus-
sions (see [16] for a review) into lower and higher order thinking within a fixed number
of levels. This method is based on the assumption that discussion postings are indicative
of learning processes [17], learners’ interaction [17, 18], critical thinking [18] or learn-
ing goals [19]. For example, Henri’s framework [17] comprised of five levels: partici-
pation, interaction, social, cognitive and metacognitive processes. In their analysis of
MOOC discussions, Kellogg and colleagues [4] found that around 40% of the discus-
sion comments in their MOOCs were sharing opinions and around 3% are metacogni-
tive. Categorization of discussion postings based on these frameworks typically privi-
leges the highest levels and is used to evaluate the quality of discussion. However, this
emphasis is in contrast to Laurillard’s framework, which values all the different kinds
of conversations that comprise the cycle of the learning process.
3.2 Social Network Analysis
Social Network Analysis has been widely used to examine the social structures and the
interactions among individuals in computer-mediated collaborative learning [20, 21]
and MOOC discussions [8, 9, 22]. In the social network analysis, a tie or an edge is
formed between two learners if one of them comments on the other’s post, and the
number of times they interact is taken as the weight of the tie. The density and disper-
sion of the ties among learners in turn indicate the social structure of the discussion
forums or activities. Thus, social network analysis is useful for examining individuals’
39
interactions with others as well as the overall social structure of a network. For the
overall social structure, the density of small-scale online discussions [21] was found to
be high with only one or two learners at the periphery, whereas the MOOC discussion
tended to be dispersive and fragmented [8]. At the individual level, Sunar and col-
leagues [9] found only 1.75% of learners had recurrent interactions with each other
across the MOOC course period, and 20% of learners were identified to be at the core
of the MOOC discussions analyzed by Kellogg and colleagues [4]. These studies
seemed to show that the social structure and interaction among learners is not close-
knit in MOOCs, perhaps due to the massive numbers of learners who are free to join or
leave the discussion [8].
3.3 Summary
Undeniably, the content analysis and social network analysis are beneficial for educa-
tors to understand the discussion and social dynamics that has happened in their
MOOCs overall. However, as indicated by [2, 14, 23], an analysis at the course level
may not be helpful in the search for a theoretical or design explanation about the distri-
bution of the comments of differing cognitive levels or the social dynamics in the dis-
cussion. One way of tackling this issue is to associate the content and social network
analysis with the course step design. More importantly, categorizing discussion post-
ings by using the content analysis frameworks mentioned above may mask the dynam-
ics in a conversation because this method normally codes individual postings without
recognizing turn-taking, replying, initiating conversation, or posting without receiving
reply. Similarly, although social network analysis could reveal social interactions
among learners across the course period, it did not reveal their interaction within a con-
versation or a discussion thread.
As discussed earlier, conversations with oneself and others is fundamental to social
learning in FutureLearn. Yet, content analysis and social network analysis may not nec-
essarily reveal interaction among learners within conversations, such as turn-taking,
replying to others’ posts, replying to others’ reply to one’s own posts, or lone posting
as conversations with oneself. Therefore, in the present study, we aimed to examine
this aspect of interaction in the FutureLearn by classifying comments and learners in
terms of conversational dynamics based on its unique discussion function that is not a
forum. This classification may provide visualization that help educators or mentors to
facilitate discussions during the course periods, and may set up the next step for in-
depth content analysis, discourse analysis, conversational analysis or linguistic analysis
[11, 24] that will inform the nature of social constructive learning in MOOC discus-
sions.
4 Categorizing Learners’ Comments in FutureLearn
As mentioned earlier, in FutureLearn learners are encouraged to comment under or be-
side the content of each course step, except in steps for quizzes and exercises. The
commenting area takes a simple structure, differentiating between posts and replies
40
only. There is no hierarchical or threaded structure amongst replies under a post, and
the replies are ordered by the time of posting. Learners receive notification by email
when somebody replies under their posts, or when somebody replies after their reply
under the same post.
Given that not every post receive replies, posts that receive no replies are categorized
as lone posts, whereas those receives replies are initiating posts. An initiating post is
the start of a conversation, if we assume a conversation consists of at least two turns,
i.e., the post and one reply. Learners creating the initiating posts could be considered
as initiators, to differentiate them from replying learners who reply to them.
Underneath an initiating post, a long list of replies probably does not reveal any
conversational interaction at first glance, given that the commenting structure in Fu-
tureLearn is not threaded. However, a close examination of the replies underneath an
initiating post could at least reveal the interaction among replying learners and the ini-
tiator within that conversation. A learner replying more than once underneath an initi-
ating post indicates that they come back to the conversation again after making the first
reply, possibly in order to respond to others’ reply in relation to their first reply. Simi-
larly, an initiator replying underneath his/her own initiating post suggest that he/she
might be responding to other learners’ reply to their initiating post. These two situations
suggest that there are turn-takings between two learners or initiator within a conversa-
tion initiated by an initiating post [25]. Nonetheless, there are also cases where learners
only reply once underneath an initiating post, or initiators never reply underneath
his/her own initiating post. In short, the replies underneath an initiating post could be
differentiated based on the order and number of times an individual learner reply under
that initiating post, rather than the order in the long list of replies per se.
Based on the observation just mentioned, each comment on FutureLearn could thus
be categorized into one of the five categories (See Figure 1 for a mock illustration):
1. Initiating posts: Posts that receive replies
2. Lone posts: Posts that receive no replies, also include the replies posted by the same
learners in response to his/her own post, where no other learners reply to that post
3. Replies: Replies to others’ initiating post, i.e., the first time or the only time a learner
replies to an initiating post created by others
4. Further replies: Further replies under an initiating post that one has already replied
to, i.e., the learner replies more than one time under an initiating post
5. Initiator’s replies: Replies to others’ replies to one’s own initiating post, i.e., initi-
ator replies under his/her own initiating post.
41
Fig. 1. Mock Illustration of Different Types of Comments within a Conversation
42
These five categories capture turn-takings in conversations, that is somewhat shaped
by the FutureLearn platform [24, 25], despite it being not as neat as it could have been
if content had been taken into account [17]. Nonetheless, we argue that these five cate-
gories could be used as a proxy for discussion dynamics among learners. Essentially,
the differentiation between lone posts and initiating posts allows us to capture posts that
initiate conversations, in contrast to lone posts that imply no explicit interaction among
learners. Of note, lone posts could be read and ‘liked’ by many learners, suggesting an
implicit interaction among learners. A reply is an explicit interaction between at least
two learners. A further reply or initiator’s reply shows that learners do get back to each
other on the issue raised in their posts or replies, in other words a turn-taking.
5 Research Questions
To illustrate how this categorization approach could help educators and researchers
in understanding the conversational dynamics in the discussion activities in Future-
Learn, we are going to apply this categorization to one FutureLearn MOOC and conduct
both quantitative analysis and qualitative analysis to explore the following questions:
1. What are the characteristics of conversations occurring in the FutureLearn discus-
sions?
2. Are there different groups of social learners with distinctive commenting behav-
iours?
3. Is there a relationship between course step design and distribution of comment
types?
By addressing these questions, a learning analytic approach is realized by quantify-
ing the discussion dynamics in terms of the distribution of different types of comments,
conversations and social learners. Educators could then make use of this information
for their revision of course step design in the future run of their course, or for their
intervention of on-going discussion activities. Prototype dashboards showing the ana-
lytics are presented after we address each research question.
6 Methods
6.1 Data Set
The comments data to be analyzed are from the first run of the FutureLearn Course
“Inequalities in Personal Finance: the Baby Boom Legacy” offered by The Open Uni-
versity. The course explores the controversial issue of inequality with particular atten-
tion to pensions, housing, social benefits in developed countries, and presents alterna-
tive solutions to inequality that could be adopted by individuals and governments. The
course content is mainly about the UK, but with some mentions of other developed
countries, such as the US, Germany and Singapore. A recurrent contentious theme in
the comments is whether governments should use taxpayers’ money to help those at the
poor end of the inequality continuum. The course lasted for four weeks, yet the data
43
captured is available from the start of the course until two weeks after the course period
ends. There were 1951 learners, 636 (33%) of whom were social learners that contrib-
uted 10033 comments. Four educators also contributed 363 comments. Wherever space
allows, the analysis of educators’ comments are presented along with learners’ com-
ments, but interpretation and discussion will be on learners.
Based on the definition offered by FutureLearn, learners are those who visit at least
one step of the course, whereas social learners comment at least once in the course. It
should be noted that we are analyzing the comments data, so only social learners, rather
than all learners, were included in the present study. Social learners did not necessarily
comment in each course week, as shown in Table 1, and the number of social learners
decreased across the weeks. The number of comments also dropped from Week 1 to
Week 3, and plateaued between Week 3 and week 4. The reference to the week here is
based on the four weeks of the course. For example, if a comment was posted in a week
2 step after the course ended, it was considered a week 2 post.
Table 1. Number of Social Learners and Comments in Each Course Week
Social Learners Learners’ Educators’
Comments Comments
Week 1 516 2723 91
Week 2 366 2872 100
Week 3 290 2213 65
Week 4 266 2225 107
6.2 Data Analysis
All comments posted in the courses were categorized into the five proposed types
and are shown in Table 2. There were more lone posts than initiating posts, yet replies
were the most frequent comment types in this course. Additionally, some learners did
engage in turn-takings, i.e., responding to others’ replies to their own initiating posts or
replying again under an initiating post, although the number of initiator’s replies and
further replies were lower than the other categories. This could be because they could
only happen when an initiator receives a reply for their posts so that they could respond,
or when there are other learners replying after a learner has replied to an initiating post.
As argued earlier, an overview of the distribution of the comments at the course level
may not be informative for unpacking the conversational interaction among learners in
the discussion activities in FutureLearn. We thus investigate the distribution of each
type of comments at three levels: conversations, learners and course step design, corre-
sponding to the three research questions raised.
44
Table 2. Distribution of Each Type of Comments
Types Number of % Number of %
Learners’ Educators’
Comments Comments
Initiating Posts 1845 18% 23 6.3%
Lone Posts 2612 26% 39 11%
Replies 3871 39% 242 67%
Initiator’s Replies 708 7% 10 2.7%
Further Replies 997 10% 49 13.4%
Total 10033 100% 363 100%
7 Conversations
Research Question 1: What are the characteristics of conversations occurring in the
FutureLearn discussions?
7.1 Analysis and Results
A conversation is started by an initiating post and composed of all the replies, initiator’s
replies and further replies underneath it. Thus the number of conversations is equal to
the initiating posts, which is 18451 in this course. These initiating posts were contrib-
uted by 404 social learners, whilst all the conversations involved 506 social learners
(including both initiators and replying learners). Thirteen percent of the initiating posts
in a conversation elicited more than five turns, i.e., replies, initiator’s replies or further
replies together2, which was more than the 2.5% shown in [10] findings on their Fu-
tureLearn courses. There were 72 conversations with at least ten turns, the longest of
which consisted of 51 turns. Table 3 shows the number and percentage of conversations
identified based on the number of turns, number of initiator’s replies, replies and further
replies contributed by replying learners, and number of unique learners/educators in-
volved.
The longest conversation in this course happened in a step that was not specifically
designated as a discussion step. The initiating post was the initiator’s3 interpretation of
a cartoon on that step that portrayed “the 'rich' family had one child and the 'poor' had
1
There were an additional 23 conversations initiated by educators as shown in Table 2, but are
not included in the current analysis.
2
The number of turns also included replies and further replies made by educators to ensure the
completeness of a conversation.
3
The comment data retrieved did not contain learners’ name and the registration for this run of
the course had ended before we started the analysis, so there is no way for us to obtain consent
from learners to cite their comments, or to acknowledge them under the terms of Creative
Commons License. Therefore, we anonymized them instead to protect their privacy.
45
two … some people have children they cannot afford but expect someone to pick up the
tab by having more benefits …”. This initiating post garnered 38 replies from 12 learn-
ers and 1 educator, and the initiator responded to them 13 times, such that there were a
total of 51 turns. This, along with another 14 conversations with more than 20 turns,
might make a case study for conversational analysis about learners addressing a con-
troversial issue among themselves and how learners address a specific learner among
all the learners involved in the conversation. At the same time, this finding also suggests
that not only discussion prompts designed by educators, but also contents of initiating
posts contributed by learners could generate discussion. A comparison between lone
posts, initiating posts with only one reply and initiating posts eliciting more than 20
turns might help us to understand more about learners’ roles in initiating conversation
in MOOCs. The first reply in each conversation may also need to be taken into account,
to understand if the reply stifles further conversation or if it is simply a supportive state-
ment to a reflective post that hardly invites replies, given that there were
705 conversations (38%) with only one reply.
Table 3. Overview of Conversations
Nature of the conversations Frequency %
Conversations with only 1 reply 705 38%
Conversations with more than 5 turns 247 13%
Conversations with at least 10 turns 72 4%
Conversations with at least 20 turns 15 1%
Conversations with initiator's replies, i.e., repeated occur- 466 25%
rence of the initiator
Conversations with further replies, i.e., repeated occurrence 413 22%
of replying learners
Conversations with both initiator's replies and further replies 207 11%
from others
Conversations involving at least 5 unique learners/educators 170 9%
Conversations involving at least 10 unique learners/educators 12 1%
Secondly, in 466 conversations, learners who contributed the initiating post responded
to replies from others at least once, and there were eleven conversations in which the
initiators replied more than five times. A conversation with large number of initiator’s
replies may imply a conscientious initiator who responds to each reply he/she receives.
Furthermore, in 413 conversations, learners who replied to the initiating post further
replied at least once after other learners reply after their first reply. In short, in almost
a quarter of the conversations generated in this course, learners engaged in turn-takings
by getting back to each other on issues raised in their comments. It also pointed to the
fact that, despite its simple commenting structure, there are turn-takings and discourse
structure in the discussion in FutureLearn, and this information has not been captured
46
in previous content analysis research in MOOC discussion where postings were ana-
lyzed individually, or social network analysis where analysis is based on individuals
rather than on a conversation. Detailed analysis of the initiator’s replies and further
replies in these conversations will shade light on how learners react to each other, es-
pecially when there is a disagreement.
Lastly, there were 12 long conversations involving more than ten unique learners/ed-
ucators, containing replies, and initiators’ replies or further replies from some of the
replying learners/educators. In one such conversation, six out of the ten learners in-
volved addressed the initiator’s name at the start of their replies. At the same time there
were replies addressing four other learners by name in the same conversation. Such
conversations seem to be containing multiple sub conversations directed to the initiator,
and overlapping turn-taking between different pairs of learners, similar to other com-
puter-mediated communication such as Facebook [26] and Internet Relay Chat (IRC)
[25].
7.2 Interim Discussion
The analysis above showed that the conversations in FutureLearn could be character-
ized by the number of turns, presence of initiator’s replies or further replies and number
of unique learners involved, despite the complexity revealed. A thorough conversa-
tional analysis on the different types of conversations identified above will surely un-
ravel this complexity further and answer some of the questions arising from the find-
ings, including how initiating posts, instead of lone posts, elicit conversations, how
learners react to each other when they engage in turn-takings, and how multiple learners
engaged in a single conversation.
On the other hand, this analysis could be a learning analytic tool for educators and
mentors as they look into conversations with an unusual number of turns, initiator’s
replies, further replies, unique learners while monitoring the discussion activities when
the course was still running. Conversations attracting huge number of learners or com-
prising repeated exchanges between few learners may contain some heated discussion
in need of intervention by educators or mentors. For example, in a conversation with
ten replies and seven learners, the last reply started with “Thanks for the insult XX. A
problem with the public schools is that …” Although the ‘like’ function may have
helped educators filter for popular posts, it was found that in this course, there were 43
initiating posts receiving more than ten turns, but fewer than five ‘likes’. Therefore,
besides ‘like’, the replies and conversational dynamic measures of each initiating post
will also help to focus attention of educators or mentors among the overwhelming com-
ments contributed by learners.
8 Social Learners
Research Question 2: Are there different groups of social learners with distinctive com-
menting behaviours?
47
8.1 Analysis and Results
Preliminary k-mean clustering and hierarchical clustering were conducted with the aim
of identifying no more than 10 groups of social learners. The clustering was based on
the number of each type of comments contributed by a learner and the number of replies
and likes received. Both clustering resulted in one group consisting of 85% of the social
learners and other groups each with fewer than five learners. These results were not
interpretable and led us to a simpler way of categorization - permutation and combina-
tion, and base our categorization on whether a learner contributed a certain type of
comments rather than the number of each type of comments contributed.
32 (25) permutation and combination of social learner groups are possible based on
whether a social learner contributes each of the five categories of comments. However,
the learners enrolled for this MOOC did not show such a diverse pattern, and only 17
combinations were found (Table 4).
Table 4. Permutation and Combination of Social Learner Groups
Initiat- Lone Reply Further Initiator's Received Number 7 Groups of So-
ing post post reply reply Replies? of cial Learners
Learners
04 1 0 0 0 0 131 Loner
0 1 1 0 0 0 14
0 0 1 0 0 0 19
0 0 1 0 0 1 40 Replier
1 0 0 0 0 1 41 Initiator without
1 1 0 0 0 1 73 replying
1 0 0 0 1 1 11 Initiator who re-
1 1 0 0 1 1 26 spond
1 1 1 1 1 1 91 Active Social
1 0 1 1 1 1 1 Learner
1 1 1 1 0 1 27
1 1 1 0 1 1 51
1 0 1 1 0 1 7
1 0 1 0 1 1 1
1 1 1 0 0 1 75 Active social
1 0 1 0 0 1 23 learners without
turn-taking
0 1 1 1 0 1 5 Reluctant active
social learners
4
0 indicates no contribution of the particular type of comment, 1 indicates at least one
contribution of the particular type of comment.
48
We further reduced the number of groups into seven by imposing three more criteria.
Firstly, learners who received no reply at all for their posts and/or after they replied
underneath others’ initiating posts were categorized into one group (Loners). Second,
learners who contributed either further replies or intiator’s replies were deemed as en-
gaging in the same type of commenting behaviour, given that both types of comments
indicate turn-takings (see Active Social Learners and Active Social Learners without
Turn-takings below). Third, as long as learners contributed initiating post, whether or
not they contributed lone post is not taken into account (see Initiators, Initiators who
respond, Active Social Learners, Active Social Learners without Turn-taking below).
The comments contributed by these seven identified social learner groups could be sub-
jected to further analysis and/or intervention during the period of the course.
Group 1: Loners. 164 (26%) out of the 636 social learners never received a reply from
others either for their own posts or their replies under others’ initiating posts, although
110 of them received at least one like for their posts or replies. Among them, 131 con-
tributed only lone posts, 19 contributed only replies, and 14 contributed both lone posts
and replies. All except three of them commented fewer than 10 times, which might
decrease the probability of their postings being seen and replied to. Some of them might
have dropped out of the course, yet 71 of them completed at least half of the total steps
in the course. 82 of them only commented in week 1, whereas the remaining 83 com-
mented in other weeks. The inconsistency and infrequency of their commenting at the
start of the course made it hard to tell whether or not receiving no reply dissuaded them
from contributing any further to the discussion.
We examined all the comments by one loner. This learner created posts and replies
14 times across the four-week course period but never received a reply and received
only one like. Four of his/her lone posts and one reply stated only “I agreed”, which
did not provide substance to invite replies. In his/her longest lone post, “i am single
and not thinking to buy a house as far as i am single but for my family kinds and wife i
prefer to buy a own[sic] house ……we can profit in the end mortgage a house in term
of money,” there seems to be no sign of inviting others to comment. But he/she actually
answered the discussion prompt “Do you rent or buy” that encourages self-reflection.
He/she raised a question in one of his/her lone posts, but without a question mark, “i
paid, tax, unemployment insurance and pension from my salaries, how it would effect
[sic] me if i will no longer live before i would able to use them.” It seems that this
learner needs to improve his/her commenting skills and English writing, especially in
elaborating his/her ideas and making his/her questions explicit so that other learners
would have something concrete to comment on. Nonetheless, this learner completed
the course despite not receiving replies from others.
There was one loner who only contributed one post which received 12 likes despite
not receiving any reply. It was a reflection on his/her pension choice in response to a
discussion prompt in week 2, “I am retired so have some experience of the different
schemes and what they might buy in retirement. I was very lucky to have a final salary
scheme …. The problem with personal pensions is that you are at the mercy of the
insurance companies… Saving throughout your working life for retirement does not
necessarily mean you will receive the sort of income you envisaged.” This comment
49
resembles life advice from a senior that you will listen to rather than replying back.
Another loner also commented once only by sharing information during week 3 about
housing issues in his/her country by saying “Here in Hungary, 80 % of the population
have own property (flat or house). I live with my parents, we have a house,” without
receiving any likes or replies. These two lone posts provided information and personal
story only without reference to others’ viewpoints or invitation for others’ input. They
could be considered monologic, and this might be the reason for not receiving any reply
[27]. This preliminary analysis of the three loners’ comments showed that lone posts
could be of varied nature.
Group 2: Repliers. 60 people only replied to others without creating their own initiat-
ing posts. 20 of them have all their replies as the last reply under an initiating post, i.e.,
nobody replies after them, and were already categorized as loners above. The remaining
40 people sometimes attracted replies after their replies under an initiating post, alt-
hough we could not determine if the replies were targeted to their replies without con-
sidering the content of the replies. Only three of them contributed more than 10 replies
throughout the course, and nine of them engaged in further replies after other learners
replied after them.
Group 3: Initiators without Replying. 114 learners contributed only posts (both ini-
tiating posts and lone posts) but never replied to others’ initiating posts despite receiv-
ing replies from others. Among them, 41 always had people replying to their posts,
whereas 73 of them sometimes had lone posts that received no reply. Both groups never
responded to others’ replies under their initiating posts. Nonetheless, 91% of them did
not contribute more than 10 posts. Yet, there were two initiators each created 90 posts,
with only 31 and 8 posts receiving replies respectively.
Interestingly, there were three initiators received more than 10 replies to one of their
initiating posts yet they never responded. One of them had been asked by other learners
in their replies to him/her throughout the course for copying others’ comments. The
other one made only two initiating posts, both about state benefits. In both conversa-
tions initiated, there were replies for and against the initiator’s negative attitudes to-
wards those who claim benefits. The shorter initiating post was “where i live work is a
four letter word and they would love to receive[sic] a share without contributing, even
when they can.” which elicited ten replies from six learners. The first reply asked “So
you or your parenst[sic] has never received child benefit, used the NHS or attended
state school?” and one spoke for the initiator by saying “…we don't know what is hap-
pening there, and therefore can't really criticise her comments”. The third initiator
made only one post,” All animals are equal, but some animals are more equal than
others. George Orwell, Animal Farm”, with the first reply asking “Ah, XXX, but what
chances of change? Are all politicians hypocrites wh[sic] won't effect change?”
From these excerpts, it seems that initiators provoked some discussion about politics,
and they might be unwilling to respond to others’ ‘hostile’ replies with views which
50
were different from them. This awaits a full analysis of the initiating posts and the re-
plies to them. Yet, it indicates a need to monitor long conversations, especially when
the initiator is not responding at all.
Group 4: Initiators who Responded. 37 learners responded to others’ replies to their
initiating posts, yet they never replied to others’ initiating posts. They were similar to
the initiator group except they responded under their initiating posts. In one instance,
the learner responded to a reply full of strong language. Their initiating post was “The
course comments have become a happy hunting ground for left wing/right wing preju-
dices. I welcome the presentation in the course of arguments derived from a broad base
of statistical data,” which attracted one hostile reply “Can you explain to me what is
wrong with people putting their various analysis of the circumstances they see in the
world… I presume, despite your lip service to balance- that views…”. The learner of
the initiating post in turn responded with “I believe in open discussion and strive not to
be judgmental. However I confess to a prejudice towards discussion that is illuminated
by hard evidence.” Presumably, not every learner had the courage to respond to such a
hostile reply, and might choose to keep quiet, as the example in the initiator group sug-
gests.
Group 5: Active Social Learners. 91 out of the 636 social learners contributed all five
types of comments. Sometimes their posts initiated a conversation, sometimes they
were lone posts. They replied to others under an initiating post, responded to those who
replied under their initiating posts, and further replied after their replies in others’ ini-
tiating posts. Another 87 social learners were also similar to this group, except that one
of them always received replies from others and never had a lone post, 27 of them never
responded to others under their initiating posts but further replied to others under oth-
ers’ initiating posts, 51 of them never further replied under others’ initiating posts but
responded to others under their own initiating posts, and eight of them always received
replies from others and either never responded or further replies. Although they did not
contribute all five types of comments, these learners initiated posts that received replies,
replied to others’ posts and engaged in turn-taking as indicated by initiator’ replies or
further replies. Therefore, they were considered as active as the 91 learners who con-
tributed all five types of comments. Putting these learners together, there were 178 ac-
tive social learners in this course, comprising 28% of all social learners. 158 of them
completed at least half of the course, and 85% of them commented more than ten times.
Group 6: Active Social Learners without Turn-taking. 75 learners contributed ini-
tiating posts and lone posts, and also replied to others’ initiating posts, but never re-
sponded under their own initiating posts or further replied. Although they received re-
plies for their initiating posts, they never responded to those replies. Similarly, they
never replied further after other learners replies after them under the same initiating
post. Although they never got back to others on the issues they commented before, they
were still considered active given that they created posts as well as replied to others’
initiating post. Additionally, there were 23 learners behaved similarly except that they
51
were not so fortunate to receive any reply for their posts such that they did not have any
initiating posts. In sum, there were 98 learners in this group.
Group 7: Reluctant Active Social Learners. Lastly, five learners could be considered
reluctant social learners as they created posts, replied to others’ initiating posts, and
engaged in turn-taking by replying further when other learners replied after they replied
under an initiating post. They could be similar to the active social learners, just that
they were not so fortunate to receive any replies to their lone posts. All of them con-
tributed more replies than lone posts.
8.2 Interim Discussion
Table 5 summarizes the classification of the social learners discussed above and their
distinctive features.
Table 5. Seven Groups of Social Learners
Groups Distinctive Features Number of Learners
Learners with more
than ten
comments
Loners Never received replies 164 3
Repliers Only replied to others 40 3
Initiators without Never replied to others’ posts or underneath own in- 114 8
replying itiating posts
Initiators who re- Never replied to others’ posts but responded to oth- 37 9
sponds ers' replies underneath own initiating post
Active social Initiated posts, replied to others, and engaging in 178 148
learners turn-taking by replying under own initiating post or
further replying
Active social Initiated posts, replied to others but never replied un- 98 30
learners without der own initiating post or further replied
turn-taking
Reluctant active Created lone posts, replied to others, further replied 5 3
social learners
This classification shows that the learners’ commenting behaviour in FutureLearn is
not homogenous and could be distinguished by the types of comments they contributed
and replies received. Also shown are the number of learners commenting more than ten
times in each group. 204 social learners contributed comments more than ten times
across the course periods, and more than half of them were active social learners. The
52
content of the comments and learning experience of the different groups of social learn-
ers warrant further research to understand the discussion activities in MOOCs. Analyz-
ing comments contributed by individual learners from each group by using well-estab-
lished content analysis frameworks, discourse analysis or conversational analysis will
further shed light on social learners and inform course educators about their audiences,
although social learners comprised only about one third of all the learners. Conducting
in-depth interviews with learners from different groups about why and how they com-
ment in the discussion might triangulate findings from the analysis of their comments,
especially the initiators who only create posts, repliers who reply only and loners who
contribute very few posts.
This classification also allows educators or mentors to target specific groups of
learners for discussion monitoring. We suggest automatizing classification of learners
in the middle term of the course, and looking into some of their latest comments for
interventions. For loners, educators could reply to them by asking them to elaborate
more on their arguments, or direct other learners to read their posts that are worthy of
commenting. For initiators, educators may want to encourage them to either reply to
others’ posts or responded to others’ replies to their initiating posts. However, as men-
tioned earlier, some initiators did not respond probably because of the hostility ex-
pressed by others’ replies to their controversial initiating posts, so educators might want
to look into the replies they received at the same time.
9 Course Step Design
Research Question 3: Is there a relationship between course step design and distri-
bution of comment types?
9.1 Analysis and Results
Within the 91 steps that allow commenting in this course, 76 steps were dominated
by replies, 15 steps by lone posts, whereas lone posts were the least in seven steps,
initiator’s replies in 48 steps and further replies in 36 steps. Closer inspection shows a
difference among steps. For example, lone posts (53%) dominated step 1.2, which was
the introduction to the course, and most posts were self-introduction. In contrast, in step
1.3, there were overwhelmingly more replies (43%), compared to initiating posts (17%)
or lone posts (15%), suggesting learners were explicitly interacting with each other. In
fact, step 1.3 was a series of explanations of terminology without any explicit discus-
sion prompts, except the title “Inequalities of what?” Perhaps this is a big question and
learners are at the initial stage of concept formulation, so they tended to discuss with
each other without the need of being prompted. In contrast, in the dedicated discussion
step 1.18, lone posts (60%) dominated. This might be due to the fact that the discussion
prompt “Think about the factors influencing your own income and consumption profile
so far and what you expect in the future” asked for self-reflection and sharing rather
than discussion , and it is highly likely that learners were not critical or judgmental of
others’ personal choices and circumstances.
53
To systematically examine the relationship between course step designs and
distribution of comment types, we conducted a preliminary analysis on the design of
each course step based on the content in each step. The first author went through every
step of the course as a learner, by watching all the videos and reading all the contents.
However, no comment was read, in order to avoid any bias that might arise when we
examined the relationship between course step design and comments posted by learn-
ers. After four iterations of categorizing the course step, five major categories were
drawn, although there are 27 steps in the course that remain unclassified due to their
multiple components. Therefore, in this exploratory study, we are only investigating 64
course steps that could be classified into the five categories we have come up with for
this particular course:
1. Concept (18 Steps): explanation of concepts by using definitions
2. Countries comparison (11 Steps): concepts are explained and relevant issues are
compared across countries, accompanied by graphs or charts.
3. Discussion (14 Steps): all dedicated discussion steps are included in this cate-
gory because in this course, explicit questions are only raised in these steps.
However, there are nuanced differences among the questions.
4. Expert opinion (12 Steps): expert opinions were featured either by their speech
shown in a video or by summary of their published works.
5. UK issues (9 Steps): explanation of concepts with a focus on the UK, although
it should be noted that UK affairs were constantly mentioned throughout the
course.
Table 6. Comment types and course step design
Course Step Design Initiating Posts Lone Posts Replies Initiator’s Further Replies
Replies
Concept 261 193 630 109 137
Countries comparison 151 152 408 52 98
Discussion 596 1341 1023 194 208
Expert Opinion 268 246 655 112 206
UK issues 115 96 276 53 82
A chi-square statistical test showed that the distribution of comments types is signif-
icantly associated with course step design, χ2(16) = 623.68, p<.001 (Table 6). Particu-
larly, replies seem to be the most frequent types of comments in all course step designs
except in the discussion steps, where lone posts dominated. This is rather counter-intu-
itive given that discussion steps are intended for learners to discuss with each other.
However, in this course, quite a few discussion prompts ask for reflection about one’s
own financial status, the pension scheme of one’s own country or personal choice on
housing, so learners might not reply to each other but created a post in response to the
discussion prompt instead. Nonetheless, discussion steps attracted most number of
comments in each category compared to other course step design. Further analysis is
needed for all the discussion steps and the comments posted. Another surprising result
54
is that the course steps dedicated for explaining concepts attracted fewer lone posts than
assumed by the null hypothesis distribution in the chi-square goodness of fit test. One
possible explanation is that most concepts introduced in the course are universal to all
countries and people, so learners had a shared topic such that they could relate and reply
to others’ post more easily. Similarly, steps featuring country comparisons and UK,
also garnered more replies, and fewer lone posts. Lastly, expert opinions not only elic-
ited more replies but also more further replies than expected by the null hypothesis
distribution. Learners seemed to be engaging in more turn-takings when discussing
about opinions by eminent experts.
9.2 Interim Discussion
Overall, this results suggest that learners tended to interact with each other even without
a prompt as long as there is a shared topic or a prominent opinion to converse about. It
could also be possible that learners not only engaged in conversation with oneself when
first encountering a concept, but also attempted to modulate or expand it by communi-
cating with peers [13]. Although this result seems to suggest that the discussion steps
do not produce learners’ interaction as desired despite generating most number of com-
ments, it awaits future research on the nature of discussion prompts in each step, to
determine whether the prompt is designed for reflection or is not well-designed to get
learners to talk to each other. An ad hoc analysis revealed that only one discussion step
has a very high volume of replies (more than double of lone posts), and the step was
presented with arguments against austerity measures in the UK, which inevitably pro-
voked more discussion among learners, the majority of whom were presumably based
in the UK.
Despite being inconclusive, this preliminary analysis demonstrates the potential of
quantifying discussion activities in relation to course steps, such that educators could
be informed of the unusual number of comment types in a particular course step, and
intervene while the course is still running or make revision of the next run of their
course accordingly.
10 Discussion Analytics
As argued in the interim discussions, the analytics of the five comment types at the
level of conversations, social learners and course steps could be leveraged for research
on FutureLearn discussion activities, course monitoring or course revision. The analyt-
ics are visualized below.
10.1 Conversations
In a weekly dashboard on conversations (Figure 2), educators and researchers could
sort the conversations according to the types of comments and the total number of turns
composing them, or the unique learners involved (equal to the number of replies), and
be directed to the initiating post that elicits the particular conversation by clicking on
55
the conversation ID, such that they could read or analyze all the replies underneath the
initiating post. A column indicating the number of replies made by educators could also
be included to monitor educators or facilitators’ activities.
Fig. 2. Prototype Dashboard on Conversations in Week 2
10.2 Social Learners
As suggested earlier, different groups of social learners could be visualized when the
course is half-way through so that educators could support identified individuals for the
remaining of the course. In the dashboard (Figure 3), educators could filter for a partic-
ular group of social learners, and sort according to the number of comment types con-
tributed, replies or likes received, and then be directed to the comments contributed by
individual learner by clicking on their ID. It is also possible to include a column to
indicate whether a learner receive a reply from the educators, if it helps educators to
pay attention to learners that they never interacted with before.
Fig. 3. Prototype Dashboard on Social Learners
10.3 Course Step
Lastly, a dashboard highlighting the most and the least frequent types of comments
in each step could be created (Figure 4). Educators then could identify a step that has
56
unexpectedly generated too many comments of a certain type that is not in line with
their teaching objective. For example, if there are disproportionate number of lone posts
in a step, the educators may want to intervene while the course is still running by post-
ing comments directing learners to some lone posts that are worth reading and com-
menting, or by posting more comments to guide learners how to create initiating posts.
Fig. 4. Prototype Dashboard on Each Course Step
This visualization awaits to be developed and tested with educators to evaluate its
implication in course monitoring, especially its function in sifting through the sheer
number of comments posted in the discussions activities [7]. The discussion analytics
visualized here could be automated from the comment data provided by FutureLearn to
partner institutions who offer courses through its platform. It does not require intensive
manual coding or accuracy check for machine learning because the categorization is
purely based on the commenting structures of FutureLearn. Most importantly, it pro-
vides a systematic way for both educators and researchers to leverage the data currently
available for the investigation of the patterns of conversations and dynamics in the dis-
cussion activities in FutureLearn.
11 General Discussion
To unveil the patterns of conversations among learners in FutureLearn, we catego-
rized the comments in a FutureLearn course into five types: initiating posts that receive
replies, lone posts that receive no replies, replies to others’ initiating post, initiator’s
replies to others’ replies to his/her initiating posts, and further replies when one replies
again to an initiating post. This categorization could be further applied at the level of
conversations and social learners. Beside the number of replies each conversation in-
volves, conversations under each initiating post also vary by the presence of initiator’s
replies, replying learners’ further replies, number of unique learners involved. Lastly,
based on the types of comments contributed and the replies received by each social
learner, seven groups of learners were identified. The preliminary quantitative and qual-
itative analysis based on this categorization revealed the heterogeneity of social learners
as well as the complex conversations that happened in a FutureLearn course. At the
same time, these categorizations provided an analytics method for educators and men-
tors to negotiate through the seemingly overwhelming discussion postings. Educators
57
could identify conversations, course steps and learners with an unusual number of cer-
tain types of comments and intervene accordingly during discussion monitoring or use
this information to revise their course step design for the next run. Lastly, these cate-
gorizations also set the scenes for future research on FutureLearn discussion activities
such that researchers could base their analysis on certain categories of interest to them.
Below we highlight the theoretical basis of our categorization of comments and several
findings from our preliminary analysis that warrant further investigation.
The proposed categorization extended previous research [4, 15] on MOOC discus-
sion that analyzed postings individually by taking into account dynamics and discourse
structures of conversations. It also allows examination of interaction patterns among
learners within the context of conversations and their posting behaviours, instead of the
number of people they have ties with or the position they occupy in a social network.
The differentiation between new (both initiating posts and lone posts) and replies (re-
plies, initiator’s replies and further replies) reflects the computer-mediated discourse
that learners engaged in [24]. Similar to other computer-mediated communication, cre-
ating a post normally is to address all learners in general, whereas replying to a specific
post is to target the initiator of that post or other learners who have replied to that post.
This difference between a conversation in the global context of a course step and a
conversation contextualized to an initiating post is well illustrated by the fact that most
discussion steps in this course generated more lone posts, whereas the longest conver-
sation with 51 turns was elicited by a provoking post in a step without any discussion
prompt. This finding also pointed to the fact that not only discussion prompts and
course steps designed by educators could generate discussion, but learners also play a
role in eliciting discussion. The differentiation between an initiating post and lone post
thus provides us a way to examine how learners’ post could elicit discussion. Further-
more, a differentiation between initiating posts attracting one reply and many replies,
and analysis of the first reply under each initiating post might provide additional insight
on learners’ role in discussion generation.
Secondly, we recognized turn-takings in a conversation by identifying initiator’s re-
plies and replying learner’s further replies, which are contributed by learners who came
back to a conversation that they initiated or replied before. Analyzing conversations
with such turn-takings might reveal how learners react to each other, especially when
there is a disagreement. On the contrary, conversations without such turn-takings beg
the question about why there is a lack of initiator’s replies or further replies, and how a
conversation could be sustained by multiple interlocutors who only contributed once.
As shown in the example cited in this paper, hostile replies might put learners off from
replying again to engage in turn-taking. On the other hand, within both conversations
with and without turn-taking, multiple sub-conversations exist such that individual
learner may only address a specific reply among the many replies underneath the initi-
ating post. This phenomenon might be similar to other computer-mediated communi-
cations where users only addressed the initiating post or one of the many replies under-
neath it [25, 26], by using linguistic strategies such as name addressing or back-chan-
neling to indicate their intended target user [11]. An investigation of these strategies
among the FutureLearn learners, especially in a conversation longer than ten turns, will
58
provide an insight on how learners negotiate through the sheer number of comments in
MOOC discussion.
Although our preliminary analysis on the different types of comments and conver-
sations have been mainly qualitative in nature, it is possible to conduct quantitative
analysis to understand the social dynamics and discourse in MOOC discussion. Chen
and Chiu [28] used content analysis and dynamic multi-level modelling to take both the
content and sequential nature of discussion postings into account in their research on a
university course class forum. They found that earlier messages that expressed disa-
greement or new ideas were more likely to elicit replies from others. It is possible that
the conversations we analyzed in this paper were elicited by initiating posts or replies
with such contents, and this warrant future research. Under our categorization frame-
work, their method could be applied to the level of initiating posts and lone posts to
understand how the discussion evolves in a particular course step. It could also be ap-
plied to the conversational level under each initiating post with an additional variable
that differentiates replies, initiator’s replies and further replies. However, their content
analysis framework was tailored to the sequential nature of discussion, and was differ-
ent from the well-established content analysis framework [17, 18] that considered mes-
sages individually.
Besides considering the conversational dynamics of discussion postings, we also
conducted a preliminary quantitative analysis to investigate the relationship between
the distribution of comment types and course step design, given that teaching design
could facilitate learners’ conversation with themselves and others [14]. FutureLearn
“discussion in context” approach allows such an analysis on the course step levels,
compared to the centralized discussion forum in other MOOC platforms that is detached
from course step design. However, the results in this paper showed that discussion steps
seem to elicit more lone posts than other steps in this particular course that we analyzed.
Further analysis on the prompts in each discussion steps may shed more insights on
design and conversation generated, as Golanics and Nussbaum [29] found in an exper-
imental study that goal instruction and question elaboration in discussion prompts pro-
moted more argumentation in a university course online discussion forum.
Lastly, this paper also recognized the individuality of various types of social learners
when previous MOOC research has only focused on a minority of super-posters [13].
Further analysis of learners’ comments and personal backgrounds such as education
levels and language abilities, as well as in-depth interview on their experiences and
commenting strategies related to participation in discussion will provide valuable in-
sights to educators and MOOC designers to better understand the different groups of
social learners. Furthermore, social network analysis could also be used to understand
the network position of each group of social learners identified in the present study, and
a more detailed classification of social learners may be achieved by combining social
network analysis with our proposed categorization.
59
12 Caveats
Admittedly, the categorization of comment types in this paper is based solely on the
structure in the discussion activities such that the categorization label for some com-
ments may not be valid in light of their content. Specifically, a lone post in our catego-
rization may not be ‘lone’ in content, but could be addressing comments that have been
posted earlier [17], and/or could be similarly responded to by subsequent comments.
For example, “I see comments about Australian pensions… does anyone have a link to
information as to the level of pensions and how they are funded across the developed
world.”. There was a lone post that even explicitly mentioned the name of the other
learner, “I agree with XX. State pension system is as good as it gets. ….” These learners
might choose not to reply directly to the other learners perhaps because there were too
many learners with similar ideas to reply to. As discussed earlier, these lone posts may
be components of a conversation at the global level in a course step that address all
learners in general.
Nonetheless, these posts were considered lone posts as they were standalone, and in
the FutureLearn system, such posts will not trigger notification emails to any learners,
and will not receive notification from anybody, except when being ‘liked’. This con-
trasts with initiating posts whose poster will receive notification for every reply re-
ceived, or a reply that will trigger notifications to the initiator and other learners who
replied before the reply. Therefore, lone posts also differ from the other four comments
types on the ground of this interactional feature.
The second issue with this categorization concerns with vicarious learning and learn-
ers who do not post. It is possible that some of the learners who do not take part in the
discussion activities (67% in the course analyzed in this study) read and ‘like’ some of
the comments. However, we do not know who likes what from the data provided, there-
fore we could not incorporate it into our categorization of learners. It is possible that
the initiators who never replied will have read and liked others’ posts or replies, rather
than fixating on their own initiating posts only. Another way of determining if initiators
read others’ replies to them might be the data of them clicking the notification sent to
them when they receive replies. On the other hand, the ‘like’ count has allowed us to
establish that lone posts received ‘likes’ despite not receiving replies whereas initiating
posts could receive many replies but without any ‘likes’. Still, in this exploratory study,
we did not add this dimension into our categorization of comment types and conversa-
tions, mainly due to the fact that no solid basis has been established to operationalize
the ‘like’ count, for example, the cut-off point to differentiate between well-liked and
less-liked comments.
Third, the classification of social learners is based on the types of comments con-
tributed across the whole course periods. It did not take into account the number and
proportion of different types of comments contributed, ‘like’ received by each social
learners, as well as the weekly participation in the discussion activities. Not every social
learners commented every week in this course, and learners in FutureLearn do not nec-
essarily engage with course materials every week [3]. Therefore, the active social learn-
ers identified in the present study does not necessarily mean that they participated in
60
the discussion every week or every step, but simply refers to the fact that they contrib-
uted initiating post, replied to others’ posts, and engaged in turn-taking by further re-
plying or replying to others’ reply to their initiating posts. An earlier attempt in our
cluster analysis in trying to include weekly participation and proportion of different
types of comments resulted in too many groups that elude any meaningful interpreta-
tion. Nonetheless, our classification successfully identifies every social learners, in-
stead of only the super-posters who received attention so far in MOOC research [12].
Fourth, the comments quoted in the present paper are not representative of all the
comments in the identified categories, conversations or group of social learners. They
happened to be the first instance to show up when we filtered for the examples. A sys-
tematic analysis of the contents of each comment by using well-established content
analysis techniques reviewed in the introduction, conversational analysis or discourse
analysis [11] is warranted to further shed light on the characteristics of each type of
comments, conversations and social learners identified by the categorization approach
we proposed.
Fifth, the analysis is based solely on the first run of one course and this course is full
of contentious issues due to its topic on inequality. The distribution of the comments
types, conversations and group of learners may differ in other courses of different na-
ture or course step design. Nonetheless, because the categorization is based solely on
structural relationships, it could readily be applied to other FutureLearn courses. Addi-
tionally, an analysis of the other runs of the same course will be particularly useful in
understanding the relationship between the distribution of comment types and course
step design presented in this paper. Consistent patterns may point to the influence of
the course design whereas inconsistent patterns may reveal a cohort effect.
REFERENCES
1. Wise, A.F., Cui, Y., Vytasek, J.: Bringing order to chaos in MOOC discussion
forums with content-related thread identification. In: Proceedings of the Sixth
International Conference on Learning Analytics & Knowledge - LAK ’16. pp.
188–197. ACM Press, New York, New York, USA (2016).
2. Wang, X., Wen, M., Rosé, C.P.: Towards triggering higher-order thinking
behaviors in MOOCs. Proc. 6th Int. Conf. Anal. Knowl. 398–407 (2016).
3. O’Riordan, T., Millard, D.E., Schulz, J.: How should we measure online
learning activity? Res. Learn. Technol. 24, 1–28 (2016).
4. Kellogg, S., Booth, S., Oliver, K.: A social network perspective on peer
supported learning in MOOCs for educators. Int. Rev. Res. Open Distrib.
Learn. 15, (2014).
5. Ferguson, R., Sharples, M.: Innovative pedagogy at massive scale: Teaching
and learning in MOOCs. In: Lecture Notes in Computer Science (including
subseries Lecture Notes in Artificial Intelligence and Lecture Notes in
Bioinformatics). pp. 98–111 (2014).
6. Onah, D.F.O., Sinclair, J.E., Boyatt, R.: Exploring the Use of MOOC
Discussion Forums. Proc. London Int. Conf. Educ. 1–4 (2014).
61
7. Hew, K.F., Cheung, W.S.: Students’ and instructors’ use of massive open
online courses (MOOCs): Motivations and challenges. Educ. Res. Rev. 12, 45–
58 (2014).
8. Gillani, N., Eynon, R.: Communication patterns in massively open online
courses. Internet High. Educ. 23, 18–26 (2014).
9. Sunar, A.S., Abdullah, N.A., White, S., Davis, H.C.: Analysing and Predicting
Recurrent Interactions among Learners during Online Discussions in a MOOC.
11th Int. Conf. Knowl. Manag. 4-6 Novemb. 2015. (2015).
10. Tubman, P., Oztok, M., Benachour, P.: Being social or social learning? A
sociocultural analysis of the FutureLearn MOOC platform.
11. Herring, S.C.: Computer-mediated discourse analysis: An approach to
researching online communities. In: Barab, Sasha A ; Kling, Rob; Gray, J.H.
(ed.) Designing for virtual communities in the service of learning. pp. 338–376.
Cambridge ; New York : Cambridge University Press (2004).
12. Huang, J., Dasgupta, A., Ghosh, A., Manning, J., Sanders, M.: Superposter
behavior in MOOC forums. In: L@S ’14 Proceedings of the first ACM
conference on Learning @ scale conference. pp. 117–126 (2014).
13. Laurillard, D.: Rethinking university teaching: A conversational framework for
the effective use of learning technologies. 288 (2001).
14. Laurillard, D.: Teaching as a Design Science : Building Pedagogical Patterns
for Learning and Technology. Hoboken : Taylor and Francis, Hoboken (2012).
15. Wang, X., Yang, D., Wen, M., Koedinger, K., Rosé, C.P.: Investigating how
student ’ s cognitive behavior in MOOC discussion forums affect learning
gains. Proc. 8th Int. Conf. Educ. Data Min. 226–233 (2015).
16. De Wever, B., Schellens, T., Valcke, M., Van Keer, H.: Content analysis
schemes to analyze transcripts of online asynchronous discussion groups: A
review. Comput. Educ. 46, 6–28 (2006).
17. Henri, F.: Computer conference and content analysis. In: Collaborative
Learning Through Computer Conferencing. pp. 117–136 (1992).
18. Garrison, D.R., Anderson, T., Archer, W.: Critical thinking, cognitive presence,
and computer conferencing in distance education. Am. J. Distance Educ. 15, 7–
23 (2001).
19. Bloom, B.S.: Taxonomy of educational objectives : the classification of
educational goals. Handbook 1, Cognitive domain. New York: David McKay
Company., London (1956).
20. Rienties, B., Giesbers, B., Tempelaar, D., Lygo-Baker, S., Segers, M.,
Gijselaers, W.: The role of scaffolding and motivation in CSCL. Comput. Educ.
59, 893–906 (2012).
21. De Laat, M., Lally, V., Lipponen, L., Simons, R.-J.: Online teaching in
networked learning communities: A multi-method approach to studying the
role of the teacher. Instr. Sci. 35, 257–286 (2007).
22. Kellogg, S.B., Booth, S., Oliver, K.M.: A Social Network Perspective on Peer
Support Learning in MOOCs for Educators. Int. Rev. Res. Open Distance
Learn. 15, 263–289 (2014).
23. Rienties, B., Tempelaar, D., Giesbers, B., Segers, M., Gijselaers, W.: A
62
dynamic analysis of why learners develop a preference for autonomous learners
in computer-mediated communication. Interact. Learn. Environ. 22, 631–648
(2014).
24. Herring, S.C.: A Faceted Classification Scheme for Computer - Mediated
Discourse. Discourse. 1–37 (2007).
25. Herring, S.: Interactional Coherence in CMC. J. Comput. Commun. 4, (1999).
26. Tagg, C., Seargeant, P.: Facebook and the discursive construction of the social
network. In: Georgakopoulou, A. and Spilioti, T. (eds.) The Routledge
Handbook of Language and Digital Communication. pp. 339–353. Routledge
(2015).
27. Martin, J. R., White, P.R.: The language of evaluation: Appraisal in English.
Basingstoke: Palgrave Macmillan.
28. Chen, G., Chiu, M.M.: Online discussion processes: Effects of earlier
messages’ evaluations, knowledge content, social cues and personal
information on later messages. Comput. Educ. 50, 678–692 (2008).
29. Golanics, J.D., Nussbaum, E.M.: Enhancing online collaborative
argumentation through question elaboration and goal instructions. J. Comput.
Assist. Learn. 24, 167–180 (2008).