This paper examines how an explicit theory of learning as conversation has informed design of the FutureLearn MOOC platform. We describe a process of pedagogy-informed systems design and show how Conversation Theory has provided a framework for design that combines learning as conversation with instruction through structured content. The paper compares performance metrics across three MOOC platforms. Results show higher levels of social engagement, with comparable completion rates, for the FutureLearn platform.
This paper describes how the FutureLearn MOOC platform was designed from an explicit pedagogy of learning through conversation. It addresses the question: Can a free open platform based on a pedagogy of learning as conversation enhance learning at large scale?
In order to answer the question, we describe the educational design of the FutureLearn platform and learner experience. We also cover: the process for designing, implementing and testing new elements of the FutureLearn learner experience, theories of learning and interaction that informed the platform design, evidence of effectiveness of the design decisions based on a comparison of key performance indicators for MOOC platforms, and indications of the learner and educator experience based on learner comments.
We should make clear from the start that the aim of this paper is not to discuss whether one MOOC platform is better than another. Instead, the claim is that the performance of learners with FutureLearn is comparable to two other major platforms (Coursera and edX) and, in addition, FutureLearn enables a form of learning through asynchronous conversation that works at scale and complements direct instruction. Learning by building shared understanding through conversation offers a distinctly different experience to learning through direct instruction.
The focus of this paper is pedagogy-informed design of educational technology, whereby a study of existing theory and practice of learning informs a series of design experiments, each producing new implementations, practices and theories of learning. There are many pedagogies that can inform development of future tools for learning, including game-based learning, experiential learning, inquiry-led learning and adaptive teaching. Understanding which of these can work at scale and how they can be incorporated into existing platforms can provide a rationale for design of new technologyenhanced learning. 2
The method of pedagogy-informed design has similarities to contextual design [
In early 2013, the software development team at FutureLearn was challenged to
implement a robust and scalable platform capable of supporting courses provided by the
initial 12 partner universities, and further anticipated institutions, with the first course
running within 10 months (by October 2013). The company established an Agile
software development process with Scrum [
Scrum has the advantage of delivering rapid incremental changes to the software product, providing clarity and evidence that the software is being implemented on schedule. It also places severe constraints on innovation. Items added to the product backlog must be capable of implementation within a two-week timespan, requiring a rapid iterative process of design, implementation and testing. This approach can lead to ‘hillclimbing’ with each incremental feature added to the previous ones, missing opportunities for more radical innovation in a different part of the design space.
Thus, to enable pedagogy-informed design of the FutureLearn platform, it was necessary to provide the software development team with a framework that instantiated a theory of effective learning at scale that could also be implemented in parts, with each part being developed during a two-week sprint session. Since the sprint process was being used to deliver the entire product, not just the educational elements, the educational software design had to fit with the Scrum approach. Through discussion between advisors from The Open University and the software team, a framework of learning through conversation was adopted, described below.
To evaluate effectiveness of the pedagogy-informed design of FutureLearn, we present a comparison of high-level indicators (registrants, learners, active learners, social learners, and completing learners – these categories are defined below) across three major MOOC platforms: FutureLearn, Coursera and edX. To provide further comparative data with control for international differences, a comparison is made of courses run by The University of Edinburgh, which offers MOOC courses on both the Coursera and FutureLearn platforms.
The MOOC platforms have evolved over time, incorporating features from other MOOC providers. Coursera and edX now include elements of conversational learning (such as conversation alongside content) and FutureLearn has added elements of direct instruction. Thus, to give the most direct answer to our research question, we use historic data (from 2013-2014) to compare pedagogies at the time when the platforms were first implemented. 3
An important point to note as we outline learning as conversation is that this is a
comprehensive theory of the cognitive and social processes of learning, not simply a
description of online discussions. It is based on a cybernetic systems theory of learning
that stands alongside behaviourist, cognitivist and socio-cultural theories. As initially
formulated by Pask [
Cybernetics is the study and design of systems for regulation and control, not through external diktat or command, but from self or mutual feedback and modification. In cybernetic terms, learning through conversation is more than an exchange of knowledge – it is a self-regulating process in which intelligent organisms employ a mutually evolving language to share and negotiate differences in understanding, with the aim of constructing new knowledge and reaching agreements.
A cybernetic conversation is a recursive process of coordinating understanding at differing levels of abstraction. Higher level coordinations (mutual reflections) refer to lower-level coordinations (shared objects and events). For their interactions to constitute a conversation, learners must be able to formulate descriptions of their reflections on actions, explore and extend those descriptions, and carry forward the understanding to a future activity. This is the basis for human action-oriented dialogue – for example, when two students perform an experiment together, discuss the results and what went wrong, then plan how to re-run the experiment.
Fig. 1 depicts learning as conversation, based on diagrams from Scott [
Conversations with others can occur at the levels of actions and descriptions. Each level requires a shared medium and an evolving language. At the level of actions, a learner and one or more partners (who may be other learners, teachers, or even a computer-based tutoring system) discuss a practical activity or model of the world. For example, a teacher may set a maths problem to solve, or a historical event to interpret, or a computer may provide a runnable model of a mechanical system to explore. The learners converse in the context of that model or problem, asking ‘how’ questions, sharing experiences and interpretations. The aim is to coordinate the action so that the learners’ expectations and understandings mesh with the teaching materials. Not only must the teaching content and models be appropriately designed and relevant, they must provoke reflective conversation. For example, learning computer science involves understanding the structure and process of computer code alongside terms such as ‘variable’, ‘conditional’ and ‘recursion’. If the learners find it difficult to converse, because their shared language cannot adequately express ways in which materials and understanding are coordinated, then learning may not take place.
At the level of descriptions, learners converse about why things happen, offering conceptions of their learning and questioning the understanding of others, in attempts to reach agreement about their reflective understandings. A different shared medium is needed for these conversations, one that can support a process of coming to know through constructive argumentation, where each learner expresses and adjusts conceptions in relation to the expressed understanding of others.
At both levels, learners need to agree on clear goals and objectives. Although the process of learning through conversation is exploratory, with learners managing their own activities and reflective discussions, there is a strong role for an educator in proposing goals and objectives, creating suitable activities and models to explore, facilitating discussions and prompting reflection. 4
What makes the conversational framework different to other theories of experiential and reflective learning, and learning through mutual discussion, is that it is intended to be an implementable model for learning mediated by technology. Pask was an educational technologist. He designed adaptive teaching systems and examined how people engage in self-managed learning of formalised topic maps. He also studied systems of conversation in theatre, art, architecture and music.
The formation of FutureLearn provided an opportunity to rethink the Pask/Laurillard framework for learning at large scale. A group of academics from The Open University, led by the authors, advised the FutureLearn software product team on pedagogy-informed design of the platform. Together, they considered a range of social constructivist learning theories and chose to implement Conversation Theory, as it provides explicit guidance on how to implement a process of learning through social interactions.
The framework, as outlined above and depicted in Fig. 1, indicates that MOOCs, to support effective conversations for learning at scale, should satisfy the following requirements:
Consistent language: Platform developers and educators should develop a consistent language to describe educational actions and descriptions and this should be made explicit to learners. This is more than a software usability principle – the shared language of pedagogy is the means for educators and learners to discuss their progress towards mutual understanding.
Pedagogy elements: The platform should distinguish different types of pedagogic media and practice (e.g. narrative, interactive, communicative, argumentative, assessing) so that educators can design for appropriate conversational sequences.
Conversations for action and description: The platform should distinguish conversations for learning at the levels of action and description, with the former directed towards understanding topics and solving problems, and the latter aimed at exploring differences in conception and reaching agreements.
Conversations in context: Conversations at the level of action should be directed towards specific topics and conducted in the context of narrative and interactive media.
Reflective conversations: The platform and learning design should prompt internal reflective conversations as well as social interactions. Learning can occur by analysing and modifying one’s own actions, and by reflecting on the conversations of others.
Conversations for coordinating agreements: Conversations at the level of descriptions should be initiated and structured to enable a process of ‘coming to know’ by sharing perspectives, synthesising new knowledge, and reaching agreements.
Conversational media: Each conversation should be associated with a medium that allows open investigation of the topic, revealing the contribution made by each learner to the evolving discussion.
Explicit objectives, outcomes and goals: Educators should set specific objectives and outcomes for a course, and provide a medium that enables learners to negotiate personal and shared goals.
Facilitation: The primary role of educators should be to facilitate conversations for learning around well-structured content.
Tracking conversations: Educators should be supported to understand the learners’ conceptions and, where appropriate, to adjust learning objectives as the conversations progress.
This is a distinctly different foundation for design of a MOOC platform and courses
to the ones that were in currency when FutureLearn was developed. At that point,
connectivist cMOOCs were based on learning through active connection of knowledge
objects [
Learning through conversation on MOOCs is not a replacement for direct instruction, but an adjunct to it. The shared medium comprises video, audio and text materials that are designed for individual reflective learning as well as prompted discussion. 5
Design of the FutureLearn platform and its courses is directly informed by the requirements for a conversational framework outlined in the previous section.
Consistent language: FutureLearn has developed a consistent ‘pattern language’ for its pedagogy, as well as visual design and interaction1. The courses are designed according to principles of effective learning through conversation, narrative, visible learning and shared celebration of progress. Each new feature on the platform is accompanied by a terminology and set of evidence-based concepts that are communicated to course developers, educators and learners2.
Pedagogy elements: The platform was designed to support construction of courses using pedagogy building blocks (named ‘steps’) that can be put together in different combinations to form learning activities. The main FutureLearn step types are: article, video & audio, discussion, peer review, quiz, poll, and exercise. A FutureLearn ‘activity’ is the minimum complete pedagogy element. It is associated with learning objectives, enacted through a sequence of learning steps. 1 https://www.futurelearn.com/pattern-library 2 https://www.futurelearn.com/using-futurelearn/why-it-works
Conversations for action and description: The platform supports learning as conversation in various ways. Associated with each step in the course is a flow of comments and replies (Fig. 2). A learner can read the comments, leave a new comment, or reply to a previous comment. An educator can also add comments and replies. In addition to this conversation at the level of actions, discussion steps provide spaces for conversation at the level of descriptions. Typically, learners are guided to reflect on their recent activity, share current understanding, and engage in constructive argument.
Conversations in context: To stimulate conversations alongside article, audio and video steps, course designers are encouraged to pose questions that can be answered by a short comment building on a learner’s prior experience or current knowledge (see Fig. 2 for an example). The platform is designed so that learners add short comments alongside the teaching content to continue the flow of conversation.
Reflective conversations: A fundamental design principle is that learners should not be required to interact socially: it should be possible to complete any course without contributing to conversation, but by reading and reflecting on the conversations of others. Contributing to conversations with others enhances this internal process.
Conversations for coordinating agreements: Peer review is designed as a conversation to reach agreements. Learners are asked to respond to assignments from other learners, following a three-part rubric, typically: ‘How clear is the argument’, ‘How well has the writer demonstrated a knowledge of the concepts?’, and ‘How well has the writer expressed the argument in a coherent way?’. Peer review is typically followed by a discussion step, where learners are encouraged to reflect on and discuss their assignments and reviews. In quizzes and tests, the responses to correct and incorrect answers are presented as educator comments, continuing the conversational style of interaction. More recently, Study Groups have been implemented to enable conversation within small groups aimed at reaching agreements in relation to key questions or problems.
Conversational media: Each learning element (step) is associated with a distinct medium where learners can engage in conversation, usually with a question from the educator prompting the addition of comments based on personal experience or response to the teaching materials (see Fig. 2). Learners (who are aged 13 or over) are asked to be open in revealing their real names and are encouraged to introduce themselves and post relevant details on their personal profile.
Explicit objectives, outcomes and goals: The course team is encouraged to provide objectives for a course in the form of ‘big questions’ that are presented to learners in the course description. In the platform’s course creator tool, the academic team sets explicit learning outcomes using a structured format. At the start of a course, learners have an opportunity to set personal goals that not only provide a reference for them to reflect on progress but also provide data to the course team about how learner goals align with course objectives.
Facilitation: A course team can assign specific roles to members, including the roles of educator – with specialist knowledge of the subject – and mentor to guide discussions. These roles are visible to learners (see Fig. 2). Team members can facilitate study group discussions through broadcast prompts and by contributing to any discussion.
Tracking conversations: The course team has access to a dataset that includes each comment, its associated step, timestamp and author. A dashboard allows the team to track the progress of each study group conversation.
The FutureLearn platform was designed from the outset to be scalable and responsive, with cloud hosting to cope with a large number of users and an interface that can be accessed on mobile devices. It was not, however, possible to predict in advance the scale and nature of the conversations for learning. Contingencies were made for abusive comments (through reactive moderation of learner comments) and lack of engagement (by enabling the contextual commenting to be switched off). What was not anticipated by the FutureLearn design team was the large scale of positive commentary. During early course runs, in autumn 2013, some steps of a course attracted over 1,000 comments and replies.
To manage this scale of learner contribution, the FutureLearn team added elements of social networking. Learners can ‘follow’ their peers and educators. They can ‘like’ comments and replies, and filter comments in various ways, including by ‘most liked’ (see Fig. 2). Each personal profile shows the learner’s recent comments with a link from each to its place in the conversation, as well as followers and people being followed.
A typical interaction with a course step is to read or view the teaching content ending with the educator’s prompt for conversation, click to view the most recent items in the flow of comments and replies (perhaps ‘liking’ a good comment and ‘following’ its author), click ‘Most liked’ to see favoured comments, then add a reply or post a new comment to the flow. Learners are encouraged to follow educators and can receive email notifications of new comments by people they follow. If an educator replies to a posting, followers are notified of that action, meaning that the reply is likely to attract viewings and likes, raising it up the list of most-liked comments. These social network mechanisms enable comments favoured by educators and other learners to ‘rise to the top’ of the conversation so, for any step in the course, a learner can switch between its current conversational flow and its prioritised comments and replies. 6
Not only the design of the FutureLearn platform, but also the courses that run on it, have been informed by a pedagogy of learning as conversation. In this paper, we focus on the conversational and social networked learning aspects of FutureLearn. However, for comparison with other aspects of the learning experience, Fig. 3 shows learner responses in a typical large FutureLearn course to the post-course survey item: “Please rate from ‘strongly disliked’ to ‘strongly liked’ how you felt about learning on FutureLearn”. A majority of respondents strongly liked to learn by reading articles and watching videos. Sixty percent of respondents liked or strongly liked to learn by reading the comments of other learners. Thirty percent liked or strongly liked to learn by online discussion with other learners.
Open responses to survey questions and in User Voice feedback indicate that some learners find the large number of comments and responses daunting. Others view the contextual conversation as an additional resource for learning.
FutureLearn educators now routinely design activities to support learning through conversation. To give one example, the course “Rome: A Virtual Tour of the Ancient city” from the University of Reading is based on an interactive 3D model of ancient Rome. For each location, such as the Colosseum, learners are asked to explore the environment, then discuss their personal experiences and how these changed their understanding of how people lived in the city. 7
There can, in general, be no direct measures of learning on MOOC courses, since learners volunteer to study and their topic knowledge is not compared at the start and end of a course. Instead, all major course providers use measures of activity, engagement and course completion as proxies for learning. For this paper we also provide a comparison of social engagement as a proxy for learning through conversation.
Table 1 presents a top-level view of these analytics. In FutureLearn, of those who register for a course (‘registrants’), a mean of 53% visit at least one step (‘learners’). The number of learners on a course is taken as the baseline for further analysis.
FutureLearn learners are asked to mark each step as complete when they finish it (see Fig. 2). The table shows the percentage of learners who: mark as complete one step or more (active learners), post one or more comments (social learners), or finish the course by marking at least half the steps as complete and submitting all assessment (completing learners).
Table 1 offers comparative analytics with the two largest MOOC platforms,
Coursera and edX [
To provide a fair comparison, only courses lasting five weeks or more and offering tests were included for FutureLearn, since this is the typical format for courses on the other platforms. The figures for all FutureLearn courses in 2014 (including shorter courses and those without tests) are more favourable for FutureLearn: mean registrants: 12,354; learners: 52%; active learners: 82%; social learners: 39%; completing learners: 23%.
Published data for Coursera are only available for a single course on Business
Strategy. However, a general figure for the percentage of Registrants who completed
Coursera courses was given in 2013 as 5%: “In total, roughly 5 percent of students who
signed up for a Coursera MOOC earned a credential signifying official completion of
the course” [
The results show that the number of registrants for FutureLearn during 2013-14 was smaller than for Coursera and edX, as the platform had been more recently established. The figures for learners (those who started a course) and active learners (learners who engaged with the course material) are comparable across the three platforms. The figure of 17% who complete a FutureLearn course compares well with other major MOOC platforms and suggests that a MOOC platform based on learning as conversation is overall more effective in retaining learners than those founded on instructivist approaches. The most striking difference is in the percentage of learners who post at least one comment, with a figure of 36% for FutureLearn, compared to 9% for Coursera and 12% for edX. This indicates that a substantial proportion of learners are engaging actively in learning through conversation.
The University of Edinburgh has provided the authors with unpublished data on the 22 Coursera courses it had run by 1 March 2015. These data offer an even closer comparison of platforms, since Edinburgh is both a partner of FutureLearn and of Coursera and all its courses are designed to foster community and engagement. The figures are broadly similar, except for social learners, with 13% for Edinburgh Coursera and 20% for Edinburgh FutureLearn courses. 8
We return to the question posed at the start of the paper: Can a free open platform based on a pedagogy of learning as conversation enhance learning at large scale?
Evidence of effectiveness comes from the scale of contributions plus data from the post-course survey. From the outset, FutureLearn courses generated a large number of comments and replies, especially when compared to other MOOC platforms. Because of this, mechanisms from social networks – including liking, following, user profiles and filters – were added to manage social engagement. Surveys of learners who have completed courses show that approximately two-thirds like or strongly like reading comments and one third like contributing to discussion online. FutureLearn partners are guided in designing courses that can appeal to the two-thirds who do not contribute, while offering easy ways to participate in conversation for people with widely differing skills and backgrounds, as well as opportunities to learn by viewing and browsing contributions from peers.
We have conducted a fair comparison of learner experience with the two other largest MOOC platforms, by covering courses during 2013 and 2014 (before other platforms adopted some facilities for social networked learning). Table 2 only includes those FutureLearn courses that lasted five weeks or more and offered tests, since that was the standard format for courses on Coursera and edX. A separate comparison was made with Coursera courses offered by the University of Edinburgh, which is also a partner in FutureLearn. The comparisons show a similar or greater retention rate for FutureLearn courses and much higher active social engagement. That social engagement continues in current courses, with a current average of 38% of learners making one or more contributions.
The conversational framework could be further extended for large-scale learning. Pask proposed that learners should interact not only with individual instructional materials but also with ‘entailment structures’ of interlinked topics, similar to concept maps. Structural relations between topics, similar to Pask’s entailment structures, already underlie adaptive teaching systems such as Knewton (www.knewton.com) and ALEKS (www.aleks.com). These topic networks could be made visible to learners, allowing them more agency to choose strategies for exploring the learning content.
A challenging new area of research is to explore how providing tools for personalised exploration of topics can be reconciled with supporting social and collaborative learning. When learners take different paths through a course, or adopt different strategies for exploration, they may find it difficult to identify a shared basis for discussion. Yet if many learners from diverse backgrounds can be helped to explore a complex subject along differing pathways, they could each contribute a perspective to an emergent conversation. By focusing design effort on the interwoven conversations for learning (rather than on individuals or content) and enabling many people to contribute in individual ways, we may be able to create a MOOC equivalent to Wikipedia, where people learn by building, intersecting, discussing and sharing pathways to learning. 9
Pickering [
Other pedagogies could also offer variety in learning at scale and allow new forms of respectful interaction. Inquiry-based learning supports learners in thinking like scientists. Game-based learning is based on learners working together to solve problems and create self-organising communities. Experiential learning builds knowledge from the shared experiences of learners. Collaborative design thinking encourages learners to work together on creative projects.
These pedagogies privilege exploration, diverse opinions and play over structured instruction and assessment of learning outcomes. In developing the FutureLearn platform and courses we have attempted to provide a space for a process of learning through global conversation alongside a system of direct instruction through online video, text and formative assessment.
We thank members of the FutureLearn advisory group, including Russell Beale, Simon Buckingham Shum, Andrew Law, Patrick McAndrew, Peter Scott and Martin Weller, as well as colleagues at FutureLearn including Laura Kirsop, David Major, Simon Nelson, Kathryn Skelton and Matt Walton.