Designing Conversational Agent Interventions that Support Collaborative Chat Activities in MOOCs

Abstract

Although conversational agent technology has matured over time, there is still a need for research on how agents can appropriately add value to real-world technological learning environments. This paper presents an ongoing research effort towards the design of low-cost, reusable conversational agents that deliver unsolicited interventions during online chat-based activities. These interventions aim at helping learners sustain a productive peer dialogue in the context of online courses. We expect this work to enlighten researchers, conversational interface designers and bot developers on the potential of conversational agents in serving as automated facilitators of synchronous collaborative learning in MOOCs.

Keywords

• Conversational Agents
• Computer-Supported Collaborative Learning
• MOOCs
• Productive Dialogue
• Conversational Interventions

-

1.1

Background

Conversational Agents

Conversational interfaces are on the rise. Humans are increasingly communicating with computers in human terms, leveraging the power of natural language interactions [ 1 ]. This is accomplished by utilizing the advancements in conversational agent technology. A conversational agent, also known as a chatbot or virtual assistant, is a computer-based artificial entity developed to engage in a dialogue with one or more human users [ 2 ]. Based on their design, conversational agents may be available on websites, smartphones, or smart speakers and communicate with users via audio, text or other non-verbal methods, such as gestures.

Given the recent advances in natural language understanding, conversational agents can now provide a new convenient way of interacting with users in a personalized and engaging manner. Considering that such agents can be effectively used to automate a series of tasks and processes, interesting new implementations of conversational agents have emerged in numerous scenarios and applications [ 3 ]. There are many success stories surrounding the usage of conversational agents, created to meet a wide variety of needs in many sectors, such as healthcare, finance, customer service, marketing, retail, human resources and tourism. Nevertheless, although the increased level of engagement and support provided by conversational agents has been shown to hold tremendous potential for enterprises, the realization of such agents in the real-world educational environments has been limited. 1.2

Agents Potential in Education

In the field of technology-enhanced learning, research has indicated that using conversational agents to engage learners in one-to-one (student-agent) tutorial dialogues can improve students’ comprehension and foster students’ engagement and motivation [ 4 ]. Such agents try to simulate the behavior of a human instructor or tutor and engage in a discussion with a learner on a series of predefined topics.

Although the main research interest of the past focused on the creation of agents operating in individual learning settings, researchers have also explored the design of conversational agents supporting collaborative learning activities [ 5 ]. Research in the field of Computer-Supported Collaborative Learning has revealed that unsolicited conversational agent interventions can intensify the knowledge exchange among learning partners and increase students’ explicit reasoning and participation levels [ 6 ]. Agent-based supportive mechanisms can positively impact the quality and conceptual depth of students’ conversations and, consequently, the learning outcomes [ 7 ].

With the recent rise in focus on MOOCs and the positive impact of conversational agents in social learning settings, researchers have recently begun to explore the utilization of agent-based facilitation in the context of MOOCs [ 8 ]. It was found that conversational agents can increase students’ engagement, minimize dropout rates and leverage the support that students often provide to each other by themselves [ 9 ]. In large-scale learning scenarios, such as universities or massive open online courses (MOOCs), agents can be really useful for providing continuous learning support. Indeed, a conversational agent may be able to compensate for the insufficient individual support of instructors, which constitutes one of the key factors negatively affecting retention rates [ 10 ]. Overall, conversational agents supporting group activities appear to have a direct application in MOOCs. 2

Conversational Agent Interventions in MOOC Chat Activities

Under the prism of a research project, called “Integrating Conversational Agents and Learning Analytics in MOOCs (colMOOC)”, this section summarizes our work-inprogress towards the design of a new generation of conversational intervention modes, which are domain-independent and provide collaborative learning support in the context of MOOCs. Considering the lack of guidelines as regards the proper design of conversational agent interventions that support learning in groups, we believe that this work can contribute to the understanding of how the facilitation of collaborating groups in MOOCs can be automated using conversational agents.

The colMOOC project focuses on the creation of teacher-configurable, reusable agents that are primarily rule-based and have a low developmental cost [ 11 ]. More specifically, those agents are designed as intelligent tools that enhance the impact of the facilitation strategies employed by the teacher. The configuration of the agents can be performed in a visual agent builder environment, called ‘editor’, by creating a series of conceptual links, such as the one displayed in Fig. 1 (for example, [computational thinking] [is not the same as] [algorithmic thinking]). Eventually, all the conceptual links created in the editor shape the agent domain model for a teacherdefined task. A task typically asks learners to collaborate in dyads in the context of a chat-based MOOC activity in order to provide a joint response to an open-ended question defined by the teacher.

The creation of those agile conversational agents is inspired by the work of the teachers’ community on modelling useful classroom discussion practices and norms, forming what is known as the framework of Academically Productive Talk (APT) [12]. Drawing upon this framework, these agents aim at delivering a series of interventions (or moves) as a means to trigger productive forms of peer dialogue and scaffold students’ learning, regardless of the educational domain (see Fig. 2). Considering that researchers universally value the explicit articulation of reasoning, as well as the existence of references and connections among items of articulated reasoning [13], this kind of agent interventions aims to help learners sustain a transactive form of dialogue. The latter is regarded as a highly productive form of peer dialogue, where students use one another as information resources and build on each other’s reasoning. While building such conversational intervention mechanisms for collaborative learning, attention is not given on thoroughly modelling each learner’s understanding by using complex knowledge structures for each different domain, but on identifying efficient techniques of modelling and triggering constructive peer interactions through fine-tuned agent interventions. Intervention Strategy Welcome Enhances group awareness Onboarding Contributes to the user onboarding process and serves as an “ice-breaking” tactic Disconnection Enhances group awareness Exit Enhances group awareness and suggests a solution to the user left alone Re-connection Enhances group awareness Task completion Enhances group awareness

Conversational agent interventions emerge following the identification of an associated ‘pattern’. A pattern refers to an event or a combination of events, which take place in an online chat activity and may be of interest for the agent to detect and analyze as an opportunity for triggering an intervention. Usually, a pattern is regarded as a combination of something uttered by the human discussants along with some contextual information of what is going on in the chat environment. In the scope of the colMOOC project, patterns fall into one of the following categories: (a) static patterns, referring to one or more events that are independent of the dynamic evolvement of the peer interaction, and (b) dynamic patterns, referring to contextual events arising from the analysis of peer utterances and the identification of certain key concepts (keywords/phrases and their synonyms), set by the teacher when building the conceptual links, i.e. the agent domain ontology.

Table 1 presents a list of agent intervention strategies arising from the detection of one or more static patterns. Such patterns do not require a real-time text analysis of the peer dialogue and may refer, for example, to MOOC platform events like entering a chat, disconnecting from a chat activity, completing an activity or submitting a task team answer. Those are predefined events that relate to the context of the chat activity but not to the agent domain model, i.e. the available conception links.

Table 2 presents a list of agent intervention strategies that emerge from the identification of dynamic patterns, utilizing information that derives from both conceptual links and predefined contextual chat events. These strategies emphasize the critical role of social interaction in inducing beneficial mental processes, drawing on the findings of previous CSCL studies that have already shown the benefits of displaying APT agent interventions during students’ synchronous collaboration [ 6 ][ 7 ]. This paper provides an empirical foundation for automating conversational interventions in the context of small-group chat activities in MOOCs. The proposed facilitation strategies are operated by a teacher-configurable conversational agent, which adopts an event-driven approach and operates on the basis of specific patterns that serve as intervention opportunities. Without requiring a large development effort, this kind of agent-based facilitation can enable MOOCs to provide valuable contextresponsive support during chat-based learning activities, scaffolding and improving the quality of peer discussions. Nevertheless, future experimentation is needed to finetune the design of agents and create intervention mechanisms, which have considerable pedagogical value and are flexible enough to be used in different discussion contexts without requiring a lot of setup effort. We plan to conduct a series of studies to explore the use of the proposed agent intervention modes in MOOCs. Acknowledgements & Disclaimer. This research has been funded by the Erasmus+ Programme of the European Commission (project No 588438-EPP-1-2017-1-ELEPPKA-KA). This document reflects the views only of the authors. The Education, Audiovisual and Culture Executive Agency and the European Commission cannot be held responsible for any use which may be made of the information contained therein.

References

1. Klopfenstein , L. C. , Delpriori , S. , Malatini , S. , & Bogliolo : A. The rise of bots: A survey of conversational interfaces, patterns, and paradigms . In: Proceedings of the 2017 Conference on Designing Interactive Systems , pp. 555 - 565 ( 2017 ).

2. Araujo , T. : Living up to the chatbot hype: The influence of anthropomorphic design cues and communicative agency framing on conversational agent and company perceptions . Computers in Human Behavior , 85 , 183 - 189 ( 2018 )

3. McTear , M. , Callejas , Z. , & Griol , D. : Creating a conversational interface using chatbot technology . In: The Conversational Interface , pp. 125 - 159 , Springer, Cham ( 2016 ).

4. Graesser , A. , & McDaniel , B. : Conversational agents can provide formative assessment, constructive learning, and adaptive instruction . In: The Future of Assessment , pp. 85 - 112 ( 2017 ).

5. Caballé , S. , & Conesa , J. : Conversational Agents in Support for Collaborative Learning in MOOCs: An Analytical Review . In: International Conference on Intelligent Networking and Collaborative Systems , pp. 384 - 394 ( 2018 ).

6. Adamson , D. , Ashe , C. , Jang , H. , Yaron , D. , & Rosé , C. P. : Intensification of group knowledge exchange with academically productive talk agents . In: N. Rummel , M. Kapur , M. Nathan , & S. Puntambekar (Eds.), CSCL Conference Proceedings, pp. 10 - 17 ( 2013 ).

7. Tegos , S. , & Demetriadis , S. : Conversational agents improve peer learning through building on prior knowledge . Journal of Educational Technology & Society , 20 ( 1 ), 99 - 111 ( 2017 ).

8. Rosé , C. P. , & Ferschke , O. : Technology support for discussion based learning: From computer supported collaborative learning to the future of massive open online courses . International Journal of Artificial Intelligence in Education , 26 ( 2 ), 660 - 678 ( 2016 ).

9. Ferschke , O. , Yang , D. , Tomar , G. , & Rosé , C. P. : Positive impact of collaborative chat participation in an edX MOOC . In: International Conference on Artificial Intelligence in Education , pp. 115 - 124 , ( 2015 ).

10. Hone , K. S. , & El Said , G. R. : Exploring the factors affecting MOOC retention: A survey study . Computers and Education , 98 , 157 - 168 ( 2016 ).

11. Demetriadis , S. , Tegos , S. , Psathas , G. , Tsiatsos , T. , Weinberger , A. , Caballé , S. , Dimitriadis , Y. , Sánchez , G. E. , Papadopoulos , M. , & Karakostas , A. : Conversational Agents as Group-Teacher Interaction Mediators in MOOCs . In: 2018 Learning With MOOCS (LWMOOCS) , pp. 43 - 46 ( 2018 ).