Personalization Approaches in Learning Environments

Keywords

• Proceedings edited by

-

PALE 2015 held in conjunction with http://ceur-ws.org/

The Student Advice Recommender Agent: SARA

Jim Greer, Stephanie Frost, Ryan Banow, Craig Thompson, Sara Kuleza, Ken Wilson and Gina Koehn

Personalising e-Learning Systems: Lessons learned from a vocational education case study

Lie Ming Tang and Kalina Yacef

Modeling Learner information within an Integrated Model on standard-based representations

Mario Chacón-Rivas, Olga C. Santos, Jesus G. Boticario

Patterns of Confusion: Using Mouse Logs to Predict User’s

Emotional State

Avar Pentel

Using Problem Statement Parameters and Ranking Solution

Difficulty to Support Personalization

Rômulo C. Silva, Alexandre I. Direne and Diego Marczal

International Workshop on Personalization

Approaches in Learning Environments (PALE 2015)

Preface

Milos Kravcik1, Olga C. Santos2, Jesus G. Boticario2,

Maria Bielikova3, Tomas Horvath4 Abstract. Personalization approaches in learning environments are crucial to foster effective, active, efficient, and satisfactory learning. They can be addressed from different perspectives and also in various educational settings, including formal, informal, workplace, lifelong, mobile, contextualized, and selfregulated learning. PALE workshop offers an opportunity to present and discuss a wide spectrum of issues and solutions. In particular, this fifth edition includes 6 papers dealing with adapting the study plan (with highlighting), student’s performance (i.e., academic distress), self-regulating learning skills, interoperability in learner modelling by integrating standards (i.e., IMS specification), confusion detection by monitoring mouse movements in a computer game, and knowledge acquisition of mathematical concepts. 1

Introduction

The 5th International Workshop on Personalization Approaches in Learning Environments (PALE)1 took place on June 30th, 2015 and was held in conjunction with the 23rd conference on User Modeling, Adaptation, and Personalization (UMAP 2015). Since the topic can be addressed from different and complementary perspectives, PALE workshop aimed to offer a fruitful crossroad where interrelated issues could be 1 http://adenu.ia.uned.es/workshops/pale2015/ contrasted and discussed. PALE 2015 was a follow-up of the four previous editions of PALE (which took place at UMAP 2011 – 2014).

In order to foster the sharing of knowledge and innovative ideas on these issues, PALE format follows the Learning Cafe methodology2 to promote discussions on open issues regarding personalization in learning environments. Three Learning Café sessions were set up for this year PALE edition. Each one consisted of brief presentations of the key questions posed by two workshop papers and subsequent small group discussions with participants randomly grouped at tables. Each table was moderated by the presenter of the paper. In the middle of the session, participants changed tables to promote sharing of ideas among the groups. The workshop ended with a summary of the discussions on each paper. In this way, participants attending the workshop could benefit both from interactive presentations, constructive work and knowledge sharing.

The target audience of the PALE workshop includes researchers, developers, and users of personalized and adaptive learning environments. As a long-standing workshop series (for 5 years now, annually run at UMAP) PALE workshop has established itself as a mature channel for disseminating research ideas on personalization of learning environments. This could not be possible without the very much appreciated involvement of the program committee members (many of them supporting PALE all along these years) as well as the active participation of authors who have selected this venue to disseminate and discuss their research. To compile the progress achieved in this field, a special issue on User Modeling to Support Personalization in Enhanced Educational Settings taking into account extended versions of previous contributions to PALE (in addition to papers from an open call) is being guest edited by PALE organizers in the International Journal of Artificial Intelligence in Education3.

In the following, we introduce PALE 2015 motivation and themes as well as present an overview of the contributions accepted and discussed in the workshop. 2

Motivation and Workshop Themes

Personalization is crucial to foster effective, active, efficient, and satisfactory learning, especially in informal learning scenarios that are being demanded in lifelong learning settings, with more control on the learner side and more sensitivity towards context. Personalization of learning environments is a long-term research area, which evolves as new technological innovations appear.

Previous PALE editions have shown several important issues in this field, such as behavior and embodiment of pedagogic agents, suitable support of self-regulated learning, appropriate balance between learner control and expert guidance, design of personal learning environments, contextual recommendations at various levels of the learning process, tracking affective states of learners, harmonization of educational and technological standards, processing big data for learning purposes, predicting student outcomes, adaptive learning assessment, and evaluation of personalized learning solutions.

From the past experience, we have identified new research areas of interest to complement the previous ones. Nowadays there are new opportunities for building interoperable personalized learning solutions that consider a wider range of learner situations and interaction features in terms of physiological and context sensors. However, in the current state of the art it is not clear how this enhanced interaction can be supported in a way that positively impacts on the learning process. In this context, suitable user modeling is required to understand the current needs of learners. There are still open issues in this area, which refer to providing open learner models in terms of standards that cover the extended range of available features and allow for interoperability with external learning services as well as taking advantage of the integration of ambient intelligence devices to gather information about the learner interaction in a wider range of learning settings than the classical desktop computer approach.

Therefore, these new features are paving the way to other related topics that are to be considered in the learner modeling, including affective states of the learner as well as changing situations in terms of context, learners' needs and their behavior. Another broad research area addresses personalization strategies and techniques, considering not only the learner model, but the whole context of the learning experience, including the various technological devices that are available in the particular situation.

In this workshop edition we drew attention to sharing and discussing the current research on how user modeling and associated artificial intelligent techniques contextualize the world and provide the personalization support in a wide range of learning environments, which are increasingly more sensitive to the learners and their context, such as: intelligent tutoring systems, learning management systems, personal learning environments, serious games, agent-based learning environments, and others. We are especially interested in the enhanced sensitivity towards learners' interactions (e.g., sensor detection of affect in context) and technological deployment (including web, mobiles, tablets, tabletops), and how this wide range of situations and features may impact on modeling the learner interaction and context. Furthermore, we aim to cover the every time more demanding need of personalized learning at large-scale, such as in massive open online courses (MOOCs).

The higher-level research question addressed in this workshop edition was: “Which approaches can be followed to personalize learning environments?” It is considered in various contexts of interactive, personal, and inclusive learning environments. The topics of the workshop included (but were not limited to) the following: • Affective computing • Ambient intelligence • Personalization of MOOCs • Learning recommendation • Learner and context awareness • Cognitive and meta-cognitive scaffolding • Social issues in personalized learning environments • Open-corpus educational systems • Adaptive mobile learning • Successful personalization methods and techniques • Reusability, interoperability, scalability • Evaluation of adaptive learning environments 3

Contributions

A peer-reviewed process has been carried out to select the workshop papers. Three members of the Program Committee with expertise in the area have reviewed each paper. As a result, 6 submissions (out of 8) were accepted, which discuss ideas and progress on several interesting topics, such as adapting the study plan (with highlighting), student’s performance (i.e., academic distress), self-regulated learning skills, interoperability in learner modelling by integrating standards (i.e., IMS specification), confusion detection by monitoring mouse movements in a computer game, and knowledge acquisition of mathematical concepts.

Tintarev et al. [ 1 ] focus on the effect of emphasis adaptation in a study plan, which is represented as a workflow with prerequisites. They compare the effectiveness of highlighting when the adaptation was correct (participants responded quicker and more correctly), and when it did not highlight the most relevant tasks (detrimental effect). They found that false statements took longer to process than positive statements (deciding about things that were not in the plan), but also surprisingly had lower error rates than positive statements. In their view, these findings imply that errors in the adaptation are harmful, and may cause students to incorrectly believe that they do not need to do certain tasks.

Greer et al. [ 2 ] present SARA, the Student Advice Recommender Agent, which is similar to an early alert system, where predictive models of learners’ success combined with incremental data on learners’ activity in a course are used to identify students in academic distress. SARA can detect when the student is struggling academically and then provides notifications with a personalized advice how to get back on track. The system represents a scalable advice personalization environment in large university courses and delivers weekly advices. The authors have observed a significant year over year improvement in unadjusted student grades after the SARA’s advice recommender was implemented in a 1200-student freshman STEM course.

Tang and Yacef [ 3 ] address the challenge of time and environment management. They report on their experience with a leading vocational education provider in Australia (i.e., training of specific skills or trades, often done part time or in personal time over a lengthy period) who is transitioning from classroom-based training to a pilot elearning system. They present the key lessons learned and the prototype goal-setting and time management interface designed to improve user self-regulation. A growing body of evidence suggests that these self-regulating skills are a key determinant in learning performance and can be improved with computer aided support, increasing engagement and motivation of trainees.

Chacón-Rivas et al. [ 4 ] identify open issues when it comes to integrate the information from the learner activity in standards-based learner models, which covers learning styles, competences, affective states, interaction needs, context information and other learner´s characteristics. In particular, there are standards that can be used to cover several of the subjects to be integrated into those models, such as IMS-LIP, IMS-RDCEO, IMS-AFA. Authors present their on-going work in implementing a learner model that aims at providing a holistic user modelling perspective, which is able to hold and collects all relevant information, thus supporting its real-life usage. This approach is expected to facilitate interoperability and sustainability, while still research needs progressing where representation and management is required.

Pentel [ 5 ] describes an unobtrusive method for user confusion detection by monitoring mouse movements. A special computer game was designed to collect mouse logs. Users’ self-reports and statistical measures were used in order to identify the states of confusion. Mouse movement’s rate, full path length to shortest path length ratio, changes in directions and speed were used as features in the training dataset. Support Vector Machines, Logistic Regression, C4.5 and Random Forest were used to build classification models. Those models generated by Support Vector Machine yield to best classification results with fscore 0.946, thus showing that frequent direction changes in mouse movement, are good predictors of confusion.

Silva et al. [ 6 ] approach theoretical and implementation issues of a framework aimed at supporting human knowledge acquisition of mathematical concepts. They argue that personalization support can be achieved from problem statement parameters, defined during the creation of Learning Objects and integrated with the skill level of learners and problem solution difficulty. The last two are formally defined as algebraic expressions based on fundamental principles derived from extensive consultations with experts in pedagogy and cognition. Their implemented prototype framework, called ADAPTFARMA, includes a collaborative authoring and learning environment that allows short- and long-term interactions. 4

Conclusions

In this 5th edition of PALE contributions address several gaps identified in the state of the art, such as adapting the study plan (with highlighting), student’s performance (i.e., academic distress), self-regulated learning skills, interoperability in learner modelling by integrating standards (i.e., IMS specification), confusion detection by monitoring mouse movements in a computer game, and knowledge acquisition of mathematical concepts..

Nevertheless, other issues remain open such as the integration of ambient intelligence devices to gather information about the learner interaction in a wider range of learning settings than the classical desktop computer approach, aimed to enhance the sensitivity towards learners' interactions through diverse technological deployments (including web, mobiles, tablets, and tabletops), impacting on modeling the learner interaction and context. We expect that future editions in PALE can progress on aforementioned directions.

Acknowledgements

PALE chairs would like to thank the authors for their submissions and the UMAP workshop chairs for their advice and guidance during the PALE workshop organization. Moreover, we also would like to thank the following members of the Program Committee for their reviews (in alphabetical order): Miguel Arevalillo, Mihaela Cocea, Sabine Graf, Peter Henning, Mirjana Ivanovic, Jelena Jovanovic, Iolanda Leite, Noboru Matsuda, Alexander Nussbaumer, Alexandros Paramythis, Lubomir Popelinsky, Elvira Popescu, Sergio Salmeron-Majadas, Natalia Stash, Christoph Trattner, Carsten Ullrich, Stephan Weibelzahl, Michael Wixon.

The organization of the PALE workshop relates and has been partially supported by the following projects: BOOST: Business perfOrmance imprOvement through individual employee Skills Training, LEARNING LAYERS: Scaling up Technologies for Informal Learning in SME Clusters (FP7 ICT-318209), MAMIPEC: Multimodal approaches for Affective Modelling in Inclusive Personalized Educational scenarios in intelligent Contexts (TIN2011-29221-C03-01), MARES: Multimodal and Machine learning techniques to recognize emotions in educational settings (TIN2011-29221C03-02), Supervised Educational Recommender System (VEGA 1/0475/14), and Virtual Learning Software Lab for Collaborative Task Solving (KEGA 009STU4/2014).

References

1. Tintarev , N. , Green , M. , Masthoff , J. , Hermens , F. Benefits and risks of emphasis adaptation in study work flows . In proceedings of the 5th Workshop on Personalization Approaches for Learning Environments (PALE 2015 ). Kravcik, M. , Santos , O.C. , Boticario , J.G. , Bielikova , M. , Horvath , T. (Eds.). 23rd conference on User Modeling , Adaptation, and Personalization (UMAP 2015 ), CEUR workshop proceedings, this volume, 8 - 15 ( 2015 )

2. Greer , J. , Frost , S. , Banow , R. , Thompson , C. , Kuleza , S. , Wilson, K. , Koehn , G.. The Student Advice Recommender Agent: SARA . In proceedings of the 5th Workshop on Personalization Approaches for Learning Environments (PALE 2015 ). Kravcik, M. , Santos , O.C. , Boticario , J.G. , Bielikova , M. , Horvath , T. (Eds.). 23rd conference on User Modeling , Adaptation, and Personalization (UMAP 2015 ), CEUR workshop proceedings, this volume, 16 - 23 ( 2015 )

3. Tang , L.M. , Yacef , K. Personalising e-Learning Systems: Lessons learned from a vocational education case study . In proceedings of the 5th Workshop on Personalization Approaches for Learning Environments (PALE 2015 ). Kravcik, M. , Santos , O.C. , Boticario , J.G. , Bielikova , M. , Horvath , T. (Eds.). 23rd conference on User Modeling , Adaptation, and Personalization (UMAP 2015 ), CEUR workshop proceedings, this volume, 24 - 30 ( 2015 )

4. Chacón-Rivas , M. , Santos , O.C. , Boticario , J.G. Modeling Learner information within an Integrated Model on standard-based representations . In proceedings of the 5th Workshop on Personalization Approaches for Learning Environments (PALE 2015 ). Kravcik, M. , Santos , O.C. , Boticario , J.G. , Bielikova , M. , Horvath , T. (Eds.). 23rd conference on User Modeling , Adaptation, and Personalization (UMAP 2015 ), CEUR workshop proceedings, this volume, 31 - 39 ( 2015 )

5. Pentel , A. Patterns of Confusion: Using Mouse Logs to Predict User's Emotional State . In proceedings of the 5th Workshop on Personalization Approaches for Learning Environments (PALE 2015 ). Kravcik, M. , Santos , O.C. , Boticario , J.G. , Bielikova , M. , Horvath , T. (Eds.). 23rd conference on User Modeling , Adaptation, and Personalization (UMAP 2015 ), CEUR workshop proceedings, this volume, 40 - 45 ( 2015 )

6. Silva , R.C. , Direne , A.I. , Marczal , D. Using Problem Statement Parameters and Ranking Solution Difficulty to Support Personalization . In proceedings of the 5th Workshop on Personalization Approaches for Learning Environments (PALE 2015 ). Kravcik, M. , Santos , O.C. , Boticario , J.G. , Bielikova , M. , Horvath , T. (Eds.). 23rd conference on User Modeling , Adaptation, and Personalization (UMAP 2015 ), CEUR workshop proceedings, this volume, 46 - 51 ( 2015 )