LEA’s BOX: Practical Competence-oriented Learning Analytics and Open Learner Modeling Michael D. Kickmeier-Rust Susan Bull Dietrich Albert Graz University of Technology University College London Graz University of Technology Knowledge Technologies Institute Knowledge Technologies Institute 8010 Graz, Austria 8010 Graz, Austria +43 316 873 30636 +43 316 873 30640 michael.kickmeier-rust@tugraz.at dietrich.albert@tugraz.at ABSTRACT Big Data and data technologies increasingly find their way into 2. LEA’s BOX school education. Learning Analytics and Educational Data LEA’s BOX (www.leas-box.eu) is a project, funded under the Mining are focal research areas. However, technical solutions EU’s Seventh Framework Programme and stands for a practical often fail to meet the practical requirements of teachers or to LEarning Analytics tool Box, that provides really mirror human learning processes. The LEA’s BOX project aims at developing a practical web platform that hosts tools for a • a competence-centred, multi-source formative theory-based approach to Learning Analytics and that offers tools assessment methodology, to open and negotiate learner models. • based on sound psycho-pedagogical models (i.e., Competence-based Knowledge Space Theory and Keywords Formal Concept Analysis), Learning analytics, data visualization, open learner models. • intelligent model-based reasoning services, • innovative visualization techniques, 1. INTRODUCTION Using Learning analytics and educational data mining are more • and features to open and negotiate learner models; than recent buzz words in educational research: they signify one LEA’s BOX is dedicated to develop a learning analytics toolbox of the most promising developments in improving teaching and that is intended to enable educators to perform competence- learning. While many attempts to enhance learning with mere centered, multi-source learning analytics, considering their real technology failed in the past, making sense of a large amount of practical needs. Thus, the project spends significant efforts on a data collected over a long period of time and conveying it to close and intensive interaction with educators in form of design teachers in a suitable form is indeed the area where computers and focus groups and piloting studies. technology can add value for future classrooms. However, The tangible result of LEA’s BOX manifest in form of a Web reasoning about data, and in particular learning-related data, is not platform (Figure 1) for teachers and learners provide links to the trivial and requires a robust foundation of well-elaborated psycho- existing components and interfaces to a broad range of pedagogical theories. educational data sources. Teachers will be able to link the various tools and methods that they are already using in their daily The fundamental idea of learning analytics is not new. In essence, practice and that provide software APIs (e.g., Moodle courses, the aim is using as much information about learners as possible to electronic tests, Google Docs, etc.) in one central location. More understand the meaning of the data in terms of the learners’ importantly, the platform hosts the newly developed LA/EDM strengths, abilities, knowledge, weakness, learning progress, services, empowering educators to conduct competence-based attitudes, and social networks with the final goal of providing the analysis of rich data sets. A key focus of the platform will enable best and most appropriate personalized support. Thus, the concept teachers not only to combine existing bits of data but to allow of learning analytics is quite similar to the idea of formative assessment. “Good” teachers of all time have strived to achieve exactly this goal. However, collecting, storing, interpreting, and aggregating information about learners that originates from a school year, or even in a lifelong learning sense) requires smart technology. To analyse this vast amount of data, give it educational meaning, visualize the results, represent the learner in a holistic and fair manner, and provide appropriate feedback, teachers need to be equipped with the appropriate technology. With that regard, a substantial body of research work and tools already exist. This project aims to continue and enrich on-going developments and facilitate the broad use of learning analytics in the “real educational world. Figure 1. Central web platform Copyright © 2016 for the individual papers by the papers' authors. Copying permitted only for private and academic purposes. This volume is published and copyrighted by its editors. LAL 2016 workshop at LAK '16, April 26, 2016, Edinburgh, Scotland. them to “generate” and collect data in very simple forms, not provides a basis for structuring a domain of knowledge and for requiring sophisticated hard- or software solutions. Finally, we representing the knowledge based on prerequisite relations. We want to open new ways to display the results of learning analytics interpret the performance of a learner (e.g., mastering an addition - leaving the rather statistical dashboard approach, moving task) in terms of holding or not holding the respective towards structural visualizations and towards opening the internal competency. In addition, recent developments of the approach are learner models. based on a probabilistic view of having or lacking certain competencies. In our example, mastering one specific addition 3. Open Learner Models task allows the conclusion that the person is able to add two Learner models contain and dynamically update information numbers (to hold this competency) only to a certain degree or regarding users’ learning: current knowledge, competencies, probability. When thinking of a multiple-choice item with two misconceptions, goals, affective states, etc. There is an increasing alternatives, as another example, mastering this item allows only trend towards opening the learner model to the user (learner, to 50 percent that the person has the required competencies/ teacher or other stakeholders) to support reflection, encourage knowledge. On the basis of these fundamental views, CbKST is greater learner responsibility for their learning, and help teachers looking for the involved entities of aptitude (the competencies) to better understand their students (Bull & Kay, 2010). The core and a natural structure, a natural course of learning in a given requirement is that such visualizations must be understandable to domain. For example, it is reasonable to start with the basics (e.g., the user. Although this may appear to be similar to the more the competency to add numbers) and increasingly advance in the recent work on LA, open learner models (OLM) concentrate more learning domain (to subtraction, multiplication, division, etc.). As on the current state of learners, with less references to activities indicated above, this natural course is not necessary linear, which undertaken, scores obtained, materials used, contributions made, bears significant advantages over other learning and test theories. etc. OLMs typically focus on concepts, competencies, and guiding learners with regard to conceptual issues rather than specific As a result we have a set of competencies in a domain and activities and performance. Various OLM visualization examples potential relationships between them. In terms of learning, the have been described in the literature for university students (see relationships define the course of learning and thus which Bull & Kay 2010, for a more detailed overview). The most competencies are learned before others. Because of the common visualizations used in courses include skill meters, mathematical nature, we can develop LA algorithms on this basis. concept maps and hierarchical tree structures. The results, in turn, provide not only summative analyses but also In addition to visualizing the learner model, various methods of formative and predicative information, as well as model-based interacting with the learner model exist, ranging from simple recommendations (Kickmeier-Rust & Albert, 2015). inspectable models, which allow some kind of additional evidence to be input directly by users, to negotiated learner models, in 5. OUTLOOK which the content of the learner model is discussed and The project is now in in its second year and a stable, open, and potentially updated. We focus on the latter. Key features of configurable web platform with a set of internal tools as well as a negotiated learner models are not only that the presentation of the negotiable OLM instance is available. Presently we are running learner model must be understandable by the user, but also that use case studies on a large scale in Austria, Germany, the Czech the aim of the interactive learner modelling should be an agreed Republic and Turkey. The results so far a highly promising and model. Most negotiated learner models are negotiated between the reveal the potential added value of theory-driven LA and open student and the teaching system. However, other stakeholders can presentation of results to educators and teachers. also be involved, and the notion of “the system” can be broadened to include a range of technologies, such as the ones used in 6. Acknowledgements technology-enhanced learning. Here we consider (i) fully- This work is supported by the European Commission (EC) under negotiated learner models; (ii) partially-negotiated learner models; the 7th Framework Programme for research and development as and (iii) other types of learner model discussion. They are all well as the running LEA’s BOX project, contracted under number relevant to our notion of negotiating the learner model or its 619762. This document does not represent the opinion of the EC content, and they are adapted for LEA’s BOX (Figure 2). and the EC is not responsible for any use that might be made of its theoretical approach to do so is Competence-based Knowledge content. Space Theory (CbKST, Albert & Lukas, 1999). The approach is a mathematical psychological, set-theoretic framework for 7. References addressing the relations among problems (e.g., test items). It Bull, S. & Kay, J. (2010). Open Learner Models, In R. Nkambou, J. Bordeau and R. Miziguchi (Eds.), Advances in Intelligent Tutoring Systems (pp. 318-338), Berlin-Heidelberg: Springer- Verlag. Albert D. & Lukas J. (1999). Knowledge spaces: Theories, empirical research, applications. Mahwah: Lawrence Erlbaum Associates. Kickmeier-Rust, M. D., & Albert, D. (2015). Competence-based Knowledge Space Theory: Options for the 21st Century Classrooms. In P. Reimann, S. Bull, M.D. Kickmeier-Rust, R. 4. COMPE-TENCE-BASED KNOWLEDGE Vatrapu, and B. Wasson (Eds.), Measuring and Visualizing SPACES Learning in the Information-Rich Classroom (pp. 109-120). New As claimed initially, in the context of formative LA, a York, NY: Routledge. Figure competence-oriented 2. OLM in approach is LEA’s BOX An elaborated necessary.