AN OPEN LEARNER MODEL USED BY TEACHERS TO MONITOR SPEED READING LEARNERS Blandine Ginon Matthew Johnson School of Engineering, School of Engineering, University of Birmingham, UK University of Birmingham, UK b.ginon.1@bham.ac.uk d.johnson.1@bham.ac.uk Ali Turker Michael Kickmeier-Rust SEBIT Education and Information Technologies, Knowledge Technologies Institute, Ankara, Turkey Graz University, Austria Ali.Turker@sebit.com.tr michael.kickmeier-rust@tugraz.at ABSTRACT 1 INTRODUCTION The benefits of Open Learner Model for learners have been widely demonstrated: supporting learning An Open Learner Model (OLM) is a learner model and metacognition, facilitating self-monitoring and that is accessible to a user, in an understandable way planning, improving self-assessment skills… In this [2]. The aims to make the model accessible to paper, we investigate the benefits of using an OLM learners are to support learning and cognition, and to for teachers. We present Hızlıgo, a speed reading facilitate self-monitoring and planning [4]. OLMs application and the LEA’s Box OLM, a can also be useful as well for other stakeholders of competency-based OLM intended for both teachers learning, like teachers and parents, in order to help and learners. They have been linked in order build in them help learners and facilitate learners monitoring learner models composed of 50 competencies [8; 10]. Access the learner model can help teachers relative to speed reading that the level is calculated to identify learners’ strengths and difficulties and to based on the outcomes of activities performed in plan and adapt their teaching [11]. Thus, several Hızlıgo. 10 teachers have been using the OLM in order to monitor their class in the context of a 12 day OLM are intended for both teachers and learners intensive course using the speed reading application (e.g. [12;7]), some OLMs offer different Hizligo and involving 87 students. The OLM have visualisations for learners and teachers (e.g. [5]), been regularly used by teachers, using different especially in the cases where the learners are visualisations, mainly in the aim to identify the children (e.g. [6]). However, in these OLMs the strengths and weakness of both their class and their model cannot be built from data coming from an individual students. Teachers found the OLM easy external data source, with a competency-based to use and to understand and helpful for their approach. teaching. In this paper, we investigate the benefits for teachers to use a competency-based OLM, in the CCS Concepts context of a speed reading course. First, we • Human-centered computing➝Human computer introduce Hızlıgo, a speed reading application and interaction (HCI); user model, user studies the LEA’s Box OLM, a competency-based OLM ➝Visualization; Visual analytics; Visualization systems intended for both teachers and learners. Then, we and tools • Applied Computing ➝ Education; present how the OLM have been used in the context Interactive learning environments. of a 12 day intensive course with Hızlıgo, involving 10 teachers and 87 students. KEYWORDS Open learner model, learning analytics, teaching analytics. 2 HIZLIGO SPEED READING APPLICATION Hızlıgo (www.hizligo.com) is an online application intended to help learners to improve their speed reading competencies during a 12 day intensive course. Hızlıgo provides learners with 20 types of activities (see Fig. 2), like ‘letter counting’ where the learner has to count the number of occurrence of a character in a character cloud and ‘picture text’ where the learner has to recognise a picture briefly shown. Several measures can be used to calculate the scores of an activity, like the proportion of good answers, the number of answers and the speed to answers. Hızlıgo recommends a sequence of activities for every day of the course, Fig. 2. Hızlıgo: statistics. but the learners can also access the activities independently. Using Hızlıgo, learners and teachers can visualise statistics regarding the completion rate 3 LEA’S BOX PERSUADABLE of the course and the activity scores (see Fig. 2), OPEN LEARNER MODEL however, it do not provided information with a competency-based approach. The LEA’s BOX OLM is a competency-based open learner model that provides teachers and learners with 12 visualisations [3], from the most simple like skill meters (Fig. 4) to more complex multidimensional visualisations like across time (Fig. 5) and heatmap (Fig. 6). They can be used to visualise different information: groups’ overall level, students’ overall level, the level of one or several students or groups for each competency in the model, and the data coming from activities or information sources. In the simple example of Fig. 3, the teacher monitors 4 students coming from 2 groups. In the ‘Groups/Students’ part of the interface, we notice that the average level of group 9/A is a little stronger than group 9/A. The empty skill meters means that the OLM has no evidence of the level of this student, i.e. the student has not yet performed any activities linked to the OLM. In the ‘Competencies’ part of the interface’, we see the average level of the students for the competency ‘Focusing’ and its 4 sub- competencies. In the ‘activities’ part of the interface, we notice that in average, the students had better scores from the activity ‘Letter counting’ than from ‘Picture Text’ and ‘Word hunt’. Using the filters (see left part of Fig. 3), it is possible to visualise a sub-set of the information available in the OLM. For instance, the teacher can use the filters to see only the information relative to one student in order to identify his/her weakest and strongest competencies [4]. Fig. 1. Hızlıgo: letter counting and picture text activities. Fig. 3. LEA’s Box OLM: teacher interface Fig. 4. Visualisation of the competencies using Table. Fig. 5. Visualisation of the evolution of the students’ models across time. Fig. 6. LEA’s Box OLM heatmap visualisation: link between activities (horizontally) and competencies (vertical). The darker is a case, the stronger are the evidence coming from the corresponding activity for the corresponding competency. 4 EVALUATION the course, a questionnaire has been send to teachers about the OLM. In the context of a 12 day intensive speed reading course in Turkey, 87 secondary-school students from 4.2 Results grade 7 to 11 have been using Hızlıgo. They have been encouraged to use Hızlıgo daily, on the base of The 87 students have performed an average of 61,76 activities in Hızlıgo (median=33, minimum=1, 30 minutes per day. maximum=275). The usages of the OLM by the Teachers have defined in the LEA’s Box OLM teachers are presented in Table 1.The 10 teachers 50 competencies and sub-competencies related to have been using the OLM in an average 7,9 times the speed reading and divided into 5 area (improving during the course; a session of use of the OLM eye muscles, seeing rapidly, focusing, reading and lasted in average 17 minutes. All teachers have been reading and understanding), that have then been using several visualisations, 3 in average, but only linked to the activities provided by Hızlıgo. Every two visualisations have been very regularly used: the time a learner performs an activity in Hızlıgo, the across time (used in 86% of the OLM sessions) and outcome, using several measures, is sent to the OLM skill meters visualisations (used in 56% of the OLM as a piece of evidence for each competencies linked sessions). Teachers frequently used the filters, to this activity. mainly to monitor a given students, in 33% of the In order to monitor their students’ engagement in OLM sessions. the course and the evolution of their competencies, their 10 teachers had the possibility to use the LEA’s Box OLM. Students also had the possibility to use Table 1. Use of the OLM by teachers. the OLM for self-monitoring. In this section, we focus on how the OLM has been used by teachers. Average Median Range 4.1 Participants, Materials and Methods Session of use of the 7.9 5 2 - 29 OLM The 10 teachers came from a same school in Turkey, Time per session 1 teacher had a class of grade 7, 4 teachers had a 17 12 3 – 104 (in min) class of grade 9, 4 teachers had a class of grade 10 and 1 teacher had a class of grade 11. At the Number of 3 2 1 - 10 visualisations used beginning of the course, students and teachers have been introduced to Hızlıgo and the LEA’s Box OLM. All usages have been logged. At the end of Fig. 7. Reasons to use the OLM, perceived ease of use and usefulness of LEA’s OLM. The teachers claimed several reasons to use the online learning applications, teacher assessment and OLM (see left art of Fig. 7): 9 teachers used it to student self-assessment. identify the weaknesses and strengths of individual 10 teachers have been using the LEA’s Box students and of the group, 8 teachers used it to OLM in order to monitor their class in the context of identify the weaknesses of the group and 7 teachers a 12 day intensive involving 87 secondary school used it to identify the strengths of the group. 5 students. The teachers have been using the OLM teachers also used the OLM to compare individual regularly during the course. They were particularly students’ levels or the group’s in different interested in using the across time visualisation in competencies. Most teachers found LEA’s OLM order to see the overall evolution of a student or a easy to use and useful (see right part of Fig. 7): 6 group, the evolution of the level of a competency teachers found it easy to use and found the and the evolution of the scores to an activity. interaction with the system clear and understandable, Teachers were also interested in using the filters 5 teachers found it useful for their teaching and 6 facility, in order to focus on one student or teachers claimed that using LEA’s OLM make their competency. Most of teachers found the LEA’s Box teaching easier and enhance their effectiveness. In OLM easy to use and to understand, and helpful for their comments, teachers also claim an interest of in their teaching, notably to identify the strengths and monitoring the students’ engagement in the course weaknesses of their class as a group or of individual and their regularity. students. These promising results show an Open Learner 5 DISCUSSION AND Model intended for teachers can be a powerful tool CONCLUSION for teachers in order to help them in their teaching by providing relevant learning analytics in a suitable Using the LEA’s Box OLM, it has been possible to way. Teachers seem to be particularly interested in define a set of 50 competencies related to speed seeing an overview of their students’ levels and their reading, and to link them to the activities provided evolution across time, but there are also interested in focusing on one student or one competency. by Hızlıgo. Thus the learner models are built based on evidence coming from the scores to Hızlıgo activities, using different measures. The OLM provided teachers with learning analytics that were not available in Hızlıgo, in order to help them in their teaching. Although it was not the case in this first study, the LEA’s Box OLM can gather information from different data sources, like several 6 ACKNOWLEDGMENTS 5. Bull, S., McEvoy, A. T., Reid, E. (2003). Learner Models to Promote Reflection in Combined Desktop PC / Mobile. Intelligent Learning Environments. This project is supported by the European Workshop on Learner Modelling for Reflection, Commission (EC) under the Information Society Sydney, Australia, p. 199-208. Technology priority FP7 for R&D, contract 619762 6. Bull, S., McKay, M. (2004). An open learner model LEA’s Box. This document does not represent the for children and teachers: Inspecting knowledge level opinion of the EC and the EC is not responsible for of individuals and peers. In Intelligent tutoring any use that might be made of its contents. systems, Springer Berlin Heidelberg, 646-655. 7. Ginon, B., Jean-Daubias, S. (2011). Models and tools to personalize activities on learners profiles. Ed- Media, Portugal. REFERENCES 8. Lee, S. J., Bull, S. (2008). An open learner model to help parents help their children. Technology 1. Bull, S. (in press). Negotiated learner Modelling to Instruction Cognition and Learning, 6(1), 29-51. Maintain Today's Learner Models. Research and 9. Siemens, G., & Long, P.: 2011. Penetrating the fog: Practice in Technology Enhanced Learning. Analytics in learning and education. EDUCAUSE 2. Bull, S. Kay, J. Student Models that Invite the Review, 46(5), 31-40. Learner In: The SMILI:() Open Learner Modelling 10. Van Leeuwen, A. (2015). Learning analytics to Framework. Int. J. Artif. Intell. Educ., vol. 17, 89– support teachers during synchronous CSCL: 120, 2007. balancing between overview and overload. Journal of 3. Bull, S., Ginon, B., Boscolo, C., Johnson, M.D. (in Learning Analytics, 2(2), 138-162. press). Introduction of Learning Visualisations and 11. Vatrapu, R., Teplovs, C., Fujita, N., Bull, S. (2011). Metacognitive Support in a Persuadable Open Towards visual analytics for teachers' dynamic Learner Model, Proceedings of Learning Analytics diagnostic pedagogical decision-making. Proceedings and Knowledge. of the 1st International Conference on Learning 4. Bull, S., Kay, J. (2013). Open Learner Models as Analytics and Knowledge. ACM, 93-98. Drivers for Metacognitive Processes. In International 12. Zapata-Rivera, J. D., Greer, J. (2001). Externalising Handbook of Metacognition and Learning learner modelling representations. In Proceedings of Technologies, Springer New York, 349-365. Workshop on External Representations of AIED: Multiple Forms and Multiple Roles, 71-76.