The focus of education is increasingly set on students' ability to regulate their own learning within technology-enhanced learning environments. Scaffolds have been used to foster self-regulated learning, but scaffolds often are standardized and do not do not adapt to the individual learning process. Learning analytics and machine learning offer an approach to better understand SRL-processes during learning. Yet, current approaches lack validity or require extensive analysis after the learning process. The FLORA project aims to investigate how to advance support given to students by i) improving unobtrusive data collection and machine learning techniques to gain better measurement and understanding of SRL-processes and ii) using these new insights to facilitate student's SRL by providing personalized scaffolds. We will reach this goal by investigating and improving trace data in exploratory studies (exploratory study1 and study 2) and using the insight gained from these studies to develop and test personalized scaffolds based on individual learning processes in laboratory (experimental study 3 and study 4) and a subsequent field study (field study 5). At the moment study 2 is ongoing. The setup consists of a learning environment presented on a computer with a screen-based eye-tracker. Other data sources are log files and audio of students' think aloud. The analysis will focus on detecting sequences that are indicative of micro-level self-regulated learning processes and aligning them between the different data sources.
The FLORA project aims to improve measurement of self-regulated learning by using multimodal
learning analytics. Self-regulated learning (SRL) occurs when learners monitor and regulate content
they access and operations they apply to content as they pursue goals to augment and edit prior
knowledge [
SRL consists of cognitive activities related to learning the content and meta-cognitive activities
related to regulation. Sub-categories of cognition refer to student’s strategic information processing
during learning such as reading the information, repeating it as well as deeper information
processing like elaboration, and organization of information processed. The metacognitive activities
include five categories: planning, goal specification, orientation, monitoring, reflection, and
evaluation which refer to the postulated metacognitive activities during SRL. When zooming in on
these categories, many micro-level processes might be detected, such as content evaluation and
monitoring progress towards learning goals in the category: monitoring [
The aim of study 1 was to measure micro-level SRL processes. A learning environment was presented to students. The task for the students was to learn about three topics and to write an essay. Before and after this task, students’ knowledge about the topics was assessed. Preliminary results show that there is a significant learning gain. The challenge is to link the learning gain to SRL processes. The objective is to analyze each data source (think aloud, log data, and eye-tracking) and extract behaviors that are indicative of micro-level SRL processes, see Fig. 1 for a schematic overview.
To improve traceability of SRL processes, instrumentation tools have been added in study 2. These include a timer, a note-taking and highlighting application, a planner tool, a search function, and a hybrid read-write mode of the essay in which both the text and essay is visible. These tools allow learners to reveal SRL processes, which should be reflected in improved traceability in think aloud, log, and eye-tracking data. Aside from the instrumentation tools the setup is the same is in Study 1.
Three types of data were gathered: log files (mouse and keyboard), audio, and gaze. To record data a mouse, keyboard, microphone, and eye-tracker were used. The participant was seated in front of a monitor with a screen-based eye-tracker, microphone, keyboard, and mouse. The stimuli were presented on this monitor. In future studies, feedback will be provided as well, see Fig. 2 for an overview of the technical infrastructure.
Copyright © 2020 for this paper by its authors. Use permitted under Creative Commons License Attribution 4.0 Internationa2l (CC BY 4.0). Audio was recorded to measure think aloud data. The participants were instructed to think aloud. Think aloud consisted of reading text, stating learning goals, mentioning the creation of notes, stating navigational actions, etc.. To make sense out of the log files, logs have to be interpreted in the context of the learning environment. Mouse clicks mostly indicate navigation, while typing is most common for the note-taking function and the essay. Gaze data was recorded using a screenbased eye-tracker. The raw data consisted of a timestamp (sampling rate is 300 Hz), coordinates of the where the participants is looking, pupil dilation, Areas Of Interest (AOIs) if enabled, and more. For all data sources, a coding scheme is needed to label and analyze the data. Analysis will focus on sequences of actions that can be indicative of specific SRL processes.
The first part of the workshop consists of collecting data with the presented setup and a shorter task (15 minutes). In the second part, three groups will each investigate a single data source (think aloud, log data, or eye-tracking). The goal is to extract and analyze SRL processes and identify the value of instrumentation tools. To do so, data and a coding scheme will be provided. During this process, each group evaluates the data source in relation to detection of SRL. Advantages and disadvantages will be identified and discussed at the end when the groups come together to share the results.
Copyright © 2020 for this paper by its authors. Use permitted under Creative Commons License Attribution 4.0 Internationa3l (CC BY 4.0). Copyright © 2020 for this paper by its authors. Use permitted under Creative Commons License Attribution 4.0 Internationa4l (CC BY 4.0).