This work aims to propose a learner model-based feedback model for self-direction skills (SDS) acquisition to address the challenge of providing learning services with multimodal data in K-12 settings. The feedback model leverages students' daily life activity data from learning systems and wearable devices, creates a learner model of SDS, and provides multi-dimensional feedback to K-12 students: (1) feedback on contextual activity; (2) feedback on self-direction management and (3) feedback on skill assessment. We also implement the feedback model with two learning dashboards in English learning and physical activity contexts for illustration and show the potential effects of the feedback model on student engagement and skill improvement with two case studies in K-12 settings. The results of the case studies indicate that K-12 students can continuously engage in both learning and physical activities, and take the feedback regularly with the learning dashboard support. The implications and challenges of SDS development with multimodal data in K-12 settings are also discussed. Learning analytics, feedback, self-direction skill, multimodal data, learner model
Self-direction skills (SDS) are the basis of lifelong learning and a healthy lifestyle [1, 2]. They have been identified as increasingly important skills in the 21st century [3, 4, 5]. SDS can be defined as a process in which learners take their own responsibility to direct one’s learning and life to meet personal goals [6, 7, 8]. Similarly to self-regulated learning, SDS is also concepted as a combination of internal monitoring and external management of the learning experience [28]. The strength of SDS is its external management features involving the design of learning environment and the planning of learning trajectory [29]. There is a great need for students to develop SDS following the shift from teachercentered traditional classrooms to learner-centered learning environment with advanced technologies [9].
Learners face several critical challenges when they develop a high level of SDS in K-12 educational settings. The K-12 young learners lack the self-directed contexts to practice their SDS [10] and they also have limited technology support to assess their SDS with actionable feedback [11].
It is becoming a trend to utilize technologies in education, and students’ learning behaviors in a learning environment can be automatically recorded by learning tools or systems [13]. Meanwhile, available physical and physiological data in learner’s daily life are rapidly increased since the development of wearable devices and sensing techniques [14, 15]. The physical and physiological data expands the educational choices available for traditional K-12 educational settings. Such multimodal trace data provide new contexts to influence young learners’ learning processes and outcomes in their daily life [12, 16]. EMAIL:
1);
2020 Copyright for this paper by its authors.
Learning Analytics (LA) can be used to deal with the large amount of data in learner’s daily life and make sense of it for the learner [20]. Multimodal data (MMD) from various data channels is increasingly being deployed in LA research to help us better understand, model and support learning processes [21, 22]. Therefore, it’s possible to create more practice opportunities by utilizing MMD and provide meaningful feedback through LA techniques.
We aim to propose a learner-centered feedback model for the development of SDS by integrating MMD with LA techniques to address the challenges of providing LA service with MMD in K-12 settings [24]. The model will leverage the MMD synchronized from the learners’ daily activity contexts and provide three-dimensional feedback for the contextual activity, self-direction management, and skill assessment. In order to implement the feedback model, we work out two LA dashboards in learning and physical activity contexts. Finally, we show two case studies to evaluate the effectiveness of implemented learning dashboards in K-12 settings. The challenges of supporting lifelong learning by leveraging MMD will be discussed.
One of the objectives of this work is to present how to utilize MMD to support skill development. We begin with the description of SDS acquisition layers as our fundamental design principle.
Figure 1 gives a general view of the SDS acquisition layers in the learners’ daily life activities. The bottom layer is the contextual activity of daily life with MMD, the middle layer is the self-direction management with multimodal indicators and the upper layer is the SDS assessment.
Self-direction skill assessment
Self-direction management • Multimodal indicator collection
• Activity status analysis • Planning • Progress monitoring
• Strategic Reflection
Contextual activities of daily life • Learning (i.e., Extensive reading with e-book tools) • Physical (i.e., Walking with smartwatch sensors) • Health (i.e., Sleep with wearable devices)
Firstly, contextual activities are executed by learners in their daily life. The activity is any daily routine activity such as reading in learning or walking in physical activity. The raw activity data could be recorded by a large range of tools or sensors. Once the raw multimodal data is captured, the data could be extracted to high-level multimodal indicators for learners by using a human rubric such as body mass index (BMI) or a computer algorithm such as computer vision algorithm. • Learners can learn a wide range of subject knowledge in the daily learning context. For example, learners can improve their English reading fluency and the joyful of learning through extensive reading activities. • Learners can engage in various exercise in the daily physical activity context. For example, learners can increase their fitness levels and teamwork ability through basketball play.
Secondly, self-direction actions are managed by learners themselves in different activity contexts. The key self-direction actions are extracted from a data-informed process model named DAPER (Data collection - Analysis - Planning - Execution monitoring - Reflection) [25]. It is synthesized by the theory of self-directed learning, self-regulated learning, and quantified self. It is a process model with five phases, the initial phase of data collection, which gives learners the initiative in their contextual activities, followed by the other four phases: data analysis, planning, execution monitoring, and reflection. The self-direction actions in management are classified into the following five interactive phases: (a) multimodal indicator collection; (b) activity status analysis; (c) planning; (d) progress monitoring; (e) strategic reflection.
Finally, learners’ SDS is assessed by human tutors or computers. Importantly, both contextual activity data and self-direction actions contribute to the skill assessment. The result of the skill assessment can be seen by learners for their skill improvement in the specific context.
The overview of learner model-based feedback for self-direction skill acquisition is shown in Figure 2. The model starts with the three layers of SDS acquisition in the previous section: contextual activities, self-direction management, and skill assessment. Then it creates a learner model using activity data, management data, and assessment data. Through the learner model, the feedback will be provided to learners in three dimensions: feedback on contextual activity, feedback on self-direction management, and feedback on skill assessment. Three-dimensional feedback will scaffold learners from specific activity engagement to general skill development.
Feedback for
activity Activity data Contextual activities
Feedback for management Learner Model Management
data Self-direction Management
Feedback for assessment Assessment data
Skill assessment
We implement our learner model-based feedback for illustration with two LA dashboards. One dashboard is provided for self-directed extensive reading in learning contexts and another one is given for self-directed daily walking in physical activity contexts. Two dashboards provide different contextual information but give similar feedback for self-direction management and skill assessment. We deliver the feedback through learning dashboard instead of email or notification intervention because learning dashboards could provide real time visual feedback to the students.
To create the LA dashboard for self-directed extensive reading, an existing e-book learning system named BookRoll[26] and a self-direction support system named GOAL [27] are connected with the learning dashboard. The reading activity data is automatically synchronized from a cloud-based Learning Record Store (LRS), which records learning logs from the e-book learning system and stores the data in the Experience API (xAPI) standard format. An example log of e-book reading in the xAPI format is “Learner A read the page 2 of the e-book B58 for 25 seconds from 3 pm today”. The selfdirection actions in management are synchronized with a self-direction support system, which tracks all self-direction events. An example action of self-direction is “Learner A created a daily plan for time spent in extensive reading”.
The user interface of the dashboard for self-directed extensive reading is shown in Figure 3. Three
feedback modules are created in the dashboard: (
To create the LA dashboard for self-directed daily walking, a physical activity data aggregator is
connected with the self-direction support system and learning dashboard. The physical activity data is
(
The user interface of the dashboard for self-directed daily walking is shown in Figure 4. Three
feedback modules are also created in the dashboard: (
To evaluate the learner model-based feedback, we carried out two case studies in learning and physical activity contexts respectively. In the learning context, self-directed extensive reading was conducted using the first implemented learning dashboard. A total of 108 seven-graders (average age of 13 years old) from a junior high school in Japan participated in the 5-week study. Students were encouraged to read picture e-books extensively in the e-book learning system and managed selfdirection actions in the self-direction support system at their own pace. For instance, students could select a fiction book to read from over 500 e-books and set personal reading goals with a daily target value in reading pages. In the physical activity context, self-directed daily walking program was conducted using the second implemented learning dashboard. A total of 66 seven-graders from the same junior high school participated in the 11-week study. Students were recruited to wear a Garmin smartwatch voluntarily. They were encouraged to track their daily walking activity using the smartwatch, synchronize their walking data to Garmin smartwatch data platform through their computer application, and manage self-direction actions in the self-direction support system at their own pace. For instance, students can record total walking time of after-school walking and set personal walking goals with a daily target value in walking time.
We show the preliminary results of the two case studies by analyzing one active learner’s engagement with feedback support. The learner, named learner A, was selected from the group whose engagement levels are both high in learning and physical activity contexts.
The engagement level of learner A in self-directed extensive reading is shown in Figure 5. The engagement levels in reading activity, self-direction management, skill assessment update, and feedback check are presented by a heatmap form. Each line has one color scale and a darker cell means a higher level of engagement in the cell. The results show that the reading pages of reading activity increased after week 1 and the highest number of reading pages was achieved in week 4. The action count of self-direction management reached its highest in week 5. Skill assessment was updated more frequently in week 2 and week 5 than the ones in other weeks. Importantly, the participant checked the feedback on the dashboard more frequently in week 4 and week 5 than the ones in other weeks.
Learner A
The engagement level of learner A in self-directed daily walking is shown in Figure 6. The engagement levels in physical activity, self-direction management, skill assessment update, and feedback check are also presented by a heatmap form. The results show that the step count of daily walking in week 1 was the lowest and the one in week 6 was the highest. The action count of selfdirection management reached its highest in week 11. Skill assessment was updated more frequently in week 1, week 8 and week 11 than the ones in other weeks. Importantly, the student checked the feedback on the dashboard more frequently in week 1 and week 8 than the ones in other weeks.
Learner A
This work presents a conceptual feedback model for SDS acquisition by integrating multimodal technologies with the design of learning technologies. Then we implement the feedback model with two LA dashboards in learning and physical activity contexts. We carried out two case studies in K-12 settings to evaluate the effectiveness of implemented LA dashboards. The results are shown by visualizing the engagement levels in contextual activity, self-direction management, skill assessment, feedback check from one of the active participants.
The LA dashboard provided summative visual feedback with activity data, management data and skill assessment data together. That makes students can engage in skill development after they understand the activity data and the relation between activity data and other data. The potential of utilizing MMD for K-12 students’ skill development was shown in our feedback model. The effectiveness of implemented feedback model was also confirmed since the student can continuously engage in both learning and physical activities, and take feedback regularly through the LA dashboards.
Several crucial challenges could be highlighted from this work regarding the application of MMD to support skill development in real learning environments. Firstly, technical challenges should be addressed in order to support a variety of daily life activities. It would be beneficial to create technical infrastructures to interconnect learning tools and physiological sensors [17]. High diversity of MMD in multi-context can rich the context of skill practice. Secondly, it is a quite challenging task in humancomputer interactions to create LA interfaces that could be understood and used to nudge learners to take action [18]. It is becoming more critical to simplify complex MMD during the feedback design in K-12 settings because K-12 students have relatively low data literacy and learning skill level. Thirdly, more significant challenges need to be addressed for privacy and ethics concerns when integrating MMD and LA technologies [19, 23]. There are other rich sensor data available (i.e., eye-tracking, electroencephalography, video) in students’ daily life; however, the privacy and ethical issues make it difficult to embed these sensors into K-12 classrooms, comparing to the laboratory settings.
This work could be extended to two research directions in the future. One is to investigate the transferability of SDS in learning, physical activity, and other daily activity contexts. Another is to improve the feedback model across activity contexts from a learner’s lifestyle perspective by integrating open learner model and multimodal fusion techniques.
This study is supported by NEDO Special Innovation Program on AI and Big Data JPNP18013,
JSPS KAKENHI Grant-in-Aid for Early-Career Scientists 20K20131, JSPS KAKENHI Grant-in-Aid
for Scientific Research (B) 22H03902.
[7] Boyer, S. L., Edmondson, D. R., Artis, A. B., & Fleming, D. (2014). Self-directed learning: A tool
for lifelong learning. Journal of Marketing Education, 36(
Journal of Learning Analytics, 3(
British Journal of Educational Technology, 50 (