Capturing Self-Regulated Learning During Search

Abstract

Researchers in the learning sciences have demonstrated the benefits of efective self-regulated learning (SRL) in improving learning outcomes. The search-as-learning community aims to improve learning outcomes during search, but ofers limited research exploring the impact of SRL on learning during search. Current limited research in search-as-learning explores only perceptions of SRL processes after the search process [1]. Results from such analyses are limited in that SRL is a dynamic, active process and participant perceptions of SRL can be unreliable [2, 3]. In this paper, we propose the implementation of an SRL coding framework to capture SRL processes as they unfold throughout a search session. Additionally, we ofer several implications for future work using the proposed methodology.

Keywords

• eol>search-as-learning
• self-regulated learning
• qualitative coding

1. Introduction

The search-as-learning community was established to address the limitations of current search systems in supporting learning during search [ 4, 5 ]. Prior search-as-learning work has focused on several main factors that afect learning during search: (1) the user [ 6, 7, 8, 9 ]; (2) the task [ 10, 11, 12 ]; or (3) the system [ 13, 14, 15, 16 ]. Less work has focused on better understanding the learning process during search [ 12, 8, 17 ]. More exploration is necessary to uncover when, where, why, and how learning occurs during search. Critical to this understanding is the process of self-regulated learning (SRL) during search.

SRL is an active, reflective process in which a learner monitors and controls their own learning to achieve their learning objectives [ 18, 19, 20 ]. For decades, researchers in the learning sciences have shown that efective SRL improves learning outcomes [ 21, 22, 23, 24, 19, 25, 26, 27, 28 ]. However, little work has considered the role of SRL in learning during search [ 1, 29 ]. Such studies have used questionnaire data to explore perceptions of SRL processes after a search session, but arguably no work has explicitly explored SRL processes during search.

Learning sciences research has shown the limitations of particular methodologies for capturing SRL processes [ 30 ]. SRL is an active, dynamic process that occurs over time. Questionnaire data captures SRL perceptions after the learning session, making the methodology less-suited to capturing the changing, evolving process of SRL. Think-aloud protocols, on the other hand, code learning comments and aim to capture specific SRL processes as they unfold across a learning session. In this paper, we propose and apply an SRL coding framework adapted from Greene et al. [ 3 ] to an example search-as-learning scenario. Additionally, we discuss how understanding SRL processes during search has several implications for future search-as-learning research.

2. Motivation

Prior search-as-learning research has considered how factors afect learning during search. The majority of this work has investigated the impact of three main factors on learning during search—(1) the user [ 6, 7, 8, 9 ]; (2) the task [ 10, 11, 12 ]; or (3) the system [ 13, 14, 15, 16 ]. Fewer studies have considered the learning process during search. Of those that have, Liu et al. [ 12 ] investigated knowledge shift patterns during the search process. To capture knowledge shifts, Liu et al. used mind maps. Participants were ask to create a mind map before the task to capture prior knowledge. Participants then modified these initial mind maps throughout the search process to capture how learning evolved. During analysis, the authors categorized diferent types of changes participants made to their mind maps (e.g., adding, modifying, or deleting nodes). The authors also categorized the location of the change within the mind map (e.g., level 1-2, level 3, or higher level changes). These categories were analyzed to better understand common and uncommon changes to mind maps across participant learning processes (e.g., adding nodes was more common than structural changes to mind maps). Additionally, the changes and locations of changes to mind maps were used to group search sessions (e.g., those learning processes with frequent changes early vs. late in the search session).

Roy et al. [ 8 ] also investigated the learning process, exploring the impact of domain knowledge on learning across the search session. While searching, participants were intermittently presented with vocabulary learning assessments to measure changes in learning. The authors found that prior knowledge impacted when participants had the highest knowledge gains. Participants with less prior knowledge had greater gains at the beginning of the search session and those with more prior knowledge had greater gains toward the end of the search session.

Urgo & Arguello [ 17 ] explored how a searcher’s learning objective may impact the learning process during a search session. To manipulate learning objectives, the authors leveraged the Anderson & Krathwohl (A&K) taxonomy [ 31 ]. Specifically, learning objectives were situated at the intersection of a specific cognitive process (apply, evaluate, create) and knowledge type (factual, conceptual, procedural). Similarly, the A&K taxonomy was used to analyze the learning pathways followed by study participants toward a given objective. Pathways were defined as sequences of learning instances that were also assigned to a cell from the A&K taxonomy. Results found several important trends. First, procedural knowledge objectives had longer pathways, mostly due to participants iterating on create-level processes (e.g., iteratively modifying a procedure based on preferences or constraints). Second, irrespective of the objective, participants tended to iterate more on simple processes (e.g., remember, understand) than complex processes (e.g., analyze, evaluate). Finally, the authors explored common and uncommon cognitive process transitions conditioned on the objective. For example, conceptual objectives had fewer transitions from analyze to evaluate.

Although a small number of studies have investigated the learning process during search, there are still large gaps in our understanding of when, where, how, and why learning occurs during search. Additionally, very limited research has investigated SRL in search-as-learning [ 1, 29 ]. Importantly, these studies have exclusively used questionnaire data to examine participants’ post-task perceptions of engagement in SRL processes during the search session. In this paper, we argue that search-as-learning research should investigate SRL processes as they unfold across the search session, to better understand the role of SRL on learning during search. We describe an existing SRL coding framework [ 3 ] that is well-suited for this purpose. Additionally, we describe how think-aloud data (in conjunction with recorded search activities) can be used to detect and characterize SRL processes during search.

3. SRL Models

Self-regulated learning (SRL) is an active and reflective process that involves a learner monitoring and controlling their learning to achieve specific learning goals [ 18, 19, 20 ]. Research in the learning sciences has underscored the important role of efective SRL in improving learning outcomes [ 21, 22, 23, 24, 19, 25, 26, 27, 28 ]. From prior work, several models of SRL have emerged [ 32, 33, 34, 35, 21 ]. These models originate from various fields (e.g., social foundations of cognition and behavior [ 36 ]), theories (e.g., Action Control Theory [ 37 ]), and/or motivating factors (e.g., learner motivation). We propose the Winne & Hadwin (W&H) model [ 23 ] because it is supported by evidence from much prior work [ 38, 39, 40, 41, 42, 43, 44 ] and it emphasizes metacognitive knowledge (i.e., a learner’s knowledge of their own learning and general knowledge of learning strategies) and metacognitive skills. Metacognitive skills include monitoring and control and are integral to the W&H model discussed next.

The W&H model of SRL consists of four phases—(1) task definition; (2) planning and goalsetting; (3) studying tactics; and (4) adaptation. In the task denfiition phase, a learner generates an understanding of the requirements of the task. In the planning and goal-setting phase, a learner sets goals to monitor progress. In the studying tactics phase, a learner uses strategies (e.g., summarizing, note-taking, selecting sources) to accomplish their goals. Finally, in the adaptation phase, the learner reflects on their choices, progress, successes, and failures to make decisions about what to do next. Conditions (e.g., motivation, task understanding, time, resources), operations (e.g., note-taking, summarizing), and standards (e.g., criteria learner deems important to achieve task) are important components throughout the W&H model. During each phase of the model, a learner uses conditions to make decisions about operations and standards. Metacognitive monitoring and control are the “pivots upon which each of the four phases turn.” [45, p. 469] Metacognitive monitoring is the learner’s process of using standards to judge what has been learned and produced in order to assess progress toward their learning goals. Metacognitive control is the implementation of strategies based on feedback from monitoring. For example, a learner may read through a section of text and, after monitoring, realize they are not understanding anything. In response, the learner enacts control by selecting a new informational source that may be better suited to their level of understanding. In the next section, we discuss how the W&H model has been operationalized to capture SRL in prior work outside of search-as-learning.

4. Capturing SRL Outside of Search-as-Learning

SRL processes can be dificult to capture because they are dynamic and adaptive [ 46 ]. To address this challenge, researchers in the learning sciences have developed two primary means of capturing SRL, using—(1) questionnaires [ 47, 48, 49 ]; or (2) coded think-aloud data [ 50, 51, 3 ]. While static questionnaire data captures perceptions of SRL after the learning session, coded think-aloud data is more suited to capturing the dynamic, adaptive process of SRL by coding processes as they occur across the learning session. Greene et al. [ 30 ] have proposed a “right tool for the job” approach to collecting SRL data. Greene et al. assert that while “motivational and dispositional aspects of SRL may be best captured by self-report data [...] more transient, dynamic task-specific aspects may be best captured by TAPs [think-aloud protocols].” [ 30, p. 323] We argue that in search-as-learning research it is precisely the transient, dynamic, taskspecific aspects that are unknown and crucial to understanding how best to support learning during search. For this purpose, we propose the Greene et al. framework [ 3 ] to understand SRL processes during search.

5. Applying SRL Coding Framework to Search-as-Learning

In response to the gaps left behind by questionnaire data alone, SRL coding frameworks have been developed to analyze think-aloud data exhibited by users of computer-based learning enivironments [ 52, 53 ]. Greene et al. developed an SRL coding framework [ 3 ] (adapted from [ 52, 53 ]) that is rooted in the W&H model of SRL. The framework breaks SRL processes into ifve macro-SRL processes—(1) planning; (2) monitoring; (3) strategy use; (4) task dificulty & demands; and (5) interest. Each macro-SRL process is associated with one or more components of the W&H model and contains micro-SRL processes that can be coded using think-aloud comments. In this section, we review the macro-SRL processes and the micro-SRL processes contained within each. Many of the descriptions are pulled directly from Greene et al., while some have been slightly modified. Extending the application proposed by Greene et al., we also provide novel example think-aloud comments and search activities that may be indicative of each micro-SRL process. The example, fictional think-aloud comments and search activities are inspired by actual participant comments and behaviors from a learning-oriented search task operationalized in a search-as-learning study: “Determine, which best explains the notion of lift and why: Bernoulli’s principle or Newton’s laws of motion?” [ 17 ] In the study, participants were provided with a search system and word document to take notes (as is common to many search-as-learning studies).

Macro-SRL: Planning The macro-SRL process of planning is associated with the second phase of the W&H model: planning and goal-setting. Shown in Table 1, there are four micro-SRL processes associated with planning. While the examples provided are all think-aloud comments, there may be search activities that are also indicative of these micro-SRL processes. For example, the subgoals process may be engaged when a searcher adds an additional heading to their notes or queries “definition of Bernoulli’s principle”.

Macro-SRL: Strategy Use The macro-SRL process strategy use is associated with the third phase of the W&H model studying tactics. Shown in Table 2, there are nineteen micro-SRL processes associated with strategy use. Many of the micro-SRL processes associated with strategy use quite naturally fit with typical search-as-learning participant behaviors (e.g., summarization, taking notes, select new informational source, manipulate representation).

We omitted one strategy use micro-SRL process from the original Greene et al. framework: search. Search has been excluded as it arguably encapsulates an entire search-as-learning session and is therefore not relevant to this field as a specific micro-SRL process. Historical Perspective Taking Hypothesizing Inferences Inferring Source Content Knowledge elaboration Manipulate representation Memorization Prior Knowledge Activation Reading notes Re-reading Self-knowledge activation Select new informational source Summarization Taking notes

Participant puts self in position of a historical figure; infers that figure’s perspective, thinking, emotions; expresses understanding of that figure’s decision making at that time.

Making a tentative conclusion or informed guess (about content relevant to the task) based upon information either in the environment or from prior knowledge.

Drawing a conclusion based on two or more pieces of information that were read, seen, or heard in the search session.

Participant makes a guess as to the content available in a source.

Making a definitive conclusion by elaborating on what was just read, seen, or hear with prior knowledge.

Using pause, start, rewind, zoom, or other controls with a graphical representation.

Learner tries to memorize text, diagram, etc.

Learner searches memory for relevant prior knowledge either before beginning performance of a task or during task performance.

Learner reads over notes, drawings, etc.

Re-reading or revisiting a section of the search environment.

The participant verbalizes that they are going to invoke a strategy because it is personally helpful, or that they are NOT going to invoke a strategy because it is NOT helpful to them, or, they say something about their own knowledge, beliefs, disposition, etc.

Using the search environment to access a new representation of the desired information (e.g., navigating to new webpage).

Verbally restating or writing what was just read, inspected, or heard in the search session.

Learner writes down information. “Based on the timeline of when these were developed, I would guess that Bernoulli wasn’t trying to apply this idea specifically to lift.” “I think the paper will move upward when he blows across it because the air will be moving faster on the top rather than the bottom of the paper.” “Ok, so if pressure is important to lift in the wing example and the paper example, then I think Bernoulli’s principle is important to lift.” “The snippet [on the SERP] mentions energy in a system, so I think this site should be about conservation of energy.” [Viewing diagram of a wing indicating pressure, velocity, and lift] “Ok, so if this diagram were showing forces from Newton’s third law, then it would show the flow of the air here and downwash here from conservation of momentum.” [Rewinds YouTube video of Bernoulli’s principle explanation] “Once again without looking, Bernoulli’s principle is the inverse relationship between velocity and pressure of a fluid.” “Oh yes! I do remember that Bernoulli’s principle has something to do with pressure. . . ” “I’m going to read over my notes.” [Reads through notes] “I’m going to read through this section again.” “I’m going to summarize what I just read in my notes because that will help me remember it better.” [Returns to SERP and clicks on a diferent result] [Writes summary in text editor] [Writes notes in text editor]

Macro-SRL: Monitoring The macro-SRL process monitoring is associated with the central component metacognitive monitoring that functions throughout the W&H model. Shown in Table 3, there are twelve micro-SRL processes associated with monitoring. While many of these processes are quite intuitive (e.g., monitoring progress toward subgoals, time monitoring), there are two important concepts related to monitoring that may be less familiar to search-as-learning researchers, feeling of knowing and judgment of learning. Feeling of knowing (FOK) involves a learner reflecting on whether or not they are familiar with a piece of information. [ 52, 53 ]. Judgment of learning (JOL) involves a learner reflecting on whether or not they currently understand a piece of information [ 54, 55 ]. There are two micro-SRL processes associated with JOL: JOL and JOLT. JOL involves a learner expressing that they explicitly do or do not understand something. JOLT, on the other hand, involves a learner expressing that they have some understanding that may not be fully accurate. They will then continue learning with their current understanding until something proves or disproves that their understanding is correct.

Macro-SRL: Task Dificulty & Demands The macro-SRL process task dificulty & demands is associated with the conditions component of the W&H model. Shown in Table 4, there are three micro-SRL processes associated with task dificulty & demands . Such processes ofer a more nuanced look at task dificulty than questionnaire methods typically used in search-aslearning [ 56 ].

Macro-SRL: Interest The macro-SRL process task interest is associated with the conditions component of the W&H model. Task interest most closely aligns with a subset of conditions called cognitive conditions. Cognitive conditions include variables such as prior knowledge, motivation, and task understanding. Shown in Table 5, there is one micro-SRL process, interest statement. Similar to task dificulty & demands , documenting interest statements may ofer a more nuanced look at task interest than questionnaire methods alone [ 56 ].

6. Implications

While efective SRL has been shown to improve learning outcomes, little research in searchas-learning has explored where, when, why, and how frequently SRL processes occur while learning during search. The proposed coding framework adapted from Greene et al. afords a nuanced look into SRL processes during search-as-learning studies. We ofer five major implications for future search-as-learning work given the proposed methodology.

First, capturing SRL during the search process (versus post-search perceptions with questionnaire data) allows researchers to explore the efects of observed SRL on learning outcomes. Prior search-as-learning research has not investigated the efects of SRL processes on learning outcomes. Regardless of particular micro- or macro-SRL processes, SRL-coded think-aloud and search activity data may give researchers the opportunity to better understand the overall efect of SRL on learning.

Second, capturing SRL processes during search may enable search-as-learning researchers to better understand the impact of specific SRL processes on learning. By coding think-aloud comments and search activities into micro- and macro-SRL processes, researchers can calculate the frequency of each process. This information can then be analyzed to better understand the relationship between particular SRL processes and learning outcomes.

Third, a major goal of search-as-learning is to develop tools that better support learning during search. Efective SRL is critical for learning. Therefore, future work should develop tools to directly support SRL processes. We propose four types of tools to encourage and support distinct SRL processes during search. One, future work should consider tools that allow searchers to develop subgoals, take notes with respect to subgoals, and mark subgoals as completed. Such tools can encourage and support planning and monitoring. Two, future work should consider note-taking tools with diferent types of structures to organize information. Diferent structures might be able to support diferent micro-SRL processes within strategy use. For example, tables might support comparing and contrasting, lists might support establishing chronology, concept maps might support drawing inferences, and diagram capabilities might support drawing. Three, future work should consider tools that prompt self-reflection in contextually relevant ways. Such tools might encourage and support both strategy use and monitoring. For example, tools might prompt searchers with various questions: “How does this relate to the article you just read?”; “What do you already know about this topic?”; and “What remaining questions do you still have about this topic?” Finally, future work should consider tools that search for alternate representations of a given piece of information to support task dificulty & demands . For example, such a tool might enable a searcher to highlight a passage of text and search for non-textual representations of the content, such as images, videos, and tables.

Fourth, future search-as-learning research should investigate the relationship between particular SRL processes and search activities that can be logged by a system. In this paper, we ofer several potential examples (e.g., a new header in a note-taking tool may suggest planning). Better understanding the types of search activities that are associated with particular SRL processes would allow future work to potentially predict the occurrence of SRL simply using search interaction data.

Finally, researchers in the learning sciences assert that the relative importance of particular SRL processes vary based on academic domain (e.g., science versus history) and context (e.g., physical space, time constraints, available resources) [ 3 ]. For this reason, future searchas-learning research should investigate whether and how the importance of particular SRL processes vary based on the task domain (e.g., biology, statistics); specific task constraints (e.g., task importance, timeframe); or the learning objective (e.g., cognitive process, knowledge type). For example, undergraduate biology students searching to learn about osmosis and difusion may need support for diferent SRL processes than professional data analysts searching to learn about a new statistical method.

7. Conclusion

For decades, researchers in the learning sciences have observed the positive impacts of efective SRL on learning outcomes. Additionally, researchers have developed methods for capturing SRL within computer-based learning environments. Leveraging these existing coding frameworks is important to advancing future search-as-learning research. While prior work has considered the extent to which searchers perceive engagement in SRL processes, it is critical to explore when, where, why, and how frequently SRL processes actually occur during a search session. Doing so will allow researchers to explore—(1) the overall efect of SRL on learning during search; (2) the impact of particular SRL processes on learning during search; (3) tools that support important SRL processes during search; (4) the relationship between particular SRL processes and search activities; and (5) factors that may impact which SRL processes are most critical to supporting learning during search.

References

[1] A. Crescenzi , A. R. Ward , Y. Li , R. Capra , Supporting Metacognition during Exploratory Search with the OrgBox , in: Proceedings of the 44th International ACM SIGIR Conference on Research and Development in Information Retrieval , Association for Computing Machinery, New York, NY, USA, 2021 , pp. 1197 - 1207 . URL: http://doi.org/10.1145/3404835.3462955.

[2] P. H. Winne , D. Jamieson-Noel, Exploring students' calibration of self reports about study tactics and achievement , Contemporary Educational Psychology 27 ( 2002 ) 551 - 572 . URL: https://www.sciencedirect.com/science/article/pii/S0361476X02000061. doi: 10 . 1016/ S0361 -476X( 02 ) 00006 - 1 .

[3] J. A. Greene , C. M. Bolick , W. P. Jackson , A. M. Caprino , C. Oswald , M. McVea, Domainspecificity of self-regulated learning processing in science and history , Contemporary Educational Psychology 42 ( 2015 ) 111 - 128 . URL: https://linkinghub.elsevier.com/retrieve/ pii/S0361476X15000260. doi: 10 .1016/j.cedpsych. 2015 . 06 .001.

[4] J. Allan , B. Croft , A. Mofat , M. Sanderson , Frontiers, challenges, and opportunities for information retrieval: Report from SWIRL 2012 the second strategic workshop on information retrieval in Lorne, ACM SIGIR Forum 46 ( 2012 ) 2 . URL: http://dl.acm.org/ citation.cfm?doid= 2215676 .2215678. doi: 10 .1145/2215676.2215678.

[5] K. Collins-Thompson , P. Hansen , C. Hauf , Search as Learning ( Dagstuhl Seminar 17092) ( 2017 ). doi:10.4230/dagrep.7.2 .135.

[6] T. Willoughby , S. A. Anderson , E. Wood , J. Mueller , C. Ross , Fast searching for information on the Internet to use in a learning context: The impact of domain knowledge , Computers & Education 52 ( 2009 ) 640 - 648 . URL: https://www.sciencedirect.com/science/article/pii/ S0360131508001802. doi: 10 .1016/j.compedu. 2008 . 11 .009.

[7] H. L. O'Brien , A. Kampen , A. W. Cole , K. Brennan , The Role of Domain Knowledge in Search as Learning , in: Proceedings of the 2020 Conference on Human Information Interaction and Retrieval , CHIIR '20, Association for Computing Machinery, Vancouver BC , Canada, 2020 , pp. 313 - 317 . URL: http://doi.org/10.1145/3343413.3377989. doi: 10 .1145/ 3343413.3377989.

[8] N. Roy , F. Moraes , C. Hauf , Exploring Users' Learning Gains within Search Sessions , in : Proceedings of the 2020 Conference on Human Information Interaction and Retrieval , CHIIR '20, Association for Computing Machinery, Vancouver BC , Canada, 2020 , pp. 432 - 436 . URL: http://doi.org/10.1145/3343413.3378012. doi: 10 .1145/3343413.3378012.

[9] G. Pardi , J. von Hoyer, P. Holtz , Y. Kammerer , The Role of Cognitive Abilities and Time Spent on Texts and Videos in a Multimodal Searching as Learning Task , in: Proceedings of the 2020 Conference on Human Information Interaction and Retrieval , ACM , Vancouver BC Canada, 2020 , pp. 378 - 382 . URL: https://dl.acm.org/doi/10.1145/3343413.3378001. doi: 10 . 1145/3343413.3378001.

[10] S. Ghosh , M. Rath , C. Shah , Searching As Learning: Exploring Search Behavior and Learning Outcomes in Learning-related Tasks , in: Proceedings of the 2018 Conference on Human Information Interaction & Retrieval , CHIIR '18, ACM , New York, NY, USA, 2018 , pp. 22 - 31 . URL: http://doi.acm. org/10 .1145/3176349.3176386. doi: 10 .1145/3176349.3176386.

[11] R. Kalyani , U. Gadiraju, Understanding User Search Behavior Across Varying Cognitive Levels , in: Proceedings of the 30th ACM Conference on Hypertext and Social Media , HT '19, Association for Computing Machinery, New York, NY, USA, 2019 , pp. 123 - 132 . URL: http://doi.org/10.1145/3342220.3343643. doi: 10 .1145/3342220.3343643.

[12] H. Liu , C. Liu, N. J. Belkin , Investigation of users' knowledge change process in learningrelated search tasks , Proceedings of the Association for Information Science and Technology 56 ( 2019 ) 166 - 175 . doi:https://doi.org/10.1002/pra2. 63 .

[13] Y. Kammerer , R. Nairn , P. Pirolli , E. H. Chi , Signpost from the masses: learning efects in an exploratory social tag search browser , in: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, CHI '09 , Association for Computing Machinery, Boston, MA, USA, 2009 , pp. 625 - 634 . URL: http://doi.org/10.1145/1518701.1518797. doi: 10 . 1145/1518701.1518797.

[14] L. Freund , R. Kopak , H. O'Brien, The efects of textual environment on reading comprehension: Implications for searching as learning , Journal of Information Science 42 ( 2016 ) 79 - 93 . URL: https://doi.org/10.1177/0165551515614472. doi: 10 .1177/0165551515614472, publisher: SAGE Publications Ltd.

[15] N. Roy , M. V. Torre , U. Gadiraju , D. Maxwell , C. Hauf, Note the Highlight: Incorporating Active Reading Tools in a Search as Learning Environment , in: Proceedings of the 2021 Conference on Human Information Interaction and Retrieval , CHIIR '21, Association for Computing Machinery, New York, NY, USA, 2021 , pp. 229 - 238 . URL: http://doi.org/10. 1145/3406522.3446025. doi: 10 .1145/3406522.3446025.

[16] A. Câmara , N. Roy , D. Maxwell , C. Hauf , Searching to Learn with Instructional Scafolding , in: Proceedings of the 2021 Conference on Human Information Interaction and Retrieval , CHIIR '21, Association for Computing Machinery, New York, NY, USA, 2021 , pp. 209 - 218 . URL: http://doi.org/10.1145/3406522.3446012. doi: 10 .1145/3406522.3446012.

[17] K. Urgo , J. Arguello , Understanding the "Pathway" Towards a Searcher's Learning Objective , ACM Transactions on Information Systems 40 ( 2022 ) 77 : 1 - 77 : 42 . URL: http://doi.org/10. 1145/3495222. doi: 10 .1145/3495222.

[18] P. H. Winne , Self-regulated learning viewed from models of information processing, in: Self-regulated learning and academic achievement: Theoretical perspectives , 2nd ed, Lawrence Erlbaum Associates Publishers, Mahwah, NJ, US, 2001 , pp. 153 - 189 .

[19] B. J. Zimmerman , D. H. Schunk , Handbook of self-regulation of learning and performance, Handbook of self-regulation of learning and performance , Routledge/Taylor & Francis Group, New York, NY, US, 2011 . Pages: xiv, 484 .

[20] D. H. Schunk , Self-Regulation Through Goal Setting, ERIC Digest ( 2001 ) 2 .

[21] M. Boekaerts , M. Zeidner , P. R. Pintrich , P. R. Pintrich , Handbook of Self-Regulation, Elsevier Science & Technology, San Diego, UNITED STATES, 1999 . URL: http://ebookcentral. proquest.com/lib/unc/detail.action?docID= 300645 .

[22] T. Sitzmann , K. Ely , A meta-analysis of self-regulated learning in work-related training and educational attainment: What we know and where we need to go , Psychological Bulletin 137 ( 2011 ) 421 - 442 . doi: 10 .1037/a0022777, place: US Publisher: American Psychological Association.

[23] P. H. Winne , A. F. Hadwin , Studying as self-regulated engagement in learning , in: Metacognition in educational theory and practice , 1998 .

[24] D. H. Schunk , C. W. Swartz , Goals and Progress Feedback: Efects on Self-Eficacy and Writing Achievement , Contemporary Educational Psychology 18 ( 1993 ) 337 - 354 . URL: https://www.sciencedirect.com/science/article/pii/S0361476X83710246. doi: 10 . 1006/ceps. 1993 . 1024 .

[25] B. J. Zimmerman , M. M. Pons , Development of a Structured Interview for Assessing Student Use of Self-Regulated Learning Strategies , American Educational Research Journal 23 ( 1986 ) 614 - 628 . URL: https://doi.org/10.3102/00028312023004614. doi: 10 .3102/ 00028312023004614, publisher: American Educational Research Association.

[26] B. J. Zimmerman , M. Martinez-Pons , Construct validation of a strategy model of student self-regulated learning , Journal of Educational Psychology 80 ( 1988 ) 284 - 290 . doi: 10 . 1037/ 0022 - 0663 . 80 .3.284, place: US Publisher: American Psychological Association.

[27] D. H. Schunk , Sequential attributional feedback and children's achievement behaviors , Journal of Educational Psychology 76 ( 1984 ) 1159 - 1169 . doi: 10 .1037/ 0022 - 0663 . 76 . 6.1159, place: US Publisher: American Psychological Association.

[28] D. H. Schunk , Modeling and attributional efects on children's achievement: A self-eficacy analysis , Journal of Educational Psychology 73 ( 1981 ) 93 - 105 . doi: 10 .1037/ 0022 - 0663 . 73 .1.93, place: US Publisher: American Psychological Association.

[29] J. F. von Hoyer , J. Kimmerle , P. Holtz , Acquisition of false certainty: Learners increase their confidence in the correctness of incorrect answers after online information search , Journal of Computer Assisted Learning 38 ( 2022 ) 833 - 844 . doi: 10 .1111/jcal.12657.

[30] J. A. Greene , V. M. Deekens , D. Z. Copeland , S. Yu , Capturing and modeling self-regulated learning using think-aloud protocols, in: Handbook of self-regulation of learning and performance , 2nd ed, Educational psychology handbook series, Routledge/Taylor & Francis Group, New York, NY, US, 2018 , pp. 323 - 337 .

[31] L. W. Anderson , D. R. Krathwohl , P. W. Airasian , K. A. Cruikshank , R. E. Mayer , P. R. Pintrich , J. Raths , M. C. Wittrock , A Taxonomy for Learning, Teaching, and Assessing: A Revision of Bloom's Taxonomy of Educational Objectives , Abridged Edition , 1 edition ed., Pearson , New York, 2000 .

[32] P. H. Winne , N. E. Perry , Measuring self-regulated learning , in: Handbook of self-regulation, Academic Press, San Diego, CA, US, 2000 , pp. 531 - 566 . doi: 10 .1016/ B978-012109890-2/ 50045 -7.

[33] P. R. Pintrich , Chapter 14 - The Role of Goal Orientation in Self-Regulated Learning , in: M. Boekaerts , P. R. Pintrich , M. Zeidner (Eds.), Handbook of Self-Regulation, Academic Press, San Diego, 2000 , pp. 451 - 502 . URL: http://www.sciencedirect.com/science/article/ pii/B9780121098902500433. doi: 10 .1016/B978-012109890-2/ 50043 -3.

[34] B. J. Zimmerman , Chapter 2 - Attaining Self-Regulation : A Social Cognitive Perspective , in: M. Boekaerts , P. R. Pintrich , M. Zeidner (Eds.), Handbook of Self-Regulation, Academic Press, San Diego, 2000 , pp. 13 - 39 . URL: http://www.sciencedirect.com/science/article/pii/ B9780121098902500317. doi: 10 .1016/B978-012109890-2/ 50031 -7.

[35] B. J. Zimmerman , Becoming a Self-Regulated Learner: An Overview , Theory Into Practice 41 ( 2002 ) 64 - 70 . URL: http://www.tandfonline.com/doi/abs/10.1207/s15430421tip4102_2. doi: 10 .1207/s15430421tip4102_ 2 .

[36] A. Bandura , Social foundations of thought and action: A social cognitive theory, Social foundations of thought and action: A social cognitive theory , Prentice-Hall, Inc, Englewood Clifs, NJ, US, 1986 . Pages: xiii, 617 .

[37] J. Kuhl , From Cognition to Behavior: Perspectives for Future Research on Action Control, in: J. Kuhl , J. Beckmann (Eds.), Action Control: From Cognition to Behavior, SSSP Springer Series in Social Psychology, Springer, Berlin, Heidelberg, 1985 , pp. 267 - 275 . URL: https: //doi.org/10.1007/978-3- 642 -69746-3_ 12 . doi: 10 .1007/978-3- 642 -69746-3_ 12 .

[38] J. A. Greene , R. Azevedo , A Theoretical Review of Winne and Hadwin's Model of Self-Regulated Learning: New Perspectives and Directions , Review of Educational Research 77 ( 2007 ) 334 - 372 . URL: https://doi.org/10.3102/003465430303953. doi: 10 .3102/ 003465430303953, publisher: American Educational Research Association.

[39] G. van den Boom , F. Paas , J. J. G. van Merriënboer , Efects of elicited reflections combined with tutor or peer feedback on self-regulated learning and learning outcomes , Learning and Instruction 17 ( 2007 ) 532 - 548 . URL: http://www.sciencedirect.com/science/article/pii/ S0959475207000953. doi: 10 .1016/j.learninstruc. 2007 . 09 .003.

[40] I. Glogger , R. Schwonke , L. Holzäpfel , M. Nückles , A. Renkl , Learning strategies assessed by journal writing: Prediction of learning outcomes by quantity, quality, and combinations of learning strategies , Journal of Educational Psychology 104 ( 2012 ) 452 - 468 . doi: 10 . 1037/a0026683, place: US Publisher: American Psychological Association.

[41] R. Santhanam , S. Sasidharan , J. Webster , Using Self-Regulatory Learning to Enhance E-Learning- Based Information Technology Training, Information Systems Research 19 ( 2008 ) 26 - 47 . URL: https://pubsonline.informs.org/doi/abs/10.1287/isre.1070.0141. doi: 10 . 1287/isre.1070.0141, publisher: INFORMS.

[42] M. Bannert , M. Hildebrand , C. Mengelkamp , Efects of a metacognitive support device in learning environments , Computers in Human Behavior 25 ( 2009 ) 829 - 835 . URL: http://www.sciencedirect.com/science/article/pii/S0747563208001350. doi: 10 .1016/ j.chb. 2008 . 07 .002.

[43] S. Kistner , K. Rakoczy , B. Otto , C. Dignath-van Ewijk , G. Büttner , E. Klieme, Promotion of self-regulated learning in classrooms: investigating frequency, quality, and consequences for student performance, Metacognition and Learning 5 ( 2010 ) 157 - 171 . URL: https: //doi.org/10.1007/s11409-010-9055-3. doi: 10 .1007/s11409-010-9055-3.

[44] R. Azevedo , S. Ragan , J. G. Cromley , S. Pritchett , Do Diferent Goal-Setting Conditions Facilitate Students' Ability to Regulate Their Learning of Complex Science Topics with RiverWeb? , 2002 . URL: https://eric.ed.gov/?id= ED482509 .

[45] P. H. Winne , Students' calibration of knowledge and learning processes: Implications for designing powerful software learning environments , International Journal of Educational Research 41 ( 2004 ) 466 - 488 . URL: https://linkinghub.elsevier.com/retrieve/pii/ S0883035505000601. doi: 10 .1016/j.ijer. 2005 . 08 .012.

[46] J. A. Greene , K. R. Dellinger , B. B. Tüysüzoğlu , L. -J. Costa , A Two-Tiered Approach to Analyzing Self-Regulated Learning Data to Inform the Design of Hypermedia Learning Environments , in: R. Azevedo , V. Aleven (Eds.), International Handbook of Metacognition and Learning Technologies , Springer International Handbooks of Education, Springer, New York, NY, 2013 , pp. 117 - 128 . URL: https://doi.org/10.1007/978-1- 4419 -5546- 3 _8. doi: 10 . 1007/978-1- 4419 -5546- 3 _ 8 .

[47] P. R. Pintrich , E. V. de Groot, Motivational and self-regulated learning components of classroom academic performance , Journal of Educational Psychology 82 ( 1990 ) 33 - 40 . doi: 10 .1037/ 0022 - 0663 . 82 .1.33, place: US Publisher: American Psychological Association.

[48] G. Schraw , R. S. Dennison , Assessing Metacognitive Awareness, Contemporary Educational Psychology 19 ( 1994 ) 460 - 475 . URL: https://www.sciencedirect.com/science/article/pii/ S0361476X84710332. doi: 10 .1006/ceps. 1994 . 1033 .

[49] C. Weinstein , D. Palmer , A. Schulte , Learning and Study Strategies Inventory (LASSI), Clearwater , FL: H & H Publishing ( 1987 ).

[50] R. Azevedo , Using Hypermedia as a Metacognitive Tool for Enhancing Student Learning? The Role of Self-Regulated Learning , Educational Psychologist 40 ( 2005 ) 199 - 209 . doi: 10 . 1207/s15326985ep4004_ 2 .

[51] J. A. Greene , L. A. Hutchison , L. -J. Costa , H. Crompton , Investigating how college students' task definitions and plans relate to self-regulated learning processing and understanding of a complex science topic , Contemporary Educational Psychology 37 ( 2012 ) 307 - 320 . URL: http://www.sciencedirect.com/science/article/pii/S0361476X12000070. doi: 10 .1016/ j.cedpsych. 2012 . 02 .002.

[52] R. Azevedo , J. G. Cromley , Does Training on Self-Regulated Learning Facilitate Students' Learning With Hypermedia? , Journal of Educational Psychology 96 ( 2004 ) 523 - 535 . URL: https://auth.lib.unc.edu/ezproxy_auth.php?url=http://search.ebscohost. com/login.aspx ?direct=true&db=pdh&AN= 2004 -18154-011& site=ehost-live&scope=site . doi: 10 .1037/ 0022 - 0663 . 96 .3.523, publisher: American Psychological Association.

[53] R. Azevedo , J. T. Guthrie , D. Seibert , The Role of Self-Regulated Learning in Fostering Students' Conceptual Understanding of Complex Systems with Hypermedia , Journal of Educational Computing Research 30 ( 2004 ) 87 - 111 . URL: http://journals.sagepub.com/doi/ 10.2190/ DVWX-GM1T- 6THQ-5WC7. doi: 10 .2190/ DVWX-GM1T- 6THQ-5WC7.

[54] R. Ariel , J. D. Karpicke , A. E. Witherby , S. K. Tauber , Do Judgments of Learning Directly Enhance Learning of Educational Materials? , Educational Psychology Review 33 ( 2021 ) 693 - 712 . URL: https://link.springer. com/10.1007/s10648-020-09556-8 . doi: 10 .1007/ s10648-020-09556-8.

[55] J. Dunlosky , A. R. Lipko , Metacomprehension: A Brief History and How to Improve Its Accuracy, Current Directions in Psychological Science 16 ( 2007 ) 228 - 232 . URL: https:// doi.org/10.1111/j.1467- 8721 . 2007 . 00509 .x. doi: 10 .1111/j.1467- 8721 . 2007 . 00509 .x, publisher: SAGE Publications Inc.

[56] K. Urgo , J. Arguello , R. Capra , The Efects of Learning Objectives on Searchers' Perceptions and Behaviors , in: Proceedings of the 2020 ACM SIGIR on International Conference on Theory of Information Retrieval , ACM , Virtual Event Norway, 2020 , pp. 77 - 84 . URL: https://dl.acm.org/doi/10.1145/3409256.3409815. doi: 10 .1145/3409256.3409815.