=Paper=
{{Paper
|id=Vol-2699/paper17
|storemode=property
|title=The Other Side of the Same Coin: From Learning-centric Search Systems to Search-centric Learning Systems
|pdfUrl=https://ceur-ws.org/Vol-2699/paper17.pdf
|volume=Vol-2699
|authors=Catherine L. Smith,Soo Young Rieh
|dblpUrl=https://dblp.org/rec/conf/cikm/SmithR20
}}
==The Other Side of the Same Coin: From Learning-centric Search Systems to Search-centric Learning Systems==
https://ceur-ws.org/Vol-2699/paper17.pdf
The other side of the same coin: From learning-centric search
systems to search-centric learning systems
Catherine L. Smitha and Soo Young Riehb
a
Kent State University, Kent, Ohio, USA
b
University of Texas at Austin, Austin, Texas, USA
Abstract
This short paper proposes a framework for designing search-centric learning systems that support
search as learning. Our argument draws on Jackson’s purpose-centric design concepts for
software, and from research on self-regulated learning, an established paradigm that intersects
psychology, education, and learning sciences. In introducing these ideas we also examine
searching for information as self-regulating activity and the design of experimental learning
systems that support self-regulation. We argue that embedding search functionality within
learning systems holds promise for better supporting students engaged in self-regulated learning.
Keywords 1
searching as learning; self-regulated learning; software design; metacognition
1. Introduction how those processes affect learning in an
academic setting [8, 21, 25]. This is a decidedly
Smith and Rieh [18] presented design goals design-centric research orientation and we
focused on information literate action and the acknowledge that goals such as basic research
need for learning-centric search systems are also essential.
designed for supporting metacognitive This paper is organized as follows. The first
engagement. One of the key ideas of learning- three sections present ideas and selected work
centric search systems was to better facilitate from purpose-centric design, self-regulated
active engagement with information that would learning, and learning system design. Next we
result in long-term learning and creative briefly examine results showing that searching
endeavor. In this paper, we flip that design goal for information is a process integral to SRL. We
over and focus on self-regulated learning to then present an example of a search-centric
argue for the design of a search-centric learning system, define the construct more
learning system. Such a system would embed broadly, and discuss a short scenario
search functions within a learning system. explicating the need for search concepts that
Our argument draws on two constructs. First better meet purposes for searching during SRL.
is Jackson’s conceptual design paradigm for The paper concludes with a brief summary. The
software [11], which focuses on alignment paper contributes a framework for considering
between a users’ purpose and functional design goals for learning systems that support
concepts within a software application. More search as learning.
specifically, our goal is to focus on the users’
purposes for information search during self- 2. Purpose-centric design.
regulated learning (SRL) [16]. SRL is a
psychological construct focused on cognitive, Jackson [11] proposed that good software
metacognitive and emotional processes design aligns user-centric purposes with
students use when engaged in learning, and on software concepts. Within this paradigm, “a
Proceedings of the CIKM 2020 Workshops, October 19-20,
Galway, Ireland
EMAIL: csmit141@kent.edu (A); rieh@ischool.utexas.edu (B)
ORCID: 0000-0002-8433-0829 (A); 0000-0002-2978-692X (B)
©️ 2020 Copyright for this paper by its authors. Use permitted under Creative
Commons License Attribution 4.0 International (CC BY 4.0).
CEUR Workshop Proceedings (CEUR-WS.org)
concept is a self-contained, reusable, increment full perspective on design includes a rich set of
of functionality that is motivated by a purpose ideas that we do not cover here, however, the
defined in terms of the needs of an end user” purpose/concept heuristics serves as a useful
[17]. Within a design, concepts exist at all framework for considering design goals for a
levels of granularity and are independent of search-centric learning system. We return to
their instantiations in code. For example, purpose-centric concepts later in the paper.
Twitter’s purpose is viral public expression.
Twitter serves its purpose with three concepts: 3. Self-regulated learning
tweet, hashtag and following, each concept with
a single purpose. The purpose of a tweet is short Hypothesizing a search-centric learning
public posting (a variant of the concept system provides an opportunity to focus on the
posting). The purpose of a hashtag is to purpose for information search within the
establish associations between tweets (a variant context of a system designed for learning. The
of the concept label). The purpose of following construct of SRL is particularly compelling as a
is to receive messages from a specific account. framework because it is domain-independent
All three concepts may be used for similar and centers on the general behaviors and mental
purposes in other applications or they may be processes students use when engaged in
instantiated in concept variants with similar effective learning. Also, its theories are
functionality (as tweet is for posting and embedded in much recent work on learning
hashtag is for label). Further, each of these system design and related analytics [21].
concepts comprises sub-concepts, ideally, each SRL has been defined as “self-directive
with its own single purpose. processes and self-beliefs that enable learners
The purpose of a search application is to find to transform their mental abilities, such as
information. Search applications use two verbal aptitude, into an academic performance
concepts: query and results. The purpose of a skill, such as writing.” [25]. At its most basic,
query is to express an information need. The SRL posits the recursive use of cognitive and
purpose of results is to expose the information metacognitive skills in three phases during
sources most likely to meet the need. These task-focused learning: forethought,
purposes apply in many contexts thus these performance, and assessment. Each of these
concepts have many applications. Examples of may be variously named or decomposed, but
sub-concepts for query include suggestion, there is consensus on a minimal three [16].
completion, and structure. The purpose of Experimental research often focuses on subsets
suggestion is to clarify the need by helping of specific skills within each phase.
users reformulate queries. The purpose of Forethought generally encompasses
completion is to minimize typing and typing interpreting, understanding, strategizing, and
errors. The purpose of structure is to improve planning a learning task. Performance focuses
the precision of results. The concept of structure on monitoring and control of plans and
includes sub-concepts such as filter and logic. strategies while learning. Assessment includes
Good software uses concepts that each serve using performance feedback, reacting,
a single purpose, where the purpose is defined adapting, and reflecting on cognition. Theories
well enough to motivate one and only one differ on the roles, types, and importance of
concept. Unmotivated concepts serve no motivation, skill, context, individual
purpose and are of no intrinsic value to users; differences, and prior knowledge that affect
typically these involve patching over a design transitions between phases. Increased use of
flaw or simply superfluous functionality. When SRL skill reliably enhances learning outcomes
software contains redundant concepts that [26], thus much work has been done on the
fulfill the same purpose the application is design of instructional methods that enhance
confusing, hard to learn, and inefficient for self-regulation [16].
users. Problems also arise when a concept SRL has a large, rich, and growing literature
serves more than one purpose; overloaded of empirical study and convergent theory
concepts are likely to require design tradeoffs covering task, affect, and motivational factors
that render the concept suboptimal for at least in individual, shared, and collaborative learning
one purpose. Of course, unfulfilled purposes scenarios [16]. It is studied sufficiently to have
with no concept are often opportunities for new spawned multiple handbooks, literature
applications and enhanced designs. Jackson’s
reviews, and meta-analyses [1, 10, 16]. study and facilitate SRL using prompts [12],
Protocols and self-report instruments exist for and a system of prompts selected by learners
measures of learning and self-regulation [1]. [19]. These examples use some form of
Current research uses behavioral logs collected navigable content structure for the target
in online learning environments [21]. SRL learning domain.
contrasts with the concept of self-directed Also an early design, the general purpose,
learning (SDL), which unlike SRL, focuses on domain-independent gStudy system was
individual initiative and adult learners’ different [23]. The design sought to facilitate
formulation of their own learning objectives SRL through behaviors such as note-taking,
[13]. We acknowledge that SDL and other labeling, glossary building, concept mapping,
learning theories may be equally valid and coaching, chatting, and collaborating. The
useful for consideration in search-as-learning. system also included a learner’s display of
It is not our objective to claim SRL is the only analytics derived from logged interaction
useful paradigm. behavior. Much of the functionality involved
As implied above, SRL is a large and information search and interaction such as
complex research domain that bridges several “indexing, annotating, analyzing, classifying,
areas of psychology and practical aspects of organizing, evaluating, cross referencing and
education. Generally, results from experimental searching ” ([23] page 107). Later versions of
studies have informed models of factors the system (nStudy) incorporated a Web
affecting the use of cognitive and browser, webpage linking, tagging, hypertext
metacognitive skill as related to learning authoring, and a library of information
outcomes. As learning has moved to computers resources filterable on various bibliographic
and then online, these methods and attendant and user-generated metadata [22, 24].
research have moved to online learning Experimental systems from the SRL
systems. community have not used explicit models of the
individual learner (but see [15] for a notable
4. Learning system design exception), however a large, parallel body of
research in learner modeling has done so. Early
Learning systems (computer-based learning learner models tracked and facilitated content
systems; CBLEs) are designed for many navigation and summative assessment within a
purposes. Within the SRL community, designs closed system, with data generated during
derive from pre-computer classroom and observable behavior [5]. Modern systems use
tutoring approaches that enhance SRL and various forms of statistical modeling, where the
ultimately, learning outcomes. Studies on product of the model is generally a visual
experimental SRL systems focus typically on display. Open learner models (OLMs) make
methods for facilitating SRL, usage of SRL their underlying data accessible to the learner,
skill, and differential learning outcomes. Early who may initiate, append, or update the data
experimental systems focused on SRL within directly. OLMs may model states associated
the context of learning tasks such as homework with SRL, including data and reports on
assignments on a topic. The first published reflection, planning, monitoring, and formative
systems were domain-independent, general- evaluation [5].
purpose, and operated over the Encarta [10] reviewed 64 published OLMs designed
encyclopedia [2, 23]. Research with the early for higher education. The vast majority (89%)
MetaTutor system focused on scaffolding of models supported learning in STEM
learning goals for domain knowledge and specifically. Most of the OLMs (63%) operated
students’ use of SRL skills [2, 3]. A later within a closed learning system such as an
version of MetaTutor used animated automated tutor. The most common modeling
pedagogical agents to scaffold skills in SRL, objectives focused on predicting and tracking
with prompts and feedback delivered as student learners’ attainment of domain knowledge.
learning progressed [9]. Other examples Within the three-phase view of SRL
include a dashboard that prompts forethought (forethought, performance, assessment), fewer
and provides feedback on learning behavior than one-third of OLMs reviewed addressed
[14], a system that uses curricula structured in any part of a learner’s forethought, with support
pedagogical concept maps to guide a course of of performance and summative assessment
more common.
The above brief review suggests that more complex, covering not only the Web but
currently published learning systems often also tags, bookmarks, folders, saved work,
address domains where knowledge content can online textual material, media, library
be structured to scaffold and support the resources, a learning-management system, and
attainment of domain knowledge. Importantly, so forth, we expect the role of searching to also
learning also occurs in less structured domains be more complex. As a central psychological
where problems, goals, and standards for process for learning, a learner’s purpose for
success are relatively underspecified. For information search may involve accessing
example, success in information-intensive domain knowledge and self-regulation of
learning tasks such as writing a research paper learning. In the next section we consider how
require considerable SRL. This less structured these purposes may fit concepts for a search-
learning scenario provides context for centric learning system.
considering the purposes for search during
learning. 6. Purpose and search-centric
learning systems
5. Search in SRL learning systems
For a search-centric learning system, good
The development of experimental computer design provides search concepts that fit the
systems for SRL enabled researchers to trace user’s purposes within the context of the
students’ use of strategy and skills during study. learning application. The design of an
With those advances, the capture of data intelligent textbook provides a clear example.
indicative of the internal SRL processes has The purpose of the Inquire Biology textbook
been a key need, thus think-aloud methods are [6] is to assist students in learning complex
common. One early study used think-aloud concepts and their associations within the
during assignment completion in a hypertext biology domain. One of the central concepts
encyclopedia [4]. The environment included a used in the textbook’s design is question-and-
search function, which students were free to answer search, with attendant concepts and sub-
use. Utterances indicative of SRL were coded concepts such as question generation,
within the authors’ four-part model of SRL. vocabulary lookup, and term association
Monitoring (awareness of self, task, and search. These concepts fit the types of SRL
context) included identifying the adequacy of strategies that work well in highly structured
information and information content domains such as those found in STEM:
evaluation. Strategy use (control and regulation memorization, knowledge elaboration, self-
of self, task, and context) included coordinating test, and self-questioning. Within the design,
information sources; selecting a new search is not an overloaded monolithic concept.
information source; goal-directed information Rather, each purpose for searching is met with
search; free search (searching with no a concept fit for purpose. One may consider the
articulated goal); and evaluating content textbook a search-centric learning system,
relative to a learning sub-goal. Later work on albeit one that does not search beyond its
how students sequenced SRL activities also internal resources.
used think-aloud in a closed hypertext In the limited view presented in this paper,
environment [19]. Although the system did not learning systems may have two distinct
offer query-based search, searching for purposes: (1) to facilitate the learner’s
information and judging information relevance acquisition of special knowledge in a single
were found among key metacognitive domain (e.g., the Inquire Biology textbook) or
activities. The authors examined patterns of (2) to facilitate the development of transferable
SRL processes, finding prominent effects of the knowledge and skill in any domain; for
SRL system on the position of search within the example: critical thinking, reading for
patterns of SRL activity. comprehension, synthesis, and expository
The above results suggest that information writing. How well a single system can fulfil
search, interaction, and judgment are frequent both purposes is a matter for empirical study,
and central aspects of SRL, even in relatively but information search is essential in both
simple environments like a closed hypertext cases. As the Inquire Biology textbook
system. Where the options for searching are demonstrates Jackson’s notion of fitting
functional concepts to search purposes, we be accessible in textbooks, readings, prior work
argue that new system concepts can be designed on assignments, and other attendant sources.
to fulfill the purposes for searching in the Chris’s progress relative to instructional
second case. Indeed, we have argued that the scaffolding may also be available. Before
need for this view is compelling [18] due to working on a paper, Chris is likely to engage in
psychological effects on metacognition explicit forethought captured for later self-
associated with current system designs. Like reflection. When Chris works on one of the
others studying undergraduate learners [20] our papers, features of the assignment are
recent observations of 100+ college students accessible to the search system, along with
working on transfer-focused assignments concurrent evidence of engagement with search
revealed heavy reliance on Web search. Those functions and tools and supports for SRL. This
observations led us to consider the ways in context provides rich data for the search system
which search functionality may fit Jackson’s and for research examining the purposes for
definition of an overloaded concept. We believe searching external information sources during
there is need for design concepts that better SRL. For example, one such purpose is the
facilitate information search purposes in the notion of sourcing, a metacognitive skill used
context of SRL. in reading for comprehension where the reader
For example, we consider Chris, a freshman attends to “who says what” [7]. We argue that
nursing student taking two courses requiring a search functionality can be designed using
research paper. For a first-year writing course concepts that fulfil varied and complex
the paper can be on any topic. The paper needs purposes for searching during SRL.
to demonstrate research and writing skill;
pedagogically this is learning meant to transfer 7. Conclusion
to any general learning situation. For Chris’s
nursing class, the paper must go beyond the This paper makes three contributions to
course content to demonstrate understanding of search as learning. First, we reviewed self-
a chronic disease condition. Here the goal is to regulated learning as a useful paradigm for
show deep knowledge and synthesis, so Chris research on search as learning, focusing on how
wants to choose a condition that has already search activities may be conceptualized as self-
been introduced in class. regulated learning. Second, we introduced
Considering Chris’s goals through the lens Jackson’s [11] software design paradigm,
of search system design, observation of Chris’s focusing on alignment of the purposes for
current and past search behavior enables searching with functional concepts that fulfill
inference on the structure of the two tasks and those purposes. Third, we presented a design-
topics. Within this task-centric view, we may centric framework for considering the purpose
infer Chris’s more specific information goals of searching in academic tasks, proposing that
and internal state as interaction proceeds over a search-centric learning system may fulfil
possibly multiple sessions. Having inferred those purposes with the design of new
tasks, topics, goals, and internal state, functional concepts. We look forward to
inferences may be updated with the goal of discussing these ideas with IWILDS workshop
exposing only the information sources most attendees. [32, 18, 33, 34].
likely to optimize learning and task completion.
From Chris’s perspective, the search concepts
8. REFERENCES
used for this purpose are queries and results in
a Web search engine.
[1] Araka, E. et al. 2020. Research trends in
Flipping the design goal over embeds search
measurement and intervention tools for
functionality within a learning system. The
self-regulated learning for e-learning
purpose of SRL-focused systems is to facilitate
environments—systematic review (2008–
the development and use of effective strategies
2018). Research and Practice in
for study and academic achievement. Here one
Technology Enhanced Learning. 15, 1
or more OLMs may be in use, providing
(2020), 1–21.
information that obviates the need for inference
[2] Azevedo, R. et al. 2004. Does adaptive
on individual differences and preferences for
scaffolding facilitate students’ ability to
Chris’s learning processes and skills. Further,
regulate their learning with hypermedia?
specific sources of contextual information may
Contemporary Educational Psychology. Regulated Learning and Conceptual
29, 3 (Jul. 2004), 344–370. Thinking for Learning Analytics
DOI:https://doi.org/10.1016/j.cedpsych.2 Applications. J Learning Analytics. 6, 3
003.09.002. (2019), 106–121.
[3] Azevedo, R. et al. 2004. The role of self- [13] Loyens, S.M. et al. 2008. Self-directed
regulated learning in fostering students’ learning in problem-based learning and its
conceptual understanding of complex relationships with self-regulated learning.
systems with hypermedia. J. Educational Educational Psychology Review. 20, 4
Computing Research. 30, 1–2 (2004), 87– (2008), 411–427.
111. [14] Molenaar, I. et al. 2020. Personalized
[4] Azevedo, R. and Cromley, J.G. 2004. visualizations to promote young learners’
Does training on self-regulated learning SRL: the learning path app. Proceedings of
facilitate students’ learning with the Tenth International Conference on
hypermedia? J Educational Psychology. Learning Analytics & Knowledge (2020),
96, 3 (2004), 523. 330–339.
[5] Bull, S. and Kay, J. 2007. Student models [15] Nussbaumer, A. et al. 2015. A
that invite the learner in: The SMILI:() competence-based service for supporting
Open learner modelling framework. self-regulated learning in virtual
International J Artificial Intelligence in environments. J Learning Analytics. 2, 1
Education. 17, 2 (2007), 89–120. (2015), 101–133.
[6] Chaudhri, V.K. et al. 2013. Inquire [16] Panadero, E. 2017. A review of self-
Biology: A textbook that answers regulated learning: Six models and four
questions. AI Magazine. 34, 3 (2013), 55– directions for research. Frontiers in
72. Psychology. 8, (2017), 422.
[7] Delgado, P. et al. 2020. To click or not to [17] Perez De Rosso, S. et al. 2019. Declarative
click: investigating conflict detection and assembly of web applications from
sourcing in a multiple document hypertext predefined concepts. Proceedings of the
environment. Reading and Writing. 2019 ACM SIGPLAN International
(2020), 1–24. Symposium on New Ideas, New
[8] Efklides, A. 2011. Interactions of Paradigms, and Reflections on
metacognition with motivation and affect Programming and Software (2019), 79–
in self-regulated learning: The MASRL 93.
model. Educational Psychologist. 46, 1 [18] Smith, C.L. and Rieh, S.Y. 2019.
(2011), 6–25. Knowledge-Context in Search Systems:
[9] Harley, J.M. et al. 2017. Let’s Set Up Toward Information-Literate Actions.
Some Subgoals: Understanding Human- Proceedings of the 2019 Conference on
Pedagogical Agent Collaborations and Human Information Interaction and
Their Implications for Learning and Retrieval (New York, NY, USA, 2019),
Prompt and Feedback Compliance. IEEE 55–62.
Transactions on Learning Technologies. [19] Sonnenberg, C. and Bannert, M. 2015.
11, 1 (2017), 54–66. Discovering the effects of metacognitive
[10] Hooshyar, D. et al. 2020. Open learner prompts on the sequential structure of
models in supporting self-regulated SRL-processes using process mining
learning in higher education: A systematic techniques. J of Learning Analytics. 2, 1
literature review. Computers & Education. (2015), 72–100.
(2020), 103878. [20] Warwick, C. et al. 2009. Cognitive
[11] Jackson, D. 2015. Towards a theory of economy and satisficing in information
conceptual design for software. 2015 seeking: A longitudinal study of
ACM International Symposium on New undergraduate information behavior.
Ideas, New Paradigms, and Reflections on JASIST. 60, 12 (2009), 2402–2415.
Programming and Software (Onward!) DOI:https://doi.org/10.1002/asi.21179.
(2015), 282–296. [21] Winne, P.H. 2018. Cognition and
[12] Kivimäki, V. et al. 2019. Curricular Metacognition Within Self-regulated
Concept Maps as Structured Learning Learning. (2018). In D. H. Schunk & J. A.
Diaries: Collecting Data on Self- Greene (Eds.), Educational psychology
handbook series. Handbook of Self-
Regulation of Learning and performance
(p. 36–48). Routledge/Taylor & Francis
Group.
[22] Winne, P.H. et al. 2019. nStudy: Software
for learning analytics about processes for
self-regulated learning. J Learning
Analytics. 6, 2 (2019), 95–106.
[23] Winne, P.H. et al. 2006. Supporting self-
regulated learning with gStudy software:
The Learning Kit Project. Technology
Instruction Cognition and Learning. 3, 1/2
(2006), 105.
[24] Winne, P.H. and Hadwin, A.F. 2013.
nStudy: Tracing and supporting self-
regulated learning in the Internet.
International Handbook of Metacognition
and Learning Technologies. Springer.
293–308.
[25] Zimmerman, B.J. 2008. Investigating self-
regulation and motivation: Historical
background, methodological
developments, and future prospects.
American Educational Research Journal.
45, 1 (2008), 166–183.
[26] Zimmerman, B.J. and Schunk, D.H. 2011.
Self-regulated learning and performance:
An introduction and overview. Handbook
of Self-regulation of Learning and
Performance. 1–12.