=Paper=
{{Paper
|id=Vol-2699/paper18
|storemode=property
|title=Modeling Knowledge Change Behaviors in Learning-related Tasks
|pdfUrl=https://ceur-ws.org/Vol-2699/paper18.pdf
|volume=Vol-2699
|authors=Chang Liu,Xiaoxuan Song,Hanrui Liu,Nicholas J. Belkin
|dblpUrl=https://dblp.org/rec/conf/cikm/LiuSLB20
}}
==Modeling Knowledge Change Behaviors in Learning-related Tasks==
https://ceur-ws.org/Vol-2699/paper18.pdf
Modeling Knowledge Change Behaviors in Learning-related Tasks
Chang Liua, Xiaoxuan Songa, Hanrui Liua, and Nicholas J. Belkinb
a
Peking University, No.5 Yiheyuan Road Haidian District, Beijing, P.R.China
b
Rutgers University, 57 US Highway 1, New Brunswick, NJ, USA
Abstract
In Search as Learning (SAL) research, when and how learning occurs during the search process
has been a focus that attracts research attention. The goal of this study is to explore and characterize
searchers’ knowledge change patterns in the context of learning-related tasks from a process perspective.
A user experiment was conducted, and participants were asked to search for two learning-related search
tasks in a laboratory environment, and draw mind maps before and during search to keep a record of
what they know about the task. Searchers’ knowledge change behaviors during the search process were
extracted from their mind maps and analyzed based on the "Actions-Tactics-Strategies (ATS)" research
path. In this study, we report current preliminary analysis, which discovered twenty-five types of
knowledge change actions, and identified eight types of knowledge change tactics using bottom-up
clustering methods. The findings are the basis for our further exploration of searchers’ learning
strategies during the whole session, also present a complete behavioral and cognitive picture of
searchers’ knowledge change process, for search systems providing assistance at different stages of
searching and learning.
Keywords
Knowledge change behaviors, Learning-related tasks, Knowledge structure, Actions-Tactics-Strategies
(ATS), Search as Learning (SAL)
evaluated their knowledge gain as a search or
learning outcome [3, 5, 15, 18].
1. Introduction In addition to learning outcomes, in SAL,
researchers strive for demonstrating when and
“Search as Learning (SAL)” considers search how learning occurs during the search process.
systems as learning technologies rather than Some previous studies regarded users’ writing
merely information retrieval tools, and allows for behaviors and strategies as learning indicators [12,
an understanding of users’ information search 13]. However, it is difficult to infer learners’
behavior in the broader context of human learning. knowledge structure and their knowledge gain
Interpreting users’ information search behaviors solely through such textual evidence.
from the learning perspective is not a new topic. Research in sense-making has examined
Belkin’s [2] ASK model argues that users’ changes of knowledge structures using interview
knowledge state is anomalous and inadequate to or think-aloud protocols, e.g. Zhang & Soergel
achieve some goal and ASK is the motivation why [19]. They used three broad classes of conceptual
people turn to search. However, ASK did not fully changes: accretion, tuning and restructuring. Then
describe how users’ knowledge would change they further identified nine types of change
during search. Marchionini [10] described patterns. However, the think-aloud method may
information seeking as “a process, in which interfere with users’ searching behavior or
humans purposefully engage in order to change learning process. It may be difficult for some
their state of knowledge”. Kuhlthau’s ISP model users to simultaneously articulate their thoughts
[9] examined users’ emotional and cognitive and complete complex tasks [8].
changes during the search process. Recently, In the current study, we applied the mind-
more empirical studies focused on searchers’ mapping technique to elicit users’ knowledge
knowledge change during the search process, and changes during their search process, in order to
Proceedings of the CIKM 2020 Workshops, October 19-20, 2020, Galway, Ireland
EMAIL: imliuc@pku.edu.cn(A. 1); songxiaoxuan@pku.edu.cn(A. 2); lhr2013@pku.edu.cn(A. 3); belkin@rutgers.edu(A. 4)
ORCID: 0000-0002-9183-6385(A.1); 0000-0002-6589-4071 (A.2)
© 2020 Copyright for this paper by its authors. Use permitted under Creative
Commons License Attribution 4.0 International (CC BY 4.0).
CEUR
Wor
Pr
ks
hop
oceedi
ngs
ht
I
tp:
//
ceur
-
SSN1613-
ws
.or
0073
g
CEUR Workshop Proceedings (CEUR-WS.org)
clarify how learning occurs during the search and nineteen postgraduates whose majors
process. In our previous study [11], the mind-map included Information Science, Computer Science,
technique has been shown to be an effective tool Chemistry, Psychology, Sociology, Medical
to represent knowledge changes during the search Science and Environmental Science. We first sent
process. In this study, we developed a out a recruitment questionnaire, and then only
comprehensive coding system that considers selected those participants who were familiar with
users’ actions on both nodes and links in their the basic operations of mind-map and had drawn
mind maps. The sequence clustering method from a mind-map at least once in their daily work or
Hendahewa et al.’s two studies [4, 6] was study, to ensure that they all had sufficient
expanded and applied to identify knowledge knowledge in drawing mind-maps.
change patterns during the search process based During the experiment, participants used a
on users’ actions on their mind maps. desktop computer in our research lab to search for
Inspired by Bates’s [1] study on search moves, two learning-related search tasks. They first filled
tactics, and strategies, we propose a three-level out a background questionnaire. Before the search
analysis path, "Actions-Tactics-Strategies (ATS)" started, participants read the task description, and
to identify searchers’ knowledge change tactics then were asked to draw a mind-map using
and strategies (as shown in Figure 1). First, we XMind8 (https://www.xmind.cn/xmind8-pro/, a
coded manually to characterize and identify tool for supporting construction of mind-maps
different types of users’ knowledge change online) to represent knowledge they knew about
actions from mind maps; then we used the the topic. The next step was to complete a pre-
sequence clustering method to obtain knowledge search questionnaire to elicit data like topic
change tactics; and finally, knowledge change familiarity. Participants were instructed to modify
strategies were abstracted from the the mind-map during their search whenever they
transformational relationship of knowledge thought they learned something while searching,
change tactics in each session. The bottom-up and were told to stop searching when they
analysis could help us describe searchers’ believed that their mind-map represented the
knowledge change process comprehensively. knowledge needed to answer the task. After the
search, participants were asked to write an essay
in a notepad file to answer the task, only referring
to their mind-map records. When the essay was
submitted, a post-search questionnaire was given
Figure 1: The Actions-Tactics-Strategy (ATS) to participants to evaluate task difficulty. After
analysis path that, participants began work on the second
In this paper, we present our preliminary search task with the same procedure. Finally, the
results of the first two levels of the ATS path: participants completed a post-experiment
searchers’ knowledge change actions and tactics questionnaire about their general search
during the search process. Specifically, we have experience. The order of the two search tasks
two main research questions: were balanced among all the participants, that is
RQ1. During the knowledge change process half of participants completed task 1 first, the
while searching, how many knowledge change other half completed task 2 first. During search,
actions are there? What is the relationship participants’ interactions with the computer were
between knowledge change action types and their recorded by Morae Recorder 3.3.
associated duration?
RQ2. During knowledge change process while 2.1. Learning-related search tasks
searching, how many types of knowledge change
tactics are there? What are the characteristics of
each type of knowledge change tactic? We adopted the cognitive learning mode
model introduced by Rieh et al. [14] to construct
the learning-related tasks in our experiment. Two
2. Data Collection Method types of tasks were designed: Receptive learning
and Critical learning tasks. The receptive learning
A user experiment was conducted to address task is defined as understanding, remembering
our research questions. We recruited thirty-five and reproducing what is taught, and the critical
students from Peking University. Among all the learning task is defined as criticizing and
participants, there were fifteen males and twenty evaluating ideas from multiple perspectives. The
females, with ages between seventeen and descriptions of the two tasks are as follows.
twenty-nine. There were sixteen undergraduates Task 1 (Receptive learning, Topic: iPhoneX
face recognition): Your brother has just entered
college and wants to change to a new mobile They were encouraged to modify or update the
phone. He heard that Apple has launched a very mind-map to organize their thoughts after
powerful face recognition technology in iPhoneX, obtaining new information. They were also told
which makes the use of mobile phones more that, after done with searching, they would write
convenient and interesting. He hopes that you can down their answers to the task, referring only to
introduce him to functions and usage scenarios their mind-maps they drew during their search,
using face recognition technology in iPhoneX; at without checking any webpages at that point.
the same time, to describe the advantages and
innovations of face recognition in iPhoneX
compared with previous face recognition
3. Data Analysis Method
technology. You need to search for relevant
information to explain the above questions to your The data analysis in this study involves two
brother. main steps. The first step is to characterize users’
Task 2 (Critical learning, Topic: Bitcoin): knowledge change actions in Mind maps, which
Recently, Bitcoin has set off another wave of was achieved using manual two-layer coding. The
enthusiasm. Many students are interested in it but second step is to identify the knowledge change
don’t know much about it. In the "Internet tactics through clustering of sub-sequences of
Finance" course, you chose the topic of "Bitcoin" knowledge change actions. This section describes
to give a presentation of about 3 minutes. You these two main steps in detail.
intend to introduce the differences between
Bitcoin and common currency (such as RMB, 3.1. Encoding of users’ knowledge
USD). At the same time, analyze whether Bitcoin
can become a currency that is generally circulated change actions
in reality, and finally give your conclusion
(Yes/No). In order to complete this presentation, Through watching video recordings, we
you need to search for relevant information and encoded the knowledge change process in two
prepare your lecture material. layers. The first layer aims to reflect the process
of knowledge change actions in detail; the second
layer is to match the knowledge change actions
2.2. Mind-map drawing with types of conceptual changes in cognition for
the next step analysis.
Previous studies have demonstrated that • First layer coding
knowledge in human brains is organized An example participant’s pre-search mind-
semantically in networks, built piece by piece map is shown in Figure 2, and the mind-map after
with small units of interacting concepts and search is shown in Figure 3. The purpose of these
frameworks [7]. Visualizations could help figures is to indicate the general nature of a mind-
externalize and elicit the abstract structure of map, and to show how the structure of such a
knowledge, to support learners’ cognitive representation could change. The actual meanings
processing and retain knowledge in long-term of the nodes are not at issue here. We then coded
memory [17]. The mind-map technique provides the level of each node in the mind-map in the post-
a means to visually represent knowledge search map. There was at least one knowledge
structures, which could possibly support learning, tree in the mind-map, and each tree had a central
as well. node, which was the primary node and coded as
In the current study, participants were asked to level 1, and its children nodes were coded as level
first draw a pre-search mind-map after they read 2. We coded each node's level according to the
the task description, before they filled in the pre- above rules. An example of the coding is shown
search questionnaire and before they conducted in Figure 4.
the search. For this mind-map, they were asked to In general, users could change two types of
draw a mind-map which represented what they objects in mind maps: nodes or links. As there
already knew about this search task. They were exist differences in knowledge change, we
instructed that they could draw a mind map designed separate coding schemes for nodes and
structure directly if they had a structure in mind; links, as shown in Table 1 and Table 2, to code
otherwise, they could just list as many points as knowledge changes in the mind maps.
possible, and then choose which of them to
include in the mind map structure later.
While they were searching for information for
the tasks, they could modify and improve their
mind-maps using the information they collected.
Figure 2: A sample pre-search mind-map for Task
1 (S07) Figure 4: A coding example for node levels
Figure 3: A sample post-search mind-map for
Task 1 (S07)
Table 1: The coding scheme of node actions in Mind maps
Dimensions Coding Description
Add Add a node
Delete Delete a node
Actions Move Move a node
Modify Modify a node
Observe No action on the mind map
Modification Structural change Change happened on the nodes that belong to level 1 or 2
Degree Detail change Change happened on the nodes that belong to level 3 or below
Start node The first modification to the mind map, unable to describe the relative position
Father node The superior node contains this node
Child node The subordinate node of this node
Sibling node The same level as current node and has the same parent node
Same node The modified node is the same as the last modified node
Modification The node is a summary of some existing nodes generated by the summary
position Summary node
function in Xmind
Disordered node There is no direct connection between the current node and last modified node
Special disordered nodes. Although the modified node is not directly related to
Ordered node
the last modified node, the modification still follows a certain order
Actions after moving and observing, the nodes relationship cannot be clearly
Others
defined
Table 2: The coding scheme of link actions in Mind maps
Dimensions Coding Description
Actions The same as nodes' actions
Parallel The nodes at the both ends of links are in the same level
Link level
Cross The nodes at the both ends of links are in the different level
Association There is no content on links
Link significance
Differentiation There is content on links
Table 3: The mapping of knowledge structure change and associated actions in Mind maps
Change of knowledge structure Description Associated actions in Mind maps
The addition of new information
Accretion Node-Add- Detail change
without structural change
Node-Modify-Structural change
Node -Modify- Detail change
Organization and interpretation of
Tuning Node -Move- Detail change
information
Node -Delete- Detail change
All the actions on links
Node-Add-Structural change
Major change to existing knowledge
Restructuring Node-Move-Structural change
structure or creation of new structure
Node-Delete-Structural change
Only checking the mind map without Only checking the mind map without any
Observation
any actions actions
• Second layer coding words, multiple sub-sequences of knowledge
On the basis of the first layer coding, we can change actions were extracted from each session.
integrate part of the first layer of coding with The length of each sub-sequence was 18 (the length
Rumelhart and Norman’s [16] three kinds of of the last sub-sequence of the session may be less
concept change: accretion, tuning and restructuring. than 18), and the repetition rate between adjacent
Accretion refers to the addition of new information sub-sequences was 50%. The results of sequence
into existing knowledge, but does not cause cutting resulted in 1100 sub-sequences.
changes in the knowledge structure. Tuning focuses ③ The tactics are obtained through cluster
on the organization and interpretation of analysis. Then we carried out a hierarchical
information, which will cause weak changes in the clustering analysis on all sub-sequences to obtain
knowledge structure. Restructuring is a major the users’ knowledge change tactics.
change to the existing knowledge structure or the
creation of a new structure. We found that, during
search, sometimes searchers checked the mind map
4. Results
without making any changes. Such actions might 4.1. Knowledge change actions
serve to get an overview of their knowledge
structure or to confirm certain details. Thus, we add Twenty-five types of searchers’ knowledge
“Observation” as a new type of interaction with change actions were identified in this study,
knowledge structures, as shown in Table 3. considering the actions and positions of knowledge
change. As shown in Figure 5, Accretion actions
3.2. Identification of tactics were the most frequent actions, which accounted
for 51%, followed by Tuning (26%) and
Observation (13%), and Restructuring actions
This section describes the three steps involved
happened least frequently (10%).
in the identification of knowledge change tactics.
① Constructing knowledge change action
sequences. The complete description of knowledge
change actions includes two parts: action types and
duration. We used Hendahewa and Shah's [6]
method in constructing action sequences to
generate repeated action sequences according to the
duration of each knowledge change action. In order
to reduce the impact of the total task completion
time on the duration of a single behavior, we
normalized action durations in the session using the
following function: (Action length i - Action length Figure 5: Percentage of knowledge change actions
minimum) divided by (Action length maximum -
Action length minimum). The sequence of repeated With respect to knowledge change positions
actions is generated according to the standardized (Figure 6), searchers preferred moving between
duration, and then the sequence of repeated actions sibling nodes in the form of horizontal expansion,
is concatenated according to the occurrence order with the highest proportion (31%). Actions on
of knowledge change action to form the sequence father nodes and summary nodes only accounted
of knowledge change action for each session. for 2% and 1%, which suggests that searchers rarely
② Cutting knowledge changes action sequences restructured or summarized knowledge they
into sub-sequences. Because the length of collected during the knowledge change process.
knowledge change action sequences varies from 33
to 456 in different sessions, it was difficult to
directly compare sequences of knowledge change
actions. Therefore, each long sequence was divided
into sub-sequences of equal length. We wanted the
extracted sub-sequences to be realistic. So we
checked the number of knowledge change actions
each time users opened the XMind software, and
found that among all the sessions, the length of
knowledge change action sequence was shorter Figure 6. Percentage of knowledge change positions
than 18 in 95% cases. We therefore set window
length to 18 with a sliding distance of 9. In other
We further measured the duration of each type were identified, based on the maximization of
of knowledge change action and examined the difference, approximate equivalence of level, and
relationship between duration and the type of scale of clusters, as shown in Figure 9.
knowledge change action. The results in Figure 7
show that it took searchers the longest time to
Restructure their knowledge structure, followed by
Tuning and Accretion. Even though the duration of
Observation was the shortest, the average duration
was still 8.89 seconds. This indicates that learning
was a complex cognitive process and the more
complex the cognitive process, the longer it took
searchers to generate the output. This also
demonstrates that the coding of the knowledge
change actions in this study was reasonable.
Figure 9: Dendrogram of knowledge change tactics
In terms of the characteristics of each cluster, we
named the eight knowledge change tactics as
follows: Tactics of Accretion of Child nodes (TAC),
Tactics of Accretion of S nodes (TAS), Tactics of
Accretion of Disordered nodes (TAD), Tactics of
Tuning of Same nodes (TTS), Tactics of Tuning of
Disordered nodes (TTD), Tactics of Tuning of Link
actions or Node Position (TLP), Tactics of
Figure 7: Average duration for each type of Observation and Thinking (TOT), Tactics of
knowledge change action (seconds) Restructuring of Nodes (TRN).
Table 4 shows the percentage of each of the
The examination of duration for each position knowledge change actions that occurred in each
(Figure 8) reveals that users often spent the longest cluster of knowledge change tactics. In three of the
time on summary nodes, followed by discorded tactics, TAC, TAS, and TAD, Accretion actions
node and self-node. There was not much difference were dominant, accounting for more than 70% of
among child nodes, sibling nodes, father nodes or all actions, and the difference was the modification
ordered nodes. objects (either nodes or links, or the position of the
nodes). For example, in the sub-sequence of TAC,
users mainly added child nodes vertically. In TAS,
adding Sibling nodes actions were dominant in the
sub-sequence, which demonstrated a horizontal
expansion type of knowledge change pattern. In
TAD sub-sequences, users also frequently added
new nodes, but these new nodes were mostly
discorded nodes, neither child nodes, parent nodes,
nor sibling nodes. In these sub-sequences, users
may already have a certain amount of knowledge
Figure 8: Average duration for each position of
points, and were checking to fix certain gaps or
knowledge change action (seconds) deficiencies if there is any in their knowledge
structure.
4.2. Identification of knowledge In another three tactics, TTS, TTD, and TTL,
Tuning was the dominant actions during the sub-
change tactics sequences, the occurrences were all above 50%.
When Tuning actions were conducted, users
We calculated the Hamming distance among the usually modified, deleted or moved the detail nodes.
1100 sub-sequences, used the cluster package in R In the TTS sub-sequence, users consistently
to carry out hierarchical clustering analysis, and modified the same node every time they worked on
adopted the Ward method to calculate the distance the mind map, which showed an excelsior attitude
among the clusters. According to the results of the toward the knowledge structure. Besides
dendrogram, eight types of distinguishing clusters consistently modifying the same node, users in
TTD may modify different nodes and did not follow often frequently observe the knowledge map, and
any order in selecting the nodes to be modified, so then modify the content of some nodes. Therefore,
this tactic is named Tactics of Tuning of Disordered this tactic is named as Tactics of Observation and
nodes (TTD). Another type of Tuning dominant Thinking.
tactic is called Tactics of Tuning of Link actions or The final type of tactic is called Tactics of
Node Position (TTL), in which the main actions Restructuring of Nodes (TRN), in which the
were to add or modify links or move the position of percentage of Restructuring actions was
some existing nodes. Such modifications were particularly high (about 43.31%), and this action
mainly not to change the semantic meaning but only occurred around 5% in other types of tactics.
focusing on optimizing the structure. In this tactic, users also conducted certain amount
In terms of the TOT tactic, the occurrences of of accretion and tuning. This may indicate that
Tuning and Observation actions were similar, restructuring type of knowledge change is least
accounted for about 35% each, and the proportion frequently occurred during search, and this type of
of Accretion was slightly lower (22.16%). This is knowledge change often occur together with
apparently a special type of tactic, in which users accretion and tuning.
Table 4: The proportion of knowledge change actions for different types of tactics
The proportion of knowledge change actions %
Knowledge change tactics
Accretion Tuning Restructuring Observation
Tactics of Accretion of Child nodes (TAC) 80.45% 8.48% 5.29% 5.78%
Tactics of Accretion of Sibling nodes (TAS) 76.04% 17.50% 2.30% 3.95%
Tactics of Accretion of Disordered nodes (TAD) 70.26% 14.96% 2.12% 8.00%
Tactics of Tuning of Same nodes (TTS) 26.92% 58.27% 4.91% 8.68%
Tactics of Tuning of Disordered nodes (TTD) 30.99% 58.52% 5.12% 3.39%
Tactics of Tuning of Link actions or Node Position
27.07% 64.27% 6.02% 2.43%
(TTL)
Tactics of Observation and Thinking (TOT) 22.16% 38.94% 3.29% 34.36%
Tactics of Restructuring of Nodes (TRN) 28.82% 19.83% 43.31% 7.00%
5. Discussion Restructuring occurred least and last for the longest
time.
With respect to the knowledge change positions,
The aim of this study is to reveal users’ results show that sibling nodes were the most
knowledge change tactics base on the analysis of common nodes to be added or modified. This
users’ actions on mind maps during search. By implies that users often adopt a horizontal
adopting the "Actions-Tactics-Strategies (ATS)" expansion method when editing their knowledge
research path, this study first examined all kinds of
map. The second frequent change position the
actions that searchers could do on mind maps. Each disordered nodes, which were neither parent nodes,
action (add, delete, modify, or observe) were child nodes nor sibling nodes. This may indicate
mapped to one of the conceptual change types: that when users were searching information, they
Accretion, Tuning, Restructuring and Observation. may not always be oriented by the pre-defined
The frequency analysis and during analysis knowledge structure, but often edit the knowledge
showed that Accretion was the most frequent map according to the new information they
knowledge change type and it often took short time. acquired through searching. Future research would
Such result is consistent to Rumelhart & Norman also examine the relationship between content of
(1978) that Accretion is the most common form of webpages examined and the position of knowledge
learning during search, which may not require high change and how such relationship is related to
cognitive load, and relatively easy for users to document usefulness.
accomplish. The frequency and duration of Tuning When analyzing knowledge change tactics, we
are both at the medium level. Since Tuning may used sequence clustering methods on sub-sequence
involve weak structural change of knowledge and of knowledge change actions and identified eight
require more thinking and interpretation, the types of knowledge change tactics. The benefits of
occurrence is fewer than that of Accretion, and the identification of knowledge change tactics is that it
duration is a bit longer. The knowledge structure is could reveal a series of knowledge change actions
important for users since they need to rely on the users have conducted, and examine how users
structure to organize all the information they process the information they get through searching.
acquired through searching, and after deciding the
Among these eight types of tactics, three of them
structure, they seldom change it. Therefore, were Accretion dominant: TAC, TAS, and TAD,
each demonstrated vertical depth, horizontal This paper was supported by Natural Social
expansion and gap fixing pattern of learning. The Science Foundation Project “Examination of the
first two tactics show certain sequence for Relationship between User Interaction Behavior and
knowledge accretion, while the last tactic seems to Learning Effect in Learning Search” (#18BTQ090).
rely on the available new information they get from
searching. Future research could examine the 8. References
occurrence stage of these three tactics to see if the [1] Bates, M. J. (1979). Information search tactics. Journal of the
TAC and TAS happen at the early stage and the American Society for Information Science & Technology,
30(4) :205–214.
TAD happen at later stage of searching. [2] Belkin, N.J. (1980). Anomalous states of knowledge as a
There are also three types of tactics dominant by basis for information retrieval. Canadian Journal of
Tuning, TTS, TTD, and TTL. In these tactics, Information and library Science, 5, 133-143.
[3] Bhattacharya, N. & Gwizdka, J. (2019). Measuring Learning
participants often modify the same node several During Search: Differences in Interactions, Eye-Gaze, and
times consistently, or add/delete links between Semantic Similarity to Expert Knowledge. CHIIR '19:
nodes, or modify the content of nodes without Proceedings of the 2019 Conference on Human Information
Interaction and Retrieval, March 2019, pp 63–71.
following any order. [4] Cole, M. J., Hendahewa, C., Belkin, N. J., & Shah, C. (2015).
There is only one tactic that was dominant by User activity patterns during information search. ACM
Restructuring, TRN. Restructuring is related to the Transactions on Information Systems, 33(1), 1-39.
main knowledge structure and this structure may be [5] Gadiraju, U., Yu, R., Dietze, S., & Holtz, P. (2018).
Analyzing knowledge gain of users in informational search
related to how users organize their thoughts and the sessions on the web. In Proceedings of the 2018 Conference
information they receive through searching. We on Human Information Interaction & Retrieval (pp. 2-11).
may speculate TRN tactics often occur at the [6] Hendahewa, C., & Shah, C. (2014). Segmental Analysis and
Evaluation of User Focused Search Process. International
beginning and the end of the search and this will be Conference on Machine Learning & Applications. IEEE.
further validated in future studies. [7] Jonassen, D.H. (2000). Computers as Mindtools for Schools:
In future research, we will continue to connect Engaging Critical Thinking. Columbus, OH: Prentice-Hall.
[8] Kelly, D. (2009). Methods for evaluating interactive
these knowledge change tactics with users’ search information retrieval systems with users. Foundations and
behaviors to reveal how their search behaviors or Trends in Information Retrieval, 3(1-2), 1-224.
the content they read lead to different types of [9] Kuhlthau, C. C. (1991). Inside the search process:
Information seeking from the user’s perspective. JASIS&T,
knowledge change tactics. In addition, we will also 42(5), 361–371.
investigate the distribution of these knowledge [10] Marchionini, G. (1995). Information seeking in electronic
change tactics during task completion process to environments. New York: Cambridge University Press.
[11] Liu, H., Liu, C., & Belkin, N. J. (2019). Investigation of
summarize users’ knowledge change strategy, and users' knowledge change process in learning‐related search
whether different strategies may lead to different tasks. Proceedings of ASIS&T, 56(1).
learning performance. The ultimate goal is to reveal [12] Liu, J., & Belkin, N. J. (2013). Searching vs. writing: factors
the most effective learning strategy or to provide affecting information use task performance. Proceedings of
ASIS&T, 49(1), 1-10.
effective learning tools embedded in the search [13] Hagen, M., Potthast, M., Völske, M., Gomoll, J., & Stein, B.
system to help searchers achieve better learning (2016). How Writers Search: Analyzing the Search and
performance. Writing Logs of Non-fictional Essays. In Proceedings of
2016 ACM Conference on Human Information Interaction
and Retrieval, 193–202.
6. Conclusion [14] Rieh, S. Y., Collins-Thompson, K., Hansen, P., & Lee, H. J.
(2016). Towards searching as a learning process. Journal of
This study explored searchers’ knowledge Information Science, 42(1), 19-34.
change patterns in the context of learning-related [15] Roy, N., Moraes, F., & Hauff, C. (2020). Exploring Users'
Learning Gains within Search Sessions. In Proceedings of the
tasks from a process perspective. We first coded 2020 Conference on Human Information Interaction and
participants’ knowledge change actions in mind Retrieval (pp. 432-436).
maps, and found twenty-five types of knowledge [16] Rumelhart, D.E. & Norman, D.A. (1978). Accretion, Tuning,
change actions. Then we used sequence clustering and Restructuring: Three Modes of Learning, In Semantic
Factors in Cognition, J. W. Cotton & R. L. Klatzky (ed.),
methods and identified eight types of knowledge Lawrence Erlbaum and Associates, Hillsdale, NJ.
change tactics. The findings are the basis for our [17] Wang, M., Peng, J., Cheng, B., Zhou, H., & Liu, J.. (2011).
further exploration of searchers’ learning strategies Knowledge Visualization for Self-Regulated Learning.
Educational Technology & Society, 14 (3), 28–42.
during the whole session, also present a complete [18] Yu, R., Gadiraju, U., Holtz, P., Rokicki, M., Kemkes, P., &
behavioral and cognitive picture of searchers’ Dietze, S. (2018). Predicting user knowledge gain in
knowledge change process, for search systems informational search sessions. In The 41st International
ACM SIGIR Conference on Research & Development in
providing assistance at different stages of searching Information Retrieval (pp. 75-84).
and learning. [19] Zhang, P. & Soergel, D. (2016). Process patterns and
conceptual changes in knowledge representations during
information seeking and sensemaking: a qualitative user
7. Acknowledgements study. Journal of Information Science, 42(1), 59-78