Analyzing Frequent Sequential Patterns of Learning Behaviors in Concept Mapping Shang Wang Erin Walker Ruth Wylie School of Computing, Informatics, and School of Computing, Informatics, and Mary Lou Fulton Teachers College Decision Systems Engineering Decision Systems Engineering Arizona State University, Tempe AZ, Arizona State University, Tempe AZ, Arizona State University, Tempe AZ, USA USA USA Ruth.Wylie@asu.edu swang158@asu.edu Erin.A.Walker@asu.edu ABSTRACT This paper explores the use of data mining methods to Computer-based concept mapping learning environments can systematically build and analyze models of student behaviors as produce large amounts of data on student interactions. The ability they interact with our concept map environment. This paper to automatically extract common interaction patterns and approaches student modeling by analyzing similar and different distinguish between effective and ineffective interactions creates behavior patterns between various types of student groups. opportunities for researchers to calibrate feedback and assistance to better support student learning. In this paper, we present an 2. WORKFLOW METHOD exploratory workflow that assesses and compares student learning 2.1 Data Inputs behaviors with concept maps. This workflow employs a The raw data are xml files, where each item in corresponds to a sequential pattern mining technique to classify interaction patterns specific action performed by students on the system. There are 8 among students and determine specific behavior patterns that lead fields of information being logged in each student action. to better learning outcomes. 1. Student ID, identifying the student interacting with the Keywords system. 2. Session ID, denoting the session of the study. Data mining, sequential pattern mining, student behavior, concept mapping. 3. Time, recording the time stamp of the action. 1. INTRODUCTION 4. Time zone, indicating the time zone of the system. Concept maps are visual representations of knowledge, with 5. Selection, representing where student is interacting with. concept nodes representing concepts in the knowledge structure For example, concept map view, textbook view, etc. and links denoting relationships among concepts. Concept mapping has been widely used as an active learning tool in 6. Action, denoting the specific student action. For educational contexts and research has shown the positive effect of example, adding a concept node from the textbook, concept mapping in helping students organizing and summarizing navigating to a new page, linking two concepts, knowledge [1][2]. One of the main disadvantages of concept hyperlinking navigation, etc. mapping is the complexity of the task. Learners who lack expertise often feel overwhelmed and de-motivated [3]. 7. Input, representing the input of the action. For example, an input for adding a concept from the textbook would To facilitate students in concept map construction, we designed a be “root” and an input for navigating to a new page personalized and interactive concept mapping learning would be “page 5”. environment integrated within a digital textbook. Students are 8. Page number, indicating the text page when the action able to create maps directly from the textbook, which allows them is performed. to better relate concepts with the textbook content. The system offers a hyperlinking navigation feature where, after creating the These raw data are generated in real-time and are sent to a server concept map from the textbook, students are able to click on the after each session for further analysis. concept nodes and navigate to where these nodes were added from the textbook. We hypothesize that this feature supports learning Apart from the log files, we also use pre and post test results and by offering flexibility in comparing and finding connections final concept maps for analysis. Pre and post tests consist of 30 between concepts that are located in different pages, multiple choice questions. The test results can be used to classify students into high and low performance groups and help us To examine the effect of interactive concept mapping learning determine specific behavior sequences that distinguish the better environments, we have conducted a week-long study with 32 high groups from the weaker ones. Similarly, the concepts created by school students using the system as a substitute for a paper-and- students enable us to understand how different behavior patterns pencil based concept mapping activity while they learn about their affect concept mapping. current science textbook chapter. Students in the study were randomly assigned into two conditions: A hyperlinking condition, 2.2 Workflow Model where nodes in the concept maps were hyperlinked with the Action abstraction is the first step of our workflow, in which we textbook, and a non-hyperlinking condition. Pre and post tests categorize a specific sequence of low granularity actions into were given before and after the study to measure learning aggregated actions that indicate specific learning behaviors. This outcomes. step filters out irrelevant information and combines qualitatively similar actions (Table 1). For example, a student might flip 10 pages in the textbook quickly when searching for certain sections 1. Hyperlinking and No-hyperlinking: Comparing in the textbook. Instead of analyzing these 10 navigation actions sequential patterns between hyperlinking and non- separately, we consider them as one aggregated action called hyperlinking conditions suggests how hyperlinking “Quick Search” (QS). navigation affects student behaviors. Aggregated Log Action 2. High performance and low performance: Comparing Behavior frequent patterns in these two conditions identifies Quick Search Students flip several pages quickly to go to certain behavior patterns that distinguish better learning (QS) a specific page groups than the lower ones. Long Stay (LS) Students don’t perform any actions for a 3. Better concept maps and weaker concept maps: long period of time Comparing sequential patterns in these two conditions would help us understand how behavior patterns affect Read and Add Students read the textbook and add a the final concept maps created by students. (RA) concept node into the concept map Read and Link Students read the textbook and link two 3. DISCUSSION (RL) concepts in the concept map We present a workflow that first creates aggregated behaviors Add and Link Students add a concept node to the concept from the log files and then applies sequential pattern mining to (AD) map and quickly link it to another node extract behavior patterns from various conditions. Comparisons of student behaviors between the hyperlinking and non-hyperlinking Read and Delete Students read the textbook and delete a condition would help us understand how the hyperlinking feature Node (RD) node from the concept map affects student navigation. Questions like does the navigational Hyperlinking Students click on a concept node to flexibility in the hyperlinking condition yield more comparison Navigation (HN) navigate to the page where it’s created between concepts located in different pages in the textbook would Back and Forth Student navigate between a few pages be interesting to explore. Comparisons of student behaviors (BF)d back and forth within a short period of between different types of student groups would help us examine time specific behavior patterns that lead to high learning outcomes and better concept maps, which provides opportunities for researchers Table 1. Student actions and aggregated behaviors to develop feedback or scaffolding methods to support these We classify all the student actions into 8 aggregated student behaviors. This work opens doors for teachers or automated behaviors, which are easier for sequential pattern mining and systems to intervene and provide feedback more appropriately. It student modelling. For example, a back and forth (BF) behavior also enables researchers to develop concept mapping learning could be an indication that the student is comparing two linked environment that offers automation to replace the ineffective concepts in the concept map. A long stay (LS) behavior might behaviors while preserving and supporting behaviors that yield suggest that the student is spending a lot of effort reading the better learning outcomes. textbook or distracted and not motivated. 4. ACKNOWLEDGMENTS After this classification, we apply sequential pattern mining This research was funded by NSF CISE-IIS-1451431 EAGER: techniques to extract interesting behavior patterns. Research in the Towards Knowledge Curation and Community Building within a literature has applied sequential pattern mining techniques to a Postdigital Textbook. variety of educational data. Perera and colleagues showed the importance of leadership and group interactions towards learning 5. REFERENCES success using k-means clustering to find groups of similar teams [1] Nesbit, John C., and Olusola O. Adesope. "Learning with and similar individuals, and employing a modified version of the concept and knowledge maps: A meta-analysis." Review of Generalized Sequential Pattern (GSP) mining algorithm to extract educational research 76.3 (2006): 413-448. student behavior patterns [4]. Martinez et al. applied clustering [2] Novak, Joseph D., and Alberto J. Cañas. "The theory and sequential pattern mining techniques to determine the underlying concept maps and how to construct and use sequences of actions that characterize high-achieving and low- them." (2008). achieving learners [5]. [3] Davies, Martin. "Concept mapping, mind mapping and In our workflow, we plan to use sequential mining techniques to argument mapping: what are the differences and do they identify the frequent sequential patterns from the two conditions matter?." Higher education 62.3 (2011): 279-301. for further analysis. [4] Perera, Dilhan, et al. "Clustering and sequential pattern 2.3 Workflow Outputs mining of online collaborative learning data." Knowledge The first output from the workflow model is a list of sequential and Data Engineering, IEEE Transactions on 21.6 (2009): patterns extracted from the log files depending on the minsup. 759-772. These patterns represent frequent student behaviors that occurred [5] Martinez-Maldonado, Roberto, et al. "Analysing frequent during the concept mapping task. After extracting frequent sequential patterns of collaborative learning activity around behavior patterns, we further cluster these patterns based on an interactive tabletop." Proceedings of the International different student groups. Conference on Educational Data Mining 2011 (EDM 2011). 2011.