Online computer-supported collaborative learning (CSCL) has risen in popularity in knowledge sharing and problem-solving. This research explored students' online activity in online problem-based learning (PBL) using process and sequence mining approaches. Process mining modeled students' time-stamped activities and links between them. Sequence mining provided an overview of the flow and frequencies of students' activities through sequential process maps. Our finding showed that the most frequent students' activities were nonargument discussions followed by sharing knowledge and social interactions. The process model of the students' discussion started with sharing knowledge most of the time and then students either evaluate or argue others' messages to end discussions through social interactions. The sequence mining model showed that social interaction and non-argument discussion are the most common starting activities by students. It is concluded that process and sequence mining allowed us to identify different stages of online forum discussion in PBL.
The recent growth of online education systems resulted in a vast amount of data from various resources and in different forms [1], [2]. Such educational big data led to learning analytics development to improve teaching and learning [3]. Learning Analytics (LA) is defined as “the measurement, collection, analysis and reporting of data about learners and their contexts, for purposes of understanding and optimizing learning and the environments in which it occurs” [4]. LA mainly focus on analysis and modeling of educational data through the prediction of students’ achievement and dropout, tracking of students’ progress and exploring of learners’ behavior [3]. Using analytics in high education could enhance decision making by relying on data informed insights [5]. LA has utilized several methods and techniques such as descriptive statistics, social network analysis, process mining, sequence mining, text mining, prediction, clustering, causal mining and visualization, classification, and association techniques for solving different educational problems [2].
The students’ participation in various learning activities may vary over time. Thus, research methods were developed to study temporal aspects of learning activities [6]. Process and sequence mining are among these LA methods that are technically related but have different objectives [7]. Process mining (PM) is an educational data mining technique that forms a link between the data and process sciences. PM allows the extraction of log data from information systems to develop a process model, test compliance, assess obstacles, compare processes, and provide recommendations for improvements [8]. In PM, researchers mainly visualize and analyze the students’ learning process rather than getting predictive insights from educational data [9]. The educational application of PM includes detecting students’ learning strategies [10], identify selfregulated learning in massive open online courses (MOOC) [11], understand students’ approaches in learning [12], and analysis of the reasons of students' dropout [13]. PM was also used to investigate problem-solving in online forums [14] and understand the physiological arousal events in collaborative problem-solving [15]. Learning strategies of students with different performance levels and personalities can be compared using PM [16], [17]. The main limitation of PM that it is frequencybased with lack of inferential statistics, so it is unsuitable to investigate learners’ behavior heterogeneity or visualize behavior trajectories in courses. Thus, sequence mining was developed to study the learners’ trajectories and clustering them while maintaining the data sequence and timing [18].
Sequence mining (SM) is adopted in LA research to extract significant information from timeseries data that summarize learning sequence patterns and investigate students’ learning approaches and strategies [19]. SM can visualize learning activities succession, recurring activities, and overview of learning strategies. Moreover, SM data can be categorized into subgroups according to sequences of learning approaches [20] and behaviors [21]. Researchers used SM to explore students’ tactics and strategies in learning and organize them into comparable clusters according to students’ achievements [10], [22]. Multi-channel sequence mining was recently used for students’ timestamped records analysis to detect transitions between various learning activities [12], [18]. In the game-based learning environment, SM was used to track students’ metacognitive scientific reasoning [23] and identify problem-solving strategies and trajectories [24]
Addressing the temporal and sequence of learning activities can be beneficial in collaborative learning [25]. Computer-supported collaborative learning (CSCL) is based on knowledge construction through students’ social interaction in a positive environment that gives the opportunity to discuss, debate and respond to ideas [26], [27]. To exploit this approach, various pedagogical strategies have been adopted such as problem-based learning (PBL) and team-based learning (TBL) that help students to collaborate and interact in small groups. In PBL, the problem can trigger students to interact, debate and elaborate their knowledge [28]. Students’ discussion in online PBL starts by posting the question and waiting for other participants to interact in one of two forms: content or relationship spaces. In the content space, students gain knowledge, collect information, and discuss problems and their solutions. The relationship space —including social interaction— focus on communication activities, and how students interact with their colleagues during discussions [29]. Tutors in PBL encourage discussion, sharing knowledge and collaboration to achieve learning objectives. Moreover, tutors monitor students through PBL sessions to recognize group dynamics, minimize conflict, avoid interaction dysfunctions, and respect different opinions [31]. Analysis of PBL data in an organized way using LA help to make the proper academic decisions [32]. Previous studies have utilized LA to study collaborative learning in PBL such as network analysis [31], [33] and temporal network to monitor the students’ interactions [34].
Participation and social interactions are critical elements of online CSCL to establish a favorable environment for effective knowledge building among learners [35]. This study builds on monitoring the students’ activities through coding of the student’s interaction in online PBL sessions using the process and sequence mining approaches. We try to address the temporal activities and answer the following research question How can learning analytics explore the temporal dynamics in students’ discussions during problem-based learning?
Data from first-year dental college students enrolled in a general histology course. The course is delivered through Moodle learning management system (LMS). The proposal of this research has been submitted to the ethical committee of the college and ethical approval was issued. Students were provided with information about the research to consent participation. We retrieved the student’s ID, groups, timestamp, and submitted posts by each user. All students’ IDs were blinded to confirm anonymization. The dataset consists of total 1,265 posts from the interaction of the 68 enrolled students (23 females and 45 males) in total five PBL scenarios. Students are required to attend two physical appointments every week. In between these appointments, students are given access to an online discussion where they can continue to communicate through the LMS. The discussions begin after the first appointment to discuss problems, share information, collaborate to build the necessary knowledge, and work toward the PBL objectives. Each group is required to conclude what they've learned together at the end of the week. The PBL sessions are designed to address all the course learning outcomes. Other educational activities like lectures or seminars are intended to help students grasp the PBL. In the mid-course and final exams, students are assessed using multiple choice questions (MCQ) [31].
Two coders coded the students’ posts, considering eight activities in the discussion board (Table 1). The codes were modified from previous codes used by Saqr and López-Pernas [36] for discourse analysis of online PBL. For each post, the action was coded as “one” for having the action and “zero” for not having the action. The inter-coder agreement amongst the two coders showed a strong reliability using Cohen's Kappa test (κ= 0.88) [37], and the coders met to solve cases with disagreements.
The data were ordered in a chronological order for data analysis A. Frequency analysis: the frequency of each action during the students’ discussion was investigated throughout the PBL sessions. These frequencies were used for process and sequence mining.
B. Process mining using First Order Markov models: The methods entail the creation of a process model from a series of time-stamped activities of the overall student interactions i.e., takes a longitudinal overview of students’ typical interactions in the whole PBL discussions over the full course. This process model links different activities, and possible tactics of learning. The pMineR R-package [38] was utilized according to the methods described in Peeters et al. [39] C. Sequence mining: the sequence model was created using TraMineR package [40], by grouping the chronologically ordered learning activities using the methods described in detail in López‐ pernas et. al [12]. Sequence mining analyzed the typical learning session which is a typical episode (minutes to hours of length) of the student’s interactions in the online PBL.
The total posts of the course were 1265 posts with 3803 activities in the PBL threads. The posts were coded according to the codebook. Table 2 shows the descriptive statistics for the students’ activities for the eight activities. The most common activity recorded by students was non-argument discussion in more than one thousand record (29.3%) followed by sharing knowledge in 21.5%. The least activities of students recorded was team management in 3.47% of discussion board activities.
The process model of all students’ interactions aims at charting the process of student’ interactions in the PBL process over the whole duration of the course. Based on the students’ activities is modeled in (Figure 2). Share knowledge is the central activity that 87% of students begin their interaction with e.g., “Proteins are large biomolecules, or macromolecules, consisting of one or more long chains of amino acid residues". The students commonly start argumentation after that (transition probability (TP)=0.5) e.g., “researchers argue that it's BMI not the most accurate way to measure body weight. For years, scientists have said that BMI can't distinguish between fat and muscle” or start evaluating others’ posts (TP=0.3) e.g., “That's right. Fiber refers to certain types of carbohydrates that our body cannot digest”. Most of the sessions ended by non-argument discussion e.g., “We now know the difference between exocrine and endocrine glands” which is followed by social interactions between students e.g., “very nice topic; interesting information and illustrations”. Some students may ask and by doing so, the question triggers discussions (TP=0.6) e.g., “I have a question Do you know what is the recommended intakes of carbohydrates?”. The resuming discussions have followed argumentation (TP=0.2), as some students may interact in multiple threads e.g., “I will complete what you have started: The second type of connective tissue fibers is white collagenous fibers which can be found in tendons, ligaments, and capsules of organs”.
Sequence mining was applied to coded activities within sessions (limited durations of interactions). The overall sequence was plotted in Figure 2. Most sessions would start with argumentation in almost one third of activities, the argumentation decreases by the time to increase near the end of the sessions. The second most common starting activity was non-argument discussion, and it continued as the most common activity with increased frequency to end up as the most common activity by the end of the sessions. Although social interaction was among the less frequent activities at the start, it increased to become alternating with the non-argument discussion as the most common activity throughout the sessions. In summary, the sequence shows a mix of arguments, discussions with social activities that are intertwined together.
In this study we applied process mining (PM) and sequence mining (SM) to explore the temporal dynamics of students’ online collaboration during PBL. In PBL, the students’ discussions are dense as they are involved in sharing knowledge, debate and arguing rather than simply solve the problem [36].
The contents of the online interactions were unpacked and coded according to the dynamic activities of the students. PM showed that most of the students started their posts by sharing knowledge, this may be explained by the idea that students need to share resources to understand the basic information related to the problem to fill the knowledge gap with their colleagues. After sharing knowledge students started discussing such information through evaluation and argumentation. This is mirroring the students’ efforts for reflection which is supportive of deep learning and communal knowledge building [41]. The sequence mining showed that the social interactions were increasing with time during the students’ discussions – with non-argument discussion – to be the most common activities throughout the sessions. SM can help go beyond analyzing learning sequence patterns to do clustering of students into subgroups according to different variables [20] [21] and detection of transitional activities using multi-channel SM [10], [22], which we aim to use in future research.
The social interactions were progressively increasing during students’ collaborative discussion, which include sharing feelings, off topic messages and complimenting others this may indicate that online collaborative learning motivate students’ social interaction. Strong and effective social interactions are considered as an essential component of collaborative learning [42]. Kim, 2010 identified social relationships as a strong determinant of student progress and satisfaction [43]. Moreover, other scholars have recognized the link between social interactions and reflection to improve knowledge building and enhance learning outcomes [44]. It was well established that social interaction alone will not assure the emergence of critical online discussion, however, the development of such discussion is challenging without the social interactions’ basis [45] .
This study is not free from limitations. One of the limitations of this study is the small sample size, so including more students in different courses may give us more insights into the students’ collaboration strategies in PBL. Another limitation of the study that it analyzes the students’ posts only while the role of the teacher is unclear for students’ guidance and support which can be studied in future work. Future research may also compare students’ activities of different performance levels and cluster these activities according to the time-series of action using a sequence mining cluster algorithm.
In summary, we suggested using process mining and sequence mining as dynamic learning analytics methods to analyze the changes in activities across the time in PBL. The results showed that most of the students start their interaction on the discussion board by sharing knowledge with their colleagues to start building their discussion. Although social interaction was among the less frequent activities when a session started according to sequence mining, it increased to become the most common activity throughout the sessions with non-argument discussion.
A. Hadwin, S. Järvelä, and M. Miller, “Self-Regulation, Co-Regulation, and Shared Regulation in Collaborative Learning Environments,” in Handbook of Self-Regulation of Learning and Performance, 2nd Ediiti., D. Schunk and J. Greene, Eds. New York: Routledge, 2017, pp. 83–106. M. J. Gomez et al., “Applying learning analytics to detect sequences of actions and common errors in a geometry game,” Sensors (Switzerland), vol. 21, no. 4, pp. 1–16, 2021, doi: 10.3390/s21041025.
W. van der Aalst, “Process Mining: The Missing Link,” in Process Mining, Berlin, Heidelberg, Heidelberg: Springer Berlin Heidelberg, 2016, pp. 25–52.
H. Alqaheri and M. Panda, “An Education Process Mining Framework: Unveiling Meaningful Information for Understanding Students’ Learning Behavior and Improving Teaching Quality,” Information, vol. 13, no. 1, p. 29, Jan. 2022, doi: 10.3390/info13010029.
W. Matcha et al., “Analytics of Learning Strategies: Role of Course Design and Delivery
Modality,” J. Learn. Anal., vol. 7, no. 2, pp. 45–71, 2020, doi: 10.18608/jla.2020.72.3.
J. Maldonado-Mahauad, M. Pérez-Sanagustín, R. F. Kizilcec, N. Morales, and J. Munoz-Gama,
“Mining theory-based patterns from Big data: Identifying self-regulated learning strategies in
Massive Open Online Courses,” Comput. Human Behav., vol. 80, pp. 179–196, Mar. 2018, doi:
10.1016/j.
S. López‐pernas, M. Saqr, and O. Viberg, “Putting it all together: Combining learning analytics methods and data sources to understand students’ approaches to learning programming,” Sustain., vol. 13, no. 9, p. 4825, Apr. 2021, doi: 10.3390/su13094825.
J. P. Salazar-Fernandez, M. Sepúlveda, and J. Munoz-Gama, “Influence of student diversity on educational trajectories in engineering high-failure rate courses that lead to late dropout,” IEEE Glob. Eng. Educ. Conf. EDUCON, vol. April-2019, pp. 607–616, 2019, doi: 10.1109/EDUCON.2019.8725143.
G. A. Wang, H. J. Wang, J. Li, and W. Fan, “Mining knowledge sharing processes in online discussion forums,” Proc. Annu. Hawaii Int. Conf. Syst. Sci., pp. 3898–3907, 2014, doi: 10.1109/HICSS.2014.483.
M. Dindar, S. Järvelä, A. Nguyen, E. Haataja, and A. Çini, “Detecting shared physiological arousal events in collaborative problem solving,” Contemp. Educ. Psychol., vol. 69, p. 102050, Apr. 2022, doi: 10.1016/j.cedpsych.2022.102050.
D. Gasevic, J. Jovanovic, A. Pardo, and S. Dawson, “Detecting Learning Strategies with Analytics: Links with Self-reported Measures and Academic Performance,” J. Learn. Anal., vol. 4, no. 2, pp. 113–128, 2017, doi: 10.18608/jla.2017.42.10.
J. Malmberg, S. Järvelä, H. Järvenoja, and E. Panadero, “Promoting socially shared regulation of learning in CSCL: Progress of socially shared regulation among high- and low-performing groups,” Comput. Human Behav., vol. 52, pp. 562–572, Nov. 2015, doi: 10.1016/j.chb.2015.03.082. S. López-Pernas and M. Saqr, “Bringing Synchrony and Clarity to Complex Multi-Channel Data: A Learning Analytics Study in Programming Education,” IEEE Access, vol. 9, no. 1, pp. 166531–166541, 2021, doi: 10.1109/ACCESS.2021.3134844.
N. A. A. A. Uzir, D. Gaševic, J. Jovanovic, W. Matcha, L.-A. A. Lim, and A. Fudge, “Analytics
of time management and learning strategies for effective online learning in blended
environments,” in The 10th International Conference on Learning Analyti
A. Gabadinho, G. Ritschard, N. S. Müller, and M. Studer, “Analyzing and visualizing state sequences in R with TraMineR,” J. Stat. Softw., vol. 40, no. 4, pp. 1–37, 2011, doi: 10.18637/jss.v040.i04.
J. S. Kinnebrew and G. Biswas, “Identifying learning behaviors by contextualizing differential sequence mining with action features and performance evolution,” in 5th International Conference on Educational Data Mining (EDM 2012), 2012, pp. 57–64.
W. Matcha et al., “Detection of Learning Strategies: A Comparison of Process, Sequence and Network Analytic Approaches,” in Transforming Learning with Meaningful Technologies. ECTEL 2019. Lecture Notes in Computer Science, vol. 11722, M. Scheffel, J. Broisin, V. PammerSchindler, A. Ioannou, and J. Schneider, Eds. Cham: Springer, 2019, pp. 525–540.
M. Taub, R. Azevedo, A. E. Bradbury, G. C. Millar, and J. Lester, “Using sequence mining to
reveal the efficiency in scientific reasoning during STEM learning with a game-based learning
[24]
[25]
[26]
[27]
[28]
[29]
[30]
[31]
[32]
[33]
[34]
[35]
[36]
[37]
[38]
[39]
[40]
[41]
[42]
environment,” Learn. Instr., vol. 54, pp. 93–103, Apr. 2018, doi:
10.1016/j.learninstru
T. Liu and M. Israel, “Uncovering students’ problem-solving processes in game-based learning environments,” Comput. Educ., vol. 182, p. 104462, Jun. 2022, doi: 10.1016/j.compedu.2022.104462.
J. Malmberg, S. Järvelä, and H. Järvenoja, “Capturing temporal and sequential patterns of
self, co-, and socially shared regulation in the context of collaborative learning,” Contemp. Educ.
Psychol., vol. 49, pp. 160–174, Apr. 2017, doi: 10.1016/j.
Y. Woo and T. C. Reeves, “Meaningful interaction in web-based learning: A social constructivist interpretation,” Internet High. Educ., vol. 10, no. 1, pp. 15–25, 2007, doi: 10.1016/j.iheduc.2006.10.005.
E. Michael Nussbaum, “Collaborative discourse, argumentation, and learning: Preface and literature review,” Contemp. Educ. Psychol., vol. 33, no. 3, pp. 345–359, 2008, doi: 10.1016/j.cedpsych.2008.06.001.
A. J. Neville, “Problem-Based Learning and Medical Education Forty Years On,” Med. Princ. Pract., vol. 18, no. 1, pp. 1–9, 2008, doi: 10.1159/000163038.
B. Slof, G. Erkens, P. A. Kirschner, J. G. M. Jaspers, and J. Janssen, “Guiding students’ online complex learning-task behavior through representational scripting,” Comput. Human Behav., vol. 26, no. 5, pp. 927–939, 2010, doi: 10.1016/j.chb.2010.02.007.
A. P. Rovai, “Sense of community, perceived cognitive learning, and persistence in asynchronous learning networks,” Internet High. Educ., vol. 5, no. 4, pp. 319–332, Jan. 2002, doi: 10.1016/S1096-7516(02)00130-6.
M. Saqr and A. Alamro, “The role of social network analysis as a learning analytics tool in online problem based learning,” BMC Med. Educ., vol. 19, no. 1, p. 160, Dec. 2019, doi: 10.1186/s12909-019-1599-6.
A. Joshi, P. Desai, and P. Tewari, “Learning Analytics framework for measuring students’
performance and teachers’ involvement through problem based learning in engineering
education.,” Procedia Comput. Sci., vol. 172, pp. 954–959, 2020, doi: 10.1016/j.pro
M. Saqr and J. Nouri, “High resolution temporal network analysis to understand and improve
collaborative learning,” in 10th International Conference on Learning Analyti
M. Saqr and S. López-Pernas, “Modelling diffusion in computer-supported collaborative learning: a large scale learning analytics study,” Int. J. Comput. Collab. Learn., vol. 16, no. 4, pp. 441–483, Dec. 2021, doi: 10.1007/s11412-021-09356-4.
M. L. McHugh, “Interrater reliability: The kappa statistic,” Biochem. Medica, vol. 22, no. 3, pp. 276–282, 2012, doi: 10.11613/bm.2012.031.
R. Gatta, J. Lenkowicz, M. Vallati, and A. Stefanini, “pMineR ver. 0.31 : Pro
W. Peeters, M. Saqr, and O. Viberg, “Applying learning analytics to map students’
selfregulated learning tactics in an academi
A. Gabadinho, M. Studer, N. Müller, R. Bürgin, P.-A. Fonta, and R. Gilbert, “TraMineR: Trajectory Miner: a Toolbox for Exploring and Rendering Sequences,” 2022. https://cran.rproject.org/web/packages/TraMineR/index.html.
J. van Aalst, “Distinguishing knowledge-sharing, knowledge-construction, and knowledgecreation discourses,” Int. J. Comput. Collab. Learn., vol. 4, no. 3, pp. 259–287, Sep. 2009, doi: 10.1007/s11412-009-9069-5.
M. Saqr, O. Viberg, and H. Vartiainen, “Capturing the participation and social dimensions of computer-supported collaborative learning through social network analysis: which method