Exploratory Process Analysis of Teacher Learning of AI Integration through Collaborative Design

Abstract

Little research has been done on the study of computersupported collaborative learning (CSCL) in the context of teacher learning, especially the temporal analysis of the knowledge construction process and its impact on learning outcomes. The purpose of our research is to explore multiple temporal analysis methods to understand the knowledge construction in K-12 teacher CSCL of ML-empowered lesson plan design using the video transcript data. The social network analysis yielded high and low meta-cognition across groups and indicated the association with the design artefact quality. Sankey diagram visualization demonstrated the macro-level cognition activity flow in the process. Lag sequential analysis found patterns of transition of technological, pedagogical, and content knowledge during collaborative design contributing to the group learning outcome. A discussion on the results is provided, which sheds light on analyzing and facilitating teacher learning in CSCL settings.

Keywords

• eol>Computer-supported collaborative learning (CSCL)
• temporal analysis
• teacher learning
• knowledge construction

1. INTRODUCTION

Artificial intelligence (AI) plays an increasingly critical role in K-12 education as technology advancement [ 19 ]. It imposes a new requirement for K-12 teachers with limited computing backgrounds to develop an understanding of teaching with AI technologies in classrooms [ 13, 32 ]. Recent research eo↵rts started initiating professional development programs to prepare teachers with sucient knowledge about utilizing AI to support student learning with subject matters [ 47, 52, 58 ]. These studies, however, provide little information about how teachers engaged in sense-making activities of AI technologies, which are essential in guiding the design of teacher ∗These three authors contributed equally.

Copyright ©2021 for this paper by its authors. Use permitted under Creative Commons License Attribution 4.0 International (CC BY 4.0) education programs for AI integration.

Our study attempts to address this inquiry by investigating teachers’ learning process using existing data collected from a professional development program, ML4STEM, in April 2020 [ 59 ]. It introduced an ML-enhanced scientific discovery learning environment to 18 in-service K-12 teachers and engaged them in several learning activities to learn to teach with a new tool. This paper specifically focuses on the collaborative design activity in a computer-supported collaborative learning context (CSCL). Teachers created MLenhanced lesson plans (Fig. 7) facilitated by a web-based learning environment, SmileyDiscovery (Fig. 6), enabling novice learners to apply k-means clustering in science context to discover patterns and new knowledge [ 59 ]. Collaborative design has been argued as the most e↵ective method to support teachers’ understanding of technology integration in classrooms [ 24 ]. However, the desired learning outcomes are not naturally guaranteed [ 26 ]. An e↵ective CSCL learning process depends on constructing new knowledge and generating new understandings during the collaboration process [ 10, 15 ]. Particularly, d [ 21 ]. To understand the quality of such knowledge construction process, a key to di↵erentiating the quality of cognitive activities at high and low levels becomes necessary, as we expected the desired high level of knowledge construction during the collaboration [ 43 ]. Previous research related to CSCL lacks such e↵orts in studying the knowledge construction process in the teacher learning context [ 26 ] or limited to descriptive analysis to reveal the factors within CSCL that contribute to teacher learning [ 18, 34, 4, 27 ].

To uncover patterns of teachers’ collaborative learning and investigate the knowledge construction process demonstrated by the quality of cognitive processing, we explored multiple methods to analyze the temporal data at both individual and group levels. Social network analysis is to uncover both the interactions between participants and with cognitive activities to identify group collaboration patterns and suggest collaboration strategies based on groups’ end-product of learning [ 21 ]. Lag sequential analysis (LSA) is to investigate how knowledge is transiting between groups during collaborative design. Sankey diagram visualizes macro-level cognitive activity flow through the collaborative design process. The results provide insights into how die↵rent collaboration patterns across teams ae↵ct learning and how knowledge constructs across groups. The implications of such findings are discussed at the end of this paper.

2. RELATED WORK

2.1 Artificial Intelligence in K-12 Teaching

AI technologies have become increasingly crucial in education by playing four roles: intelligent tutor, intelligent tutee, intelligent learning tool & partners, and the policy-making advisor [ 19 ]. Implementing them in classrooms, however, is a challenge for K-12 teachers. One of the primary obstacles is that guiding students to learn with AI tools require teachers to understand relevant technological knowledge [ 19 ]. It is another challenge to prepare K-12 teachers for teaching with AI in classrooms due to their limited computing backgrounds [ 13 ], and the lack of teaching materials [ 32 ]. To provide insights on potential solutions to the aforementioned challenges, we analyzed the learning process showing how K-12 STEM teachers learned collaboratively to design MLempowered lesson plans.

2.2 Collaborative Design & Teacher Learning

Collaborative design is viewed as a form of professional development [ 2, 54 ] and has been advocated as a desirable way for sustaining teachers to implement innovative practices enhanced by advanced technologies [ 2, 16, 34 ]. It is an activity in which teachers and technology designers work together to create teaching materials that comply with the function of technologies, and the realities of teaching contexts [ 54 ]. It argues that active engagement, as well as the shared process of collaborative design, o↵ers ample opportunities for teachers to reflect on and deepen their understanding of the usage of the new technology in classroom teaching [ 54 ]. The model of technological, pedagogical, and content knowledge (TPACK) [ 24 ] is frequently applied in this research field to describe what knowledge that teachers should develop for technology integration. It consists of seven dimensions: technology knowledge (TK), pedagogy knowledge (PK), content knowledge (CK), technology pedagogy knowledge (TPK), technology content knowledge (TCK), pedagogical content knowledge (PCK), and technological pedagogical content knowledge (TPCK). Previous research has identified two kinds of support necessary for developing teachers’ TPACK in collaborative design activities. One is expert support, which means the design teams should involve participants who are knowledgeable in the area of content, pedagogy, and technology on the materials that are being developed [ 18 ]. The other is process support, referring to monitoring the design process for ensuring the design intention is achieved [ 4, 27 ]. These studies employ a descriptive analysis method, while our research examines the learning process using statistics and visualization techniques.

2.3 Learning Process Analysis in CSCL

Understanding the temporal aspect of learning is essential as learning, by nature, is a process that occurs over time [ 23, 41 ]. In the context of CSCL, two reasons stand out to study the temporal data of the collaboration process: 1) CSCL is a complex social process, including characteristics of multiple actors (e.g., learners, technology, etc.) between events over time [ 26, 8 ]; 2) collaboration has a great potential to provide a learning environment with the shared learning process and shared learning activities for knowledge construction [ 20 ]. However, such a shared process does not necessarily lead to productive knowledge outcomes. A Previous study showed the interrelations between cognitive events under knowledge building discourse and uncovered the sequential pattern of events using frequent pattern mining and latent sequential analysis [ 7 ]. In addition, another work studied how lowperforming and high-performing groups progress through a framework of socially shared regulation of learning and argue the importance of recognizing challenges and strategies in group collaboration [ 31 ]. These two studies provided insights for uncovering the patterns of knowledge construction in the teacher learning context.

Various methods have been used to study the process of CSCL, mainly in inferential statistics and the coding-andcount approach [ 26, 26 ], including social network analysis analyzing the group interactions over time (e.g., [ 51, 50, 28 ]), sequential analysis studying the learning event patterns (e.g, [ 9, 57 ]), and di↵erent types of visualizations studying online discussions (e.g., [ 12, 25 ]). Social network analysis (SNA) served as a primary research method for studying group interactions, characteristics of relations, and influence of these relations in online teaching and learning [ 40, 46, 36 ]. For a CSCL process, the participants’ presence, roles, and their interactions with other participants in the network are critical factors that influence the collaboration process [ 35 ] and lead to di↵erent levels of learning performance [ 11 ] or knowledge construction [ 1 ].

3. METHODOLOGY

3.1 Research Questions

The following three questions guide our analysis of the learning process: RQ1 What are the interaction patterns of group participants (teacher-to-teacher)and knowledge construction (teacher to cognitive activities) during teachers’ collaborative design? RQ2 What are the sequential patterns of cognitive activities during the individual learning journey? RQ3 What are the sequences of knowledge construction concerning discussion contents at the group level during teachers’ collaborative design?

3.2 Research Context

The data was collected from a two-week teacher learning program conducted in April 2020. The program aims to equip teachers with sucient knowledge about teaching with an ML-enhanced scientific discovery learning environment, SmileyDiscovery, designing for supporting STEM teaching and learning in K-12 contexts [ 59 ]. In this study, we mainly focused on the second session - teacher-as-designer in which teachers worked collaboratively to design ML-enhanced lesson plans by using SmileyDiscovery components (Fig. 6). Eighteen teachers were divided into four groups (noted as group A, group B, group C, and group D) based on their teaching grades and subjects. Each group included a participant (for example, a1) volunteering for a mediator and a researcher(for example r1) playing as a facilitator (Table 1). Due to the COVID-19 lockdown, teachers communicated with each other via ZOOM and created the lesson plans on design canvas supported by an online collaborative platform Lucidchart. The design canvas contains draggable cards representing di↵erent SmileyDiscovery system components (Fig. 6) for teachers to select for specific instructional steps in their lesson plan. The collaborative design activity consists of four phases: Deciding topic (10min)- teachers select a subject matter to work on; Discussing learning objectives (10min)- teachers identify the targeted grade levels of students, questions of their interests, and other materials required to fulfill the learning materials; Developing learning activities (25min)- teachers determine the pedagogical steps according to the 5E instructional model [ 5 ], then design the related instructional activity in this step, and select the appropriate SmileyDiscovery features that could support implementations of each instructional activity; Reflecting the design (20min)- teachers critically reflect on the current lesson plan design and propose the desired improvement on a specific aspect of SmileyDiscovery.

The end product of teachers’ collaboration design are the designed lesson plans that include specific instructional steps (e.g., Fig. 7) listed along with corresponding SmileyDiscovery components (Fig. 6). We assessed the quality of lesson plans as the group learning outcome using an empirically validated framework [ 17 ] (see Table 5). It was created for measuring the quality of technology-enhanced teaching materials built from the TPACK model [ 24 ]. Two researchers independently evaluated the lesson plans, achieving a nearperfect agreement (Cohen’s kappa = 0.92).

3.3 Data and Analytical Approach

We collected recordings of four groups’ collaborative design and transcribed the verbal data for analysis. The raw transcripts contain 1869 turns. The social talk and incomplete talk were dropped o↵ as they are less relevant with knowledge construction, ended with 1765 turns in total (Group A = 504, Group B = 328, Group C = 478, Group D = 455).

3.3.1 RQ1: Social Network Analysis

A social network analyzes the patterns of connections (represented as ties or edges with strengths and directions) among entities (individual, groups, events, etc.), represented by nodes with sizes, and relations between entities [ 40, 46 ]. This research question investigates participants’ positions and their interactions in groups and their engagement in die↵rent cognitive activities involved in the knowledge construction process. Thus, we utilized SNA to visualize the relations and participants’ roles in the network and quantified the relations using both the node-level measures and network measures with the Igraph library in the R programming language. Conceptually, a social network can be structured as a one-mode network [ 30 ] and a two-mode network with mode referred to the set of nodes [ 46 ]. One-mode analysis is used to study the relations of people (e.g., interactions between teachers and participant’s positions and roles). Two-mode analysis is used to analyze networks that involve participants and events (e.g., teacher’s participation frequency engaged in the knowledge construction process). Before running the analysis, we segmented the transcripts to the turn level. First, two researchers reviewed each group’s transcript independently to code the source and target of each turn of speech, reaching the agreement (Cohen’s Kappa = 0.97). Second, we coded the cognitive activity for each turn using an adapted version of the meta-cognitive regulation coding scheme [ 21 ] (See Table 7, 8, 9). The original coding framework (see [ 21 ]) is developed by [ 53 ] to analyze group knowledge construction behavior and validated in [ 48 ]. We extended it with response tokens (e.g. right, yeah, Uh huh, and hmm), showing that a talk sent by a speaker has been received by the audience). These response tokens are important for analyzing discussions since they serve to forward the course of a conversation [ 33 ]. Four low-level codes, thus, were generated after we conducted an open coding for the transcripts: follow-up response (FU), show uncertainty (SU), show hearing (SH), agree with peers (AP). Two researchers independently coded all the transcripts, reaching a near-perfect agreement (Cohen’s kappa = 0.95). For the one-mode analysis, we structured two data files recording 1) an edge list (all source-target directions and each tie weight) and 2) a node list (all participant id and their roles) of the network. The weight of each turn is assigned according to the level of cognitive activity: high-level (value = 2) and low-level (value = 1). A two-dimensional co-concurrence matrix was constructed for the two-mode analysis, calculating each participant’s participation frequency engaged in each type of cognitive activity. The measures of the social network analysis are shown in the Appendix (Table 3).

3.3.2 RQ2: Sankey Diagram

A Sankey diagram is a visualization tool that illustrates quantitative information of the activity flow of individual participants by using directed, and weighted graphs [ 42 ]. Thus, we applied it to discern the patterns of cognitive activity flow each participant engaged in across die↵rent phases of the collaborative design. Moreover, we can explore the sequential patterns of teachers’ engagement and role-switching in di↵erent types of cognitive activities. To simplify the visualization, we grouped all cognitive activities into six categories based on the purposes of learning: plan the next step (pl, ph), evaluate the design purpose (el, eh), enhance the group’s conceptual understanding (vm, ei, jd, rm, sm, qm), seek or provide basic information (si, ai), follow up without creating much new information (sh, su, ci, fu, ap), and conclude an episode of discussion (cd, sd). And the x-axis represents the sequence of a participant’s cognitive behaviors (e.g., one node with x = 7 represents the 7th cognitive activity a participant conducted).

3.3.3 RQ3: Lag Sequential Analysis

Lag sequential analysis (LSA) is an analytical approach used for determining if a statistically significant dependence exists between sequential events [ 3 ]. Many researchers have adopted it to understanding the sequential patterns of participants’ behaviors in learning activities and what the desired patterns would be for learning [ 55, 56 ]. We applied it to explore the sequential patterns of knowledge construction occurring in the discussion contents that di↵erent groups engaged in collaborative design activities.

We first chunked the transcripts into segments, whereby each segment corresponded to a unique topic of conversation related to the design contents. For example, teachers were required to identify the learning objectives of the design lesson plan. A conversation around it, from the initiation to the end, is considered a topic. Second, we adapted the TPACK model [ 24 ] to code the knowledge dimensions shown by the specific speech of a turn (Table 6). Two researchers independently coded the TPACK and reached an almost perfect agreement, Cohen’s kappa = 0.95. Third, since we are interested in understanding the sequential pattern of die↵rent knowledge dimensions for each topic of conversation, the duplicated codes were dropped o↵ for each segment. For example, if TK occurs several times in one segment, we only counted it occurred once.

The LSA is performed for each group using the program Generalized Sequential Query (GSEQ) [ 3 ]. First, we run the Pearson chi-square test to check if a significant dependence exists between knowledge dimensions. Then, we used the program to calculate the adjusted residual between any two knowledge dimensions.

4. RESULTS

4.1 RQ1.1 Teacher-teacher interaction

The sociogram of the teacher-teacher interaction (see Fig. 1) showed di↵erent roles of participants (researcher, mediator, teacher) and their levels of contributions to the discussion, demonstrated by the position and size of nodes in the network. First, for the researcher position, r1(group A) and r4 (group D) had a higher degree of centrality (especially out-degree centrality) than r2 (group B) and r3 (group C), with r2 holding the least out-degree centrality, evident by the node-level measures (see Appendix Table. 4). This illustrated that r1 and r4 played more proactive roles in facilitating the discussion and o↵ering guidance, whereas r2 intervened less and relied more on the mediator b4 to facilitate the discussion. Second, for the mediator position, d3 (out-degree centrality = 500) and c4 (out-degree centrality = 469) played dominant roles in the group discussions, taking responsibility for note-taking and guiding the discussion than a1 (out-degree centrality = 215) and b3(out-degree centrality = 260). Third, compared the out-degree centrality for teacher participants and visual positions in the network, b1, c5, and d4 are relatively peripheral in contributing to the group discussions. The reason might be due to the teacher’s insucient technology knowledge about SmileyDiscovery. As to the closeness measure, we observed the highest closeness of participants in group A, demonstrating their relatively equal participation and greater mutuality in the discussions.

Comparing the one-mode network attribute (see Fig. 2), we found all groups shared a relatively high density value, between 30-40%. This indicated highly active and cohesive participation in the group discussion across four groups, with no isolated participants. On average, group C (avg.degree = 360) and group D (avg.degree = 394) had a higher frequency of group interactions than group A (avg.degree = 320) and group B (avg.degree = 268). However, the contribution among participants was rather equal for group A and group B, indicated by the smaller standard deviation(SD) of the degree centrality-12.40 and 13.50 respectively- compared to that of group C (SD = 26.44) and group D (SD = 21.05). Reciprocity refers to the balance of the network. The values of all groups are larger than 0.5, showing a relatively high mutual communication between participants. Compared the one-node network attributes with the scores of teacher-designed lesson plans, group C and group D with high avg degrees and high density and more proactive roles of researcher and mediator had better final sores. Although not tested statistically, the association might demonstrate a need for explicit facilitation and mediation training of researchers and mediators in the future.

4.2 RQ1.2 Teacher-cognitive activity interaction

The two-mode network attributes Table 2 showed the distribution of low and high-level cognitive activity frequency during teachers’ collaborative design. While group A had the highest number of cognitive activity events (N = 504), the ratio of engagement in the high-level cognitive activities (37.70% ) was smaller than that of Group C (44.98%) and group D (43.74%). Group B had the slightest participation in cognitive activities (N = 328) and the high-level cognitive activities (ratio = 37.20%). This indicates that the better quality of the lesson plans designed by group C and group D might result from the high-quality discussion they engaged in the knowledge construction process.

The two-mode sociogram (Fig. 2) shows interaction patterns of the participants with the cognitive activities, with the size of the participant node indicating the degree of participation. Group A had a relatively equal distribution of cognitive activities. Participants b1 and c5 had the relatively less engagement compared to the other participants in their groups. Group D was unequal as the mediator d3 undertook more cognitive activities than other participants.Furthermore, participants of group C shared a relatively higher frequency of the high-level cognitive activity; because the nodes of cognitive activities are near each other and surround all participant nodes. The researcher and mediator in group C both actively participated in high-level planning (PH) and explore ideas (EI). For group D, we observed a proactive role of mediator in facilitating discussions, demonstrated by d4’s engagement in the high-level cognitive activities such as QM- Questioning, RM- reflect meaning, and PH- high-level planning. Also, the mediator engaged in low-level cognitive activities for responding to the participants and transitioning the discussion, as SI- seek information, SD- stop discussion, and SU- show uncertainty. We argue that for future study, the researcher or mediator should be oe↵red a more explicit strategy to help facilitate the discussion and actively participate in the collaborative design to support participants engaging in high-level cognitive activities. 4.3

RQ2 Sequential patterns of cognitive activities

Preparation phase. Die↵rent from the rest phases, there is no enhance of conceptual understanding during this phase (Fig. 3). There are two main patterns identified in participants’ cognitive flows: (1) some participants mainly focused on seeking information or providing information; (2) some participants (e.g., researchers) illustrated how the next step should take place and evaluated the current progress. The phases for learning objective setup, learning activity development and reflection. During these three phases (Fig. 4), most participants switched frequently between the role to enhance the group’s conceptual understanding (‘enhance’), the role to seek or add basic information (‘info’), and the role to follow up with someone else (‘follow’). Fig. 4 showed that, during these three phases, a few participants generated much longer cognitive activity flows than other participants (long-tail flows). Those participants who contributed more to the discussions tended to be more constantly and frequently engaged in cognitive activities that enhance the group’s conceptual understanding (‘enhance’). This indicates that individual participant’s frequency of the cognitive activities might be positively related to one’s contribution to the group’s conceptual understanding.

4.4 RQ3 Sequential patterns of TPACK transition in knowledge construction

Fig. 5 illustrates the patterns of TPACK transition in knowledge construction of di↵erent groups. The curve indicates the sequence between two knowledge dimensions and the values are the adjusted residual of the sequential transition. Any sequential transition below 1.96 is dropped o↵ as it indicates a non-significance between two knowledge dimension. Also, the information of group B is not presented here because the overall pattern is not significant assessed by the chi-sqaure test (p = 0.97). By comparing the knowledge construction patterns of group A, group C, and group D with the quality of their design artifacts, we found the level of complexity of TPACK transition patterns in knowledge construction corresponds to the groups’ outcomes.

Group C (with the highest group outcome) shows the most complex patterns of knowledge transition, including six sets of transitions with six knowledge dimensions involved (TK– >TPK, CK–>TCK, CK–>PCK, TPK–>TK, PCK–>CK, PCK–>PK). Group D (the second-highest group outcome) consists of four sets of transitions with five knowledge dimensions involved (PCK–>PK, PK–>TPK, TCK–>CK, CK– >TCK). Compared with group C, group D lacks the transition between TK with any other knowledge dimensions. That means teachers in group D were more likely to discuss TK exclusively for a design component than connecting it to others. Group A (the third highest group outcomes) has a simpler pattern, containing four sets of knowledge construction with four knowledge dimensions (TK–>TPK, CK– >TCK, TPK–>TK, TCK–>CK). Compared with group C and group D, group A lacks PK and PCK as well as their transitions to other knowledge dimensions. Group B (the lowest group outcomes) does not show significant dependence between any two knowledge dimensions. According to the LSA results, the higher group outcomes, the more complex TPACK transition patterns displayed in the knowledge construction process. Nevertheless, given the small sample data, further studies are needed to validate this finding.

DISCUSSION

Effective group discussion and the role of participants.

Group C and group D outperformed group A and group B as for the discussion quality, and the result is potentially associated with the previously-graded design artefacts scores, with lesson plan scores (group C > group D > group A > group B). The result, consistent with the previous literature, indicates that groups engaged in a large amount of high-level conceptual understanding, elaboration, and justification of content material were also associated with better overall conceptual understanding demonstrated in the end learning product [ 21 ]. We expected to see high group density, active participation, and high-level knowledge construction To promote a high-quality discussion [ 22, 53 ]. The mediator and the researcher might play a pivotal role in promoting the discussion quality. Group C and group D’s mediators (c4 and d3) played a dominant role in directing the discussions and taking notes for the whole group, demonstrated by their engagement in planning, evaluation, and direction while group A had a relatively equal engagement in the cognitive activities. Previous literature showed that assigning students with leadership roles (e.g., facilitators) could empower students to engage in the discussions [ 39 ], which echoed with the case in group C and group D, but how to empower and engage the peripheral members would be a future discussion. Also, discussion facilitation strategies make a significant di↵erence in the extent of collaboration [ 40, 49 ]. In our case, researchers who take an active role in facilitation and help address the technological and content knowledge gap of participants promoted better quality of discussion. Thus we need to explicitly train mediators and researchers to facilitate discussions.

Engage learners in more meaningful discussion. The

Sankey diagrams show that teachers who participated in the discussion constantly and frequently engaged in more activities enhancing the group’s conceptual understanding. One interpretation is that note-takers in each group who had to talk more throughout the design activity needed to take responsibility for the learning activity construction and reflection; in turn, they got involved in more cognitive activities that enhance the conceptual understanding. Another potential explanation is that participants who produced more dialogues of “enhance” had more opportunities to explore their ideas further. This suggests the facilitation is needed to prompt learners with fewer discourses or fewer “enhance” cognitive activities to share their ideas with the group. Knowledge transition in group discussions contributes to the TPACK development. The results of the lag sequential analysis suggest that the transitions between the subdimensions of TPACK in collaborative design might contribute to groups’ learning outcomes. This finding adds new evidence to the research of TPACK, showing that grasping the connections between TK, PK, and CK is significant for developing an integrative understanding of TPACK. Previous research has found the impacts of TK, PK, and CK on teachers’ TPACK by a regression model using pre-post assessments [ 6 ]. Few studies, however, have examined it taking a process perspective. Given the importance of collaborative design in developing TPACK [ 24 ], our research suggests the design contents are better to be addressed by discussing the knowledge dimensions and the related sub-dimensions. For example, when teachers are engaged in talking about TPK for a while, the facilitator can intervene and guide teachers to discuss TK or/and PK related to the TPK. Such a process provides teachers with opportunities to understand each dimension of TPACK and its connections. Our further step is to conduct a qualitative analysis of the transcript data, primarily how knowledge transition occurred in some interaction units but not others. The findings generated can o↵er more insights on how to facilitate teachers’ collaborative design activities.

APPENDIX

Number and ratio of low-level cognitive activities in the network.

Number and ratio of high-level cognitive activities in the network.

Two-mode Network

Roles Reflect on meaning Follow-up response Confirm information Show uncertainty Show hearing Seek meaning Volunteer meaning Conclude from discussions Codes Evaluate without justification Evaluate with justification

References

[1] R. Aviv , Z. Erlich , G. Ravid, and A. Geva . Network analysis of knowledge construction in asynchronous learning networks . Journal of Asynchronous Learning Networks , 7 ( 3 ): 1 - 23 , 2003 .

[2] M. A. Bakah , J. M. Voogt , and J. M. Pieters . Updating polytechnic teachers' knowledge and skills through teacher design teams in ghana . Professional development in education , 38 ( 1 ): 7 - 24 , 2012 .

[3] R. Bakeman and V. Quera . Sequential analysis and observational methods for the behavioral sciences . Cambridge University Press, 2011 .

[4] H. Becuwe , J. Tondeur , N. Pareja Roblin , J. Thys , and E. Castelein . Teacher design teams as a strategy for professional development: The role of the facilitator . Educational Research and Evaluation , 22 ( 3-4 ): 141 - 154 , 2016 .

[5] R. W. Bybee , J. A. Taylor , A. Gardner, P. Van Scotter , J. C. Powell , A. Westbrook , and N. Landes . The BSCS 5E instructional model: Origins and e↵ectiveness . BSCS , 5 , 88 - 98 , Colorado Springs , Co, 2006 .

[6] C. S. Chai , J. H. L. Koh , and C.-C. Tsai . Facilitating preservice teachers' development of technological, pedagogical, and content knowledge (tpack) . Journal of Educational Technology & Society , 13 ( 4 ): 63 - 73 , 2010 .

[7] B. Chen , M. Resendes , C. S. Chai , and H.- Y. Hong . Two tales of time: uncovering the significance of sequential patterns among contribution types in knowledge-building discourse . Interactive Learning Environments , 25 ( 2 ): 162 - 175 , 2017 .

[8] B. Chen , A. F. Wise , S. Knight , and B. H. Cheng . Putting temporal analytics into practice: the 5th international workshop on temporality in learning data . In Proceedings of the sixth international conference on learning analytics & knowledge , pages 488 - 489 , 2016 .

[9] T. H. Chiang , S. J. Yang , and G.-J. Hwang . Students' online interactive patterns in augmented reality-based inquiry activities . Computers & Education , 78 : 97 - 108 , 2014 .

[10] B. Collis and J. Moonen . Contribution-oriented pedagogy . In Encyclopedia of Distance Learning, Second Edition , pages 439 - 446 . IGI Global, 2009 .

[11] M. S. Despotovi´ c-Zraki´c, A. B. Labus , and A. R. Mili ´c. Fostering enginering e-learning courses with social network services . In 2011 19thTelecommunications Forum (TELFOR) Proceedings of Papers , pages 122 - 125 . IEEE, 2011 .

[12] N. Ding , J. Wei , and M. Wolfensberger . Using epistemic synchronization index (esi) to measure students' knowledge elaboration process in cscl . Computers & education , 80 : 122 - 131 , 2015 .

[13] I. Evangelista , G. Blesio, and E. Benatti . Why are we not teaching machine learning at high school? a proposal . In 2018 World Engineering Education Forum-Global Engineering Deans Council (WEEF-GEDC) , pages 1 - 6 . IEEE, 2018 .

[14] L. C. Freeman . Centrality in social networks conceptual clarification . Social networks , 1 ( 3 ): 215 - 239 , 1978 .

[15] R. Ha ¨ma¨l¨ainen and K. Va¨h¨asantanen. Theoretical and pedagogical perspectives on orchestrating creativity and collaborative learning . Educational Research Review , 6 ( 3 ): 169 - 184 , 2011 .

[16] A. Handelzalts . Collaborative curriculum development in teacher design teams. In Collaborative curriculum design for sustainable innovation and teacher learning , pages 159 - 173 . Springer, Cham, 2019 .

[17] J. Harris , N. Grandgenett , and M. Hofer . Testing a tpack-based technology integration assessment rubric . In Society for Information Technology and Teacher Education International Conference , pages 3833 - 3840 . Association for the Advancement of Computing in Education (AACE ), 2010 .

[18] T. Huizinga , A. Handelzalts , N. Nieveen , and J. M. Voogt . Teacher involvement in curriculum design: Need for support to enhance teachers' design expertise . Journal of curriculum studies , 46 ( 1 ): 33 - 57 , 2014 .

[19] G.-J. Hwang , H. Xie , B. W. Wah , and D. Gaˇsevi´c . Vision, challenges, roles and research issues of artificial intelligence in education , 2020 .

[20] A. Khanlari , M. Resendes , G. Zhu, and M. Scardamalia . Productive knowledge building discourse through student-generated questions . Philadelphia, PA: International Society of the Learning Sciences. , 2017 .

[21] D. K. Khosa and S. E. Volet . Productive group engagement in cognitive activity and metacognitive regulation during collaborative learning: Can it explain die↵rences in students' conceptual understanding? Metacognition and Learning , 9 ( 3 ): 287 - 307 , 2014 .

[22] A. King . Structuring peer interaction to promote high-level cognitive processing . Theory into practice , 41 ( 1 ): 33 - 39 , 2002 .

[23] S. Knight , A. F. Wise , and B. Chen . Time for change: Why learning analytics needs temporal analysis . Journal of Learning Analytics , 4 ( 3 ): 7 - 17 , 2017 .

[24] M. Koehler and P. Mishra . What is technological pedagogical content knowledge (tpack)? Contemporary issues in technology and teacher education, 9(1 ): 60 - 70 , 2009 .

[25] J. La ¨msa¨, R. Ha¨ma¨l¨ainen, M. Aro, R. Koskimaa , and S. -M. A ¨ yra¨mo¨. Games for enhancing basic reading and maths skills: A systematic review of educational game design in supporting learning by people with learning disabilities . British Journal of Educational Technology , 49 ( 4 ): 596 - 607 , 2018 .

[26] J. La ¨msa¨, R. Ha¨ma¨l¨ainen, P. Koskinen, J. Viiri , and E. Lampi . What do we do when we analyse the temporal aspects of computer-supported collaborative learning? a systematic literature review . Educational Research Review, page 100387 , 2021 .

[27] T. H. Levine and A. S. Marcus . How the structure and focus of teachers' collaborative activities facilitate and constrain teacher learning . Teaching and teacher education , 26 ( 3 ): 389 - 398 , 2010 .

[28] J.-W. Lin , L.-J. Mai , and Y.-C. Lai . Peer interaction and social network analysis of online communities with the support of awareness of di↵erent contexts . International Journal of Computer-Supported Collaborative Learning , 10 ( 2 ): 139 - 159 , 2015 .

[29] Z. Liu , L. Kang , M. Domanska , S. Liu, J. Sun , and C. Fang . Social network characteristics of learners in a course forum and their relationship to learning outcomes . In CSEDU (1) , pages 15 - 21 , 2018 .

[30] D. A. Luke . A user's guide to network analysis in R. Springer, 2015 .

[31] J. Malmberg , S. Ja¨rvela¨, H. Ja¨rvenoja, and E. Panadero . Promoting socially shared regulation of learning in cscl: Progress of socially shared regulation among high-and low-performing groups . Computers in Human Behavior , 52 : 562 - 572 , 2015 .

[32] L. S. Marques , C. Gresse von Wangenheim, and J. C. HAUCK. Teaching machine learning in school: A systematic mapping of the state of the art . Informatics in Education , 19 ( 2 ): 283 - 321 , 2020 .

[33] M. McCarthy . Talking back:” small” interactional response tokens in everyday conversation . Research on language and social interaction , 36 ( 1 ): 33 - 63 , 2003 .

[34] S. McKenney , F. Boschman , J. Pieters , and J. Voogt . Collaborative design of technology-enhanced learning: what can we learn from teacher talk ? TechTrends , 60 ( 4 ): 385 - 391 , 2016 .

[35] C. S. Ng , W. S. Cheung , and K. F. Hew . Interaction in asynchronous discussion forums: peer facilitation techniques . Journal of Computer Assisted Learning , 28 ( 3 ): 280 - 294 , 2012 .

[36] M. Oliveira and J. Gama . An overview of social network analysis . Wiley Interdisciplinary Reviews: Data Mining and Knowledge Discovery , 2 ( 2 ): 99 - 115 , 2012 .

[37] T. Opsahl , F. Agneessens , and J. Skvoretz . Node centrality in weighted networks: Generalizing degree and shortest paths . Social networks , 32 ( 3 ): 245 - 251 , 2010 .

[38] F. Ouyang . Using three social network analysis approaches to understand computer-supported collaborative learning . Journal of Educational Computing Research , page 0735633121996477 , 2021 .

[39] F. Ouyang and Y.-H. Chang . The relationships between social participatory roles and cognitive engagement levels in online discussions . British Journal of Educational Technology , 50 ( 3 ): 1396 - 1414 , 2019 .

[40] F. Ouyang and C. Scharber . The influences of an experienced instructor's discussion design and facilitation on an online learning community development: A social network analysis study . The Internet and Higher Education , 35 : 34 - 47 , 2017 .

[41] P. Reimann . Time is precious: Variable-and event-centred approaches to process analysis in cscl research . International Journal of Computer-Supported Collaborative Learning , 4 ( 3 ): 239 - 257 , 2009 .

[42] P. Riehmann , M. Hanfler , and B. Froehlich . Interactive sankey diagrams . In IEEE Symposium on Information Visualization , 2005 . INFOVIS 2005 ., pages 233 - 240 . IEEE, 2005 .

[43] T. K. Rogat and L. Linnenbrink-Garcia . Socially shared regulation in collaborative groups: An analysis of the interplay between quality of social regulation and group processes . Cognition and Instruction , 29 ( 4 ): 375 - 415 , 2011 .

[44] M. Saqr and A. Alamro . The role of social network analysis as a learning analytics tool in online problem based learning . BMC medical education , 19 ( 1 ): 1 - 11 , 2019 .

[45] M. Saqr , O. Viberg , and H. Vartiainen . Capturing the participation and social dimensions of computer-supported collaborative learning through social network analysis: which method and measures matter? International Journal of Computer-Supported Collaborative Learning , 15 ( 2 ): 227 - 248 , 2020 .

[46] J. Scott and P. J. Carrington . The SAGE handbook of social network analysis . SAGE publications , 2011 .

[47] F. Sullivan , E. Su´arez, E. Pektas, and L. Duan . Developing pedagogical practices that support disciplinary practices when integrating computer science into elementary school curriculum . 2020 .

[48] M. Summers and S. Volet . Group work does not necessarily equal collaborative learning: evidence from observations and self-reports . European Journal of Psychology of Education , 25 ( 4 ): 473 - 492 , 2010 .

[49] J. Thormann and P. Fidalgo . Guidelines for online course moderation and community building from a student's perspective . Journal of Online Learning & Teaching , 10 ( 3 ): 374 - 388 , 2014 .

[50] R. Tirado , A ´ . Hernando, and J. I. Aguaded. The e↵ect of centralization and cohesion on the social construction of knowledge in discussion forums . Interactive Learning Environments , 23 ( 3 ): 293 - 316 , 2015 .

[51] A. Traxler , A. Gavrin , and R. Lindell . Networks identify productive forum discussions . Physical Review Physics Education Research , 14 ( 2 ): 020107 , 2018 .

[52] A. Vazhayil , R. Shetty , R. R. Bhavani , and N. Akshay . Focusing on teacher education to introduce ai in schools: Perspectives and illustrative findings . In 2019 IEEE Tenth International Conference on Technology for Education (T4E) , pages 71 - 77 . IEEE, 2019 .

[53] S. Volet , M. Summers , and J. Thurman . High-level co-regulation in collaborative learning: How does it emerge and how is it sustained? Learning and Instruction, 19 ( 2 ): 128 - 143 , 2009 .

[54] J. Voogt , T. Laferriere , A. Breuleux , R. C. Itow , D. T. Hickey , and S. McKenney . Collaborative design as a form of professional development . Instructional science , 43 ( 2 ): 259 - 282 , 2015 .

[55] S.-Y. Wu and H.-T. Hou . How cognitive styles a↵ect the learning behaviors of online problem-solving based discussion activity: A lag sequential analysis . Journal of educational computing research , 52 ( 2 ): 277 - 298 , 2015 .

[56] T.-C. Yang , S. Y. Chen , and G.-J. Hwang . The influences of a two-tier test strategy on student learning: A lag sequential analysis approach . Computers & education , 82 : 366 - 377 , 2015 .

[57] S. Zhang , Q. Liu , W. Chen , Q. Wang , and Z. Huang . Interactive networks and social knowledge construction behavioral patterns in primary school teachers' online collaborative learning activities . Computers & Education , 104 : 1 - 17 , 2017 .

[58] Y. Zhang , J. Wang , F. Bolduc , W. G. Murray , and W. Sta ↵en. A preliminary report of integrating science and computing teaching using logic programming . In Proceedings of the AAAI Conference on Artificial Intelligence , volume 33 , pages 9737 - 9744 , 2019 .

[59] X. Zhou , J. Tang , M. Daley , S. Ahmad , and Z. Bai . “now, i want to teach it for real!”: Introducing machine learning as a scientific discovery tool for k-12 Table 9: Coding scheme for cognitive activities during the group regulation of evaluation. Abbr. Level Description EL Low EH High