The temporal component has been pointed out as an impactful dimension for educational research, as learning is not a static phenomena. In this paper, we investigate how the students' activity and interaction dynamics in an online forum difer among teaching modalities. The analysis is based on three years of forum interaction data in an undergraduate course where the only significant change was the teaching modality, from on-site (2019) to fully online (2021). We build and analyse temporal networks by studying changes in several networks' measures and features. Our preliminary results show changes in the students' and teachers' interaction dynamics as well as changes in the posts' content with fully online teaching. This on-going research is focused on studying the impact of teaching modalities on the discussion forum interactions. Initial results are promising, and other features such as the students' closeness and betweenness, as well as the dynamics changes' relationship with the academic performance are being explored to improve our understanding on teaching modalities and help generalisation of Network Science and Learning Analytics research in wider educational contexts.
The education provision has over the course of the last two years, since spring 2020, adapted to
major changes and accelerated technology adoption. The impact of digitally enhanced learning
and teaching, for instance, blended or online formats, has called attention to revision of teaching
methods and modalities [
Network analysis is one of the research approaches in Learning Analytics (LA) for analysis
and modelling of relational data in educational settings [
This on-going research focuses on exploring and describing the diferences in the usage,
content and interaction dynamics in a discussion forum platform commonly used in undergraduate
courses while taking into account diferent teaching modalities. We aim to answer the research
question: To what extent does the change in teaching modality impact the usage and interaction
patterns in discussion forum platforms? To answer the question, three years of forum data
were gathered and analysed to outline the changes in interaction dynamics when the modality
of teaching shifted from on-site in 2019, transitioning to emergency remote teaching [
The rest of the paper is organised as follows. In Section 2, we present related research on Social Network Analysis, discussion forums, and temporal networks. In Section 3, we present the data and the method followed in the study. Finally, in Section 4 we present the preliminary results obtained, and in Section 5 the discussion and next steps are outlined.
Networks are created based on existing relationships among two or more entities, a social
network refers to a collection of people sharing connections within an environment [
Temporal networks, (i.e., networks where the nodes’ connections change over time) are useful
for modelling the dynamic behaviour of many complex real-world systems because their
interactions are rarely constant over time [23, 24]. Considering the moment when the connections
happen when creating networks has an impact on several properties and measures, such as
connectedness, shortest paths, and centrality [25]. Evolution over time is an important dimension in
learning processes. Consequently, in the latest years, the attention paid to the temporality and
its efects on learning has increased; focusing on varied elements of educational settings [ 26, 27].
Saqr et al. [28] studied the role of temporal measures for predicting academic performance;
their findings underline that the temporal dimension provides essential information on learning
patterns and can potentially support instructors in addressing students’ performance. Xu et al.
[
3.1. Data This study encompasses the discussion forum interaction data of one undergraduate course from 2019 to 2021. The course is a first-year course for the computer science programme at Reykjavik University and a second year course for the engineering programme. The course is always taught in the spring term, with a high number of students enrolled each year: 215, 233 and 318 students respectively. The main characteristic of this case study is that during the three years included, most of the course’s features remained without changes. The course had the same teacher, syllabus, book, assessment structure, and an active online discussion forum available for the students’ use. The only component with significant changes was the teaching modality. In 2019, the teaching was delivered on-site at the university premises. In 2020 it started on-site, but due to the meeting restrictions imposed as response to the COVID-19 pandemic outbreak, the teaching modality was suddenly moved to online learning at the beginning of March. Finally, in 2021, the course was planned and delivered completely online.
The data were gathered from Piazza, an online discussion forum platform widely used among undergraduate courses to support and enhance student-student and student-instructor communication. In this course, the students were allowed to post questions, notes, and polls to the forum participants registered in the course section in Piazza. In addition, they were able to post answers, updates and follow-up questions to any public post previously published. The discussion forum was included into the course settings with the intention to: (1) encourage the students to collaborate with their classmates, and (2) provide the students with a direct communication channel with the instructors (teacher and teaching assistants). Neither their registration into the course section nor their participation in the threads were mandatory or included in the assessment structure. In addition, categorical grade placements from A to D were created based on the numerical grades obtained from the Learning Management System. Students with outstanding performance were allocated in A grade placement, whereas students who did not pass the course were allocated in D grade placement.
To answer our research question, the analysis performed includes two components. The first component, the social network analysis is included with the objective of studying the evolution of communication events along the term and compare it across years and teaching modalities. The second component, related to posts’ content features, complements the social networks analysis and is useful to get a better understanding of the diferences between teaching modalities. Table 1 displays the features included in the posts’ content analysis.
For the network analysis, the networks were created based on the edge list created using the forum threads archive, as described in Figure 1. Each node in the network represents a forum participant (teacher, teaching assistant, or student). The categorical grade placements were included in the network as an attribute of each student node.
As noted above, the interaction network in the discussion forum is dynamic, since not all participants registered are present since the beginning, and more edges are added as the course progresses. Representing dynamic interactions as networks can be addressed in several ways [31]. In this study, following the syllabus structure and the asynchronous dynamic on the online discussion forum; we decided to create twelve static networks per year. Each network represents one week of interactions between the participants (See Fig. 2). The networks were created based on the post’s timestamp to analyse their growth and their features’ evolution during the term; and compare them among teaching modalities to study their diferences. Table 2 displays the network measures that were calculated and analysed for all years and networks created. The data pre-processing and social network analysis were conducted in Python using the NetworkX package.
1. With fully online teaching, a higher proportion of students registered in the forum platform. 2. The percentage of active students increased from 41% and 52%, to 66% in 2021. 3. Active students had on average grades 35% higher than non-registered and observer students in online teaching, whereas with on-site teaching the increment was 5%, and 15% in the transition in 2020. Length of the first post in each thread Number of unique words within the post Whether the post was published anonymously or not Time in weeks until a student’s first post or contribution Time in hours until the first answer received to each post Follow-up questions in each thread
Number of answers provided by the teacher, teaching assistants, and students
Meaning in the discussion forum’s network context Number of students taking part in the forum threads by posting or answering threads.
Number of unique edges as an indicator of how active the participants are by contacting diferent participants each week.
Weight assigned to the edges as an indicator of how strong the connections are. The more communication events occur between each pair of participants, the higher the size will be.
Number of peers answering a student’s post as an indicator of the nodes’ posts popularity.
Number of peers the student answered to by posting an answer or follow-up question.
Number of posts relative to the number of possible students’ connections as an indicator of the network’s completeness.
Proportion of a nodes’ friends answering to each other, as an indicator of how tightly connected the students are.
Teacher’s influence on the information spreading among the course participants. 4. The networks of fully online teaching grew faster than with the other modalities, as a result of a higher percentage of students participating in the forum each week; the number of unique connections and contributions was also higher. 5. In 2021, with a higher average in-degree, out-degree, and clustering coeficient; the students got in contact with more people and were more connected with their close peers. 6. In fully online teaching, there was no diference in the week when the students started posting questions on the platform, in contrast to previous years, when D students started posting later in the term. 7. The students were more involved in the discussion threads, the number of follow-up questions posted increased in 2021.
In regards to the instructors’ activity, in 2021 with fully online teaching: (i) Number of nodes participating (ii) Unique edges (iii) Size (iv) Average in-out degree (v) Density (vi) Average clustering coeficient (vii) Teacher’s betweenness 1. The teacher’s betweenness indicates the influence on the spreading of information among the students was stronger, highlighting the importance of the teacher’s presence when the teaching modality was fully online (Fig. 3vii). 2. The number of questions increased by 75% and 51% compared to the questions posted in 2019 and 2020 respectively. 3. The instructors responded to a higher percentage of the questions. 4. The answers were provided faster, the median waiting time with fully online teaching was 56 minutes, whereas with on-site teaching the median time was calculated at 4.2 hours.
These results help to dimension the work overload experienced by teachers and teaching assistants since the beginning of the pandemic.
Regarding the comments’ structure and usage analysis: 1. The posts were longer and more complex than those posted in previous years. 2. The contributions posted anonymously decreased with fully online teaching. Possible explanations include; the students’ want to be identified by their instructors and peers due to the lack of direct contact; as the students never met in person, neither the teacher nor their peers, they could feel less exposed participating in the forum.
This paper presents the initial conclusions of an exploratory analysis performed with three years of discussion forum data. Previous research focusing on discussion forum interactions and their evolution usually relies on a specific course or teaching modality. In contrast, this paper combines the descriptive analysis of the networks’ structure with the posts’ features to compare the impact of the teaching modality choice on the usage of the forum, and over the participants’ interactions. This approach allowed us to notice that the teaching modality impacted not only the activity levels, but also the way the students’ and instructors’ connected with other participants, as well the comments features (e.g. length and anonymity status).
Initial insights from the descriptive analysis of the temporal networks created and their measures indicate that this approach is appropriate for researching the diferences in the interactions and usage over time between teaching modalities. However, one limitation of this research is the learning context of the discussion forum in this study; which might prevent our conclusions from being extended to courses with mandatory or graded forum participation.
The subsequent work of this study includes the exploration of temporal network measures for students and their diferences across teaching modalities. Furthermore, the relationship between the temporal network measures, their evolution, and the academic performance should be explored in more detail to develop models that account for teaching modality and help to identify students at risk of failing or drop out.
Both the initial and final results obtained from this research will contribute to the LA field by enhancing our understanding on the efect of teaching modality choices in higher education. In addition, this research will help to advance in the still limited research on temporal networks in educational contexts.