When students develop their academic activities in collaborative contexts mediated by technology, they manifest their behaviors through the interactions that are exchanged between the members of the different learning groups. The analysis of these interactions is one of the variables that teachers consider when measuring student performance. With the appearance of Covid-19 and the impossibility of developing face-to-face activities, a drastic and unexpected change took place in educational scenarios all over the world. Particularly in Argentina, the technology-mediated collaborative activities that students developed in combination with faceto-face classes were later carried out exclusively online. This study allowed to evaluate changes in the behavior and the performance of Computer Science students at a public university in Argentina by comparing the pandemic and pre-pandemic periods. The analysis of the results obtained, discriminated by gender, allowed to recognize a certain improvement in the behavior and academic performance of the students during the pandemic period.
In a Computer-Supported Collaborative Learning environment students are organized in groups to
work together in collaborative activities proposed by the teacher, and they use computer as a
communication, collaboration, and coordination channel [
During 2000 and 2021 the national government of Argentine Republic adopted several measures to protect people health against the spread of Covid-19 virus. In first place the government stablished a strict isolation (Social, Preventive, and Obligatory Isolation- SPOI), and secondly a careful distancing (Social, Preventive and Obligatory Distancing-SPOD). These measures affected severely the Argentine Educational System and produced a drastic change in educational scenarios. The usual face-to-face classes were replaced by fully, unexpected, and challenging virtual classes. This change showed that neither teachers nor students were prepared for virtual classes.
The aim of this work was to analyze conversations of students to discover if collaborative behavior had changed because of pandemic, and to find out the impact of behavior changes on academic performance. To achieve this aim, interactions of university students were collected and analyzed during two periods: 2018-2019 (pre-pandemic) and 2020-2021 (pandemic). Genre and grades of students during the two periods were considered. All participants were 4th and 5th year students of Bachelor’s in information systems at National University of Santiago del Estero in Argentina. This paper is organized as follows. In section 2 the step by step of this study is described. Then, the obtained results are discussed in section 3. In section 4, some related works are presented. Finally, some conclusions and further works are stated in section 5.
The 42 small learning group that participated on this study were constituted by students of the following subjects: Data Base II, Artificial Intelligence and Cryptography. Students were randomly assigned to the groups and each group had an average of 3 members. During pre-pandemic and pandemic periods students worked for two weeks on the same kind of activities and they communicate through the forum tool of COLLAB2 application to solve a problem allocated by the teacher. Prepandemic and pandemic problems had identical characteristics and difficult level. To complete activities students had to create a final product (report or graphic) and the final product was built through consensus of opinions posted on the forum that each group had in COLLAB. Figure 1 shows a snapshot of dialogue between members of a group of students that participated on this study.
As a result of collaborative activities between students, a total of 3.768 interactions were collected. 1.683 interactions took place during pre-pandemic and 2.085 during pandemic. Considering the gender of the author, interactions are distributed in the following way: 884 female interactions and 839 male 2 This tool was developed by researchers at the Instituto de Investigación en Informática y Sistemas de Información of the FCEyT. The tool is available at http://chat.fce.unse.edu.ar/chat/web/ interactions in pre-pandemic, and 1.023 female interactions and 1.062 male interactions in pandemic. This way there are 4 subsets of data to be analyzed.
Pre-processing tasks were applied over the 4 subsets of data to improve the data quality. The first
pre-processing task was stopwords removal, commonly used words (articles, prepositions, pronouns,
conjunctions, etcetera) that does not add much meaning to a sentence were removed. The text mining
software called GATE (developed by [
Once subsets of data were pre-processed, content analysis was applied over the content of 4
spreadsheets. The content analysis process followed the steps of the methodology suggested by
Krippendorff [
In this research reliability data were calculated under test-test conditions since two teachers,
experienced in e-learning and group dynamics, tagged each word of dataset (analysis unit) with the
conceptual content and collaborative skill that they considered the word was linked to. The term
conceptual content is understood as all word related to specific knowledge of each subject, for example
the word “Agent” is typical of Artificial Intelligence or the word “cryptosystem” that is related with
cryptography. Regarding to collaborative skills, the two analysts considered the following categories
proposed by Soller [
The two-analysts or experts worked independent of each other and applied the same categories to
the 4 subsets of words. The agreement level between analysts was very high as reliability index reached
the value of 0.99. Despite α=0.99 surpasses the 0.80 recommended by Krippendorff [
Figure 4 summarizes in a graphic the steps followed from the collection of interactions to their analysis.
Finally, 237 words out of 394 belonged to male students and 157 to female students. For male students 31% of the 100 words posted during pandemic were related to conceptual content and 69% related to collaborative skills. Regarding to pre-pandemic, the 137 words posted by male students were distributed on the following way: 27% conceptual content and 73% collaborative skills. Figure 5 shows the distribution of words during pre-pandemic and pandemic periods.
Female students generated a set of 157 words, 79 during pandemic and 78 during pre-pandemic. The 37% out of words published in pandemic were associated with conceptual content and the remaining 63% was tagged as collaborative skills. In pre-pandemic the situation was different as 28% of words indicated conceptual content and 72% manifested collaborative skills. Figure 6 shows the distribution of interactions of female students in pandemic and pre-pandemic. Interesting discoveries can be made when considering the grades that teachers assigned. In prepandemic as well as pandemic teachers evaluated groups through the evaluation scale of “excellent”, “very good”, “good”, “fair” and “poor”.
As it is showed in Figure 7, the percentage of male students graded as “excellent” during pandemic (37%) far exceed the amount of male student graded as “excellent” during pre-pandemic (13%). On the other hand, the number of male students with “very good” grade was 16% higher in pre-pandemic than in pandemic. Regarding to “good” grade, there was no significant difference between two periods, 16% in pandemic and 13% in pre-pandemic. The number of students with “fair” grade was noticeably lower during pandemic (7%) than the number recorded during pre-pandemic (18%). No “poor” grades were recorded in either period. Considering that a satisfactory academic performance could be represented by “excellent” and “very good” grades, could be stated that 77% of male students had a satisfactory performance during pandemic and 69 % during pre-pandemic. These numbers show a certain positive difference in favor of the pandemic period. If “good” and “fair” grades are considered as evidence of deficient academic performance, it could be stated that deficient academic performance was mainly observed in pre-pandemic period. Unsatisfactory academic performance was not recorded in any periods since there were no “poor” grades.
The performance of female students can be observed in Figure 8. The number of “excellent” grades changed considerably from a period to the other as 52% of female student reached “excellent” during pandemic and only 44% in pre-pandemic. Considering the “very good” grade there was no significant difference between two periods, 26% in pandemic and 28% in pre-pandemic. Students that were graded as “good” represented 22% during pre-pandemic and this number decreased to 13% in pandemic; while the number of “fair” grades in pandemic was 4 point higher than the number of “fair” grades recorded in pre-pandemic. There were no students with “poor” grade in either period. If “excellent” and “very good” grades define satisfactory academic performance, Figure 8 shows that in pandemic 77% of female students had satisfactory performance while in pre-pandemic the satisfactory performance represented 72%. These numbers demonstrate a certain positive difference in favor of the pandemic period. Considering that “good” and “fair” grades define deficient academic performance, 23% of female students were rated as deficient performance in pandemic and 28% in pre-pandemic. This confirm the positive difference in favor of the pandemic period that was detected when analyzing satisfactory performance. No unsatisfactory performances were recorded because there were no students with “poor” grades.
Professors from different universities have modified their classes with curricular innovations because of COVID-19 pandemic and have evaluated their impacts. This section presents some of these related works describing such curricular innovations.
Area-Moreira et al. (2020) describe an experience at Universidad de La Laguna (Spain) where a blended learning classes were designed for a distance modality. Students had to choose one of two learning itinerates: the first itinerary was based on group and project learning, and the second itinerary was individual and based on learning by topics. The two itineraries included the same topics and skills to be achieved. In both itineraries students had a virtual classroom and remote classes throughout Google Met. Students could ask questions by e-mail or forums. At the end of the experience students answered a survey. Considering the results of survey and the performance of students, Area-Moreira et al. (2020) evaluate their innovation as successful.
Castro et al. (2020) worked with students of Immunology, and they incorporated videos (recorded classes), assignments to complete in the virtual classroom, remote synchronous meeting (Zoom platform), problem-based learning, gamification and flipped learning. The authors pointed out that videos and some assignment caused tiredness, lack of attention and mistakes on students since they were very extensive. However, the authors highlight that most of the activities were positive, motivating, and encouraged students to use specific technical vocabulary.
Professors of mathematic at the Engineering Faculty of Universidad Nacional de La Plata (Argentina) worked with Google Classroom as virtual classroom, Zoom and Webex platforms for remote meeting, Jamboard and Microsoft Whiteboard as virtual blackboard, and the open software called GeoGebra for solving assignments (Del Río et al., 2020). In spite of evaluating the experience with a survey, the authors do not explain results or measure the impact on students learning.
Moreno-Correa (2020) describes the innovations made in following subjects at the Faculty of Health Sciences of the Pontificia Universidad Javeriana Cali (Colombia): Infections and Immunity of the Medicine degree, and Immunology and Infections of the Nursing degree. The author used active learning throughout flipped classroom. Students had to deliver activity guides to be revised by teachers and all activities were recorded in a virtual classroom in Blackboard. Moreno-Correa states that a survey to gather the opinions of the students would be conduct, yet he does not provide the results of processing the survey and he does not analyze the impact on students learning.
Llorens-Largo et al. (2021) implemented project-based learning in a subject of the Multimedia Engineering at the Universidad de Alicante (Spain). The first part of the subject was developed faceto-face, but in the second part the virtual modality was adopted because of pandemic. In the virtual modality the authors used videos to explain the theoretical content, videoconferences to answer questions of students, and video-streaming for students to present their work. Authors conduct an opinion survey for each modality of classes (virtual and face-to-face) and students expressed their satisfaction with both modalities. Llorens-Largo et al. states that the performance of the students was similar in both modalities.
Lacave and Molina (2021) designed a cross-sectional pilot study to explore, from both social and technological points of view, how students carried out their group programming activities during the pandemic period. The authors indicated that the students adopted a programming model based on work division or distributed peer programming, and very few made use of synchronous distributed collaboration tool. However, no significant differences are detected in the students’ needs between prepandemic and pandemic period.
This paper described a comparative study of interactions and academic performance of Computer Science university students along two periods: pre-pandemic (2018 and 2019) and pandemic (2020 and 2021). Results were discriminated by gender: male students and female students. The study considered the content of interactions recorded while students used COLLAB as a communication tool to complete collaborative activities. Pre-processing tasks and the content analysis technique were applied on set of interactions collected in pre-pandemic and pandemic. That way the percentage of conceptual content and collaborative skills identified in group interactions was defined. The results pointed out that there were no significant changes in the behavior of male students. However, female students showed a greater willingness to focus dialogue on interpersonal relationships instead of conceptual content.
The grades assigned to the students and the academic performance of both genre (male and female) was analyzed to compare pre-pandemic and pandemic. The analysis showed an improvement of academic performance during pandemic, and this improvement was noticeable in male students.
When analyzing the relationship between behavior of learning groups (conceptual content and collaborative skills) and academic performance, it seems that pandemic had a positive effect on dynamic of learning groups regardless of the gender of group members. A possible explanation of this positive effect can be found on the features of students that participated on this research: university students of Computer Sciences are used to utilize technology for doing their academic activities and it is possible that the drastic transition to virtual classes far from being conflictive, was beneficial. Furthermore, at the end of classes all students stated that virtuality and isolation due to pandemic led them to allocate more time to academic activities. The increase on the time spent on academic activities is probably one of the causes of the improvement in performance during pandemic.
The findings of this study are original and contrast with those obtained by other researchers, as can be seen in the section on related works. On the other hand, it is worth noting that no studies have been found that focus on assessing the impact of the pandemic on students' collaborative interactions, analyzing their skills, concepts, and academic performance.
In conclusion, this research provides relevant information on the impact of COVID-19 pandemic on teaching and learning processes when these processes are supported by computer and based on collaborative activities. However, results should be careful considered because they might not directly apply to other academic disciplines. It is well known that features, preferences, and skills of students change from one discipline to other. In near future the impact of COVID-19 pandemic on teaching process and collaborative learning process will be evaluated, yet this time the research will focus on students of Computer Sciences of other universities. In the medium term, an attempt will be made to analyze the interactions considering variables such as emotional states of the students, preferences regarding activities, and preferences regarding tools.
This research was funded by SECYT UNSE project 23/C138 “Mejorando escenarios de aprendizaje colaborativo soportado por computadora”, from January 2017 to December 2021.
[8] L. Del Río, P. Knopoff, E. Boero, L. Ciliberti. Innovación e improvisación en el marco de la pandemia de COVID-19: relato de una experiencia. Trayectorias Universitarias 6.10, 2020: e020. doi:10.24215/24690090e020. [9] S. Moreno-Correa. La innovación educativa en los tiempos del Coronavirus. Salutem Scientia
Spiritus 6.1, 2020: 14-26. [10] F. Llorens-Largo, C. Villagrá-Arnedo, F. Gallego-Durán, R. Molina-Carmona. CoVId-proof: cómo el aprendizaje basado en proyectos ha soportado el confinamiento. Revista Campus Virtuales 10.1, 2021: 73-88. [11] C. Lacave, A. Molina. The Impact of COVID-19 in Collaborative Programming. Understanding the Needs of Undergraduate Computer Science Students. Electronics 10.14, 2021: 1728. doi: 10.3390/electronics10141728.