Improving the collaboration process has long been a subject of inquiry. Yet, evaluating collaboration quality is a significant challenge for researchers and practitioners. Recently, the generalized competency model of collaborative problem solving (CPS) has been suggested, encompassing facets, sub-facets, and indicators (verbal and nonverbal) that directly align with CPS skills. Here we discuss the integration of physiological data to potentially further improve the detection of cognitive and affective aspects of CPS. This paper aims to bridge the gap between physiological data features or characteristics and collaboration quality. specifically, we present our attempts to integrate physiological data with verbal and nonverbal indicators of a generalized competence model of CPS in small groups comprising four individuals. Moreover, this integration can be further developed into interventions such as reflective exercises or real-time feedback provided by AI agents, with the goal of enhancing collaborative skills. Physiological data, collaborative problem solving, collaboration quality framework.
As the field of collaborative problem solving (CPS) continues to evolve, researchers have recognized the importance of incorporating physiological data into the measurement and assessment of this generalized competence. Physiological data can provide valuable insights into the cognitive, affective and metacognitive processes that underlie collaborative problem solving. For example, from a systematic literature review (SLR) by Wicaksono et al. [1] found a positive relationship between students Electrodermal Activity (EDA) and their stress level, emotion and mental effort during collaborative problem solving. Other contributions include the detection of cognitive aspects such as learning outcomes, learning gain, and collaboration quality. Several papers on the SLR also focus on metacognitive monitoring using physiological data. By analyzing physiological data in the context of collaborative problem solving, researchers can gain a deeper understanding of the regulatory patterns and trigger events that contribute to effective collaboration in this domain. This new knowledge can inform the development of more accurate and comprehensive assessments of collaborative problemsolving skills. Moreover, incorporating physiological data into the measurement and assessment of collaborative problem solving can enhance the validity and reliability of existing models. However, it is important to note that there remains controversy surrounding the use of physiological data in detecting emotional experiences.
Several assessment frameworks exist for collaborative learning. McGrath's [2] model introduced the concepts of input, process, and outcome. A decade later, Thomson and Perry [3] developed a similar model that described the antecedents, processes, and results of collaboration. Schneider et al. [4] proposed a classification that identified the conditions for successful cooperation, the interactions during collaborative processes, and the final outcome. Notable examples include a rating scheme for the quality of computer-supported collaboration processes[5]. More recently, Sun et al. [6] developed
2023 Copyright for this paper by its authors. a generalized competency model of collaborative problem solving. These studies primarily utilized verbal (utterance/dialogue) and non-verbal behavioral data, collected through video and audio recordings, as indicators of collaboration quality.
A recent development in the field involves the automatic detection of collaborative states in small groups (triads) using multimodal features, as demonstrated by M. Bradford et al. [7]. They achieved successful automatic detection of collaborative problem solving (CPS) at the facet level through the application of multiple machine learning classifier models. Bradford utilized the generalized competency model of CPS by Sun et al. as the basis for their study, particularly in laboratory settings involving CPS tasks that require real-time interaction with physical objects.
Another recent advancement has been made by Yan et al. [8] in their research on physiological synchrony and arousal as indicators of stress and learning performance in embodied collaborative learning (ECL), specifically in the context of high-fidelity healthcare simulation involving four-person groups. Their findings align with previous SLR conducted by Wicaksono et al. [1], which emphasized the significance of physiological synchrony measures as indicators of perceived stress and task performance. Additionally, it was observed that physiological arousal measures remain significant indicators of task performance even after accounting for the variations explained by individual and group differences. An interesting aspect of the ECL study is the utilization of heart rate as a measure of physiological data, in contrast to other research in collaborative learning that predominantly employs EDA. This choice was made due to the heart rate's reliability, especially in situations involving constant hand movements during collaborative activities.
Based on the SLR conducted by Wicaksono et al. [1] there are currently no studies that have successfully linked physiological data features or characteristics to a collaboration quality framework. Exploring the possibility of incorporating physiological data features as additional indicators within a collaboration quality framework would be an intriguing avenue for future research. Therefore, there are several potential research directions to consider. One such direction involves the detection and modeling of physiological data, utilizing measures such as EDA or heart rate, to serve as additional indicators within a collaboration quality framework. In the next phase of research, an intervention phase, physiological data will be integrated as feedback to the groups in order to enhance their collaborative learning activities. The primary focus of this study will be on the detection and modeling phase. Consequently, the following research questions will guide this study: 1. How can physiological data be incorporated into a collaboration quality framework? 2. Are there any differences in physiological data between high-quality and low-quality collaboration groups within the collaboration quality framework?
Several CPS models have been developed to suit specific contexts. Sun et al. [6] introduced a competency model of CPS that focuses on three primary facets: constructing shared knowledge, negotiation/coordination, and maintaining team function. Their model has shown convergence, discriminant, and predictive validity. Notably, two studies have provided support for the generalized CPS model, despite variations in tasks, environments, demographics, durations, roles, and coding schemes. The first study involved middle school students playing educational games to solve physics problems in face-to-face settings, while the second study involved college students participating in a visual programming task in online settings. Both studies implemented triads as the collaborative structure. The research conducted by Sun et al. was inspired by the earlier works of Roschele and Teasly [9], Nelson in Reigelut [10], Griffin et al. [11], and the PISA Framework [12]. 2.2
One of the methods to illustrate the multimodal data’s relationship to higher level constructs is using multimodal matrix (MM) developed by Echeverria et al. [13] The adaptation of MM can be seen in table 1.
The representation of the performed modeling on the data of a single student is presented in Table 1. The MM (multimodal matrix) is a data structure with dimensions (m x n), where each data modality m is encoded into n columns of the matrix, referred to as multimodal observations. For instance, the first row illustrates the analysis of the student's initial utterances, which were annotated by two trained coders. In these utterances, the coders identified that the student proposed specific solutions and/or built upon others' ideas to enhance the solution, thus marking '1' in the "shared understanding of problems and solutions" column. Simultaneously, we recorded the EDA/Heart rate Peak. The count of arousal peaks per role, detected through an increase of 0.05 µS, was registered in each row of the MM for each utterance. Additionally, the heart rate peak was determined based on the value at the time of utterance exceeding the average heart rate value.
This study will attempt to replicate and take a step further several research findings that emphasize physiological arousal and synchrony as a predictor of collaborative learning features such as stress and learning performance as mentioned in SLR by Wicaksono, et al. This experiment will engage 40 participants, divided into 10 groups of four individuals each, with a focus on the domain of education and technology CPS. Additionally, the experiment will utilize the general competence of CPS by Sun, et al. [6].
others question/ideas
execution
roles of the team
The proposed framework that we intend to expand is illustrated in Table 2. To construct this framework, we have adopted Sun et al.'s model, which focuses on six sub-facets along with their corresponding verbal and non-verbal indicators. Our aim is to further expand upon this framework by implementing it in a different context, specifically face-to-face collaborative problem solving among a group of four college students. This implementation will allow us to validate context-specific features, which was one of the main findings highlighted by Yan et al. [8] regarding physiological data as indicators of stress and learning performance in collaborative learning.
Our first objective is to replicate the work of Bradford et al. [7]. We aim to automatically detect collaborative states using physiological data as additional features, supplementing the prosodic features they utilized. Furthermore, we aim to incorporate physiological data, specifically physiological synchrony, by examining the event of interest within each collaboration sub-facet and relating it to peaks (sharp increases), oscillations (jolts), and valleys which extending the works of Schneider et al. [4]. These physiological data will be cross validated with coding utterances, log contents, and selfreports. The selection between using electrodermal activity (EDA) or heart rate as the physiological measure will depend on the availability of data. 3.2
The hands icon on Figure 1 indicates when participants were instructed to "tag" using their Empatica wristbands. Participants (students) are randomized at random to a group of four members of Master Student of Education Technology with diverse backgrounds, ethnicities, genders, employment experiences, and 'English as a first language' status. Collaborative Problem Solving pertains to the educational technology trend of determining the optimal use of generative AI in higher education. In the introduction, participants will receive the following information regarding generative AI and work instructions: 1. Begin by providing participants with a clear overview of generative AI and its potential applications in higher education. 2. Instruct participants to discuss and evaluate the advantages and risks of utilizing artificial intelligence tools to enhance learning outcomes. 3. Encourage participants to consider ethical considerations associated with the use of generative AI in educational settings. 4. Facilitate a discussion among the participants, guiding them to explore the optimal ways to incorporate generative AI in higher education. The time will be 45 minutes. 5. Encourage brainstorming and idea generation within the diverse group, ensuring each member has an opportunity to contribute. 6. Remind participants to consider the potential benefits, risks, and ethical implications of their proposed solutions. 7. Instruct the groups to document and present their design project on the Miro platform, using visuals, text, and any other relevant media.
During the primary task, participants will be monitored using a video camera, microphone, and Empatica E4 smartwatch to collect physiological data. Additionally, participants will be required to complete pre- and post-discussion surveys, evaluating their perceptions of their collaborative performance in terms of cognitive and non-cognitive aspects, including emotions and motivation. The survey items used for this evaluation are based on those developed by Yan et al. [8].
At the conclusion of the discussion, the collaborative results will be assessed based on five criteria: understanding the problem, innovative solutions, feasibility of solutions, ethical considerations, and presentation of solutions.
The evaluation of group collaboration will be conducted based on the six sub-components of the CPS general competence model proposed by Sun, et al [6]. The analysis primarily focuses on detecting indicators of collaboration sub-facets during collaborative problem-solving by examining specific patterns of electrodermal activity (EDA) or heart rate, such as peaks, valleys, and oscillations, at designated time points. These patterns will be cross-referenced with utterances at the sub-facet and indicator levels, as well as self-reports. For instance, when a group member responds to another member's idea (a part of the negotiation/coordination facet), whether supporting or refuting a certain claim and negotiating when disagreements occur, they might exhibit similar patterns of physiological arousal. This could manifest as consistent EDA or Heart Rate Peaks among group members with differing viewpoints. Furthermore, when a group member enters the phase of constructing shared knowledge, as indicated by sharing ideas/information and building common group understanding by addressing and clarifying misunderstandings, they might exhibit physiological synchrony at certain points. This research builds upon the previous work of [6] and [7] who identified CPS at the facet level. To preprocess the audio recordings and obtain final utterances, we plan to utilize a method inspired by M. Bradford et al. [7].
Furthermore, we aim to investigate potential differences in physiological arousal and synchrony, detected through EDA/heart rate patterns, between high-quality and low-quality collaborative groups across all six sub-components of the overall CPS competency model. We will seek to distinguish these patterns between high-performing and low-performing groups using a triangulation of physiological data with other gathered data, including verbal annotations, physiological data, log data, and selfreports. Each group is expected to participate in 30 sessions, resulting in a total of 60 sessions for both high and low-performing groups, with each session lasting 45 minutes.
This study has a specific scope focused on incorporating physiological data into a collaboration quality framework at the sub-facet level. The context of investigation is collaborative problem-solving (CPS) in a higher education setting, specifically addressing the problem of utilizing generative artificial intelligence (AI) in higher education. The group involved in the study comprises four individuals. Additionally, the study explores the automatic detection of collaborative states using physiological data. It is important to note that this study represents the initial phase of a broader research endeavor aimed at providing interventions and feedback by incorporating physiological data. These interventions could take the form of student reflections after meetings or real-time feedback using AI agents, with the goal of assisting the group in improving their collaborative skills.
W. Febriantoro, A. Gauthier, and M. Cukurova, “The Promise of Physiological Data in Collaborative Learning: A Systematic Literature Review,” in Proceedings of the European Conference of Technology-enhanced Learning (ECTEL), Springer New York, 2023. McGrath, Joseph. “Groups : Interaction and Performance”. Prentice-Hall, Inc New Jersey, 1984. A. M. Thomson and J. L. Perry, “Collaboration Processes: Inside the Black Box,” Public Adm. Rev., vol. 66, no. s1, pp. 20–32, Dec. 2006, doi: 10.1111/j.1540-6210.2006.00663.x. B. Schneider, G. Sung, E. Chng, and S. Yang, “How Can High-Frequency Sensors Capture Collaboration? A Review of the Empirical Links between Multimodal Metrics and Collaborative Constructs,” Sensors, vol. 21, no. 24, p. 8185, Dec. 2021, doi: 10.3390/s21248185. [5] A. Meier, H. Spada, and N. Rummel, “A rating scheme for assessing the quality of computersupported collaboration processes,” Int. J. Comput.-Support. Collab. Learn., vol. 2, no. 1, pp. 63– 86, Mar. 2007, doi: 10.1007/s11412-006-9005-x. [6] C. Sun, V. J. Shute, A. Stewart, J. Yonehiro, N. Duran, and S. D’Mello, “Towards a generalized competency model of collaborative problem solving,” Comput. Educ., vol. 143, p. 103672, Jan. 2020, doi: 10.1016/j.compedu.2019.103672. [7] M. Bradford, I. Khebour, N. Blanchard, and N. Krishnaswamy, “Automatic Detection of Collaborative States in Small Groups Using Multimodal Features,” in Artificial Intelligence in Education, N. Wang, G. Rebolledo-Mendez, N. Matsuda, O. C. Santos, and V. Dimitrova, Eds., in Lecture Notes in Computer Science, vol. 13916. Cham: Springer Nature Switzerland, 2023, pp. 767–773. doi: 10.1007/978-3-031-36272-9_69. [8] L. Yan, R. Martinez-Maldonado, L. Zhao, X. Li, and D. Gašević, “Physiological Synchrony and Arousal as Indicators of Stress and Learning Performance in Embodied Collaborative Learning,” in Artificial Intelligence in Education, N. Wang, G. Rebolledo-Mendez, N. Matsuda, O. C. Santos, and V. Dimitrova, Eds., in Lecture Notes in Computer Science. Cham: Springer Nature Switzerland, 2023, pp. 602–614. doi: 10.1007/978-3-031-36272-9_49. [9] J. Roschelle and S. D. Teasley, “The Construction of Shared Knowledge in Collaborative Problem Solving,” in Computer Supported Collaborative Learning, C. O’Malley, Ed., in NATO ASI Series. Berlin, Heidelberg: Springer, 1995, pp. 69–97. doi: 10.1007/978-3-642-85098-1_5. [10] C. M. Reigeluth, Instructional-design Theories and Models: A New Paradigm of Instructional
Theory. Routledge, 2013. [11] P. Griffin, E. Care, and B. McGaw, “The Changing Role of Education and Schools,” in Assessment and Teaching of 21st Century Skills, P. Griffin, B. McGaw, and E. Care, Eds., Dordrecht: Springer Netherlands, 2012, pp. 1–15. doi: 10.1007/978-94-007-2324-5_1. [12] OECD, PISA 2015 Results (Volume V): Collaborative Problem Solving. Paris: Organisation for Economic Co-operation and Development, 2017. Accessed: Apr. 13, 2023. [Online]. Available: https://www.oecd-ilibrary.org/education/pisa-2015-results-volume-v_9789264285521en;jsessionid=GFCdRpXfxQEOi1xN3rxke6GO2c7PRjjy-nfqPiMi.ip-10-240-5-126 [13] V. Echeverria, R. Martinez-Maldonado, and S. Buckingham Shum, “Towards Collaboration Translucence: Giving Meaning to Multimodal Group Data,” in Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems, in CHI ’19. New York, NY, USA: Association for Computing Machinery, May 2019, pp. 1–16. doi: 10.1145/3290605.3300269.