Towards a comprehensive framework for situated collaborative learning tools

Abstract

CSCL (Computer Supported Collaborative Learning) is a dynamic field that has considerably evolved in recent years. The result is a myriad of tools and theories that have emerged from numerous studies. While different studies shed light on different aspects of collaboration, a comprehensive connection between collaboration processes they support has not been established yet. This PhD aims at providing a joint conceptual framework and environment to achieve this objective. Computer Supported Collaborative Learning, framework, collaborative processes, tool lemans.fr (A. 3)

1. Introduction

The field of CSCL (Computer Supported Collaborative Learning) aims at analyzing and improving collaborative learning activities through digital tools. Collaboration has become especially prominent with the rise of learning theories such as Social Constructivism and has been found to be a key property of learning [ 1 ]. The research focus has therefore shifted from the individual to the group, as unit of analysis [ 2 ]. Researchers argue that the process of learning in groups becomes more explicit since individuals have to communicate intentions, knowledge and actions – which, in turn, allow researchers to capture parts of learning that would remain invisible if only the individual was studied [ 3 ]. However, groups also add complexity to investigate learning since they form complex systems in which individuals influence each other in various ways.

CSCL tries to address this by providing digital tools that help analyze and improve collaboration. Studies have proven superiority of digital tools over traditional means to support Proceedings of the Doctoral Consortium of the Seventeenth European

Conference

Learning, EMAIL:

[e-mail] (A.

1);

2);

2022 Copyright for this paper by its authors. Use permitted under Creative tangible tokens for collaboration

CSCL has seen numerous theoretical frameworks emerge on the nature of collaboration these past years. Indeed, CSCL is a cross-domain discipline drawing on concepts and theories from Psychology, Computer Science, Education and Sociology [ 7 ] and is in close neighborhood to CSCW (Computer Supported Collaborative Work). Consequently, overlapping concepts and varying views from researchers across disciplines have resulted in a variety of frameworks. Even the definition of collaboration itself is not unique and has evolved over time [ 8 ]. The first challenge is therefore to establish a unified conceptual framework of collaboration. If this challenge can be mastered, a second challenge would be to identify links between collaborative processes and tool functionalities. Indeed, even though studies have proven that digital tools can provide better assistance for collaboration then traditional means in many aspects [ 5 ], there is no clear link between activity, low-level functionalities and collaboration.

The objective of our Computer Science PhD thesis is to help overcome the aforementioned challenges. In the next section of this paper, we first present the related work on different aspects of collaboration. In section 3, we then propose a conceptual framework that combines key insights of previous work and provide an overall vision of collaboration on a process level (challenge 1). In section 4, we build on this conceptual framework to provide a tool framework in order to identify links between the high-level collaboration processes and the low-level functional parts of digital tools (challenge 2). In section 5, we present the work that has already been done during this first year of PhD and finally, in section 6, we present the upcoming work to validate our propositions.

2. Related work

High-level definitions of collaboration mainly diverge when it comes to cooperation. It is disputed whether cooperation should be a part of collaboration or a separate concept. In our work, we settle with the vision of Roschelle et al. [ 9 ] and consider collaboration a distinct concept from cooperation. Collaboration requires group members to act as one, while cooperation splits a task into smaller parts. It seems important to make this distinction between cooperation and collaboration in the context of collaborative learning since “acting as one” requires members to agree on their vision of the task, yielding group behavior patterns beneficial to learning not present in cooperative tasks. When both collaboration and cooperation occur, we group them under the concept of collective activity [ 10 ]. As an example, the activity of brainstorming is a collective activity since participants may split up the mental work of idea generation (cooperative activity) but organizing themselves involves joint planning and coordination (a collaborative activity).

In an attempt to detail the concept of collaboration further, two types of frameworks have emerged: on one hand, frameworks based on the notion of collaborative skills (e.g. [ 11 ]) and on the other hand, frameworks on the notion of collaborative processes (e.g. [ 4 ] [ 12 ]). We present three main frameworks and important work on peripheral concepts that will be the basis for our own proposition.

Meier et al. have identified five aspects of collaboration in their attempt of assessing the quality of computer supported collaboration processes [ 12 ]: Communication, Joint Information Processing, Coordination, Interpersonal Relationship and Motivation (figure 2).

Communication includes processes such as “grounding” to build a shared vision of concepts [ 13 ], Joint information processing refers to reaching consensus on decisions and processing available information collectively. To do so, members need to know what others know within the group and may use transactive memory systems [ 14 ]. Coordination concerns the organization of resources and monitoring critical subtask sequences while interpersonal relationship is characterized by Meier et al. by the absence of hierarchies where members have the same status, referring to Dillenbourg’s notion of symmetrical relationships [ 15 ]. Finally, the Motivation category involves motivation by members to their individual contribution as well as to the group task result.

Mateescu et al. identified five dimensions of collaboration in their systematic review on collaborative studies [ 4 ]: Workspace Awareness, Verbal and gestural communication, Participation, Coordination Flow, Artifact interaction and Level of Reasoning (figure 3).

Workspace Awareness means understanding another person’s interactions with the shared workspace. Verbal and gestural communication corresponds to the number of assertions, questions and answers. Participation is defined as a level of involvement by the participants in the problem solving process. Coordination flow embodies the strategies on how a group links or orchestrates individual contributions. Artefact interaction refers to the use of any object (e.g. tangible tokens). Finally, the level of reasoning is defined as “Measures that reflect the level of reasoning observed in or expressed by group members”.

Hesse et al. distinguish conceptual skills from social skills in their framework for teachable collaborative problem solving skills [ 11 ]. Social skills comprise Participation, Perspective taking and Social regulation whereas Conceptual skills concern Learning and knowledge building and Task regulation.

Hesse et al. describe participation skills as “observable action of engaging in discourse” and distinguish between action, interaction and task completion. Perspective taking is the capability to understand what other people think and know. Social regulation refers to the capacity of group members to be aware of and overcome biases (e.g. confirmation biases) so as to fully exploit the potential of the group’s mental resources. Task regulation is a synonym for planning and coordination skills. Learning and knowledge building is a two-folded category in which knowledge building designate the “ability to take up ideas from collaborators to refine problem representations, plans, and monitoring activities” and learning as “the ability to identify and represent relationships, understand cause and effect, and develop hypotheses based on generalizations.”

Collaborative processes and skills are only a part of collaboration and how it emerges. As Dillenbourg notes, there is no guarantee collaborative learning will take place, but chances that it will occur can be increased by setting the right conditions [ 15 ]. The choice and design of activities are crucial to collaboration. The reason why collaboration is nothing natural is that it is not the most effective way to accomplish a task. Cooperation, in contrast, provides the advantage of task parallelization and a lower cognitive load per individual. Hierarchical structures further reduce cognitive load by limiting information spaces necessary for the execution of specialized subtasks. However, this intuitive modus operandi is counterproductive to learning since learning takes place in exchanges [ 1 ]. In order to make collaboration emerge in a team setting, Johnson & Johnson thus defined conditions for successful collaboration featuring social skills, promotive interaction, positive interdependence, group processing and individual & group accountability [ 14 ] (figure 5).

Social skills and promotive interaction refer to how individuals encourage and facilitate each other’s efforts to complete tasks in order to reach the group’s goals [ 16 ]. Group processing consists of multiple layers: selfreflection and regulation with respect to the needs and goals of the others in the group, coreflection and regulation, and shared reflection and regulation (Kirshner et al). Such metacognitive skills require meta-cognitive evaluations: members must give feedback to each other and reflect on these to elicit which individual or group actions were helpful or unhelpful and to make decisions as to whether to continue or to change particular actions. Positive Interdependence links member of a team together so one cannot succeed unless all group members succeed [ 17 ]. This can be done for example through the design of the activity, by strategically providing knowledge for task completion among different members of a team. By doing so, members are constrained to collaborate and exchange. Finally, group and individual accountability in activities hold people responsible for their individual as well as the group performance. “When a person’s performance affects the outcomes of collaborators, the person feels responsible for their welfare as well as his or her own (Matsui, Kakuyama, & Onglatco, 1987). Failing oneself is bad, but failing others as well is worse.” [ 16 ]

One last important concept related to collaboration is described in literature: cognitive artefacts. These are mental representations that help the group keep track of shared knowledge and a common representation of the task state. Since collaboration requires significantly more attention and cognitive resources than cooperation, groups organize and manage transactive memory systems. Such systems only require individuals to know what others know (meta-knowledge) to pool and process distributed knowledge within a group [18]. A joint problem space is established when members of a group successfully communicate a shared vision of the task or problem at hand. The notion of a joint problem space was first introduced by Roschelle et al. [19] .

3. PhD thesis propositions

As presented in the previous section, researchers have proposed various types and categories of collaborative processes, including related concepts such as skills, conditions and cognitive artefacts. The problem is, for the purpose of establishing links between processes and tools, to reunite these different visions under a common framework.

3.1 Conceptual framework

We attempt to provide a comprehensive conceptual framework that encompasses all of these views.

3.1.1 Process categories

We combine the collaborative process categories, proposed by Mateescu et al. [ 4 ] Hesse et al. [ 11 ] and Meier et al [ 12 ], into three main categories: Perception, Participation and Coordination (the three categories are colored in shades of green throughout the presented frameworks in figure 2 – 4 and match our framework proposition in figure 7).

The participation category contains collaboration processes that Meier et al. grouped under communication and Mateescu et al. within Verbal and Gestural communication. We widen Hesse’s definition of participation as an “observable action of engaging in discourse” into an observable action of engaging in communication. We further follow Hesse in his distinction of different levels of participative processes along actions, interactions and task completions. This category definition allows us to include processes considered by Mateescu et al. as artefact interaction. Examples of participative collaborative processes are grounding (the process of building a common vision by adapting individual knowledge to the other person’s level of understanding), dialogue management, building on existing ideas, challenging arguments or managing transactive group memory (by creating and managing shared knowledge across group members).

The awareness category relates to knowledge about the environment, more specifically about cognitive awareness (what do I and other people know), behavioral awareness (what do other people do) and social awareness (emotional state of other group members [20]. As such, Hesse’s social skill of Perspective Taking corresponds to a type of social awareness as well as Mateescu’s workspace awareness to behavioral awareness in the presence of a shared tool. It also englobes Meier’s interpersonal relationship category since it involves processes such as sensibility for hierarchical orders and potential conflicts that are a type of social awareness essential to maintain collaboration. Examples of awareness processes include self-evaluation (gaining awareness of personal strengths and weaknesses), pooling from transactive memory (gaining awareness of knowledge, strengths and weaknesses of others) or assuming responsibility for aspects of the activity itself. While those processes are not directly visible for an observer, they feed participative processes that reflect their presence within a group (such as taking part in an activity and informing others about its progress).

The coordination category relates to collaboration processes that coordinate how the task is resolved by the group. This category exists in all three frameworks (named task regulation in Hesse’s framework). This category encompasses processes for resource management and planning (goal negotiation and expectations). Group processing is another important process which refers to the capability of a group to assess and evaluate their strategies for task completion and adapt them accordingly [21].

In addition, we propose to link several peripheral concepts to these three collaboration processes: conditions, skills and artefacts.

3.1.2 Preconditions, skills and cognitive artefacts

In order for collaborative processes to take place, we consider favorable conditions, such as format and design of the activity itself (providing rule sets to create forms of positive interdependence) and existing social and cognitive skills among team members.

In particular collaborative Skills can facilitate collaboration but can also be acquired and enhanced by engaging in collaboration, therefore being a reciprocal system in which processes act on skills and vice-versa.

Successful collaboration yields cognitive artefacts and group behavior patterns such as a joint problem space [ 9 ] (consisting of content and relational spaces [22]) and a shared group memory [18]. These cognitive artefacts can be detected and their quality measured for both analysis and tool support. This is the reason why Level of reasoning is colored in orange in figure 2: The use and quality of those cognitive artefacts allow us to assess the level of reasoning that participants deploy during collaboration.

In conclusion, the proposed conceptual model consists of a collaborative process hierarchy that groups different collaborative processes together under the following three main categories: Participation, Awareness and Coordination. When collaborative processes take place, they yield cognitive artefacts such as a joint problem space, shared group memory etc. For collaborative processes to take place, preconditions have to be met such as positive interdependence, accountability and, in particular, existing social collaborative skills.

3.2 Linking processes & tool functionalities

Previous studies on CSCL have mainly been concerned with providing evidence that digital tools provide advantages over more traditional means of collaboration, such as pen and paper. While this aspect is now widely accepted, studies are now starting to consider the impact of tools on the various collaboration processes. However, these tools are often composed of several functionalities, making it difficult to identify which of these functionalities, or a combination, is really supporting collaboration.

Prominent examples include Hwang et Su 2012: The study of surface computer supported cooperative work and its design efficiency and challenges, where a number of concepts such as territoriality and multiple gesture/action visualisations and have been condensed in a single tool. Caretta is another example of a tool that combines functionalities such as voting, shared and private screens, physical tokens, action visualisation and other functionality in one tool.

Having established a common framework on collaborative process level, the main question of our work is the following: Can we link tool functionality to collaborative processes and if so, is there a combination that optimizes collaboration for a given activity and context?

Investigating the potential existence of such links requires a notion of functionality that has the potential to be linked to one or more collaborative processes.

3.3 Functional bricks

We envision every tool to be a set of modular functional bricks, configured to work together. A functionality may be a shared mobile screen, or a widget to balance participation as demonstrated by Bachour et al. [23]. Another functional brick could be a shared mobile display to augment a static surface using a peephole approach. The tool presented in Figure 1, for example, has a functionality to filter the information presented on the shared screen and a functionality to interact with the screen by manipulating tangible tokens [24].

These functional bricks may directly impact certain collaborative processes or indirectly, by impacting related concepts. For example, a functional brick that manages positive resource interdependence helps at upholding conditions for collaboration. Another type of indirect functional bricks are those supporting cognitive artefacts, such as maintaining a joint problem space (e.g. by visualizing group findings).

A tool based on our framework is a mere aggregation of one or more functional bricks, each configured and orchestrated by a class of core bricks. The orchestration bricks allow for dynamic configuration of functional bricks included in the tool. Thereby, researchers can trigger the use of certain bricks at different moments of the experimentation or provide different groups with different functional bricks and information, effectively testing positive or negative impact of functional bricks (or variations thereof) on collaboration in an experimental manner (figure 8).

4. Conclusion and perspectives

During this first year of PhD, we have tried to form a comprehensive view of all the literature related to collaborative learning. We propose a conceptual framework that combines the important concepts and show the relations between them. In particular, this framework groups the collaboration processes into three main categories: participation, awareness and coordination. Our objective is now to build on this conceptual framework to identify links between the functional bricks, found in digital tools, and the collaborative processes they support. Understanding these links between functionalities and collaborative processes will be a significant breakthrough in CSCL because it will allow designers to implement only the necessary functionality to support the type of collaborative activity they want to create. However, there is still a long way to go before we can identify the effect of functional bricks.

To start with, we intend to analyze previous studies on collaborative tools. This will provide insight on the possible effects of the functional bricks on the collaborative processes. However, this will not be very precise, as systemic reviews are limited in depth and explanatory power due to heterogeneity of study parameters such as activity design, domain context, experimental parameters such as group size and composition but also tool design.

Ideally, more studies should be led with all the existing functionalities to help measure their impact on collaboration. Our intention is not to do this ourselves (which would be impossible within the given time of a PhD) but rather to provide a framework on which the community can build on. We also intend on providing an open-source software architecture to facilitate the implementation of these functional bricks and there orchestration. We plan on developing the core orchestration module and two functional bricks as a proof of concept. These functionalities and there combinations will be tested in 2023, during three experimentations planed in diverse contexts: a field trip in geography with master students, an orienteering race with disabled students in secondary school and a historygeography field trip with novice primary school teachers. The design of learning activities will be based on the MoCoGa model developed by Marfisi-Schottman et al. [26].

We believe that using a modular approach, under a common framework, allows for a better comparability and reproducibility of studies and strengthening identified links between functionalities and collaboration. In addition, developing tools takes up a significant amount of available resources. Sharing development efforts in a collaborative matter has the potential to liberate resources that can be used elsewhere. In the medium term, data and results from the scientific community using this framework for further experiments will validate modules and combinations that cannot be tested during this project and provide insights to enhance the interaction model that our experimentations will yield. In implementing the before mentioned methodology, we hope to also address the ongoing reproducibility crisis which is not exclusive to domains such as psychology or medicine [25].

While approaches like open data or preregistrations can improve reproducibility, the variety of tools (and their limited availability for replication studies) used in CSCL make it near to impossible for other researchers to validate results. Not only may software not be available to other researchers but software is usually built for specific hardware (e.g. interactive tabletop), further limiting reproducibility and comparability. The latter is especially important in CSCL since study group sizes are small. Large size studies on situated collaborative learning are uncommon and thus, generalizing results is difficult [ 7 ].

5. Acknowledgements

The research published in this article was carried out for the SituLearn project, financed by the French Agence National de la Recherche (ANR-20-CE38-0012).

[18] B. Biancardi, L. MaisonnaveCouterou, M. Mancini, and G. Varni, ‘Tu sais qui sait quoi ? Suggestions pour l’étude du système de mémoire transactive dans un groupe à partir des patterns comportementaux et conversationnels.’, Saint Pierre d’Oléron, France, Jun. 2020. Accessed: May 24, 2022. [Online]. Available: https://hal.inria.fr/hal02934517

[19] J. Smith, A. Disessa, and J. Roschelle, ‘Misconceptions Reconceived: A Constructivist Analysis of Knowledge in Transition’, Journal of the Learning Sciences, vol. 3, Nov. 1993, doi: 10.1207/s15327809jls0302_1.

[20] J. Fransen, P. A. Kirschner, and G. Erkens, ‘Mediating team effectiveness in the context of collaborative learning: The importance of team and task awareness’, Computers in Human Behavior, vol. 27, no. 3, pp. 1103–1113, May 2011, doi: 10.1016/j.chb.2010.05.017.

[21] D. W. Johnson, R. T. Johnson, M. B. Stanne, and A. Garibaldi, ‘Impact of Group Processing on Achievement in Cooperative Groups’, The Journal of Social Psychology, vol. 130, no. 4, pp. 507–516, Aug. 1990, doi: 10.1080/00224545.1990.9924613.

[22] B. Barron, ‘When Smart Groups Fail’, Journal of the Learning Sciences, vol. 12, no. 3, pp. 307–359, Jul. 2003, doi: 10.1207/S15327809JLS1203_1.

[23] K. Bachour, F. Kaplan, and P. Dillenbourg, ‘An Interactive Table for Supporting Participation Balance in Face-toFace Collaborative Learning’, IEEE Trans. Learning Technol., vol. 3, no. 3, pp. 203–213, Jul. 2010, doi: 10.1109/TLT.2010.18.

[24] H.-C. Jetter, J. Gerken, M. Zöllner, H. Reiterer, and N. Milic-Frayling, ‘Materializing the query with facet-streams: a hybrid surface for collaborative search on tabletops’, in Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, New York, NY, USA, mai 2011, pp. 3013–3022. doi: 10.1145/1978942.1979390.

[25] J. Janssen and I. Kollar, ‘Experimental and Quasi-Experimental Research in CSCL’, in International Handbook of ComputerSupported Collaborative Learning, U. Cress, C. Rosé, A. F. Wise, and J. Oshima, Eds. Springer International Publishing, Cham, 2021, pp. 497– 515. doi: 10.1007/978-3-030-65291-3_27.

[26] I. Marfisi-Schottmann, A. Laine, and P. Laforcade, ‘Towards an Authoring Tool to

References

[1] J. Lave and E. Wenger , Situated Learning: Legitimate Peripheral Participation, 2nd ed. Cambridge University Press, Cambridge, UK, 1991 .

[2] G. Stahl , T. Koschmann , and D. Suthers , ' Computer-supported collaborative learning: An historical perspective', Cambridge handbook of the learning sciences , p. 20 , 2006 .

[3] G. Stahl , Group Cognition: Computer Support for Building Collaborative Knowledge a book by Gerry Stahl . 2006 .

[4] M. Mateescu , C. Pimmer , C. Zahn , D. Klinkhammer , and H. Reiterer , ' Collaboration on large interactive displays: a systematic review', Human-Computer Interaction , vol. 36 , pp. 1 - 35 , Dec. 2019 , doi: 10.1080/07370024. 2019 . 1697697 .

[5] P. Dillenbourg , Collaborative Learning: Cognitive and Computational Approaches. Advances in Learning and Instruction Series. Elsevier Science , Inc, New York, NY, 1999 .

[6] M. R. Morris , A. Cassanego , A. Paepcke , T. Winograd , A. M. Piper , and A. Huang , 'Mediating Group Dynamics through Tabletop Interface Design', IEEE Computer Graphics and Applications , vol. 26 , no. 5 , pp. 65 - 73 , Sep. 2006 , doi: 10.1109/ MCG . 2006 . 114 .

[7] C. E. Hmelo-Silver , E. Chernobilsky , O. Mastov , C. Chinn , A. O'Donnell , and G. Erkens, ' Analyzing collaborative learning: Multiple approaches to understanding processes and outcomes' , ICLS , 2006 .

[8] L. Tong , ' Designing and Analyzing Collaborative Activities in Multi-Surface Environments' , Ph.D. thesis , Université de Lyon, Lyon, 2017 .

[9] J. Roschelle and S. D. Teasley , ' The Construction of Shared Knowledge in Collaborative Problem Solving' , in Computer Supported Collaborative Learning, Berlin, Heidelberg, 1995 , pp. 69 - 97 . doi: 10 .1007/978- 3- 642 -85098- 1 _ 5 .

[10] S. George , ' Apprentissage collectif à distance . SPLACH  : un environnement informatique support d'une pédagogie de projet' , Ph.D. thesis , Université du Maine, Le Mans , 2001 .

[11] F. Hesse , E. Care , J. Buder , K. Sassenberg , and P. Griffin , ' A Framework for Teachable Collaborative Problem Solving Skills' , 2015 . doi: 10 .1007/ 978 -94-017-9395- 7 _ 2 .

[12] A. Meier , H. Spada , and N. Rummel , ' A rating scheme for assessing the quality of computer-supported collaboration processes' , International Journal of Computer-Supported Collaborative Learning , vol. 2 , no. 1 , pp. 63 - 86 , 2007 .

[13] H. H. Clark and S. E. Brennan , ' Grounding in communication', in Perspectives on socially shared cognition , American Psychological Association, Washington, DC, US, 1991 , pp. 127 - 149 . doi: 10 .1037/ 10096 - 006 .

[14] D. W. Johnson and R. T. Johnson, ' Cooperation and the Use of Technology', in Handbook of research on educational communications and technology , 2nd ed, Lawrence Erlbaum Associates Publishers, Mahwah, NJ, US, 2004 , pp. 785 - 811 .

[15] P. Dillenbourg , ' What do you mean by collaborative learning' , undefined, 1999 , Accessed: Jul. 14 , 2022 . [Online]. Available: https://www.semanticscholar.org/paper/Whatdo-you -mean-by-collaborative-learningDillenbourg/9874b4cfd9e8ef89fd0b753af18c1 4cbc7c42744

[16] D. W. Johnson , 'Social Interdependence: Interrelationships Among Theory, Research, and Practice', American Psychologist, p. 12 , 2003 .

[17] D. Johnson , R. Johnson, and K. Smith , ' Active Learning: Cooperation in the College Classroom' , The Annual Report of Educational Psychology in Japan , vol. 47 , Jan . 1998 , doi: 10.5926/arepj1962.47.0_ 29 . Help Teachers Create Mobile Collaborative Learning Games for Field Trips . In: Proceedings of EC-TEL 2022 , In press', Toulouse, France, Jul. 2022 .