=Paper=
{{Paper
|id=Vol-2501/paper8
|storemode=property
|title=An Appraisal of a Collaboration-Metric Model based on Text Discourse
|pdfUrl=https://ceur-ws.org/Vol-2501/paper8.pdf
|volume=Vol-2501
|authors=Adetunji Adeniran,Judith Mastho,Nigel Beacham
|dblpUrl=https://dblp.org/rec/conf/aied/AdeniranMB19a
}}
==An Appraisal of a Collaboration-Metric Model based on Text Discourse==
https://ceur-ws.org/Vol-2501/paper8.pdf
66
An Appraisal of a Collaboration-Metric Model based on
Text Discourse
Adetunji Adeniran1, Judith Mastho1,2 & Nigel Beacham1
1University of Aberdeen, 2Utretch University
r01aba17@abdn.ac.uk
Abstract. This paper presents a more in-depth analysis based on discourse of the collab-
oration-metric model, Word-Count/Gini-coefficient measure of symmetry (WC/GCMS)
which was introduced in [3]. We discuss the validity of the model in regards to how well
it represents what happens in the groups' discourse content. We discuss the application
and implication of WC/GCMS based on the goal to incorporate collaborative learning
and its cognitive advantages to E-Learning environments.
Keywords: Group discourse, online group, E-learning, collaboration-metrics
1 Introduction and related work
Online learning provides access to education for millions of learners through many en-
vironments offered by Universities and other organizations world-wide (e.g. Mass
Open Online Courses). This motivates Computer Supported Collaborative Learning
(CSCL) research towards leveraging the cognitive advantages of collaboration [5, 16,
24, 26, 32] for online learning, as it is preeminent in traditional classroom settings.
Online collaboration however has two major concerns: (i) media richness- the degree
to which a virtual medium conveys the immediacy of face-to-face (F2F) conversation
[28] and (ii) social presence- communication that fosters immediate interaction/feed-
back and permits people to communicate with multiple senses (e.g. verbal and visual
clues) [28]. Media richness and online social presence are inter-dependent; the richer a
media, the more social presence it conveys during online collaboration. For example,
there is more social presence in teleconferencing which conveys both the verbal and
real-time image of collaborators compared to email exchange or other text-based con-
versation media. However, implementing robust media that conveys both verbal and
visual clues for online programs comes with costs and complexity of deployment,
which may inhibit the integration of group learning. Also, a group that is media enabled
with verbal and visual interaction is most times synchronous; this excludes the time
flexibility to participate, to think, and to search for extra information, and to contribute
in a group discussion, which comes with on-line collaboration (e.g. in asynchronous
text-based media) [11, 25, 27]. Text-based group media is cost efficient and prospec-
tively effective for online collaborative learning; De Wever et al. [11] posit that, text-
based discussion makes individual contributions more explicit and provides a better
reflection of the process of collaboration for both researchers and instructors. It is a
good data source to evaluate both collaboration and individual participation within
group [18, 23].
Copyright Β© 2019 for this paper by its authors. Use permitted under Creative Commons
License Attribution 4.0 International (CC BY 4.0).
67
Online learners who interact via a text-based environment strive to maximize the
social presence in the media [28]; a comparative study between text-based & F2F verbal
discourse attests to similarities in both, despite a lack of facial expressions and gestures
in the former [7]. Features such as frequency of agreement or disagreement, use of
negative affect terms and frequency of punctuation use in text contributions reveal emo-
tions of discussants, which is similar to facial expressions and gestures in F2F verbal
discussions [14].
Online discussions provide evidence of collaboration as seen in F2F, although it has
different representations in both; text or verbal information containing the same content
will provide the same emotional or cognitive effect although processed differently [10].
Soller [31] corroborated this position stating that learners pose the natural inclination
to adapt and maximize social presence when they use text-based media to interact; she
however suggested that CSCL research needs to design a new adaptive method to sup-
port interaction in this environment.
1.1 Measure of Collaboration with Text Discourse
The instructors' view of collaboration via textual interaction had depended on a review
of the transcripts of the groups' discourse [12]; analysis about how well groups have
collaborated is possible only after the Joint Problem Solving (JPS) process has ended
and any feedback from such analysis is useful to moderate future group work. In order
to accord online groups the kind of real-time support obtainable in F2F groups, we
require a real-time approach to view what goes on during online JPS.
Schwarz and Asterhan [29] explored this objective and presented a real-time view
of group interaction using the social network of the connections between the activities
within the group (see Fig. 1 a); the measure of participation by members was visualized
with a bar charts, each bar representing different variables of activities involved in the
task, for each group member (see Fig. 1 b).
Our model contributes to existing knowledge by providing a simpler, scalable and
generically adaptable computational mechanism that informs the level of collaboration
during online JPS; applicable in real-time. In the following sections, we assess submis-
sions from existing work about indicators and metrics of collaboration. The ideas from
these studies are aggregated and extrapolated for text-based online interaction. Addi-
tionally, we present the rationale and mathematical relation that inform the Word-
count/Gini-coefficient measure of symmetry (WC/GCMS) model [3]. Finally, we dis-
cuss the method we used to validate this model by triangulating qualitative assessment
of the groups' discourse transcript, with the output of the WC/GCMS model. We con-
clude with a discussion on the implications of the model in regard to a design frame-
work for sustainable and effective online group learning environments.
68
(a) Group collaboration measure (b) Individual group members' ac-
with Social network tivities
Fig. 1: E-moderation of online group collaboration, Schwarz and Asterhan [28]
1.2 Indicators and Metrics of Collaboration within Groups
Much work has been done to identify indicators of collaboration during group JPS;
more of these studies explored F2F or co-located groups. For example, Martinez et al.
[19] mined the frequent sequential pattern of the log trace of groups' JPS activities
around a table-top application to categorize groups into high achieving and low
achieving. In a similar study, Martinez et al. [20] proposed an approach to automati-
cally distinguish between groups that engaged in a collaborative or non-collaborative
activity during JPS.
Meier et al. [22] presented a rating scheme to quantify collaboration, Cukurova et al.
[9] explored how group synchrony and individual accountability, equality and intra-
individual variability informs good collaboration. The consensus found in these exist-
ing studies in regards to indicators of collaboration during JPS are: (i) Symmetry of
contribution (ii) Volume of contribution (iii) Connectivity/links between contribu-
tions of different group members and (iv) the quality of contributions with respect to
context of JPS. In the next section, we will discuss how this informed the WC/GCMS
collaboration metric model.
2 Word-count/gini-coefficient measure of symmetry
The components of the WC/GCMS presented in [3] are given by:
69
WC/GCMS metric of collaboration is given by:
πΊ(π€ππ‘ )
πΊππ = (1)
πΊπ
πΊ(π€ππ‘ ): represents the volume of activities/texts that the group generate during JPS;
assuming that this volume informs the quality of the JPS process [21].
πΊπ : represents the symmetry of the activities within the group and is based on the gini-
coefficient measure of symmetry. It ranges from 0-1; 0 being perfect symmetry and 1
asymmetry. Assuming that symmetry of JPS activities is an indication of group collab-
oration, the numerical value of the πΊπ is inversely proportional to the group collabora-
1
tion level i.e. ππππππππππ‘πππ = .
πΊπ
πΊππ : measures the collaboration within a group.
Volume of group activities: A member i within a group contributes textual State-
ments βββ βββ2 , β¦ , βββββ
ππ , π ππ , at time intervals during JPS. All text contributions by member i is
a collection of statements, π βββπ . The measure of contribution during JPS by member i, is
thus given by equation 2.
π
π€ππ‘ = βπ βββ βββ
π=1 ππ , π€βπππ π = ππ (2)
1 2 3 4
Hence, within a group of 4 members, we have contributions π€ππ‘ , π€ππ‘ , π€ππ‘ , π€ππ‘ . Con-
sidering that a non-collaborating member may contribute very little and an extrovert
may provide an excessively high text contribution, we represent the group activity vol-
π
ume measure, πΊ(π€ππ‘ ) with the median π€ππ‘ in the group:
1 2 3 4
πΊ(π€ππ‘ ) = ππππππ(π€ππ‘ , π€ππ‘ , π€ππ‘ , π€ππ‘ ) (3)
Symmetry of activity within group: This is based on the gini-coefficient measure of
symmetry adapted from [20]. Firstly, we compute the mean number of contributions by
group members (equation 4a), then the symmetry of contributions within the group
(equation 4b):
1
πππππ = βππ=1 |ππ | (4a)
π
βπ π
π=1 βπ=1 |ππ βππ |
πΊπ = (4b)
2π2 πππππ
Next, we describe the output of WC/GCMS with data from 5 groups. The study proce-
dure, a brief discussion about the model and findings was presented in [3]. Here we
provide an expanded and more exploratory discussion on the validity of WC/GCMS
for quantifying collaboration with text-based discourse.
70
3 Text-based discourse data source
The text-based discourse of 5 groups was collected in a study by Adeniran et al. [3].
The groups were formed from a convenience sample of undergraduate/postgraduate
students. Each group had 4 members: (Group 1) 3 male, 1 female, all aged 18-25;
(Group 2) 3 male, 1 non-disclosed; all 18-25; (Group 3) 2 male, 2 female; all 18-25;
(Group 4) 4 male, all 26-35; (Group 5) 4 male, 3 26-35, 1 36-45. In the study, the groups
solved a joint task, the task [1], is an open-ended problem without clear cut answers as
recommended by [8] for group work. JPS was via a text-based chatroom designed for
the study [3]. Discourse is collected in a database; contributions are time-stamped, and
uniquely but anonymously identified with the contributor. This data serves as input for
our WC/GCMS model, which tells how well the groups have collaborated relatively.
3.1 Visualization with WC/GCMS metric
Figure 2a shows the relative measure of collaboration between groups based on total
discourse, Figure 3 simulates a real-time view of this measure during JPS. Figures 2
and 3 can inform a remote teacher about which group is collaborating less well. We did
not define a measure for a collaborative or non-collaborative group; WC/GCMS de-
pends on the comparison between the groups to determine which group needs attention
most, at a given time during JPS.
The measure of individuals' participation within the group (shown in Figure 2 b)
provides a hint about non-participating members; for example, M3 in group 1 or M4 in
group 4. The components of WC/GCMS i.e. πΊ(π€ππ‘ ) & πΊπ , are viewed in real-time as
shown in Figure 4; this provides information about the groups' JPS process as discussed
below. Figure 4a visualizes πΊ(π€ππ‘ ), we can observe a higher ripple in the line repre-
senting Groups 3 and 5, showing that the symmetry of contribution within the group
changes more rapidly during JPS. It is a sign of high frequency of contribution within
the groups which can be hypothesized as an indication of members' interest in the dis-
cussion or a relatively higher knowledge about the task (i.e. the members have more to
contribute). On the contrary, the lines representing Groups 1 and 2 are smoother and
the Group 4 line the smoothest, indicating that the participation rate in these groups is
lower.
From Figure 4 b, which visualizes πΊπ , we can observe that the volume of text con-
tribution in Groups 3 and 5 is higher and increases steadily during their JPS discussion,
corroborating that if the contribution rate is higher, then the contribution volume will
be higher. This also confirms the position of Maldonado [19], that a high verbal activity
is an indication of collaboration; in our context: high textual contribution indicates col-
laboration in a text-based discourse.
4 Validation: WC/GCMS output versus Qualitative Assessment
of Discourse
To validate the WC/GCMS's visualizations, we use the groups' discourse transcripts to
make a comparative analysis with the inferences from the visualization. Contributions
that aid collaboration were conjectured to assume one of the following activity-states:
71
task coordination, acknowledgement, request, inform, argue and motivate [2, 30]. We
assess groups' discourse to determine how much evidence of these collaborative activ-
ity-states exist therein and compare these between the groups. Firstly, at the group dis-
course start, there is evidence of initial coordination within Groups 3 and 5, indicating
an interest and enthusiasm to participate; contrary to our observation in the discourse
of Groups 1, 2, and 4. Participants in the latter groups did not make any effort to famil-
iarize with the task nor with group members; they went ahead to give suggested solu-
tions (See Table 1).
Secondly, there is evidence of informed argument and planning in Groups 1,2, and
4, where the contributions were mostly erroneous. These groups suggested solutions
with blind acceptance and acknowledgement. Most contributions from Groups 1,2, and
4 are similar to what Webb [33] refers to as βgiving and receiving non-elaborated helpβ
(i.e. unexplained solutions to the JPS task). Such contributions during group learning
provide no cognitive benefit to the giver of the information nor to other members. The
extract from Group 5 discourse particularly contains cognitive elaboration, which is
posited to be an evidence of collaboration [33]. The relative level of collaboration be-
tween groups shown by WC/GCMS (as shown in Figures 2a and 3) is thus justified.
(a) Group measure (b) Individual Activity Measure within group
Fig. 2: Final collaboration measure between groups based on discourse content (a), and
individual participation measure (of members M1-M4) within groups based on the num-
ber and word count of contributions (b).
Fig. 3: Simulated Real-time view of collaboration level
72
1
(a) measure between groups (b) πΊ(π€ππ‘ ) measure between groups
πΊπ
Fig. 4: Components of WC/GCMS model
Table 1. Evidence of Coordination.
1
Group 3 ο· Group
measure 5
between groups
πΊπ
Beowulf: Good Morning everybody
Epigha: What item do we think should have the highest ranking of sir D: Hello
1? I suggest oxygen Cg: Hello
Epigha: Any other suggestion? Cg: I have just been writing
Anonymous1: I think safety is most important, so life raft is my sug- notes on all the items whilst
gestion waiting.
Beowulf: My ratings were based on a few things know about the Mide: Hi Cg
moon. Cg: I just submitted my
// First: there is no atmosphere thoughts on the items and the
// Second: It is very cold system deleted the message.
// Third: there is no magnetic field Cg: Hi
Epigha: If there is no atmosphere, how can you breathe without ox- Ku: hi everyone
ygen? I think you need to breathe before considering safety
Group 1 Group 2 Group 4
olu: which do you think should be first
Lucas: Hello... fellas? smart : Oya, so what is your
charis: how are we starting the Carbon: Since we dont knw when d view? Swiftly
ranking
4th member will be available Swift: Obviously, first place im-
charis: β¦
Ranco: Watson smith: I think oxygen portant thing is oxygen
Lucas: Guess we are waiting for Carbon: Stellar mapYeahNo 1 item = Swift: Then water, followed by
one more participant oxygen food
Ranco: Can we start pls?
Ranco: Most important (1) oxygen olu: what about first aid? Swift: What do you think?
Lucas: "I think we can Charis, olu: I think I agree with map i.e direc- smart: Yes, oxygen... Correct
still online?" tionmap, compass, first aid smart: Yes.. In that order
Lucas : I will go for oxygen as the
smith: Oxygen should be the most im- smart: Without it
most important (1).
Ranco: Opinion pls? portant smart: All those in order,
Ranco: Hello.... smith: Oxygen is needed for survival in Swift: So, what do you think
Lucas : I think water should be 2
space should be the next?
Carbon: First aid should be later
Non-participating group members: Logically, the rate of participation by an individ-
ual group member is directly proportional to the collaboration level within the group.
73
The discourse transcript shows that member "charis" in Group 3 and "unknown" in
Group 4 did not participate relatively well within their respective groups. This explains
the low bar for M3 in Group 2 and M4 in Group 4 as shown in 2 and further validates
the WC/GCMS output.
Quality of contribution and knowledge about task: Assessment of discourse of
Groups 2 and 5, shows evidence of information sharing, of new knowledge, and sug-
gestions based on logical reasoning about the task. Their discussion conveyed
knowledge of context (the moon environment) and transfer of knowledge (see Table
2). This kind of elaborated discussion indicates participants' socializing during small
group discussion as posited by [15]. On the contrary, the discourse of Groups 1,2, and
3 lacks such knowledge-based interaction; this inhibits socialization within the groups
[15]. In line with the Vygotskian perspective as mentioned in [33] that collaboration
provides cognitive benefits when βa more expert member helps less-expert onesβ. Stud-
ies have also shown that there is a knowledge level threshold for a task that can foster
optimum collaboration within groups; below it, a group will not attempt a solution at
all or suggest unexplained erroneous solutions which hinders collaboration and cogni-
tion [4].
Table 2: Evidence of Other collaborative activity-states
Group 3 Group 5
Cg: The parachute is useful in that it is a large piece of
material. But I do not think that high
Beowulf: My ratings were based on a few things I know about sir D: no gravity
the moon. // First: there is no atmosphere // Second: It is very Cg: It can be used for things other than its intended use.
cold // Third: there is no magnetic field Cg: There is gravity but no atmosphere.
Epigha: If there is no atmosphere, how can you breathe without Mide: U WIL NEED PARACHUTE SINCE U ARE
oxygen? I think you need to breathe before considering safety AIR, FOR LANDIND, SAFETY
cls603: I rated Solar-powered FM receiver-transmitter 1st be- Cg: Was the scenario that you had landed, or were away
cause the to communicate with their base to land? Also there is no information about the para-
Epigha: Beowulf and cls603, what is your contribution? chute. How big is it?
Epigha: cls603, I think communication can come after survival Ku: I think we should first have clusters like A. survival
and safety B. Safety and C. Set Objective Then from this clusters
Beowulf: I rated the oxygen tanks as the most important item, we rank the items in each cluster. And naturally we solve
for breathing the problem
Epigha: I give oxygen tank highest priority too Cg: Is it not your objective to survive?
Epigha: Do we all agree with oxygen tank = 1 sir D: yes it is
Beowulf: I agree with that sir D: "and the scenario says "" mother ship on the
cls603: Alright I agree with the oxygen lighted surface of the moon """
Anonymous1: Oxygen OK Cg: "I think we need to start with either the number 1 or
Epigha:We move to the next item then. what item is the second the number 15 and say ""OK, which item would leave
highest priority? behind if we had to"". That is 15. Then do it again, again
Beowulf: Water is a priority, but because the moon is cold, the etc."
heater is needed to make the water liquid rather than frozen.
74
Group 1 Group 2 Group 4
smith: I think oxygen smart: Yes, oxygen... Correct
Carbon: Stellar mapYeahNo 1 item = oxygen smart: Yes.. In that order
Luca: I think water should be 2 olu: what about first aid? smart: Without it
Ranco: Ok....same
here...good olu: I think I agree with map i.e directionmap, smart: All those in order,
Lucas: "Ranco, Any sugges- compass, first aid Swift: So, what do you think should be the next?
tion for 3?" smith: Oxygen should be the most important smart: A feel a magnetic compass
Ranco: One case of dehy-
drated milk smith: Oxygen is needed for survival in space smart: Cos they would have to knw
Ranco: ??? Carbon: First aid should be later smart: Where they wanna go
Lucas: Hmmm....is that really olu: oxygen is as well important Swift: Yeahh...I agree
important? Remember we
Carbon: After all navigation tools has been pick smart: Then the receiver-transmitter To keep contact
have water already
Lucas: "With oxygen and olu :you can fix oxygen without the first aid kit smart: What do u think?
water, i think the instruments smith: first aid is as well important Swift: The stellar map should come before the com-
to get them to their location smith: but proactive measures should be taken be- pass
should come next"
Ranco: "Ok, at what point do fore reactive measures Swift: Then the receiver-transmitter should come af-
think we will need milk and olu: 1. map2. compass3. oxygen ter the compass
food? Jst asking?" Carbon: Health first smart: Oh
Lucas: After the basic instru-
ments; So the stellar map, olu: 4. first aid smart: That true
United Carbon: I think first aid and oxygen shoyld be smart: What about the heating unit
first smart: I feel the moon is kinda cold u know
olu: any other opinion for the first four rating smart: For a 200miles journey
5 Conclusion
The major contributions of this paper are: first, based on literature, we argue that a text-
based media is efficient and can be optimized to maximize social presence within an
online group [7, 11, 25, 27, 28]. Second, existing studies proposed measures of collab-
oration that use the text discourse transcript, providing an analysis after the discourse
has been completed [6, 13, 17], whilst WC/GCMS is intended to be used in a real-time
group monitoring dashboard for a remote teacher. Third, we present an explicit com-
parative analysis of the WC/GCMS metric output with an assessment of the groups'
discourse, to validate the model's sensitivity in regards to quantifying text-based group
collaboration. We posit that WC/GCMS can provide simple, easily interpretable graph-
ical outputs is upgradeable (to capture verbal and visual clues when using richer inter-
action media) and generic (can be extrapolated to the collaboration context).
Whilst the indicators of collaboration exceed the characteristics of the text discourse
content used in this paper, WC/GCMS is sensitive enough to serve as a proxy-effective
metric of collaboration and participation within online groups. We plan to run a larger
scale study to further investigate the indicators, factors and models presented. We will
also investigate the use of our metrics and visualizations to provide real-time feedback
to learners to scaffold collaboration, and measure both quantitatively and qualitatively
the effect of such feedback on JPS. We further aim to develop algorithms for a computer
agent (taking our models as input) to stimulate participation and consequently scaffold
collaboration.
75
6 References
1. Moon exercise. humber.ca/centreforteachingandlearning/assets/files/pdfs/MoonExer-
cise.pdf.
2. Adeniran, A. Investigating feedback support to enhance collaboration within
groups in computer supported collaborative learning. In AIED Conference, pp.
487-492. Springer. (2018).
3. Adeniran, A., Mathoff, J., Beacham, N. Model-based characterisation of discourse content:
An evaluation of collaboration within online groups. AIED, (2019).
4. Adeniran, A., Masthoff, J., Beacham, N. Analyzing groups problem-solving process to char-
acterize collaboration within groups. In CEUR Workshop Proceedings. (2018).
5. Alfonseca, E., Carro, R.M., Martin, E.,Ortigosa, A., Paredes, P. The impact of learning
styles on student grouping for collaborative learning: a case study. User Modeling and User-
Adapted Interaction, 16(3), pp.377-401. (2006).
6. Andresen, M.A.. Asynchronous discussion forums: Success factors, outcomes, assessments,
and limitations. J. Educ. Techn. & Society, 12(1), pp.249-257.(2009).
7. Bromme, R., Hesse, F.W., Spada, H. Barriers and biases in computer-mediated knowledge
communication: and how they may be overcome. Springer. (2006).
8. Cohen, E.G., Brody, C.M., Sapon-Shevin, M. Teaching cooperative learning: The challenge
for teacher education. Suny Press. (2004).
9. Cukurova, M., Luckin, R., Mavrikis, M., Millan, E.. Machine and human observable differ-
ences in groups collaborative problem-solving behaviours. In European Conference on
Technology Enhanced Learning, pp. 17-29. Springer. (2017).
10. Curtis, D.D., Lawson, M.J. Exploring collaborative online learning. Journal of Asynchro-
nous learning networks, 5(1), pp. 21-34.(2001).
11. De Wever, B., Schellens, T., Valcke, M., Van Keer, H. Content analysis schemes to analyze
transcripts of online asynchronous discussion groups: A review. Computers & education,
46(1), pp.6-28 (2006).
12. Dringus, L.P., Ellis, T. Using data mining as a strategy for assessing asynchronous discus-
sion forums. Computers & Education, 45(1), pp.141-160. (2005).
13. Driscoll, A., Jicha, K., Hunt, A.N., Tichavsky, L., Thompson, G. Can online courses deliver
in-class results? A comparison of student performance and satisfaction in an online versus a
face-to-face introductory sociology course. Teaching Sociology, 40(4), pp.312-331. (2012).
14. Hancock, J.T., Landrigan, C., Silver, C.. Expressing emotion in text-based communication.
In SIGCHI conf. on Human factors in comp. Sys. ACM. (2007).
15. Ho, M.-c. Academic discourse socialization through small-group discussions. System,
39(4), pp.437-450.(2011).
16. Liu, S., Joy, M., Griffiths, N. Incorporating learning styles in a computer-supported collab-
orative learning model. (2008).
17. Loncar, M., Barrett, N.E., Liu, G.-Z. Towards the refinement of forum and asynchronous
online discussion in educational contexts worldwide: Trends and investigative approaches
within a dominant research paradigm. Computers & Education, 73, pp.93-110. (2014).
18. Macdonald, J. Assessing online collaborative learning: process and product. Computers &
Education, 40(4), pp.377-391. (2003).
19. Maldonado, R.M., Yacef, K., Kay, J., Kharrufa, A., Al-Qaraghuli, A. Analysing frequent
sequential patterns of collaborative learning activity around an interactive tabletop. In Edu-
cational Data Mining 2011, (2010).
76
20. Martinez, R., Wallace, J., Kay, J., Yacef, K. Modelling and identifying collaborative situa-
tions in a collocated multi-display groupware setting. In Artificial Intelligence in Education,
pp.196-204. Springer.(2011).
21. Martinez-Maldonado, R., Dimitriadis, Y., Martinez-Mones, A., Kay, J., Yacef, K. Capturing
and analyzing verbal and physical collaborative learning interactions at an enriched interac-
tive tabletop. I.J of CSCL, 8(4), pp.455-485 (2013).
22. Meier, A., Spada, H., Rummel, N. A rating scheme for assessing the quality of computer-
supported collaboration processes. I.J. of CSCL, 2(1), pp.63-86, (2007).
23. Meyer, K.A. Evaluating online discussions: Four different frames of analysis. Journal of
Asynchronous Learning Networks, 8(2), pp.101-114 (2004).
24. Newman, D.R., Webb, B., Cochrane, C. A content analysis method to measure critical think-
ing in face-to-face and computer supported group learning. Interpersonal Computing and
Technology, 3(2), pp.56-77. (1995).
25. Pena-Shaff, J., Nicholls, C. Analyzing student interactions and meaning construction in
computer bulletin board discussions. Comp. & Educ., 42(3), pp.243-265.(2004).
26. Savery, J.R., Duffy, T.M.. Problem based learning: An instructional model and its construc-
tivist framework. Educational technology, 35(5), pp.31-38. (1995).
27. Schellens, T., Valcke, M.. Collaborative learning in asynchronous discussion groups: What
about the impact on cognitive processing? Computers in Human behavior, 21(6), pp.957-
975. (2005).
28. Schneider, S., Kerwin, J., Frechtling, J., Vivari, B. Characteristics of the discussion in
online and face-to-face focus groups. Soc.Sc.Comp.Rev., 20, pp.31-42. (2002).
29. Schwarz, B.B., Asterhan, C.S.. E-moderation of synchronous discussions in educational set-
tings: A nascent practice. J. of the Learn. Sc., 20(3), pp.395-442.(2011).
30. Soller, A. Supporting social interaction in an intelligent collaborative learning system. Int.
JAIED, 12, pp.40-62.(2001).
31. Soller, A., Martinez, A., Jermann, P., Muehlenbrock, M. From mirroring to guiding: A re-
view of state of the art technology for supporting collaborative learning. Int. JAIED, 15(4),
pp.261-290.(2005).
32. Suh, H. Lee, L.. Collaborative learning agent for promoting group interaction. ETRI journal,
28(4), pp. 461-474. (2006).
33. Webb, N.M. The teacher's role in promoting collaborative dialogue in the classroom. British
Journal of Educational Psychology, 79(1), pp.1-28 (2009).