COMPS is a web-delivered computer-mediated problem solving environment for student collaborative exploratory learning. The primary mode of interaction is typed dialogue, but COMPS also provides problem-specific affordances for exploring a problem. This paper reports qualitatively on dialogues from students employed in four different activities: two logical reasoning problems in a quantitative literacy class and two different problems in object-oriented Java in an elementary programming class. In all domains we observe behaviors consistent with quality collaborative learning experiences: co-construction of knowledge, mixed initiative dialogue, coming to common agreement, and students adopting different roles in the problem-solving process. These observations confirm that COMPS indeed facilitates true collaborative activity.
COMPS is a web-delivered computer-mediated problem
solving environment designed for supporting
problemsolving activities in mathematics and computing
[
The goal of the COMPS project is to provide a computer-aided instrument for collaborative learning of concepts through problem-solving dialogue. We anticipate that the computer will aid the instructor, who is effectively looking over the shoulders of the students as they work, by providing a status display of progress toward solving the problem and degree of cooperative behavior.
This paper illustrates examples of dialogues collected using COMPS. Usage to date has been for testing, revising and refining the prompts for the problems and protocols, as well as collecting data in preparation for developing computer monitoring technology.
Thus there are two categories of phenomena that we look for in our dialogues. A) Observable instances of students making their thinking visible are necessary for computer monitoring. B) Observable examples of students helping each other learn are necessary for validating that the dialogues are facilitating collaboration.
The purpose of this paper is to illustrate examples of these phenomena as observed in our class usage.
Background
The student skills that are the focus of this project are
oriented toward understanding and manipulating concepts.
This is what [
Collaborative problem solving directly addresses the
goal of helping students learn from each other. Our
approach so far has employed the theory of group cognition
studied by the Virtual Math Teams project, where students
solve math problems through computer-mediated chat
[
We observed similar examples of team cognition in our
observation of students studying a nim-like game in small
groups in person [
There is also research showing that collaborative activity
is a desirable pedagogical approach specifically for
creating conceptual understanding [Tchoukine et al., 2010]. Key
to engendering learning is dialogue that engages in domain
reasoning, such as explaining, negotiating, or inferring
[
Regarding our goal of making thinking visible, discourse
pragmatics provides the theoretical justification that it
should occur in problem-solving dialogues. Koschmann's
studies of doctors in training [
Following Grice's Cooperative Principle [Grice, 1975] the important discourse obligation we observe in these dialogues is to make sure everybody is aware of your knowledge state. We claim that if you are actively participating in a problem-solving dialogue, and you don't signal your knowledge state, Grice's maximum of quality permits the implicature that you have a state approximately the same as everybody else. It would be pragmatically odd to continue listening or participating in the dialogue without communicating to the others that you have figured it out when the others haven't. Similarly it would be odd to let the others continue on to the next problem without signal ing that you still do not understand the current one.
This often holds in ordinary situations, however in pedagogical situations there are alternative explanations for a lack of grounding. For example, students may save face by not revealing their lack of understanding. One question our data will answer is how often we observe students working to achieve common understanding.
We have used COMPS with four problems in classes at NC A&T State and Valparaiso Universities. These problems were administered as approximately one hour regular class exercises in a computer lab, with the students deliberately seated far from each other. Collaboration groups were typically four students, but could be as large as six. The four problems are: 1. The Poison problem, a logic exercise usually assigned in a quantitative literacy class. Students must figure out the winning strategy for a Nimlike game where players remove one or two stones from a pile. The person to remove the last stone loses. In these exercises, the students have been elementary education majors attending the mathematics content course. We have 19 sessions from three class sections. 2. The Patagonian Congress problem, an exercise in voting mathematics. Students must figure out how proposals pass or fail in the congress under various voting procedures and numbers of people ranking the proposals in different orders. This is another quantitative literacy problem assigned in an education methods class. We have 5 sessions. 3. A puzzle in Java programming language class inheritance (Figure 1). Students must figure out the inheritance relationships between four different Java classes, based on the output of a method call. Students were in the second semester programming class. We have 13 sessions from two class sections. 4. A puzzle in Java Swing programming, deriving program and class structural relationships from the screenshot and functionality of a simple graphical user interface. Students were also in the second semester programming class. We have 10 sessions from two class sections.
The quantitative literacy exercises have on-screen manipulatives. Playing the Poison game is configured to encour age experimentation rather than competition. For example students can play any side without dividing up into teams. The Java problems use COMPS as a chat interface only.
In the quantitative literacy protocols the students are required to meet in-person as a group after about three-quarters hour of online work in order to write up their results. This mimics the normal face-to-face classroom administration of these problems. But in our experiments it meant we did not capture the end of the collaborations and their summarized results, which happened in-person. In the programming protocols one student from each group obtains the correct answers from the professor after the group agrees on an answer. Then, continuing to work online, the group has to explain the correct answer. This protocol has the effect of forcing the one student to communicate online to the others what he or she has learned, but we don't see what the professor said.
Figure 3 illustrates a dialogue solving the Poison problem. This example is noteworthy because it illustrates the power of discourse obligation, and how those obligations serve our twin goals of collaborative learning and making thinking observable. In this dialogue there is no one student setting the agenda as in the first example. And no student demands an explanation so as to achieve the same level of enlightenment as the others. However the evidence is that the students were attending to each others knowledge state anyway.
Much of the activity in the Poison dialogues is concerned with repeatedly playing the game multiple times, with occasional analysis. In this extract we pick up at dialog turn number 124, the first moment when student C figured out the first insight. She articulated this insight for everybody.
But not everybody understood the point. They played another game so C could show the others. Then they played more games, progressively smaller to make the point more evident. Student B interrupted a game to point out and explain the winning strategy to the remaining two students (turns 165 to 177).
The process of ensuring that the whole group achieved a common level of understanding, starting from turn 124, took 16 minutes out of a one hour lab. Students B and C knew that this first realization was not the entire solution, they could have proceeded to finish the problem. Ultimately they sacrificed their opportunity to finish the problem in order to aid the other two students.
Evidence that our students are actively participating in group cognition is easy to find in our dialogues.
The Java inheritance exercise in Figure 2 shows that the [Kersey, et al., 2009] experiment in initiative is incomplete. That study of programming problem collaborative dialogues found that taking the initiative correlated highly with learning gains. This held true for both dialogue initiatives (controlling discourse focus, e.g.) and task initiatives (coordinating problem-solving tasks, e.g.). In our example above, student A both controlled both the conversational focus and the problem-solving agenda, yet this student had arguably the weakest grasp of the answer.
Evidence of both student knowledge and knowledge
state is visible in almost all of our transcripts. The students
indeed make knowledge visible as [
The primary factors that explained when our dialogues lacked evidence of collaboration were: • Some of the Java problems were too easy to solve. In many groups there was little evidence of knowledge co-construction. Somebody simply said the answer. • Playing the Poison game can be too much fun.
Groups can expend a lot of time simply playing
the game without addressing the problem
statement [
These factors can be addressed by changes to the protocols and the COMPS environment.
These preliminary sessions show that COMPS usage is consistent with our goals of a) facilitating high-quality collaborative problem-solving, and b) producing visible evidence of student knowledge state and cooperative behaviors that the computer can potentially monitor.
Thank you to our hard-working students at North Carolina A&T and Valparaiso Universities. This work is supported by the National Science Foundation under awards 0851721 and 0634049 to Valparaiso University and 0633953 to North Carolina A&T State University.
What should blanks 1 – 4 contain to produce the following output:
From Foo_3 From Foo_2 From Foo_1 From Foo_3 From Foo_4 From Foo public class Foo { public static class Foo_2 extends _______ { // 1 public Foo_2() {
System.out.print("from Foo_2 "); }
} public static class Foo_1 extends _______ { // 2 public Foo_1() {
System.out.print("From Foo_1 "); } } } } } } } public static class Foo_4 extends _______ { // 3 public Foo_4() {
System.out.print("From Foo_4 "); public static class Foo_3 extends _______ { // 4 public Foo_3() {
System.out.print("From Foo_3 "); public static void main(String[] args) {
Object foo_2 = new Foo_1(); Object foo_3 = new Foo_4();
System.out.println("From Foo "); } Possible Answers: a) 1) Object 2) Foo_1 b) 1) Foo_4 2) Foo_2 c) 1) Foo_3 2) Foo_2 d) 1) Foo_3 2) Foo_2 e) None of the Above 3) Foo_2 4) Foo_3 3) Object 4) Foo_1 3) Foo_3 4) Object 3) Object 4) Foo_3