COMPS computer-mediated typed-chat collaborative learning exercises permit the students to type at the same time. People can see and respond to each other's text in real time. Although everybody talking at the same time does not work in spoken conversation, students quickly discover they can type at the same time without interfering with each other. About 40% of typing occurs while other students have not yet formally ended a dialogue turn by pressing “enter.” In COMPS dialogues normal conversational turn-taking often occurs when students pause to wait for each other, without pressing “enter.” In this paper we estimate the polite delay that people use for deciding when the other person has relinquished a turn. Studies of educational dialogue will have to take into account the interactions that the new computer-mediated communication regime affords. This paper also characterizes the varieties of interaction that are observed during non-turn-taking simultaneous typing in COMPS dialogues. While students are typing together, they do not engage in tightly-interleaved two-way exchanges. Instead, each student individually responds to something that another student said earlier.
A feature of the COMPS (Computer-Mediated Problem Solving) online chat environment is that students can all type at once. They see each other’s words in real time and can respond at the same time without interrupting. This adds an interactive dimension that spoken language does not support.
Most forms of human dialogue require that people take turns as they talk. Typed-chat lacks many of the signals such as prosodic effects that people use to regulate turn-taking in verbal conversation. COMPS follows the common convention that pressing <enter> signals the end of a dialogue turn. However the simultaneous typing feature means that pressing <enter> is not necessary for relinquishing the turn, other people can simply start typing. Hence people often do not press enter to end a turn. They simply pause, and other participants © Copyright retained by the authors infer that a turn transition is possible. In this paper we estimate how long is the pause that the other participants wait before deciding the turn has ended.
Since overlapping dialogue for more than a short time is a phenomenon new to computer-mediated communication, this study examines overlapping dialogue manually to see how students are using it. One possibility is that the students are violating the turn-taking structure of regular dialogue, engaging in tightly intertwined dialogue where a person responds immediately to the words being typed by the other person. Manual characterization of 30 such interactions reveals that this doesn’t happen. We have identified three common patterns of interaction, in all of them the overlapping students are effectively responding to earlier dialogue. Thus they follow conversation protocols akin to normal turntaking even though several people are talking at once.
The two main results in this paper are then: a) an estimate of the pause time that signals a possible turn-switching point in typed chat, and b) showing, qualitatively, examples of the three main patterns of dialogue interaction that students employ when using the simultaneous chat, c) show that with these patterns of dialogue interaction students are still using conversational turn-taking.
These results can be used to inform the COMPS project’s efforts at text analytics, which will identify characteristics of the dialogue in real time that will be indicative of students engaging in interactive conversation. 2
The COMPS project deploys and studies small group
collaborative problem-solving exercises in college computer
science and mathematics classes [
Students discuss the problem through the COMPS software, a web-delivered chat interface that permits everybody to type and see each other’s dialogue all at the same time. The software logs the chat for later analysis.
Figure 1 illustrates how simultaneous chat can differ from
normal conversational turn-taking [
Potentially the instructor could be aided in knowing which discussion groups could benefit from instructor intervention and participation. The COMPS project is developing text analytics for this purpose.
Having the computer detect and measure the prevalence of transactive turns in the conversation might provide an estimate of conversation quality for an instructor’s dashboard. Transactive dialogue is a key element of group cognition, where a dialogue turn a) contributes to the knowledge construction and b) responds to a previous dialogue move, usually by another person [Weinberger & Fischer, 2006]. There are a variety of different categories of transactive contribution, ranging from “eliciting” to “conflict-oriented consensus building.”
We have simplified the computer-recognition task to two
smaller steps: recognizing whether student dialogue turns are
discussing the topic [Willis et. al, 2017] and whether dialogue
turns are responding to other turns [
One possible step toward disentangling the Initiate/Respond problem between overlapping dialogue turns would be to identify the Transition Relevance Places (TRPs) and Turn Allocational Components [Schegloff, 1990; Sacks et al., 1974]. In Conversation Analysis, a TRP is a place where turn reallocation is possible. TRPs can occur in the dialogue when a person stops speaking, but they can also occur, e.g., when a complete thought has been finished and another person can jump in and respond.
According to this analysis, the Figure 1 dialogue could be thought of as follows:
A: “labels 1, 2, 3, 4, 5, and 14” <TRP>”can be …” B: “What about 6 and 7?”<TRP>
A: “Because they do not have to be changed” In this analysis, B starts typing in response to A’s first TRP. In speech this could have been an interruption. In typed-chat, B typing does not interrupt A’s ability to type, so A and B can type simultaneously. Similarly, A’s final sentence is a response to B’s recently-ended turn.
Many TRPs will be determined semantically, as the first TRP in Figure 1 after “14.” We observe that a pause in the typing represents a TRP, a possible place for the of a new turn for dialogue purposes. If student A paused to read B’s question in Figure 1, for example, detecting that pause could be helpful both in manual and automated dialogue analysis.
two exercises from a CS2 class in Fall 2015. One exercise was about inheritance concepts in Java, the other about the concepts in a Java SWING graphical user interface. There were 14 discussions from each exercise
This study examined two phenomena: how long is a pause that represents a turn relevance place, and how do people use simultaneous typing to address each other.
For the timing study we used data on how long did person B wait to start typing after person A paused. This was applied to all the typing in Table 1.
For the varieties of interaction, we manually analyzed examples of simultaneous typing observed in the log files. The question was the relationship of each overlapping dialogue turn to the rest of the conversation: was it initiating or responding, and if responding where was the antecedent language. During this process we maintained a coding manual listing the varieties of interaction the annotators had found.
Only a small sample of about 30 of the instances have been manually annotated at this time according to the current manual. The three coders periodically compared annotations and arrived at a consensus. Interrater reliability thus has not been tested. The result reported here is qualitative, a description of the categories that were found and coded. 3
The explicit component for turn allocation in typed-chat is the <enter> which ends a turn. COMPS also has a button for ending a turn, the effect is the same. We start by defining a delimited turn as the time from the first keystroke until the <enter> delimiter that ends the turn. A simultaneous keystroke occurs if person B types while any other person A is still within a delimited turn. This is similar to the usual definition of interruption in spoken dialogue, except that B typing does not interrupt A’s ability to type. Using this definition of a turn, between 25% and 50% of keystrokes in COMPS dialogues are simultaneous typing.
Just as in spoken conversation a turn can end when a person
stops speaking, a chat turn can also end without explicit
marking if the student stops typing. In spoken conversation,
3 seconds is an awkwardly long pause [
What causes people to formally end their turns is the COMPS scrolling text box behavior. Turns properly work within the interleaved dialogue scroll only if they have been <enter> delimited. Furthermore, a single turn containing several logical dialogue turns concatenated together in a single stretch of text becomes hard to read. The result is that turntaking by pausing rarely continues past two logical turns. It is, however, common for a turn to end with a long pause followed by the <enter> before the new turn.
We hypothesize that there is some time that will be recognized as meaning that the typist has given up the turn, allowing somebody else to speak. We extended the definition of simultaneous typing to include a release time. A participant is not inside a dialogue turn if a) the last thing typed was <enter> (explicitly ending the turn) or b) a release time has elapsed (implicitly releasing the turn).
How long after people pause typing before the other people recognize it as a typed-chat release of turn? We can look at typing behavior. If there is a well-recognized time that releases the turn, starting typing without waiting for the release time to elapse would be a form of interrupting. Under this hypothesis, we would expect to see many fewer keystrokes from other persons during the interval of the release time than after it has passed. After the release time the rate of keystrokes from other typists will reflect the average rate for conversation in general.
Figure 4 (at end) shows the percent of keystrokes classed as overlapping according to different trial values of release time. As expected, for larger times the curve is approximately linear, representing the rate of new dialogue added into the conversation. However for the first 2 seconds after a person pauses typing the other participants are much more reluctant to type.
Accordingly, for computer analysis we use 2 seconds to declare the end of a turn and subsequent typing is non-simultaneous.
We have not considered other mechanisms for turn
allocation. In conversation a turn can be ended, for example, by
asking a question. The same holds true for typed chat. Spoken
dialogue also employs prosodic features and other
mechanisms for turn allocation that are not available in typed chat.
The paucity of conventions for turn allocation in
computermediated typed chat produces quite different behavior than
spoken conversation. One experiment using a chat system
similar to ours observed about 30% of turns were overlapping
with other turns, an amount of overlap not possible in speech.
It also observed long pauses where nobody was typing,
another feature rarely observed in speech [
An issue occurs if person A pauses shortly after person B begins simultaneous typing. A’s ability to type unimpeded is not compromised, however A sometimes pauses, perhaps to read what B is saying. Clearly B’s first keystrokes constitute simultaneous typing. After A pauses for a while, it becomes It's only one answer for this one. It's not A. Sorry About that◄ It’s just E.◄
A B
A B
C clear that they are no longer typing simultaneously. If A were to pause for only the time for few keystrokes it should probably still be call simultaneous. In our work we use the same release time for this determination. Up until the 2 second release time has elapsed, B’s keystrokes are still counted as simultaneous typing. 4
The main observation is that overlapping dialogue usually addresses earlier dialogue. Overlapped dialogue never in our sample of 30 simultaneous typing incidents directly addressed or responded to the text that was being typed at approximately the same time. People were not simultaneously reading, thinking, and writing.
There are three patterns of overlapping dialogue, characterized by what parts of earlier dialogue are being addressed by the simultaneous participants: • Overlapping response, where participant B responds to something that A has recently uttered, while A continues the same turn. • Simultaneous response, where two participants respond to an earlier turn by a different participant. • Simultaneous initiation, where two participants simply have different ideas to insert into the conversation and happen to type them in overlapped fashion. Both utterances fit into the conversation as of the point when they started.
Examples of the first two varieties follow.
says “Its just E.” According to our analysis, the end of A’s first sentence represents a Transition Relevance Place.
The Figure 1 example shows overlapped responses. Student B responds after the first part of A’s answer, noticing that some of the numbered SWING components were not mentioned. Later, Student A responds to B after A has paused a short moment to read B’s question.
Figure 3 shows an example of a simultaneous response, student B agreeing with student A’s suggestion and student C disagreeing. Simultaneous responses occur at TRPs.
One interesting aspect of this example is there is a three second gap between the end of A’s turn and the beginning of both B and C. This gap is visible in the keystroke timing diagram of the same dialogue turns shown in Figure 5 (at end). Student A is the bottom line, B and C above it in that order. We think this gap represents cognitive processing, rather than waiting for the release time, since A’s turn had been terminated with <newline>. 5
There are considerable differences between computer-mediated typed chat and spoken communication. The ability for multiple participants to talk simultaneously for extended periods is new to computer communication. It has not been well-studied how people use this ability as they engage in problem-solving dialogues.
Regarding turn allocation, it has been observed by other
researchers turn allocation is not always controlled by simple
<enter> and pause times. People have been observed to type
<enter> in the middle of a logical thought, for example, and
then quickly continue typing [
Regarding simultaneous interaction, cognitive science
suggests it is unlikely that when several people are typing
simultaneously they are multitasking, viz: reading other
students’ dialogue, thinking about it, and writing responses.
Generally people cannot multitask between two tasks that
require attention and cognition without switching back and
forth between them and degrading performance [
The manual analysis and annotation of simultaneous events has proven to be difficult. This will have to be addressed in order to get meaningful statistics on the prevalence of categories. The main issue is that Conversation Analysis was developed for two-person dialogues. Categorizing utterances as initiate or respond (some versions include a third category “followup”) seems to break down when there are more than two people in the group. In a group dialogue it becomes necessary to determine whose utterance is being responded to. Once it becomes possible to have multiple responses to one utterance, the same analysis produces chains of responses. Did C respond to B’s response to A? Or did C respond to A directly? The same utterance can often be read both ways. The methods of linguistic analysis of multi-party conversations hinge on this distinction. But when the dialogue turns overlap, it becomes harder to resolve the ambiguities. Although there are many cases where three independently-working annotators readily agree, there are a similar number where agreement comes only after consensus discussion or not at all. In the future we will try categories that do not depend on identifying the antecedent of a response.
The determination of the release time will be refined by statistical analyses of the gaps between the end of one turn and the start of the next. There are other phenomena that potentially affect pauses in the conversations. We could try to disambiguate between a cognitive reason and a social reason. Conversational gaps might be expected to be more prevalent in problem-solving dialogues due to cognitive processing. The Figures 3 and 5 example of simultaneous response exhibits just such a gap. However there is possibly a social explanation: with three or more participants, no one person is responsible for filling the gap in the conversation after one speaker pauses. We plan to study this by two methods. One is by separating the conversations into segments where students are attending to the problem vs. attending to other matters. We hypothesize longer pauses in the problem-solving segments. Another method will be to check for correlation between the length of a pause and the length of the dialogue turn which follows the pause. Longer thinking pauses might result in longer utterances subsequently.
Another aspect of simultaneous chat that could be studied is whether the degree of simultaneity correlates with student engagement or other positive measures of dialogue quality. Anecdotally, students report being engaged by this facility. If degree of simultaneity correlates with some positive or negative outcomes of group problem-solving chat, it would be useful to put this measure on the instructor dashboard.
Partial support for this work was provided by the National Science Foundation's Improving Undergraduate STEM Ed ucation (IUSE) program under Award No. 1504917. Any opinions, findings, and conclusions or recommendations expressed in this material are those of the author(s) and do not necessarily reflect the views of the National Science Foundation Sacks, H., E. A. Schegloff, and G. Jefferson. (1974). A Simplest Systematics for the Organization of Turn-Taking for Conversation. Language, vol. 50, pp. 696-735.
Schegloff, E. A. (1992). On the Organization of Sequences as a Source of “Coherence” in Talk-in-Interaction. In: B. Dorval, ed. Conversational Organization and Its Development, Norwood NJ: Ablex, pp. 51-77.
Stahl, Gerry (2004). Building Collaborative Knowing: Elements of a Social Theory of CSCL. In J. W. Strijbos, P. Kirschner and R. Martens, eds., What We Know About CSCL and Implementing It in Higher Education, Boston, MA: Kluwer Academic Publishers, pp. 53‒86.
Stahl, Gerry, ed. (2009). Studying Virtual Math Teams,
Springer, pp. 57‒73.
Stubbs, Michael. (1983). Discourse Analysis, U. of Chicago
Press.
Weinberger, A., and F. Fischer. (2006). A Framework to Analyze Argumentative Knowledge Construction in Computer-Supported Collaborative Learning. Computers & Education, vol. 46 no. 1, pp. 71–95.
Willis, Angelica, Ashana Edwards, Jung Hee Kim, Kelvin Bryant, and Michael Glass. (2017). Topic Modeling to Detect Student Expressions of Understanding in Collaborative Problem-Solving Dialogues. Poster abstracts, Thirtieth Florida Artificial Intelligence Research Symposium (FLAIRS-30). (to appear)