=Paper=
{{Paper
|id=Vol-1964/NLP2
|storemode=property
|title=Not Interfering: Simultaneous Typed Chat in COMPS Computer-Mediated Dialogues
|pdfUrl=https://ceur-ws.org/Vol-1964/NLP2.pdf
|volume=Vol-1964
|authors=Michael Glass,Anthony Nelson,Chinedu Emeka,Jung Hee Kim
|dblpUrl=https://dblp.org/rec/conf/maics/GlassNEK17
}}
==Not Interfering: Simultaneous Typed Chat in COMPS Computer-Mediated Dialogues==
https://ceur-ws.org/Vol-1964/NLP2.pdf
Michael Glass et al. MAICS 2017 pp. 107–113
Not Interfering: Simultaneous Typed Chat in
COMPS Computer-Mediated Dialogues
Michael Glass Anthony Nelson Chinedu Emeka Jung Hee Kim
Valparaiso U. U. of Southern Cal. Valparaiso U. North Carolina A&T State U.
michael.glass anthoncn chinedu.emeka1 jungkim@ncat.edu
@valpo.edu @usc.edu @valpo.edu
Abstract infer that a turn transition is possible. In this paper we esti-
mate how long is the pause that the other participants wait
COMPS computer-mediated typed-chat collabora-
before deciding the turn has ended.
tive learning exercises permit the students to type at
Since overlapping dialogue for more than a short time
the same time. People can see and respond to each is a phenomenon new to computer-mediated communi-
other’s text in real time. Although everybody talking
cation, this study examines overlapping dialogue manually
at the same time does not work in spoken conversa-
to see how students are using it. One possibility is that the
tion, students quickly discover they can type at the students are violating the turn-taking structure of regular dia-
same time without interfering with each other.
logue, engaging in tightly intertwined dialogue where a per-
About 40% of typing occurs while other students
son responds immediately to the words being typed by the
have not yet formally ended a dialogue turn by other person. Manual characterization of 30 such interactions
pressing “enter.” In COMPS dialogues normal con-
reveals that this doesn’t happen. We have identified three
versational turn-taking often occurs when students
common patterns of interaction, in all of them the overlap-
pause to wait for each other, without pressing “en- ping students are effectively responding to earlier dialogue.
ter.” In this paper we estimate the polite delay that
Thus they follow conversation protocols akin to normal turn-
people use for deciding when the other person has
taking even though several people are talking at once.
relinquished a turn. Studies of educational dialogue The two main results in this paper are then: a) an estimate
will have to take into account the interactions that
of the pause time that signals a possible turn-switching point
the new computer-mediated communication regime
in typed chat, and b) showing, qualitatively, examples of the
affords. This paper also characterizes the varieties three main patterns of dialogue interaction that students em-
of interaction that are observed during non-turn-tak-
ploy when using the simultaneous chat, c) show that with
ing simultaneous typing in COMPS dialogues.
these patterns of dialogue interaction students are still using
While students are typing together, they do not en- conversational turn-taking.
gage in tightly-interleaved two-way exchanges. In-
These results can be used to inform the COMPS project’s
stead, each student individually responds to some-
efforts at text analytics, which will identify characteristics of
thing that another student said earlier. the dialogue in real time that will be indicative of students
engaging in interactive conversation.
1 Introduction1
A feature of the COMPS (Computer-Mediated Problem Solv- 2 Background
ing) online chat environment is that students can all type at
once. They see each other’s words in real time and can re- 2.1 COMPS dialogues
spond at the same time without interrupting. This adds an in- The COMPS project deploys and studies small group collab-
teractive dimension that spoken language does not support. orative problem-solving exercises in college computer sci-
Most forms of human dialogue require that people take ence and mathematics classes [Kim et al., 2016]. The exer-
turns as they talk. Typed-chat lacks many of the signals such cises are designed to address student conceptual knowledge
as prosodic effects that people use to regulate turn-taking in through group cognition [Stahl, 2004; Stahl, 2009]. Typically
verbal conversation. COMPS follows the common conven- students work in groups of three, with a TA or instructor ad-
tion that pressing signals the end of a dialogue turn. ditionally participating intermittently. The exercise protocol
However the simultaneous typing feature means that pressing for the exercises requires the students to solve a problem in
is not necessary for relinquishing the turn, other peo- steps, coming to agreement on each step. The students show
ple can simply start typing. Hence people often do not press the agreed-upon answer to the instructor, receive feedback or
enter to end a turn. They simply pause, and other participants hints, then further discuss the step or proceed to the next item.
© Copyright retained by the authors
107
Not Interfering: Simultaneous Typed Chat in COMPS Computer-Mediated Dialogues pp. 107–113
A Labels 1, 2, 3, 4, 5, and 14can be instantiated anonymously. Because these do not have to be changed ◄
B What about 6 and 7?
◄
Figure 1. Example of simultaneous dialogue. End of turn is marked by “◄”.
This is consistent with accepted practices for good collabora- there sometimes isn’t even a clear separation of turns that can
tive exercise design, requiring creative interdependence be tagged as which is responding to which.
[Eberly, 2016]. This protocol also discourages social loafing, One possible step toward disentangling the Initiate/Re-
all students must participate at multiple instances during the spond problem between overlapping dialogue turns would be
exercise. to identify the Transition Relevance Places (TRPs) and Turn
Students discuss the problem through the COMPS soft- Allocational Components [Schegloff, 1990; Sacks et al.,
ware, a web-delivered chat interface that permits everybody 1974]. In Conversation Analysis, a TRP is a place where turn
to type and see each other’s dialogue all at the same time. The reallocation is possible. TRPs can occur in the dialogue when
software logs the chat for later analysis. a person stops speaking, but they can also occur, e.g., when a
Figure 1 illustrates how simultaneous chat can differ from complete thought has been finished and another person can
normal conversational turn-taking [Glass et al., 2015]. In this jump in and respond.
and subsequent figures the ending a turn is marked According to this analysis, the Figure 1 dialogue could be
with “◄”. Student A was listing widgets on a screenshot of a thought of as follows:
Java GUI, it was an answer to one of the exercise questions. A: “labels 1, 2, 3, 4, 5, and 14” ”can be …”
Part way through A’s dialogue turn student B inquired why B: “What about 6 and 7?”
certain widgets had been left out. Student A was still typing A: “Because they do not have to be changed”
and had not typed yet. A, continuing to type in the In this analysis, B starts typing in response to A’s first TRP.
same chat dialogue turn, then answered B’s question. As the In speech this could have been an interruption. In typed-chat,
Figure 1 example illustrates, students indeed adapt to this B typing does not interrupt A’s ability to type, so A and B
simultaneous chat regime and engage in productive interac- can type simultaneously. Similarly, A’s final sentence is a re-
tions while doing so [Glass et al., 2015]. sponse to B’s recently-ended turn.
Many TRPs will be determined semantically, as the first
2.2 Dialogue Turns TRP in Figure 1 after “14.” We observe that a pause in the
Potentially the instructor could be aided in knowing which typing represents a TRP, a possible place for the of a new turn
discussion groups could benefit from instructor intervention for dialogue purposes. If student A paused to read B’s ques-
and participation. The COMPS project is developing text an- tion in Figure 1, for example, detecting that pause could be
alytics for this purpose. helpful both in manual and automated dialogue analysis.
Having the computer detect and measure the prevalence Table 1. Statistics of Group Discussions.
of transactive turns in the conversation might provide an es-
timate of conversation quality for an instructor’s dashboard.
Group discussions 28
Transactive dialogue is a key element of group cognition,
where a dialogue turn a) contributes to the knowledge con-
Dialogue turns 2723
struction and b) responds to a previous dialogue move, usu-
ally by another person [Weinberger & Fischer, 2006]. There
Keystrokes (not ) 113,808
are a variety of different categories of transactive contribu-
tion, ranging from “eliciting” to “conflict-oriented consensus
Participant/group 4 (3 students, 1 TA)
building.”
We have simplified the computer-recognition task to two
Duration: Swing exercise 35 min
smaller steps: recognizing whether student dialogue turns are
discussing the topic [Willis et. al, 2017] and whether dialogue
Duration: Inheritance exercise 70 min
turns are responding to other turns [Glass et al., 2014]. For
the latter task, we adopted the conventions of Conversation
Pct of keystroke overlapping 17%
Analysis, a framework from discourse linguistics [Sacks et
al., 1974; Stubbs, 1983]. In Conversation Analysis dialogue
is segmented, each segment starting with an “Initiate” turn
and containing the other person’s “Respond” turn plus possi- 2.3 Data for this Study
ble “followup” turns. However COMPS project attempts to Table 1 shows the descriptive statistics for the chat data used
train classifiers to recognize Initiate and Respond turns had in this study. COMPS exercises have been deployed in two
difficulty partly because of many turns where people type different quantitative literacy exercises for pre-service teach-
simultaneously [Glass et al., 2014a, 2014b]. As Figure 1 il- ers and in four different object-oriented Java concept exer-
lustrates, where each dialogue turn responds to the other, cises in CS2 classes [Kim et al., 2016]. In this study we used
108
Michael Glass et al. MAICS 2017 pp. 107–113
two exercises from a CS2 class in Fall 2015. One exercise What causes people to formally end their turns is the
was about inheritance concepts in Java, the other about the COMPS scrolling text box behavior. Turns properly work
concepts in a Java SWING graphical user interface. There within the interleaved dialogue scroll only if they have been
were 14 discussions from each exercise delimited. Furthermore, a single turn containing sev-
eral logical dialogue turns concatenated together in a single
2.4 Method of Analysis stretch of text becomes hard to read. The result is that turn-
This study examined two phenomena: how long is a pause taking by pausing rarely continues past two logical turns. It
that represents a turn relevance place, and how do people use is, however, common for a turn to end with a long pause fol-
simultaneous typing to address each other. lowed by the before the new turn.
For the timing study we used data on how long did person
B wait to start typing after person A paused. This was applied 3.3 Measuring a Release Time
to all the typing in Table 1. We hypothesize that there is some time that will be rec-
For the varieties of interaction, we manually analyzed ex- ognized as meaning that the typist has given up the turn, al-
amples of simultaneous typing observed in the log files. The lowing somebody else to speak. We extended the definition
question was the relationship of each overlapping dialogue of simultaneous typing to include a release time. A partici-
turn to the rest of the conversation: was it initiating or re- pant is not inside a dialogue turn if a) the last thing typed was
sponding, and if responding where was the antecedent lan- (explicitly ending the turn) or b) a release time has
guage. During this process we maintained a coding manual elapsed (implicitly releasing the turn).
listing the varieties of interaction the annotators had found. How long after people pause typing before the other peo-
Only a small sample of about 30 of the instances have ple recognize it as a typed-chat release of turn? We can look
been manually annotated at this time according to the current at typing behavior. If there is a well-recognized time that re-
manual. The three coders periodically compared annotations leases the turn, starting typing without waiting for the release
and arrived at a consensus. Interrater reliability thus has not time to elapse would be a form of interrupting. Under this
been tested. The result reported here is qualitative, a descrip- hypothesis, we would expect to see many fewer keystrokes
tion of the categories that were found and coded. from other persons during the interval of the release time than
after it has passed. After the release time the rate of key-
3 Measuring the Pause That Signals Turn- strokes from other typists will reflect the average rate for con-
versation in general.
Taking Figure 4 (at end) shows the percent of keystrokes classed
as overlapping according to different trial values of release
3.1 Turn Allocation Using time. As expected, for larger times the curve is approximately
The explicit component for turn allocation in typed-chat linear, representing the rate of new dialogue added into the
is the which ends a turn. COMPS also has a button conversation. However for the first 2 seconds after a person
for ending a turn, the effect is the same. We start by defining pauses typing the other participants are much more reluctant
a delimited turn as the time from the first keystroke until the to type.
delimiter that ends the turn. A simultaneous key- Accordingly, for computer analysis we use 2 seconds to
stroke occurs if person B types while any other person A is declare the end of a turn and subsequent typing is non-simul-
still within a delimited turn. This is similar to the usual defi- taneous.
nition of interruption in spoken dialogue, except that B typing
does not interrupt A’s ability to type. Using this definition of 3.4 Other Means of Turn Allocation
a turn, between 25% and 50% of keystrokes in COMPS dia- We have not considered other mechanisms for turn alloca-
logues are simultaneous typing. tion. In conversation a turn can be ended, for example, by
asking a question. The same holds true for typed chat. Spoken
3.2 Ending a Turn by Pausing dialogue also employs prosodic features and other mecha-
Just as in spoken conversation a turn can end when a person nisms for turn allocation that are not available in typed chat.
stops speaking, a chat turn can also end without explicit The paucity of conventions for turn allocation in computer-
marking if the student stops typing. In spoken conversation, mediated typed chat produces quite different behavior than
3 seconds is an awkwardly long pause [McLaughlin, 1984]. spoken conversation. One experiment using a chat system
In COMPS dialogues there are many pauses. In one extreme, similar to ours observed about 30% of turns were overlapping
participant A may pause typing without pressing , with other turns, an amount of overlap not possible in speech.
waiting for the other participants to respond. Everyone else It also observed long pauses where nobody was typing, an-
can see what A has written so far, and can observe that A has other feature rarely observed in speech [Anderson, et al.,
stopped. Another participant B can then type without inter- 2010].
ference. After B pauses, A can resume, inserting a second An issue occurs if person A pauses shortly after person B
logical turn in the single -delimited turn. In this typ- begins simultaneous typing. A’s ability to type unimpeded is
ing regime, there may not be much incentive to type not compromised, however A sometimes pauses, perhaps to
to formally end a dialogue turn. read what B is saying. Clearly B’s first keystrokes constitute
simultaneous typing. After A pauses for a while, it becomes
109
Not Interfering: Simultaneous Typed Chat in COMPS Computer-Mediated Dialogues pp. 107–113
A It's only one answer for this one. It's not A. Sorry About that◄
B It’s just E.◄
Figure 2. Example of overlapping response
A i think its B D and E◄
B I agree with B,D, and E◄
C Actually no im changing to just E◄
Figure 3. Example of simultaneous response.
clear that they are no longer typing simultaneously. If A were says “Its just E.” According to our analysis, the end of A’s
to pause for only the time for few keystrokes it should prob- first sentence represents a Transition Relevance Place.
ably still be call simultaneous. In our work we use the same The Figure 1 example shows overlapped responses. Stu-
release time for this determination. Up until the 2 second re- dent B responds after the first part of A’s answer, noticing
lease time has elapsed, B’s keystrokes are still counted as that some of the numbered SWING components were not
simultaneous typing. mentioned. Later, Student A responds to B after A has paused
a short moment to read B’s question.
4 Varieties of Interaction
4.2 Simultaneous Response
The main observation is that overlapping dialogue usually ad-
Figure 3 shows an example of a simultaneous response, stu-
dresses earlier dialogue. Overlapped dialogue never in our
sample of 30 simultaneous typing incidents directly ad- dent B agreeing with student A’s suggestion and student C
disagreeing. Simultaneous responses occur at TRPs.
dressed or responded to the text that was being typed at ap-
One interesting aspect of this example is there is a three
proximately the same time. People were not simultaneously
reading, thinking, and writing. second gap between the end of A’s turn and the beginning of
both B and C. This gap is visible in the keystroke timing dia-
There are three patterns of overlapping dialogue, charac-
gram of the same dialogue turns shown in Figure 5 (at end).
terized by what parts of earlier dialogue are being addressed
by the simultaneous participants: Student A is the bottom line, B and C above it in that order.
We think this gap represents cognitive processing, rather than
• Overlapping response, where participant B responds
waiting for the release time, since A’s turn had been termi-
to something that A has recently uttered, while A
continues the same turn. nated with .
• Simultaneous response, where two participants re-
spond to an earlier turn by a different participant. 5 Discussion and Future Work
• Simultaneous initiation, where two participants There are considerable differences between computer-medi-
simply have different ideas to insert into the conver- ated typed chat and spoken communication. The ability for
sation and happen to type them in overlapped fash- multiple participants to talk simultaneously for extended pe-
ion. Both utterances fit into the conversation as of riods is new to computer communication. It has not been
the point when they started. well-studied how people use this ability as they engage in
Examples of the first two varieties follow. problem-solving dialogues.
Regarding turn allocation, it has been observed by other
4.1 Overlapping Response researchers turn allocation is not always controlled by simple
Figure 2 shows an overlapping response. The students were and pause times. People have been observed to type
asked to pick one or several from the lettered answers to a in the middle of a logical thought, for example, and
multiple choice question. The first student, who has sug- then quickly continue typing [Markman, 2013]. This circum-
gested multiple letters, corrects that by saying “It’s only one vents the turn allocation mechanism, permitting a single per-
answer for this one. It’s not A. Sorry about that.” Another son to hold the floor for longer stretches of dialogue.
student starts typing in the middle of this turn, responding to Regarding simultaneous interaction, cognitive science
the assertion that there is only one answer letter. That student suggests it is unlikely that when several people are typing
110
Michael Glass et al. MAICS 2017 pp. 107–113
simultaneously they are multitasking, viz: reading other stu- Acknowledgments
dents’ dialogue, thinking about it, and writing responses.
Generally people cannot multitask between two tasks that re- Partial support for this work was provided by the National
quire attention and cognition without switching back and Science Foundation's Improving Undergraduate STEM Ed
forth between them and degrading performance [Bermúdez, ucation (IUSE) program under Award No. 1504917. Any
2014]. Our sampling of simultaneous dialogue events is con- opinions, findings, and conclusions or recommendations ex-
sistent with this. When people chat simultaneously, they are pressed in this material are those of the author(s) and do not
interacting with events in the dialogue that occur before they necessarily reflect the views of the National Science Founda-
initiated their chat turn. tion
The manual analysis and annotation of simultaneous
events has proven to be difficult. This will have to be ad- References
dressed in order to get meaningful statistics on the prevalence Anderson, J. F., F. K. Beard, and J. B. Walther. (2010). Turn-
of categories. The main issue is that Conversation Analysis Taking and the Local Management of Conversation in a
was developed for two-person dialogues. Categorizing utter- Highly Simultaneous Computer-Mediated Communica-
ances as initiate or respond (some versions include a third tion System. Language@Internet vol. 7 article 7.
category “followup”) seems to break down when there are
more than two people in the group. In a group dialogue it be- Bermúdez, José Luis. (2014) Cognitive Science: An Intro-
comes necessary to determine whose utterance is being re- duction to the Science of the Mind. 2nd ed. Cambridge
sponded to. Once it becomes possible to have multiple re- University Press.
sponses to one utterance, the same analysis produces chains Eberly Center (2016). Carnegie Mellon, Eberly Center for
of responses. Did C respond to B’s response to A? Or did C Teaching Excellence and Innovation. What are best prac-
respond to A directly? The same utterance can often be read tices for designing group projects?
both ways. The methods of linguistic analysis of multi-party http://www.cmu.edu/teaching/designteach/design/in-
conversations hinge on this distinction. But when the dia- structionalstrategies/groupprojects/design.html. Re-
logue turns overlap, it becomes harder to resolve the ambigu- trieved March, 2016.
ities. Although there are many cases where three inde-
Glass, Michael, Jung Hee Kim, Kelvin Bryant, Melissa
pendently-working annotators readily agree, there are a sim-
Desjarlais, Micayla Goodrum, and Thomas Martin.
ilar number where agreement comes only after consensus dis- (2014a). Toward Measurement of Conversational Interac-
cussion or not at all. In the future we will try categories that
tivity in COMPS Computer Mediated Problem Solving.
do not depend on identifying the antecedent of a response.
Proceedings of the 25th Modern Artificial Intelligence
The determination of the release time will be refined by and Cognitive Science Conference (MAICS-14), Spo-
statistical analyses of the gaps between the end of one turn
kane, WA.
and the start of the next. There are other phenomena that po-
tentially affect pauses in the conversations. We could try to Glass, Michael, Jung Hee Kim, Kelvin Bryant, and Melissa
disambiguate between a cognitive reason and a social reason. Desjarlais. (2014b). Indicators of Conversational Interac-
Conversational gaps might be expected to be more prevalent tivity in COMPS Problem-Solving Dialogues, Third
in problem-solving dialogues due to cognitive processing. Workshop on Intelligent Support for Learning in Groups
The Figures 3 and 5 example of simultaneous response ex- (ISLG) at Twelfth International Conference on Intelligent
hibits just such a gap. However there is possibly a social ex- Tutoring Systems, Honolulu, Hawaii, June.
planation: with three or more participants, no one person is Glass, Michael, Jung Hee Kim, Kelvin Bryant, and Melissa
responsible for filling the gap in the conversation after one Desjarlais (2015). Come Let Us Chat Together: Simulta-
speaker pauses. We plan to study this by two methods. One neous Typed-Chat in Computer-Supported Collaborative
is by separating the conversations into segments where stu- Dialogue. Journal of Computing Sciences in Colleges, 31
dents are attending to the problem vs. attending to other mat- no. 2, Dec.
ters. We hypothesize longer pauses in the problem-solving
Kim, Jung Hee, Michael Glass, Taehee Kim, Kelvin Bryant,
segments. Another method will be to check for correlation
between the length of a pause and the length of the dialogue Angelica Willis, Ebonie McNeil, Zachery Thomas. 2016.
Student Understanding and Engagement in a Class Em-
turn which follows the pause. Longer thinking pauses might
ploying COMPS Computer Mediated Problem Solving.
result in longer utterances subsequently.
Another aspect of simultaneous chat that could be studied Modern AI and Cognitive Science Conference, Dayton,
is whether the degree of simultaneity correlates with student OH. pp. 69–74
engagement or other positive measures of dialogue quality. Markman, K. M. (2013) Conversational Coherence in Small
Anecdotally, students report being engaged by this facility. If Group Chat. In Herring, S.,D. Stein.,and T. Virtanen, eds.
degree of simultaneity correlates with some positive or nega- Pragmatics of Computer-Mediated Communication., Ber-
tive outcomes of group problem-solving chat, it would be lin: De Gruyter Mouton, pp. 539-564C
useful to put this measure on the instructor dashboard. McLaughlin, M. (1984). Conversation: How Talk is Orga-
nized. Newbury Park, CA: Sage Publication.
111
Not Interfering: Simultaneous Typed Chat in COMPS Computer-Mediated Dialogues pp. 107–113
Sacks, H., E. A. Schegloff, and G. Jefferson. (1974). A Sim-
plest Systematics for the Organization of Turn-Taking for
Conversation. Language, vol. 50, pp. 696-735.
Schegloff, E. A. (1992). On the Organization of Sequences as
a Source of “Coherence” in Talk-in-Interaction. In: B.
Dorval, ed. Conversational Organization and Its Develop-
ment, Norwood NJ: Ablex, pp. 51-77.
Stahl, Gerry (2004). Building Collaborative Knowing: Ele-
ments of a Social Theory of CSCL. In J. W. Strijbos, P.
Kirschner and R. Martens, eds., What We Know About
CSCL and Implementing It in Higher Education, Boston,
MA: Kluwer Academic Publishers, pp. 53‒86.
Stahl, Gerry, ed. (2009). Studying Virtual Math Teams,
Springer, pp. 57‒73.
Stubbs, Michael. (1983). Discourse Analysis, U. of Chicago
Press.
Weinberger, A., and F. Fischer. (2006). A Framework to An-
alyze Argumentative Knowledge Construction in Com-
puter-Supported Collaborative Learning. Computers &
Education, vol. 46 no. 1, pp. 71–95.
Willis, Angelica, Ashana Edwards, Jung Hee Kim, Kelvin
Bryant, and Michael Glass. (2017). Topic Modeling to
Detect Student Expressions of Understanding in Collabo-
rative Problem-Solving Dialogues. Poster abstracts, Thir-
tieth Florida Artificial Intelligence Research Symposium
(FLAIRS-30). (to appear)
112
Michael Glass et al. MAICS 2017 pp. 107–113
Figure 4. Percent of turns classified as simultaneous vs. pause time that would start a new turn
Figure 5. Keystroke timing diagram for dialogue in Figure 3. Student A on bottom line, Students B and C above.
113