We analyze gazes during silence in multi-party conversation and compare them between conversations among nativelanguage speakers and those among second-language speakers. The duration of gaze during silence shows a significant difference between these two conditions: Gaze during silence is longer in a second-language conversation. Correlation analyses for gazes during silence and the values from questionnaire responses show unique characteristics of second-language free-flowing conversations: Gazes during silence are associated with a negative impression toward the conversation partners. Consequently, the monitoring functions of gazes during silence might affect the partners' emotional states in such second-language free-flowing conversations. These results suggest that gazes during silence in secondlanguage conversations have different functions from those in native-language conversations.
Interdependence among nations is increasing these days,
and the structure of international interaction is becoming
complex and changing dynamically. Such interdependency
is characterized by increased integration not only in trade,
capital flow, and movement of labor
Since this interdependency covers a wider area with more participants speaking a wider range of native languages, the role played by second-language conversation becomes even more important in collaboration. However, second-language conversation in international collaboration typically involves participants with different levels of linguistic proficiency in the common language, and such differences can form a barrier to equal opportunity of contribution to the collaboration (Beyene, Pamela, Hinds, & Crampton 2009). Supporting conversation among people with different levels of linguistic proficiency is thus an urgent issue in efforts to fully elicit the abilities of all participants in international collaboration.
In native-language conversation, people use non-verbal
information such as gazes and facial expressions in
conversation
In this paper, we analyze gazes during silence in both
second-language and native-language conversations. Silence
not only signals difficulty in speech production and
communication management but also reflects affection,
judgments, affirmation, negation, thought process, and so on
Our analysis of total gaze duration showed a significant difference between a native-language and a secondlanguage conversation: Gazes during silence were longer in the second-language conversation. The correlation analyses for gazes during silence and the values from a questionnaire’s responses showed unique characteristics of free conversation in a second language: Gazes during silence are associated with a negative impression toward the partners. These results suggest that gazes during silence in second-language conversations have different functions from those in native-language conversations.
We collected data from conversations in a mother tongue and those in a second language made by the same interlocutors (for details, refer to Umata et al. 2013).
Thirty university students (18 females and 12 males) between the ages of 18 and 24 years were divided into ten conversational groups of three strangers. All were native Japanese speakers whose second language was English. We measured their English communication levels based on the Test of English for International Communication (TOEIC). Participants were ranked within the group into three degrees of linguistic proficiency according to their TOEIC scores. Thus each participant had two partners of different linguistic proficiency, and we defined the participant with higher proficiency between the two as the “higher-ranked” and the one with lower proficiency as the “lower-ranked” partner.
The three participants sat 1.5 m apart in a triangular formation around a table. Each participant sat in the same position for all four of the experiment’s trials. Three sets of NAC EMR-9 head-mounted eye trackers and headsets with microphones recorded their eye gazes and voices. The participants talked about two predetermined topics in English (second language) and in Japanese (mother tongue). Each group participated in two conversations in each language.
One of the two conversational topics was assigned before each trial. The first was a free-flowing one in which they chatted about their favorite foods. The second was a goaloriented task in which they collaboratively decided what to take with them on a trip to a deserted island or the mountains. We randomly arranged the order of the conversation topics to counterbalance any order effect. We also randomly arranged the order of the languages used in the conversations. Each group had six-minute conversations on the free-flowing and goal-oriented topics in both Japanese and English. The participants filled out questionnaires after each of their four conversations. We analyzed the data from the free-flowing and goal-oriented conversations in Japanese and English.
From the fourth group on, the participants were asked to fill out the questionnaire after each session. Consequently, we analyzed the submitted questionnaires for seven of the ten groups. The questionnaire required participants to express their interactional attitudes in each conversation, and it consisted of 29 items, each of which was ranked on a Likert scale from 1 (negative to the question) to 7 (positive to the question). Each item’s question was categorized into such communicational features as participant-gazing activities, feelings toward other participants, interest in the conversation topic, conversational skills in English, and evaluation of the conversation content.
The utterances were transcribed, and annotators manually annotated the time spans for the utterances and gazes at other participants to integrate the utterance and eye gaze data. Utterances were divided by pauses of more than 500 msec, and intervals where no utterance took place were regarded as silence. Since we failed to record the eye gaze data of two participants due to equipment trouble, they were excluded from the analyses. We used the EUDICO Linguistic Annotator (ELAN) developed by the Max Planck Institute as an annotation tool.
A N A LY S ES
We expected that the total duration of silence would be longer in the second-language conversations than in the native-language conversations, due to the higher possibility of communicative problems in turn management, speech production, and mutual understanding in a second-language conversation. We also expected that differences in topic would affect the duration of silence: Goal-oriented conversations would show a longer duration of silence because participants must organize their utterances to match the needs of their task. We conducted an ANOVA for the total duration of silence within groups, with language difference and topic difference being within-subject factors. The results showed a main effect of language difference (F(1, 9) = 59.5, p < .001) and a main effect of topic difference (F(1.9) = 25.9. p < .01), and no interactions were observed. As expected, the total duration of silence was significantly longer in the second-language conversations and the goaloriented conversations (Table 1).
Table 1: Duration of silence (sec.)
Goal-oriented (JAP)
A previous study found that the duration of the listeners’
gazes during their partners’ speech was significantly longer
in second-language conversations than in native-language
ones
First, the average being gazed at ratio during silence is defined as ! !!! () = ! ×100(%)
!!! ()
Here, S(i) is the duration of the i-th silence, and DOS(i) is the duration when other participants are looking at the participant in the i-th silence.
The average gazing at ratio during silence is defined as ! !!! () = ! ×100 %
!!! ()
Here, DSO(i) is the duration when the participant is looking at other participants in the i-th silence.
We categorized each participant’s gaze during silence into two categories: 1) gaze toward the higher-ranked of the two partners and 2) gaze toward the lower-ranked partner. We conducted an ANOVA for gaze during silence, with language difference, topic difference, and gaze category (toward higher/lower partner) as the within-subject factors. The results revealed a significant main effect of language (F(1, 29) = 5.175, p < .05), but no other main effect or interaction was observed.
Table 2: Average duration of gaze during silence
30 Free-flowing (JPN) to 30.60 11.14 30 lower-ranked Goal-oriented (JPN) to 30.53 16.61 30 higher-ranked Goal-oriented (JPN) to 28.41 12.73 30 lower-ranked Free-flowing (ENG) to 33.76 10.50 30 higher-ranked Free-flowing (ENG) to 31.93 11.52 30 lower-ranked Goal-oriented (ENG) to 32.27 15.93 30 higher-ranked Goal-oriented (ENG) to 31.12 12.18 30 lower-ranked The average duration of gaze during silence was significantly longer in the second-language conversations as expected, but the topic difference did not affect the gaze duration during silence (Table 2).
A nal y s i s III: Co rre l ati o n A nal y s e s o f Gaz e duri ng S i l e nc e
The duration of gaze during silence was significantly longer in the second-language conversations than in the native-language conversations. This result suggests that there may be some functional difference in gaze during silence between these two kinds of conversations. We conducted Spearman’s correlation analysis on gaze during silence, gaze during speech, and questionnaire items. The items that exhibited significant correlation are shown with correlation values (Spearman’s ρ) in the following tables, where a single asterisk * denotes p < .05, a double asterisk ** denotes p < .01, a sharp # denotes p < .1, and n.s. denotes no significant correlation.
There were positive significant or marginally significant correlations between the duration of being gazed at by the higher-ranked partner and that by the lower-ranked partner, in both the native-language and second-language conversations in free-flowing conversation and in secondlanguage conversation in goal-oriented conversation. Fre e - f l o w i ng C o nv e rs at i o n Native-language conversation: Being Gazed at by higher-ranked partner during silence <-> Being Gazed at by lower-ranked partner during silence .650**
Being Gazed at by higher-ranked partner during silence <-> Being Gazed at by lower-ranked partner during silence G o al - o ri e nt e d C o nv e rs at i o n .630**
Being Gazed at by higher-ranked partner during silence <-> Being Gazed at by lower-ranked partner during silence .747** There was an interesting negative correlation between the durations of gazing at the higher-ranked partner and being gazed at by the higher-ranked partner only in the secondlanguage free-flowing conversations, but no such correlation was found in the other direction (i.e. gazing at the lowerranked partner and being gazed at by the lower-ranked partner).
Fre e - f l o w i ng C o nv e rs at i o n Second-language conversation: Gazing at higher-ranked partner during silence s<i-l>encBeeing Gazed at by higher-ranked partner during -.539**
We conducted correlation analyses for the duration of gazes during utterances and those during silence. First, the average being gazed at ratio while speaking is defined as ℎ ! !!! () = ! ×100(%)
!!! ()
Here, D(i) is the duration of the i-th utterance, and DPOS(i) is the duration when other participants are looking at the speaker in the i-th utterance.
The average gazing at ratio while speaking is defined as ℎ ! !!! () = ! ×100 %
!!! ()
Here, DSOP(i) is the duration when the speaker is looking at other participants in the i-th utterance.
The duration of gaze during silence showed correlations with the duration of gaze during speech for both gazing categories (toward higher-ranked/lower-ranked) in both language conditions.
Free-flowing Conversation Native-language conversation:
Gazing at higher-ranked partner during silence spe<a-k>ingGazing at higher-ranked partner while .669**
Gazing at lower-ranked partner during silence <-> Gazing at lower-ranked partner while speaking
Gazing at higher-ranked partner during silence .556**
<-> speaking
Gazing at lower-ranked partner during silence <-> Gazing at lower-ranked partner while speaking
Goal-oriented Conversation Native-language conversation: Gazing at higher-ranked partner during silence <-> Gazing at higher-ranked partner while speaking
Gazing at lower-ranked partner during silence <-> Gazing at lower-ranked partner while speaking
Gazing at higher-ranked partner during silence <-> Gazing at higher-ranked partner while speaking
Gazing at lower-ranked partner during silence <-> Gazing at lower-ranked partner while speaking .605** .725** .792** .782** .695** .697**
Second-language free-flowing conversations showed unique characteristics for the correlations between gaze during silence and values from the questionnaire responses.
The duration of being gazed at by the partners and the self-evaluation of content understanding showed interesting correlation in the second-language free-flowing conversations but not in the native-language ones nor in the goal-oriented conversations.
Fre e - f l o w i ng C o nv e rs at i o n Second-language conversation: Being Gazed at by higher-ranked partner during silence d<i-s>couDrose?you think you could understand his/her -.407*
Being Gazed at by lower-ranked partner during silence <-> Do you think you could understand his/her discourse? -.474* The values for evaluating the mental concentration of the higher-ranked partner also showed a negative correlation with being gazed at during silence by the higher-ranked partner in the second-language free-flowing conversations, but this was not the case for these values of the lowerranked partner or in the native-language conversations. Fre e - f l o w i ng C o nv e rs at i o n Second-language conversation: Being Gazed at by higher-ranked partner during silence <-> Do you think your partner concentrated when -.465* he/she spoke?
The duration of being gazed at by the higher-ranked partner showed a negative correlation with the selfevaluation of interest only in the second-language freeflowing conversation.
Free-flowing Conversation Second-language conversation: Being Gazed at by higher-ranked partner during silence <-> Did you become interested in the discourse of your partner? -.424*
Gaze during silence and feelings toward partners also showed interesting correlations only in the second-language free-flowing conversations. Gaze from the lower-ranked partner and favorable impression toward this partner showed a negative correlation.
Free-flowing Conversation Second-language conversation: Being Gazed at by lower-ranked partner during silence <-> Did you like your partner? -.571**
D IS CU S S ION We confirmed that the total duration of silence is longer in second-language conversations than in native-language conversations. This result apparently reflects the communication difficulties in second-language conversations. Differences in topic also affect the duration of silence, suggesting that managing utterance production while attempting to contribute to a specific task makes the communication more difficult.
Then we compared the durations of gaze during silence in the native-language and in the second-language conversations. An ANOVA showed a significant main effect of language difference, suggesting that gaze during silence might play different roles in native-language and second-language conversations.
The correlation analyses showed interesting characteristics of second-language free-flowing conversations. There was a negative correlation between the duration of gazing at the higher-ranked partner and being gazed at by the higher-ranked partner only in the secondlanguage free-flowing conversations. The results from questionnaire analyses discussed below suggest that the gazes from the higher-ranked partner are associated with negative evaluations toward the partner, and this might be one of the causes of this phenomenon. Although these causes are not yet clear, the results seem to indicate a functional difference between gazes in native-language conversations and those in second-language conversations. Content analyses of the utterances are now in progress to clarify the causes.
Correlation analyses for gazes during silence and the values from the questionnaire responses also showed interesting results. The durations of being gazed at by the higher and lower partners during silence and the values of content understanding by self-evaluation showed significant negative correlations only in the second-language freeflowing conversations. Such a result suggests that the participants who were gazed at more by their partners felt that they could not understand the partners’ discourse. This may reflect the monitoring function of the gaze: the participants gazed at their partners to check their degree of understanding in the second-language conversations. The reason why this happens only in free-flowing conversations is not clear either, but one possibility is that the need to check the partners’ understanding is stronger in free-flowing conversations because the contents of the next utterance are less predictable, whereas the contents are expected to follow along a sequence of task requirements in goal-oriented conversations. We are now conducting analyses of the function of each utterance.
Duration of being gazed at by the higher-ranked partner showed significant negative correlations with the results of questionnaire items evaluating favorable impression, interest in the partner’s discourse, and the partner’s concentration only in the second-language free-flowing conversations. These results suggest that the gazes from the higher-ranked partner during silence are associated with negative evaluations toward the partner in the secondlanguage free-flowing conversations but not in the nativelanguage conversations or in the second-language goaloriented conversations. One possible cause of this may be the monitoring function of gaze in the second-language conversations: People who had their understanding checked closely by partners might not have had a good impression toward them.
The analyses in this paper were conducted for each
participant, and no intra-group structure was observed due
to the insufficient amount of data. For the same reason, the
correlation analyses of gazes were done for each question
item rather than subscales. We have recorded 10 additional
groups and are now processing the data. We are planning to
conduct multilevel analyses that also consider intra-group
factors
CON CLU S ION
We examined gaze during silence in native-language and second-language conversations. The duration of such gaze showed a significant difference between the two language conditions as we expected, suggesting that there are functional differences between gazes during silence in a second-language conversation and those in a nativelanguage conversation.
Correlation analyses showed interesting characteristics of second-language free-flowing conversations. Gazes from the partner with higher linguistic proficiency are associated with negative evaluations toward that partner. The results suggest that the monitoring function of gazes during silence might affect the partner’s emotional state in secondlanguage free-flowing conversations. Content analyses are now being conducted to confirm this possibility.
A c kno w l e dg me nts This research was supported in part by a grant from the Japan Society for the Promotion of Science (JSPS) (No. 22520598, 15K00293). The authors also thank Professor Kristiina Jokinen of the University of Helsinki and Professor Mariko Sugawara of Doshisha University for their suggestions and discussions. We also deeply appreciate the reviewers for their valuable suggestions and comments.
Re f e re nc e s