In group discussion, it is not always easy for the participants to effectively control the discussion to make it fruitful. With the goal of contributing to facilitating group discussions, this study proposes a method of segmenting a discussion. Predicted discussion boundaries may be useful for tracking the discussion topics, analyzing the discussion structure, and determining a timing for intervention. We created a multimodal embedding space using an autoencoder, and represented each multimodal utterance data in the embedding space. Then, a simple unsupervised approach was used to detect the discussion boundary. In a preliminary analysis, we found that the proposed method can generate discussion segments that are comprehensible for analyzing a discourse structure. But, the performance in the discourse segmentation task should be improved as future work.
INTRODUCTION Group discussion is widely used for decision-making and idea generation. However, it is not always easy for the participants to effectively control the discussion by themselves. A facilitator is a person who helps the participants establish common understanding and reach consensus during the conversation. In order to make an effective contribution, the facilitator needs to choose a right timing for intervening to the discussion while observing the discussion. Thus, for the purpose of exploiting information technology in supporting a discussion, tracking a discussion is one of the basic function for a computer system to facilitate the discussion.
There were many previous studies for topic tracking and discourse segmentation. There were mainly two approaches in this research area. Unsupervised approach is based on lexical cohesion, such as identical words, synonyms, and hypernyms [1, 2]. Discourse boundary is determined by the © 2018. Copyright for the individual papers remains with the authors. Copying permitted for private and academic purposes.
SymCollab '18, March 11, Tokyo, Japan. concise similarity between word vectors. The other approach is the supervised approach, where a set of features are calculated and a classifier is learned to decide a boundary or non-boundary [3]. While the motivation of these previous studies is to use discourse boundaries to identify more informative segments, retrieve specific information more accurately, and generate a summary of the discourse. The purpose of discourse segmentation in this study is slightly different. We aim to identify discussion boundaries, each of which is a kind of shift in the discussion and may be an appropriate intervention timing for facilitation. Thus, each discourse segment divided by a boundary should be a coherent discourse.
Moreover, group discussion is not well-structured compared to texts, and discussion segmentation would be more difficult than text segmentation. The discussion sometimes does not go straightforwardly, and the same topic may be discussed multiple times. As more closely related work, [4, 5] proposed a discourse segmentation model by employing a featurebased supervised classification approach.
However, feature selection is a painful process. In this study, we employ an autoencoder to learn multimodal embedding space to represent each utterance as a vector. The advantage of this approach is that feature selection is not necessary. Then, we employ unsupervised approach to decide a discourse boundary by calculating cosine similarity between the vectors.
GROUP DISCUSSION CORPUS
Task and Subjects
We recruited 30 subjects (10 groups of 3 people), who were
native Japanese speakers. They participated in a group
discussion for 30 minutes to create a one-day travel plan for
foreigners. The group of participants cooperatively filled in
a work sheet in which they described (
Experimental Environment Figure 1 shows a snapshot of the experiment. Three people were seated at a table, and each of them wore a head set microphone (Audio-technica HYP-190H) to record speech data. Inertial Motion Unit (IMU, ATR-Promotions: WAA010) were attached to the back of each participant’s head. These sensors measured head acceleration, angular velocity, and terrestrial magnetism in the x, y, and z coordinates at 20 fps. A Kinect sensor placed on the other side of each participant was used to collect face tracking data individually1. In addition, two video cameras were set to record the overview of the communication. Speech data were manually transcribed.
MULTIMODAL EMBEDDING SPACE From the speech audio2, we obtained 7052 utterances, for each of which we calculated following verbal and nonverbal features.
Features
(
ܣܪ ൌ ට ݔ ଶ ݕ ଶ ݖ ଶ Then, maximum, minimum, average, and variance were computed for each participant per utterance. (9) Wavelet features for the composite head acceleration: This feature is used for measuring the synchrony of the head motions between discussion participants. Multiresolution analysis with Daubechies wavelets [6] was applied to the composite acceleration calculated in (8). Then, maximum, minimum, average, and variance were computed for a wavelet at the highest resolution. (10) Doc2Vec features: A Doc2Vec [6] model, which was trained by using Wikipedia articles written in Japanese, was applied to each utterance, and a 200-dimensional vector was obtained. All elements of the vector were used as features. LEARNING A MULTIMODAL EMBEDDING SPACE All the features described in the previous section were concatenated, and each utterance was represented as a 214dimentional vector, including 12-dimentions for Wavelet analysis, 4-dimentions for speech intensity, 4-dimentions for head rotation, and 200-dimentions for Doc2Vec features. We 2 One group was excluded from the analysis because the speech audio was not recorded by mistake.
4 5 6 4 5 6 0.52 0.60 0.67 0.47 0.57 0.65
0.70 0.52 0.62 0.70 0.55 0.62 0.68 0.50 0.60 0.67 used this 214-dimentional vectors as the input to an autoencoder.
We built an autoencoder consisting of one input layer, one hidden layer, and one output layer. We used ReLu as the activation function in the hidden layer, and a linear function for output layer. Minimum square error was used in the cost function. The 241-dimensionnal data in the input layer was reduced to150-dimensions in the hidden layer. The data from 7 out of 9 groups (4044 utterances) were used for training, and the data from the remaining two groups (1124 utterances) were used for testing.
ANALYSIS The test data obtained from two groups were used in the following analysis. Each utterance was represented as an output vector from the autoencoder. Then, the cosine similarity values were calculated by pairing the current utterance with the previous three utterances, and the average of three similarity values was calculated. If the average similarity with the recent three utterances was lower than 0.75, the current utterance was identified as a discussion boundary.
Coherence In order to test the coherence of each discussion segment, we calculated the lexical similarity between the segments. First, a word vector was generated for each segment by extracting nouns and verbs from the transcription. Then, the cosine similarity was calculated for all the pairs of segments. The cosine similarity was generally very low. In more than 90% of the pairs, the cosine similarity is lower than 0.2. While in 0.7% of the pairs, the similarity was over 0.5, the content of these segments was quite similar to each other (e.g. discussing the same place). These results suggest that each discussion segment had enough lexical coherence. As a qualitative analysis, we visualized the structure of the discussion based on the segments obtained. Figure 2 visualizes the structure of a discussion of Group2. Labels, such as G2-1, indicate a discussion segment. As shown in the diagram, the main stream of the discussion can be easily interpreted: starting from determining the expected travelers, followed by the discussion about the catchphrase and the visiting spots. In addition, it was also possible to assign subG2‐21 G2‐24 G2‐27 topics for some segments. For instance the topic “Midtown” (G2-16) has four sub-topics: “route”, “evaluation”, “staying time”, and “reason.” This suggests that the results of automatic segmentation is comprehensible for a human analyzer, and there is a good possibility that such segmentation is useful for supporting a human facilitator. Agreement with the segmentation by a human annotator As a preliminary analysis, we compared the result of automatic segmentation with the segmentation by a human annotator. For the last part of the group work, the participants mainly worked on filling out the task sheet, and the interaction is very different from other parts. Thus, we did not use the data for this part. So we used 404 utterances from the Group2 discussion, and 453 from the Group7 discussion. The model detected 58 boundaries for the Group2 discussion, and 79 for the Group7 discussion, while the human annotator detected 56 and 55 respectively. In order to permit near miss judgment, we set a tolerance window of size n, and judged that the model prediction was correct if there was a boundary or no boundary within the window for both model prediction and human judgement. With this tolerated agreement measure, we calculated precision, recall, and F-measure. Table 1shows the evaluation results of the proposed model, and Table 2 shows the evaluation results of a model only using Doc2Vec features. These two models were created to compare a language-based model and a multimodal model. As the average segment length in the human annotation was 7.8, we assume that window size n=4 (half the average segment size) may be reasonable. Although the model performance should definitely be improved, the multimodal model outperformed the language-based unimodal model for all window sizes. As our final goal is not finding discourse boundaries, but identifying a good timing for intervention, we need to propose more appropriate evaluation metrics as future work.
CONCLUSION This study proposed a method for finding discussion boundaries based on the similarity of utterance vectors in a multimodal embedding space which was created by using an autoencoder. Although the performance in the discourse segmentation task is not good enough, the proposed method can generate segments that are comprehensible for interpreting a discourse structure.
As future directions, we will test the model in terms of determining a timing for intervention. In addition, it is necessary to improve the model for tracking the topics of discussion.
ACKNOWLEDGMENTS This work was supported by CREST, JST.