Toward Adaptive Unsupervised Dialogue Act Classification in Tutoring by Gender and Self-Efficacy Aysu Ezen-Can Kristy Elizabeth Boyer Department of Computer Science Department of Computer Science North Carolina State University North Carolina State University aezen@ncsu.edu keboyer@ncsu.edu ABSTRACT intensive due to the manual annotation and handcrafted dialogue For tutorial dialogue systems, classifying the dialogue act (such as act taxonomies that are usually domain-specific. To overcome questions, requests for feedback, and statements) of student these challenges, unsupervised dialogue act modeling techniques natural language utterances is a central challenge. Recently, including hidden Markov models [20, 21, 30], Dirichlet Process unsupervised machine learning approaches are showing great clustering [12, 23], k-means clustering [31], and query-likelihood promise; however, these models still have much room for clustering [15] have been investigated in recent years. improvement in terms of accuracy. To address this challenge, this Despite this growing focus on developing unsupervised dialogue paper presents a new unsupervised dialogue act modeling act classifiers, these models still underperform compared to approach that leverages non-cognitive factors of gender and self- supervised approaches in their accuracy for classifying according efficacy to better model students’ utterances during tutorial to manual tags. However, while unsupervised models to date have dialogue. The experimental findings show that for females, considered such things as lexical features (the words found in the leveraging learner characteristics within dialogue act classification utterance) and syntactic features (the structure of the sentence), significantly improves performance of the models, producing they have not considered non-cognitive factors, such as gender better accuracy. This line of investigation will inform the design and self-efficacy, which are believed to influence the structure of of next-generation tutorial dialogue systems, which leverage tutorial dialogue [10]. Cognitive factors such as skill mastery has machine-learned models to adapt to their users with the help of been widely studied in learning environments. However, there is a non-cognitive factors. smaller body of work on adaptive learning environments using non-cognitive factors. A variety of learner characteristics, Keywords including non-cognitive factors, play an influential role in Tutorial dialogue, learner characteristics, dialogue act learning, not only in tutoring but in classroom settings [1], and in classification, unsupervised machine learning, adaptive learning. web-based courses [19]. Prior work on learner characteristics has focused on building adaptive systems based on different user 1. INTRODUCTION groups [16], tutorial feedback selection [9] and identifying Tutorial dialogue is a highly effective form of instruction, and students that need remedial support [27]. Identifying clusters of much of its benefit is thought to be gained from the rich natural student characteristics is also an active area of research [4, 25– language dialogue exchanged between tutor and student [7, 17, 27]. 36]. In order to model tutorial dialogue for the purposes of This paper investigates whether the performance of an building tutorial systems or for studying human tutoring, dialogue unsupervised dialogue act classifier can be improved by taking acts, which capture both cognitive and non-cognitive aspects of these factors into account. Because non-cognitive factors are dialogue utterances, provide a valuable level of representation. shown to affect language, we believe that training dialogue act Dialogue acts represent the underlying intention of utterances (for classifiers tailored to specific learner characteristics can help example, to ask a question, agree or disagree, or to give a tutorial dialogue systems to understand students better. We utilize command) [3, 32]. Within the computational linguistics and two learner characteristics: gender, as self-reported by students on dialogue systems literature, automatically classifying dialogue a survey and domain-specific self-efficacy, as measured by a acts has been a focus of research for several decades [6, 14, 35]. validated instrument for determining a student’s confidence in her For tutorial dialogue systems, dialogue act classification is crucial own abilities. Specifically, we train unsupervised dialogue act to understanding students’ utterances and developing tutorial models that are tailored to students of specific gender and self- strategies [8, 24]. efficacy level, and we compare those models to corresponding Today’s tutorial dialogue systems utilize a variety of dialogue act ones trained without restricting by that learner characteristic. This classification strategies, some rule-based and some statistical [13]. unsupervised training is conducted entirely without the use of Historically when machine learning has been used to devise manual tags. We then test all of the models on held-out test sets tutorial dialogue classifiers, these have been supervised within leave-one-student-out cross validation, and compare the classifiers, which require training on a manually labeled corpus. resulting classification accuracy according to their previously The same is true within the broader dialogue systems research applied manual tags. The results show that for female students, community: dialogue act classifiers have historically either been utilizing learner characteristics statistically significantly improves handcrafted and rule-based, or learned with supervised machine dialogue act classification models. For self-efficacy groups, learning techniques [11, 14, 22, 29]. However, supervised improvement is observed but not at a statistically reliable level. techniques face substantial limitations in that they are labor- This paper constitutes the first research toward incorporating non- cognitive factors into unsupervised dialogue act classifiers for tutorial dialogue with the overarching goal of providing questionnaire from which gender was obtained. For self-efficacy, personalized learning for students. We first administered a survey students were divided into classes based on the median score to collect these characteristics via self-report, and then learned a across all students on that scale. Along with gender, this produces dialogue act classifier tailored to those characteristics. These two partitions of the 42 students: females (12) and males (30), low results can inform the way that next-generation tutorial dialogue (24) and high self-efficacy students (18). systems conduct their real-time dialogue act classification and Table 1: Excerpt of dialogue with a male student language adaptation. in the low self-efficacy group 2. RELATED WORK Dialogue   Dialogue act modeling is an important level of representation Role   Utterance   Act   within dialogue systems. Following theories proposed several Tutor   You'll need to end every Java S decades ago within philosophy and linguistics [3, 32], dialogue statement with a semi colon   act classification aims to capture the intention of an utterance; for Student   Got it!   ACK example, in tutoring some dialogue acts involve asking questions Tutor   This is to let Java know where each S or giving or requesting feedback. While a long-standing line of statement ends   investigation has focused on handcrafted or supervised machine Tutor   Ah no prompt!   S learning techniques for dialogue act classification [11, 14, 22, 29], Tutor   Why do you think that is?   Q only recently is a body of work emerging on unsupervised Student   I wish I knew...   A approaches to this problem. Most of this work has been done Student   I don't think I spelled anything wrong   S outside of educational domains, with a proposed hidden Markov Tutor   Ah it's actually pretty easy   S model in the domains of Twitter posts [30] and emails [21], Tutor   The order of the lines matters   S Dirichlet Process Mixture Models for a train fare dialogue domain [12] and for navigating buildings [23], and a Chinese Restaurant Process approach for spoken Japanese [20]. The corpus containing 1640 student utterances was manually annotated with dialogue act tags in previous work [18] (Table 2). Another important difference between the current work and prior These dialogue act tags are not available during model training, research is in the features used, namely the non-cognitive but we use them for evaluation purposes to calculate accuracy on characteristics of gender and self-efficacy. Prior work has used a a held-out testing set. variety of features for performing supervised dialogue act classification, including prosodic and acoustic features which Table 2: Student dialogue acts and distributions involve the profile of the sound signal itself [35], lexical features Student Dialogue Act Example Distribution such as words and sequences of words [34], syntactic features A (answer) yeah I'm ready! 39.95% including part-of-speech tags [6, 24], dialogue structure features such as taking the initiative and the previous dialogue act [33] as ACK Alright 21.31% well as task/subtask features in tutorial dialogue [8, 18]. Within (acknowledgement) unsupervised dialogue act classification a subset of these features S (statement) i am taking basic fortran 21.20% have also been used such as words [12], state transition right now never seen probabilities in Markov models [23], topic words [30], function literal before words [15], a smaller subset of words containing beginning Q (question) what does that mean? 15.15% portions of utterances [31], part-of-speech tags and dependency trees [21]. While a variety of experiments have demonstrated the RF (request feedback) better? 0.98% utility of these features in several domains, no prior work has reported on an attempt to include the factors considered here, in C (clarification) *html messing 0.79% order to improve the performance of an unsupervised dialogue act O (other) haha 0.61% classifier. To investigate this, we build dialogue act classifiers that learn from utterances of specific learner groups and predict dialogue acts of students according to their learner characteristics. 4. DIALOGUE ACT MODELING BASED 3. CORPUS ON LEARNER CHARACTERISTICS The corpus used in this study consists of student-tutor interactions We hypothesize that dialogue act models built using unsupervised in an introductory computer science programming task [18]. machine learning will perform substantially better when Throughout the data collection, freshman engineering students customized to specific learner groups. Specifically, we investigate and tutors communicated through a textual dialogue-based whether by training a model only on students of a particular learning environment while working on Java programming. The learner characteristic, that model would perform significantly ethnicity of students participated in this study is distributed as better at predicting the dialogue acts of unseen students with the follows: 26 white, 9 Asian, 3 Latino, 2 African American, 1 same learner characteristic compared to a model that was trained Middle Eastern and 1 Asian American. An excerpt from the on students of all learner characteristics. corpus is shown in Table 1. We note that because the same corpus is being partitioned in two Students were given a pre-survey that included survey items on different ways, the same student will occur in one of the gender computer science self-efficacy, such as ‘I am sure I can learn groups and in one of the self-efficacy groups. This choice to programming’. This self-efficacy scale was adapted directly from partition in 2-way splits rather than 2n-way splits where n is the the Domain-specific Self-Efficacy Scale [5], with five items number of learner characteristics is because of issues that arise measured on a Likert scale from 1-5 (1 being lowest self-efficacy, with sparsity. This interdependence between partitions is a 5 being highest). Students also completed a demographic limitation to note; however, as discussed in Section 5, this interdependence can be taken into account for making decisions data points to clusters according to their closest centroids until within a tutorial dialogue system by employing a suite of convergence upon the model. In the standard k-medoids classifiers within a voting scheme. algorithm, the seeds are randomly selected. However, we employ a greedy seed selection approach intended to mitigate the effects 4.1 Experimental Design of the unbalanced distribution of dialogue acts in the corpus [2]. For gender and self-efficacy, we will test whether an unsupervised Within this greedy seed selection, an initial seed is randomly dialogue act classifier trained only on students with that selected and then each of the subsequent seeds are selected by characteristic outperforms a classifier that is not specialized by choosing the point that maximizes its distance from the already- this characteristic. In order to gather accuracy data across these selected seeds. The goal in using this approach is to select the characteristics, we conduct leave-one-student-out training and seeds from diverse utterances so the algorithm produces better testing folds. The testing set for each of the n folds (where n clusters, and our initial experiments indicated that it substantially varies depending on which learner group is being considered) improves the model. consists of all of a single student’s dialogue utterances and the model is trained on the remaining n-1 students. The average In addition to its seeding approach, the k-medoids approach number of utterances per student in the corpus is 36.8 (σ=12.07; requires the number of clusters k to be set prior to model training. min=16; max=64). These are therefore the average, minimum, and To discover the number of clusters, we experimented with X- maximum number of utterances across the leave-one-student-out Means and Expectation Maximization clustering, both of which test sets. attempt to identify the optimal number of clusters. Both of these algorithms converged at four clusters as the optimal choice, so we We compute the average test set performance of the model across proceed with k=4. However, perhaps in part due to the benefit of all folds for each non-cognitive characteristic partition. The the greedy seed selection made possible by k-medoids, these performance metric utilized in this study is accuracy compared to models performed with substantially worse overall accuracy than the manually labeled dialogue acts described in the previous k-medoids. section, where accuracy is computed as the number of utterances in the test set that were classified according to their manual label, The utterances were represented as vectors with each column divided by the number of utterances total in the test set. As matching a token (punctuation and words) in the corpus and each described in 4.2, the process of labeling via unsupervised row matching an utterance. There were a total of 877 distinct classification involves taking the majority vote within each tokens. cluster. With these parameters in place, first the clusters were formed For constructing the folds, we take an approach to balance the using each training set, and then for each utterance of the student sample size available to model training. This balancing approach held out within the leave-one-student-out fold, we computed the is needed to ensure that each model is trained on a similar size of closest cluster to that utterance as indicated by average cosine data. Consider, for example, the partition of gender. Without a distance to each point in the cluster. The closest cluster was balanced sampling approach the leave-one-student-out testing selected as the cluster to which the test utterance belongs, and the folds for the un-specialized classifier for female students would majority vote of the cluster was assigned to the test utterance as its include nfemale=12 test folds but the available data for each training dialogue act label. For each leave-one-student-out testing fold, the fold would be ntotal-1 = 41. In contrast, the specialized classifier accuracy was computed by comparing these cluster-assigned trained only on female students would still include nfemale=12 test labels to the manual dialogue act tags. points but the available data for each training fold would be 4.3 Experimental Results nfemale-1 = 11. Therefore, each un-specialized classifier was trained This section presents experimental results for unsupervised on a randomly selected subset of the corpus. In the case of dialogue act classification based on learner characteristics. We females, each of the 12 testing folds will utilize a model trained compare each model built separately by gender and self-efficacy on 11 data points. The specialized classifier will use 11 female level to the models that are built using utterances from randomly data points, and the un-specialized classifier will use 11 randomly selected students, i.e. not utilizing learner characteristic selected data points. In this way, we investigate how well a model information. Each comparison in this section is conducted with a predicts dialogue acts of a student with and without utilizing one-tailed t-test with a post-hoc Bonferroni correction. The learner characteristic information. threshold for statistical reliability after the correction has been 4.2 Unsupervised Dialogue Act Models taken as α=0.05. Our unsupervised dialogue act classification approach leverages Gender. As shown in Figure 1, the average leave-one-student-out the k-medoids clustering technique [28]. This approach groups cross-validation accuracy for the model built using female similar utterances together, and is similar to the more familiar k- students’ utterances (nfemale=12) is higher than the model built on means algorithm except that in k-medoids, the centroid of each randomly selected students. In each test run, all of one female’s cluster must be an actual data point within the corpus rather than a utterances were left out to be used as the test set, and the dialogue potentially artificial data point computed as the mean of distances. act model was built on the remaining eleven female students’ Our experiments with k-medoids have demonstrated that it utterances. This process was repeated for each female student. outperforms a variety of other unsupervised machine learning Note that for each of the eleven students, all utterances from that approaches for the task of dialogue act classification in tutorial student were considered. Average test set accuracy for the model dialogue, although the results of such experiments are beyond the with randomly selected students was 0.41 (σ=0.2), whereas the scope of this paper since our goal is to investigate the differential average test set accuracy for the dialogue act classification model benefit of adding learner characteristic features to the model, not that was built utilizing female students’ utterances only was 0.56 to compare different unsupervised approaches. (σ=0.19). After a Bonferroni correction this difference was The k-medoids algorithm requires seeding clusters at the statistically significant (pBonf<0.05). beginning of each training fold and then proceeds by distributing For male students (nmale=30), the average accuracy is only slightly Test Set Accuracies For Self-Efficacy higher with the models tailored to males 0.43 (σ =0.13) than the 0.8 models learned for randomly selected students 0.40 (σ =0.12), and this difference is not statistically significant (Figure 1). Looking 0.7 more closely at the results, we find that for eight of the thirty 0.6 Accuracy males within the corpus, a tailored model outperformed the 0.5 random model (with five of these seeing more than 10% increase 0.4 in accuracy), while twenty-two of the cases saw no difference in 0.3 accuracy between the random and tailored conditions. Two of the 0.2 males saw a decrease in accuracy for the tailored condition. 0.1 Test Set Accuracies For Gender Low-eff. test Low-eff. test High-eff. test High-eff. test random train low-eff. train random train high-eff. train 0.9 0.8 Figure 2: Leave-one-student-out test set accuracies for models 0.7 by self-efficacy 0.6 Table 3: Average test set accuracies for each learner Accuracy 0.5 characteristic (**p<0.05 after Bonferroni correction) 0.4 Learner   Model  restricted   Model  built  on   0.3 characteristic     by  learner   randomly   0.2 group   characteristic   selected  students   0.1 Females   0.56**   0.41   Female test Female test Male test Male test Males   0.43   0.40   random train female train random train male train Low  self-­‐efficacy   0.43   0.38   High  self-­‐efficacy   0.47   0.41   Figure 1: Leave-one-student-out test set accuracies for models by gender 5. DISCUSSION Self-efficacy. Models built using the self-efficacy learner Dialogue act classification is a central task for tutorial dialogue characteristic predict the unseen utterances’ dialogue acts systems. Without accurate dialogue act classification, systems marginally more successfully than models that do not use this cannot adapt and respond appropriately. Unsupervised machine information, though these differences are not statistically reliable. learning approaches to dialogue act classification are a highly For students with low self-efficacy (nlowEff=24) the average test set promising new area of study, and we have presented the first accuracy for dialogue act models that selected students randomly unsupervised dialogue act classifier tailored to learner is 0.38 (σ=0.16) and it increases to 0.43 (σ=0.17) with dialogue characteristics. The experimental results demonstrated that act models that learn only from low-self-efficacy students’ dialogue act classifiers that leverage the non-cognitive factors of utterances (Figure 2). In fifteen out of twenty-four cases the gender and self-efficacy outperform those that do not, and in the dialogue act models tailored to low self-efficacy groups case of female students the improvement was statistically outperform models that are trained on randomly selected students significant. This section presents some examples of the learned (eight of the cases with more than a 10% increase), while in seven dialogue act clusters and discusses the implications of this work of the cases the performance is decreased by utilizing the learner for tutorial dialogue systems. characteristic (five of them by more than a 5%) and in two of the First, we examine clusters from the gender-tailored unsupervised cases the accuracy remains the same. dialogue act classifier. Table 4 displays a selection of utterances The improvement obtained by utilizing learner characteristics in that were clustered together during the unsupervised training of dialogue act classification task is also marginal for high-self- the model, and afterward the clusters were labeled for testing efficacy students, where nhighEff=18. The average performance for purposes using the manual tags that comprise the majority of each the random model is 0.41 (σ=0.14) whereas the model achieves cluster. For those in Table 4 the clusters were labeled as 0.47 (σ=0.11) accuracy when trained only on utterances of high- Acknowledgments and Questions. By examining the structure of self-efficacy students. This improvement was statistically these clusters we gain some intuition as to the types of regularities significant before Bonferroni correction but not afterward. In that help the tailored models to perform significantly better. We seven out of eighteen cases, models trained on utterances of high see females in this study tended to use acknowledgment phrases self-efficacy students improved test set accuracy (five of them such as, “oh I see” and “makes sense,” while males tended to use above 15% improvement) and in two of the cases the learner the phrasing, “got it” more frequently. Within the cluster labeled characteristic decreases the performance (both of them below 5% as questions, we observe that females tended to request more decrease). Nine of the cases remained unaffected in their dialogue feedback, an observation that also emerged in prior work within a act classification accuracy. different corpus in the same domain collected approximately six years earlier [10]. On the other hand, male students tended to ask The average accuracies over the leave-one-student-out cross- more general questions. validation folds can be found in Table 3. Models tailored to learner groups uniformly outperform their counterpart, and the In addition, we observe some example clusters from the models improvement is statistically significant for females. based on self-efficacy in Table 5. Students with high self-efficacy tend to use more confident utterances such as “absolutely” compared to “ok” used by low-self efficacy students. We note that questions in the low self-efficacy group often make an implicit request for reassurance within their task-based questions, such as, 6. CONCLUSION AND FUTURE WORK “and that is it?”. In contrast, students in the high self-efficacy More accurately understanding student natural language within group more often ask contentful questions. intelligent tutoring systems is a critical line of investigation for Table 4: Selected utterances from clusters tailored to gender tutorial dialogue systems researchers. The field has only begun to explore unsupervised approaches and to investigate the range of Females Males features that are beneficial within this paradigm. We have - got it presented a first attempt to leverage non-cognitive factors within - oh I see Acknowledg - ok i got it such a dialogue act classification model, achieving statistically - make sense ements - alright i got it significant improvements in dialogue act modeling for female - yup - gotcha alrigth students, and increasing the models’ performance by small - aha! - cool margins for the self-efficacy groups. -hahaha its ok - sure thing Building upon these first steps, there are several promising future -is this right? directions. First, while sample size prohibited exploring some -so will testing always be -does that work? other learner characteristics here, other characteristics are likely related to running the -should I run it? highly influential and should be investigated. These may include program Questions -was i supposed to put that ethnicity, personality, and other non-cognitive factors. -so it is kinda like saying x before something? Additionally, while the current work focused on analyzing number or something in -so for line number could i dialogue, another aspect of the tutorial interaction that presents algebra? have typed system out challenges in understanding is the task model. Models that aim to -why does not it stop on println monopoly instead of understand students’ problem-solving activities and infer their the next line in this case goals or plans may benefit substantially from leveraging learner println x if i wanted to? characteristics. It is hoped that the research community can continue to build richer models of natural language understanding Table 5: Selected utterances from clusters tailored to self- for students of all learner characteristics in order to improve the efficacy student experience and enhance learning by adaptation. Low Self-Efficacy High Self-Efficacy ACKNOWLEDGMENTS - ok The authors wish to thank the members of the Center for Acknowled -cool! Educational Informatics at North Carolina State University for gements - yes there were a lot of -oh ok that works their helpful input. This work is supported in part by the things i felt like i had to - yep got that National Science Foundation through Grant DRL-1007962 and switch around - absolutely the STARS Alliance, CNS-1042468. Any opinions, findings, - that makes sense now conclusions, or recommendations expressed in this report are -so what exactly am i -what is the best way to those of the participants, and do not necessarily represent the supposed to be doing? do that? official views, opinions, or policy of the National Science - is there something - ok so tell me if this Foundation. Questions specific i need to call my makes sense string game declares the variable REFERENCES - i finished reading should i and then line number [1] Ames, C. and Archer, J. 1988. Achievement Goals in the click compile again? tells me what that Classroom: Students’ Learning Strategies and Motivation -and that is it? variable is value is? Processes. Journal of Educational Psychology. 80, 3, 260– Limitations. The present work has several notable limitations. 267. First, as mentioned previously, the partitions of the corpus are not [2] Arthur, D. and Vassilvitskii, S. 2007. k-means++: The independent; that is, the same student, and associated utterances, Advantages of Careful Seeding. In Proceedings Of The are present within one gender group and one self-efficacy group. Eighteenth Annual ACM-SIAM Symposium On Discrete Because these partitions are not independent, care must be taken Algorithms. 1027–1035. when interpreting the findings. Furthermore, it is possible that the self-efficacy of students can change in the course of tutoring, [3] Austin, J.L. 1962. How To Do Things With Words. Oxford which would not be handled by a classifier built using a one-time University Press. self-report. However, we believe that the current approach holds [4] Azarnoush, B., Bekki, J.M. and Bernstein, B.L. 2013. great promise for real-time tutorial dialogue classification. By Toward a Framework for Learner Segmentation. JEDM. 5, building separate classifiers by learner characteristic, a suite of 2, 102–126. classifiers (each smaller and faster than one built on the entire corpus) can be run in parallel and can vote for the classification of [5] Bandura, A. 2006. Guide for Constructing Self-Efficacy a given students’ utterance. However, as is the case with the work Scales. Self-efficacy Beliefs Of Adolescents. 5, 307–337. presented here, splitting the corpus results in a substantially [6] Bangalore, S., Di Fabbrizio, G. and Stent, A. 2008. reduced sample size on which to train, which partially explains Learning the Structure of Task-Driven Human-Human the lack of statistically reliable results observed here. Our work Dialogs. IEEE Transactions on Audio, Speech and has begun to explore the use of intrinsic metrics for accuracy Language Processing. 16, 7, 1249–1259. (rather than relying on manual tags), which has the potential to dramatically increase the available data to any dialogue act [7] Bloom, B.S. 1984. Sigma of Problem: The Methods classifier and mitigate issues of sparsity that arise when splitting Instruction One-to-One Tutoring. Educational Researcher. by learner characteristics. 4–16. [8] Boyer, K.E., Ha, E.Y., Phillips, R., Wallis, M.D., Vouk, [23] Lee, D., Jeong, M., Kim, K., Ryu, S. and Geunbae, G. 2013. M.A. and Lester, J.C. 2010. Dialogue Act Modeling in a Unsupervised Spoken Language Understanding for a Multi- Complex Task-Oriented Domain. In Proceedings of Domain Dialog System. IEEE Transactions On Audio, SIGDIAL. 297–305. Speech, and Language Processing. 21, 11, 2451–2464. [9] Boyer, K.E., Phillips, R., Wallis, M., Vouk, M. and Lester, [24] Marineau, J., Wiemer-Hastings, P., Harter, D., Olde, B., J. 2008. Balancing Cognitive and Motivational Scaffolding Chipman, P., Karnavat, A., Pomeroy, V., Rajan, S. and in Tutorial Dialogue. In Proceedings of ITS, 239–249. Graesser, A. 2000. Classification of Speech Acts in Tutorial [10] Boyer, K.E., Vouk, M.A. and Lester, J.C. 2007. The Dialog. In Proceedings of the Workshop On Modeling Influence of Learner Characteristics on Task-Oriented Human Teaching Tactics And Strategies at ITS. 65–71. Tutorial Dialogue. In Proceedings of AIED, 365–372. [25] Meece, J.L. and Holt, K. 1993. A Pattern Analysis Of [11] Buckley, M. and Wolska, M. 2008. A Classification of Students’ Achievement Goals. Journal Of Educational Dialogue Actions in Tutorial Dialogue. In Proceedings of Psychology. 85, 4, 582–590. the 22nd International Conference on Computational [26] Merceron, A. and Yacef, K. 2003. A Web-Based Tutoring Linguistics. 1, 73–80. Tool With Mining Facilities to Improve Learning and [12] Crook, N., Granell, R. and Pulman, S. 2009. Unsupervised Teaching. In Proceedings of AIED, 201–208. Classification of Dialogue Acts Using a Dirichlet Process [27] Merceron, A. and Yacef, K. 2005. Clustering Students To Mixture Model. In Proceedings of SIGDIAL. 341–348. Help Evaluate Learning. Technology Enhanced Learning. [13] Dzikovska, M.O., Farrow, E. and Moore, J.D. 2013. 171, 31–42. Combining Semantic Interpretation and Statistical [28] Ng, R.T. and Han, J. 1994. Efficient and Effective Classification for Improved Explanation Processing in a Clustering Methods for Spatial Data Mining. In Proceedings Tutorial Dialogue System. In Proceedings of AIED. 279– of the 20th International Conference on Very Large Data 288. Bases, 144–155. [14] Eugenio, B. Di, Xie, Z. and Serafin, R. 2010. Dialogue Act [29] Reithinger, N. and Klesen 1997. Dialogue Act Classification Classification , Higher Order Dialogue Structure, and Using Language Models. In Proceedings of EuroSpeech, Instance-Based Learning. Dialogue & Discourse. 1, 2, 1–24. 2235–2238. [15] Ezen-Can, A. and Boyer, K.E. 2013. Unsupervised [30] Ritter, A., Cherry, C. and Dolan, B. 2010. Unsupervised Classification of Student Dialogue Acts With Query- Modeling of Twitter Conversations. In Proceedings of the likelihood Clustering. In Proceedings of EDM, 20–27. Association for Computational Linguistics, 172–180. [16] Forbes-Riley, K. and Litman, D.J. 2009. A User Modeling- [31] Rus, V., Moldovan, C., Niraula, N. and Graesser, A.C. Based Performance Analysis Of A Wizarded Uncertainty- 2012. Automated Discovery of Speech Act Categories in Adaptive Dialogue System Corpus. In Proceedings of Educational Games. In Proceedings of EDM, 25–32. INTERSPEECH, 2467–2470. [32] Searle, J.R. 1969. Speech Acts: An Essay in the Philosophy [17] Graesser, A.C., Wiemer-Hastings, K., Wiemer-Hastings, P. of Language. Cambridge University Press. and Kreuz, R. 1999. AutoTutor: A Simulation Of A Human [33] Serafin, R. and Di Eugenio, B. 2004. FLSA: Extending Tutor. Cognitive Systems Research. 1, 1, 35–51. Latent Semantic Analysis With Features For Dialogue Act [18] Ha, E.Y., Grafsgaard, J.F., Mitchell, C.M., Boyer, K.E. and Classification. In Proceedings of the Association for Lester, J.C. 2012. Combining Verbal and Nonverbal Computational Linguistics, 692–699. Features to Overcome the ‘Information Gap’ in Task- [34] Sridhar, V.K.R., Bangalore, S. and Narayanan, S.S. 2009. Oriented Dialogue. In Proceedings of SIGDIAL, 247–256. Combining Lexical, Syntactic and Prosodic Cues For [19] Hershkovitz, A. and Nachmias, R. 2011. Online Persistence Improved Online Dialog Act Tagging. Computer Speech & In Higher Education Web-Supported Courses. The Internet Language. 23, 4, 407–422. and Higher Education. 14, 2, 98–106. [35] Stolcke, A., Ries, K., Coccaro, N., Shriberg, E., Bates, R., [20] Higashinaka, R., Kawamae, N., Sadamitsu, K., Minami, Y., Jurafsky, D., Taylor, P., Martin, R., Ess-Dykema, C. Van Meguro, T., Dohsaka, K. and Inagaki, H. 2011. and Meteer, M. 2000. Dialogue Act Modeling for Unsupervised Clustering of Utterances Using Non- Automatic Tagging and Recognition of Conversational Parametric Bayesian Methods. In Proceedings of Speech. Computational Linguistics. 26, 3, 339–373. INTERSPEECH, 2081–2084. [36] VanLehn, K., Jordan, P.W., Rosé, C.P., Bhembe, D., [21] Joty, S., Carenini, G. and Lin, C.-Y. 2011. Unsupervised Bottner, M., Gaydos, A., Makatchev, M., Pappuswamy, U., Modeling Of Dialog Acts In Asynchronous Conversations. Ringenberg, M., Roque, A., Siler, S. and Srivastava, R. In Proceedings of the Twenty-Second International Joint 2002. The Architecture Of Why2-Atlas: A Coach For Conference on Artificial Intelligence. 1807–1813. Qualitative Physics Essay Writing. In Proceedings of ITS, [22] Keizer, S., Akker, R. and Nijholt, A. 2002. Dialogue Act 158–167. Recognition with Bayesian Networks for Dutch Dialogues. In Proceedings of the SIGDIAL Workshop, 88–94.