Students' Personality and Chat Room Behavior in Synchronous Online Learning

Abstract

Synchronous chat rooms have been used to support students' collaborative learning. However, little work has empirically studied its relationship with students' sense of community and furthermore the role of personality in a ecting students' chat room behavior. In this paper, we report results of a user experiment (with 489 students). We nd that students who have actively used chat room possess signi cantly higher level of sense of community than inactive students. More notably, we identify the important role of students' personality in in uencing both their chat frequency and chat content. Online learning; synchronous learning; chat behavior; user survey; sense of community; personality.

1. INTRODUCTION

Online learning is de ned as the process of using the Internet to acquire knowledge, access learning materials, and interact with others [ 1 ]. Over the past few years, online courses have mainly depended on asynchronous mode of instruction that allows students to study at their own pace [ 9, 18 ]. However, some studies show that students feel low sense of community (a feeling of belonging to a community in a course-based context [ 13 ]) due to the lack of real-time interaction, which leads to low student retention [ 20 ]. Some researchers have hence suggested that more community-oriented features should be used to create a more inter-connected learning environment [ 4 ].

Therefore, synchronous online learning (also referred to live instruction), where instruction and learning occur at the same time, has been emphasized in recent online courses [ 17 ]. The real-time interaction may enable more e cient and active communication to reduce students' feeling of isolation and improve their engagement in online learning [ 4, 17 ].

In this paper, we focus on studying text chat, which is one of the popularly used communication tools in synchronous online learning [ 12 ]. Via real-time chat messages, students can exchange immediate feedback with their peers and instructors. They may hence see themselves as active participants rather than isolated users who communicate with the computer alone.

However, it is not conclusive whether chat room can really be used to build students' sense of community [ 3, 15 ]. Moreover, few studies have investigated the role of personal factors, especially personality, in in uencing students' chat behavior. In order to address these issues, we have performed an experiment that records 489 students' chat room behavior when attending online classes, and their personality and post-course sense of community as acquired through questionnaire. The analysis identi es signi cant di erence between active students of using chat room and inactive ones in terms of their sense of community. The results additionally reveal signi cant e ect of personality on both students' chat frequency and chat content.

In the following, we rst state our research questions, and then present experiment setup and results analysis. At the end, we draw the conclusion. 2.

PROBLEM STATEMENT

Our research questions are illustrated in Figure 1. To be speci c, we are interested in rst verifying the correlation between sense of community and students' perceived learning outcomes. Related work shows that greater sense of belonging can alleviate students' feeling of isolation and increase their satisfaction with the academic program [ 14 ], but few have empirically identi ed whether high sense of community would help improve students' learning e ectiveness, e.g., becoming more interested in the studied course (i.e., interest growth) and being motivated to accumulate more course knowledge (i.e., knowledge growth) [ 16 ]. The corresponding research question is:

RQ1: Would students' sense of community be signi cantly correlated with not only their satisfaction with learning process, but also interest growth and knowledge growth?

Secondly, it is interesting to identify whether text chat could be helpful for developing students' sense of community. In [ 3 ], an experimental comparison conducted between chat and no-chat groups shows that the two groups do not respond signi cantly di erently as to sense of community. However, this experiment did not take into account students' chat frequency during the comparison. In another related work [ 15 ], synchronous chat is found more e ective in building students' sense of community relative to asynchronous bulletin board, but the size of samples (7 students) is quite small. In our experiment, we cluster 489 students into two groups, active and inactive chat groups, according to two chat frequency measures: the average number of messages each student posted per lesson and her/his show ratio (the ratio of a number of lessons the student has posted at least one message in a lesson to the number of all lessons s/he has attended). The related question is:

RQ2: Would active chat group possess signi cantly higher level of sense of community than inactive group?

Lastly, we investigate the role of students' Myers-Briggs Type Indicator (MBTI) personality [ 10 ] in in uencing their chat room behavior. In comparison with Big-Five personality test [ 6 ], MBTI test has been more popularly used in the area of learning given that it is more indicative of a user's style when dealing with the outer world and her/his way of perceiving information and making decisions. Concretely, MBTI test de nes a user's personality in four dimensions, Introversion-Extraversion (re ecting individual perceptual orientation), Sensing-iNtuition (presenting how individuals gather information), Feeling-Thinking (in uencing how people make decisions), and Judging-Perceiving (indicating the way people connect with the outer world)1.

To our knowledge, some studies have also attempted to relate personality to student performance and behavior in online communication [ 2, 8 ]. For instance, extroverts are found more active in discussions than introverts [ 2 ]. However, there are two major limitations. One is that they mainly focus on one personality dimension Introversion-Extraversion. The other three MBTI personality dimensions have rarely been studied. The second limitation is that they did not measure the e ect of personality on students' chat content such as the appearance of social and cognitive presence words. Indeed, these two types of presence can indicate whether synchronous chat would foster collaborative and meaningful learning [ 12 ]. To be speci c, social presence is de ned as the degree of awareness of others in an interaction. It is concretely re ected by social process words (e.g., \talk" and \discuss") and a ective process words (e.g., \awesome" and \terri ed"). Cognitive presence refers to the extent of both re ection and discourse in the construction of meaningful learning outcomes. It is embodied by cognitive process words like \think" and \consider".

RQ3: Would students' personality signi cantly a ect both their chat frequency and chat content? 3. 3.1

EXPERIMENT DESIGN

Materials and Participants

The experiment was performed on a Chinese online learning website, called eBanshu (www.ebanshu.com), which pri1Due to space limit, the detailed description of these four personality dimensions can be found in [ 10 ]. marily provides synchronous learning facilities for university instructors to perform remote instruction. The instructor can use video camera and digitizer to give real-time lectures, and students can communicate with peers and the instructor through an embedded text chat interface (see Figure 2). At the end of the course, the student will take credit if s/he passes examination.

From March to June 2015, a total of 1,559 students were enrolled on 16 o ered courses (e.g., \Comparative Literature", \Inorganic Chemistry", \Discrete Mathematics"). These courses can be classi ed into three subject types: liberal art (9 courses), science (6 courses), and engineering (1 course). The average course enrollment is 97.3 (min=50, max=209, st.d.=42.2). Each course lasted for over 12 weeks, with two lessons given per week (each lesson takes 1 hour). Among those 1,559 students, 489 students (409 females) participated in our survey. Their ages range from 20 to 25, and are from 11 di erent majors (e.g., Chemistry, English, Pedagogy). 3.2

Measurement

Before each course started, we asked students to ll in a questionnaire for obtaining their pre-course interest (from 1 \very low" to 5 \very high"), pre-course knowledge (from 1 \none" to 5 \all"), and personality. The personality was acquired via a 28-item Chinese version of MBTI test. Each dimension was concretely measured via 7 questions, with each question containing two response options [ 10 ].

Then, when students nished the course, we asked them to ll out a questionnaire about their sense of community, satisfaction with learning process (from 1 \very unsatis ed" to 5 \very satis ed"), post-course interest, and postcourse knowledge. The sense of community was responded on Rovai's 20-statement Classroom Community Scale [ 13 ] (each statement was rated on a 5-point Likert scale from 1 \strongly disagree" to 5 \strongly agree"). This community scale can be divided into two dimensions [ 13 ]: connectedness that refers to students' feeling of belonging to a community; and learning that is their feeling the community helps them to acquire knowledge and meet learning goals.

The perceived learning outcomes hence include students' satisfaction with learning process, interest growth, and knowledge growth. For the latter two, the \growth" is formally calculated as the di erence between post-course and pre-course responses.

Moreover, we recorded students' chat room behavior for the purpose of analyzing their chat frequency and content. The behavioral data include the number of messages a student posted per lesson and each message's actual content.

RESULTS AND ANALYSIS

Sense of Community and Perceived Learning Outcomes

The reliability analysis of sense of community scale shows that its internal consistency coe cient (Cronbach's alpha) is 0.888, and the coe cients of the two sub-dimensions connectedness and learning are 0.724 and 0.811 respectively. These values are all above 0.70, suggesting that the corresponding statements have satisfactory internal validity [ 11 ]. Moreover, it shows that students' post-course interest is signi cantly higher than their pre-course interest (mean=4.08 vs. 3.77, t=8.7, p<0.01, via paired sample t-test). They also perceive the acquisition of signi cantly more knowledge after learning (mean=3.44 vs. 2.44, t=24.9, p<0.01).

We then calculated the correlation between sense of community and students' perceived learning outcomes, via Spearman's rank coe cient as it is applicable to both ordinal and numerical variables [ 19 ]. From Table 1, we can see that students' sense of community values are signi cantly positively correlated with their perception of learning outcomes in terms of all measures. Concretely, it indicates that students who feel stronger sense of community (not only at overall scale but also in the two sub-dimensions connectedness and learning) are likely to be more satis ed with their learning process, to be more interested in the studied course, and accumulate more knowledge after taking the course. The results thus well answer our rst research question. 4.2

Chat Frequency and Sense of Community

The 489 students posted totally 28,033 chat messages. As each student was enrolled on only one course according to our record, their average number of messages per lesson is 2.38 (st.d.=2.99) and average show ratio during the whole course is 38.9% (st.d.=15.2%). For each lesson, the average number of students who posted at least one message is 29.2 (st.d.=16.8) and their relative percentage is 36.9%.

In order to explore the relationship between chat frequency and students' sense of community, we used k-means clustering (k=2) to automatically partition all students into two groups according to both their average number of messages posted per lesson and show ratio. Figure 3 illustrates the clustering results, where red crosses represent students who are more active in posting chat messages than those marked in blue crosses. The detailed description of those two groups is given in Table 2.

Then, we compared the two groups' sense of community values. It shows that active chat users perceive signi cantly stronger sense of community than inactive chat group (mean=3.55 vs. 3.33, t=3.889, p<0.01 by t-test for two independent samples). We further compared the two groups in respect of the two sub-dimensions, connectedness and learning, which still shows signi cant di erences. To be speci c, regarding connectedness, the mean of active chat group is 3.50 (vs. 3.36 of inactive group, t=2.958, p<0.01). As for learning, it is 3.58 against 3.31 of inactive group (t=4.055, p<0.01). The results thus suggest that students who behave more actively in chat room are likely to feel higher level of community belonging and learning achievement. 4.3

Personality and Chat Behavior

The next question comes to whether personal factor will in uence students' chat behavior. As mentioned before, we particularly consider personality because it is inherently related to the way a person deals with the outer world and perceives information. Our participants' distribution among the four dimensions of MBTI personality is: Introversion (331 students (68%))-Extraversion (158 (32%)), Sensing (254 (52%))-iNtuition (235 (48%)), Feeling (383 (78%))Thinking (106 (22%)), and Judging (394 (81%))-Perceiving (95 (19%)).

Students' chat behavior was analyzed from two aspects: chat frequency (i.e., the average number of messages posted in each lesson and show ratio) and chat content. For the latter, in addition to calculating each message's length, we used Chinese Linguistic Inquiry and Word Count (CLIWC) dictionary [ 7 ] to identify its social presence and cognitive presence by counting the occurrences of social, a ective, and cognitive process words (see the de nition in Section 2). We also measured the message's engagement degree with learning topic, and manually classi ed all messages into three types: \fully-engaged" (e.g., discussing a concept), \somewhat-engaged" (e.g., asking for help of technical support), and \disengaged" (e.g., greeting).

We then conducted hierarchical multiple regression analysis for which the control variables are gender, pre-course knowledge, pre-course interest, and subject type. Through this analysis, we are able to identify whether personality would account for any additional variances that cannot be explained by control variables.

The results are shown in Table 3. It indicates that personality explains signi cant proportion of variances (p<0.05) in terms of most of chat behavior measures. Concretely, the number of messages is positively in uenced by IntroversionExtraversion dimension, and negatively by Sensing-iNtuition, implying that students who prefer moving into action (extroverts ) and learning from details (sensors ) are inclined to post more messages. Moreover, those persons also tend to use more social and cognitive presence words in their messages. Another observation is that cognitive presence words were frequently used by thinking people (in respect of Feeling-Thinking personality dimension) who are more impersonal and logical in talking. As for task engagement, the messages posted by extroverted students contain more content fully or somewhat relevant to the learning topic.

On the other hand, we nd some control variables also signi cantly a ect students' chat behavior. For instance, males posted more disengaged messages than females. Students with richer pre-course knowledge posted more social presence and cognitive presence related messages. Students who were more interested in the course before taking it (i.e., pre-course interest ) posted more messages on average. As for subject type, relative to students who were enrolled on liberal art type courses, engineering students exhibited lower show ratio and science students posted less fully and somewhat engaged messages, implying that chat room may be more e ective for liberal art students. 5.

CONCLUSION AND FUTURE WORK

In conclusion, there are several interesting ndings from this experiment: 1) Students' sense of community is signi cantly positively correlated with their satisfaction with learning process, interest growth, and knowledge growth. 2) Active chat students possess signi cantly stronger sense of community than inactive students. 3) Students' personality values, especially in terms of dimensions IntroversionExtraversion, Sensing-iNtuition, and Feeling-Thinking, signi cantly a ect their chat behavior.

Thus, the ndings not only verify the positive role of chat room in synchronous online learning, but should be constructive for related practitioners to develop more e ective chat room given students' personality. For instance, for students who are less active in using chat room, they may consider providing some personalized supports. For introverted students who need more time to re ect and respond, instructors may publish learning materials ahead of each lesson for them to obtain certain course information, so as to increase their motivation to join the chatting during the lecture. For intuitive students who are interested in fresh and abstract things, choosing more imaginative and attractive discussion topics may potentially arouse their chat intention.

For our future work, more factors that may in uence students' chat behavior will be investigated. For example, we will study whether students with di erent learning styles behave di erently in chat room, given that learning style re ects the way students absorb and deal with course materials [ 5 ]. Moreover, it will be interesting to compare text chat with other community-oriented features such as video chat and discussion forum, so as to identify its relative pros and cons in improving student learning. We will also try to consolidate the experiment's ndings among larger scale of samples who could be with diverse demographic properties (e.g., age, nationality, ethnic background).

References

[1] M. Ally . Foundations of educational theory for online learning . Theory and Practice of Online Learning , 2 : 15 { 44 , 2004 .

[2] I. Blau and A. Barak . How do personality, synchronous media, and discussion topic a ect participation? Journal of Educational Technology & Society , 15 ( 2 ): 12 { 24 , 2012 .

[3] D. Coetzee , A. Fox , M. A. Hearst , and B. Hartmann . Chatrooms in MOOCs: All talk and no action . In Proceedings of the 1st ACM Conference on Learning@ Scale Conference (L@S 2014 ), pages 127 { 136 . ACM, 2014 .

[4] D. Coetzee , S. Lim , A. Fox , B. Hartmann , and M. A. Hearst . Structuring interactions for large-scale synchronous peer learning . In Proceedings of the 18th ACM Conference on Computer Supported Cooperative Work & Social Computing (CSCW 2015 ), pages 1139 { 1152 . ACM, 2015 .

[5] R. M. Felder and L. K. Silverman . Learning and teaching styles in engineering education . Engineering Education , 78 ( 7 ): 674 { 681 , 1988 .

[6] S. D. Gosling , P. J. Rentfrow , and W. B. Swann . A very brief measure of the big- ve personality domains . Journal of Research in Personality , 37 ( 6 ): 504 { 528 , 2003 .

[7] C. Huang , C. Chung , N. Hui , Y. Lin , Y. Seih , B. Lam , and J. Pennebaker. The development of the Chinese linguistic inquiry and word count dictionary . Chinese Journal of Psychology , 54 ( 2 ): 185 { 201 , 2012 .

[8] G. M. Johnson and J. A. Johnson . Personality, internet experience, and e-communication preference . In Proceedings of the 5th Annual Conference of the International Association for Development of the Information Society (IADIS 2006 ), pages 55 { 62 . ERIC , 2006 .

[9] M. Loncar , N. E. Barrett , and G.- Z. Liu . Towards the re nement of forum and asynchronous online discussion in educational contexts worldwide: Trends and investigative approaches within a dominant research paradigm . Computers & Education , 73 : 93 { 110 , 2014 .

[10] I. B. Myers , M. H. McCaulley , N. L. Quenk , and A. L. Hammer . MBTI Manual: A Guide to the Development and Use of the Myers-Briggs Type Indicator , volume 3 . Consulting Psychologists Press, Palo Alto, CA, 1998 .

[11] J. C. Nunnally , I. H. Bernstein , and J. M. t . Berge. Psychometric Theory , volume 226 . McGraw-Hill , New York, 1967 .

[12] M. Oztok , D. Zingaro , C. Brett , and J. Hewitt . Exploring asynchronous and synchronous tool use in online courses . Computers & Education , 60 ( 1 ): 87 { 94 , 2013 .

[13] A. P. Rovai . Development of an instrument to measure classroom community . The Internet and Higher Education , 5 ( 3 ): 197 { 211 , 2002 .

[14] A. P. Rovai . Sense of community, perceived cognitive learning, and persistence in asynchronous learning networks . The Internet and Higher Education , 5 ( 4 ): 319 { 332 , 2002 .

[15] R. A. Schwier and S. Balbar . The interplay of content and community in synchronous and asynchronous communication: Virtual communication in a graduate seminar . Canadian Journal of Learning and Technology , 28 ( 2 ), 2002 .

[16] K. Swan . Learning e ectiveness online: What the research tells us . Elements of Quality Online Education, Practice and Direction , 4 : 13 { 47 , 2003 .

[17] C. A. Warden , J. O. Stanworth , J. B. Ren , and A. R. Warden . Synchronous learning best practices: An action research study . Computers & Education , 63 : 197 { 207 , 2013 .

[18] S. Wattamwar , T. Oberoi , and H. Ghosh . A novel framework for distance education using asynchronous interaction . In Proceedings of the 3rd International ACM Workshop on Multimedia Technologies for Distance Learning (MTDL 2011) , pages 7 { 12. ACM, 2011 .

[19] J. H. Zar . Signi cance testing of the spearman rank correlation coe cient . Journal of the American Statistical Association , 67 ( 339 ): 578 { 580 , 1972 .

[20] S. Zheng , M. B. Rosson , P. C. Shih , and J. M. Carroll . Understanding student motivation, behaviors and perceptions in MOOCs . In Proceedings of the 18th ACM Conference on Computer Supported Cooperative Work & Social Computing (CSCW 2015 ), pages 1882 { 1895 . ACM, 2015 .