In recent years, the use of conversational agents (CA) has been increasing at a rapid pace. Eforts have been made to leverage CAs for tackling mental health challenges. Our goal is to improve mental well-being by enabling self-help ideas through chat-based CAs. To enhance the efectiveness and user reception of CAs, we designed diferent conversational personalities with low and high variants of extroversion and conscientiousness. We used various language cues and example conversation scripts as the basis of our design process. This paper presents the design and validation process of such CA personalities. Our results indicate that the final personality characteristics presented in the scripts are recognizable in the text-based CA interactions and thus enable future research on human behavior with such agents.
Agents (CAs) for health and well-being has been steadily increasing [1]. Many CAs are also used in communication with customers on commercial platforms and are increasingly utilized in oficial government websites and general healthcare [2]. In this paper, we focus on chatbased CAs, so-called chatbots, in the context of mental health self-help. CAs have been used in mental health self-help applications, yielding promising results in promoting self-help and reducing stress [3, 4, 5].
If the CA appears too human, it can significantly lower the user’s trust in the CA and make it seem uncanny, an efect first suggested by Mori [ 6] and later researched and discussed in several other papers [7, 8, 9]. Lately, researchers have focused on how diferent personality types perform and how using diferent personalities or even altering it depending on the user can make the CAs perform better by increasing the user’s trust and acceptability towards them [10, 11, 12, 13, 14, 15]. Focusing on the personality design can also help make them more easily approachable and likable, thus increasing their alities can have a positive efect on the user experience [18] and many users wish for CAs to show some kind of personification, expect them to show emotion and more
Using CAs to provide mental health support, self-help, and digital counseling services has been researched in various studies [22, 23, 24, 25]. CAs can help mental health counseling services to be more accessible. CAs can help users engage in conversation and improve wellbeing in non-clinical use [3]. Self-help CAs provide the users with instant and interactive feedback as opposed to traditional websites and can be beneficial, especially for individuals who prefer to seek their information online [22].
Research has shown that users prefer to engage with CAs close to their personalities for information-seeking [26] and in assistance and therapy [27]. Having the CA express multiple personalities and have emotion selection improves the user engagement significantly [ 12]. In particular, the efect of high extroversion has been shown to enhance user satisfaction [10].
An individual’s personality is a combination of multiple attributes and cues, ranging from behavioral to emotional and mental characteristics. Personality can manifest as personality traits, of which the typical dimensions are the so-called Big Five personality dimensions [21]: 3. Conversational Agent Design • Extroversion vs. Introversion (sociable, assertive,
playful vs. aloof, reserved, shy) • Emotional stability vs. Neuroticism (calm, un
emotional vs. insecure, anxious) • Agreeableness vs. Disagreeable (friendly, cooper
ative vs. antagonistic, faultfinding) • Conscientiousness vs. Unconscientious (self
disciplined, organized vs. ineficient, careless) • Openness to experience (intellectual, insightful vs. shallow, unimaginative) We created five CAs with difering personalities that ofer self-help method recommendations for mental health. To appropriately compare one personality to another, we wanted to minimize efects that can be considered external. One of the factors identified was the amount of information provided by the CA. To mitigate this, we designed a simple interactive conversation structure that provides the user with mental self-help methods, which The Big Five traits have been leveraged as analysis all CA personalities follow. The main conversation struc- tools, for example, for academic success [28] and in menture is shown in Figure 1. tal health contexts [29]. Language use is frequently uti
After the initial introduction, the user can select their lized as a method of identifying Big Five personality traits current mood from three options - Not so good, I’m okay, [30, 31]. However, some of its terms like neuroticism and and I feel great. The CA reacts to the chosen option ac- language associated with such behavior can vary strongly cordingly and gives three mental health topics, including between individuals. We wanted to leverage the Big Five stress, anxiety, and low mood, that it gives self-help meth- personality traits in the design of our CAs and considered ods for. This step can be repeated as many times as the the following before beginning the design process. user wants, after which the user is asked for an open Firstly, the measurements or factors used to determine text entry about what they thought of the methods given. Big Five traits can overlap, e.g., factors perceived as high The CA responds to this message accordingly. The sec- on the extroversion-introversion dimension can also exond part of the conversation follows a similar pattern, ist on the agreeable-disagreeable dimension. Thus, if a with the CA giving information on literature, audio, and CA personality type A exhibits features of high extrowebsite sources. version, the same features can be perceived as, e.g., high agreeableness. This persuaded us to consider the CA personalities as single dimensions instead of a combination of all Big Five or a subset of them - i.e., hypothetical CA personality ”A” has high extroversion, and its personality is measured only on the extroversion scale. This division of the Big Five into dimensions is a standardized method of personality trait analysis [14].
Secondly, not all the dimensions are necessarily suitable or relevant for a CA designed to ofer self-help guidance. E.g., openness to experience as a trait of the CA plays a minor role in how i) the CA delivers information and ii) how this information is received. Thus, we wanted to select a subset of the dimensions to analyze and decided on the extroversion and conscientiousness traits.
Reaction to social assertiveness (or lack of it) is shown to vary according to an individual [32] thus how assertive (or shy) the CA personality is can afect how it is perceived. Conscientiousness can also be perceived diferently by individuals [33], as high conscientiousness can have a negative influence on individuals with depression [34] but is generally considered a positive trait. In earlier CA-related personality studies, we found 10/11 articles to study extroversion-related traits, 5/11 to investigate emotional stability-neuroticism - which we would find somewhat unfitting for our purposes, especially on the neurotic end, and 4/11 studied conscientiousness. Both of the selected dimensions are evaluated on the high-low scale, so the selected personality types are High and Low Figure 2: In the questionnaire, the participants were preExtroversion (High E and Low E) and High and Low sented with two conversations and were required to grade the Conscientiousness (High C and Low C), as well as the language cues and overall conscientiousness/extroversion on Neutral option. a 7-point Likert scale. In this example, 1 = Low conscientious
These CAs follow the same conversational structure ness 1 and 2 = High conscientiousness. and difer only in their personalities, using diferent phrases and words to deliver the same information. In addition to these, we created a CA with a neutral person- or by natural language processing methods provided by ality that aims to be neutral in both conscientiousness Dialogflow ES. The latter allows the CA to react in a few and extroversion. We started by making the neutral con- diferent ways to the participant’s response to ”What versation script, then created the other personalities by did you think of the presented methods?” during the using various language cues, some of which can be seen feedback, for example, which is responded to using text in Table 3, and example conversation scripts of diferent input. personalities from related work [14, 35]. To help users diferentiate between the CAs, we gave each CA an avatar 4. Personality Validation and a color theme.
We deployed the chatbots using BotStar chatbot engine1 and Dialogflow ES 2. BotStar was used to create the main conversational structure and host the CA. In contrast, Dialogflow ES was used for two specific messages that determine if the user is satisfied with the recommendations given and has the CA respond to their message accordingly.
To keep the design simple and to prevent the user from selecting topics the CA is not able to talk about, the primary interaction between the user and the CA is through multiple-choice buttons (method selection)
The conversation scripts of the five diferent CAs were created by the main author of the paper and then evaluated by two co-authors. We then designed a questionnaire to validate the personality design and used the online crowdsourcing platform Prolific [ 36] to recruit participants. Depending on the results we gain from the personality validation, we can change the conversation scripts accordingly and re-validate. To locate potential issues with precision, we select three conversation parts from each personality and three individual messages from the remaining script. This helped us see which specific section of the script needed to be re-written instead of re-doing the whole script. For this purpose, the participants do not directly use the CA, and the entirety of the validation is done by presenting the users with images of conversations as is presented in Figure 2 and accurate. As language cues between personalities can singular messages directly on the survey platform. overlap, which can lead to, for example, Low E being misinterpreted as High C, section three of the questionnaire 4.1. Questionnaire Design could prove dificult to get accurate results in. 40 participants were recruited, 20 for each iteration.
For the validation process, we designed a three-stage 3 participants were replaced for timing out or failing to questionnaire. To ensure each personality is perceived as answer the control question. The average age of particintended, we validated both Conscientiousness and Ex- ipants was 27.23 years (SD = 7.49). 24 were female, 16 troversion dimensions separately - High C vs. Low C and male. 31 of the participants came from Europe, with PorHigh E vs. Low E. To ensure the participants are famil- tugal and The United Kingdom being the most presented iar with the two personality types, we briefly described countries, with 7 participants for each. The remaining the personalities and presented the participants with ex- participants came from North America (4), South Amerample messages that portray each personality type. In ica (2), Africa (2), and Asia (1). English was the first sections 1 and 2, we presented the same conversation language for 10 participants. The participants were paid side-by-side, as shown in Figure 2. We then used a single- 6.70USD/hr for the first iteration and 9.24USD/hr for the choice selection to evaluate three language cues used in second iteration, with an average response time of 12min part 1 and part 2 (”Which set of the white messages is more 11s and 10min 52s, respectively.
Impulsive?”). White messages refer to the text in the con- We now introduce the results of the first iteration for versation outputted by the CA. The range of language each section, the changes the responses elicited us to cues varied from more obscure terms like ”Vague,” ”Impul- conduct, and finally, the results after the second iteration sive,” ”High verbal output” to more understandable terms round. like ”Formal” or ”Using shorter words.” The language cues we used were derived from the work of Mairesse 5.1. First Iteration et al. [14] and are based on several studies researching diferent Big Five traits [ 37, 38, 39, 40, 41, 42, 43, 44, 45]. The first two sections of the validation questionnaire The full list of language cues are presented in Table 3. rated the language cues and the overall conscientiousThen, we evaluate the conscientiousness or extroversion ness/extroversion of three sets of conversations. To valof the script using a single 7-point Likert-style Item (”How idate the language cues, we used the simple majority conscientious do you think set 1 was?”). This method was voting system, i.e., results where the correct answer is used for three conversation sets for both Conscientious- the most selected option are deemed suficient. Only ness (Questionnaire section 1) and Extroversion (section the language cue ”dissatisfaction” for Extroversion set 2). In section 3, individual messages were presented, and 3 resulted under this threshold, with 7/20 participants the participant was instructed to select a matching per- unsure and 6/20 correct. sonality type (Neutral, High C, Low C, High E, or Low For the Likert-style items, we visually inspected the E). distribution of answers, presented in Figure 3 and Figure 4. We then analyzed the diferences between Low and 4.2. Participant Recruitment High personality variants using the Wilcoxon Rank Sum test, and all the variants were significantly diferent on the .005 confidence level, full results presented in Table 2.
pants to validate individual messages of the conversation and select the personality type from all of the five possible options, including ’Neutral / I’m not sure.’ Each personality has three messages to be evaluated, for a total of twelve messages. The results are presented in Table 1. Again, we used a simple majority to validate the results. Low E was most often selected as High C (21/60) and as Low E only 13/60 times. Low C was mistaken for High C 17/60 times and correctly selected 21/60 times. 5.1.1. Modifications
distinct from each other as we had hoped. In the Low Participants were recruited on the Prolific crowdsourcing platform. Prolific helps reach out to individuals from different backgrounds and with diferent native languages, which can be an important factor when evaluating CA conversations made in English. We used the platform’s pre-screening tool to limit the participants to those with an approval rate of at least 95%, at least 50 previous submissions, and excluded the participants of the first iteration from the second.
sonalities can be dificult. Interpreting and rating more obscure language cues such as ”Impulsive” and ”Vague” can vary for each individual. Rating the overall conscientiousness or extroversion should prove to be more
C script, the language cues for ’vague’ and ’negativity’ version scripts. Some of the individual messages were were strengthened. For example, Low C’s message, ”And often confused with High C, and the ’dissatisfaction’ lanwhat do you think about the stuf I told you about right guage cue was not paired with Low E as much as we had now? Would it harm you to try it or was I being helpful?” hoped, so we ended up making minor changes to that was given an opening line: ”Finally we continue. language cue. As an example, in ”Honestly, the best ways
For High C personality, we increased the ’insightful’ to deal with low mood are the ones you derive pleasure from. and ’informativeness’ language cues. The result was a Unfortunately only you know what those are. What excites more proactive personality that gives the users more you? Try doing those things.” the ending was changed to suggestions. For the High C literature self-help message, ”... derive pleasure from. Don’t ask me what that is, none the part ”If you read at least a couple of books per month, knows those but you. Perhaps you could try to also talk to that would be really beneficial for you, and if you read even your friends if you enjoy that kind of thing?”. more, then that would be great!” was added.
We did not make any significant changes to the extro
After the changes, we conducted the second iteration of the validation. The changes made to the conversation script significantly improved the distinction between conscientiousness sets 1 and 2, as shown in Figure 3. Again, using the Wilcoxon Rank-Sum test there was a significant diference in set 2 (p < .05, W = 125) when comparing between iterations, but not for set 1 (p = .30, W = 69). Regardless, the visual inspection for set 1 indicated the separation between Low C and High C was more clear in iteration 2. Overall, the distinction between Low C and High C was now efectively communicated in the conversation script.
The language cues for extroversion showed some variance in accuracy compared to the first iteration but were still mostly correct. The only incorrect language cue was ’Realism’ for extroversion set 1. The Likert-style items show the actual improvements, as even though Low E set 2 was rated higher on the extroversion scale (even up to scores of 7) than on iteration 1, the diference between Low E and High E was clear for all three question sets in both iterations (as can be seen in Figure 4).
The third section’s results can be seen in Table 1. While the iteration did help with Low E, it was still often confused with High C.
Overall, after the second iteration, we can see significant improvements in the validation. We especially man
The next step for our generated CAs is to evaluate how their diferent personalities are perceived by individuals that are either seeking or could consider mental health help using self-help methods. We will also consider how the two scales correspond to the individual’s own personality. Before moving on, we plan on validating the neutral conversation to make sure it is actually perceived as the neutral option on the two scales. aged to improve the distinction between the conscientiousness personality types.
The use of the Big Five characteristics has seen critique regarding their use for generalization, especially for more vague traits such as neuroticism. The conception of the traits can vary from person to person, even for the more 6.1. Limitations obvious traits, such as those used in our design; extroversion and conscientiousness. 11 out of the 40 participants The use of Big Five personality traits for analysis can had one (or in some cases more than one) response which be influenced by cultural or language-based diferences significantly difered from the general consensus, e.g., rat- between study participants. We did not observe any such ing both options as high on a scale, or rating High C as diferences according to the country of origin or first Low C and vice versa. Clearly, in some cases, personal language of our study participants. perception of what is considered, for example, conscientious behavior, can alter results in studies like ours. 7. Conclusion Big Five has been critiqued for its traits to be perceived diferently according to diferent cultural upbringing or This paper presented the design and validation processes diferent language skills (first languages) for studies based of the created CA personalities to be used in future studon lexical analysis. However, we did not observe any sig- ies. We found the low and high variants to be easily nificant influence of country of origin or first language distinguishable from each other, but there are dificulties on the outliers. when choosing between extroversion and conscientiousness as several language cues are shared. Changes in the 6. Discussion and Future Work conversations improved the results. Further validation, including validation of the neutral personality, will be Our work aims to expand on previous research, e.g., conducted before the next step in our research. Heudin et al. [12] showing increased performance of multi-personality CAs. Improving the user experience Acknowledgments and making self-help tools more available could increase individuals’ mental health well-being considerably. To This research is connected to the GenZ strategic profilpair the user with the CA best matching their desired ing project at the University of Oulu, supported by the traits and personality, we need to consider ways to match Academy of Finland (project number 318930) and CRITand adapt the CA depending on the user, which has been ICAL (Academy of Finland Strategic Research, 335729). done for extroversion-based agents [27]. Our work fo- Part of the work was also carried out with the support of cuses on the mental health self-help context and expands Biocenter Oulu, spearhead project ICON. with consideration of the conscientiousness personality trait.
One of the key factors when designing personalities is References consistency [26]. Identifying both low and high variants of conscientiousness and extroversion proved reliable. [1] L. Laranjo, A. G. Dunn, H. L. Tong, A. B. Kocaballi, Cultural and language diferences can lead to a diferent J. Chen, R. Bashir, D. Surian, B. Gallego, F. Magrabi, understanding of language cues, and especially for lexical A. Y. Lau, et al., Conversational agents in healthcare: analysis, the use of Big Five has received its share of a systematic review, Journal of the American Medicritique [46]. Ultimately, one could ask, ”Do we know cal Informatics Association 25 (2018) 1248–1258. whose idea of conscientiousness has been encoded into this [2] J. L. Z. Montenegro, C. A. da Costa, R. da Rosa Righi, CA?” - and from an individual’s perspective, the language Survey of conversational agents in health, Expert used by a CA could be accurate for one but misleading Systems with Applications 129 (2019) 56–67. for others. A number of our Prolific workers stood out, [3] K. H. Ly, A.-M. Ly, G. Andersson, A fully automated but overall a vast majority were very much in agreement conversational agent for promoting mental wellin assessing the CAs. However, careful consideration for being: a pilot rct using mixed methods, Internet cultural and language backgrounds should be considered interventions 10 (2017) 39–46. in the future. [4] A. N. Vaidyam, H. Wisniewski, J. D. Halamka, M. S. 61;3267853. doi:10.1145/3267851.3267853.
Kashavan, J. B. Torous, Chatbots and conversational [17] E. Ruane, S. Farrell, A. Ventresque, User agents in mental health: a review of the psychiatric Perception of Text-Based Chatbot Personallandscape, The Canadian Journal of Psychiatry 64 ity, Chatbot Research and Design, Springer (2019) 456–464. International Publishing, Cham, 2021, pp. [5] H. Gafney, W. Mansell, S. Tai, et al., Conversational 32–47. URL: https://library.biblioboard.com/ agents in the treatment of mental health problems: viewer/3af3b103-6756-11eb-8f43-0a9b31268bf5. mixed-method systematic review, JMIR mental doi:10.1007/978-3-030-68288-0_3. health 6 (2019) e14166. [18] T. L. Smestad, F. Volden, Chatbot Personalities [6] M. Mori, K. F. MacDorman, N. Kageki, The uncanny Matters, Internet Science, Springer International valley [from the field], IEEE Robotics & Automation Publishing, Cham, 2019, pp. 170–181. URL: http: Magazine 19 (2012) 98–100. //link.springer.com/10.1007/978-3-030-17705-8_15. [7] A. Følstad, C. B. Nordheim, C. A. Bjørkli, What doi:10.1007/978-3-030-17705-8_15. makes users trust a chatbot for customer service? [19] E. Svikhnushina, A. Placinta, P. Pu, User expectaan exploratory interview study, in: International tions of conversational chatbots based on online conference on internet science, Springer, 2018, pp. reviews, in: Designing Interactive Systems Confer194–208. ence 2021, 2021, pp. 1481–1491. [8] L. Ciechanowski, A. Przegalinska, M. Magnuski, [20] C. Toxtli, A. Monroy-Hernández, J. Cranshaw, UnP. Gloor, In the shades of the uncanny valley: derstanding chatbot-mediated task management, An experimental study of human–chatbot inter- in: Proceedings of the 2018 CHI conference on huaction, Future Generation Computer Systems 92 man factors in computing systems, 2018, pp. 1–6. (2019) 539–548. [21] B. De Raad, The big five personality factors: the [9] A. Muresan, H. Pohl, Chats with bots: balancing psycholexical approach to personality., Hogrefe & imitation and engagement, in: Extended Abstracts Huber Publishers, 2000. of the 2019 CHI Conference on Human Factors in [22] G. Cameron, D. Cameron, G. Megaw, R. Bond, Computing Systems, 2019, pp. 1–6. M. Mulvenna, S. O’Neill, C. Armour, M. McTear, [10] R. Ahmad, D. Siemon, S. Robra-Bissantz, Commu- Towards a chatbot for digital counselling, in: Pronicating with machines: Conversational agents ceedings of the 31st International BCS Human Comwith personality and the role of extraversion, 2020. puter Interaction Conference (HCI 2017) 31, 2017, doi:10.24251/hicss.2021.492. pp. 1–7. [11] A. Egges, S. Kshirsagar, N. Magnenat-Thalmann, [23] Y.-C. Lee, N. Yamashita, Y. Huang, Designing a chatA model for personality and emotion simulation, bot as a mediator for promoting deep self-disclosure 2003. doi:10.1007/978-3-540-45224-9_63. to a real mental health professional, Proceedings of [12] J.-C. Heudin, Emotion selection in a multi- the ACM on Human-Computer Interaction 4 (2020) personality conversational agent, 2017. doi:10. 1–27.
5220/0006113600340041. [24] K.-J. Oh, D. Lee, B. Ko, H.-J. Choi, A chatbot for [13] H. Kim, D. Y. Koh, G. Lee, J.-M. Park, Y. kyung psychiatric counseling in mental healthcare service Lim, Designing personalities of conversational based on emotional dialogue analysis and sentence agents, CHI EA 39;19, ACM, May 02, 2019, pp. generation, in: 2017 18th IEEE International Con1–6. URL: http://dl.acm.org/citation.cfm?id= ference on Mobile Data Management (MDM), IEEE, 3312887. doi:10.1145/3290607.3312887. 2017, pp. 371–375. [14] F. Mairesse, M. A. Walker, M. R. Mehl, R. K. [25] D. Elmasri, A. Maeder, A Conversational Agent Moore, Using linguistic cues for the automatic for an Online Mental Health Intervention, Brain recognition of personality in conversation and text, Informatics and Health, Springer International The Journal of artificial intelligence research 30 Publishing, Cham, 2016, pp. 243–251. URL: http: (2007) 457–500. URL: https://search.proquest.com/ //link.springer.com/10.1007/978-3-319-47103-7_24. docview/2554117340. doi:10.1613/jair.2349. doi:10.1007/978-3-319-47103-7_24. [15] H. Qian, X. Li, H. Zhong, Y. Guo, Y. Ma, Y. Zhu, [26] C. Nass, K. M. Lee, Does computer-generated Z. Liu, Z. Dou, J.-R. Wen, Pchatbot: A large-scale speech manifest personality? an experimental test dataset for personalized chatbot, ???? doi:10.1145/ of similarity-attraction, in: Proceedings of the 3404835.3463239. SIGCHI conference on Human Factors in Comput[16] S. Castillo, P. Hahn, K. Legde, D. Cunningham, ing Systems, 2000, pp. 329–336.
Personality analysis of embodied conversational [27] A. Tapus, C. Ţăpuş, M. J. Matarić, User—robot agents, IVA 39;18, ACM, Nov 05, 2018, pp. personality matching and assistive robot behavior 227–232. URL: http://dl.acm.org/citation.cfm?id&# adaptation for post-stroke rehabilitation therapy, Intelligent Service Robotics 1 (2008) 169–183. communication, Communication and personality: [28] M. Komarraju, S. J. Karau, R. R. Schmeck, A. Avdic, Trait perspectives (1998) 95–117.
The big five personality traits, learning styles, and [41] F. Heylighen, J.-M. Dewaele, Variation in the conacademic achievement, Personality and individual textuality of language: An empirical measure, Foundiferences 51 (2011) 472–477. dations of science 7 (2002) 293–340. [29] J. Karsten, B. W. Penninx, H. Riese, J. Ormel, W. A. [42] A. Furnham, Language and personality. (1990).
Nolen, C. A. Hartman, The state efect of depressive [43] S. Nowson, The language of weblogs: A study of and anxiety disorders on big five personality traits, genre and individual diferences (2006).
Journal of psychiatric research 46 (2012) 644–650. [44] C. S. Cope, Linguistic structure and personality [30] D. Peabody, L. R. Goldberg, Some determinants development., Journal of Counseling Psychology of factor structures from personality-trait descrip- 16 (1969) 1. tors., Journal of personality and social psychology [45] A. Thorne, The press of personality: A study of 57 (1989) 552. conversations between introverts and extraverts., [31] L. R. Goldberg, An alternative” description of per- Journal of Personality and Social Psychology 53 sonality”: the big-five factor structure., Journal of (1987) 718.
personality and social psychology 59 (1990) 1216. [46] J. Block, The five-factor framing of personality and [32] J. A. Kelly, J. M. Kern, B. G. Kirkley, J. N. Patter- beyond: Some ruminations, Psychological Inquiry son, T. M. Keane, Reactions to assertive versus 21 (2010) 2–25. unassertive behavior: Diferential efects for males and females and implications for assertiveness training, Behavior therapy 11 (1980) 670–682. [33] D. M. Zeifman, Predicting adult responses to infant distress: Adult characteristics associated with perceptions, emotional reactions, and timing of intervention, Infant Mental Health Journal: Oficial Publication of The World Association for Infant
Mental Health 24 (2003) 597–612. [34] D. N. Klein, R. Kotov, S. J. Buferd, Personality and depression: explanatory models and review of the evidence, Annual review of clinical psychology 7 (2011) 269–295. [35] F. Mairesse, M. A. Walker, Towards personalitybased user adaptation: psychologically informed stylistic language generation, User modeling and user-adapted interaction 20 (2010) 227–278. URL: https://link.springer. com/article/10.1007/s11257-010-9076-2.
doi:10.1007/s11257-010-9076-2. [36] S. Palan, C. Schitter, Prolific. ac—a subject pool for online experiments, Journal of Behavioral and
Experimental Finance 17 (2018) 22–27. [37] J. Pennebaker, L. King, Linguistic styles: Language use as an individual diference, Journal of personality and social psychology 77 (2000) 1296–312.
doi:10.1037//0022-3514.77.6.1296. [38] J.-M. Dewaele, A. Furnham, Extraversion: The unloved variable in applied linguistic research, Language Learning 49 (1999) 509–544. doi:https:// doi.org/10.1111/0023-8333.00098. [39] M. Mehl, S. Gosling, J. Pennebaker, Personality in its natural habitat: Manifestations and implicit folk theories of personality in daily life, Journal of personality and social psychology 90 (2006) 862–77.
doi:10.1037/0022-3514.90.5.862. [40] J. Weaver, Personality and self-perceptions about
Many topics, higher verbal output Exaggeration Pleasure talk Agreement and compliment Sympathetic, concerned about heared Simple constructions Many conjucations Few unfilled pauses Poor vocabulary Think out loud Informal language Positive emotion words Many words related to humans Few uses of although Many verbs, adverbs, pronouns Few tentative words Few negations Many swear words Shorter words