Identifying Dominators and Followers In Group Decision Making Based on The Personality Traits Yong Zheng School of Applied Technology Illinois Institute of Technology Chicago, IL, USA yong.zheng@iit.edu ABSTRACT to group decisions, but some of other users (i.e., domina- Human factors, such as emotions, personality traits and tors) may play a dominant role in group decision making. trust network, have been proved to play an important The impact by personality in individual and group deci- role in the decision making process. The impact by per- sion making is still under investigation, especially in the sonality in individual and group decision making is still area of educational learning. For example, team work be- under investigation, especially in the area of education- comes more and more popular in the educational learn- al learning. In this paper, we propose two approaches ing. Students may be suggested to work together on the to distinguish the “dominator ” and “follower” in group assignments or projects. Decision making is involved in decision making by using an educational data. Our ex- such a scenario, e.g., how will the students build the periments also show that the characteristics of these two team, or which materials or topics should a team select user roles can further be utilized in group recommender to start learning, etc. Furthermore, it is also interest- systems to produce better item recommendations. ing to understand which group of the users may yield to group decisions. In this paper, we discuss our analysis Author Keywords to distinguish followers and dominators in the group de- recommender system, personality, group decision cision by using an educational data. Our contributions can be summarized as follows: INTRODUCTION Human factors, such as emotions, trust and personality, • We propose two approaches to identify the dominators have been recognized as influential factors in the recom- and followers in the group decision making. mender systems. For example, emotional reactions [18] • We discover and summarize the characteristics of these can be treated as strong implicit feedbacks that can rep- user roles in terms of the personality traits. resent user tastes. Trust network [8] can provide addi- • We infer the pattern in team building. tional property to infer the user preferences. User per- • We demonstrate that these characteristics are useful sonalities [10, 16] may directly affect a user’s decision, to improve the quality of group recommendations. since people with different personalities may present dis- tinct behavior patterns and preferences in the real world. RELATED WORK Recently, the importance of personality is realized not Personality has been successfully applied to improve de- only for predicting the individual tastes, but also the cision making in different areas, such as tourism [2, 14], group preferences. For example, a group of users may trading [3], career [4], etc. For example, the analysis on decide which dishes should be ordered for a group lunch. economics behaviors by Ertac, et, al. [3] helps us under- Or, a group of tourists would like to make a decision stand the role of personality in group decisions. Par- about the list of points of interests for tomorrow’s trip. ticularly, they found that openness, agreeableness and Researchers [15, 12] find out that user personality is one conscientiousness are the major three personality traits of the key factors in group decisions. For instance, some that can affect the group decisions by distinguishing the users (i.e. followers) in the group may yield their choice user roles as leaders and non-leaders. Furthermore, the personality traits are found to be useful in recommender systems, e.g., Delic, et, al. [2] observe significant patterns in user behaviors based on the personality traits which can improve the group recommender systems. In educational learning, personality has been proved to be influential. Komarraju, et al. [6] identify the impact c 2018. Copyright for the individual papers remains with the authors. of personality on the academic achievements, such as Copying permitted for private and academic purposes. GPA. Vedel, [17] focuses more on the group differences HUMANIZE ’18, March 11, 2018, Tokyo, Japan across academic majors. However, there are limited work • Collection of Individual Preferences: At the be- that explore the impact of personality on individual and ginning, each student is required to fill the question- group decision making in the learning environment. naire by himself or herself. Each subject should select at least three liked and disliked topics of the projects, In this paper, we are particularly interested in distin- and provide an overall rating to them. In addition, guishing the follower and dominators. Dominator(s) is they are asked to rate each selected project on three defined as one or more members in a group who could criteria: how interesting the application area is (i.e., be the decision leaders. By contrast, follower(s) can be App), how convenient the data processing will be (i.e., viewed as the member who may yield to the group de- Data), how easy the whole project is (i.e., Ease). The cisions. The notions are inspired by Recio-Garcia, et rating scale is from 1 to 5. al. [13]. They propose five different modes for responding to conflict situations – competing, collaborating, avoid- • Collection of Group Preferences: Finally, each ing, accommodating and compromising. The domina- student has to decide whether they will complete the tor in our paper is the user role in the competing mode, project individually. For the team work, they need while the follower represents the user role in the compro- to find partners and build the team by themselves. mising mode. However, their work relies on the Thomas- Each team will fill the same questionnaire from the Kilmann Conflict Mode Instrument (TKI) test. The sub- perspective of a team based on the group discussions. jects are required to take the test in order to be classified into these five modes. In our paper, we ignore the TKI In addition to these preference data, we collect demo- test and try to distinguish the dominator and followers graphic (e.g., age, gender, marriage status, home coun- by the rating characteristics in the data. try) information and personality traits of each studen- t. We choose the Big Five Factor (Big5) [9] which is the most popular framework to represent the personality ITMLEARNING PLATFORM traits. In the Big5 framework, the personality traits can The impact of personality on individual and group de- be described by five dimensions [6]: Openness (O) is re- cisions is under investigation in the area of educational flected in a strong intellectual curiosity and a preference learning. But unfortunately, there are no available da- for novelty and variety. Conscientiousness (C) is exem- ta sets for public research in this domain. Even in the plified by being disciplined, organized, and achievement- general area of group recommendations, most of the re- oriented. Extraversion (E) is displayed through a higher search may use the MovieLens data – the evaluation is degree of sociability, assertiveness, and talkativeness. A- usually based on the simulated groups. In this case, we greeableness (A) refers to being helpful, cooperative and start collecting our own data for the research purpose. sympathetic towards others. Neuroticism (N) indicates ITMLearning platform is built for the department of in- the degree of emotional stability, impulse control, and formation technology and management (ITM) at the Illi- anxiety. To collect the Big5 traits, we use the well-known nois Institute of Technology in USA. The platform is a Ten-Item Personality Inventory (TIPI) [5]. technology-enhanced learning system which aims to: a) The full questionnaire includes the ten statements that suggesting appropriate learning materials (e.g., books, are listed below, and the subjects are asked to give a articles, tutorials, videos); b) recommending job posi- rating in scale 1 (strongly disagree) to 7 (strongly agree) tions; c) assisting instructors in the teaching. to each of them. One of the ongoing projects from this platform is collect- • I see myself as extraverted, enthusiastic. ing students’ preferences on the topics of the projects in order to better support learning and assist teaching [19]. • I see myself as critical, quarrelsome. We start from three courses (i.e., database, data mining • I see myself as dependable, self-disciplined. and data analytics) which require students to complete a • I see myself as anxious, easily upset. project as the final evaluations. Students have their own • I see myself as open to new experiences, complex. choice to select a topic for the project, and each student can complete the project by himself/herself or by a team • I see myself as reserved, quiet. work. We ask student volunteers to complete the ques- • I see myself as sympathetic, warm. tionnaires, in order to collect the subjects’ personality • I see myself as disorganized, careless. traits and their preferences on the topics of the projects. More specifically, the questionnaires are designed to col- • I see myself as calm, emotionally stable. lect both individual and group tastes: • I see myself as conventional, uncreative. • Topics of The Projects: First of all, we provide At this moment, we have collected data for a full year a list of potential topics for each course respectively. – we obtain a data set with 194 individuals and 122 Take data analytics course for example, we provide groups. 81 out of 122 groups are composed of more the information about 50 data sets that are available than one members. More specifically, 60% of these 81 on Kaggle.com. Students should select one of them, groups are composed of two members, and the remain- define the research problems, and figure out solutions ing groups are composed of three or four members. The by using the data analytics skills. individuals leave 1951 ratings on the topics of projects, while the groups leave 745 ratings in total. In addi- Furthermore, each subject is required to provide the in- tion to the overall ratings, we collect their ratings on dividual preferences (i.e., user ratings) on the topics of three criteria as introduced above. For the purpose of the projects. Alternatively, we can represent each sub- personalization, this data is available for traditional rec- ject by his or her rating vector. The rating vector can be ommender systems (i.e., recommendations for individu- filled by the overall rating or the multi-criteria ratings on als), group recommender systems (i.e., recommendations − → app, data and ease respectively. In other words, Va and for each group), and multi-criteria recommender system- → − Vb could be rating vectors based on the overall rating s (i.e., recommendations based on multi-criteria decision or the multi-criteria ratings. The similarity between two making), as well as context-aware recommendations (i.e., subjects can be obtained by the Equation 1 accordingly. semester, year, course can be viewed as the context in- formation). The project is still ongoing and we expect 1.0 to collect more data gradually. 0.8 ANALYSIS AND DISCUSSIONS Intra−Group Similarity Personality Traits by Gender 0.6 In our data, 42% of the subjects are female. We’d like to explore whether there is a significant difference in their 0.4 personality traits in comparison with males. Table 1 presents the mean and standard deviation (SD) of the 0.2 scores in the Big5 factors for the overall, male and female individuals respectively. Big5 Rating App Data Ease Table 1. Statistics About The Personality Traits (* indi- cates significance at 95% confidence level by gender) Figure 1. Comparison of In-Group Similarities O C E A N Mean 5.22 5.05 4.63 4.85 4.11 As a result, we are able to produce the intra-group simi- Overall SD 1.28 1.34 1.49 1.47 1.53 larities by representing a user as the Big5 vector or a rat- Mean 5.15 4.89* 4.52 4.64* 4.12 ing vector. We further analyze the distribution of these Male SD 1.23 1.38 1.4 1.49 1.44 intra-group similarities, and visualize them as box plots Mean 5.32 5.27 4.78 5.14 4.1 in Figure 1. It is clear that the intra-group similarity is Female SD 1.34 1.26 1.61 1.39 1.66 significantly higher by the representations based on the Big5 factors than the ones based on user’s rating vec- In addition, we also observe that the standard deviation- tors. It implies that the subjects prefer to find the team s in neuroticism and extraversion are significantly larg- members by the personality traits, even if their tastes on er than other personality factors. The two-independent the projects may be different. The average intra-group sample hypothesis tests reveal that the difference on con- similarity based on the rating vectors is actually below scientiousness and agreeableness between males and fe- 0.5, which is surprising. males are significant at the 95% confidence level. Distinguish Dominators and Followers Team Building It has been recognized that personality can affect group We further analyze the 81 groups which are composed of decisions. Our goal is to find out distinct individuals at least two members. Students actually find their own who react differently in group decisions. More specifi- partners and build the team without intervention by the cally, we define dominator(s) as one or more members instructors. We are pretty interested in how they build in a group who are the decision leaders, and follower(s) a team or what are the most important criteria for them as the member who may yield to group decisions. We to select partners. More specifically, we try to measure try to these two user roles from the perspective of user- the intra-group similarities. group similarities and user-group conflicts which can be First of all, each subject can be represented by the Big5 further discussed as follows. Also note that our following vector. Cosine similarity, as shown by Equation 1, can analysis is based on the 81 groups which is composed of be used to produce the similarity between two subjects at least two team members. −→ → − Ua and Ub in a same team. The vectors Va and Vb are By User-Group Similarities the Big5 vectors for Ua and Ub respectively. We obtain similarity values of each pair of the subjects in a team, We have both individual and group preferences on the and the mean similarity is viewed as the intra-group sim- topics of the projects. Each individual and group can ilarity. be represented by the rating vectors. In this analysis, we focus on the overall rating only and ignore the multi- − → → − criteria ratings for simplicity. The similarity between a Va · Vb group and an individual in the group can be computed by Sim(Ua , Ub ) = −→ → − (1) ||Va ||2 × ||Vb ||2 the cosine similarity of the representations based on the Openness Openness* 6 6 5 5 4 4 3 3 Neuroticism 2 Conscientiousness Neuroticism 2 Conscientiousness 1 1 0 0 Agreeableness* Extraversion Agreeableness* Extraversion* Global Dominantor Follower Global Dominantor Follower Figure 2. Identifying User Roles by Rating Vectors Figure 3. Identifying User Roles by Latent Vectors rating vectors. If this similarity value is relatively low, We figure out a way to alleviate this problem. More it implies that this subject yields to the group decisions. specifically, we blend user ratings and group ratings to- Subjects with higher user-group similarity can be viewed gether, while each group is viewed as a special user. We as the “dominator”, while the subjects with user-group utilize a matrix factorization model based on this rating similarity smaller than a threshold can be the “follower”. matrix to find the best model which can minimize the squared prediction errors in the ratings. Finally, each us- We have two strategies to define the thresholds: er and each group can be represented by a latent vector • We can use the average value of the user-group simi- which is learned by the matrix factorization model. The larities as a single threshold. The subjects will be split user-group similarity, therefore, can be calculated by the into dominators and followers. cosine similarity of two latent factors. In our work, we use the biased matrix factorization [7] as the algorithm, and assign 10 latent factors so that each user and group • Or, we will set two thresholds. For example, we obtain will be represented by a vector with size 10. the 1st and 3rd quartile of the user-group similarities. The subjects with user-group similarity larger than 3rd Figure 3 presents new comparisons of the BIG5 traits. quartile will be viewed as dominators, while the users The dominators and followers are identified based on the with user-group similarity smaller than 1st quartile same method as mentioned above, where we represent a will be considered as the followers. team and a team member as the latent vectors learned based on the matrix factorization model. We can observe We found that the second way was better, therefore we that there are significant differences between dominators only present these results in the following sections. and followers in openness, agreeableness and extraver- Afterwards, we computer the mean Big5 vector for the sion based on the two-independent sample test at 95% subjects as dominators and followers which can be de- confidence level. More specifically, dominators present picted by the radar chart as shown in Figure 2. We higher values in the openness and extraversion, while the use “Global” to represent the mean Big5 vector of al- agreeableness value is relatively higher in the followers l the subjects. The “*” denotes a significant difference who may yield to the group decision. It is not surprising (two-independent sample test at 95% confidence level) to see that a dominator could be more extraverted since in a specific personality trait between dominators and he or she may be a talkative, confident and assertive per- followers. We can observe that the significant difference son. In terms of the openness, one explanation could be only shows up in agreeableness, while dominators actu- that dominator is usually the first person to start the ally have larger degree of agreeableness. It sounds sur- discussions in a team, and they may produce novel ideas prising to us, since we expect the followers may yield and lead the group decisions. By contrast, the follower- to the group decisions and they should present relative s present larger degree of the agreeableness, which may larger degree of agreeableness. infer that they tend to accept the group decisions even if they have different opinions. After a further investigation, we realize that the cosine similarity based on the rating vectors relies on the num- By User-Group Conflicts ber of co-ratings by a team and an individual in the team Alternatively, we can distinguish dominators and follow- – the similarity may be not reliable if the number of co- ers based on the notion of “conflicts”. Recio-Garcia, et rated items is limited. In our data, the average value of al. [13] summarized five different modes for responding to co-ratings by the teams and the team members is 3.33 conflict situations in their work – competing, collaborat- with standard deviation 2.45. We believe the results in ing, avoiding, accommodating and compromising. The Figure 2 are not reliable due to the limited number of dominator in our paper is in the competing mode, while co-ratings between a team and team member. the follower is in the compromising mode. However, the work by [13] relies on the Thomas-Kilmann Conflict – one is by the similarity between the team and its team Mode Instrument (TKI) test. In our work, we try to members, another one is by the conflicts between indi- figure out another way to define the conflicts and avoid vidual and group preferences. We find that openness, additional human efforts in the TKI test. agreeableness and extraversion are the three influential factors to recognize the dominator and followers by using More specifically, we define conflict as either the false the user-group similarities. By contrast, agreeableness is positive or the false negative case. A “false positive” the only crucial factor we find by using the method based case can be described as the situation that a subject on the conflicts. presents positive preference on one item, but his or her group finally made a negative decision on the same item. Some previous research have also identified the impor- Accordingly, the scenario that being positive on one item tant personality traits in the group decisions. For exam- by group decision but negative by a team member will ple, Ertac, et, al. [3] tried to distinguish users as leaders result in a “false negative” case. In our experiment, we and non-leaders, and they found that openness, agree- use a rating threshold to define whether it is positive or ableness and conscientiousness are the three major per- negative. More specifically, it is positive when individual sonality traits which affect the group decisions. But the or group rating on one project is larger than 3. We com- openness only takes effect if the person is a leader. Our pute the total number of conflicts (including both false findings are basically consistent with Ertac’s work. Neu- positive and false negative cases) for each team member roticism is also pointed out as a key factor by [1, 4]. But in a team. We find out that only 22.5% of the subjects we did not confirm its importance in our data. present the conflicts in our data. Accordingly, we obtain the mean, 1st and 3rd quartile RECOMMENDATIONS of the number of conflicts, and set the threshold to dis- Once we identify the dominators and followers, we fur- tinguish the followers and dominators. The process is ther exploit whether and how these findings are help- similar to the one we used to identify different user roles ful in producing the group recommendations. There are by using the user-group similarities – we can use a s- 515 ratings associated with the 81 groups which are com- ingle threshold or two thresholds. In our experiments, posed of at least two team members. We conduct a 5-fold we finally use the mean value of the number of conflicts cross validation based on these ratings – we split the 515 as the threshold to split the subjects to dominators and ratings into 5-folds. For each round evaluation, we selec- followers. t one of the five folds as the testing sets, the remaining four folds plus the data of individual ratings, information The sparsity problem is involved again since the number about group members and the user’s BIG5 traits will be of co-ratings by a team and a team member is limited. considered as the corresponding training set. We sim- We use matrix factorization model to make prediction- ply examine the recommendation performance by rating s on the unknown ratings for both subjects and teams. predictions and use mean absolute error (MAE) as the As a result, 97% of the subjects present the conflict- evaluation metric. The rating prediction for a group g ing behaviors. We use the single threshold to separate on an item t is represented by P (g, t). We adopt the the subjects to dominators and followers, while the com- following strategy in the group recommendations: parison in BIG5 can be depicted by Figure 4. We can observe that the statistical significance only presents in • Average (AVG): P (g, t) is the average predicted rating the agreeableness, while the followers usually have higher by all of the team members on the same item t. values in agreeableness. • One user choice (ONE): P (g, t) is equivalent to the preference by the dominator on the item t. If there Openness are more than one dominators, we use their average 6 5 rating predictions 1 . We set up a baseline setting for 4 the ONE method – assuming that we do not know the 3 dominators, P (g, t) will be the preference by a random Neuroticism 2 Conscientiousness team member on the item t. 1 0 • Least misery (LM): It is used to minimize the misery for the group members. P (g, t) is the minimal predict- ed rating by the team members. • Most pleasure (MP): It tries to maximize the happi- Agreeableness* Extraversion ness or pleasure for the group members. P (g, t) is the maximal predicted rating by the team members. Global Dominantor Follower For the purpose of rating predictions, we use the biased Figure 4. Identify User Roles by Conflicts matrix factorization [7] as the recommendation model. Recall that we figure out two ways to identify the dom- Summary inators and followers, we finally adopt the ones shown We try to distinguish the dominators and followers in 1 the group decision making by two proposed approaches Note that this does not happen in our experiments. by Figure 3 and 4 – we simply name them as “By Sim- explore the corresponding solutions in the future. Fur- ilarity” and “By Conflicts” in the following discussion- thermore, we evaluate the recommendation performance s. To take advantage of the identified dominators and based on simple group recommendation strategies, but followers, we simply ignore the contributions by the fol- there are several advanced work which can directly in- lowers when we execute the four recommendation strate- corporate personality in the group recommenders, such gies mentioned above. Take the AVG recommendation as the work by [11]. We will figure out how to incorpo- method for example, we will ignore the ratings by the rate the identified dominators and followers into these identified followers when we try to calculate the average advanced group recommendation models. value of the member’s rating predictions. Similar opera- tions can be applied to other recommendation strategies, REFERENCES while the ONE method will not be affected, since there 1. Kaileigh A Byrne, Crina D Silasi-Mansat, and are no followers involved. Additionally, we add another Darrell A Worthy. 2015. Who chokes under simple baseline – matrix factorization (MF) based on the pressure? The Big Five personality traits and ratings given by the teams only without considering any decision-making under pressure. Personality and dominators or followers. individual differences 74 (2015), 22–28. 2. Amra Delic, Julia Neidhardt, Thuy Ngoc Nguyen, Table 2. Comparison of MAE Francesco Ricci, Laurens Rook, Hannes Werthner, MF AVG ONE LM MP and Markus Zanker. 2016. Observing group Baseline 0.899 0.889 0.892 0.911 0.906 decision making processes. In Proceedings of the By Similarities N/A 0.874* 0.881 0.916 0.882* 10th ACM Conference on Recommender Systems. By Conflicts N/A 0.871* 0.891 0.92 0.894 ACM, 147–150. 3. Seda Ertac and Mehmet Y Gurdal. 2012. The recommendation performance based on the MAE Personality, group decision-making and leadership. metric can be presented in Table above. We can ob- Economic Research Forum Working Paper Series. serve that the AVG method is the best one among all of the baseline approaches. By incorporating the identified 4. Itamar Gati, Reuma Gadassi, Noa Saka, Yael dominators and followers, the method by user-group sim- Hadadi, Neta Ansenberg, Ronit Friedmann, and ilarities can offer significant improvement for the AVG Lisa Asulin-Peretz. 2011. Emotional and and MP strategies. The method by conflicts obtains im- personality-related aspects of career provements for the AVG method only. Note that the sig- decision-making difficulties: Facets of career nificance test was examined at the 90% confidence level. indecisiveness. Journal of Career Assessment 19, 1 Unfortunately, there are no significant improvements at (2011), 3–20. the 95% level, and the improvement is relatively small. 5. Samuel D. Gosling, Peter J. Rentfrow, and Furthermore, the method by user-group similarities William B. Swann Jr. 2003. A very brief measure of presents significant improvements in MP rather than the Big-Five Personality Domains. Journal of LM. It implies that the followers may leave false posi- Research in Personality 37 (2003), 504–528. tive contributions to the group decisions in our data. It 6. Meera Komarraju, Steven J Karau, Ronald R is because the performance can be improved if we ignore Schmeck, and Alen Avdic. 2011. The Big Five the followers in the MP method. personality traits, learning styles, and academic achievement. Personality and individual differences CONCLUSIONS & FUTURE WORK 51, 4 (2011), 472–477. In this paper, we try to distinguish the dominators and 7. Yehuda Koren, Robert Bell, and Chris Volinsky. follows in group decision making by using user-group 2009. Matrix factorization techniques for similarities and conflicts. We further take advantage of recommender systems. IEEE Computer 42, 8 the identified dominators and followers in different group (2009), 30–37. recommendation strategies. And we find that we are able to obtain significant improvements by ignoring the fol- 8. Paolo Massa and Paolo Avesani. 2007. Trust-aware lowers in the group recommendations. Furthermore, we recommender systems. In Proceedings of the 2007 find that students with similar personality tend to work ACM conference on Recommender systems. ACM, together in our data. 17–24. This paper presents our initial work, while there are plen- 9. Robert R McCrae and Oliver P John. 1992. An ty work to do. For example, the identification method introduction to the five-factor model and its by using user-group conflicts relies on the user ratings. applications. Journal of personality 60, 2 (1992), However, user bias should be taken into account in the 175–215. process to define the false positive and false negative cas- 10. Maria SN Nunes and Rong Hu. 2012. es. The approach by user-group similarities is dependent Personality-based recommender systems: an with the recommendation algorithms, since we use the overview. In Proceedings of ACM conference on algorithm to fill in the unknown ratings. We will further Recommender systems. 5–6. 11. Fernando Ortega, Antonio Hernando, Jesus Bobadilla, and Jeon Hyung Kang. 2016. Recommending items to group of users using Matrix Factorization based Collaborative Filtering. Information Sciences 345 (2016), 313–324. 12. Isaac Planas-Sitja, Jean-Louis Deneubourg, Celine Gibon, and Gregory Sempo. 2015. Group personality during collective decision-making: a multi-level approach. Proceedings of the Royal Society of London B: Biological Sciences 282, 1802 (2015), 20142515. 13. Juan A. Recio-Garcia, Guillermo Jimenez-Diaz, Antonio A. Sanchez-Ruiz, and Belen Diaz-Agudo. 2009. Personality Aware Recommendations to Groups. In Proceedings of the third ACM conference on Recommender systems. ACM, 325–328. 14. Sajad Rezaei, Faizan Ali, Muslim Amin, and Sreenivasan Jayashree. 2016. Online impulse buying of tourism products: The role of web site personality, utilitarian and hedonic web browsing. Journal of Hospitality and Tourism Technology 7, 1 (2016), 60–83. 15. Andrew Thatcher and Anthony De La Cour. 2003. Small group decision-making in face-to-face and computer-mediated environments: The role of personality. Behaviour & Information Technology 22, 3 (2003), 203–218. 16. Marko Tkalcic, Matevz Kunaver, Jurij Tasic, and Andrej Košir. 2009. Personality based user similarity measure for a collaborative recommender system. In Proceedings of the 5th Workshop on Emotion in Human-Computer Interaction-Real world challenges. 30–37. 17. Anna Vedel. 2016. Big Five personality group differences across academic majors: A systematic review. Personality and individual differences 92 (2016), 1–10. 18. Yong Zheng. 2016. Adapt to Emotional Reactions in Context-aware Personalization.. In EMPIRE Workshop @ ACM Conference on Recommender Systems. 19. Yong Zheng. 2018. Personality-Aware Decision Making In Educational Learning. In Proceedings of the 23rd International Conference on Intelligent User Interfaces Companion. ACM.