Fostering Knowledge Exchange Using Group Recommendations Alexander Felfernig Martin Stettinger Gerhard Leitner Institute for Software Institute for Software Institute for Informatics Technology Technology Systems Inffeldgasse 16b Inffeldgasse 16b Universitätsstraße 65-67 A-8010 Graz, Austria A-8010 Graz, Austria A-9020 Klagenfurt, Austria felfernig@ist.tugraz.at stettinger@ist.tugraz.at gerhard.leitner@aau.at ABSTRACT in a skiing holiday package selection process. Choicla1 is The more domain knowledge individual participants of a a group decision support environment which includes group group decision process share with each other, the higher the recommendation technologies – this system was used as a probability of high-quality decision outcomes. In this paper basis for the user study presented in this paper. we report the results of an initial empirical study conducted Psychological aspects of group decision making play an on the basis of a group decision support environment. In increasingly important role in the development of (group) this study, groups were confronted with recommendations recommendation technologies [8]. Especially decision bi- with a varying degree of diversity. The higher the diversity ases which denote suboptimal shortcuts in decision mak- of recommendations provided to groups, the higher was the ing can lead to low-quality decision outcomes. Masthoff and degree of knowledge exchange. Gatt [11] discuss approaches to the prediction of user (group member) satisfaction with recommendations – in this con- text, conformity and emotional contagion are mentioned as Keywords major influence factors. Felfernig et al. [4, 20] analyze the impact of conformity in the context of group decision mak- Group Recommenders, Decision Support, Hidden Profiles. ing and report an increasing diversity of the preferences of group members the later individual preferences are disclosed to the whole group. Chen and Pu [2] show how emotional Categories and Subject Descriptors feedback from group members can be integrated in group H.5.3. [Group and Organization Interfaces]: (music) recommendation. An outcome of their study is that Computer-supported cooperative work. emotional feedback can enhance mutual awareness of user preferences in the group. For a short overview of decision biases in recommender systems we refer to Felfernig [3, 21]. 1. INTRODUCTION The frequency of knowledge exchange within a group can In contrast to single user recommenders [7], group rec- have a major impact on the quality of the decision outcome ommenders determine relevant items for whole groups [6, [14]. The more decision-relevant knowledge is exchanged 10]. For example, Jameson [5] introduces a prototype ap- between individual group members, the higher is the prob- plication that supports groups of users in the identification ability of discovering the hidden profile which can be char- of a holiday destination. Masthoff [9] introduces concepts acterized as the relevant knowledge to take a good (if opti- for sequencing television items for groups of users on the mality criteria exist, also an optimal) decision [22]. A conse- basis of different models from social choice theory (see also quence for group decision environments is that decision sup- [10]). O’Connor et al. [16] introduce a collaborative filter- port has to include mechanisms that pro-actively encourage ing based movie recommender system that determines rec- knowledge exchange. One reason for increased knowledge ex- ommendations for groups of users. Ninaus et al. [15] show change between group members is group diversity (in terms the application of group recommendation technologies in re- of dimensions such as demographic and educational back- quirements engineering scenarios where stakeholders are in ground), i.e., the higher the degree of diversity the higher charge of cooperatively developing, evaluating, and prioritiz- the probability of higher quality decision outcomes (mea- ing requirements. Finally, McCarthy et al. [12] introduce a sured, e.g., in terms of the degree of susceptibility to the critiquing-based recommender that supports groups of users framing effect [23]). Schulz-Hardt et al. [17] discuss the role of dissent in group decision making: the higher the dissent in initial phases of a group decision process, the higher the probability that the group manages to share the decision- relevant information (discover the hidden profile). The major focus of our empirical study was to analyze the impact of recommendation diversity on the frequency of knowledge exchange between group members. A major ACM Recommender Systems 2015, Workshop on Interfaces and Human De- reason for increasing the diversity of recommendations is the cision Making for Recommender Systems (IntRS’15), Vienna, Austria. 1 Copyright held by the authors. www.choicla.com. Figure 1: CHOICLA group decision support environment: in the description of the alternatives (alternative exam modes) T Q denotes theoretical assignment and P E denotes a practical assignment. fact that otherwise recommendations are too similar to each ments) were modeled in Choicla (see Figure 1). other and thus provide only a limited coverage of the whole Each group had the task to use the Choicla group deci- item space [13, 18]. There is always a trade-off between sion support environment to cooperatively identify a ranking similarity and diversity since too diverse recommendations for the different assignment types. Each group member had can lead to situations were relevant items are omitted, i.e., to define his/her own ranking (see Figure 1) and was not able are not recommended although relevant for the user. to see the preferences of the other group members. Partic- In this paper we do not focus on the prediction quality of ipants of the study were encouraged to take a look at the recommendation algorithms but analyze in which way rec- group recommendations (tab group preferences) which was ommendations can be used to increase knowledge exchange done by 91.41% of the participants at least once. Different between the members of a group. In the context of group group decision heuristics were used in our study and each decision making it is often more important to increase the group was assigned to a Choicla version that implemented performance of the group rather than predicting decisions exactly one of these heuristics.2 Related group recommen- that will be taken by the group. In this paper we analyze dations d differ in terms of their diversity compared to the three different basic group recommendation heuristics (min, individual user ratings (rating scale: 1-5 stars) of an alter- avg, and max group distance) with regard to their impact native s determined by eval(u, s) (see Formula 1). on the communication behavior inside a group. The basis for our analysis is an empirical study that was conducted in P u∈U sers |eval(u, s) − d| a computer science course at our university. The results of diversity(d) = (1) #U sers our analysis show that recommendation diversity can trigger additional (decision-relevant) communications. The (low diversity) minimum group distance heuristic The remainder of this paper is organized as follows. In (GDmin ) returns a rating d that represents the minimum Section 2 we describe the design of our user study and dis- distance to the ratings of group members (see Formula 2). cuss related results. In Section 3 we discuss open issues for future work. With Section 4 we conclude the paper. ! X GDmin (s) = arg min |eval (u, s) − d| (2) 2. USER STUDY d∈{1...5} u∈Users The task of each group (of undergraduate students) in The (highly diverse) maximum group distance heuristic the empirical study (N=256 participants, 12% female, 88% (GDmax ) returns a rating d that reflects the maximum dis- male) was to select their preferred exam mode for their Soft- tance to current ratings of group members (see Formula 3). ware Engineering course, for example, 1 theoretical assign- ment on Object-Relational Mapping (ORM), 1 theoretical as- ! signment on Sequence Diagrams, and two practical assign- X GDmax (s) = arg max |eval (u, s) − d| (3) ments on State Charts (see Figure 1). The participants were d∈{1...5} u∈Users informed about the fact that there is no guarantee that the articulated preferences will be taken into account in upcom- Finally, average group distance represents a value between ing exams. Each participant was a member of exactly one maximum and minimum group distance (see Formula 4). group (team) that had to implement a software within the 2 Note that Choicla includes a set of group heuristics from scope of the course. Alternative exam modes (different top- social choice theory [10], GDmax and GDavg have been in- ics and different shares of theoretical and practical assign- cluded for the purpose of the empirical study. the number of decision-relevant comments given within the GDmin (s) + GDmax (s) scope of the decision process (see Table 2). Furthermore, GDavg (s) = (4) also the overall time investments increase with the diversity 2 of the decision heuristic (see Table 3). An overview of the assignment of groups to the different decision heuristics is depicted in Table 1. prefer- recom- heuristic content ences mendation heuristic #groups #participants min 22 0 27 (+4.2) min 17 92 avg 31 26 35 (+0.9) avg 12 69 max 79 91 108 (-4.4) max 16 95 total 45 256 Table 2: Content-, preference-, and recommen- dation-related comments (valence [-5 .. +5]). Table 1: Group distribution in the empirical study. avg. heuristic endtime−starttime avg. efforts (min) Hypotheses. The basic assumption of hypothesis H1 is (h) that group decision heuristics with a higher diversity lead to an increased knowledge exchange between group members. min 71.06 (13.05) 210.71 (20.19) The reason for this is that recommendations can act as an avg 85.64 (26.58) 234.56 (17.67) anchor [1] and also have the potential to induce the feeling of max 101.18 (19.48) 278.46 (16.74) dissent in the group which needs to be resolved by the group members. An increased amount of knowledge exchange can Table 3: Time (and std.dev.) invested per group for help to discover the hidden profile of a group decision [14, decision task completion (i.e., rating of alternatives). 22], i.e., the amount of decision-relevant knowledge is in- creased. Furthermore, we assume that a higher frequency We can also confirm hypothesis H2. A higher degree of of knowledge exchange is correlated with higher time efforts knowledge exchange between group members also provides per group. flexibility regarding the final group decision. Table 4 pro- Examples of knowledge exchanged within the scope of our vides an overview of the degree of opinion adaptation of empirical study are the following (see Table 2).3 groups depending on the supported decision heuristics. Content-related. A student only took a look at exercises related to Object-Relational Mapping (ORM) and asks for avg. change heuristic further information regarding the topic. Another student of of rating the same group points out that there are only a few slides min 0.67 with very simple and understandable rules which are also avg 1.32 very useful in industrial contexts. max 2.46 Preference-related. A student emphasizes that he/she prefers to include appointments on UML Class Diagrams Table 4: Changes of initial ratings depending on the compared to appointments related to the Unified Process. supported decision heuristic. Recommendation-related. A student does not like the group recommendation since it does not take into account Summarizing, the higher the diversity of the used decision his/her preferences. Furthermore, he/she articulates an ur- heuristic, the higher the frequency of knowledge exchange gent need to further discuss assignment topics that are ac- between group members. Consequently, recommendations ceptable for the group as a whole. For recommendation- in the context of group decision support can also be exploited related comments we also evaluated the valence, i.e., how to adapt a user’s group decision behavior which can lead to positive/negative a recommendation was perceived. higher quality decision outcomes. Diverse recommendations The assumption of hypothesis H2 is that a higher de- can help to detect hidden profiles [17, 19] which represent an gree of knowledge exchange increases the flexibility of group amount of global decision-relevant knowledge needed to take members to change their initial preferences. Due to the good (or even optimal) decisions. Online group decision sup- fact that more decision-relevant knowledge is exchanged port environments have to be aware of this fact and should between group members, the amount of global decision- also take into account diversity in group recommendations. relevant knowledge is increased which improves the individ- ual capabilities of taking into account additional decision 3. FUTURE WORK alternatives. Increased knowledge exchange between group Major issues for future work are the following. Our study members helps to overcome a discussion bias (group discus- is limited in the sense of having investigated a set of basic sions tend to be dominated by information group members heuristics (diversity measures) (min, avg, and max group already knew before the discussion [19]). distance). In our future research we will investigate further Hypothesis H1 can be confirmed, i.e., the amount of deci- decision heuristics (see, e.g., [10]) with regard to their capa- sion relevant knowledge exchanged between group members bility to increase the frequency of knowledge exchange and increases with the diversity degree of the used group recom- to increase decision quality. We will also focus on a more mendation heuristic. The higher the diversity, the higher fine-grained analysis of potential optimal degrees of diversity 3 The categorization into the types content-related, that help to maximize knowledge exchange while decreasing preference-related, and recommendation-related was the perceived quality of recommendations as little as possi- performed manually. ble. The average diversity (Formula 1) of recommendations determined by the three different heuristics is depicted in [8] Mandl, M., Felfernig, A., Teppan, E., and Schubert, Table 5. We want to emphasize that the satisfaction with M. Consumer Decision Making in Knowledge-based group recommendations significantly decreases if the degree Recommendation. Journal of Intelligent Information of diversity is too high – Table 6 summarizes user feedback Systems (JIIS) 37, 1 (2010), 1–22. regarding the perceived satisfaction with the group recom- [9] Masthoff, J. Group modeling: Selecting a sequence of mendations. television items to suit a group of viewers. UMUAI 14, 1 (2004), 37–85. heuristic min avg max [10] Masthoff, J. Group Recommender Systems: diversity 0.84 1.38 2.23 Combining Individual Models. Recommender Systems Handbook (2011), 677–702. Table 5: Diversity of group recommendations. [11] Masthoff, J., and Gatt, A. In Pursuit of Satisfaction very very and the Prevention of Embarrassment: Affective State heuris- satis- aver- unsat- in Group Recommender Systems. User Modeling and satis- unsatis- tic fied age isfied User-Adapted Interaction 16, 3–4 (2006), 281–319. fied fied min 67 12 9 2 2 [12] McCarthy, K., Salamo, M., Coyle, L., McGinty, L., avg 17 14 12 14 12 Smyth, B., and Nixon, P. Group Recommender Systems: A Critiquing based Approach. In IUI’06, max 2 1 15 25 52 ACM (2006), 267–269. Table 6: Satisfaction with group recommendations. [13] McGinty, L., and Smyth, B. On the Role of Diversity in Conversational Recommender Systems. In 5th Intl. Conf. on Case-based Reasoning (2003), 276–290. 4. CONCLUSIONS [14] Mojzisch, A., and Schulz-Hardt, S. Knowing Other’s In this paper we presented the results of an initial em- Preferences Degrades the Quality of Group Decisions. pirical study that focused on possibilities of increasing the Jrnl. of Pers. & Social Psy. 98, 5 (2010), 794–808. amount of knowledge exchange in group decision scenar- [15] Ninaus, G., Felfernig, A., Stettinger, M., Reiterer, S., ios. In this context, we showed that the diversity of recom- Leitner, G., Weninger, L., and Schanil, W. Intelligent mendations can have a significant impact on the frequency techniques for software requirements engineering. In of knowledge exchange – the higher the diversity of group Europ. Conf. on AI, Prestigious Applications of recommendations, the higher the corresponding number of Intelligent Systems (PAIS) (2014), 1161–1166. comments included in the group decision process. The re- [16] O’Connor, M., Cosley, D., Konstan, J., and Riedl, J. sults presented in this paper are a first step towards the PolyLens: A Recommender System for Groups of application of recommendation technologies to foster knowl- Users. In Europ. Conf. on Computer-Supported edge exchange in group decision making. Cooperative Work, ACM (2001), 199–218. [17] Schulz-Hardt, S., Brodbeck, F., Mojzisch, A., 5. REFERENCES Kerschreiter, R., and Frey, D. Group Decision Making [1] Adomavicius, G., Bockstedt, J., Curley, S., and Zhang, in Hidden Profile Situations: Dissent as a Facilitator J. Recommender systems, consumer preferences, and of Decision Quality. Journal of Personality & Social anchoring effects. In Decisions@RecSys11 Workshop Psychology 91, 6 (2006), 1080–1093. (Chicago, IL, USA, 2011), 35–42. [18] Smyth, B., and McClave, P. Similarity vs. diversity. In [2] Chen, Y., and Pu, P. Cofeel: Emotional social 4th Intl. Conf. on Case-based Reasoning (ICCBR’01), interface in group recommender systems. In RecSys’12 Springer (London, UK, 2001), 347–361. Workshop on Interfaces for Recommender Systems [19] Stasser, G., and Titus, W. Pooling of Unshared (Dublin, Ireland, 2012), 48–55. Information in Group Decision Making: Biased [3] Felfernig, A. Biases in decision making. In Information Sharing During Discussion. Jrnl. of Pers. International Workshop on Decision Making and and Social Psy. 48, 6 (1985), 1467–1478. Recommender Systems 2014, vol. 1278 of CEUR [20] Stettinger, M., Felfernig, A., Leitner, G., and Reiterer, Proceedings (2014), 32–37. S. Counteracting Anchoring Effects in Group Decision [4] Felfernig, A., Zehentner, C., Ninaus, G., Grabner, H., Making. In UMAP 2015, vol. 9146 of LNCS (Dublin, Maaleij, W., Pagano, D., Weninger, L., and Reinfrank, Ireland, 2015), 118–130. F. Group decision support for requirements [21] Stettinger, M., Felfernig, A., Leitner, G., Reiterer, S., negotiation. In Advances in User Modeling, vol. 7138 and Jeran, M. Counteracting Serial Position Effects in of LNCS, Springer (2012), 105–116. the CHOICLA Group Decision Support Environment. [5] Jameson, A. More than the sum of its members: In ACM IUI2015 (Atlanta, GA, USA, 2015), 148–157. Challenges for group recommender systems. In Intl. [22] Wittenbaum, G., Hollingshead, A., and Botero, I. Working Conf. on Adv. Visual Interf. (2004), 48–54. From Cooperative to Motivated Information Sharing [6] Jameson, A., and Smyth, B. Recommendation to in Groups: Moving Beyond the Hidden Profile Groups. In The Adaptive Web: Methods and Strategies Paradigm. Comm. Monographs 71, 3 (2004), 286–310. of Web Personalization, P. Brusilovsky, A. Kobsa, and [23] Yaniv, I. Group Diversity and Decision Quality: W. Neijdl, Eds. Springer, 2007, ch. 20, 596–627. Amplification and Attenuation of the Framing Effect. [7] Jannach, D., Zanker, M., Felfernig, A., and Friedrich, International Journal of Forecasting 27 (2011), 41–49. G. Recommender Systems: An Introduction. Cambridge University Press, 2010.