Recommender systems assist users in decision-making, where the presentation of recommended items and their explanations are critical factors for enhancing the overall user experience. Although various methods for generating explanations have been proposed, there is still room for improvement, particularly for users who lack expertise in a specific item domain. In this study, we introduce the novel concept of consequence-based explanations, a type of explanation that emphasizes the individual impact of consuming a recommended item on the user, which makes the efect of following recommendations clearer. We conducted an online user study to examine our assumption about the appreciation of consequence-based explanations and their impacts on diferent explanation aims in recommender systems. Our findings highlight the importance of consequence-based explanations, which were well-received by users and efectively improved user satisfaction in recommender systems. These results provide valuable insights for designing engaging explanations that can enhance the overall user experience in decision-making.
Recommender systems (RS) assist users in decision-making by predicting which items in a catalog
will be of interest to them1][. They help users to make eficient and satisfying decisions with
many available options in various domains, including, movie selectio2n, 3[], meal choices
[
To address these needs, we introduccoensequence-based explanations, which focus on the impact of consuming recommended items as the main argument. These explanations are particularly beneficial in domains where users have limited knowledge and struggle to understand the efects of following recommendations. By emphasizing the actual changes and outcomes resulting from a recommendation, users can gain a clearer understanding of how their choices directly influence their circumstances. To validate this concept, we conducted an online user study to assess users’ appreciation of consequence-based explanations and compared diferent ways of formulating them in two domains with diferent levels of involvemen1t0][. Our study provides initial guidance for generating consequence-based explanations.
Explaining recommendations and providing users with an understanding of why a particular item is recommended emerged as an important topic in recent ye9a]r.s P[roviding such explanations is critical for shaping the user experience of an8R].S C[ onsequently, seven explanation aims have been suggested11[]: (i) Efectiveness to help users make good decisions, (ii) Eficiency to help users make decisions faster, (iiPie)rsuasiveness to convince users, (iv) Satisfaction to improve the user experience, (Svc)rutability to allow users to correct the system, (vi) Transparency to explain how the system works, and (vTiir)ust to increase users’ confidence in the system. Studies investigating the efect of diferent methods for generating and visualizing explanations show that there is no optimal approach that fits all dimensions, and explanations should be tailored to the specific goal of the RS12[].
Explanations in RS influence users by persuading them to consume recommended items
[13], increasing the perceived usefulness of the R1S4][, and contributing to overall user
satisfaction 1[
Explanation types can be categorized depending on how they are genera9]t.edMo[delintrinsic approaches use interpretable recommendation models that directly provide explanations, whilemodel-agnostic approaches generate explanations from recommended items. One common technique at this level is thceontent-based style explanation, which utilizes item features and past consumption data for generat1i6o]n. [
When users have dificulties evaluating recommended options due to limited domain
knowledge, mentioning the impact of potential choices explicitly cinonasequence-based explanation
can help. While this explanation type is new to RS, it has proven to be useful in other applications.
Reinforcement learning agents explained their behavior in terms of expected consequences of
state transitions, to increase human understanding and to enable evaluation of the plausibility
of the agents’ decisions1[
In this paper, we introducCeonsequence-based explanations, which highlight the positive and negative impacts of consuming a recommended item. This type of explanation emphasizes the individual efects of choice rather than the underlying factors that led to it. They support users in decision-making, by highlighting how the decision toward an option influences the circumstances in a desired or undesired way. For example, a movie recommender might generate a consequence-based explanation lik”Teo:y Story has been recommended to you because it will entertain your whole family, and teach your children about the value of friendship”. Those explanations can be created using a model-agnostic approach, using the feature descriptors of the recommended items, e.g., the genre and keyword tags. To simplify the decision-making process, it is advisable to include only important consequences for users, considering the multitude of potential outcomes. This aligns with how people typically communicate explanations, focusing on relevant causes rather than overwhelming with excessive informa2t0i]o.n [
This paper explores explanations in two domains representing diferleenvtels of involvement
[
Consequences are derived using a rule-based approach customized to each domain, leveraging item features and user preferences (see Sect4io.1n.1). A sentence is prepared to explain the consequence of each feature value associated with a recommended item, taking into account the specified user preferences. These individual sentences are then combined to form the complete explanation for the recommendation. For instance, the recipe recommender considers user data such as activity level andweight aim to suggest a recipe with suitable nutrient quantities. By integrating these preferences with nutritional data of recipes, the appropriate sentence for the overall explanation is determined.
We distinguish two types of consequence-based explanations based on their formulation: (i) motivating consequence-based explanations formulate the impact in a positive way, expressing which favorable consequence the suggested item has a(ivio)iding consequence-based explanations highlight the negative impact that can be prevented by choosing the suggested item. An example for both types is presented in Tab1l.eTo ensure transparency, the downsides of a recommendation are also communicated, if the suggested item does not fulfill all user preferences. For instance, in the apartment domain, an explanation might stateyotuhracth”ildren will need to share bedrooms in this apartment”.
This study evaluates two formulations of consequence-based explanations and their impact
on explanation aims in RS1[
RQ1: How do consequence-based explanations contribute to the explanation aims in RS? RQ2: How do diferent formulations of consequences influence their contribution towards the explanation aims? 1Unhealthy nutrition can have significant consequences, but the impact of a single decision to cook a recipe is low-involvement.
Motivating Consequence-based The number of carbs, sugar, and protein in the cooked meal will give you enough energy for your activity level, and the number of calories and fat in the dish will support you in losing weight.
Avoiding Consequence-based The number of carbs, sugar, and protein in the cooked meal will not fall below the needed energy for your activity level, and the number of calories and fat in the dish will not interfere with your aim of losing weight.
RQ3: How does the level of involvement of items afect the contribution of consequence-based explanations?
RQ4: Is there a correlation between users’ demographics and preferences and the efect on the explanation aims?
Answers to those questions provide insights into (i) user appreciation, (ii) preferences, and (iii) the relevance of consequence-based explanations across domains. This helps improve user support in RS with more valuable explanations.
To prepare the user study, we created small datasets consisting of 20 items per domain. These datasets were designed with feature descriptors that included the necessary properties for generating explanations. For the recipe domain, data from recipe we2bwsitaesscollected and processed in a uniform format includintgitle, description with ingredients, and agenerated image3 using the recipe title as input. Features includecutihseine type, dificulty , diet, cooking time, nutritional data, and allergenic. For the apartment datasseizte,, rent, number of bedrooms, distance to the city center, availability of private parking and garden, and distance to leisure facilities are used as features. Those samples have been created such that the content feature values are equally distributed. Those have been extended wigtehnearated photo, title, and description in the style of an apartment advertise m4.ent
Since this paper focuses on explanations, the implemented RS is simple but generates personalized recommendations based on user preferences and item features. Preferences in the recipe domain, includefavorite cuisine, followed diets, preferred cooking time, cooking skill, ingredients to be avoided, activity level, and weight aim. In the apartment domain, preferences include the 2https://www.aheadofthyme.com, https://www.seriouseats.com, https://www.themediterraneandish.com, and https://theplantbasedschool.com 3The photo was created with DALL-E 2, an AI image generathottrps(://openai.com/product/dall-e-2) 4Descriptions were generated with ChatGPT using features as input. Their correctness has been inspected manually. number of children living in the apartment, rent, preferred distance to the city center, availability of a private car, andfavorite leisure time activities. Recommendations are generated in a two-step process of candidate generation and ranki2n1g].[Firstly, the dataset is filtered for candidates fulfilling strict user preferences, i.e., diet, ingredients, cooking time, and cooking skill for the recipe domain, and city center distance and rent for the apartment domain. The remaining candidates are then ranked by compatibility withMtuhltei-Attribute Utility Method (MAUT) [22], assigning scores to item alternatives, depending on their overlap with feature preferences.
We conducted an online study adopting the design used8i]n. [Participants were randomly assigned to either of the domains, where they were presented with a scenario of deciding on a recipe to cook later that day or searching for an apartment to move in for the next years, based on their personal situation and interests. After providing demographic information (age, gender, education) and preferences (see Sect4io.1n.2), a suggested item was presented, without any content information, only described by the generated explanation. Participants were evenly distributed among the diferent explanation variants (motivating or avoiding consequence-based and content-based). They used5-apoint Likert scale to rate their likelihood of choosing the suggested item, along with their perception of the explanation’s satisfaction and understandability. Furthermore, the importance of individual consequences was rated to evaluate suggestion quality. In the final step, participants received another suggestion with a complete item description and rated their likelihood of considering it. Notably, participants were unaware that the same item was suggested twice with diferent descriptions, enabling the assessment of explanation efects by comparing rating diferences.
In total1,14 persons participated in the stu6d3y.were part of the recipe domain, whi5l1e used the apartment recommendation. Arou2n5d% of participants identified as female. In general, the participants were rather young, with an avera2g6eyoefars. Influenced by this, 51% of the participants specifiedhigh school as their highest educational degree, wh4i5l%e had graduate degrees. These demographic factors need to be considered to interpret the results.
After gathering participant responses, we evaluated the impact of consequence-based
explanations on the explanation aim1s1[] using metrics proposed in1[
= , where is the number of pairwise follow-up tests of the
The user study aims to investigate the impact of consequence-based explanations on explanation
aims in RS [
The study analyzed thietem-based eficiency , i.e., the time users spend to evaluate an item and
decide on the likelihood of consuming it1[
Comparing the domains, the item-based eficiency was slightly better in the apartment domain, without statistical significance. The duration diferences between the motivating and avoiding consequence formulation were small within the domains. Avoiding consequences were more eficient in the apartment domain, likely because pointing out the serious avoided negative consequences helps users make faster decisions. In contrast, in the recipe domain, motivating consequences were more eficient, probably because the positive impact of consuming food is more relevant in this case.
Overall, the analysis shows that consequence-based explanations improve decision-making eficiency. Considering, that they were longer3(38-749 characters) compared to the contentbased explanations1(86-524 characters), the efect on efectiveness is further strengthened. The motivation of positive consequences seems to be more eficient in low-involvement domains, while the avoidance of negative impact tends to be more eficient in high-involvement domains. This aligns with theprospect theory [23], indicating that people are risk-averse with gains but risk-seeking with losses. People tend to avoid loss in important decisions.
To measure efectiveness, the ratings for the explanation-based and content-description-based recommendations of the same user are compare8d, [24]. Similar ratings indicate efectiveness, as both ways of presenting the suggestion led to the same result. If the full content description of a recommendation receives lower ratings than the explanation, users tend to overestimate the explanation, indicating persuasiveness. On the contrary case, users tend to underestimate the explanation, indicating weak persuasiveness.
The descriptive statistics (see Ta2bale)show that all observed explanation types, including the baseline, were efective, with minimal mean rating diferences. This is expected as the necessary information to decide is already included in the explanation. However, motivating consequences showed a trend of slight persuasiveness, while avoiding consequences and the baseline tended to be underestimated. Consequence-based explanations were efective for both low- and high-involvement item domains. For apartments, users found motivating consequences slightly more persuasive, resembling advertising tactics that highlight positive aspects. For recipes, users tended to underestimate the item, particularly when avoiding consequences were mentioned, likely because negative consequences are generally weak in this domain.
The analysis suggests that consequence-based explanations provide an efective way to explain recommendations in both low- and high-involvement item domains, where the motivating formulation leads to slight persuasiveness.
To assess user satisfaction with the explanations, participants were asked to rate their
satisfaction with the recommended item24[
Based on the descriptive statistics (see T2aab)l,ewe found that user satisfaction was higher in the high-involvement domain than in the low-involvement domain, which supports our assumption and may be explained due to the higher criticality of the decision’s impact. While no diference was observed between motivating and avoiding consequences in the apartment domain, avoiding consequences received higher ratings in the recipe domain. Overall, the results indicate that consequence-based explanations are well-received by users and positively afect user satisfaction with the RS.
To evaluate theuser-perceived transparency of the explanations2[
RQ4 explores the connection between users’ demographics, preferences, and their impact on the explanation aims. Descriptive statistics for both domains are presented in2Tbables and 2c. The analysis excluded age due to participant homogeneity. Persons with higher education levels spent more time analyzing explanations in both domains, likely due to their tendency for thorough decision-making. In the recipe domain, participants with specific requirements (e.g., preferred cooking time, weight aim) evaluated explanations faster and found them more transparent, showing a better understanding of the recommendation process. Female participants, those without a preference for cooking time, and very active individuals tended to underestimate the recommended recipe. This suggests that more detailed and persuasive explanations could be beneficial for those. For apartments, participants without a university degree found the explanations more persuasive, while graduates tended to underestimate the recommended item. This indicates that graduates may prefer more detailed and comprehensible explanations. Conclusions for user satisfaction are not drawn due to homogeneous results.
To determine which features in consequences influence users’ decisions to consume a recommended item, participants rated their importance. Analyzing the data usKinrugstkhael-Wallis test showed significant diferences across the features in both domains, which have been identiifed with pairwise Mann-Whitney-U tests. The results indicate that consequences with a strong personal impact were valued higher than those describing the item more generally. In the apartment domai nfin,ancial liability is an important factor to be included, withmtohnethly rate consequence being more important than tdhiestance to the city center, a private garden, a private parking spot, number of bedrooms, anddistance to preferred leisure activities. Furthermore, thenumber of bedrooms and thedistance to the city center are more important than having a garden. In the recipe domainp,reparation time is more important than thcueisine type, dificulty level of a recipe, and individuaalctivity level. The weight aim, preferred diet, andavoidance of ingredients are more important than tahcetivity level consequence.
This paper introduced and evaluated consequence-based explanations as a new explanation type for recommendations, emphasizing the individual impact of a recommended item. Our user study confirmed user appreciation, showing a trend toward increased satisfaction. These explanations support users in making efective and eficient decisions, particularly in highinvolvement domains where the impact is crucial, and users may lack expertise. By highlighting personal impact, valuable insights for understanding the reasons behind recommendations and making informed decisions are provided.
The initial study analyzes this novel explanation type, while it acknowledges certain limitations that will be addressed in future work. One limitation is the lack of statistically significant results for some measured dimensions. A reason might be the small number of participants in the study. To overcome this, we plan to conduct a larger study with a more diverse group of participants, in terms of age, gender, and educational background, to gain a deeper understanding of the findings. Another limitation is the use of synthetic datasets, which may impact the evaluation of recommended items. To address this concern, we intend to validate the consequence-based explanations in a real-world study, which will provide practical insights. Finally, the current methods for generating our explanations are limited to the recommended item content. In the future, we aim to enhance this method by incorporating collaborative data and refining personalization techniques. Additionally, we expect the integration of generative AI to ofer diverse and natural explanations, ensuring their authenticity and eliminating any false consequences, as a beneficial possibility for improving the explanations.
The presented work has been developed within the research proSjtercetamdiver which is funded by the Austrian Research Promotion Agency (FFG) under the project num88b6e2r05. [13] S. Gkika, G. Lekakos, The persuasive role of explanations in recommender systems., in:
BCSS@ PERSUASIVE, 2014, pp. 59–68. [14] M. Zanker, D. Ninaus, Knowledgeable explanations for recommender systems, in: 2010 IEEE/WIC/ACM International Conference on Web Intelligence and Intelligent Agent Technology, volume 1, 2010, pp. 657–660. doi:10.1109/WI-IAT.2010.131. [15] T. N. T. Tran, V. M. Le, M. Atas, A. Felfernig, M. Stettinger, A. Popescu, Do users appreciate explanations of recommendations? an analysis in the movie domain, in: Fifteenth ACM Conference on Recommender Systems, 2021, pp. 645–650. [16] N. Tintarev, J. Masthof, Beyond Explaining Single Item Recommendations, Springer US,
New York, NY, 2022, pp. 711–756. doi1:0.1007/978-1-0716-2197-4_19. [17] J. van der Waa, J. van Diggelen, K. van den Bosch, M. Neerincx, Contrastive Explanations for Reinforcement Learning in terms of Expected Consequences, arXiv e-prints (2018). doi:10.48550/arXiv.1807.08706. [18] J. T. Ripberger, C. L. Silva, H. C. Jenkins-Smith, M. James, The influence of consequencebased messages on public responses to tornado warnings, Bulletin of the American Meteorological Society 96 (2015) 577–590. [19] A. E. Fano, S. W. Kurth, Personal choice point: helping users visualize what it means to buy a bmw, in: International Conference on Intelligent User Interfaces, 2003. [20] T. Miller, Explanation in artificial intelligence: Insights from the social sciences, Artificial
Intelligence 267 (2019) 1–38. doi1:0.1016/j.artint.2018.07.007. [21] J. Davidson, B. Livingston, D. Sampath, B. Liebald, J. Liu, P. Nandy, T. Van Vleet, U. Gargi, S. Gupta, Y. He, M. Lambert, The YouTube video recommendation system, in: Proceedings of the fourth ACM conference on Recommender systems - RecSys ’10, ACM Press, Barcelona, Spain, 2010, p. 293. do1i:0.1145/1864708.1864770. [22] S. Jansen, The Multi-attribute Utility Method, 2011, pp. 101–125.10d.o1i0:07/ 978-90-481-8894-9_5. [23] D. Kahneman, A. Tversky, Prospect theory: An analysis of decision under risk, Econometrica 47 (1979) 263–291. [24] N. Tintarev, J. Masthof, Evaluating the efectiveness of explanations for recommender systems: Methodological issues and empirical studies on the impact of personalization, User Modeling and User-Adapted Interaction 22 (2012) 399–439.