Explanations have a long history in recommender systems. Researchers have studied the diferent roles explanations can play, the value of explanations for users, and diferent techniques for generating explanations for a given output. To date, we have rarely seen recommender systems make use of comparative explanations, a technique that social scientists emphasize as important in human explanatory behavior. We believe that comparative explanation could be a very powerful tool to augment explanations that recommender systems currently provide and to ofer new types of transparency. In this paper, we provide a taxonomy of diferent types of comparative explanations for recommender systems, emphasizing in particular the potential value of comparative explanations for recommender system providers. We suggest directions for future research to realize this potential rather than providing solutions.
• Scope: Comparative explanations will difer in form depending on whether we are comparing recommendations of individual items or the general behavior of the recommender system over time or over some set of items.
Because this paper is interested in comparative explanation and because we are taking a multistakeholder approach, there are a wide variety of diferent scopes to be considered, most of which have not seen any research attention. For our purposes in this paper, we consider scope as a two-level construct. First, there is the question of what aspect of recommendation output is being compared. Possible answers include: • The presence of an item in a recommendation list, • The rank of an item in a recommendation list, or • A pattern of recommendation that spans multiple items in a list or multiple lists.
The next aspect of scope for comparative explanation is the question of the source of the comparison. We can compare: • Recommendations given to a single recommendation consumer, • Recommendations given to two diferent consumers: cross-consumer comparison, or • Recommendations of items from a given provider.1
Finally, we have the question of audience. Not all types of explanation are appropriate for all audiences. There may be privacy or business reasons why not all types of explanations should be available to all audiences.2 Table 1 illustrates the space of comparative explanations using a letter code to uniquely identify each possibility. We use dashes to indicate those explanations that are unlikely to be used in real systems because of potential violations of users’ privacy or confidentiality expectations.
For example, consider an item-oriented cross-consumer explanation being delivered to an individual, the missing iccC cell in the table. Such an explanation would answer a question like “Why is User A being recommended Item X and I am not?”. There are several reasons to doubt whether such an explanation could or should be part of an explainer’s output. First, this question assumes that the user knows what is being recommended to someone else, and in addition, any answer to this question would by necessity divulge information about User A’s profile. Similar considerations exist about divulging user information to providers or cross-provider information, which might be considered confidential business data in an e-commerce setting. In this paper, we focus on explanation types that would not create such risks to confidentiality.
There are nine types of comparative explanations that we consider. Examples of the questions such explanations would respond to are shown here: • Consumer-oriented questions • Provider-oriented questions – Why is Item A being recommended to me instead of Item B? (icC) – Why is Item A being ranked ahead of Item B in my recommendations? (rcC) – Why are items of type A being recommended to me more than items of type B? (tcC) – Why is my Item A being recommended to users of type X and not users of type Y? (iccP) – Why is my Item A being recommended and my Item B is not being recommended? (ipP) – Why is my Item A being ranked higher to users of type X than users of type Y? (rccP) – Why is my Item A being ranked higher than my Item B in recommendation lists? (rpP) 1It is possible to have comparative explanations across providers, but these would likely violate providers’ privacy and so would be accessible only to system stakeholders. 2As noted above, we are setting aside the System stakeholder for future work. Providing explanations to this set of users amounts to building a dashboard for examining all aspects of recommendation outputs. This is a worthy task, but we are more interested in users who would not be experts in the design and operation of the system.
Scope
Item (i) Rank (r) Pattern (t)
Source of Comparison Consumer (c) Cross-Consumer (cc) Provider (p) Consumer Cross-Consumer Provider Consumer Cross-Consumer Provider
Audience Consumer (C) Provider (P) icC — — iccP — ipP rcC — — rccP — rpP tcC — — tccP — tpP
– Why are my items of type A being recommended to users of type X and not users of type
Y? (tccP) – Why are my items of type A being recommended more often than my items of type B? (tpP) In the rest of this paper, we will examine in detail hypothetical scenarios that illustrate each type of comparative explanation, discuss the benefit of such explanations, and consider what would be required to generate appropriate explanations in response to these hypothetical questions. Our scenarios are non-exhaustive and created to illustrate our comparative explanation taxonomy.
This paper brings together ideas from explanation in recommender systems, multistakeholder recommendation, and comparative / constrastive explanation from explainable AI.
Explanation has a long history in recommender systems research, starting with the seminal work of
Herlocker et al. [
The comparative explanations in recommendation explored in [
An explanation can be considered an answer to the question, "Why P?" where "P" represents the explicit
event that occurred and needs to be explained. However, studies in social science and philosophy show
that "why" questions are often more complex than this straightforward approach [
This raises an important question in the field of Explainable AI (XAI): "What constitutes a good
explanation?" To address this, Miller in [
Contrastive explanation has been studied in image classification and other areas of AI. For image classification, a contrastive explanation could be framed as follows: a classifier predicts label for input because, if were slightly modified to , the classifier would instead predict label where X can be important object pixels in an image [11, 12, 13]. [11] proposes a contrastive explanation method for explaining image black-box classifiers. This method identifies the minimal set of pixels that must be present in an image to justify its classification and the minimal set of pixels that must be absent to distinguish it from a similar input close to the original. Note that the contrastive aspect is between the actual image and a hypothetical alternative that would be classified diferently.
In text generation settings, where the output is a sequence of words rather than a single label, the explanation might be framed as the LLM outputs a reply to a given prompt because if the prompt was slightly modified, the LLM would have given a diferent response [14, 15, 16, 17].
Jacovi et al. [18] introduce a comparative explanation method for model interpretability by generating a contrastive latent representation. This method projects the latent representation of the input space into a new space that distinguishes between two diferent decisions made by the model. They use an interventionist approach [19] to determine the causality of a factor by intervening on it, thus generating a counterfactual. In all of these contrastive explanation studies, although the questions do not directly compare two types of events, they explain the event in a comparative manner for the purpose of justification.
Although the multistakeholder perspective on recommender systems has achieved recognition in recent
research [
As noted above, explanation in recommender systems has historically been directed towards consumers.
Providing consumer-oriented explanations can build trust and satisfaction among users by helping
them understand why certain recommendations are being made [
In demonstrating these explanation types, we will rely on hypothetical scenarios from two very different domains: music streaming recommendation (an application oriented towards a general audience) and the recommendation of scientific literature (an application targeted towards specialists). Also note that music streaming is more asymmetrical (with consumers less likely to have provider roles as well) whereas the consumers of recommendations about scientific papers are likely to be individuals who also publish such papers.
Question template: Why is Item A being recommended to me instead of Item B?
The first type of comparative explanation for a single consumer involves comparing two diferent
individual items when one item was explicitly recommended and the other item was not. In Miller’s
terminology, the first item is referred to as the fact, and the second as the foil [
Consider the following hypothetical example: Sarah is a frequent user of the Tunester platform, which she uses to listen to music. The platform uses collaborative filtering to recommend music to users by finding similar users (neighbors) and recommending songs that those neighbors like. Each time it recommends 10 songs in descending order based on the predicted score for each track. Sarah received a recommendation list from the platform, and the first song on the list “Alice Abroad” was from her favorite band, Wolf Law. Since she is a fan of this artist, she is familiar with other songs by them, especially “Because, Because” which she really likes. She becomes curious about why the recommendation system recommended “Alice Abroad” to her instead of “Because, Because.” In answer to this question, the comparative explainer might say, “85% of your close neighbors listened to ‘Alice Abroad’ while only 10% of your neighbors listened to ‘Because, Because’. That’s why ‘Alice Abroad’ was recommended.” This explanation makes it clear that a collaborative recommender is in use and that Sarah’s recommendations are a function of what her peers on the platform are listening to.
This kind of explanation would be useful in any environment where the user is likely to have extensive knowledge of potential items for recommendation, such as popular culture or media, because it depends on the consumer having specific knowledge of items they might expect to be recommended to them. In other settings, it might be less useful; for example, for restaurant recommendations in an unfamiliar city, the consumer might be unlikely to have an alternative ‘foil’ restaurant about which they are seeking an explanation.
To generate such an explanation, the comparative explainer needs access to key steps in the recommendation computation: the selection of peers, the characteristics of these peers’ profiles, and the extrapolation of recommendations. Since the user is supplying the two entities to be contrasted, the explainer can focus on the diference between how these entities were treated in the original recommendation calculation, or it could run the recommendation calculation again, with these two entities as targets, and extract the diferences.
Question template: Why is Item A being ranked ahead of Item B in my recommendations?
Items are typically recommended to users in a ranked list, with each item’s rank indicating its importance and priority relative to the others. Users expect that the first item is the most relevant and best aligned with their preferences. However, the ranking process is often not transparent, leaving users uncertain as to how rankings are derived.
We can examine this explanation style with reference to Sarah and the Tunester platform. Recently, Sarah noticed that in a list of recommendations, “All the Things You Are” (the first recommendation) and “Bolivar Blues” (the seventh recommendation) are both in the Jazz genre and by the same artist. Curious about the ranking, she asks why “All the Things You Are” is prioritized over “Bolivar Blues”. The comparative explainer responds by saying “Our recommender likes to promote artists’ newer work. ‘All the Things You Are’ is recommended over ‘Bolivar Blues’ because it is a recently-released track.”
We see that the explanation in this case focuses attention on a particular feature that the recommender system takes into account in ranking, one that the user might not be aware of. From Sarah’s point of view, these tracks are very similar; the explanation provides an opportunity for the recommender to indicate what distinctions it is making. This type of recommendation could also be useful in contexts where the user might not be aware of some of the key distinctions between items. For example, consider an e-commerce setting, where a consumer is purchasing an electronics product, such as a laptop dock. These products come in many diferent configurations and capabilities, so inquiring about the recommender’s ranking may help the user understand the diferences between the products.
As in our first example, the explainer needs access to the recommendation computation. To get the type of explanation we have envisioned in this case, we can imagine a causal model linking the songs’ features to their weight in the item representation and to the recommendation calculation. The release date would be a key diference between the casual reasoning in each case since the songs are similar in other ways, and could be used as the basis for generating our (admittedly hypothetical) explanation.
Question template: Why are items of type A being recommended to me more than items of type B? A pattern of recommendation can span multiple items in a list or across multiple lists. There are many diferent types of patterns that users might ask about. Questions that are related to item features will have explanations similar in nature to those discussed above about comparative ranking.
A diferent type of pattern is one that takes place over time, when the recommender is making certain recommendations at one time and diferent recommendations at another. Such temporal changes in recommendation patterns can be categorized as either contextual shifts or preference drifts. Contextual shift refers to changes in the context or environment in which recommendations are made. This context can include observable factors such as the time of day, location, the user’s current mood, or even external factors that might not be observable to users. Preference drift, on the other hand, describes the gradual evolution in a user’s preferences over time. Unlike contextual shifts, which are temporary and context-dependent, preference drift reflects long-term changes in what a user likes or needs [ 26]. While users might not always be aware of these underlying changes, they may notice the changed nature of the content or products recommended to them. Providing explanations can help users recognize and understand these changes, such as why certain recommendations may have shifted or diversified over time.
Consider the following hypothetical example: Sanjay is a final-year PhD student in mathematics; the title of his dissertation is “Unconstrained Optimization: a New Faster Method for Solving Nonlinear Equations.” During his PhD studies, he has published several papers. He is a regular user of Moogle Scholar, a website that allows users to subscribe to a daily newsletter of recommended scientific papers according to their interests. He has found that he generally receives paper recommendations that are highly relevant to his research area. However, recently, he has noticed a shift in the recommendations coming from the system, with a focus on optimization methods for machine learning papers – an area in which he has neither published nor has substantial knowledge. Sanjay is curious about why this change in recommendation patterns has occurred and asks: “Why are papers on machine learningoriented optimization methods now being recommended more frequently than those on mathematical optimization that I used to get?” The comparative explainer responds: “Sometimes we recommend papers based on patterns of citations coming from your work. There has been a recent increase in citations of your work from researchers in the machine learning topic area and we are recommending papers by some of those researchers to you.”
Without this comparative explanation, Sanjay might not understand the reasons for the changing patterns in the recommendations he see. With the insights the explanation provides, he may want to learn more about the researchers who have picked up on his work and what they are doing. Without the transparency provided by the explanation, he might have been tempted to ignore the recommendations of these papers. We envision that temporally-oriented comparative explanations could be very helpful in explaining trends in any area. The recommender has access to the constantly changing item catalog and the evolution of user behavior and typical users will not. We will also see the value of these types of explanations for providers in the next section.
Generating such temporal explanations is not simple, however. First, it assumes that the explainer either has access to or can re-generate a trace of the recommender system’s behavior suficient to derive an explanation for the phenomenon of interest. We cannot assume that the explainer is tracking every kind of change in anticipation of some user possibly asking about it. There will be too many possible changes and too many users and most such computation would be wasted. With this capability in place, the system would still need the ability to identify the trend to which the user is referring. In this specific case, the system would need to be able to make enough sense of the question to detect that the user is asking about a change in the topic area of recommended papers. After that, the types of casual inference that we have referred to above would be needed to isolate the efect of changes in citation patterns which cause the changes in recommendations that the user sees.
A multistakeholder approach to recommendation requires that we consider the needs of stakeholders
other than recommendation consumers, especially item providers [
Question template: Why is my Item A being recommended and my Item B is not being recommended? Ideally, providers want all their items to have a fair chance of being recommended and purchased or otherwise being received by interested consumers or audiences. There is of course natural variation in the appeal of products, but it is very hard for a provider, especially one whose work is intrinsically tied to a recommendation-oriented platform (think YouTube or TikTok), to understand the interplay between the item’s properties and the relatively inscrutable behavior of the recommender system. This lack of transparency in existing algorithms often leads providers to try to develop their own folk theories from others’ experiences or ad-hoc experimentation [27, 28]. Such theories can be far afield from actual algorithm behavior. Comparative explanations of the type that we are advocating for in this paper can help close this gap. Analogous to the consumer-oriented comparative explanations discussed above, we can imagine explanations through which providers can compare the treatment of diferent items at the hands of the recommender system.
Consider the following hypothetical example: Maria is a PhD student working on the evaluation of large language models. In the final years of her PhD, she writes a conference paper based on her results and then follows it up with a more comprehensive journal article. While she is developing her application materials in advance of a job search, she consults the provider-side interface of Moogle Scholar which gives her information about her profile relative to the recommendations it generates. She notices that the conference paper is being recommended more often than the journal article, even though the journal article was published in a top journal and the conference was a relatively small one. Curious about the reasons behind this diference, she asks the system: “Why is my conference paper A being recommended more often than my journal article B?” The system responds: “One factor in our recommendation algorithm is how often a paper is cited and by whom. Publication A has a higher citation count and higher quality score (citations * centrality of citing author) than Publication B, which outweighs Publication B’s higher venue centrality score and so Publication A is more likely to be recommended in contexts where both items might be relevant.”
We see that this explanation provides a valuable bit of transparency to the researcher who can use it to understand the diference in reception of her work and might cause her to investigate how her work has been cited. We imagine such explanations to be highly useful in other recommendation applications. The YouTube creators interviewed in [24] explicitly noted their inability to understand why some of their videos were viewed heavily and other, similar, videos received hardly any attention. ipP explanations could go a long way towards helping such individuals have a more productive relationship with the recommendation platform.
Our example here assumes a capability not present in today’s recommendation platforms: a providerside interface that helps a provider understand when and in what contexts their items have been recommended. We believe that robust interfaces of this type are necessary for providers to be incorporated as first-class users of recommender systems; this is corollary of adopting a multistakeholder view of recommendation in the first place. In this example and others in this section, we proceed under the assumption that such an interface exists and comparative explanation can become one of its features.
The types of casual explanation mechanisms that we have discussed earlier in this paper could come into play in generating explanations of this type. There is a key diference, however. Providers would not have access to individual recommendation lists to be able to ask questions about how individual lists were generated. So, provider-focused explanations are by necessity explaining a pattern that has developed over multiple recommendation interactions: item A was recommended over time, item B was not. This creates complexities similar to those discussed in relation to the pattern-oriented explanations of type tcC. The recommender would need to be able to revisit a history of recommendation decisions and the underlying processes that generated them.
Question template: Why is my Item A being ranked higher than my Item B in recommendation lists? As described earlier, items are usually recommended to users in a ranked list, where items higher in the list are expected to be the most relevant and the most aligned with user preferences. While providers likely do not have access to individual user’s recommendation lists, they may be told average recommendation ranks across many users’ recommendation lists of their own items, and may wonder why there is a discrepancy in rank between two of their items (on average).
Consider the following hypothetical example: Saylor Twift is an up-and-coming singer-songwriter who just released a new EP. Her new album has five songs on it, and she and her manager thought the first song, “Orange” – a poppy, upbeat, fun song – would perform better than “Mosaic”, a slow, acoustic ballad. Twift and her team have been advertising the whole album but have been focusing on the first song. Twift’s album is on Tunester, the hypothetical music streaming platform we have seen before. The artist interface for Tunester shows that in recommendation lists, “Mosaic” is ranked, on average, higher than “Orange” on these daily playlists. Twift wants to know why this is the case. The comparative explainer responds by saying, “Our recommender considers the music tastes of listeners when recommending songs. ’Mosaic’ is recommended over ’Orange’ on average because listeners in the Pop music category who have you and artists like you in their listening history tend to listen more to slow acoustic songs with guitar than uptempo songs with synthesizer.”
In this case, the comparative explainer reveals a glimpse of how the recommender algorithm works – clearly, some assigned categorization of the songs are considered and playlists depend on engagement with artists. We can also see that the recommender characterizes music listeners and uses that to make decisions. Here, comparative explanation is particularly advantageous over simple factual explanation – asking “why does my song not have a higher ranking” may be dificult to explain by itself, since ranking is inherently rivalrous. The simple answer is that other things are ranked higher, but the comparative explanation can help the provider narrow in on what features of their items the recommendation algorithm is picking up on, which can help influence decisions on newer productions or marketing. Armed with this new knowledge, maybe Twift decides to focus more on the acoustic songs featured on her new EP. This type of explanation would be informative to any provider with multiple items being recommended and wanting greater transparency in recommender system operation.
As with the prior provider-oriented explanation, the explainer needs access to the history of recommendation slates delivered to users and the ability to generalize over multiple executions of the recommendation algorithm. A casual approach would be useful as we have seen, to identify the key features distinguishing between the treatment of one item versus another. Our hypothetical explanation assumes the system performs certain kinds of categorization (“slow” vs “uptempo”, “artists like you”) and these can be included in the explanation.
Question template: Why are my items of type A being recommended more often than my items of type B?
Although comparing individual items or individual items’ rankings may be useful for a provider, a provider might notice larger discrepancies in the recommendations of items according to specific types of their items or across time. Instead of considering a single recommendation list, the provider would ask about the recommendations of their items across multiple users.
Consider the following hypothetical example: Jo is a tenured professor of information science of 10 years. They have 600+ citations, and although much of their early work was in ethical AI and policy, recently they’ have been working more on research into ethical AI curriculum. Their papers are all available on Moogle Scholar. Recently, Jo noticed that their ethical AI and policy papers have been recommended significantly more to users. Jo wants to know why their older papers on ethical AI and policy are being recommended more often than their more recent papers on ethical AI curricula and pedagogy. The comparative explainer responds by saying, “We recommend papers that are highly relevant to our subscribers’ research interests. There are fewer scholars working on ethical AI curriculum and fewer opportunities to recommend these papers compared to the number of AI and policy researchers.”
Here the explainer is focusing on the recommendation opportunities, based on the popularity of the research area. Jo might not have a good sense for the relative popularity of diferent research fields, especially if they are not currently doing research in this area. Similar to the provider-side explanations above, this kind of insight could be useful to any provider seeking to understand larger scale dynamics of which the recommender system is aware by its position.
The explainer would need the same historical perspective that the other provider-oriented explanation types require. It would also need to understand what Jo is referring to in describing the classes of papers they wish to contrast (“ethical AI and policy” vs “ethical AI curriculum”). A causal model of how recommendations are delivered over time could highlight the diference between papers with many potential opportunities for recommendation and those with fewer.
Providers often aim for equitable distribution or recommendations of their products across diferent consumer groups (e.g. geographic location, gender, age, race/ethnicity, or stated preferences), but achieving this can be challenging due to various factors including consumer behavior, marketing strategies, or potential biases. Therefore, providers may be particularly interested in understanding the rationale behind difering levels of engagement or utility among these consumer groups. To the best of our knowledge, this type of explanation has not been studied yet in recommender systems.
Question template: Why is my Item A being recommended to users of type X and not users of type
Y? In previous examples, the key comparison occurred between two items, item rankings, or groups of items. For the cross-consumer source, however, the comparison is instead between groups of users. A provider may want to know why a specific item of theirs is recommended to one group of users and not another.
We’ll return to the example of Jo, a tenured information science professor. They have recently published a new paper, “Ethical AI Curriculum in U.S. Public High Schools: A Literature Review”. When looking at their Moogle Scholar dashboard, which shows analytics of recommendations on their papers, Jo notices that this paper is only being recommended to professors and not to grad students. Jo wants to know why this is the case, as they have recently been talking to several grad students interested in this area who might benefit from reading such a paper. The comparative explainer responds by saying, “We recommend papers that are similar in topic to papers that users have published and depend on a minimum of five papers to establish a user’s topic profile. Grad students have fewer published papers on average than professors and may not have established profiles in the system.”
The explanation reveals more about how the underlying recommender system works. Clearly, if one doesn’t have any published papers, their recommendations may not be the same as someone with published papers, even if they share common interests. This may not be desirable behavior, but at least Jo now knows why their paper is not being recommended in the way that they expect. The added benefit of comparative explanation in this context is that it can serve two purposes at once, by allowing the provider to discover why a certain group is not being recommended to and to discover why a certain group is being recommended to. Each of these may be useful in diferent contexts. For example, a queer Instagram content creator might want to know why their content is being served to transphobic people, which they do not want, and why their content is not being served to LGBTQIA+ people, which they do want. Asking only one of these questions may not reveal the full picture.
On top of the capabilities already discussed for provider-oriented explanations, this type of explanation requires information about diferent user categories: “professor” vs “graduate student”. Such categorization is assumed in the ability for Jo to even formulate their question, so this capacity is similar to what we have articulated for the other explanation types.
Question template: Why is my Item A being recommended more highly to users of type X than users of type Y? There are many factors that may cause recommendations to vary across demographic groups. There may be shared group preferences; consider teenage music listeners versus elderly ones. There may be cultural factors, such as the language in which lyrics are sung, and there may be third parties diferentially marketing items to one group over another.
Consider again our musical artist Saylor Twift. From Tunester’s artist dashboard, she discovered that “Mosaic” was recommended to European listeners at a higher average rank compared to U.S. listeners, despite equal advertising eforts in both regions. Since a higher ranking increases the likelihood of consumption, she wants to understand why the “Mosaic” track is ranked higher for European listeners than for listeners in the U.S. The comparative explainer replies: “Our recommender considers the demographics of listeners when recommending songs. ’Mosaic’ is recommended more highly in the European market on average because listeners in the Pop music category who have you and artists like you in their listening history tend to be older than their U.S. counterparts and older listeners tend to prefer acoustic music.”
It is easy to imagine explanations of this type being of use to any type of creator interested in the audience for their work and wanting to understand how the recommender is (or is not) helping to reach those audiences or markets. Analysis of this type might highlight undesired bias in a recommender system. For example, consider a company that discovers that its job listings for engineers were being recommended much more often to male applicants than female applicants. If the job recommender exhibited the type of gender bias seen in [29], the hiring manager might get an explanation saying that “We recommended your engineering positions to male applicants because they were more similar in background to prior successful applicants for similar positions than the female applicants were.” This would be a sign of unacceptable gender bias in the recommender and one would expect the manager to demand that the platform change how its algorithm operates.
Question template: Why are my items of type A being recommended to users of type X and not users of type Y? Besides asking why one specific item is recommended diferently between groups of users, a provider may wonder why a group of their items is recommended diferently. All of the benefits of a comparative explanation for a single item and comparing across consumers holds in this case as well – namely, knowing that your content is reaching your intended audience, or not, is extremely useful. The only diference here is that by looking at a pattern, group, or trend of items, a provider can potentially understand how more of their content is being recommended (or not) by the recommendation algorithm, which could lead to deeper understanding of how particular content is afected.
Let’s return to the example of Saylor Twift, a singer-songwriter who just released a new EP. The artist interface for Tunester shows that the songs from the EP overall are recommended at much higher rates to listeners in the Japan than to listeners than in the U.S., even though Twift is American and is based in the U.S. Twift wants to know why this group of songs – her EP – is recommended more to Japanese listeners and not to U.S. listeners. The comparative explainer responds by saying, “We recommend items based on trends that we anticipate when there are new releases. The new EP was just released in Japan and so is getting more recommendations due to our new release expectations while in the U.S., it has been out for three months and is outside the new release window.”
This explanation is again similar to those we have already seen in terms of its demands on the explainer. This explanation in particular requires that the explainer have access to what might be considered business rules (“release window”) about promotional activity that the platform engages in.
We have presented a taxonomy of diferent types of comparative explanations for recommender systems, where a comparative explanation is one that compares two diferent outcomes from the recommender system with the aim of providing greater transparency into the system’s operation. We note that such comparisons are recognized as a common feature of explanations in interpersonal communication.
Our taxonomy considers diferent types of explanations based on the kinds of comparisons that are being made, between which kinds of recommender output, and for which audiences / stakeholders. We note in particular that not all comparative explanations are appropriate for all audiences for privacy or confidentiality reasons. We emphasize the importance of explanations for multiple stakeholders, in particular providers, who are for the most part neglected as users in recommender systems writ large. As far as we know, this is the first work to date to examine the potential of comparative explanations broadly in recommender systems, and to suggest the possibility of and potential impacts of explanations for providers.
This taxonomy is intended as a pointer to future work as the title would suggest, since there is little or no work on the types of explanations that we describe here. Some explanation types require significant advances in how we represent and track system activity within recommender systems, and how we interface with users to solicit their explanation-oriented questions and to produce acceptable answers. We note that comparative explanation, as a task, orients recommender systems explanation squarely in the direction of system transparency. A question like “Why is A ranked higher than B in my list?” cannot be legitimately answered in a post-hoc manner, as a rationalization. It can only be answered by reference to the process by which the ranking is actually generated, prioritizing transparency. Thus, we expect that new techniques may be needed to generate explanations for complex recommendation models, but as we discuss, we believe that causal techniques may ofer some solutions.
Although some existing work has surveyed providers on their perspectives of the platforms they use, no studies have interviewed providers with the intent of designing explanations for them. A detailed survey of providers where their perspectives on explanations are gathered, as well as the current information they receive from a platform’s interface, could help influence explanation design and help articulate the value of comparative explanation. Such relationships would also be useful in future user studies of a fully developed comparative explanation system.
We opted to leave comparative explanations targeted at system stakeholders, such as platform owners and operators, from our discussion. We expect that systems designers might want all of the types of explanations included here and then some, in order to understand how their systems are treating diferent kinds of content and diferent classes of users. Most recommender system designers and operators function within well-resourced organizations, but with the rise of Very Small Online Platforms (VSOPs) [30], the need for explanation interfaces to support recommender systems in these contexts may increase.
Author Burke was supported by the National Science Foundation under grant award IIS-2107577. Author McKinnie was supported by the National Science Foundation under grant award IIS-2232555. [10] P. Ylikoski, The idea of contrastive explanandum, in: Rethinking explanation, Springer, 2007, pp.
27–42. [11] A. Dhurandhar, P.-Y. Chen, R. Luss, C.-C. Tu, P. Ting, K. Shanmugam, P. Das, Explanations based on the missing: Towards contrastive explanations with pertinent negatives, Advances in neural information processing systems 31 (2018). [12] Y. Wang, X. Wang, “why not other classes?”: Towards class-contrastive back-propagation explanations, Advances in Neural Information Processing Systems 35 (2022) 9085–9097. [13] Y. Lei, Z. Li, Y. Li, J. Zhang, H. Shan, Lico: explainable models with language-image consistency,
Advances in Neural Information Processing Systems 36 (2024). [14] R. Luss, E. Miehling, A. Dhurandhar, Cell your model: Contrastive explanation methods for large language models, ArXiv abs/2406.11785 (2024). URL: https://api.semanticscholar.org/CorpusID: 270560761. [15] S. A. Chemmengath, A. Azad, R. Luss, A. Dhurandhar, Let the cat out of the bag: Contrastive attributed explanations for text, ArXiv abs/2109.07983 (2021). URL: https://api.semanticscholar. org/CorpusID:237532193. [16] N. Madaan, I. Padhi, N. Panwar, D. Saha, Generate your counterfactuals: Towards controlled counterfactual generation for text, in: AAAI Conference on Artificial Intelligence, 2020. URL: https://api.semanticscholar.org/CorpusID:228063841. [17] T. S. Wu, M. T. Ribeiro, J. Heer, D. S. Weld, Polyjuice: Generating counterfactuals for explaining, evaluating, and improving models, in: Annual Meeting of the Association for Computational Linguistics, 2021. URL: https://api.semanticscholar.org/CorpusID:235266322. [18] A. Jacovi, S. Swayamdipta, S. Ravfogel, Y. Elazar, Y. Choi, Y. Goldberg, Contrastive explanations for model interpretability, in: Conference on Empirical Methods in Natural Language Processing, 2021. URL: https://api.semanticscholar.org/CorpusID:232092617. [19] J. Woodward, Making things happen: A theory of causal explanation, Oxford university press, 2005. [20] Y. Deldjoo, T. D. Noia, F. A. Merra, A survey on adversarial recommender systems: from attack/defense strategies to generative adversarial networks, ACM Computing Surveys (CSUR) 54 (2021) 1–38. [21] J. J. Smith, A. Satwani, R. Burke, C. Fiesler, Recommend me? designing fairness metrics with providers, in: Proceedings of the 2024 ACM Conference on Fairness, Accountability, and Transparency, FAccT ’24, Association for Computing Machinery, New York, NY, USA, 2024, p. 2389–2399.
URL: https://doi.org/10.1145/3630106.3659044. doi:10.1145/3630106.3659044. [22] K. Dinnissen, C. Bauer, Amplifying artists’ voices: Item provider perspectives on influence and fairness of music streaming platforms, in: Proceedings of the 31st ACM Conference on User Modeling, Adaptation and Personalization, UMAP ’23, Association for Computing Machinery, New York, NY, USA, 2023, p. 238–249. URL: https://doi.org/10.1145/3565472.3592960. doi:10.1145/ 3565472.3592960. [23] A. Ferraro, X. Serra, C. Bauer, What is fair? exploring the artists’ perspective on the fairness of music streaming platforms, in: C. Ardito, R. Lanzilotti, A. Malizia, H. Petrie, A. Piccinno, G. Desolda, K. Inkpen (Eds.), Human-Computer Interaction – INTERACT 2021, Springer International Publishing, Cham, 2021, pp. 562–584. [24] Y. Choi, E. Kang, M. Lee, J. Kim, Creator-friendly algorithms: Behaviors, challenges, and design opportunities in algorithmic platforms, in: CHI 2023 - Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems, Conference on Human Factors in Computing Systems - Proceedings, Association for Computing Machinery, 2023. doi:10.1145/3544548.3581386, publisher Copyright: © 2023 ACM.; 2023 CHI Conference on Human Factors in Computing Systems, CHI 2023 ; Conference date: 23-04-2023 Through 28-04-2023. [25] N. Tintarev, J. Masthof, A survey of explanations in recommender systems, in: Data Engineering
Workshop, 2007 IEEE 23rd International Conference on, IEEE, 2007, pp. 801–810. [26] N. Hariri, B. Mobasher, R. Burke, Adapting to user preference changes in interactive recommendation, in: Twenty-Fourth International Joint Conference on Artificial Intelligence, 2015.