State-of-the-art recommender systems have the ability to generate high-quality recommendations, but usually cannot provide explanations to humans due to the usage of black-box prediction models. The lack of transparency has highlighted the critical importance of improving the explainability of recommender systems. In this paper, we propose to construct causal explainable recommendation which aims to provide post-hoc explanations for the recommendations by answering “what if” questions, e.g., “what would the recommendation result change if the user's behavior history had been diferent?” Our approach first obtains counterfactual user histories and counterfactual recommendation items with the aid of a perturbation model, and then extracts personalized causal relationships for the recommendation model through a causal rule mining algorithm. Diferent from some existing explainable recommendation models that aim to provide persuasive explanations, our model aims to find out the true explanations for the recommendation of an item. Therefore, in addition to evaluating the fidelity of discovered causal explanations, we adopt the average causal efect to measure the quality of explanations. Here by quality we mean whether they are true explanations rather than their persuasiveness. We conduct experiments for several state-of-the-art sequential recommendation models on real-world datasets to verify the performance of our model on generating causal explanations.
As widely used in decision-making, recommender sys- User Recommender tems have been recognized for its ability to provide high- Counterfactual History quality services that reduce the gap between products and uWsehr'asthiifsttohrey customers. And many state-of-the-art models achieves hasbeen outstanding expressiveness by using high-dimensional different? user/item representations and deep learning models with Recommender thousands or even millions of parameters [1, 2]. However, this excessive complexity easily go beyond the comprehension of a human who may demand for intuitive Figure 1: An example of causal explanation. Comparing the explanations for why the model made a specific decision. recommendation of real history and counterfactual histories, Moreover, providing supportive information and inter- if replacing one certain item will result in the change of recpretation along with the recommendation can be helpful tohmemsyesntdeamtiorenc,otmhemceenrtdasinthieteomrigcoinualdlibteemth.e true reason that for both the customers and the platform, since it improves the transparency, persuasiveness, trustworthiness, efectiveness, and user satisfaction of the recommendation systems, while facilitating system designers to refine the dation. algorithms [3]. Thus, people are looking for solutions One typical method to solve explainable recommendathat can generate explanations along with the recommen- tion is to construct a model-intrinsic explanation module that also serves as an intermediate recommendation stage[4, 5]. However, this approach has to redesign the original recommendation model and thus may sacrifice model accuracy in order to obtain good explanations [6]. Moreover, for complex deep models, it is even more challenging to integrate an explainable method into the original design while maintaining recommendation performance [3]. In contrast, post-hoc models (a.k.a modelagnostic explanation) consider the underlying recommendation model as a black-box, and provide explanations after the recommendation decision has been made. Al- able framework for a wide range of sequential though such explanations may not strictly follow the recommendations. exact mechanism that generated the corresponding rec- • We show that this framework can generate perommendations, they ofer the flexibility to be applied to a sonalized post-hoc explanations based on itemwide range of recommendation models. Furthermore, the level causal rules. explanation model and recommendation model work sep- • We conduct several experiments on real-world arately, we obtain the benefit of explainability without data to demonstrate that our explanation model hurting the prediction performance. outperforms state-of-the-art baselines in terms of
While it is still not fully understood what informa- ifdelity. tion is useful to generate the explanations for a certain • We apply average causal efect to illustrate that recommendation result, Peake [7] argued that one can the causal explanations provided by our frameprovide post-hoc item-level explanations. Specifically, in- work are essential component for most sequential teracted items (the causes) in a user’s history can be used recommendation model. as explanations for the future item recommendations (the efect).The authors propose to solve this by association For the remainder of this paper, we first review related rule mining which finds co-occurred items as explanation. work in Section 2, and then introduce our model in SecHowever, explanations generated by association rules are tion 3. Experimental settings and results are provided in not personalized, i.e., diferent users would receive the Section 4. Finally, we conclude this work in Section 5. same explanation as long as the rules are only applied to their overlapped histories. This makes it incompatible with modern recommender systems, which aim to pro- 2. Related Work vide personalized services to users. Moreover, we believe that the true explanation of a recommendation model 2.1. Sequential Recommendation should be able to answer the questions like “which item Sequential recommendation takes into account the hiscontribute to the system’s decision?” as well as “ Will torical order of items interacted by a user and aims to the system change the decision if a diferent set of items capture useful sequential patterns to make consecutive were purchased by the same user? ” In other words, predictions of the user’s future behaviors. Rendle et al. the explanation should be aware of the counterfactual [10] proposed Factorized Personalized Markov Chains world of the unobserved user histories and their corre- (FPMC) to combine Markov chain and matrix factorizasponding recommendation when analyzing the cause of tion for next basket recommendation. The Hierarchical a recommendation in real world. Representation Model (HRM) [11] further extended this
In this paper, we explore a counterfactual analysis idea by leveraging representation learning as latent facframework to provide post-hoc causal explanations for tors in a hierarchical model. However, these methods can any given black-box sequential recommendation algo- only model the local sequential patterns of very limited rithm. Fig.1 shows an example to illustrate our intuition. number of adjacent records. To model multi-step sequenTechnically, we first create several counterfactual histo- tial behaviors, He et al. [12] adopted Markov chain to ries which are diferent but similar to the real history provide recommendations with sparse sequences. Later through a Variational Auto-Encoder (VAE) based per- on, the rapid development of representation learning turbation model, and obtain the recommendation for and neural networks introduced many new techniques the counterfactual data. Then we apply causal analysis that further pushed the research of sequential recomon the combined data to extract causal rules between a mendation to a new level. For example, Hidasi et. al. user’s history and future behaviors as explanations. Un- [13] used an RNN-based model to learn the user hislike other explainable recommendation models [4, 8, 9] tory representation, Yu et. al. [14] provided a dynamic that focus on persuading users to keep engaged with recurrent model, Li et. al. [15] proposed an attentionthe system, this type of explanation focuses on model based GRU model, Chen et. al. [16] developed user- and transparency and finds out the true reason or the most item-level memory networks, and Huang et. al. [17] furessential item that leads to a specific recommendation. ther integrated knowledge graphs into memory networks. Therefore, instead of taking user studies or online evalu- However, most of the models exhibit complicated neural ations to evaluate the persuasiveness or efectiveness of network architectures, and it is usually dificult to interexplanations, we use the average causal efect to measure pret their prediction results. To make up for this, we plan whether the item used for explanation can explain how to generate explanations for these black box sequential the system works. recommendation models.
The key contributions of this paper are as follows:
• We design and study a counterfactual explain2.2. Explainable Recommendation and reinforcement learning [43], etc. With respect to recommendation tasks, large amount of work is about Explainable recommendation focuses on developing mod- how to achieve de-bias matrix factorization with causal els that can generate not only high-quality recommen- inference. The probabilistic approach ExpoMF proposed dations but also intuitive explanations, which help to in [44] directly incorporated user exposure to items into improve the transparency of the recommendation sys- collaborative filtering, where the exposure is modeled as tems [3]. Generally, the explainable models can be either a latent variable. Liang et. al. [45] followed to develop model-intrinsic or model-agnostic. As for model-intrinsic a causal inference approach to recommender systems approaches, lots of popular explainable recommendation which believed that the exposure and click data came methods, such as factorization models [4, 18, 9, 19], deep from diferent models, thus using the click data alone to learning models [20, 16, 21, 22], knowledge graph models infer the user preferences would be biased by the expo[23, 5, 17, 24, 25, 26], explanation ranking models [27], sure data. They used causal inference to correct for this logical reasoning models [1, 28, 29], dynamic explana- bias for improving generalization of recommendation tion models [30, 31], visual explanation models [8] and systems to new data. Bonner et. al. [40] proposed a new natural language generation models [32, 33, 34] have domain adaptation algorithm which was learned from been proposed. A more complete review of the related logged data including outcomes from a biased recommodels can be seen in [3]. However, they mix the recom- mendation policy, and predicted recommendation results mendation mechanism with interpretable components, according to random exposure. Besides de-bias recomwhich often results in over-complicated systems to make mendation, Ghazimatin et. al. [46] proposed PRINCE successful explanations. Moreover, the increased model model to explore counterfactual evidence for discovering complexity may reduce the interpretability. A natural causal explanations in a heterogeneous information netway to avoid this dilemma is to rely on model-agnostic work. Diferently, this paper focuses on learning causal post-hoc approaches so that the recommendation system rules to provide more intuitive explanation for the blackis free from the noises of the down-stream explanation box sequential recommendation models. Additionally, generator. Examples include [35] that proposed a bandit we consider [47] as a highly related work though it is approach, [36] that proposed a reinforcement learning originally proposed for natural language processing tasks. framework to generate sentence explanations, and [7] As we will discuss in the later sections, we utilize some that developed an association rule mining approach. Ad- of the key ideas of its model construction, and show why ditionally, some work distinguish the model explanations it works in sequential recommendation scenarios. by their purpose [37]: while persuasive explanations aim to improve user engagement, model explanation reflexes how the system really works and may not necessarily be 3. Proposed Approach persuasive. Our study fall into the later case and aims to find causal explanations for a given sequential recommendation model.
and then introduce our model as a combination of two parts: a VAE-based perturbation model that generates the 2.3. Causal Inference in Recommendation counterfactual samples for causal analysis, and a causal rule mining model that can extract causal dependencies Originated as statistical problems, causal inference [38, between the cause-efect items. 39] aims at understanding and explaining the causal effect of one variable on another. While the observational 3.1. Problem Setting data is considered as the factual world, causal efect inferences should be aware of the counterfactual world, We denote the set of users as = {1, 2, · · · , ||} thus often being regarded as the questions of "what-if". and set of items as ℐ = {1, 2. · · · , |ℐ|}. Each user The challenge is that it is often expensive or even im- is associated with a purchase history represented as a possible to obtain counterfactual data. For example, it sequence of items ℋ. The -th interacted item in the is immoral to re-do the experiment on a patient to find history is denoted as ∈ ℐ. Without specification, the out what will happen if we have not given the medicine. calligraphic ℋ in the paper represents user history, and Though the majority of causal inference study resides in a straight represents an item. A black-box sequential the direction of statistics and philosophy, it has recently recommendation model ℱ : ℋ → ℐ is a function that attracted the attention from AI community for its great takes a sequence of items (as will discuss later, it can be power of explainablity and bias elimination ability. Ef- the counterfactual user history) as input and outputs the forts have managed to bring causal inference to several recommended item. In practice, the underlying mechamachine learning areas, including recommendation [40], nism usually consists of two steps: a ranking function learning to rank [41], natural language processing [42], first scores all candidate items based on the user history, and then it selects the item with the highest score as the ifnal output. Note that it only uses user-item interaction without any content or context information, and the scores predicted by the ranking function may difer according to the tasks (e.g. {1, . . . , 5} for rating prediction, while [0, 1] for Click Through Rate (CTR) prediction).
Our goal is to find an item-level post-hoc model that captures the causal relation between the history items and the recommended item for each user.
To capture the causal dependency between items in history and the recommended items, we want to know what would take place if the user history had been diferent. To avoid unknown influences caused by the length of input sequence (i.e., user history), we keep the input length unchanged, and only replace items in the sequence to create counterfactual histories. Ideally, for each item in a user’s history ℋ, it will be replaced by all posDefinition 1. (Causal Relation) For two variables and sible items in ℐ to fully explore the influence that , if triggers , then we say that there is a causal re- makes in the history. However, the number of possible lation ⇒ , where is the cause and is the efect . combinations will become impractical for the learning system, since recommender systems usually deal with hundreds of thousands or even tens of millions items. In
When agitvoean rreeccoommmmeennddaetdiointemmodel ℱ maps a user fact, counterfactual examples that are closest to the orighistory ℋ ∈ ℐ, all items inal input can be the most useful to a user as shown in in ℋ are considered as potential causes of . Thus [48]. Therefore, we pursue a perturbation-based method we c=an{f(orm,ula)te|the∈sℋeto}f.causal relation candidates as ittheamt sgeinnethraeteorcigoiunnatleurfsaecrthuiasltoerxyamℋpl.es, which replaces Definition 2. (Causal Explanation for Sequential Rec- There are various ways to obtain the counterfactual ommendation Model) Given a causal relation candidate history, as long as they are similar to the real history. set for user , if there exists a true causal relation The simplest solution is randomly selecting an item in (, ) ∈ , then the causal explanation for recom- ℋ and replacing it with a randomly selected item from mending is described as “Because you purchased , ℐ ∖ ℋ. However, user histories are far from random. the model recommends you ”, denoted as ⇒ . Thus, we assume that their exists a ground truth user history distribution, and we adopt VAE to learn such a dis
Then the remaining problem is to determine whether tribution. As is shown in Figure 2, we design a VAE-based a candidate pair is a true causal relation. perturbation method, which creates item sequences that
We can mitigate the problem by allowing a likelihood are similar to but slightly diferent from a user’s genuine estimation for a candidate pair being a causal relation. history sequence, by sampling from a distribution in the Definition 3. (Causal Dependency) For a given candi- latent embedding space centered around the user’s true date pair of causal relation (, ), the causal depen- history sequence. dency , of that pair is the likelihood of the pair being In detail, the VAE component consists of a probabilisa true causal relation. tic encoder (, ) = ENC( ) and a decoder ˜ = DEC(). The encoder ENC(· ) takes a sequence of item
In other words, we would like to find a ranking func- embeddings into latent embedding space, and extracts tion that predicts the likelihood for each candidate pair, the variational information for the sequence, i.e., mean and the causal explanation is generated by selecting the and variance of the latent embeddings under independent pair with top ranking score from these candidates. One Gaussian distribution. The decoder DEC(· ) generates a advantage of this formulation is that it allows the possibil- sequence of item embeddings ˜ given a latent embedity of giving no causal relation between a user’s history ding sampled from the Gaussian distribution. Here, and the recommended item, e.g., when algorithm rec- both and ˜ are ordered concatenations of pre-trained ommends the most popular items regardless of the user item embeddings based on pair-wise matrix factorization history. (BPR-MF) [49]. We follow the standard training regime of VAE by maximizing the variational lower bound of the 3.2. Causal Model for Post-Hoc Explanationdata likelihood [50]. Specifically, the reconstruction error involved in this lower bound is calculated by a softmax In this section, we introduce our counterfactual explana- across all items for each position of the input sequence. tion framework for recommendation. Inspired by [47], We observe that VAE can reconstruct the original data we divide our framework into two models: a perturbation set accurately, while ofering the power of perturbation. model and a causal rule mining model. The overview of After pretraining ENC(· ) and DEC(· ), the variational the model framework is shown in Fig.2. nature of this model allows us to obtain counterfactual history ℋ˜ for any real history ℋ. More specifically, it ifrst extracts the mean and variance of the encoded item user history
Real hi st o r y Co unt er f act ual hi st o r y . . .
Encoder Decoder . . .
Perturbation Model Samplem Times
Co unt er f act ual hi st o r y Co unt er f act ual Resul t
Recommendation
Model .
. .
Co unt er f act ual pai r s
m+1 Causality.Mining .
.. ..
Causal Dependencies Rank and Selection
Personalized
Explanation and output item ˆ . We consider that the occurrence of a single output can be modeled as a logistic regression on causal dependencies from all the input items in the sequence: sequences in the latent space, and then the perturbation model samples latent embeddings based on the above variational information. These sampled embeddings are then passed to the decoder DEC(· ) to obtain the perturbed versions ˜ . For now, each item embedding in ˜ may not represent an actual item since it is a sampled vector from the latent space, as a result, we find its nearest neighbor in the candidate item set ℐ ∖ ℋ through dot product similarity as the actual item. In this way, ˜ is transformed into the final counterfactual history ℋ˜ . One should keep in mind that the variance should be kept small during sampling, so that the resulting sequences can be similar to the original sequence.
Finally, the generated counterfactual data ℋ˜ together with the original ℋ will be injected into the black-box recommendation model ℱ to obtain the recommendation results ˜ and , correspondingly. For any user , after completing this process, we will have diferent counterfactual input-output pairs: {(ℋ˜ , ˜ )}=1, as well as the original pair (ℋ, ). Here the value of is manually set, but it cannot exceed the number of all possible item combinations. where is the sigmoid function defined as () = (1 + exp(− ))− 1 in order to scale the score to [0, 1].
Additionally, in recommendation task, the order of a user’s previously interacted items may afect their causal dependency with the user’s next interaction. A closer behavior tends to have a stronger efect on user’s future behaviors, and behaviors are discounted if they happened earlier [13]. Therefore, we involve a weight decay parameter to represent the time efect. Here is a positive value less than one.
For an input-output pair in , the probability of its occurrence generated by Eq.(1) should be close to one.
As a result, we learn the causal dependencies by maximizing the probability over . When optimizing , they are always initialized as zero to allow for no causation between two items. When learning this regression model, 3.2.2. Causal Rule Learning Model we are able to gradually increase until they converge to Denote as the combined records of counterfactual the point where the data likelihood of is maximized. input-output pairs {(ℋ˜ , ˜ )}=1 and the original pair After gathering all the causal dependencies, we select (ℋ, ) for user . We aim to develop a causal model the items that have high scores to build causal explanathat first extracts causal dependencies between input and tions. This involves a three-step procedure. outputs items appeared in , and then selects the causal 1. We select those causal dependencies ^ ,^ rule based on these inferred causal dependencies. of Lthete ℋˆ-th=re[cˆord1,oˆf2, ··,· wh,eˆreˆ]be tihsethinepju-tthseiqteumencine wNohtoesethoautttphuetseis(thˆeo,rigin)aplairs (mi.ea.y, cˆom=efrom). either the original sequence or counterfactual setℋˆhat. tLheits ˆinclruedpersesthenetotrhigeicnoarlrreesaplopnadiirn(gℋou,tput)..NTohtee iqsufeendceinst,obetchaeubsleawckh-ebnoxa rceocuonmtemrfeancdtuaatilosneqmueondceel, model should be able to infer the causal dependency the output may happen to be the same as the (refer to Definition 3) ^ ,^ between input item ˆ original sequence . ^ ,^ · − )︁ (1) 4: 5: 6: 7: 8: 9: 10: 11: 12: 13: 14: 15: {(ℋ˜ , ˜ )
{(ℋˆ , ˆ ) }=1 +1 }=1 ← { ^ ,^ ← ℳ Rank
^ , {( , )}=1 if ∃min{} ∈ ℋ else end if pairs.
then
No explanation for the recommended item 16: end for 17: return all causal explanations ⇒ 2. We sort the above selected causal dependencies in descending order and take the top- (ˆ , ) 3. If there exist one or more pairs in these top- pairs, which cause item ˆ appears in the user’s input sequence ℋ, then we pick such pair of the highest rank, and construct ˆ ⇒ as the causal explanation for the user. Otherwise, i.e., no cause item appears in the user history, then we output no causal explanation for the user.
Note that the extracted causal explanation is personalized since the algorithm is applied on , which only contains records centered around the user’s original record (ℋ, ), while collaborative learning among users is there are two phases - perturbation phase (line 4-7) and causal rule mining phase (line 8-15).
counterfactual number , black-box model ℱ , embedding model ℰ , causal mining model ℳ Output: causal explanations ⇒ where ∈ ℋ
model 3: for each user do 1: Use embedding model ℰ to get item embeddings ℰ (ℐ) 2: Use ℰ (ℐ) and true user history to train perturbation ˜ ℋ ← end for for from 1 to do (ℋ); ˜ ← ℱ
(ℋ˜ ) Construct counterfactual input-output pairs (ℋ˜ , ˜ )}=1 ∪ (ℋ, )
︁( {(ℋˆ , ˆ ) +1)︁ }=1 and select top-
pairs indirectly modeled by the VAE-based perturbation model. turbation model to analyze our model. We include both The overall algorithm is provided in Alg.1. For each user, shallow and deep models for the experiment.
GRU4Rec [13]: A session-based recommendation decoder are Multi-Layer Perceptrons (MLP) with two model, which uses recurrent neural networks – in partic- hidden layers, and each layer consists of 1024 neurons. ular, Gated Recurrent Units (GRU) – to capture sequential The only diference between our model and the varipatterns for prediction4. ant model CR-AE is that the variant model applies fixed
NARM [15]: A sequential recommendation model normal distribution as variance instead of learned personwhich utilizes GRU and attention mechanism to estimate alized variance. The default number of counterfactual the importance of each interactions5. input-output pairs is = 500 on both datasets. The
Caser [51]: The ConvolutionAl Sequence Embedding default time decay factor is = 0.7. We will discuss the Recommendation (Caser) model, which adopts convo- influence of counterfactual number and time decay lutional filters over recent items to learn the sequential factor in the experiments. patterns for prediction6. In the following, we will apply our model and all base
AR-sup [7]: A post-hoc explanation model, which lines on the black-box recommendation models to evaluextract association rules from interactions from all users ate and compare the generated explanations. In particuand rank based on support value to generate item-level lar, we evaluate our framework from three perspectives. explanations. First, a explanation model should at least be able to ofer
AR-conf [7]: Extracting association rules and rank explanations for most recommendations, we will show it based on confidence value to get explanations. in the result (explanation fidelity). Second, if our model is
AR-lift [7]: Rank based on lift value among extracted capable of generating explanations for most recommenassociation rules to generate explanations. dations, we need to verify that the causal explanations
CR-AE: A variant of our causal rule model which learned by our framework represent the key component applies fixed variance in hidden layer of AutoEncoder of recommendation mechanism (explanation quality). Fimodel as the perturbation model. Compared with our nally, since counterfactual examples are involved in our VAE-based perturbation model, this variant apply non- framework, our framework should be able to generate personalized variance. closer counterfactual examples (counterfactual quality).
For black-box recommendation models FPMC, Additionally, we shed light on how our model difers GRU4Rec, NARM and Caser, we adopt their best from other models on statistical metrics. parameter selection in their corresponding public implementation. For the association rule-based explanation 4.3. Model Fidelity model, we follow the recommendations in [7] to set the optimal parameters: support = 0.1, confidence = 0.1, lift = 0.1, length = 2 for MovieLens100k, and support = 0.01, confidence = 0.01, lift = 0.01, length = 2 for Amazon dataset due to its smaller scale. We accept top 100 rules based on corresponding values (i.e. support/confidence/lift) as explanations
For our causal rule learning framework, we set the item embedding size as 16, both the VAE encoder and
is to generate explanations for most recommendations.
Therefore, an important evaluation measure for explanation models is model fidelity, i.e., what’s the percentage of the recommendation results can be explained by the model [3]. The results of model fidelity are shown in Table 2. In this experiment, we only report the results of keeping the number of candidate causal explanations as 1 for our framework and variant. For the association rule explanation model (section 4.2), we apply the global association rules [7] ranking by support, confidence, and lift, respectively. 4https://github.com/hungthanhpham94/GRU4REC-pytorch 5https://github.com/Wang-Shuo/Neural-Attentive-SessionBased-Recommendation-PyTorch 6https://github.com/graytowne/caser_pytorch
We can see that on both datasets, our causal explana- Here () represents an external intervention, which tion framework is able to generate explanations for most forces a variable to take a specific value. Specifically, in of the recommended items (including the variant), while our case, for an extracted causal rule ⇒ , we define ˜ the association explanation approach can only provide ex- the binary random variable as 1 if ∈ ℋ , 0 else. We planations for significantly fewer recommendations. The also define another variable as a binary random variable, underlying reason is that association explanations have which is 1 if ˜ = , otherwise it will be 0. We then to be extracted based on the original input-output pairs, report average ACE on all generated explanations. Note which limits the number of pairs that we can use for rule that since the ACE value is used for causal related models, extraction. However, based on the perturbation model, we cannot report it on the association rule baseline. our causal explanation framework is capable of creating Suppose the perturbation model (section 3.2.1) cremany counterfactual examples to assist causal rule learn- ates counterfactual input-output pairs for each user : ing, which makes it possible to go beyond the limited {(ℋ˜ , ˜ )}=1. Here ℋ˜ is created by our perturbation original data to extract causal explanations. Moreover, model (i.e. not observed in the original data), and thus when the number of input and output items are limited observing ∈ ℋ˜ implies we have ( = 1) in ad(e.g. five history items as input and the model recom- vance. Let ⇒ be the causal explanation extracted mends only one item in our case), it is harder to match by the casual rule learning model (section 3.2.2). Then global rules with personal interactions and recommen- we estimate the ACE based on these counterfactual dation, which limits the flexibility of global association pairs as, rules.
Another interesting observation is that GRU4Rec and E[|( = 1)] = Pr( = 1|( = 1)) Caser have significantly ( < 0.01) lower fidelity than = #Pairs( ∈ ℋ˜ ∧ = ) FPMC and NARM when explained by the association #Pairs( ∈ ℋ˜ ) model. This is reasonable because FPMC is a Markov- E[|( = 0)] = Pr( = 1|( = 0)) (2) based model that consider input as a basket and directly learns the correlation between candidate items and each = #Pairs( ∈/ ℋ˜ ∧ = ) items in a sequence, as a result, it is easier to extract asso- #Pairs( ∈/ ℋ˜ ) ciation rules between inputs and outputs for the model. We report the ACE value of our model and variants in NARM combines the whole session information and in- Table.3. While showing the ACE value, we still keep the lfuence of each individual item in the session, therefore, number of candidate causal explanations as 1. association rules which involve individual information We can see that our model can achieve higher ACE will be easier to be extracted for this model. However, value than the variant for most recommendation models it also means that the fidelity performance of the asso- on both dataset. But here we can observe an interesting ciation approach highly depends on the recommenda- results that the ACE value for FPMC model is much lower tion model being explained. Meanwhile, we see that our than other recommendation models (GRU4Rec, NARM, causal approach achieves comparably good fidelity on all Caser). Meanwhile, the variant model has slightly larger three recommendation models, because the perturbation ACE than our model when applying on FPC model. model is able to create suficient counterfactual exam- The diference between FPMC and other recommenples to break the correlation of frequently co-occurring dation models is that FPMC is based on Markov chain items in the input sequence. This indicates the robust- that only considers the last behavior while other modness of our causal explanation framework in terms of els involve the whole session information. For FPMC model fidelity. model, although we take a session as input to recommend next item, this model actually considers it as a 4.4. Average Causal Efect basket and linearly combines the influence of each item from the basket. In this case, every part of the session We then verify our causal explanations are true expla- will have independent influence towards next item prenations that explanation are important component for diction. So changing a small part of input items may not recommending original item. A common way is to mea- significantly change the next item prediction which high sure the causal efect on the outcome of the model[ 52]. likely results in same recommendation item. Based on First of all, we show the definition of Average Causal our experiment, when we keep counterfactual histories Efect. same for all recommendation models, FPMC model only Definition 4. (Average Causal Efect) The Average gets 98 counterfactual histories (19.6%) in average with Causal Efect (ACE) of a binary random variable on diferent recommendation (diferent from the recommenanother random variable is define as E[|( = 1)] − dation item based on real history), while other models E[|( = 0)] have at least 315 counterfactual histories (63%) in average with diferent recommendation item. This diference makes the FPMC model has much lower ACE value compared with other recommendation models. Comparing our model with CR-AE, the variant model will generate less similar counterfactual histories which more likely result in diferent recommendation item than our model.
Therefore, CR-AE has slightly higher ACE values than CR-VAE. 4.5. Proximity
are closest to the original can be the most useful to users. Similar with [48], we define the proximity as the distance between negative counterfactual examples (i.e. generate recommendation item diferent from original item) and original real history. Intuitively, a counterfactual example that close enough but get totally diferent results will be more helpful. For a given user, the proximity can be expressed as CR-AE CR-VAE
FPMC 4.6. Influence of Parameters
parameters. The first one is time decay parameter – in = − ( ∑︁ (ℋ˜ , ℋ)) (3) our framework, when explaining the sequential recom˜ ̸= mendation models, earlier interactions in the sequence will have discounted efects to the recommended item.
Here the distance is defined in latent space. The repre- A proper time decay parameter helps the framework to sentation of any history sequence is the concatenation reduce noise signals when learning patterns from the of the latent representation of each item in the sequence. sequence. The second parameter is the number of perThe latent representations of items are learned from pre- turbed input-output pairs – in our framework, we trained BPRMF [49] model. The distance of any two use perturbations to create counterfactual examples for sequence is defined as Euclidean distance between the causal learning, but there may exist trade-of between efrepresentation of two sequence. The reported proximity ficiency and performance. We will analyze the influence value would be the average over all users. of these two parameters.
Given that association rule model does not involve Time Decay Efect : Figure 3 shows the influence of counterfactual examples, this metric can only be reported on diferent recommendation models and datasets. From on our model and the variant model on both datasets, as the result we can see that the time decay efect indeed shown in Table.4 We can observe that our model can afects the model performance on fidelity. In particular, achieve higher proximity compared with the variant when is small, the previous interactions in a sequence model. In other words, counterfactual examples gen- are more likely to be ignored, which thus reduces the erated with learned latent variance is more similar with performance on model fidelity. When is large (e.g., real history. Therefore, higher proximity implies coun- = 1), old interactions will have equal importance with unknown Association Rule Explanation
Our Explanation latest interactions, which also hurts the performance. We can see from the results that the best performance is gives >95% confidence that the estimated probability erachieved at about = 0.7 on both datasets. ror is <0.1.
Number of Counterfactual Examples: Figure 4 shows the influence for the number of counterfactual 4.7. Case Study input-output pairs . A basic observation from Figure 4 is that when increases, model fidelity will decrease first In this section, we provide a simple case study to comand then increase. The underlying reason is as follows. pare causal explanations and association explanations.
When is small, the variance of the counterfactual Compared with the association explanation model, our input-output pairs will be small, and fewer counterfac- model is capable of generating personalized explanations, tual items will be involved. Then the model is more likely which means that even if the recommendation model recto select original item as explanation. For example, sup- ommends the same item for two diferent users and the pose the original input-output pair is , , → . In users have overlapped histories, our model still has the the extreme case where = 1, we will have only one potential to generate diferent explanations for difercounterfactual pair, e.g., , ˜, → ˜ . According to the ent users. However, the association model will provide causal rule learning model (section 3.2.2), if ˜ ̸= , then the same explanation since the association rules are ex ⇒ will be the causal explanation since the change tracted based on global records. An example by the Caser of results in a diferent output, while if ˜ = , then [51] recommendation model on MovieLens100k dataset is either ⇒ or ⇒ will be the causal explanation shown in Figure 5, where two users with one commonly since their scores will be higher than or ˜. In either watched movie (The Sound of Music) get exactly same case, the model fidelity and percentage of verified causal recommendation (Pulp Fiction). The association model rules will be 100%. However, in this case, the results do provides the overlapped movie as an explanation for the not present statistical meanings since they are estimated two diferent users, while our model can generate peron a very small amount of examples. sonalized explanation for diferent users even when they
When increases but not large enough, then random got the same recommendation. noise examples created by the perturbation model will reduce the model fidelity. Still consider the above example, 5. Conclusions if many pairs with the same output are created, then the model may find other items beyond , , as the Recommender systems are widely used in our daily life. cause, which will result in no explanation for the origi- Efective recommendation mechanisms usually work nal sequence. However, if we continue to increase to through black-box models, resulting in the lack of transsuficiently large numbers, such noise will be statistically parency. In this paper, we extract causal rules from user ofset, and thus the model fidelity and percentages will history to provide personalized, item-level, post-hoc exincrease again. In the most ideal case, we would create all planations for the black-box sequential recommendation of the |ℋ||ℐ| sequences for causal rule learning, where models. The causal explanations are extracted through a |ℋ| is the number of item slots in the input sequence, and perturbation model and a causal rule learning model. We |ℐ| is the total number of items in the dataset. However, conduct several experiments on real-world datasets, and |ℋ||ℐ| would be a huge number that makes it compu- apply our explanation framework to several state-of-thetational infeasible for causal rule learning. In practice, art sequential recommendation models. Experimental we only need to specify suficiently large. Based on results verified the quality and fidelity of the causal exChebyshev’s Inequality, we find that = 500 already planations extracted by our framework.
In this work, we only considered item-level causal relationships, while in the future, it would be interesting to explore causal relations on feature-level external data such as textual user reviews, which can help to generate ifner-grained causal explanations.
IIS-2007907. Any opinions, findings, conclusions or
recommendations expressed in this material are those of
the authors and do not necessarily reflect those of the
sponsors.
[9] N. Wang, H. Wang, Y. Jia, Y. Yin, Explainable
recommendation via multi-task learning in opinionated
text data, in: The 41st International ACM SIGIR
Conference on Research & Development in
Factorizing personalized markov chains for nextbasket recommendation, in: Proceedings of the 19th international conference on World wide web,
ACM, 2010, pp. 811–820. [11] P. Wang, J. Guo, Y. Lan, J. Xu, S. Wan, X. Cheng,
Learning hierarchical representation model for nextbasket recommendation, in: Proceedings of the 38th International ACM SIGIR conference on Research and Development in Information Retrieval,
ACM, 2015, pp. 403–412.
[12] R. He, J. McAuley, Fusing similarity models with
[1] H. Chen, S. Shi, Y. Li, Y. Zhang, Neural collaborative markov chains for sparse sequential
recommendareasoning, in: Proceedings of the Web Conference tion, in: 2016 IEEE 16th International Conference
2021, 2021, pp. 1516–1527. on Data Mining (ICDM), IEEE, 2016, pp. 191–200.
[2] S. Zhang, L. Yao, A. Sun, Y. Tay, Deep learning [13] B. Hidasi, A. Karatzoglou, L. Baltrunas, D. Tikk,
based recommender system: A survey and new Session-based recommendations with recurrent
perspectives, ACM Computing Surveys (CSUR) 52 neural networks, in: International Conference on
(2019) 1–38. Learning Representations, 2016.
[3] Y. Zhang, X. Chen, Explainable recommendation: [14] F. Yu, Q. Liu, S. Wu, L. Wang, T. Tan, A dynamic
A survey and new perspectives, Foundations and recurrent model for next basket recommendation,
Trends® in
Y. Zhang, Reinforcement knowledge graph reason- [16] X. Chen, H. Xu, Y. Zhang, J. Tang, Y. Cao, Z. Qin,
ing for explainable recommendation, in: Proceed- H. Zha, Sequential recommendation with user
memings of the 42nd International ACM SIGIR Confer- ory networks, in: Proceedings of the eleventh ACM
ence on Research and Development in Information international conference on W
Language Processing (2017). [48] R. K. Mothilal, A. Sharma, C. Tan, Explaining machine learning classifiers through diverse counterfactual explanations, in: Proceedings of the 2020 Conference on Fairness, Accountability, and Transparency, 2020. [49] S. Rendle, C. Freudenthaler, Z. Gantner, L. Schmidt
Thieme, Bpr: Bayesian personalized ranking from implicit feedback, UAI (2012). [50] D. P. Kingma, M. Welling, Auto-encoding varia
tional bayes, 2014. [51] J. Tang, K. Wang, Personalized top-n sequential recommendation via convolutional sequence embedding, in: Proceedings of the Eleventh ACM International Conference on Web Search and Data
Mining, ACM, 2018, pp. 565–573. [52] R. Morafah, M. Karami, R. Guo, A. Raglin,
H. Liu, Causal interpretability for machine learningproblems, methods and evaluation, ACM SIGKDD Explorations Newsletter 22 (2020) 18–33.