Recommender systems serve as intelligent tools to alleviate information overload and provide personalised services to users. Existing recommender systems that rely on user behavior to generate data often face the problem of data sparsity. Moreover, graph neural network-based recommendation algorithms model user feature representations by treating neighbours equally, but ignore the problem of inconsistency in neighbour preferences in a given context. In this paper, we propose CRDG, a GNN model based on context-aware denoising. Specifically, we first construct user similarity graphs and item relevance graphs from historical interaction data and capture useful information from implicit neighbours with similar preferences through higher-order relationship models to alleviate the data sparsity problem exacerbated in existing denoising-based contextual recommendations. Then, to address the context inconsistency problem, we propose a denoising GNN model to aggregate information from contextually consistent neighbours. In addition to refine the influence of different types of neighbours, we propose a dual attention model to assign different influence weights to different neighbours. Experimental results on several real datasets demonstrate the superiority and effectiveness of the proposed model.
The massive amount of information generated due to the rapid development of the Internet has
brought about the problem of information overload. Recommender systems are one of the main
solutions for addressing this problem. Among the many recommendation algorithms, Collaborative
filtering (CF) has received extensive attention from researchers due to its simplicity and efficiency
[
In recent years, graph neural networks (GNNs) have shown great advantages in recommendation
systems by virtue of their powerful modeling capabilities on non-Euclidean data [
The above analysis shows the drawbacks of ignoring specific contexts (i.e., a single user may have different context-consistent neighbors for different items). To this end, this paper proposes CRDG, a context-aware GNN recommendation model that aggregates useful information from contextually consistent neighbors. First, CRDG constructs user-similar collaboration graphs and item-related collaboration graphs from user-item history interactions for users and items, respectively, and models the impact of implicit neighbors in user-similar graphs and the role of item associations in item-related graphs through a higher-order relationship-aware module. Next, to mitigate the effect of context-inconsistent neighbors, we construct a context-aware denoising module. The model removes context-inconsistent neighbors by sampling and aggregates information only from contextconsistent neighbors. Then, to refine the influence of neighbors, we propose a dual -attention model to assign weights to contextually consistent neighbors. Finally, we connect the initial feature representations modeled based on higher-order perceptual modules with the final features based on consistent neighbor aggregation for recommendation prediction. Extensive experiments on real datasets demonstrate the effectiveness of our model.
Let a bipartite graph G = (U ,V , EUV ) of user-item interactions consist of two different types of node sets (User setU = {u1, u2 ,, um} and Item set I = {i1, i2 ,, i } n ) and edge setEUV , where m and n denote the number of users and items, respectively. We denote the interaction matrix as A ∈ m×n , where au,v otherwise au,v = 0 .
∈{0,1}
. and only au,v = 1 denotes that user u interacts with item v , i.e., (u, v) ∈ EUV , Formally, the recommendation algorithm aims to construct an interaction prediction matrix R ∈ m×n between users and items based on user-item interaction data. i.e., it first learns the latent feature representations of users and items, and then predicts user-item interactions based on the product of the feature representations of users and items.
ruv = eu ⋅ ev (1) where eu ∈ d , ev ∈ d denote the final embedding of the user and the item, respectively, and d is the embedding dimension.
This section first outlines the overall framework of the proposed model. The overall architecture of the model is shown in Fig. 2. The four main component parts into which the model is divided will then be described in detail.
graphs and item relevance graphs based on historical user-item interactions, and then generate preliminary feature representations of users and items by aggregating implicit similarity information within the neighborhood via a higher-order relation-aware graph neural network (RGNN).
Context-aware denoising module: In this module, we first construct embedded representations of context pairs through the query layer, and then filter and denoise the candidate neighbors to obtain context-consistent neighbors.
u1 u2 u3
Prediction module: In this part, we link the preliminary representations of users and items obtained based on RGNN and the feature representations based on context-consistent neighbors aggregation as the final representations, and then output the prediction results based on the inner product of the point feature representations.
In order to capture the influence of implicit neighbors during feature representation, based on the original user-item interaction bipartite graph G = (U ,V , EUV ) , we construct a user similarity graph sim(ui , u j ) =
N VG (ui ) N VG (u j )
N VG (ui ) ⋅ N VG (ui ) rel(vi , v j ) =
N UG (vi ) N UG (v j )
N UG (vi ) N UG (v j ) GUs = (U , EUs ) and an item correlation graph GVr = (V , EVr ) , where EUs and EVr are the sets of edges of the two collaboration graphs, respectively: (2) (3) where N VG (ui ) and N VG (u j ) denote the neighbors of ui and u j on the user item bipartite graph G , i.e., the set of interacting items, respectively. If sim(ui , u j ) >τ , we add an edge between ui and u j where τ is the hyperparameter. Similarly, we compute the correlation between items as follows.
where N UG (vi ) and N UG (v j ) denote the neighbors of vi and v j on the user item bipartite graph G , i.e., the set of users who have interacted with vi and v j , respectively. Similarly if rel(vi , v j ) > ξ , we then add an edge between vi and v j , where ξ is a hyperparameter.
We construct a higher-order relationship-aware graph neural network (RGNN) that can effectively aggregate the relevant information in the neighboring nodes in the two collaborative graphs, and the detailed process is as follows.
Aggregating Similar User Neighbours: for each user ui , given the layer l feature xl representation ui , we will update the user feature representation at layer l +1 as follows: (4) xul+i1 =∑(W1s,l σ xuli + W2s,l ( xuli xul j ))
u j∈NUs (ui ) σ is the LeakyReLU activation function, and where W1s,l and W s,l 2 are the learnable parameters of layer l , denotes the element-wise product,
N Us (ui ) denotes the neighbourhood of ui on U , i.e.,
Gs the similar user neighbourhood of userui .
Aggregating Related Item Nneighbours: Similarly, for each item vi , given the layer l feature xl representation vi , we will update the item feature representation at layer l +1 as follows: (5) xvli+1 =vj∈∑NVr σ (vi ) (W1r,l xuli + W2r,l ( xvli xvlj )) where W r,l 2 are the learnable parameters of layer l , N Vr (vi ) denotes the neighbourhood 1 and W r,l r of vi on GV .
Finally, after L iterative propagation, we obtain the set of user and item feature representations, xu and xv where l = [0,1, 2,, L] . Then connecting the feature representations of users and items at l l each level yields a preliminary feature representation xus = xu0 || xu1 || || xuL of the user's similaritybased feature representation and a preliminary feature representation xvr = xv0 || xv1 || || xvL of the 0 0 item's correlation-based feature representation , where xu and xv are the original input feature representations.
To solve the context-inconsistent problem, we design a context-aware denoising model, which consists of a query layer and a context- consistent denoising-based module.
eu00 ev =WU xus , u ∈U =WV xvr , v ∈V
A recommendation context refers to a user-item pair of a user's preference for a specific item , and in order to capture the representation of a specific context, CRDG builds the query layer to (u, v) select context-consistent neighbors specifically for a specific recommendation context . Specifically, it generates context embeddings by mapping the connection between the initial embeddings of users and items: qu,v =(Wq σ ( xus ⊕ xvr ))
s r where qu,v is the context embedding, xu and xv are the preliminary feature representations of u
W and v , respectively. q is the learnable parameter, and ⊕ denotes the connection operation. Based on the query layer, we can dynamically sample neighbors according to different contexts.
In the higher-order relationship-aware module above, we ignore the interaction information between the user and the item. Therefore, in this phase we will introduce the user-item interaction graph G . After the introduction of G , for user u two different types of neighbor nodes exist, i.e., item neighbors N VG (u ) in G and user neighbors N Us (u) in GUs . Similarly, item v has user
N UG (v) neighbors in G and item N Vr (v) neighbors in GVr . In order to realize the information transfer between heterogeneous nodes, we map the preliminary user and item feature representations to the same embedding space as follows:
where WU and WV are learnable parameters. Given context(u, v) , we obtain the embedding qu,v of (u, v) through the query layer, and then the context consistency score for user u 's neighbor nu ∈ N Us (u ) N VG (u ) is computed as follows
exp (− qu,v − en0u 22 ) p (nu ; qu,v ) = ∑nu′∈NUs (u)NVG (u) exp − qu,v − en0u′ 22
Similarly, the context consistency score for item nv ∈ N Vr (v) N UG (v) as follows: v 's neighbors B is calculated (u, v) (6) (7) (8) p (nv ; qu,v ) =
exp (− qu,v − en0v 22 ) ∑nv′∈NVr (v)NUG (v) exp − qu,v − e0 2 nv′ 2 where nu and nv denote the neighbors of user u and item v , respectively. It is worth noting that they can be item nodes as well as user nodes. Based on the above definitions, we can compute the consistency scores of all neighbors of u and v with the given context (u, v) .
We then select neighbors with the top percent γ of the consistency score, where 0 ≤ γ ≤ 1is a hyperparameter. In this way, we can filter out most of the context-inconsistent neighbors, thus eliminating their negative impact. After completing the denoising process, user u retains two types of contextually consistent neighbors: (9) (10) (11) (12) (13) (14) (15) NUs (u ) = {nu | p ( nu ; qu,v ) ∈ topγ ( N Us (u ))} NVG (u ) = {nu | p ( nu ; qu,v ) ∈ topγ ( N VB (u ))} NVr (v) = {nv | p ( nv ; qu,v ) ∈ topγ ( N Vr (v))}
NUG (v) = {nv | p ( nv ; qu,v ) ∈ topγ ( N UG (v))} where NUs (u ) denotes the set of context (u, v)
Gs consistent user neighbors of user u on U , and NVG (u ) denotes the set of context (u, v) consistent item neighbors of user u on G . topγ (⋅) denotes the selection of the top percent γ of elements in the set. Similarly, for item v we retain two contextually consistent neighbors: where NVr (v) denotes the set of context-consistent item neighbors of item v on GV and NUG (v) r denotes the set of context-consistent user neighbors of item v on G . Next, to aggregate different types of contextually consistent neighbor information, we design a dual -attention based consistent neighbor aggregation module.
Impact of different neighbors of the same type: Given that users u ,
Gs user neighbors of u in U , we aggregate the features of these consistent user neighbors as follows: are consistent NUs (u ) l−1 where eu′ denotes the. feature representation of u 's neighbor u′ in layer l −1 , and α ul ,u′ is the eNlUs (u) = ∑u′∈NUs (u)α ul ,u′eul−′1 eNVG (u) = ∑v′∈NVG (u)α ul ,v′evl′−1
l weight corresponding to u′ . The calculation is as follows.
α ul ,u′ =
exp (σ (W l (eul−1 ⊕ eul−′1 ))) ∑u′∈NUs (u) exp (σ (W l (eul−1 ⊕ eul−′1 ))) where W l is the learnable parameter of layer. Similarly for user u 's contextually consistent item neighbor v′ ∈ NVG (u ) on G we take the same aggregation approach and compute the following: where ev′ denotes the potential embedding of item neighbor v′ in layer l −1 , α ul ,v′ is the weight l−1 parameter corresponding to v′ .
Impact of different types of neighbors: We propose a secondary attention for aggregating information from consistent user neighbors and consistent project neighbors, the aggregated embedding obtained by user u is shown below:
AGGul =NlUs(u)eNUs β (u) + β NlVG (u)eNVG (u)
l l l l where eNUs (u) and eNVG (u) are the aggregation embeddings of consistent user neighbors and consistent item neighbors of useru , respectively. β NlUs (u) and β NlVG (u) are the attention weights of consistent user neighbors and consistent item neighbors, respectively, as follows: α ul ,v′ =
exp (σ (W l (eul−1 ⊕ evl′−1 ))) ∑v′∈NVG (u) exp (σ (W l (eul−1 ⊕ evl′−1 ))) β NlUs (u) = β NlVG (u) =
exp (σ (W l (eul−1 ⊕ eNl−U1s (u) ))) ∑ g∈NUs (u)NVG (u) exp (σ (W l (eul−1 ⊕ egl−1 )))
exp (σ (W l (eul−1 ⊕ eNl−V1G (u) ))) ∑ g∈NUs (u)NVG (u) exp (σ (W l (eul−1 ⊕ egl−1 ))) eul =(Wul σ (el−1 ⊕ AGGul ))
u where W l
u is the l -layer learnable parameter. Similarly given the item v , we can follow the above method to obtain the updated representation as follows: evl =(Wvl σ (el−1 ⊕ AGGl ))
v v where Wvl is the l -layer learnable parameter. AGGvl is calculated similarly to AGGul . 3.4. Prediction module where W l is the learnable parameter of layerl . Finally, the embedding of user u in layer l is updated as follows.
eu ev =(Wu σ (eu0 ⊕ eL ))
u =(Wv σ (ev0 ⊕ evL )) L 1
∑ (ruv − au,v ) = 2 EUV u,v∈EUV 2 After completing the consistent neighbor information aggregation, we can get the hierarchical l l embedding of user and item features, i.e., eu and ev where . We select the embedding values of the first and the last layer among them, so the final representations of users and items are as follows: l = [0,1, 2,, L] where Wu and Wv are the learnable weight matrices. Then we take the following loss function to measure the deviation between the predicted and true values: (17) (18) (19) (20) (21) (22) (23) (24) (25)
where EUV is the edge set of user-item interactions, and each recommendation context. d is the predicted value of user-item interactions.
In this section, the author mainly introduces the research content involved in the experiment, and then describes the datasets, evaluation metrics, experimental settings and experimental results used in this work.
The proposed model CRDG is compared with the following baselines.
FM [
NCF [
GCN [
NGCF [
LightGCN [
DiffNet++ [
GraphDA [13]: A denoised and augmented user-item matrix is generated by capturing the correlation between user-user and item-item, and by top-K sampling.
The performance comparison results are shown in Table 2. From the results, the following observations can be made:
First FM, NCF performs poorly on both datasets. This is due to the fact that traditional collaborative filtering-based methods are difficult to comprehensively model the interaction between users and items compared to graph-based recommendation algorithms. Secondly, among the graphbased learning methods Diffnet++ compared to the traditional graph learning methods NGCF and LightGCN it is designed with a diffusion method that can effectively cross the limitation of one-hop neighboring nodes, and thus can capture richer graph attributes. The reason why GraphDA is able to achieve a better approach than the above methods may be due to the fact that the interaction matrix based on denoising and enhancement can mitigate the effect of noise in the existing interaction matrix, which in turn can model the feature representation of the user and the project more efficiently.
The proposed CRDG model performance baseline algorithm for the following reasons:1. Higherorder relationship-aware module, which can effectively model the implicit similarity and relevance of users and items, can be used as a complement to enhance the information sparsity problem that exists in traditional recommendation.2. The denoising model based on can effectively alleviate the noise problem that exists in the process of aggregation of information from contextually inconsistent neighbors.3. Compared to the GraphDA and other denoising methods, our proposed dual-attention based consistent neighbor aggregation module can refine the influence of different neighbors more effectively.
In order to study the impact of each component, we designed three CRDG variants as follows.
CRDG-RGNN, removes the higher-order relation-aware module from CRDG, i.e., the initial feature representation is directly passed into the subsequent denoising and attention modules.
CRDG-Denoising, removes the context-based denoising step from CRDG, i.e., the dual attention module directly aggregates the representation information of the whole neighbors.
CRDG-Datt, replaces the dual-attention model with the traditional GNN model.
The experimental results are shown in Fig3. We can observe that CRDG consistently achieves the best performance compared to the other variants, suggesting that all components are necessary to obtain the best results. CRDG-RGNN exhibits poor performance reflecting the importance of the higher-order relationship-aware module, for modeling user similarity and item relevance. CRDGDenoising performs sub-optimally and greatly reflects the fact that a mechanism based on the dual attention mechanism can effectively refine the influence of neighbors.
In this subsection, we investigate how the performance of our proposed model varies with some hyperparameters, including its embedding dimension d , the thresholdτ of the user similarity graph and the threshold ξ of the item related graph , and the experimental results are as follows. The experimental results are shown below.
The results in Figure 4 show that the model performance tends to show an increasing and then decreasing trend with increasing d . However, the optimal dimension d varies across datasets, with the best performance achieved when d = 32 on the Yelp dataset and better results when d = 64 on the Amazon dataset. The explanation for this result is that when d is small as d increases the model's modeling ability is stronger, but when d is too large it leads to overfitting problems.
Figure 5 shows the effect of different user similarity thresholds τ on the model performance on τ = 0.3 , CRDG both datasets. Specifically, whenτ = 0.5 , CRDG performs best on Yelp. When performs best on Amazon. As τ gradually increases from 0.1 to 0.9, the model performance shows a trend of increasing and then decreasing. The reason behind is that too small τ will weaken the denoising ability of the model, while too large τ will remove a lot of effective information in the denoising process, which makes the amount of information that the model can obtain decrease, thus leading to a decrease in model performance. Therefore, a suitable τ needs to be set to ensure the model’s performance. achieves the best performance on both datasets when ξ = 0.3 . Similar to the user similarity threshold, the overall performance of the model shows an increasing and then decreasing trend as the threshold increases from 0.1 to 0.9, which is also due to the fact that when ξ is too small, it will lead to a decrease in the denoising ability, and when it is too large, it will lead to a decrease in the amount of information that can be obtained by the model.
In this paper, we propose a new context-based denoising recommendation model. This model constructs user similarity graphs and item relevance graphs to model the implicit similarity and relevance of users and items from their historical interactions. Then, the higher-order relationaware graph neural network is used to learn the user similarity features and item relevance features. Considering the issue of inconsistent neighbors in context-based recommendation, we designed a context-aware denoising method. This method effectively filters out contextually inconsistent neighbors, improving the effectiveness of information aggregation. Finally, in order to refine the impact of different neighbors in the information aggregation process, we propose a dualattention based consistent neighbor aggregation module to achieve adaptive propagation of information from different neighbors. We show through extensive experiments that much of the proposed model due to existing state-of-the-art methods and verify the effectiveness of the proposed scheme.
The author(s) have not employed any Generative AI tools. [13] Fan Z, Xu K, Dong Z, et al. Graph collaborative signals denoising and augmentation for recommendation[C]//Proceedings of the 46th international ACM SIGIR conference on research and development in information retrieval. 2023: 2037-2041.