Discovering similar users from diferent perspectives plays an essential role in marketing activities regarding understanding customers. In particular, user similarity based on attributes (e.g., preferences and behavioral tendencies) efectively captures user needs. However, such attributes are defined diferently depending on the dataset, making comparing users accurately in a cross-domain setting challenging. Previous methods have focused on unifying attributes to calculate user similarity across multiple datasets. However, applicability is limited because they assume the existence of users or attributes that overlap in two datasets. In this paper, we propose Attribute TransPortation (ATP), a novel method based on optimal transport theory for calculating cross-domain user similarity without imposing the assumption of overlapping. In numerical experiments on six real-world datasets, ATP performed quantitatively better than related methods in two tasks, one for finding the same user to evaluate the similarity itself and the other for a cross-domain recommendation task to evaluate the efectiveness of utilizing the similarity.
CEUR ceur-ws.org
1
When analyzing users across internal and external data, common attributes are required to link users. A typical example of such attributes is a demographic attribute (e.g., age, gender, or occupation); however, analyzing users only with demographic attributes can cause stereotyping problems, leading to inappropriate marketing strategies. In this study, we focus on discovering similar users based on psychographic attributes, such as a user’s preferences and behavioral tendencies. While psychographic attributes are more efective than demographic attributes in directly capturing user needs and avoiding stereotyping, internal and external data rarely contain the same attributes. Therefore, discovering similar users across diferent datasets is nEvelop-O CEUR Workshop Proceedings External data is typically obtained by purchasing it from data vendors or forming a partnership between companies to share their user data, but personal information is often anonymized, and it is unknown whether external data contain target users. challenging because such psychographic attributes difer for both types of data (Fig. 1).
There have been various research in the context of cross-domain recommendation [
To resolve the overlapping problem, this paper proposes Attribute TransPortation (ATP), a
method for calculating cross-domain user similarity by connecting attributes. The core idea
of ATP is to connect attributes across diferent domains by adopting optimal transport (OT)
theory [
We investigated the efectiveness of ATP by conducting two experiments on six situations with real-world datasets containing millions of behavioral histories. In the first experiment, we evaluated ATP’s efectiveness in matching accuracy for the same user. If the same users exist across domains, the cross-domain user similarity should be high; thus, we evaluated the matching 2Even if we obtain external data (e.g., movie data crawled from review sites), there is no guarantee that users and attributes in the external data will also appear in the internal data (e.g., music data that we already own). where the highest improvement rate was over 13%.
Our contributions in this paper are summarized as follows.
• We propose Attribute TransPortation (ATP), a method for calculating cross-domain user similarity based on attributes. • ATP is versatile because it requires no overlapping users or attributes. • We conducted extensive experiments on six real-world situations and confirmed that ATP performed quantitatively better than related methods.
Throughout this paper, we denote two domains as and (source and target), and denotes either unless otherwise stated. The sets of users and attributes in domain are denoted as and , respectively. All attributes appearing in this paper are set to be written in a natural language, e.g., = {dance, anime, ...}. A user is represented by a numerical vector ( ) ∈ ℝ| | where a value ( )[ ] ∈ ℝ means the degree of the user’s interest in the attribute ∈ .
We assume all user’ features are given in this form throughout this paper (see Section 3 for how this feature is obtained). Table 1 summarizes the symbols frequently used in this paper.
Our objective is to find users of domain who are similar to a target user of domain based on attributes. Now that two users and of diferent domains are expressed as ( ) ∈ ℝ| | and ( ) ∈ ℝ| |, respectively, the features belong to diferent feature spaces. Our approach consists of transforming ( ) to a new feature → ( ) ∈ ℝ | | in order to treat users of diferent domains in the same feature space. The next subsection explains how to calculate the transformed user feature → ( ).
| |×| | is given, a transformed feature vector is implemented as
→ ( ) = W ( ). Then, each element W[ , ] is interpreted as a contributing score in
which an attribute is reworded as another attribute . If and are “similar” words in
some sense, a user who is interested in attribute is likely to be interested in as well; thus,
W[ , ] is desired to take a higher value. To implement such W, we use a semantic similarity
which is calculated by adopting a word embedding model (e.g., fastText [
The disadvantage of using Wcos is that the relationship between two attributes does not reflect their domains. For example, the semantic similarity of “rap” and “musical” might be high because both terms appear in many music-related sentences. However, if “rap” and “musical” are attributes in the music and movie domains, respectively, they would represent diferent genres. This gap stems from the diference between determining one word from among the millions of words appearing in training data for the language model or from words appearing only in the two domains.
To overcome this disadvantage, we reflect on the relationship between two domains and
propose our method on optimal transport (OT) theory [
Let = {(p , )}=1 and = {(p ,
)}=1 be weighted point sets on ℝ . In OT, a matrix
W ∈ [
(1) the optimal transportation plan is defined as one minimizing the total moving cost For example, when a point p has a weight , a transportation plan transfers the amount W, of the weight of p to p (Fig. 2 and 3). When the cost of moving one point from p to p is C , > 0,
Among the many available OT methods, we used word rotator’s distance (WRD) [
W,
C , . weighted point set ( ) ∶= {(e( ), )} ∈ whose weight is calculated in accordance with the
2-norm, as ∶= ‖e( )‖/∑∈ ‖e( )‖. The optimal transportation plan by WRD is defined where C, ∶= 1 − cos(e( ),e( ))is a cost defined by cosine similarity, > 0 , and Ω(W) ∶= movie ∈ movie. The “Opera” attribute is similar in cosine similarity to various movie attributes (right other music attributes than the “Opera” attribute. As the left figure shows, the transportation plan modifies the similarity relevant to the two domains.
Sinkhorn algorithm [
W, log(W, ) 3. To solve it numerically, we added the regularization term Ω(W) from the OT reflects the relationship of two domains to the semantic similarity as follows. Let the attribute most similar to , i.e., the moving cost from to is the minimum compared to the other attributes in domain . In this case, a transportation plan will typically transfer most of the weights of to , but if there is another attribute ′ that is most similar to , it is also going to transfer most of the weights of to ′ (Figure 3). OT balances this transferring of weights by satisfying the summation assumption of Π( ( ), ( ))and then calculates an attribute similarity for items that are not only semantic but also relevant to the two domains. be
Although the resulting matrix WOT re-describes a user feature ( ) into that in domain as → ( ) = (WOT)
( ), there remains a problem regarding the treatment of unrelated attributes. For example, we assume that a user is intensely interested in the “instrumental” attribute in the music domain, and there is no similar movie attribute to the music attribute. Although the music attribute is unrelated to any movie attributes, the construction (WOT) ( ) cannot avoid fully transferring the degree of this user’s preference for the unrelated music attribute “instrumental” to the transformed feature.
To deal with this issue, we reduce the influence of the unrelated attribute by formulating
3As the original paper [
, = ‖e(
distribution
) − e( )‖2 to the distance matrix from the cosine similarity and the mass on each word from a uniform
≡ 1/| | to a weighted distribution of which the mass is proportional to the norm of the embedding
vector.
4In experiments, we used Python Optimal Transport [
∈ → ( ) ∶= (WOT) (s ⊙ ( )) ∈ ℝ| |, where ⊙ is the Hadamard (element-wise) product for vectors.
As a mathematical remark, users who are similar in terms of user features in the original domain are also similar in terms of the transformed user features; in other words, the mapping from ( ) to → ( ) is Lipschitz continuous. To be precise, for any 1 ≤ < ∞ , there is a constant > 0 such that ‖ → ( 1) − → ( 2)‖ ≤ ‖ ( 1) − ( 2)‖ for any users 1, 2 ∈ . This is proven as a consequence of the facts that (1) the operation of the matrix product is a linear map, i.e., for any matrix M, there exists a constant > 0 such that ‖Mx − My‖2 ≤ ‖ x − y‖2 for any vectors x, y, (2) the Hadamard product can be seen as a diagonal matrix with c ⊙ x = diag(c)x, and (3) the 2-norm and the -norm (1 ≤ < ∞) are equivalent norms, i.e., there exist constants 1, 2 > 0 such that 1‖x‖ ≤ ‖‖ 2 ≤ 2‖x‖ for any vectors x.
Finally, we define the cross-domain user similarity in the same feature space ℝ → ( ) and ( ). Note that ( )[ ] can have a higher value for some attribute , while none of the transformed user features can have higher values for the attribute because s [ We also adjust the user feature ( ) on the basis of attribute-domain similarity as ] is low. | | by comparing sim ( , ) ∶= cos ( → ( ),s ⊙ ( )) .
In addition, since and can also be compared in through → ( ) and ( ), we define the cross-domain user similarity of ATP, as simATP( , ) ∶= 2 1 (sim ( , ) + sim ( , )) . (3) (4) (5) (6) (7)
We applied ATP to real-world datasets and compared its performance to other methods in defining cross-domain user similarities for linkage of the same users and cross-domain recommendation tasks.
We used the Amazon Review Dataset [
The number of intersections of pairwise datasets, where each value inside the parentheses is the Jaccard Case 1 Case 2 Case 3 Case 4 Case 5 Case 6
Music
Book Clothes Clothes
Book Kindle
Movie Music Movie Music Movie Book | ∩ |
| ∩ | 516 495 106 19 1318 6313 (0.069) (0.008) (0.011) (0.003) (0.02) (0.1) 38 50 48 34 61 (0.068) (0.078) (0.038) (0.031) (0.077) users, we limited the user set to those who had evaluated more than 50 items, as it would be challenging to capture meaningful user features for users who gave evaluations on only a few items. Regarding attributes, we separated all original attributes into words by morphological analysis, cleaned them to their lemmata (canonical forms) because some shared common components (e.g., “rock” in “classic rock”, “folk rock”, and “hard rock”)6, and set the separated and cleaned words as the attribute set . We also removed attributes that were not given to 5 or more items from .
Let and ⊂ × be sets of items and transaction logs that users reviewed items, respectively. Tables 2 and 3 summarize the statistics for the datasets and intersections of the pairwise datasets, respectively. We set the cross-domain situation by choosing pairwise datasets so that common users appeared in both datasets, denoted as linked ∶= ∩ ≠ ∅. For Table 3, we omitted the item column because all pairs of item sets had no intersections, i.e., | indicated by the lower number of intersections and the Jaccard similarity7, users and attributes ∩ | = 0. As seldom overlapped across domains, even though each dataset had a large number of users and attributes. This observation supports the necessity of this study, which aims to obtain new insights from two datasets with slight overlapping. 6We used spaCy (https://spacy.io/) to obtain the canonical forms of words. 7Jaccard similarity of two sets and is defined as | ∩ |/| ∪ | .
We created a user feature ( ) from user transactions. Let be a set of items in domain , [ ] ⊂ be a set of items that user evaluated, and ( , ) be a rating score for by . Let A be an item-attribute matrix of , where each element is given by A [ , ] = 1 if an item attribute ∈ is attached to an item ∈ ; otherwise, 0. A simple approach to obtaining a user feature is just to take a weighted sum of the attribute vectors in which the user was interested, that is, (8) (9) simple( ) ∶=
∑ ∈ [ ]
( , )A [ ] ∈ ℝ| |.
To properly adjust the degree of an attribute following its domain, we change the
itemattribute matrix A by its transformed matrix using BM25 [
∑ where is a normalization term to make ‖ ( )‖1 = 1. Then, each element in a user feature can be interpreted as a percentage of the degree of the user’s interest.
We compared our method with the following two baselines Agg and SCT for calculating crossdomain user similarity. As an ablation study, we also compared Agg and SCT with ATP-cos 8For example, assume that a song has both “rap” and “young” attributes. If we describe this song with one attribute, “rap” would be appropriate because it is more representative than “young” in the music domain. BM25 is known in the information retrieval field as an appropriate method for adjusting the importance of each attribute in this way. registered to its corpus9.
WOT in Eq. (4) with Wcos. and ATP-ads, which are partially changed components of ATP.
ATP (Proposed approach) : The embedding model e to compute WRD was set as fastText
[
Agg [
∑ A and A , and AoBuMte2r5 is the transformed matrix of Aouter by BM25.
where is a normalization term so as to make ‖ Agg( )‖1 = 1, Aouter is the outer join of
SCT- [
, | ) where ( | ) is the number of attributes on which the user evaluated that belong to a cluster . In the experiments, we investigated = 50 and 100.
On the basis of the above construction, Agg( )and SCT( )of either domain belong to the same spaces ℝ| ∪ | and ℝ , respectively. Here, we calculated the cross-domain user similarity by cosine similarity, as sim∗( , ) ∶= cos( ∗( ), ∗( )), where ∗ ∈ {Agg, SCT}.
Since all overlapping users were active on both domains, any ∈ linked = ∩ should show a higher cross-domain user similarity sim(, ) than sim(, ) of the other users ∈ ⧵ {} .
We, therefore, evaluated the efectiveness of ATP by matching accuracy for the same user. 9We used the fastText model released in https://dl.fbaipublicfiles.com/fasttext/vectors-crawl/cc.en.300.bin.gz without any fine-tuning to ensure a fair comparison. (10) (11) 3.4.1. Evaluation We utilized the mean reciprocal rank (MRR) and top- accuracy to evaluate the matching accuracy. When we calculate {sim(, ) ∣ ∈ } for ∈ linked, if the similarity of is located at the top ℓ-th position, we denote the number ℓ by rank() . Then, the MRR is defined by the mean of the inverse of the rank, i.e.,
MRR ∶=
1 ∑ 1 | linked| ∈ linked rank()
, |{ ∈ linked ∣ rank() ≤ }| | linked| . and top- accuracy is defined by the probability that the rank is less than or equal to , i.e., The higher MRR and top@ are, the better the cross-domain user similarity is.
Note that as the number of users to look for | | increases, MRR and top@ deteriorate. For example, it is more dificult to obtain a result for top@ 10 = 0.1 when | | = 105 than when | | = 103. To reduce the efect from | |, we determined whether the rank was in the top-10% of and wrote top@⌈0.1| |⌉ as top@10%, where ⌈⋅⌉ is the ceiling function. 3.4.2. Results The results shown in Table 4 indicate that ATP exhibited the highest MRR or top@10% in Cases 1–4 and the second-highest scores in Cases 5 and 6. One of the reasons ATP was defeated by Agg in these latter two is that there were many common attributes across domains (please refer to Table 3) and Agg utilized them to match users. We observed that the ratios of simAgg(, ) = 0 for ∈ linked, which indicates that did not show any interest in common attributes across domains, were less than 0.5% in Cases 5 and 6, and over 3%; otherwise. In other words, in Cases 1–4, where many overlapping users did not show interest in common attributes, ATP successfully found the same users with higher accuracy. SCT- , which also essentially utilizes common attributes in clusters, showed the highest top@10% in Case 2 by = 50 ; however, the results depended on the selections of , and there remains the dificulty in determining the proper . As an ablation study, we confirmed the necessity of OT compared to the results by ATP-cos. While ATP-ads outperformed ATP for Case 4 and 5, the diferences were slight, and (12) (13) ATP-ads was defeated in the other situations; hence, we also confirmed the necessity of using attribute-domain similarity.
In this section, we evaluated the efect of using ATP in the context of a recommendation task
that predicts which rating score a user will give to an item based on his past rating history.
However, if a user did not evaluate at all (i.e., a cold-start user), it would be dificult to make a
better recommendation because his preferences are unknown. If a cold-start user in domain
behaved in another domain , methods of cross-domain recommendation [
MAE ∶=
1 |Rtest|0 (, )∶Rtest[, ]≠0 ∑ |Rtest[, ] − (̂ , )|, where |R|0 counts the number of nonzero elements of a matrix R.
Note that methods of single-domain recommendation (e.g., matrix factorization (MF)) cannot does not contain any information of cold-start users t est (Figure 6). be applied to this task because such methods learn user representations only from Rtrain, which
The methods in [
‖R − P (Q ) ‖2 + +
1 |R |0 1 |Rtrain|0 | train|2 , ′∈ train ‖Rtrain − Ptrain(Q ) ‖2 ∑
sim(, ′)‖P[] − P[ ′]‖2,
where Q ∈ ℝ|
and P = [P , Ptrain] 10. In contrast to the original loss function used in [
| |× , Ptrain ∈ ℝ| train|× , is a positive integer, > 0 , train ∶= ∪ t rain, 10In experiments, we chose = 50 and = 1 by 5-fold cross validation. (14) (15) (16) (17) during the training stage, i.e., ∩ train = ∅ and ∩ = ∅. For this challenging situation, we address predicting ratings of cold-start users. an item ∈ by a cold-start user ∈ t est. we observed that |R |0 ≈ 7 ⋅ 106 and |Rtrain|0 ≈ 3 ⋅ 105 in Case 2. For ATP, we set sim(, ′) = cos( (), ( ′))when and ′ belong to the same domain ; simATP(, ′), otherwise. After obtaining the trained embedding vectors, we predict a rating score as (̂ , ) = ⟨P [], Q [ ]⟩ for ∈ as (̂ , improvement rate from AVE as
As stated above, single-domain recommendation methods do not work for cold-start users, but we can apply a method that utilizes average ratings, called AVE, to this situation. AVE calculates the average of the rating scores attached to the item in the training dataset as = mean{Rtrain[ , ] ∣ ∈ t rain} and predicts a rating score for an item by any user ) = . Since AVE was introduced as the minimum baseline, we calculate the Improvement rate (%) ∶= 100 × (18) MAEAVE − MAE∗
MAEAVE where ∗ ∈ {ATP, Agg, SCT} and MAEAVE is the MAE of AVE. 3.5.2. Results The results shown in Table 5 indicate that ATP exhibited the highest MAE improvement rates for all cases except Case 4. Even for Case 4, ATP achieved the second-highest improvement rate and was only slightly worse than SCT-50. Unlike the results for Cases 5 and 6 in Section 3.4, Agg did not outperform ATP. One reason is that Agg could not give the degree of dissimilarity. Even if two users showed interest in similar (but not common) attributes, Agg uniformly treated them as entirely dissimilar users because their similarity score was zero. In contrast, ATP gives the degree of dissimilarity with a particular value, which is why it worked better than Agg and SCT in the recommendation task with cross-domain user similarity.
Methods for calculating cross-domain user similarity and discovering the same user are known
as methods of user identity linkage (UIL) [
From the viewpoint of user similarity, recommendation systems can be regarded as trying
to extract user features that summarize their degrees of interest in items from their behaviors
(e.g., purchase and evaluation histories for items). When users or items are registered to
multiple datasets, standard (single-domain) recommendation methods provide each dataset’s
corresponding user features in diferent dimensions. To provide user features in a common
dimension, methods in cross-domain recommendation (CDR) [
Tag-based CDR methods [
In this paper, we proposed Attribute TransPortation (ATP), a novel method for calculating crossdomain user similarity without requiring assumptions for overlapping users or attributes. The core idea of ATP is to use optimal transport theory to provide attribute similarity for items that are not only semantic but also relevant to the two domains on which we focus. ATP transforms a target user into a feature vector of the other domain and then enables cross-domain user similarity in the same feature space to be calculated. The results of experiments using linkage of the same users and cross-domain recommendation tasks demonstrated the efectiveness of ATP compared to related methods. [24] S. Gao, H. Luo, D. Chen, S. Li, P. Gallinari, J. Guo, Cross-domain recommendation via cluster-level latent factor model, in: ECML/PKDD (2), volume 8189 of Lecture Notes in Computer Science, Springer, 2013, pp. 161–176.