Cross-Domain Recommendation is a new field of study in the area of recommender systems. The goal of this type of recommender systems is to use information from other source domains to provide recommendations in target domains. In this work, we provide a generic framework for content-based cross-domain recommendations that can be used with various classifiers. In this framework, we propose an e cient method of feature augmentation to implement adaptation of domains. Instead of defining the notion of domain based on item descriptions, we introduce user-based domains. We define meta-data features as a set of features to characterize the fields that domains come from and introduce indicator features to segment users into di↵ erent domains based on values of the meta-data features. We study an implementation of our framework based on logistic regression and perform experiments on a dataset from LinkedIn to perform job recommendations. Our results show promising performance in certain domains of the data.
1. INTRODUCTION
Recommender systems can help users to address the
information overload problem by providing related items
considering user’s interests. So far, most of the recommender
systems were focused on specific domains, recommending
one type of item (such as books) to all categories of users.
Recent research introduced cross-domain recommender
systems that aim to take advantage of shared information among
various domains [
In this paper, our proposed framework can be utilized by various algorithms defined on any notion of domain from data attributes. Our work also di↵ers from the existing literature in defining domains on the user profile side instead of item side. It can, of course, be used for domains defined on item-set features. We experiment with LinkedIn data for job recommendations. Our experiments lead to promising results for content-based cross-domain recommendations based on user job functions.
In Section 2, we briefly discuss related literature. In Section 3, we introduce our approach to content-based cross-domain recommendation. We present our dataset and experiment setup in Section 4 and we discuss the results in section 5.
Segmented regression or piece-wise regression [
Feature augmentation was introduced by Daum´e [
Our approach improves Daum´e’s model in the possibility of having multiple meta-data features for defining the domains (instead of having one dimension of source and target domains). In addition, each of the meta-data features can have multiple values and define multi-dimensional domains. Moreover, we can have separate sets of common (overlapping) and uncommon features in the main-e↵ect and domain-specific models. Our model is extensible to incorporating cross-products of domain indicator features.
Despite of the importance of job recommender systems, there
have not been many research on this subject. Rafter et
al. [
In cross-domain recommendation, we aim to build a model that can be general and flexible enough, to transfer the information in multiple related domains, and specific enough, to capture particular aspects of each individual domain. This means that we expect a trade o↵ between the bias and the variance in our model. We want all models to be close to each other in particular dimensions (having less variance) and we want them to be biased towards each domain’s specific distribution. For example, if we think of various job functions as di↵erent domains in job recommendation, we expect the user profile to have a good match to the job description in all domains (common feature of the domains). Also, we expect the skills feature to be more important for an artist than a university professor (domain-specific feature). If we consider one main model for all of the data present in various domains, we are going to have no variance in the model, but we will lose the bias we are looking for. On the other hand, if we treat each domain with a separate model, we will achieve the bias each domain is introducing, but we will have too much variance in the achieved models. In other words, we will lose the ability to transfer common information among die↵rent models. Our framework consists of two parts: the main-e↵ect model , and domain-specific models. The main-e↵ect model is to model the shared statistics among all domains. We have one domain-specific model per domain to capture the domain-specific characteristics. A general formulation of model can be seen in Equation 1.
Final-Model = Main-Ee↵ct Model+
X i2 Domains
Domain-Specific Modeli (1)
As said before, we characterize the fields that the domains come from by some features called meta-data features. Each dataset has a set of features, such as user-related features, item-related features, and features that represent similarities between users and items, that we call them “base features”. Meta-data features are a subset of base features, which specify aspects that we want to define the domains based on. Each domain is constructed based on the values of these meta-data features and their combinations. For example, if we want to recommend movies to users, base features are user features, such as age, education, language, etc, item features, such as movie genre, actors, director, etc, or the relationship between users and items, such as the similarity between each movie genre and genres that a user likes. We can choose some of these base features as meta-data features to define the notion of domain based on them. For example, we can choose the genre base feature as the meta-data feature. In this case, the defined domains will be action movies, drama movies, action-drama movies, etc. If we choose two meta-data features, the domains can be a combination of values for those meta-data movies. For example, if we choose genre of movie and age of user as meta-data features, the domains will be like: action-middle-age, drama-young, etc. In the case of user-based domains for job recommendation, we can choose some base features of users, such as job function, or job seniority of users, as meta-data features. For example, if we want to define the domains based on job function, people who have IT job function form one domain and people who have medical job function form another domain.
Each of the domain-specific models in Equation 1 works on one domain’s data. As a result, we should split the dataset for each domain and provide each domain-specific model with the section of the dataset related to that domain. To address this splitting, we expand on the idea of segmented regression model. We propose to augment the feature space based on the domain definitions and copy each datapoint into the related domain’s sub-space. The main space is then used for the main-e↵ect model and each copy of the space is used for the related domain-specific model.
However, this augmentation has a problem: if we have k die↵rent domains (e.g. k di↵erent job functions), we need to partition the data space into 2k separate segments (copies) to capture all of di↵erent settings of the dataset which takes too much space. Each one of these copies is for each subset of the possible combination of meta-data feature values (domains). For example, if we want to partition users based on the job function represented in their resume, and we have three di↵erent possible job functions (e.g. operations, education, and sales), we will have to partition the data into 23 = 8 segments: people for whom the job function is in operations, the ones who have sales function, the ones who have education job function, the ones who have sales and operation functions, and etc. In addition to the space problem, segmenting the data into combinations of domains may lead to very sparse copies of the original dataset. Additionally, looking at all various combinations of the domains and their interactions might not be necessary for our purpose.
To alleviate the problem indicated in Section 3.1.1, we expand on the idea of Segmented Regression Model and introduce indicator features for each domain. These features allow us to augment the feature space in a polynomial order, while being able to keep the main e↵ect model and control the granularity of combinations among die↵rent domains. Suppose that each meta-data feature can take k di↵erent values and segment our data into k domains and suppose that we are choosing only one meta-data feature. For each of the k possible values in each of the domains, we define a binary indicator feature, representing if a data point falls into that specific domain or not. As a result, we will end up with k binary indicator features for the selected meta-data feature. Eventually, we will augment the feature space based on the indicator features in the following way: We keep the original feature space for the common features used in the main effect model. For each of the features falling into the domainspecific models, we augment the feature space by copying it k times for each of the meta-data feature values. Consequently, we have a polynomial expansion of space. Note that we do not consider combinations of meta-data feature values yet.
Now, if we have d meta-data features to choose the domains from, each of which can take k values, and in each of the domain-specific models, f of the base-features exist, we should replicate this f dimensional space for dk +1 times. This number is polynomial in d (number of meta-data features) and k (number of values for each meta-data feature). While if we have not used the indicator features in segmented model, the f -dimensional feature space should have been replicated for dk times. Considering having c common features for the Main E↵ect Model , we can represent the new feature space by Figure 1.
An advantage of this framework is its extensibility to higher order cross-products of values between and within domains. For example, if we want to consider the ee↵ct of interaction between two domains, we can extend the model to consider an indicator feature, representing cross-products of feature values in the domains. E.g. if we want to take into account the combination of every two feature values within each of the domains, we will end up with a space that has O(c + f (dk + 1) + f (dk2 + 1)) dimensions or is expanded for dk + dk2 + 2 times. This gives us the ability to control the dimensionality of feature space while avoiding the sparsity in each of the segments.
Another challenge is choosing the features that should be in the main-e↵ect model and the ones that should remain as domain-specific features. In other words, which features should be responsible for transferring the information among domains (controlling the variance) and which ones should provide the domain-specific bias? In our approach, the model can learn which features to use in the main e↵ect model and which features to use in each of the domains using regularization. Since regularization imposes coecient values to be as close to zero as possible, the less important coecients of the model will have very small values and are removed from the model. While the model can choose between these sets of features, we can also initialize the main e↵ect and domain-specific features by the expert’s domain knowledge. Besides, in some of the cross-domain recommender problems, each domain has its own subset of a general feature set, which might die↵r in the size or type with other domains’ feature sets. We expect our cross-domain solution to consider this problem and be extensible to domains with heterogeneous number and types of features.
Although the approach we presented here can be used in various classification algorithms, we used a straightforward classification algorithm to implement the model. Suppose that fci is the ith common feature among the domains; M is the set of meta-data features; Vj is the set of values (domains) for the jth meta-data feature; Iij is the binary indicator feature for the domain i of meta-data feature j; fijk represents the kth base feature specific to the ith domain of meta-data feature j; and p is the probability of the model’s outcome. Equation 2 shows the resulting segmented regression model with indicator features. As we can see, it has a simple representation that can be implemented easily for domain adaptation.
i i Here w represent the weight (importance) of each feature in the model. In case we want to extend it to having twoway interaction e↵ects of values of each meta-data feature (belonging to two domains), we will have: In Equation 3 Ii,l,j represents the binary indicator feature for the datapoint belonging to both i and l domains (or having both i and l values) of meta-data feature j. To decide which features should be in the common set of features and which should be in each of the domain-specific models, we use L2 regularization.
The dataset we are using in this study is LinkedIn’s job application data. It contains records of users and job features and a binary label indicating if the user has applied for the job or not. Some of base features are calculated similarities between the job and the user. For example, we use TF.IDF to calculate the similarity of job description with user’s skills and store it as a feature in the user-job record. Some other features, from which we have chosen the metadata features, are user-specific. For example, the past and current job functions of a user, past and current industries the user has worked at, or the geographical location of user. Meta-data features should have categorical values, so that we can extract binary indicator features from them. In case we want to use features with continuous values, we partition values into more than one category.
The o✏ine dataset used in the following experiments consists of three million records of more than 150, 000 users. There is a one to ten ratio of positive job applications to negative job applications in the dataset. We use 100 user-job features as base features in the model.
We pick one meta-data feature (user’s current job function) from user-specific features and split domains based on it. This feature is specifically related to what a user does in his/her job, e.g., a user can be an IT (job function) engineer in a bank. We choose this feature based on our experience that people working in various functions (e.g. arts and legal domain) have di↵erent requirements and definitions for a good job recommended to them.
Based on the LinkedIn data, current job functions of a user can have 26 di↵erent values. Each user can have multiple job functions at the same time. The distribution of user job functions is not uniform in the dataset: some functions are more covered in the dataset and some include less number of users. Figure 2 shows the coverage of current job functions in the data. As we can see in the picture, “Sales”, “Operations”, and “Information Technology” are among the most covered job functions in the data and “Community and Social Services”, “Real Estate”, and “Military and Protective Services” are the job functions with least coverage.
As we discussed in section 3, we can have two extremes of modeling users as our baselines: a) when there is only one main model for all of the users, and b) when there is a separate model for each segment of users and there is no shared information among the models. In each of our studies, we compare our model to at least one of these two baselines. To capture the e↵ect of domain granularity on recommendation results, we experiment with three di↵erent settings for domains: segmenting on two domain indicator features versus all other domains (two-vs-all ), segmenting on all indicator features of a domain (all-indicators), and segmenting on clusters of indicator features of a domain (domain-clusters). Figure 3 shows a graphical demonstration of user segmentation in each case for job functions.
For the two-vs-all model, we choose the two most covered values of the selected meta-data feature and define three indicator features based on that: the indicator feature that selects users with the most covered value, the indicator feature that selects users with the second most covered value, and the indicator feature for the rest of users. For example, for user’s job function meta-data feature, we will have the following indicator features: IS for users with “Sales” job function, IO for users with “Operations” job function, and IOther for all other users. The final model is presented in Equation 4. Here, f ci shows the ith common features among domains, fSi , fOi , and fOtheri are features used in the “Sales”, “Operations”, and “Other” domains respectively, wj is the weight for feature j, and p is the probability that the target user applies for the target recommended job. Note that since we have chosen only one meta-data feature, we do not need to present it in the model (e.g. j 2 M in Eq. 2). Also, note that by including IOther as an indicator feature, we are capturing the ee↵ct of the interaction or cross-product of “Sales” and “Operations” domains. For the all-indicators model, we segment based on all values of the selected meta-data feature. If the meta-data feature can take k values, we will end up with k indicator features to segment all users into k die↵rent partitions. For user’s job function meta-data feature we end up with 26 di↵erent indicator features. Our final model is shown in Eq. 5. ent values of job function); and fi,j represents the jth base feature of domain i.
For the last set of experiments (domain-clusters), we cluster the values of meta-data features into groups. Each group represents a cluster of domains. We use one indicator feature for each group. We use spectral clustering to group 26 di↵erent job functions into 8 clusters. The clusters are based on the user transition between job functions in the data. Figure 4 shows a tag-cloud representation of these clusters. Each color indicates one cluster. As we can see in the picture, functions like “Sales” and “Marketing” are clustered together and “Research” and “Education” functions fall in one cluster. We run the segmented model having one indicator per cluster. Suppose that C is the set of cluster indicators for a meta-data feature. The final model is shown in Equation 6.
We experiment in the two-vs-all and domain-clusters settings for current job function of users as meta-data feature. We divide the data into 70% train and 30% test subsets. We measure accuracy of the algorithms on the test set. To find the performance of algorithm in each domains of the data, we partition the test set into domains in the same way that we partitioned the training set and calculate the accuracy in each domains of the dataset. Our model is compared to at least one of the two baseline models: “one-forall” and “independent” models. The “one-for-all” model only contains one model for all of the datapoints, ignoring the domain-specific models. The “independent” model trains a separate model for each of the domains independently. This model ignores the common information among the domains and treats them as independent from each other. To dig deeper into the o✏ine results, we look at the coecient values obtained by the algorithm in each of the models.
As explained in Section 4, in this two-vs-all setting two most covered domains are compared to the rest of the domains. We compare our model with the two base models: “onefor-all” and “independent” models. The most covered job functions in the data are “Sales” (about 12% coverage) and “Operations” (about 8% coverage). The accuracy results for the “job function” meta-data feature are shown in Table 1. The “Sales and Operations” row represent the domain with users in both “Sales” and “Operations” domains and the “Other” row shows the users who are not in any of “Sales” or “Operations” domains. The first two rows show the users who are only in “Sales” or only in “Operations” domains respectively.
As we see in table 1, the segmented model has slightly higher accuracy than the base models. The di↵erence is bigger for the “independent” base model, specially in the two mostcovered domains. To understand how the models work differently, we look at the coecients assigned to base variables of “two-vs-all” segmented model and “one-for-all” base model in Figure 5. In this picture, we can compare the coecient values for these two models. The “two-vs-all” segmented model can have more than one coecient for each variable: the variable might repeat in the main-ee↵ct part of the model or in each of the domain-specific parts of the model. To be able to compare the coecient values, we use the average coecient value of the main-e↵ect and domainspecific parts of the “two-vs-all” segmented model. The red dots represent this average value and the bars around them represent the variance of these coecients in the model. The blue dots are coecient values in the “one-for-all” base model. As we can see in the picture, some of coecients have a di↵erent value in the two models. For example, the similarity of user skills with job description has more importance in the “two-vs-all” segmented model. In addition, we can see that some of base variables existing in the “two-vs-all” segmented model, do not exist in the “one-for-all” base model. The reason is that these variables were removed automatically during the regularization process. For example, the similarity between user location and the location of the job is only present in the “two-vs-all” segmented model. Based on Figure 5, the “two-vs-all” model’s coecients have di↵erent variance in the main-e↵ect and domain-specific models. Some of the coecients vary more than the others. This can indicate that this model is able to capture the di↵erence between di↵erent domains.
To understand if the “two-vs-all” segmented model is capturing the di↵erence between each of the domains, we look at coecients of variables in all four job function domains (Sales, Operations, Sales and Operations, and other) in Figure 6. The coecient values are shown as stacked over each other in the picture. Since there are many base variables in the model and their names are not easily readable, we removed the names in figures of this section. As we can see, coecient values for some of the variables are die↵rent for various domains. With a closer look we can find the differences in coecient values. For example, the similarity of past positions of user to the job description is more important for the Sales domain than the Operations domain. The similarity of user skills to the job’s required skills are more important for users in the Operations domain than Sales domain.
As explained in section 4, user job function meta-data features are grouped into 8 clusters. The accuracy results for the clusters in the “job function” meta-data feature is shown in Table 2. As we can see in this table, the accuracy of the models are very close to each other, for some clusters the baseline models work better than the domain-clusters segmented model and for others it is the reverse. We compare coecient values of the one-for-all baseline and domain-clusters model to have a more detailed insight of the results. Looking at the di↵erences of coecient values for each of the domains in the segmented model, we can understand how di↵erent features are more important for each of the domains. Figure 7 shows the coecient values of domain-clusters model for the job function meta-data feature. As we can see in the picture, some of the coecients are more important in some of the domains and less in others. For example, The similarity of user’s previous searches to the job description is more important to users in cluster 2 (including sales, marketing, and similar job functions).
In this model, we pick all of the possible values for a metadata feature as domain indicators: each indicator feature is representative of one of the values the meta-data feature can take. Since we choose job function as our meta-data feature, we will end up with 26 domains related to job functions, such as IT, sales, engineering, real estates, and marketing. We can see accuracy results of the all-indicators segmented model and the two one-for-all and independent baseline models in Figure 8. As we can see here, the allindicators model is usually slightly better than the two other models. In some of the domains (such as Product Management (number 19) and Real Estate (number 23) domains) the one-for-all model has more accuracy than all-indicators model.
Comparing coecients of one-for-all and all-indicators models in Figure 9, we can see that some of the base features have a very di↵erent coecients in the model. Also, some of base features have a large variance in di↵erent domains of the all-indicators model. There are some base features that are removed from the one-for-all model by regularization, while they still play a role in the all-indicators model.
In this paper, we propose a framework for content-based cross-domain recommender systems. This framework is flexible enough to be implemented with various classifiers. The model in this framework can transfer common information among di↵erent domains while keeping them distinct. We define user-based domains based on users’ meta-data features and implement our framework using logistic regression. The regularization in the model allows us to pick important features of each of the domains automatically, while keeping it flexible to accept expert knowledge in choosing the features. We experiment on job recommendations for LinkedIn users. Our results indicate slight improvement in recommendation accuracy in the o✏ine setting. Furthermore, the experimental results are promising: i) die↵rent features have die↵rent coecient values in each of the domains; and ii) coecients are di↵erent in the cross-domain model compared to the one-for-all base model. As a result, we are hopeful that this model can be a good fit to our problem in the online experiments (A/B testing). We expect several directions for future work: implementing the framework based on various classifier algorithms, expansion of experiments of the model using di↵erent meta-data features, and experimenting on interaction of various possible domains. Automatic selection of meta-data features is another interesting direction of research.