In online platforms it is often the case to have multiple brands under the same group which may target diferent customer profiles, or have diferent domains. For example, in the hospitality domain, on-line travel platforms may have multiple brands which have either diferent traveler profiles or are more relevant in a local context. In this context, learning embeddings for hotels that can be leveraged in recommendation tasks in multiple brands requires to have a common embedding that can be induced using alignment approaches. At the same time, one needs to ensure that this common embedding space does not degrade the performance in any of the brands.
Online platforms often have multiple brands, for the same line of business, under the same group which may target diferent customer profiles. As an example, in the hospitality and retail domains the on-line platforms may have diferent brands, that can either have diferent profiles of customers or be more relevant locally. A main task in retail platforms, is to recommend products to customers which requires to learn an embedding space that captures their salient attributes. Hence, enable similarity comparisons that can be leveraged from recommendation systems.
In the recent years approaches that learn product embeddings from the interactions of the customers
with the on-line platform have been proposed1[], [
To do so, one would need to align the embedding spaces of the diferent brands and use this aligned
space to capture the intent of the users while searching on the on-line platform. In a very recent
work, Bianchi et al. [
CEUR
ceur-ws.org
In this work we propose to align embeddings from diferent brands using a simple regularization approach from domain adaptation. We build upon the hotel2vec mode4l][ for learning hotel embeddings and extend it to accommodate alignment of embedding spaces. Our approach, can also be thought of as a transfer learning one as we are able to bootstrap the learning procedure of hotel2vec in a target brand using the hotel2vec embeddings from a source brand.
The main contributions of this paper are the following: i) we propose a simple yet efective domain adaptation approach that adds a regularization term in the loss function of hotel2vec, ii) we present empirical results on the task of next-hotel prediction for two brands and iii) we also implement an alignment method borrowed from the task of alignment of cross-lingual embeddings. We show empirically, that to perform such a transpose to another domain one should be careful of the particularities of it.
In recommendation systems learning embeddings for users and items that capture the semantics is a
critical part. In the domain of electronic commerce, approaches as prod2vec have been proposed in
order to learn representations of products1][, [8], [
Aligning embedding spaces is a topic that has been extensively studied in the NLP domain and specifically in the context of cross-lingual embeddings. A simple and eficient approach to align two embedding spaces of diferent languages is to learn a linear projection [10]. In [11] the authors employ an iterative approach starting from seed mappings of words (source language to target language) and do the linear projection by imposing an orthogonality constraint in the projection matrix. A survey on cross-lingual word embedding can be found in [12]. In this work we study the efectiveness of such alignment approaches in the context of hotel embeddings.
Domain adaptation is also a topic that has attracted interest in the recommendation systems space
[13], [14] where the goal is to transfer knowledge from a source to a target task. In our setting we
employ a domain adaptation approach in order to align diferent embedding spaces and we follow a
straightforward regularization approach1[
The approach most similar to our work is that presented in7][where the authors propose to align product embeddings to enable cross-shop recommendations. Specifically, they propose diferent approaches either based on content (images and text) or using the clicked products in the diferent shops as supervised signals in methods for alignment of cross-lingual embeddings. Our use case has a simpler setting as we dispose a partial overlap among the products of the diferent embeddings spaces we wish to align.
In order to learn hotel representations, we follow the hotel2vec mod4el],[which implements a neural model and is trained with the skip-gram model and negative sampling. The model learns diferent embeddings for click ( ), hotel properties ( ) and geographical information ( ) which are fused to learn an enriched representation. Specifically, the embeddings are calculated as follows: = ( ; ), = ( ; ), = ( ; ) where (; ) = ReLU( ‖‖ 2 ) and , and refer to the hotel2vec model minimizes the following loss function: input features for the click, amenity and geographical embedding. The final hotel2vec embedding is calculated as a projection of the concatenated embedding sℎ: = ReLU([ , , ] ).
LetVhi be the representation of the hotℎel as calculated by the above equation whereℎ ∈ . Then () = −
∑ (ℎ ,ℎ )∈ + log ( ℎ ℎ ) + ∑ log (− ℎ ℎ ) ℎ ∈ (1) where + are the skip-gram pairs of clicked hotels in the session that are generated using a fixed length window. are the negative samples that are sampled from the same market of clicked hotels in the session, as a traveler searches for a specific destination. Finally,() is the sigmoid function.
In this work, we propose to extend the hotel2vec model in order to accommodate embedding spaces alignment in the case of multi-brand representations.
We do so by employing a domain adaptation scheme. Denote s the source domain where we already learned a hotel2vec model by minimizing the loss function in Equatio1n, domain t we learn hotel2vec representations by minimizing the following regularized function: () . Then in the target () = () + || ℎt − ℎs ||22 where ℎs refers to the corresponding embedding of hoteℎl in the source domain. Note that the embeddi ngs of the source domain are fixed and not re-trained. Also, is a parameter that controls how much knowledge we would like to transfer from the source domain. As in our case we want to align the embedding spaces, we would like to constraint the model to be as close as possible in the source domain, hence set equal to 1.0. In the experimental section we experiment with this parameter to understand the impact in the downstream task. Note that we define the strength of regularization globally but in a more fine version it could be defined per hotel. We leave this for future work.
The regularization framework is evaluated under the assumption of partially overlapping property inventories across brand domains, where approximate correspondences between entities are inferred for the purposes of this paper.
We evaluate the proposed approach in the next item prediction task where we want to predict the next hotel clicked in the session based on the previous hotel clicked. We collect click sessions over one year of searches for two brands, namelyBrand A and Brand B. Each dataset has millions of user interaction sessions and hundreds of thousands of unique properties distributed across distinct brand domains. We randomly split the sessions into training, validation, and test with a ration of 8:1:1.
We use a system with 64GB RAM, 8 CPU cores, and a Tesla V100 GPU. We use Python 3 as the programming language and the Tensorflow [ 25] library for the neural network architecture and gradient calculations. For hotel2vec we follow the experimentation methodology that authors employed4]info[r tuning the hyperparameters of the model. The model in both brands is trained wi2th-regularization.
We compare the regularization approach with the linear projection alignment approach presented in [10]. In that case given the embeddings that we export from the trained models in brand A and brand B domains, denoted S and T respectively, we solve the following optimization problem: min || S −
T 2 || 2 The alignment is learned only on the common hotel embeddings for the two sets of vectors in order to avoid injecting too much noise.
We compare the diferent approaches in terms of Hits@k and Mean Reciprocal Rank at k (MRR@k). Both metrics are calculated over the ranking induced by cosine similarity of the embedding vectors. Note that when we have a cross-brand setting this corresponds to a zero-shot learning framework. -29.17 -24.48 -23.16 -25.05 -37.74 -33.91 -21.87 -24.61
We presented in this work a simple yet efective regularization approach for aligning embedding spaces in a multi-brand scenario. For example, in the hospitality/retail domains the on-line platforms have multiple brands that operate in the same domain. The idea is to add a regularizer in the objective function of the model that learns the embeddings in order to force them to be as close as possible to the embeddings of the source brand, hence performing domain adaptation. This kind of approaches has also been explored in the past in Natural Language Processing tasks15[].
We evaluated the proposed approach in the next-hotel prediction task for two brands. We measured performance in terms of hits@k and MRR@k metrics. We also, compared with linear projection alignment borrowed by the cross-lingual approaches for aligning embeddings of diferent languages [10].
The results showed that the proposed approach can align the spaces of the multiple brands achieving good performance in both brands. Indeed the results showed that we can outperfom the single-domain models. We also observed that an approach like the linear projection without taking into account some particularities of the domain leads to worse performance.
For future work we would like to add a more adaptive regularization parameter that can be defined per
hotel rather than being global. In that way we may wish to transfer knowledge when we are certain
that a pair of hotels in the source and target brands should have the same embedding. We would also
like to explore the use of multiple source brands in order to align in the same time the embedding
spaces. Multi-task approaches can be leveraged to align the diferent embedding spaces1[
The author(s) have not employed any Generative AI tools. [8] O. Barkan, N. Koenigstein, Item2vec: neural item embedding for collaborative filtering, in: 2016 IEEE 26th International Workshop on Machine Learning for Signal Processing (MLSP), IEEE, 2016, pp. 1–6. [9] L. Singh, S. Singh, S. Arora, S. Borar, One embedding to do them all, arXiv preprint arXiv:1906.12120 (2019). [10] T. Mikolov, Q. V. Le, I. Sutskever, Exploiting similarities among languages for machine translation,
CoRR abs/1309.4168 (2013). URL: http://arxiv.org/abs/1309.4168.arXiv:1309.4168. [11] M. Artetxe, G. Labaka, E. Agirre, Learning bilingual word embeddings with (almost) no bilingual data, in: Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), Association for Computational Linguistics, Vancouver, Canada, 2017, pp. 451–462. URL: https://www.aclweb.org/anthology/P17-104.2doi:10.18653/v1/P17-1042. [12] S. Ruder, A survey of cross-lingual embedding models, CoRR abs/1706.04902 (2017). URL: http: //arxiv.org/abs/1706.04902. arXiv:1706.04902. [13] F. Yuan, L. Yao, B. Benatallah, Darec: Deep domain adaptation for cross-domain recommendation via transferring rating patterns, in: Proceedings of the Twenty-Eighth International Joint Conference on Artificial Intelligence, IJCAI-19, International Joint Conferences on Artificial Intelligence Organization, 2019, pp. 4227–4233. URL:https://doi.org/10.24963/ijcai.2019/587. doi:10.24963/ijcai.2019/587. [14] H. Kanagawa, H. Kobayashi, N. Shimizu, Y. Tagami, T. Suzuki, Cross-domain recommendation via deep domain adaptation, in: L. Azzopardi, B. Stein, N. Fuhr, P. Mayr, C. Hauf, D. Hiemstra (Eds.), Advances in Information Retrieval, Springer International Publishing, Cham, 2019, pp. 20–29. [15] H. Daumé III, Frustratingly easy domain adaptation, in: Proceedings of the 45th Annual Meeting of the Association of Computational Linguistics, Association for Computational Linguistics, Prague, Czech Republic, 2007, pp. 256–263. URL: https://www.aclweb.org/anthology/P07-103.3 [16] W. Lu, H. L. Chieu, J. Löfgren, A general regularization framework for domain adaptation, in: Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing, Association for Computational Linguistics, Austin, Texas, 2016, pp. 950–954. URhLt: tps://www. aclweb.org/anthology/D16-1095. doi:10.18653/v1/D16-1095. [17] S. Motiian, Q. Jones, S. Iranmanesh, G. Doretto, Few-shot adversarial domain adaptation, in: I. Guyon, U. V. Luxburg, S. Bengio, H. Wallach, R. Fergus, S. Vishwanathan, R. Garnett (Eds.), Advances in Neural Information Processing Systems, volume 30, Curran Associates, Inc., 2017. [18] Y. Ganin, E. Ustinova, H. Ajakan, P. Germain, H. Larochelle, F. Laviolette, M. Marchand, V. Lempitsky, Domain-adversarial training of neural networks, J. Mach. Learn. Res. 17 (2016) 2096–2030. [19] X. Chen, Y. Sun, B. Athiwaratkun, C. Cardie, K. Weinberger, Adversarial deep averaging networks for cross-lingual sentiment classification, Transactions of the Association for Computational Linguistics 6 (2018) 557–570. [20] Z. Xu, W. Pan, Z. Ming, A multi-view graph contrastive learning framework for cross-domain sequential recommendation, in: Proceedings of the 17th ACM Conference on Recommender Systems, RecSys ’23, Association for Computing Machinery, 2023, p. 491–501. [21] G. Lin, C. Gao, Y. Zheng, J. Chang, Y. Niu, Y. Song, K. Gai, Z. Li, D. Jin, Y. Li, M. Wang, Mixed attention network for cross-domain sequential recommendation, in: Proceedings of WSDM conference, WSDM ’24, New York, NY, USA, 2024, p. 405–413. [22] H. Ma, R. Xie, L. Meng, X. Chen, X. Zhang, L. Lin, J. Zhou, Triple sequence learning for cross-domain recommendation, ACM Trans. Inf. Syst. 42 (2024). [23] A. Petruzzelli, C. Musto, L. Laraspata, I. Rinaldi, M. de Gemmis, P. Lops, G. Semeraro, Instructing and prompting large language models for explainable cross-domain recommendations, in: Proceedings of the 18th ACM Conference on Recommender Systems, RecSys ’24, Association for Computing Machinery, 2024, p. 298–308. [24] A. Samra, E. Frolov, A. Vasilev, A. Grigorevskiy, A. Vakhrushev, Cross-domain latent factors sharing via implicit matrix factorization, in: Proceedings of the 18th ACM Conference on Recommender Systems, RecSys ’24, Association for Computing Machinery, New York, NY, USA, 2024, p. 309–317. doi:10.1145/3640457.3688143. [25] M. Abadi, A. Agarwal, P. Barham, et. al., TensorFlow: Large-scale machine learning on heterogeneous systems, 2015. URL:https://www.tensorflow.org, /software available from tensorflow.org. [26] F. Bianchi, V. D. Carlo, P. Nicoli, M. Palmonari, Compass-aligned distributional embeddings for studying semantic diferences across corpora, CoRR abs/2004.06519 (2020). URL: https://arxiv.org/ abs/2004.06519. arXiv:2004.06519.