Modern recommender systems increasingly consider diverse data modalities to enhance model performance. However, the quality of the underlying multimodal data (e.g. data from social media platforms) has received limited attention so far. Focusing on tourism recommendation, we examine real-world challenges encountered when using datasets from Yelp and Instagram. Issues include noise from fake reviews, hallucinated content from LLMs, modality contradictions, and demographic bias. We outline future directions such as quality-aware fusion, structured and controlled LLM-based content generation to reduce hallucinations, and user group diferentiation to improve the robustness and reliability of multimodal recommendation systems.
As a key application domain of recommender systems, tourism recommendation involves tasks such as
providing personalized suggestions for transportation, accommodation, and points-of-interest (POIs) [
Despite the growing reliance on such heterogeneous data sources, the quality of multimodal information remains largely underexplored. This oversight can introduce noise, bias, and contradictions into the recommendation process. In the following, we draw on findings from real-world datasets to illustrate the challenges that arise when working with imperfect, incomplete, or biased multimodal data in tourism recommender systems.
Recent investigations in tourism-related user behavior analysis and next-POI recommendation have
revealed several fundamental issues regarding the quality of multimodal data. Based on two real-world
datasets from [
noisy information that can significantly degrade the efectiveness of recommendation models. For
example, the multimodal Yelp dataset is constructed from user-generated reviews and check-in records
on the Yelp platform. However, some of these reviews originate from fake or automated accounts that
produce fabricated interactions, leading to distorted representations of businesses. Prior work [
In addition, with the increasing use of LLMs to process textual and visual modalities, a new form of
semantic noise has emerged. Hallucinated or irrelevant content generated by LLMs can significantly
degrade the quality of downstream embeddings and harm recommendation performance. One approach
to address this issue is to use structured summarizations instead of open-ended generation. By enforcing
predefined fields in the summaries, the process becomes more controllable and less susceptible to
hallucination. In particular, for visual content, summaries are generated selectively, only for images that
contain clear and interpretable visual cues [
there is a mismatch between the distribution of users in training data and the characteristics of the
target population [
Demographic bias can also arise from population imbalances in content creation. For instance, Instagram content may primarily come from younger users, whereas recommendations targeting regions like Lake Constance are often aimed at an older audience. This mismatch can reduce the relevance and efectiveness of recommendations. Moreover, the majority of media originates from the DACH region (Germany, Switzerland, Austria), reflecting cultural and behavioral patterns specific to this geographic area, which limits the generalizability of the dataset to broader international contexts. One possible approach to mitigate this issue is to further analyze the user profiles to improve audience segmentation and better align the system output with the preferences of the target user group. Semantic Contradictions Across Modalities. Contradictions refer to semantic inconsistencies between content from diferent modalities. In the current datasets, such inconsistencies mainly stem from user-generated content (e.g., misaligned captions, incorrect location tags, irrelevant hashtags) or from business-provided information that is outdated or inaccurate. A common example arises in the multimodal Yelp dataset, where an LLM-generated summary based on the majority of user reviews might describe a restaurant as “clean and quiet”, while user-uploaded photos reveal a cluttered or noisy environment. Similar inconsistencies can also be found in Instagram posts, where images, captions, and location tags do not always align. A photo may be taken in one city but tagged to another, or hashtags may be added that are unrelated to the actual content. Such contradictions reduce the coherence and reliability of the fused multimodal representation, posing challenges for both modality alignment and inference stability in downstream models.
Given such challenges, we propose several directions for future research: (1) Future models should adopt data-aware multimodal learning, where the quality of each modality is explicitly modeled during training. This way, low-quality modalities can be dynamically downweighted in the modality fusion process [9, 10]; (2) Current fusion strategies assume consistent and complete input across modalities, which rarely holds in real-world scenarios. Future research should develop robust fusion mechanisms to tolerate missing, noisy, or redundant inputs [11, 12, 13]. This includes support for preserving modalityspecific uniqueness, reducing cross-modal redundancy, and enhancing synergistic signals to better adapt to imperfect data; (3) Future studies should also focus on modeling user intent divergence across diferent groups, such as tourists versus locals. Identifying and representing such behavioral diferences during preprocessing or representation learning can mitigate distributional mismatches and improve recommendation relevance [14, 15]; (4) In addition to these directions, we observe that videos (e.g., ones posted by social media users or 360° immersive marketing videos) [16, 17] have not been leveraged much in the literature for improved tourism recommendations. We see the incorporation of these additional visual signals as a promising area for future work; (5) Given the increasing role of LLMs in multimodal recommendation, we advocate for more research on structured and controlled LLM-based generation to reduce hallucinations, thereby enhancing the reliability of generated content for downstream use [18].
In summary, integrating data quality considerations into both upstream preprocessing and downstream modeling is essential for building trustworthy, personalized, and data-aligned multimodal recommender systems.
The authors have not employed any Generative AI tools. [9] Q. Zhang, H. Wu, C. Zhang, Q. Hu, H. Fu, J. T. Zhou, X. Peng, Provable dynamic fusion for low-quality multimodal data, in: Proceedings of the 40th International Conference on Machine Learning, ICML’23, JMLR.org, 2023. [10] S. Wei, Y. Luo, Y. Wang, C. Luo, Robust multimodal learning via representation decoupling, in: Computer Vision – ECCV 2024: 18th European Conference, Milan, Italy, September 29–October 4, 2024, Proceedings, Part XLII, Springer-Verlag, Berlin, Heidelberg, 2024, p. 38–54. doi:10.1007/ 978-3-031-72946-1_3. [11] R. Lin, H. Hu, Missmodal: Increasing robustness to missing modality in multimodal sentiment analysis, Transactions of the Association for Computational Linguistics 11 (2023) 1686–1702. doi:10.1162/tacl_a_00628. [12] Y.-H. H. Tsai, M. Ma, M. Yang, R. Salakhutdinov, L.-P. Morency, Multimodal routing: Improving local and global interpretability of multimodal language analysis, in: B. Webber, T. Cohn, Y. He, Y. Liu (Eds.), Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), Association for Computational Linguistics, Online, 2020, pp. 1823–1833. URL: https://aclanthology.org/2020.emnlp-main.143/. doi:10.18653/v1/2020.emnlp-main.143. [13] C. Xu, Y. Zhang, Z. Guan, W. Zhao, Trusted multi-view learning with label noise, in: Proceedings of the Thirty-Third International Joint Conference on Artificial Intelligence, IJCAI ’24, 2024. doi:10.24963/ijcai.2024/582. [14] P. Sanchez, L. W. Dietz, Travelers vs. locals: The efect of cluster analysis in point-of-interest recommendation, in: Proceedings of the 30th ACM Conference on User Modeling, Adaptation and Personalization, UMAP ’22, Association for Computing Machinery, New York, NY, USA, 2022, p. 132–142. doi:10.1145/3503252.3531320. [15] A. Derdouri, T. Osaragi, A machine learning-based approach for classifying tourists and locals using geotagged photos: the case of tokyo, Information Technology & Tourism 23 (2021) 575–609. doi:10.1007/s40558-021-00208-3. [16] M. Casillo, F. Colace, A. Lorusso, D. Santaniello, C. Valentino, Integrating physical and virtual experiences in cultural tourism: An adaptive multimodal recommender system, IEEE Access 13 (2025) 28353–28368. doi:10.1109/ACCESS.2025.3539205. [17] L. Argyriou, D. Economou, V. Bouki, Design methodology for 360° immersive video applications: the case study of a cultural heritage virtual tour, Personal and Ubiquitous Computing 24 (2020) 843–859. doi:10.1007/s00779-020-01373-8. [18] D. King, Z. Shen, N. Subramani, D. S. Weld, I. Beltagy, D. Downey, Don’t say what you don’t know: Improving the consistency of abstractive summarization by constraining beam search, in: A. Bosselut, K. Chandu, K. Dhole, V. Gangal, S. Gehrmann, Y. Jernite, J. Novikova, L. Perez-Beltrachini (Eds.), Proceedings of the Second Workshop on Natural Language Generation, Evaluation, and Metrics (GEM), Association for Computational Linguistics, Abu Dhabi, United Arab Emirates (Hybrid), 2022, pp. 555–571. doi:10.18653/v1/2022.gem-1.51.