Traditional tourism recommender systems struggle with cold-start problems and lack the natural interaction capabilities of conversational agents. This paper introduces a modular Hybrid Retrieval-Augmented Generation (RAG)-based Conversational Tourism Recommender System (C-TRS) for European cities. Our architecture combines LLMs with a hybrid retrieval pipeline (dense and sparse vector search) and a structured dialogue state tracker. We use a curated knowledge base from Wikivoyage and Tripadvisor, encompassing over 100 European cities. User utterances are parsed for multi-label intent classification, triggering the retrieval of relevant city-level knowledge chunks to augment LLM prompts for actions like answering queries or providing recommendations. Our evaluation focuses on user intent classification (comparing traditional models with few-shot LLM prompting) and retrieval quality (using the RAGAS framework). Findings demonstrate the eficacy of our hybrid retrieval approach and the power of few-shot learning with LLMs in handling the complexities of conversational recommendation. Overall, our hybrid retrieval strategy balances recall and precision by combining dense (semantic) and sparse (lexical) embeddings, conditioned on conversational intent. This dynamic selection addresses limitations of static or purely lexical/semantic retrieval in tourism RAG systems. It represents a novel integration of intent-driven hybrid retrieval in a conversational tourism framework. Our code and artifacts are available at https://github.com/Akshat125/conversational-trs.
Recommender systems are pivotal in helping users discover new destinations, attractions, and points of
interest (POIs). In tourism, this assistance is crucial for enabling users to explore and plan enriching
travel experiences. However, traditional recommendation approaches often grapple with the "cold-start"
problem, which stems from insuficient user information and limited interaction data. This problem
hinders their ability to provide personalized suggestions, especially for new users or in novel contexts,
as the system lacks the necessary data to make accurate recommendations [
The emergence of LLMs pretrained on extensive textual corpora introduces a paradigm shift by
leveraging implicit world knowledge, advanced natural language understanding, and few-shot in-context
learning. This term refers to the ability of the model to learn from a small amount of data in a specific
context, allowing it to generate recommendations with limited explicit training data [
This paper presents a modular Hybrid Retrieval-Augmented Generation (RAG)-based Conversational Tourism Recommender System (C-TRS) tailored to recommend European city trips. Our architecture integrates LLMs with a hybrid retrieval pipeline — combining dense and sparse vector search and a structured dialogue state tracker. This design allows the system to detect user intent, maintain context across turns, and retrieve and generate grounded, contextually appropriate recommendations or answers.
To support this system, we use a knowledge base combining structured and unstructured travel information from Wikivoyage and Tripadvisor, covering over 100 European cities [4]. Each user utterance, such as "Recommend relaxing destinations in Spain for early spring", is parsed for multi-label intent classification, and depending on the system action (e.g., answer, recommend, and explain), relevant city-level knowledge chunks are retrieved from a vector database and used to augment the LLM prompt.
We focus our evaluation on two key components: (1) user intent classification using both traditional
models (BERT and BART) and few-shot prompting with LLMs, and (2) the retrieval quality of diferent
vector search strategies using the RAGAS evaluation framework [
This paper is structured as follows: Section 2 reviews the relevant literature and identifies the research gaps. Section 3 outlines our proposed system architecture and core components. In Section 4, we evaluate two critical aspects of the C-TRS: user intent classification and the RAG pipeline, with a key focus on the retrieval strategies. Finally, Section 5 concludes the paper by summarizing our contributions and findings, and discusses directions for future work.
This section surveys the existing research in CRS, user intent classification, and RAG for recommender systems in the tourism domain.
Conversational recommender systems (CRS) have emerged as a promising approach to address
challenges such as cold-start problems and information asymmetry in recommendation scenarios [
In the tourism domain, several LLM-based conversational systems have been proposed to assist users
with trip planning and itinerary generation. For instance, zIA [9] is a persona-driven assistant that
ofers localized destination suggestions and supports itinerary planning. TravelAgent [10] combines
recommendation, planning, memory, and tool-use components to deliver personalized travel itineraries
through conversational interaction. Similarly, Vaiage [
While these systems demonstrate the potential of LLMs for conversational tourism assistance, they primarily focus on POI recommendation and detailed itinerary planning, often assuming that a destination or city has already been selected. In contrast, our work targets an earlier and less explored stage in the travel planning pipeline: recommending sustainable cities or destinations. We design a conversational framework that supports natural language interaction while prioritizing sustainability, a critical but underrepresented objective in existing LLM-based tourism CRS.
Several studies have addressed user intent classification in conversational recommender systems (CRS).
Early work by Cai et al. [8] proposed a taxonomy of user intents in movie recommendation. It evaluated
traditional machine learning models such as logistic regression, XGBoost, and SVM, highlighting the
benefit of contextual features. With the rise of transformers, Moradizeyveh [
Retrieval-Augmented Generation (RAG) techniques have recently gained traction in recommender
systems for their ability to combine external knowledge retrieval with powerful language models,
improving both accuracy and contextual relevance while reducing hallucination [
More recently, Banerjee et al. [
To enable context-aware and dynamically adaptive recommendations within our CRS, we propose an architecture centered around a RAG-pipeline. This design enhances the traditional CRS interaction loop by integrating vector-based semantic search and conditional prompt augmentation, triggered based on the conversational context. The RAG pipeline primarily supports two types of recommender actions: Answer and Recommend and Explain, where external knowledge is required to generate informative and grounded responses. A high-level overview of the system architecture is depicted in Figure 1.
Our architecture follows a modular design and is composed of three primary stages: Retrieval, Augmentation, and Generation. Each user query is processed through the augmented pipeline and, depending on intent and dialogue state, may trigger the retrieval mechanism.
The system operates in a multi-turn conversational setting where users engage in natural dialogue to discover travel destinations. A typical session starts with an open-ended query such as "Can you suggest a relaxing destination in Europe for early spring?" or a factual question like "What is the best time to visit Ljubljana?". As the conversation evolves, users may provide preferences (e.g., "I prefer less crowded places with good public transport") or respond to system recommendations (e.g., "That sounds interesting—tell me more about local cuisine there"). These utterances are parsed to infer intents, preferences, and context cues, which are stored in a structured dialogue state and used to guide retrieval and generation.
The underlying knowledge base consists of structured and unstructured city-level travel information
for 160 European cities from Wikivoyage, parsed and processed using the method described in Banerjee
et al. [4]. Each article is hierarchically chunked by section headings (e.g., Get Around, Do, Eat) to
preserve semantic organization similar to Banerjee et al. [
The retrieval stage is responsible for identifying relevant external knowledge chunks to support system responses. This stage is invoked only when deemed necessary by a lightweight routing mechanism (described in the augmentation stage). This selective invocation ensures computational eficiency, avoiding retrieval during trivial turns such as acknowledgments. This stage includes document indexing, query embedding, vector similarity search, optional reranking, and strategy-specific chunk filtering.
We construct a vector database using city-level travel data from Wikivoyage using the knowledge-base
developed in Banerjee et al. [4]. Each document is parsed and chunked hierarchically using markdown
headers to preserve semantic structure. Large chunks are recursively split by subheaders, subsubsections,
and sentences, resulting in an average size of 600 characters, similar to Banerjee et al. [
Two types of embeddings are generated:
• Dense embeddings: using the all-MiniLM-L6-v2 model [
Embeddings are stored in a Milvus Lite instance 2, using FLAT indexing for dense vectors with cosine similarity and SPARSE_INVERTED_INDEX for sparse vectors with inner product similarity.
At runtime, the user query is embedded using the same model(s) as during indexing. Three retrieval modes are supported: • Dense Retrieval: Semantic similarity using cosine distance. • Sparse Retrieval: Lexical matching based on token overlap.
• Hybrid Retrieval: Combines both using Reciprocal Rank Fusion (RRF) [
We employ intent-aware retrieval workflows that utilize both the dialogue state and metadata for more targeted filtering. The underlying intent classification approach is detailed in Section 4.1.
When the user issues a factual query about a known destination: 1. Metadata filtering narrows search to chunks matching the current city and (optionally) relevant subheadings. 2. Vector search retrieves top- matching chunks using dense, sparse, or hybrid methods.
This enables precise, city-specific responses, even when city references are implicit, by using an entity extractor agent to update the dialogue state.
For Recommend and Explain Action This strategy involves a multi-stage retrieval with sustainability-aware reranking to ensure better explanation: 1. Query generation: An LLM transforms dialogue constraints into a pseudo-natural language query.
2. Initial search: Top- chunks are retrieved using dense, sparse, or hybrid methods.
3. SFI reranking: Candidate cities are reranked for sustainability using the Societal Fairness
Indicator (SFI) [
This hierarchical approach balances user constraints and contextual relevance while prioritizing sustainability to generate high-quality recommendations with meaningful explanations, improving the transparency and interpretability of our system.
To improve context quality, we apply a cross-encoder-based reranker 3 after initial search. This model
embeds chunks and queries together to obtain a similarity score for each pair, making it slower yet
more accurate than a dense retriever [
Algorithm 1 abstracts the retrieval workflow used in our RAG pipeline, unifying the retrieval strategies
discussed earlier. SearchWikivoyageDocs retrieves the top-k chunks for a given query, retriever
(dense, sparse, or hybrid), and optional filter. When the is_recommendation flag is set, as in the
Recommend and Explain action, the candidate pool undergoes SFI reranking and a city-specific search
to yield sustainable, context-rich recommendations. Building on N. F. Liu et al. [
Algorithm 1 Multi-stage retrieval function with cross-encoder reranking
The augmentation stage governs intent interpretation, dialogue state tracking, context routing, and
action selection. It is the core logic engine of the system and builds upon previous works in conversational
3https://huggingface.co/cross-encoder/ms-marco-MiniLM-L-6-v2
recommendation [
Each user utterance is passed through a multi-label intent classifier to determine one or more
conversational intents. We adopt a taxonomy comprising five categories [
Multiple intents can co-occur in a single turn, provided they are semantically compatible. For instance, the utterance ‘Amsterdam looks great to me, can you tell me more about the local cuisines there?” would be classified with the intents: Accept Recommendation, Inquire, and Provide Preference—a valid combination. In contrast, a contradictory statement such as Looks great. I don’t like it.” would be flagged for intent conflict (Accept + Reject), and corrective heuristics would be applied. The corrective heuristics, such as constraint validation and priority-based resolution, act as guardrails to handle diverse user queries. For example, when processing feedback on a recommendation, we first verify that a recommendation was issued in the previous turn.
Intent classification is performed using autoregressive LLMs, with few-shot prompting for robustness. Additionally, when dealing with accept/reject intents, we verify that a recommendation was indeed provided in the previous turn to ensure contextual coherence.
Following intent classification, the system updates the dialogue state, which is maintained as a
semistructured JSON-like object. Inspired by Kemper et al. [
current_subheadings_of_interest). • Current user intents and selected system action. (current_destination_of_interest,
This structure supports dynamic adaptation of the CRS over multiple dialogue turns and prevents premature recommendations when essential constraints are missing—unless overridden by a strong Ask Recommendation intent.
A lightweight routing mechanism determines whether external context retrieval is necessary for the current turn. This decision is based on the identified recommender action: only Answer and Recommend and Explain actions trigger the retrieval stage. For simpler actions (e.g., Acknowledge Acceptance), a pre-defined response is returned. This selective routing reduces system latency and computational cost.
When external context is needed, the system may reformulate the query using relevant elements from the dialogue state (e.g., inserting a specific destination name or subtopic). This improves semantic alignment between the user query and vector database content, enhancing retrieval precision.
Based on the classified intents and current dialogue state, the system selects a recommender action from a predefined set: • Recommend and Explain: Generate a recommendation and justify it using user preferences. • Answer: Provide information in response to a user query. • Request Information: Ask for missing hard constraints.
• Acknowledge Acceptance / Rejection: Respond to user feedback.
Unlike scoring-based approaches [
The final LLM prompt is constructed by integrating four key components: (1) the transformed user query, (2) any retrieved context passages (if applicable), (3) the serialized dialogue state, and (4) instructional framing tailored to the selected recommender action. This structured composition ensures that the LLM is guided by both the ongoing dialogue context and relevant external information, resulting in coherent, context-aware, and grounded responses. Listing 1 illustrates the combined system-user prompt template used for the Recommend and Explain action.
You are a sustainable tourism recommendation system. A city has been pre-selected for the user after considering both user preferences and sustainability factors. Using the provided context for the selected city, your task is to: 1. Summarize the key highlights and explain why the city is recommended, focusing on sustainability factors. 2. Explain how the city matches the user’s preferences based on the query and context. 3. Highlight the top 3 attractions and green hotels that align with the user’s preferences. 4. Recommend the most sustainable mode of travel from the user’s starting location, and encourage off-peak/shoulder season travel. **Instructions**: 1. Begin your response with a bold heading for the city and country (**City, Country**). 2. Follow with: "I recommend [city_name]" and explain why this destination is recommended. 3. Use only the provided context to craft your response. If insufficient, reply: "Sorry, I am unable to provide a recommendation for the given preferences." 4. Ensure your response is accurate and professional. **Query:** {{ query }}. Which city do you recommend and why? **Context:** {{ context }} **Response:**
Listing 1: Prompt template for Recommend and Explain action
In the final stage, a response is generated using an LLM: • For Answer and Recommend and Explain actions, a context-augmented prompt is passed to the
LLM for free-form response generation. • For simpler actions (e.g., acknowledgments or information requests), a lightweight template or hard-coded message is returned to the user.
This selective generation mechanism ensures that expensive model inference is performed only when warranted, optimizing both user experience and system eficiency. We use Llama-3.1-8B-Instruct as our LLM for response generation.
In summary, our RAG-driven architecture tightly integrates semantic retrieval, structured dialogue modeling, and LLM-based generation. By decoupling the intent classification, context routing, and generation processes, the system maintains high modularity, interpretability, and extensibility. This design facilitates seamless adaptation to evolving use cases, including the integration of new intents and retrieval strategies.
Due to the multifaceted nature of our CRS, evaluating every component is challenging. For the scope of this paper, we evaluate the performance of our CRS through ofline evaluation of two critical components: user intent classification and the RAG pipeline for European city tourism recommendations. To gain a meaningful understanding of the system’s performance, we use a combination of current state-of-the-art metrics, including standard classification metrics 4 for user intent classification, and metrics from a model-based evaluation framework RAGAS5 to evaluate the RAG pipeline.
Section 4.1 evaluates the system’s ability to classify diverse user intents and sentiments. Section 4.2 evaluates the RAG pipeline, specifically the information retrieval and generation stages for the Answer action. In both sections, we first discuss the experimental setup and metrics, followed by the results.
User intent classification is a multi-label classification problem in which the user intents can be classified
into multiple categories. Classifying the user sentiment is one of the most critical components of the
system because of its direct impact on the recommender action selected and the response returned
by the system. Moreover, the task’s lower complexity provides a good opportunity to benchmark the
results against smaller models. In order to efectively evaluate our user intent classifier, we compare the
performance of four diferent approaches:
1. BERT (Fine-Tuned Sequence Classification) : A fine-tuned BERT model trained on a
synthetically generated dataset [
We use GPT-4o-mini [
Due to the lack of labeled training data for our use case, we manually created user examples covering
over ten intent combinations. To augment this dataset, we leveraged Gemini-1.5-pro-001 [
Our prompt template includes: (1) natural language descriptions of each intent as described in Section 3.4.1, (2) a few-shot set of manually crafted examples, and (3) specific instructions to handle edge cases and encourage diverse phrasing. For instance, to generate utterances for the Accept_Recommendation intent, the model is prompted to act as an expert sustainable travel consultant and produce 20 distinct examples that clearly accept a recommendation. The instructions emphasize clarity, intent exclusivity (avoiding preferences or inquiries), and variation in tone and persona. Each output is structured in CSV format with binary labels across five intent categories: Ask_Recommendation, Provide_Preference, Inquire, Accept_Recommendation, and Reject_Recommendation. For example, the utterance “Looks perfect to me!” is labeled with Accept_Recommendation=1 and all others=0. This structured generation approach ensures label consistency and seamless integration with supervised intent classification models. The same template structure is adapted for other intents, with corresponding examples and tailored instructions.
This results in a final dataset with 330+ labeled user inputs, where the label is a binary vector over
the intent classes. The distribution of label combinations is varied, with between 20 and 54 examples
for over 10 unique intent combinations. The dataset is then split into 80% for training and 10% for
validation and testing each. We fine-tune BERT on a MacBook with Apple M2 and 16GB RAM, freezing
all but the last two layers to reduce overfitting given the limited dataset size. During inference, we
apply a decision threshold of 0.5 for BERT and BART-large-MNLI, while LLMs are not provided an
explicit threshold.
4.1.1. Metrics
Four key metrics are used to evaluate the performance of multi-label classification models: Accuracy,
Precision, Recall, and F1-score [
Our findings in Table 1 reveal that few-shot classification with LLMs outperforms zero-shot classification, with GPT-4o-mini notably achieving the highest score across most metrics. For the F1-score particularly, we observed an improvement of 34% for GPT-4o-mini, reinforcing the efectiveness of having clear intent descriptions and few-shot examples to guide the models. This aligns with our expectations, as few-shot examples cover several edge cases present in our dataset, which a pre-trained model may not capture. Sequence classification using BERT also emerges as an efective approach for a smaller model size, with the highest precision of 91% and an F1-score of 88%, trailing just behind GPT-4o-mini (few-shot). This shows that, with a suficiently large and diverse training dataset, smaller models like BERT can compete with, or potentially outperform, few-shot classification using LLMs. One of the common sources of inaccuracies we observed with LLMs is the misinterpretation of ambiguous user utterances as preference elicitation.
The RAG pipeline plays a central role in our C-TRS, both when responding to user inquiries and when generating recommendations. In this section, we focus on evaluating the former, as recommendations often lack a clear reference context for objective comparison. Since our primary contribution lies in the retrieval stage, we restrict our evaluation primarily to this component rather than the entire pipeline. Furthermore, the open-ended nature of the Recommend and Explain action is better suited to a comprehensive user study, which we leave for future work.
We design an ofline experiment to compare diferent vector search approaches, evaluating both the
retrieval and generation quality. LLM-judge-based frameworks have emerged as a promising tool for
evaluating RAG pipelines, with research by Zheng et al. [
The evaluation dataset consists of 50 synthetically generated Q&A pairs from the Wikivoyage corpus
using the RAGAS TestsetGenerator6. The questions span a diverse range of user personas and
question types, covering articles for the following 5 cities with 10 questions for each: Amsterdam,
Munich, Istanbul, Madrid, and Zurich. We restrict the query category to single-hop specific
queries, which can be answered in one retrieval step from a single document, as the metadata filtering
step may exclude relevant context for cross-city comparison questions [
For our evaluation, we employ GPT-4o-mini as the evaluator LLM and set the corresponding city
names as metadata filters prior to vector search. Llama-3.1-8B-Instruct is used as the generator LLM
throughout the evaluation. Additionally, we fix the top_k value to 5 and apply a cross-encoder as the
reranker.
4.2.1. Metrics
We use the following RAGAS metrics to evaluate the retrieval (context recall and precision) and
generation (faithfulness and answer relevancy) stages of the RAG pipeline [
In contrast, Hybrid Search without reranking yields the highest Faithfulness score (0.81), suggesting that this method is most efective in reducing hallucinations by ensuring that generated responses are well-supported by the retrieved content. When it comes to user-centric evaluation, Hybrid Search + Rerank achieves the best performance in Answer Relevancy (0.90), implying that it delivers the most query-relevant responses, even though it shows a slight drop in recall. Notably, reranking improves 6https://docs.ragas.io/en/stable/getstarted/rag_testset_generation/ Vector Search Type Dense Search Dense Search + Rerank Sparse Search Sparse Search + Rerank Hybrid Search Hybrid Search + Rerank precision across all methods by filtering out less relevant chunks, though this sometimes comes at the cost of reduced recall.
Finally, Dense Search underperforms relative to sparse and hybrid methods across all metrics, highlighting its limitations in this task. Overall, we observe that sparse retrieval excels in key retrieval metrics, while hybrid retrieval performs best for generation quality. Hybrid approaches ofer a balanced trade-of, making them particularly promising as we extend support for more complex, multi-hop, or abstract queries in future work.
In this paper, we introduced a modular Hybrid RAG-based Conversational Tourism Recommender System (C-TRS) for recommending European cities. Our system integrates LLMs with hybrid densesparse retrieval and a structured dialogue state tracker to support real-time, multi-turn interactions. By combining intent classification, dynamic retrieval strategies, and grounded response generation, the system delivers contextually relevant recommendations and factual answers tailored to user preferences.
Through ofline evaluation, we demonstrated that few-shot prompting with LLMs significantly improves multi-label user intent classification over traditional models. Moreover, our hybrid retrieval strategy—conditioned on conversational intent—balances recall and precision, improving answer relevance and reducing hallucinations compared to static or single-strategy retrieval methods.
Despite promising initial results, several aspects of our approach can be further improved. First, our
evaluation revealed that few-shot prompting with LLMs for intent extractions can struggle with
ambiguous or complex user utterances. Future work could explore more robust prompting strategies, such
as chain-of-thought (CoT) prompting [
During the preparation of this work, the authors used ChatGPT (OpenAI) and Grammarly to correct grammar and spelling inconsistencies and to improve the clarity of the text. ChatGPT was also used for code snippet suggestions during system development. We have critically reviewed and revised all GenAI outputs to ensure that accuracy and originality are maintained, and we accept full responsibility for the content presented in this draft.