Large Language Models (LLMs) often produce hallucinated or factually incorrect responses in domain-specific question answering (QA) tasks. To address this limitation, this work explores the integration of Knowledge Graphs (KGs) with Retrieval-Augmented Generation (RAG) as a strategy to improve factual accuracy and multihop evidence selection. Specifically, the benefits of using structured information from a KG to enhance the QA performance of the DeepSeek model are investigated. A novel framework, HyP-KGRAG, is introduced in which KG triples are retrieved via hypothetical paths and refined through an LLM-based denoising module. Experimental results on the material science MSE-KG dataset show that HyP-KGRAG significantly improves the QA performance of DeepSeek and other baseline models, achieving a ROUGE-1 F1 score of 0.532 and an SBERT similarity of 0.629.
Large language models (LLMs) have attracted significant attention due to their remarkable capabilities
in various natural language processing (NLP) tasks, including question answering (QA) [
This work focuses on addressing the following research question: How can KGs be efectively leveraged within a RAG framework to reduce hallucinations and enhance domain-specific QA performance in LLMs such as DeepSeek?
The key contributions of this work are as follows: • HyP-KGRAG framework: a novel RAG-based framework that retrieves KG triples via hypothetical paths and leverages LLMs for denoising and answer generation. • Experiments on a materials science KG: demonstrate that HyP-KGRAG significantly enhances domain-specific QA performance in terms of accuracy and semantic coherence. (G. A. Gesese) CEUR
ceur-ws.org • Ablation studies: validate the benefits of triple-structured retrieval and the denoising module to achieve a better balance between precision and recall.
DeepSeek-V3 DeepSeek-V3 [
RAG RAG [
BM25 BM25 (Best Matching 25) [
BGE-M3 BGE-M3 [
The integration of KGs into RAG has emerged as a promising direction to enhance the problem of
hallucination with LLMs. Several recent works have proposed methods for incorporating structured
knowledge into the RAG pipeline, moving beyond traditional text-based retrieval. KAPING [
For more complex query understanding and summarization tasks, GraphRAG [
KG2RAG [
In contrast to existing methods, the proposed HyP-KGRAG framework introduces a novel hypothetical
path-based retrieval approach that leverages structured knowledge from KGs and incorporates
LLMbased denoising to enhance QA performance. While it is inspired by HyDE [
This section introduces HyP-KGRAG (Hypothetical Path-Based KG Retrieval Augmented Generation with DeepSeek), a novel framework designed to enhance the retrieval quality and multi-hop evidence selection of LLMs, with a focus on DeepSeek without loss of generality, through KG integration. As illustrated in Figure 1, HyP-KGRAG contains three main components: ofline processing, hypothetical path construction, and LLM-as-Judge denoising and generation.
In the ofline processing phase, HyP-KGRAG performs two key preparatory tasks: KG verbalization and index construction. These steps transform structured knowledge into a format that enables fast and semantically meaningful retrieval during inference.
KG Verbalization. This step involves converting structured triples (i.e., subject, predicate, object)
into natural language documents. Unlike prior approaches that employ LLMs to generate free-form
verbalization of triples, HyP-KGRAG uses a linear verbalization strategy, as described in [
Index Construction. Following verbalization, the triples are encoded and indexed for retrieval. For
semantic retrieval, the bge-M3 embedding model is employed to encode verbalized triples into dense
vector representations, supporting multi-granularity encoding. The resulting embeddings are indexed
using FAISS [
As depicted in Figure 1, HyP-KGRAG leverages an LLM (specifically DeepSeek-V3) to identify potential paths underlying the query, where these hypothetical paths are structured as triplets. These triples undergo the same verbalization process as described in Section 3.1. Subsequently, BGE-M3 is employed to encode the text into vector representations. FAISS is then utilized to retrieve semantically closest factual triplets from an ofline-constructed vector database, with similarity measured via the inner product of vectors. To construct a complete retrieval paths for the query, HyP-KGRAG utilizes class information to fill in missing components, whether they are target answers or intermediate nodes, ensuring coherence and completeness of the paths. A class represents an abstract collection of entities (e.g., Person, Researcher, Organization), making it well-suited as a bridge in incomplete paths.
Efective multi-hop evidence retrieval over KGs requires generating coherent paths that span one or more hops. Accordingly, user questions are categorized into two types: Single-hop Questions. These involve a direct relation between two entities, requiring only one retrieval step to obtain the supporting evidence for the answer. These questions are relatively straightforward, as they rely on a single factual assertion represented as a triple in the KG. For example, the question ”Who published the dataset ’Nano2023’?” requires identifying an entity (likely a person or organization) directly connected to the dataset via the published relation. Although single-hop, HyP-KGRAG still constructs hypothetical paths in both directions to support more robust retrieval. Class-constrained hypothetical paths include:
published Person −−−−−−−−→ Nano2023 or
published by Nano2023 −−−−−−−−−−→ Person Multi-hop Questions. These involve more complex chains of entities and relations, requiring multihop evidence retrieval to gather the information needed to answer the question. For instance, the question “Which researcher from MIT published the dataset ’Nano2023’ cited in ’Advanced Materials Lecture 5’?” combines afiliation, authorship, and citation relationships across entities. Here, answering requires traversing a sequence of links. Two possible hypothetical paths are:
afiliation published cites
MIT ←−−−−−−− Researcher −−−−−−−−→ Nano2023 ←−−−− Advanced Materials Lecture 5 alternative,
cites published by employs
Advanced Materials Lecture 5 −−−−→ Nano2023 −−−−−−−−−−→ MIT −−−−−−−→ Researcher HyP-KGRAG dynamically adjusts the path length based on question complexity. To support the generation of such paths, a structured prompt is defined to guide the LLM. This prompt instructs the model to generate step-by-step logical triples for knowledge-grounded retrieval. The full prompt is available in our GitHub repository1.
Once a query’s hypothetical paths are generated, each triple is used to perform semantic similarity search against the ofline-constructed vector database to retrieve relevant factual triples. After deduplication, the retrieved factual triples represent the preliminary result set and are passed on to the denoising phase (see Section 3.3).
Adding all retrieved triples directly into the LLM’s input can be problematic due to input size limitations and the potential inclusion of irrelevant or noisy triples retrieved via vector similarity. To address this, HyP-KGRAG incorporates an LLM-as-Judge denoising mechanism inspired by the multi-dimensional evaluation strategies used in GraphRAG and LightRAG. In this process, the LLM scores each candidate triple based on the following three criteria, filtering out low-quality or of-topic information before answer generation.
• Directness - refers to how clearly a triple indicates the answer to a query. Triples with relevant entities and relations that directly point to the answer should receive higher scores.
Scoring: 2 = direct mention of relevant entities/relations; 1 = indirect reference. • Entity clarity - reflects the precision and unambiguity of the entities within a triple, which strongly influences its overall quality.
Scoring: 2 = explicit entities; 1 = ambiguous references. • Context relevance - captures the LLM’s ability to assess whether the answer thoroughly addresses all aspects and details of the question.
Scoring: 2 = core relevance; 1 = peripheral relevance.
Only triples with a total score of 3 or higher are retained. Triples with ambiguous entities or lacking contextual grounding are excluded, ensuring that only the most relevant and precise information supports the generation process. The full prompt guiding this filtering process is available in our GitHub repository: https://github.com/Zhaotai924/HyP-KGRAG.
After denoising, the filtered and retrieved triples, serving as core contextual information, are combined with the original query and input into the LLM (DeepSeek-V3) to generate a response.
In this section, the experiments conducted to evaluate the performance of the proposed approach are presented. The source code and the datasets are made publicly available at https://github.com/ Zhaotai924/HyP-KGRAG.
The MSE-KG2 (Material Science and Engineering KG), which represents heterogeneous research data within the NFDI-MatWerk3 community and beyond, is the dataset used for the experiments.The KG covers three main domains: i) research community structure, including researchers, projects, universities, and institutions; ii) scientific infrastructure, such as software, workflows, instruments, facilities, and educational materials; and iii) research data, spanning repositories, publications, datasets, and reference data. The graph consists of 5,153 triples, 1,533 entities, and 109 distinct relations. To evaluate the proposed approach, 37 competency questions were used. 13 of these questions are complex multi-hop questions. 1https://github.com/Zhaotai924/HyP-KGRAG 2https://demo.fiz-karlsruhe.de/matwerk/ 3https://nfdi-matwerk.de/
HyP-KGRAG is compared against four baselines: • Deepseek V3: This model operates without utilizing any information from the KG, relying exclusively on the knowledge contained within the Deepseek V3. • KGMistral: This model utilizes SPARQL to extract information from a KG to improve the QA performance of the Mistral model. • BM25 Retrieval + Pre-Retrieval Query Processing: This model first verbalizes the triples and performs sentence-level and paragraph-level sparse retrieval-based matching using BM25. It can also use LLM to process the original query to facilitate keyword-based matching in BM25. • BGE-M3 Retrieval + Pre-Retrieval Query Processing: This model first verbalizes the triples and performs sentence-level and paragraph-level dense retrieval-based matching using BGE-M3.
It can also use LLM to process the original query to facilitate semantic matching in BGE-M3.
The evaluation uses text-matching metrics at three distinct levels: ROUGE-1, ROUGE-2, and
ROUGEL, to provide a systematic assessment for lexical coverage (uni- gram), phrase structure (bigram),
and semantic coherence (LCS). Every ROUGE metric assesses several attributes, including ground
truth coverage, generation accuracy, and global balance by means of Recall, Precision, and F1 Score,
respectively. In addition, BLEU and SBERT similarity measures are used. BLEU provides stricter n-gram
precision evaluation, which is especially valuable for measuring phrase-level fidelity. The SBERT
similarity measure is based on the cosine similarity computed between sentence embeddings using the
lightweight model ”all-MiniLM-L6-v2”[
For the hyperparameter settings of retrieving document chunks or triples, without loss of generality, all retrievals are configured to return the top-k most relevant document chunks or triples for each query (whether rewritten or not), where k ranges from 10 to 100, sorted by similarity score. These retrieved documents are then fed into the LLM to generate answers using a standardized prompt structure. The DeepSeek-V3 model is the LLM chosen for the experiments conducted in this work.
As shown in Table 1, HyP-KGRAG consistently outperforms the baseline models across all precisionfocused metrics. It achieves the highest F1 scores for ROUGE-1 (0.532), ROUGE-2 (0.408), and ROUGE-L (0.491), along with leading precision score of 0.569, 0.453, and 0.528 respectively. HyP-KGRAG also scores the top BLEU score (0.231), reflecting its strong capability in producing lexically accurate and concise answers. These results are further visualized in Figure 2, where HyP-KGRAG demonstrates superior performance in generating highly accurate and lexically precise answers, significantly outperforming other models in minimizing errors and aligning with reference outputs. While BGE-M3 demonstrates competitive recall, HyP-KGRAG’s precision-centric design ensures superior reliability in scenarios demanding factual correctness. For instance, its ROUGE-1 precision (0.569) surpasses BGE-M3’s (0.473) by 20%, reflecting stricter adherence to relevant content. Similarly, its BLEU score (0.231) outperforms BGE-M3 (0.200), further validating its lexical alignment strength.
Compared to the earlier KGMistral7B baseline, which integrates SPARQL-based KG retrieval with Mistral-7B generation, HyP-KGRAG achieves substantial improvements, particularly in precision and semantic coherence. Meanwhile, the standalone DeepSeek-V3 model struggles severely on domainspecific datasets (e.g., BLEU=0.002, SBERT=0.283), underscoring its limitations in the absence of prior knowledge or retrieval augmentation.
This section presents the ablation studies in detail. First, the proposed approach is evaluated by comparing the linear-verbalization technique with LLM-based verbalization. Then, the impact of the denoising module on the overall performance of HyP-KGRAG is assessed.
HyP-KGRAG encodes both the KG and the hypothetical paths using structured triples, represented through a linear verbalization technique as described in Section 3.1. To evaluate the impact of this design choice, an ablation study is conduct in which the linear-verbalization is replaced with sentence-level natural language expressions generated by an LLM. Note that the sentence-level verbalization is chosen here because it demonstrated a better performance as compared to paragraph-level verbalization for the baseline models.
As shown in Table 2, the results demonstrate that triple-based hypothetical path matching, i.e., applying linear-verbalization, achieves significantly higher retrieval accuracy than its natural language based verbalization. This diference highlights the advantages of structured representations in RAG systems: triples ofer compact, unambiguous, and noise-resistant semantic cues that are better aligned with the query intent.
In contrast, natural language representations, while richer in context, introduce higher variability and redundancy. This leads to less efective matching during retrieval, particularly in precision-sensitive tasks. Overall, the results support the efectiveness of the linear verbalization approach in closing the semantic gap between queries and structured knowledge, thus reinforcing its role as a core design choice in HyP-KGRAG.
This ablation study evaluates the impact of removing the LLM-based denoising mechanism from HyPKGRAG. As shown in Table 3, denoising plays a critical role in enhancing both retrieval precision and the overall quality of generated responses. Across all precision-related metrics, denoising consistently leads to notable improvements indicating that noisy or irrelevant triples are efectively filtered out, allowing for more targeted and accurate content generation. One of the most noticeable improvements
F1 Precision Recall
F1 Precision Recall BM25 F1 Precision Recall BGE-M3 F1 Precision Recall
F1 Precision Recall is the BLEU score, which increases by 16.5%, confirming a stronger lexical alignment between the generated output and the ground truth. This suggests that denoising enhances the model’s ability to focus on essential content and reduce spurious or of-topic information.
However, this gain in precision comes with a slight trade-of. Specifically, ROUGE-1 recall drops by 4.2%, pointing to a potential loss of some relevant details due to the aggressive filtering of context. Despite this, the F1 scores across ROUGE and BLEU remain consistently positive, demonstrating an efective balance between precision and recall. Importantly, the semantic coherence of the outputs remains intact. The SBERT similarity score shows negligible change (from 0.598 to 0.596), indicating that the high-level semantic alignment between the generated response and the reference answer is Metric ROUGE-1-F1 ROUGE-1-Precision ROUGE-1-Recall ROUGE-2-F1 ROUGE-2-Precision ROUGE-2-Recall ROUGE-L-F1 ROUGE-L-Precision ROUGE-L-Recall BLEU SBERT Similarity preserved even when some lexical content is pruned. In summary, this analysis confirms that the denoising mechanism in HyP-KGRAG serves as a powerful tool for improving precision and refining generation quality, without compromising the underlying conceptual integrity of the output.
This work introduces HyP-KGRAG, a knowledge-grounded RAG framework that redefines how structured information from KGs is utilized in domain-specific question answering. By directly operating on triples, rather than relying on potentially noisy natural language verbalizations, HyP-KGRAG avoids linguistic ambiguities and better captures the intent. The incorporation of hypothetical path generation allows for dynamic multi-hop evidence retrieval, while the hybrid retrieval strategy (BM25 + BGE-M3) ensures high-recall access to relevant knowledge. Additionally, the LLM-as-judge denoising module significantly boosts answer quality by filtering irrelevant or noisy triples prior to generation.
Experimental results on the MSE-KG dataset validate the efectiveness of our design, showing strong gains in both factual precision (e.g., ROUGE-1 F1: 0.532) and semantic coherence (SBERT similarity: 0.629) over established baselines. These findings highlight the promise of structured, triple-based RAG approaches in enhancing reliability and accuracy in knowledge-intensive tasks.
Key limitations include: i) the system’s efectiveness still depends significantly on the reasoning capabilities of the underlying LLM and ii) current evaluation is restricted to a single dataset, i.e. MSEKG, which may limit generalizability.
Future work will focus on improving the scalability and generalizability of HyP-KGRAG by extending evaluations to diverse multi-hop KGQA benchmarks. This will allow for a more comprehensive assessment of the model’s robustness and applicability. Additionally, the integration of dynamic path retrieval mechanisms that more closely reflect ground-truth graph structures could further enhance multi-hop evidence retrieval quality. Another promising direction is the incorporation of multimodal knowledge through visual-language KGs and image-attributed triples, enabling image-augmented RAG (iRAG) models that improve contextual grounding and reduce hallucinations. The development of agentic capabilities where the system can autonomously select and sequence retrieval, rewriting, and verification modules may also lead to more adaptive and reliable performance. While the current evaluation primarily focuses on addressing hallucination challenges in domain-specific QA with the DeepSeek LLM, future work will involve more extensive experiments comparing HyP-KGRAG with state-of-the-art models such as GraphRAG and LightRAG to more rigorously assess its comparative performance.