This study explores the use of Large Language Models (LLMs) for automatic evaluation of knowledge graph (KG) completion models. Historically, validating information in KGs has been a challenging task, requiring large-scale human annotation at prohibitive cost. With the emergence of general-purpose generative AI and LLMs, it is now plausible that human-in-the-loop validation could be replaced by a generative agent. We introduce a framework for consistency and validation when using generative models to validate knowledge graphs. Our framework is based upon recent open-source developments for structural and semantic validation of LLM outputs, and upon flexible approaches to fact checking and verification, supported by the capacity to reference external knowledge sources of any kind. The design is easy to adapt and extend, and can be used to verify any kind of graph-structured data through a combination of model-intrinsic knowledge, user-supplied context, and agents capable of external knowledge retrieval.
CEUR ceur-ws.org
Knowledge Graphs (KGs) are flexible data structures used to represent structured information
about the world in diverse settings, including general knowledge [
External data
Web
unknown triples, leading to significant time and cost implications. Eforts to improve the
eficiency of human-driven KG evaluation include strategies like cluster sampling, which aims
to reduce costs by modeling annotation eforts more economically [
KGValidator Framework: Motivated by these challenges, we introduce KGValidator as a lfexible framework to evaluate KG Completion using LLMs. At its core, this framework validates the triples that make up a KG using context. This context can be the inherent knowledge of the LLM itself, a collection of text documents provided by the user, or an external knowledge source such as Wikidata or an Internet search (refer to Figure 1 for a high-level overview). Importantly, our framework does not require any gold references, which are often only available for popular benchmark datasets. This enables evaluation of a wider range of KGs using the same framework.
KGValidator makes use of the Instructor1 library, Pydantic2 classes, and function calling to control the generation of validation information. This ensures that the LLM follows the correct guidelines when evaluating properties, and outputs the correct data structures for calculating evaluation metrics. Our main contributions are: • A simple and extensible framework based on open-source libraries that can be used to validate KGs with the use of LLMs3. • An evaluation of our framework against popular KG completion benchmark datasets to measure its efectiveness as a KG validator. 1https://github.com/jxnl/instructor 2https://docs.pydantic.dev/ 3Unfortunately, IP restrictions currently prevent us from sharing our implementation, but we are happy to directly correspond with interested researchers who wish to reproduce our results • An investigation of the impact of providing additional context to SoTA LLMs in order to augment evaluation capabilities. • A straightforward protocol for implementing new validators using any KG alongside any set of knowledge sources.
Test Set
External annotator / LLM
Incomplete triple (James Joyce, author of, ?)
KGC model predictions
Ulysses Moby Dick ...
Finnegans Wake Dubliners Eveline
The rest of the paper is structured as follows: Section 2 discusses key related work, Section 3 covers our approach in detail, Section 4 presents several experiments designed to validate the framework, and Section 5 discusses results and possible extensions to this work.
Knowledge Graphs can be represented as multi-relational directed property graphs [
An early line of work on knowledge graph construction focused on the TAC 2010 Knowledge
Base Population (KBP) shared task [
Knowledge Graph Completion (KGC) is a KG construction task that has gained popularity
recently. It involves predicting missing links in incomplete knowledge graphs [
Evaluation methodologies for KG completion primarily utilize ranking-based metrics. These
include Mean Rank (MR), Mean Reciprocal Rank (MRR), and Hits@K, which gauge a model’s
ability to prioritize correct triples over incorrect ones, ofering a quantifiable measure of
performance [
Outside these tightly defined tracks, various approaches have been proposed to construct or
populate knowledge graphs. For example, NELL (Never-Ending Language Learner) [
Studies have shown that pretrained language models (PLMs) possess factual and relational
knowledge which makes them efective at downstream knowledge-intensive tasks such as open
question-answering, fact verification, and information extraction [
Pretrain-KG [
Knowledge Graph Construction Using Generative AI With the proliferation of
generalpurpose LLMs [
Khorashadizadeh et al. demonstrate the capabilities of GPT 3.5 in the task of KG construction
using an in-context learning approach [
Complementing these advancements, resources such as the Text2KG Benchmark [
User prompt Extract all named entities and any relevant properties about them in the following text: Tesla, Inc. announced on March 15, 2024, that its new electric vehicle model, the Tesla Cybertruck, will be launched in the United States in July 2024. Elon Musk, CEO of Tesla, stated that the Cybertruck represents a significant advancement in electric vehicle technology.
OpenIE System
A comprehensive survey on the unification of LLMs and KGs [
Constraining language models to produce outputs that conform to specific schemas is
challenging but essential for applications like natural language to SQL (NL2SQL) [
The tendency of LLMs to hallucinate poses a significant challenge in their application to
downstream tasks [
Evaluating automatically constructed knowledge graphs is challenging. Huaman et al. present
a comprehensive evaluation of state-of-the-art validation frameworks, tools, and methods for
KGs [
We assume the existence of a triple-extractor model, which produces a stream of candidate statements from unstructured data feeds. The triple-extractor model could be implemented by a KG completion model, one or more LLMs with well-designed prompts, or by a more traditional information extraction pipeline consisting of several distinct models that perform parsing, named entity recognition, relationship classification, and other relevant sub-tasks. For each predicted triple from the stream, we wish to validate whether it is correct in the presence of context. Once a statement has been validated, it can be written into a knowledge graph or another data store, and statements that do not pass validation can be flagged for further review. A high-level overview of the validation stage is illustrated in Figure 4.
{ }
Unvalidated triple In this work we use existing standard KGC datasets for our experiments, so in practice the candidate triples in this work are produced by streaming through existing datasets (see Section 4).
• Knowledge accrued in the LLM parameters during pretraining. • User-provided context in the form of document collections or reference KGs represented in string format. • Agents that can interact with the world to search and retrieve information in various ways.
Further detail on use of context in our validator implementations is provided in Section 3.1.
Basic Settings for Validation: The first step is to obtain KG completion predictions in the
format of a list of (ℎ, , ) triples, each consisting of a head entity ℎ, a relation and a tail entity .
All validators are instantiated in a zero-shot setting with an LLM backbone; this may be a model
from OpenAI’s model family, such as gpt-3.5-turbo-0125 [
Validation via Pydantic Models Pydantic is a data validation and settings management library which leverages Python type annotations. It allows for the creation of data models, where each model defines various fields with types and validation requirements. By using Python’s type hints, Pydantic ensures that the incoming data conforms to the defined model structure, performing automatic validation at runtime.
KG triples are passed to the validator via the Instructor library, which uses a patched version of popular LLM API clients. This patch enables the request of structured outputs in the form of Pydantic classes. It is within these Pydantic classes that we specify the structural and semantic guidelines that the LLM must follow during validation. An example of this form of prompting is shown in Figure 7. Specifically, we request that, for every triple (ℎ, , ) , the model must provide values for a number of fields: 1. triple is valid: A boolean indicating whether the proposed triple is generally valid, judged against any given context. The model can reply with True, False, or "Not enough information to say".
2. reason: An open-form string describing why the triple is or is not valid.
This section discusses the contextual information that is available to diferent validator
instantiations. We use context to mean all information that is available to a validator, including the
information stored in trained model parameters.
3.1.1. Validating with LLM Knowledge
This is the most straightforward method of triple validation. Given a triple (ℎ, , ), the objective
is to classify the triple using the LLM’s inherent knowledge about the world, learned during the
pretraining stage, and stored in the model parameters. The process is illustrated in Figure 4 and
an example can be found in appendix Figure 10. This is a powerful and simple way to verify
triples with no additional data.
3.1.2. Validation using Textual Context(s)
Inspired by the success of Retrieval-Augmented Generation (RAG) in knowledge-intensive tasks
such as question-answering [
Textual Documents
This provided corpus can be of arbitrary length and can contain a collection of documents.
The corpus will be recursively chunked and encoded by an embedding model from either
the sentence transformers library [
The Wikidata knowledge graph is built from two top-level types: Entities and Properties: Entities: Entities represent all items in the database. An item is a real-world object, concept, or event, such as “Earth” (Q2), “love” (Q316), or “World War II” (Q362). Items can be linked to each other to form complex statements via properties. In the context of KG completion, a statement can be thought of as a triple. Each entity is identified by a unique identifier, which is a Q-prefix followed by a sequence of numbers, e.g., Q42 for Douglas Adams.
Properties: Properties in Wikidata define the characteristics or attributes of items and establish relationships between them. They are the predicates in statements, linking subjects (items) to their object (value or another item). For example, in the statement “Douglas Adams (Q42) profession (P106) - writer (Q36180)”,“profession” is the property that describes the relationship between “Douglas Adams” and “writer”.
Reference KG Implementation Our approach to integrating Wikidata as a source of contextual information is simple. Given triple , an agent module searches Wikidata using the string of the subject as a query. The top Wikidata entity from the search API is returned – if no results are found for the query, a warning is thrown, and the validator will default to using its inherent knowledge. The Wikidata item is parsed to remove a list of trivial properties. Among Wikidata’s 11,000 Properties, over 7,000 of these are identifiers to external databases such as IMDb and Reddit 10. In this work, we are not interested in verifying such information, and so we discard these properties.
A string representation of the Wikidata page is now passed through the same RAG pipeline as described in Section 3.1.2, from which relevant sections are retrieved and passed to the validator as context alongside each predicted triple . This implementation is illustrated in appendix Figure 9.
In some cases, the triples we wish to validate cannot be captured with a query to Wikidata, and we do not have a collection of textual information to provide the model with additional context. To overcome this, the validator is given access to collect information relevant to the triple via a web-searching agent. The triple is formatted as a string query. An agent then searches the web using the DuckDuckGo API11. The top results for the given query are parsed and stored as a collection of documents. The validation then follows the same pattern as Section 3.1.2, whereby relevant chunks of text are retrieved as context for triple validation. This method is illustrated in appendix Figure 8.
We conduct a series of triple classification experiments to validate the efectiveness of an
LLM-backed validator for KG Completion. Our experiments make use of a number of popular
10https://wikiedu.org/blog/2022/03/30/property-exploration-how-do-i-learn-more-about-properties-on-wikidata/
11https://github.com/deedy5/duckduckgo_search
benchmark KG datasets: UMLS [
Prompt as a Hyperparameter: We emphasize the notion of a prompt as a model hyperparameter, and manually tuning it to fit a subset of data is a form of over-fitting or evaluation set leakage. In this work, we thus formulate a generic model prompt, and apply this prompt to all benchmark datasets without further changes. We include the prompt in the appendix (see Figure 6).
Through the following experiments we attempt to answer the question: Given context, can our model judge whether an unseen triple (ℎ, , ) is correct?
We are primarily interested in observing the change in evaluation performance of an LLM when it has access to context under the following settings: • LLM Inherent Knowledge: Evaluates the model’s native understanding without external data sources. • Wikidata: Uses structured data from Wikidata as the reference KG context. • Web: Incorporates information retrieved directly from the internet. • WikidataWeb: Combines data from both Wikidata and web sources. • WikipediaWikidata: Utilizes a mix of Wikipedia and Wikidata to provide a comprehensive context.
API Cost and Rate-Limiting Constraints Due to OpenAI API constraints, we run experiments using a subset of 150 examples from each dataset. This is indicated by the -150 sufix to each dataset name.
Our analysis reveals notable variations in performance across datasets, as evidenced by the results obtained using diferent validators powered by GPT-3.5 and GPT-4 language models. Specifically, the GPT-3.5 World Knowledge validator shows limited efectiveness on the FB15K237N-150, Wiki27k-150, and CoDeX-S-150 datasets (as detailed in Tables 1 and 3). However, the introduction of contextual information from Wikidata and web searches gives a strong performance boost, with the performance on the CoDeX-S-150 dataset in particular improving accuracy from 0.54 to 0.91 when using the WikidataWeb validator.
GPT-4 configurations exhibit strong performance across the board, particularly excelling in the FB15K-237N-150 and Wiki27k-150 datasets, where GPT-4 achieves the highest accuracy of 0.83 and 0.89 respectively. However, both GPT-3.5 and GPT-4 models demonstrate less satisfactory results on the UMLS-150 dataset, as indicated in Table 2.
It is noteworthy that the incorporation of context from external knowledge sources, especially web searches and Wikidata, proves beneficial for both models. Despite this, the open-source Llama2 model performs poorly on this task, as shown in Table 4 and inference examples 11 and 12. We hypothesize that future open-source LLMs may perform much better than the those currently available.
GPT-4 validators display efectiveness on the WN18RR-150 dataset, both with and without supplemental context. This robust performance is hypothesized to stem from the model’s superior grasp of English morphology and nuanced language comprehension, aligning with the linguistic focus of the WN18RR dataset.
Inherent Knowledge Insuficiency: In the case of GPT-3.5 and Llama2 70B, reliance solely on the inherent knowledge of LLM validators often leads to unquestioned acceptance of predicted triples. This indicates a limitation in the models’ ability to challenge the veracity of the information, underscoring the need for external validation mechanisms. This corroborates with ifndings from prior studies which find that LLMs struggle to memorize knowledge in the long tail [51].
Challenge in Verifying Ambiguous Triples: Our evaluation of each dataset reveals that additional information is neccesary to verify many triples. For example, a positive triple in the UMLS dataset reads ("age_group", "performs", "social_behavior"). Ambiguous triples in the UMLS and WN18RR datasets require understanding of specific ontologies, rendering web or Wikidata searches inefective for retrieving relevant context. This complexity is contrasted by datasets like FB15K-237N and Wiki27k, which involve concrete entities or facts (e.g., people, locations) more amenable to validation through widely available external sources. For example, a positive example in FB15K-237N reads "Tim Robbins", "The gender of [X] is [Y] .", "male", The Importance of Relevant Context: Performance is weaker on datasets requiring domain-specific knowledge, such as UMLS, where no model tested achieved satisfying results. This is attributed to the challenge of sourcing pertinent context for validation, as exemplified by the clinical domain triple from UMLS: ("research_device", "causes", "anatomical_abnormality"). This highlights the critical role of context in enabling accurate validation, emphasizing the need for targeted search strategies to augment the model’s knowledge base.
Limitations in Zero-Shot Triple Classification by Current Open-Source LLMs: Table 4
shows the performance of a version of the LLama-2-70B-chat model [
Adoption of Other Open-Source LLMs: At present, we find that only OpenAI and Llama models are usable with the Instructor framework. More recent models, such as Mixtral [52] and Gemma [53], are beginning to receive support under this library, but issues with constraining model output has delayed implementation. We are particularly interested in observing how other open-source models perform at this task in the future.
Building on the framework by Weidinger et al. [54], we highlight key ethical and social risks associated with using LLMs for KG validation. LLMs, trained on large-scale internet datasets, may perpetuate biases [55], discriminating against marginalized groups and potentially reinforcing stereotypes within KGs. Additionally, the alignment of LLM outputs with human preferences can introduce biases favoring certain languages and perspectives [56]. Privacy concerns also arise from LLMs potentially leaking sensitive information [57]. Furthermore, the risk of spreading misinformation through inaccurate validation poses serious challenges, especially in sensitive domains like medicine or law. Lastly, the environmental impact of training and deploying LLMs, including significant carbon emissions and water usage, underscores the need for sustainable practices in LLM-driven KG validation [58, 59].
We have introduced a flexible framework for utilizing large language models for the validation of triples within knowledge graphs, capitalizing on both the inherent knowledge embedded within these models, and upon supplementary context drawn from external sources. As demonstrated in our experiments (Section 4), the approach significantly enhances the accuracy of zero-shot triple classification across several benchmark KG completion datasets, provided that the appropriate context can be retrieved from external sources.
Use Cases: From experimentation, LLMs have demonstrated the potential to be efective validators for KG completion methods. They also open up the possibility of updating existing KG datasets with new knowledge from external sources, ensuring their relevance as gold-standard benchmarks. A practical application of this is the development of automated systems, such as bots, designed to enrich platforms like Wikidata with real-world data. These bot contributions could be systematically verified by SoTA LLMs to ensure accuracy and relevance.
As of January 2024, Wikidata encompassed nearly 110 million pages, a figure increasing at an accelerating rate. The decade between 2014 and 2023 saw an annual average of 9.57 million new pages and 191.5 million edits, and cumulative annual growth rates of 12.83% and 12.16% respectively [60]. The volume and pace of such expansion highlights the challenge of relying on manual verification methods. Leveraging LLMs to flag incorrect or unsupported edits made by users or bots could be an excellent aid to the Semantic Web community.
Future Research: As the quality of general-purpose LLMs improves, this framework should become increasingly efective in validating KG completion models. Instructor has already begun work to support other open-source LLMs, which would enable even greater flexibility in validator configuration.
Enriching models with domain-specific context and graph structural features could boost their performance across diverse datasets. Moreover, fine-tuning strategies tailored to LLMs may unlock even better performance when a model is fine-tuning specifically for the KG validation task.
As discussed in Sections 1 and 2, a growing body of work studies knowledge graph creation and augmentation using generative models. Knowledge graph creation is outside the scope of this paper, but we plan to explore this in future work. Given an information extraction model which produces KG triples from raw text, our verification pipeline could be connected to the entity and property stores of an existing KG, and automatically update the KG with high-accuracy information extracted from textual data feeds such as news. We note this is likely to be easier for some domains then others, and current SoTA LLMs will probably not be good verifiers for domain specific KGs. doi:10.18653/v1/2020.emnlp-main.669. [51] A. Mallen, A. Asai, V. Zhong, R. Das, D. Khashabi, H. Hajishirzi, When not to trust language models: Investigating efectiveness of parametric and non-parametric memories, 2023. arXiv:2212.10511. [52] A. Q. Jiang, A. Sablayrolles, A. R. et al., Mixtral of experts, 2024. arXiv:2401.04088. [53] G. Team, T. Mesnard, C. H. et al., Gemma: Open models based on gemini research and technology, 2024. arXiv:2403.08295. [54] L. Weidinger, J. Mellor, M. e. a. Rauh, Ethical and social risks of harm from Language Models, 2021. URL: http://arxiv.org/abs/2112.04359. doi:10.48550/arXiv.2112.04359, arXiv:2112.04359 [cs]. [55] E. M. Bender, T. Gebru, A. McMillan-Major, S. Shmitchell, On the Dangers of Stochastic Parrots: Can Language Models Be Too Big?, in: Proceedings of the 2021 ACM Conference on Fairness, Accountability, and Transparency, FAccT ’21, Association for Computing Machinery, New York, NY, USA, 2021, pp. 610–623. URL: https://dl.acm.org/doi/10.1145/ 3442188.3445922. doi:10.1145/3442188.3445922. [56] M. J. Ryan, W. Held, D. Yang, Unintended Impacts of LLM Alignment on Global Representation, 2024. URL: http://arxiv.org/abs/2402.15018. doi:10.48550/arXiv.2402.15018, arXiv:2402.15018 [cs]. [57] N. Carlini, F. Tramer, E. Wallace, M. Jagielski, A. Herbert-Voss, K. Lee, A. Roberts, T. Brown, D. Song, U. Erlingsson, A. Oprea, C. Rafel, Extracting Training Data from Large Language Models, 2021. URL: http://arxiv.org/abs/2012.07805. doi:10.48550/arXiv.2012.07805, arXiv:2012.07805 [cs]. [58] D. Mytton, Data centre water consumption, npj Clean Water 4 (2021) 1–6. URL: https:// www.nature.com/articles/s41545-021-00101-w. doi:10.1038/s41545-021-00101-w, publisher: Nature Publishing Group. [59] D. Patterson, J. Gonzalez, Q. Le, C. Liang, L.-M. Munguia, D. Rothchild, D. So, M. Texier, J. Dean, Carbon Emissions and Large Neural Network Training, 2021. URL: http://arxiv. org/abs/2104.10350. doi:10.48550/arXiv.2104.10350, arXiv:2104.10350 [cs]. [60] Wikimedia Foundation, Wikimedia statistics - wikidata, https://stats.wikimedia.org/, ????
Accessed: 2024-03-07. [61] Tom Jobbins, Thebloke/llama-2-70b-chat-gguf, https://huggingface.co/TheBloke/
Llama-2-70B-Chat-GGUF/, ???? Accessed: 2024-03-13. [62] G. Gerganov, gguf.md, https://github.com/ggerganov/ggml/blob/master/docs/gguf.md, 2023. Accessed: [2024-03-15].
A.1. Prompt Templates @staticmethod def validate_statement_with_no_context(entity_label, predicted_property_name, ↪ predicted_property_value): '''Validate a statement about an entity with no context a statement is a triple: entity_label --> predicted_property_name --> ↪ predicted_property_value
e.g Donald Trump --> wife --> Ivanka Trump ], max_retries=3, temperature=0, model=MODEL, ) return resp Figure 6: The prompt used across all experiments. The LLM response is captured as a Pydantic model.
Web Search API
Search Results “subject”: “anaheim_ducks”, “relation”: “teamplaysport”, “object”: “football” query: the subject of the unvalidated triple e.g. anaheim_ducks
Relevant and iltered triples ((aannaahheeiimm__dduucckkss,, lheoamgueev,eNnautei,oHnaolnHdaocCkeenytLere)ague) } properties to keep (anaheim_ducks, instance of, ice hockey team)
... (anaheim_ducks, Instagram username, AnaheimDucks) } properties to discard (anaheim_ducks, X username, anaheimducks)
...
LLM
To evaluate the capabilities of open-source LLMs in our framework, we employed a version
of the LLama-2-70B-chat model [
12https://github.com/abetlen/llama-cpp-python 13https://huggingface.co/docs/hub A.5. Validator Inference Examples >>> inp = [{"predicted_subject_name": "Heinrich Rudolf Hertz", "predicted_relation": [ "occupation" ], "predicted_object_name": "theologian",} ] >>> WorldKnowledgeKGValidator(**{'triples': inp}) >>> { {
} "triple_is_valid": false, "reason": "Heinrich Rudolf Hertz was a German physicist who made significant ↪ contributions to the field of electromagnetism and is best known for his ↪ discovery of electromagnetic waves. There is no evidence or indication that he ↪ was a theologian. The predicted relation 'occupation' with the object ↪ 'theologian' is not valid based on historical and factual information about ↪ Hertz's life and work.", "predicted_subject_name": "Edward Norton", "predicted_relation": "The profession of Edward Norton is record producer", "predicted_object_name": "record producer", "triple_is_valid": true, "reason": "Edward Norton is indeed a record producer, as he has produced several films ↪ and documentaries through his production company, Class 5 Films.", "sources": [ { "relevant_text": "['Edward Norton - IMDb Edward Norton - IMDb ![]()', 'Edward ↪ Norton\u2019s impact on the film industry goes beyond his on-screen performances. ↪ His dedication to storytelling, whether through acting, directing, or producing, ↪ reflects a commitment to meaningful and thought-provoking narratives. ↪ Norton\u2019s ability to tackle complex characters and engage with diverse ↪ genres has left an enduring mark on cinema, earning him a place among the most ↪ respected and accomplished figures in the entertainment world. As he continues ↪ to navigate the evolving landscape']" "predicted_subject_name": "Ricky Jay", "predicted_relation": "The gender of Ricky Jay is female", "predicted_object_name": "female", "triple_is_valid": true, "reason": "Ricky Jay was born as Richard Jay Potash, but he legally changed his name to ↪ Ricky Jay in 1982. Although he has been known to keep his personal life private, it ↪ is generally accepted that he identifies as male."