The Virtual Record Treasury of Ireland (VRTI) is an initiative to digitally recreate the contents of the Irish central archive which was destroyed during the Civil War. The project has created a Knowledge Graph (KG) to facilitate information discovery and reasoning over the recovered items. However, complex queries must be created to retrieve data in the KG, which require a high level of technical expertise. In this paper, we explore the application of Large Language Models (LLMs) to facilitate searching of the VRTI-KG by users who lack this technical expertise and to decrease workload for those who do not. The VRTI-ChatGPT framework is proposed which uses ChatGPT to interpret requests from users and to facilitate the creation of queries which can be executed on the KG.
The Virtual Record Treasury of Ireland (VRTI) [1–3] is a state-funded programme hosted at
Trinity College Dublin. The VRTI began with the objective of digitally reconstructing
archival records destroyed during the 1922 Irish Civil war [2]. A fire during the war
destroyed the Public Record Office of Ireland, which damaged records dating back more
than 700 years. The staff at the time spent months to recover documents which were
recreated a century later. The initial VRTI Knowledge Graph (KG) was created as a result of
the lead
This section discusses the design and implementation of the VRTI-ChatGPT framework. The implementation of the framework is available online2. The framework is configurable to 1 https://www.ontotext.com/blog/natural-language-querying-of-graphdb-in-langchain/ 2 https://vrti-graph.adaptcentre.ie/gpt-search allow changes in the VRTI-KG to be easily synchronised with the involved prompts and queries. Figure 1 presents an overview of the activities of the framework. Several Python libraries3 were used to implement the framework. Flask is a customizable web framework which was used to create the web application. SPARQLWrapper is used to execute queries on the endpoint of the VRTI-KG. The Open-AI library is used to communicate with ChatGPT 4.04. Python string formatting are used to create prompts and queries from the templates. Figure 2 presents search results displayed on the implementation. 3 https://github.com/alex-randles/VRTI-ChatGPT/blob/main/libraries.pdf 4 https://openai.com/index/gpt-4/
Initial Processing of Users Input. First, the user enters a question into the search bar (A –Figure 2) or selects a suggested question. For instance, a question could ask about a specific (<person>) such as “Tell me about <person>?” “, “Where and when was <person> born”, “Was <person> in the army?” or “What job did <person> have?”. For the running example, the user inputs “Tell me about Michael Collins”. Michael Collins5 is a notable Irish person who was involved in the Irish civil war. The question is inserted in a prompt template which extracts the name of people and places from the user’s input. The generated prompt is “Extract the names of people and places in this text ‘Tell me about Michael Collins’ and output the result into a JSON dictionary”. Then, the prompt is fed into ChatGPT 4.0 using a request carried out by the OpenAI library. The result is a dictionary containing key-value pairs of names of people and places which is stored in memory.
Creation of SPARQL query. The extracted entity ("Michael Collins”) is inserted into a SPARQL [6] query template6 defined in the configuration file. The insertion involves translating the key-value pairs from the created JSON dictionary into FILTER conditions (FILTER CONTAINS(?Name, "Michael Collins”)) using string formatting methods. The query retrieves resources with a matching name along with their related properties, such as birth date and place. Thereafter, the query is executed on the VRTI-KG using the SPARQLWrapper library to retrieve matching resources. The query results are represented in dictionary format.
Creation of Natural Language Response. The dictionary containing the query result is inserted into a prompt template to generate the natural language answer. For this example, the generated prompt is “Answer this question ‘Tell me about Michael Collins’ using only the information in this dictionary ‘{Person: <….>, Occupation: <…>, BirthDate: “…”}’. Do not include any external information in the answer.”). The prompt template is designed to constrain ChatGPT to use only the information from the query results from the VRTI-KG rather than external information it has on the topic. The response (B – Figure 2) from ChatGPT is then displayed on the interface. In addition, the URI of each resource returned from the query are presented in a tabular format (C – Figure 2), which allows further exploration with the application.
Future work includes usability testing of the framework with a cohort of historians. The testing will allow the user requirements to be refined and validated. The testing will involve the participants interacting with the framework to complete several tasks which mimic the expected user interaction. In addition, it is hoped to configure the framework to answer questions from information stored in other KGs.
The VRTI-ChatGPT framework proposed in this paper provides possible direction for the integration of generative AI, such as LLMs in the VRTI-KG system [1]. It is hoped the proposed approach can facilitate searching by users who lack relevant technical expertise. Thus, reducing workload and improving the uptake of information by domain experts. 5 https://kb.virtualtreasury.ie/person/Collins_Michael_c20_dib_a1860 6 https://github.com/alex-randles/VRTI-ChatGPT/blob/main/sample-query.rq Finally, the prompts used by the framework are hoped to provide guidance for researchers who propose similar approaches.
Virtual Record Treasury of Ireland (VRTI) is funded by the Government of Ireland, through the Department of Tourism, Culture, Arts, Gaeltacht, Sport and Media, under the Project Ireland 2040 framework. The project is also partially supported by the ADAPT Centre for Digital Content Technology under the SFI Research Centres Programme (Grant 13/RC/2106_P2).