We will demonstrate KGQAn, a question-answering platform trained independently of KGs. KGQAn transforms a question into semantically equivalent SPARQL queries via a novel three-phase strategy based on natural language models trained generally for understanding and leveraging short English text. Without preprocessing or annotated questions on KGs, KGQAn outperformed the existing systems in KG question answering by an improvement of at least 33% in F1-measure and 61% in precision. During the demo, the audience will experience KGQAn for question answering on real KGs of topics of interest to them, such as DBpedia and OpenCitations Graph, and review the generated SPARQL queries and answers. A demo video is available online 1.
of predicates, e.g., <http://dbpedia.org/property/leadfigures>, and entities, e.g., <http://dbpedia.org/resource/Boston_Tea_Party>, and (ii) identify basic graph patterns (BGP), e.g., a set of triple patterns connected through a common subject/object. Thus, forming a SPARQL query is challenging.
To easily explore KGs, KG question answering systems map a question into
semantically equivalent SPARQL queries. Existing systems need thousands of
fully annotated questions or require excessive preprocessing. DTQA [
We developed KGQAn, a question-answering platform trained independently of KGs, to address the above challenges. NLP-based models are effectively used to construct KGs from text.This fact inspired us to develop a three-phase strategy based on NLP models trained generally for understanding and leveraging a short English text. KGQAn outperforms the state-of-the-art systems by improving F1-measure and precision by least 33% and 61%, respectively. In this demo, Section 2 outlines the KGQAn architecture. Section 3 gives a glimpse on the evaluation. Section 4 explains the demo scenario and concludes. 2
We outline our three-phase strategy and demonstrate each phase as illustrated in Figure 1. Unlike existing systems, our strategy utilizes NLP models trained independently of the targeted KG, then uses lightweight SPARQL queries to annotate PGP’s node and edges with corresponding vertices and predicates using semantic similarity models. KGQAn starts by extracting relation triples from the question to construct a phrase graph pattern (PGP). The relation triples are extracted from a question using our relation triple generator model pretrained for short English text. KGQAn fetches via SPARQL queries vertices and predicates, ranks them semantically to annotate the PGP with top-k ones. Finally, KGQAn executes the BGP queries with the highest rank, then filters the queries whose answers do not match the predicted answer data type.
The PGP Predictor extracts a set of relation triples patterns from the question to construct the PGP. A relation triple pattern is a triple of a relation phrase connecting two entities, e.g., a relation triple is hBoston Tea Party, take PHRASE GRAPH PATTERN (PGP) PREDICTOR A Seq2Seq Model to extract Relation Triples Boston Tea Party, take place, ?when Boston Tea Party, led by, ?who .
BASIC GRAPH PATTERN SPARQL QUERY (BGP) MATCHER AND ANNOTATOR MANAGER
Entity Name: ?when Queries Builder and
Relation Phrase: “take place” n2 TPyopses:ibDlaeteanswers: Ambiguity-Based Filtering n1 Predicates: {((pdlaatcee,,00.4.83,,SS))} {<answer, score>} “EBnotisttyonNaTemaeP:arty” PRreeladtiicoantePsh:r{ase: “led by” Type: Named Entity (leadfigures, 0.52, S) Vertices: (partof, 0.49, S)} {<v_uri, score>} BGP_I : { dbpediaR:Boston_Tea_Party dbpediaP:date ?When .
Entity Name: ?whom dbdpbepdeidaiRa:PB:olsetaodnf_iTgeuar_ePsar?twyhome.
n3 TPyopses:ibPleersaonnswers: }
{<answer, score>}
place, ?wheni in Figure 1. Our triple generator model is a Sequence to Sequence
(Seq2Seq) Deep Learning model based on BART [
Preparing our annotated dataset does not need the annotated SPARQL query from LC-QuAD. We only extract the English questions from LC-QuAD to annotate them. For example, for the input Q: How many movies did Stanley Kubrick direct?, the output is ”Subject”: Stanley Kubrick, ”Object”: ?unknown, ”Predicate”: direct. The dataset covers a wide variety of question types. In our relation triples generation model, we fine-tuned the BART large model to work on the triples generation task. First, the triples generation model is trained separately using the prepared dataset on a GPU machine. We trained the model for 3 epochs with a batch size of 4. Moreover, we used Adam optimizer with a learning rate of 0.0005, and a gelu activation function. The resulting trained model is saved to be integrated with KGQAn. After that KGQAn uses this trained model to extract the relation triples from the question under consideration. Then, the predicted triples are post-processed to construct the PGP. The generated triples are in (subject, predicate, object) format where the subject, object, and variables correspond to PGP nodes, and the relation phrases correspond to PGP edges as shown in Figure 1.
The BGP Matcher aims to annotate the PGP nodes and edges with
candidate vertices and predicates in the target KG. The main idea is to prune the
search space, i.e., the KG, by identifying a set of vertices that match
syntactically the detected entities. The BGP Matcher then calculates the semantic
affinity between the vertex’s label and the entity to rank the top-k vertices. Our
semantic affinity model is based on Word Embeddings [
To annotate the PGP edges, KGQAn fetches, for each vertex v, a set of predicates connected to v when v is a subject and when it is an object. The PGP is an undirected graph, as the direction of the edge is based on the actual triple in the KG. KGQAn annotates each edge in the PGP with a set of tuples. Each tuple indicates a specific predicate, its direction, and its semantic affinity score to the extracted predicate. We formulate intermediate SPARQL queries searching for (i) a subject whose label matches a set of keywords, or the subject is of a particular type to fetch v, and (ii) a set of predicates connected with a given v as subject or object.
SPARQL Query Manager: This component generates the top-k SPARQL queries by traversing the annotated PGP to select the top-k basic graph patterns (BGP) semantically equivalent to the question. Our algorithm generates all the combinations of hsubject, predicate, objecti using these lists. Our Ambiguity-Based Ranking and Filtering module adjusts the final score of a BGP based on the semantic affinity between the predicate and the predicted answer type. Furthermore, the result of the queries is filtered depending on the answer data type. KGQAn filters the query whose result does not match the predicted answer data type. 3
The state-of-the-art systems, such as DTQA, gAnswer, and WDAqua, were
evaluated using the most recent Challenge on Question Answering over Linked
Data (QALD-9) [
For the word embedding, we used a GloVe model pre-trained on 16B tokens from a large English corpus. We deployed the DBpedia dataset used with QALD-9 at a virtuoso SPARQL endpoint running at a remote VM. KGQAn submits SPARQL queries via HTTP calls to the remote SPARQL endpoint. We evaluated KGQAn using the QALD-9 test dataset, i.e., the 150 questions on DBpedia. Unlike other systems, KGQAn did not use the the QALD-9 training dataset to tune its performance. Without preprocessing on DBpedia or annotated questions, KGQAn significantly outperformed DTQA and gAnswer, especially in terms of precision and F1-measure, as shown in Table 2. Moreover, KGQAn is efficient in terms of time. The average time is less than 3 seconds for the endto-end KGQAn pipeline, including the execution time of all the intermediate SPARQL queries, which fetch the candidate URIs for vertices and predicates, plus the execution of the final top-k semantically equivalent queries.
In this demo, we will use real datasets in the order of billions RDF triples from DBpedia and OpenCitations Graph. KGQAn uses models trained independently of the KG. Thus, we will also allow participants to explore a KG of their choice if the KG has a public SPARQL endpoint. We integrated our Seq2Seq model into a pipeline of lightweight queries to efficiently map a question to its semantically equivalent SPARQL queries. The demo will give a chance to discuss exciting research ideas inspired by such a pipeline to solve similar problems.