DBLP is currently serving as a source of structured information for the computer science community. Among the ofered services, DBLP provides users with a SPARQL endpoint interface, enabling them to write and execute SPARQL queries on the DBLP Knowledge Graph (KG). However, not every user is familiar with the SPARQL syntax and the KG schema. Having an automated method, such as semantic parsing-based KG Question Answering (KGQA), bridges the user's SPARQL familiarity gap, where KGQA converts natural language questions into structured queries to retrieve relevant data from the KG. Nevertheless, existing KGQA systems over DBLP are not robust enough to reflect the recent changes in the DBLP schema. Hence, we propose ASK-DBLP, which accepts natural language questions, converts them to SPARQL, and provides answers. In case of unclear questions, ASKDBLP advises users to reformulate their questions. Also, it empowers users to select their preferred correct entities among the candidate linked entities and update the SPARQL. The user can also modify the resulting SPARQL query. Finally, if the user confirms the correctness of the SPARQL query and the answer, ASK-DBLP updates the training set to further improve SPARQL generation. ASK-DBLP achieves a competitive performance over the DBLP-QuAD benchmark. The current deployed version of ASK-DBLP is available at https://ask-dblp.nliwod.org.
Knowledge Graphs (KGs) have emerged as a powerful means to represent entities and their rich
interrelations in a structured and machine-actionable way [
However, interaction with KGs often requires expertise in formal query languages, such as SPARQL,
making access challenging for non-experts. Even KG experts cannot master every detail of a new
KG; they need to understand its schema to query it efectively. Having a Question Answering 1 (QA)
system on top of the KG also benefits experts by enabling them to quickly explore and comprehend
the schema and content of the KG without needing to inspect its structure manually [
The first KGQA benchmark developed over DBLP is DBLP-QuAD [ 8], which introduced a dataset and a baseline model. DBLP-QuAD consists of 10K templated question-query-answer pairs that cover a range of complexities. To assess the models’ generalization ability, specific template tuples and question templates are excluded from the training set but are present in the validation and test sets. As a baseline, DBLP-QuAD employs a fine-tuned T5 model (both T5-Base and T5-Small versions), following the approach in [9]. The T5 model is trained to generate SPARQL queries, taking the concatenation of the question with entity and relation URIs as input.
NLQxform [10] leverages the BART2 model to convert questions into logical forms that structurally resemble SPARQL queries but retain entity mentions instead of URIs. Subsequently, entity mentions are resolved to their corresponding DBLP URIs using the KG search APIs. The resulting logical forms are then refined with templates and executed against the SPARQL endpoint to retrieve answers. More recently, NLQxform-UI [11] introduced a human-in-the-loop interface for QA over DBLP, while maintaining the same underlying methodology as NLQxform. BERTOlogyNavigator [12] first retrieves one-hop neighboring entities of the mentioned entity, then ranks candidate pairs by calculating the cosine similarity between the question and the pairs using BERT. The most relevant pairs are selected and ifltered through heuristic rules to provide the final answer.
Despite their merits, these approaches have certain limitations: the DBLP-QuAD baseline requires
extensive training data and is highly dependent on the availability of entity and relation URIs. Both
NLQxform and BERTOlogyNavigator sufer from limited robustness, as they heavily rely on training
sets and SPARQL templates. In contrast, our approach ofers greater robustness by guiding SPARQL
query generation with the DBLP schema and drawing on similar question-SPARQL pairs from the
training set. Furthermore, our system has a self-learning feature, incrementally expanding the training
data by incorporating newly generated and user-verified question-SPARQL pairs. This schema-driven
query generation significantly enhances robustness compared to previous methods, including those
that utilize Large Language Models (LLMs) [
ASK-DBLP follows the steps shown in Figure 1. The following sub-sections describe individual components.
ASK-DBLP first assesses the completeness and clarity of a user’s question. This is achieved by requesting Qwen 2.53 hosted by Chat-AI [13] using the prompt found in the ASK-DBLP additional resources wiki page of this paper4. If a question is vague or incomplete, the system politely prompts the user to revise the question, providing guidance for clarification. For example, ambiguous questions like "Give me the best database paper" are flagged for revision because it is unclear whether the user refers to a best-paper award or a citation impact. This initial phase helps refine user input and ensures that subsequent steps operate on well-formed questions. 2facebook/bart-base, fine-tuned on question-SPARQL pairs from DBLP-QuAD. SPARQL syntax elements (such as SELECT, COUNT, ORDER BY), parentheses, and DBLP-specific relations are added as special tokens to improve the logical form conversion. 3https://qwenlm.github.io/blog/qwen2.5-coder-family/ 4https://github.com/semantic-systems/ask-dblp/wiki/ASK%E2%80%90DBLP-ISWC-2025-Publication-Additional-Resources
The user question is then passed to an entity linker module [14, 15], which extracts candidate entities
and maps them to their respective URIs in DBLP. If the entity linker fails to identify relevant entities, it
alerts the SPARQL generator, signaling that no entities are recognized in the input question. Following
that, ASK-DBLP identifies the top 5 similar questions from the training set, following the approach
described in [
The SPARQL generator constructs a prompt that includes the question, the retrieved examples, the linked entities, and the DBLP schema. The SPARQL generation prompt (see the ASK-DBLP additional resources wiki page of this paper5) instructs the Qwen 2.5 LLM to generate an appropriate SPARQL query for the given user question. If no entities are identified in the question, the prompt is modified to include only the schema and the example questions. When similar questions cannot be determined, the LLM receives the question and the schema for SPARQL generation. Upon successful SPARQL generation, if the query produces valid results and the user provides “thumbs up" feedback, the pair comprising the user question and the generated query is added to the training set, enabling the system to improve incrementally through new examples.
Moreover, the system allows users to edit the generated SPARQL queries or select diferent entities from the entity linker results, and then request a regeneration of the query as needed. This flexibility supports iterative improvements and better aligns the query to user intent. Finally, when the user chooses to run the SPARQL query, it is executed against DBLP’s SPARQL endpoint. As shown in Figure 2, the resulting answers are presented to the user in tabular format. 5https://github.com/semantic-systems/ask-dblp/wiki/ASK%E2%80%90DBLP-ISWC-2025-Publication-Additional-Resources
The user interface6 is shown in Figure 2, with diferent sections labeled by numbers. At first, the user is prompted to enter a natural language question in the input field (1). For inspiration, a couple of example questions are also displayed. Once the question is formulated, the “Generate SPARQL”-button (2) can trigger the generation process. Once this is completed, the user is presented with a selection of entities and the candidates linked to them by the entity linker. Various candidates are presented for each entity, and the user can select the correct entity using the dropdown menus. When the selection process is finished, the query can be updated with the selected values by clicking the “Update SPARQL 6ASK-DBLP UI is built with Next.js (https://nextjs.org/docs) and seamlessly integrated with a backend implemented entirely in Python using the Flask framework. with Selected Entities”-button (3). In (4), the obtained query is presented and can be modified by the user, if needed. By pressing the “Run Query”-button (5), the query is evaluated by using the public DBLP SPARQL endpoint7 and the results are displayed (6). When the user is satisfied with the query and the results, it is optional to use the “thumbs up”-button (7) to provide the question and the SPARQL query for further training as mentioned in Section 3.3.
The evaluation of ASK-DBLP employs DBLP-QuAD8, a dataset created from an earlier DBLP dump9. The experimental result of our model is compared to the existing method’s performance in Table 1. Our approach achieves an F1 score of 0.7614, outperforming the DBLP-QuAD baseline (T5 Small), Jiang et al. [16], BERTologyNavigator [12], and PSYCHIC [17]. Although the DBLP-QuAD baseline (T5 Base) achieves the highest F1 score of 0.868, our model demonstrates strong performance given its relative simplicity and eficiency. The strength of NLQxform lies in its in-house developed entity linker. In contrast, our approach utilizes an openly available entity linking tool. The DBLP-QuAD baseline operates in a pre-linked setting, which contributed to higher performance, using pre-identified entities and relations during the SPARQL generation.
Method DBLP-QuAD baseline (T5 Base) [8] NLQxform [10] DBLP-QuAD baseline (T5 Small) [8] Jiang et al. [16] BERTologyNavigator [12] PSYCHIC [17] ASK-DBLP (Ours)
Unlike the DBLP-QuAD T5 Base model, our method requires only a minimal training set, utilizing similar questions as inline context during SPARQL generation. Moreover, we do not train any language model from scratch; instead, we leverage a publicly available model, specifically Qwen 2.5 Coder 10 In contrast to NLQxform, which relies heavily on SPARQL templates derived from the DBLP-QuAD dataset before SPARQL generation, ASK-DBLP generates SPARQL queries without using any predefined templates. Instead, it is guided by automatically selected examples and the DBLP schema. Alternative methods necessitate retraining and template updates from scratch to adhere to the continuous evolution of the DBLP KG. In comparison, ASK-DBLP is more robust, as it dynamically follows the schema and generates SPARQL queries driven by a small set of example queries. Furthermore, ASK-DBLP incorporates a self-improving feature by leveraging user feedback. Incorporating user-verified and voluntarily submitted questions and queries enables continual improvements and adaptation. 7https://sparql.dblp.org/ 8DBLP-QuAD is a well-established KGQA benchmark tailored to the computer science domain, providing a reliable and widely used basis for evaluating KGQA methods. Using DBLP-QuAD enables direct comparison with existing approaches, facilitating objective assessment and tracking progress within the research community. 9https://blog.dblp.org/2022/03/02/dblp-in-rdf/ 10Since our goal was not to comprehensively evaluate LLMs but to select one that best suited our task, we performed a manual random evaluation of a few open-source models and selected Qwen 2.5 Coder. Its familiarity with structured query languages such as SQL enhances its ability to produce well-structured SPARQL queries. Moreover, Qwen 2.5 Coder follows instructions reliably and, for our specific task, makes it a highly suitable choice. We propose ASK-DBLP, a KGQA system over DBLP, that leverages the KG schema and a few examples during SPARQL generation, making it easier to adapt to the continuously evolving DBLP KG. Moreover, our proposed method enables the collection of self-checked question-SPARQL pairs from users. Those new examples are used to improve the SPARQL generation and can be shared with the research community.
One limitation is that we are currently using an entity linker and LLM APIs; we plan to decouple from the high API usage calls by deploying them locally and further reducing latencies. As one direction of future work, we will explore diferent entity linking methods. We currently use few-shot in-context learning-based SPARQL generation over an open-source general purpose LLM; fine-tuning the LLM for SPARQL generation is another future direction.
This work is supported by the German Research Foundation (DFG) consortium NFDI4DataScience under Grant No. 460234259 and consortium NFDIxCS Grant No. 501930651. We utilized two NVIDIA RTX A5000 24GB GPUs, kindly provided by the NVIDIA Academic Hardware Grant Program.
While preparing this work, the author(s) used Grammarly and DeepL to: Grammar and spelling check. Further, the author(s) used the Flux11 model hosted by Chat-AI to: Generate images (the ASK-DBLP logo shown on top of Figure 2). After using these tool(s)/service(s), the author(s) reviewed and edited the content as needed and take(s) full responsibility for the publication’s content. [7] A. Perevalov, X. Yan, L. Kovriguina, L. Jiang, A. Both, R. Usbeck, Knowledge Graph Question Answering Leaderboard: A Community Resource to Prevent a Replication Crisis, in: N. Calzolari, F. Béchet, P. Blache, K. Choukri, C. Cieri, T. Declerck, S. Goggi, H. Isahara, B. Maegaard, J. Mariani, H. Mazo, J. Odijk, S. Piperidis (Eds.), Proceedings of the Thirteenth Language Resources and Evaluation Conference, European Language Resources Association, Marseille, France, 2022, pp. 2998–3007. URL: https://aclanthology.org/2022.lrec-1.321/. [8] D. Banerjee, S. Awale, R. Usbeck, C. Biemann, DBLP-QuAD: A Question Answering Dataset over the DBLP Scholarly Knowledge Graph, in: I. Frommholz, P. Mayr, G. Cabanac, S. Verberne, J. Brennan (Eds.), Proceedings of the 13th International Workshop on Bibliometric-enhanced Information Retrieval co-located with 45th European Conference on Information Retrieval (ECIR 2023), Dublin, Ireland, April 2nd, 2023, volume 3617 of CEUR Workshop Proceedings, CEUR-WS.org, 2023, pp. 37–51. URL: https://ceur-ws.org/Vol-3617/paper-05.pdf. [9] D. Banerjee, Semantic Parsing for Knowledge Graph Question Answering with Large Language Models, in: C. Pesquita, H. Skaf-Molli, V. Efthymiou, S. Kirrane, A. Ngonga, D. Collarana, R. Cerqueira, M. Alam, C. Trojahn, S. Hertling (Eds.), The Semantic Web: ESWC 2023 Satellite Events - Hersonissos, Crete, Greece, May 28 - June 1, 2023, Proceedings, volume 13998 of Lecture Notes in Computer Science, Springer, 2023, pp. 234–243. URL: https://doi.org/10.1007/ 978-3-031-43458-7_42. doi:10.1007/978-3-031-43458-7\_42. [10] R. Wang, Z. Zhang, L. Rossetto, F. Ruosch, A. Bernstein, NLQxform: A Language Model-based Question to SPARQL Transformer, in: D. Banerjee, R. Usbeck, N. Mihindukulasooriya, G. Singh, R. Mutharaju, P. Kapanipathi (Eds.), Joint Proceedings of Scholarly QALD 2023 and SemREC 2023 co-located with 22nd International Semantic Web Conference ISWC 2023, Athens, Greece, November 6-10, 2023, volume 3592 of CEUR Workshop Proceedings, CEUR-WS.org, 2023. URL: https://ceur-ws.org/Vol-3592/paper2.pdf. [11] R. Wang, Z. Zhang, L. Rossetto, F. Ruosch, A. Bernstein, NLQxform-UI: An Interactive and Intuitive Scholarly Question Answering System, in: N. Ferro, M. Maistro, G. Pasi, O. Alonso, A. Trotman, S. Verberne (Eds.), Proceedings of the 48th International ACM SIGIR Conference on Research and Development in Information Retrieval, SIGIR 2025, Padua, Italy, July 13-18, 2025, ACM, 2025, pp. 3990–3993. URL: https://doi.org/10.1145/3726302.3730153. doi:10.1145/3726302.3730153. [12] S. Rajpal, R. Usbeck, BERTologyNavigator: Advanced Question Answering with BERT-based Semantics, CoRR abs/2401.09553 (2024). URL: https://doi.org/10.48550/arXiv.2401.09553. doi:10. 48550/ARXIV.2401.09553. arXiv:2401.09553. [13] A. Doosthosseini, J. Decker, H. Nolte, J. M. Kunkel, Chat AI: A Seamless Slurm-Native Solution for
HPC-Based Services, 2024. URL: https://arxiv.org/abs/2407.00110. arXiv:2407.00110. [14] D. Banerjee, Arefa, R. Usbeck, C. Biemann, DBLPLink: An Entity Linker for the DBLP Scholarly Knowledge Graph, in: I. Fundulaki, K. Kozaki, D. Garijo, J. M. Gómez-Pérez (Eds.), Proceedings of the ISWC 2023 Posters, Demos and Industry Tracks: From Novel Ideas to Industrial Practice colocated with 22nd International Semantic Web Conference (ISWC 2023), Athens, Greece, November 6-10, 2023, volume 3632 of CEUR Workshop Proceedings, CEUR-WS.org, 2023. URL: https://ceur-ws. org/Vol-3632/ISWC2023_paper_428.pdf. [15] Banerjee, Debayan and Tafa, Tilahun Abedissa and Usbeck, Ricardo, DBLPLink 2.0 - An Entity Linker for the DBLP Scholarly Knowledge Graph, arXiv preprint https://arxiv.org/abs/2507.22811 (2025). [16] L. Jiang, X. Yan, R. Usbeck, A Structure and Content Prompt-based Method for Knowledge Graph
Question Answering over Scholarly Data., 2023. [17] H. A. Akl, PSYCHIC: A Neuro-Symbolic Framework for Knowledge Graph Question-Answering Grounding, in: D. Banerjee, R. Usbeck, N. Mihindukulasooriya, G. Singh, R. Mutharaju, P. Kapanipathi (Eds.), Joint Proceedings of Scholarly QALD 2023 and SemREC 2023 co-located with 22nd International Semantic Web Conference ISWC 2023, Athens, Greece, November 6-10, 2023, volume 3592 of CEUR Workshop Proceedings, CEUR-WS.org, 2023. URL: https://ceur-ws.org/Vol-3592/ paper1.pdf.