In this work, we present a web application named DBLPLink, which performs entity linking over the DBLP scholarly knowledge graph. DBLPLink uses text-to-text pre-trained language models, such as T5, to produce entity label spans from an input text question. Entity candidates are fetched from a database based on the labels, and an entity re-ranker sorts them based on entity embeddings, such as TransE, DistMult and ComplEx. The results are displayed so that users may compare and contrast the results between T5-small, T5-base and the diferent KG embeddings used. The demo can be accessed at https://ltdemos.informatik.uni-hamburg.de/dblplink/. Code and data shall be made available at https://github.com/uhh-lt/dblplink.
CEUR ceur-ws.org
Entity Linking (EL) is a natural language processing (NLP) task that involves associating named
entities mentioned in text to their corresponding unique identifiers in a knowledge graph (KG).
For example, in the question: Who is the president of USA?, the named entity span of USA has to
be linked to the unique identifier Q301 in the Wikidata KG [
A scholarly KG is a special sub-class of KGs, which contains bibliographic information
about research publications, authors, institutions etc. Some well-known scholarly KGs are
the OpenAlex2, ORKG3 and DBLP4. In this work, we focus on the DBLP KG, which caters
specifically to computer science, and as a result, is smaller in size than other scholarly KGs.
DBLP, which used to stand for Data Bases and Logic Programming5, was created in 1993 by
Michael Ley at the University of Trier, Germany [
DBLPLink can handle simple and complex questions pertaining to authorship, venues, institutions and other information available in the DBLP KG.
As shown in Figure 1, the UI consists of three main parts. In Section A, the user can either type a question as input or select a question from the drop-down menu. Further, the user can select which model to use for label span detection, and which embeddings to use for re-ranking of entities. In Section B, the results of DBLPLink are displayed. First, the top-ranked entity for each detected span is displayed, with a corresponding label and type from the DBLP KG. A hyperlink to the entity, which points to the original DBLP entity web page is also shown. Additionally, a distance metric is shown which denotes how close a match this entity is to the input question. A lower distance means a better match. Towards the bottom of the UI, we can briefly see collapsible boxes called ”Ranked Entities”, which further display the top 10 ranked entities for each of the detected label spans. Lastly, in Section C, the user has an option to remove certain combinations of results from the screen, if the UI becomes too cluttered. Our expectation is that the user shall try multiple combinations of T5 and entity embeddings to compare and contrast the results, which may need occasional cleanup from the UI.
As seen in Figure 2, the first step is to produce salient labels and types from the given input
question. For this purpose, we use the DBLP-QuAD [
With the entity labels and types produced in the previous step, a free-text-search is performed on an Elasticsearch7 instance, which contains entity URLs with their corresponding labels. The results are further filtered by the types. This gives us a list of candidate entities. In normal operation of the demo application, we present the top-ranked candidate as the final linked entity. We only proceed to the disambiguation stage if the top entity candidate has a label, that is the same as another entity in the candidate list.
In case two entities in the candidate list share the same label, we proceed with disambiguation,
which requires a further re-ranking of the candidate list. For this, we follow a common approach
of using Siamese neural networks [
During inference, a question and an entity candidate are vectorised and passed through the trained Siamese network. The cosine distance between the two resulting embeddings is computed, and the pair with the lowest distance is considered the most suitable match.
We evaluate our entity linker on the 2.000 questions of the test split of the DBLP-QuAD dataset and measure the F1-score. In Table 1, under the heading ‘Label Sorting‘, we consider the
0.698 0.698
conditional-disambiguation TransE ComplEx DistMult 0.700 0.692 0.699 0.701 0.692 0.701
hard-disambiguation TransE ComplEx DistMult 0.511 0.482 0.537 0.521 0.484 0.547 top-ranked candidate after the label sorting phase as the linked entity. We perform no further disambiguation. Under the ‘conditional-disambiguation‘ setting, we perform disambiguation only if two entities in the candidate list share the same label. Under the ‘hard-disambiguation‘ setting, re-ranking based on Siamese network cosine distances is always run after the candidate generation phase, essentially ignoring the label sorting order.
We see that hard-disambiguation lags behind significantly in performance when compared to plain label sorting, which points to the learning that for DBLP KG, degree of string match of an author or a publication is more important than the KG embeddings. Based on this finding, we allow the web application to run in ‘conditional-disambiguation‘ mode for better performance. In the case of conditional disambiguation, performance is marginally better when using TransE and DistMult when compared to label sorting, because not many cases of ambiguous labels exist in the DBLP-QuAD test set. However, it is evident from the hard disambiguation case, that DistMult performs the best on a pure disambiguation task. This may be explained by the inherent suitability of DistMult for 1-to-N relationships, which is close to the nature of the DBLP KG model, where one author may have several papers. On the contrary, TransE expects 1-to-1 relationships, while ComplEx works better for symmetric relationships. Another interesting outcome of the experiments is that the diference in parameter sizes of T5-small and T5-base does not produce any diference in performance. This may be explained by the fact that in the span label production task, much of the focus is on copying the right part of the input to the output. Since the learned knowledge of the model weights from the pre-training task is not being exploited, the larger size of T5-base does not seem to matter.
In this work, we presented DBLPLink, which is a web-based demonstration of an entity linker over the DBLP scholarly KG. In the future, we would like to add further interactivity to the UI where users can provide feedback on quality of the results. Additionally, a conversational interface for question answering would be desirable for question answering tasks, and we would like to build it in a future version.
This research is performed as a part of the ARDIAS project, funded by the “Idea and Venture Fund“ research grant by Universität Hamburg, which is part of the Excellence Strategy of the Federal and State Governments. This work has additionally received funding through the German Research Foundation (DFG) project NFDI4DS (no. 460234259).