In the field of Question Answering on Knowledge Graphs (KGQA), entity linking is an essential substep to transform a natural language (NL) question to the formal query language SPARQL. The required entities need to be spotted in the text and the correct resource in the knowledge graph (KG) has to be identified. The latter step is especially hard for entities in questions when there is only little or no context. We already presented a previous approach using abstract meaning representation (AMR) of the question to spot the surface forms of entities. With this paper, we present our adapted approach for the domain of scholarly questions - in specific DBLP QuAD.
approach when the underlying KG is modelling a very specific domain, such as the DBLP RDF KG.
The general approach utilizes the abstract meaning representation (AMR) of the question. Within the syntactic structure of the graph, named entities are represented as name nodes and the surface form is included as child nodes. For the subsequent entity linking process, a extensive entity dictionary is required as well as context information for all entities and from the input question. We examined several options for the DBLP RDF KG and the DBLP-QuAD.
The remainder of the paper is structured as follows: Related work is described in Section 2. The entity linking approach including the description of the dictionary and ranking options is depicted in Section 3. With our approach, we took part in the Scholarly QALD challenge and the results are discussed in Section 4. We summarize our approach and discuss future work in Section 5.
One of the first entity linking approaches has been introduced by Milne et al. in 2008 [
As we are using the AMR graph for the generation of the SPARQL query, other approaches
based on AMR graphs might be interesting. Besides our own approach [
Both latter approaches do not use the AMR graph itself, but BLINK[
Our presented approach is based on the entity linking process introduced in [
RDF • exclusion of publishers as named entities • disambiguation using complete question
We will further describe the specific approach for the DBLP QALD questions in the following sections.
The DBLP QUAD dataset contains questions about authors and publications. Paper titles are enclosed by quotation marks which, in general, makes it easy to identity the surface form of the titles. As we utilize the AMR graph for the further generation of the SPARQL query, we aim to maintain the linguistic embedding of the entities within the NL question and therefore focus on identifying the named entities (including paper titles) within the AMR graph. Unfortunately, AMR graphs and their generation have some limitations, including that it ”drops grammatical number, tense, aspect, quotation marks, etc.”1. Dropping the quotation marks results in embedding the paper title in the context of the question and using parts of title in the linguistic structure. For instance, the question How many authors does ’Measuring the impact of temporal context on video retrieval’ have? results in an incorrect AMR graph as shown of the left below. The correct graph is shown on the right: Incorrect AMG graph with dropped quotation marks: (h / have-03 :ARG0 (p / publication :name (n / name :op1 "Measuring" :op2 "The" :op3 "Impact" :op4 "of" :op5 "Time" :op6 "Context")) :ARG1 (p2 / person :ARG0-of (a / author-01) :quant (a2 / amr-unknown)) :ARG2 (r / retrieve-01 :ARG1 (v / video)))
Correct AMR graph: (h / have-03 :ARG0 (p / publication :name (n / name :op1 "Measuring" :op2 "The" :op3 "Impact" :op4 "of" :op5 "Temporal" :op6 "Context" :op7 "On" :op8 "Video" :op9 "Retrieval")) :ARG1 (p2 / person :ARG0-of (a / author-01) :quant (a2 / amr-unknown)) 1https://github.com/amrisi/amr-guidelines/blob/master/amr.md
Obviously, there are two problems with the handling of the question: 1. the surface forms of the paper titles are changed (temporal → time) 2. parts of the paper title are excluded from the name node (video retrieval as separate node under :ARG2) We therefore use a placeholder for the paper titles before the AMR generation and then replace the placeholder with the actual title at the correct position within the AMR graph. The respective AMR graph for the question including the placeholder looks like this: (h / have-03 :ARG0 (p / publication :name (n / name
:op1 "PaperOne")) :ARG1 (p2 / person :ARG0-of (a / author-01) :quant (a2 / amr-unknown))
In this way, we preserve the syntactic structure of the question and the position of the paper title within the graph.
Another issue with the generation of the AMR graphs which seems to be specific to the DBLP QuAD dataset, is the mentioning of author names in certain ways. In some cases, authors are mentioned in the question with their lastname first following the given name separated by a comma, as e.g. in What are the papers written by Ben-Simon, E. and Andrey Zhdanov together?. The AMR for that question looks like this: (p / paper :ARG1-of (w / write-01 :ARG0 (a / and :op1 (p2 / person :name (n / name
:op1 "Ben-Simon")) :op2 (p3 / person :name (n2 / name
:op1 "E.")) :op3 (p4 / person :name (n3 / name :op1 "Andrey" :op2 "Zhdanov")) :mod (t / together))) :domain (a2 / amr-unknown))
Apparently, the comma is treated like a separator for a list of person names. We therefore preprocess the question using regular expressions to identify reverse author names and rewrite the author names in the pattern [first name(s)] [space] [last name(s)] before generating the AMR graph.
In general, DBLP RDF contains named entities (in terms of URIs) of type publication and author. As described in Section 3, we collect descriptive information for the entities in order to disambiguate ambiguous surface forms.
For publications, we do not generate or collect alternative titles. Our dictionary contains the exact title as contained in the DBLP RDF KG. We assume that titles of publications are referenced in questions as exact and complete as possible. As the DBLP RDF KG does not contain references of publications to other publications, we did not include a popularity measure for publications in the dictionary. As context information, we collected the author names and the publisher for each publication.
In some cases, DBLP QuAD references authors in the questions only using their first name or even middle name. In other cases, one of the names is abbreviated and the others are not. Therefore, we generate an extensive set of alternative labels from their names. For instance, the author Antonio Manuel Fernandez Villamor has 21 entries in our dictionary with labels generated from the combination of parts of his name, as e.g. antonio manuel fernandez v., a. m. f. villamor, or antonio m. fernandez villamor. Overall, our dictionary contains almost 30 million entries for DBLP authors. For the popularity measure, we counted the number of publications for each author. The context information for authors consists of the titles of all their publications transformed to a set of keywords and filtered for stop words. Also, we added all names of co-authors and all publishers the author have published in.
Table 1 shows the overview of information collected for both entity types. alternative labels indegree / popularity measure contextual information
Publications Authors / each part of the name separately / number of publications author names & publisher set of keywords from publications & co-author names & publishers
DBLP QuAD contains questions mentioning publications by specific publishers, as e.g. Has J. Florens published in Computer Graphics in the last 5 years?. One could assume that Computer Graphics is the surface form of a named entity represented by an URI in the DBLP RDF KG. But, this is not the case. The respective SPARQL query must use the following triple to express that something has been published in Computer Graphics: ?x <https://dblp.org/rdf/schema#publishedIn> ’Computer Graphics’ .
In the AMR graph, the publisher Computer Graphics is represented in a name node as a named entity as shown in the graph below: (p / publish-01 :ARG0 (p2 / person :name (n / name :op1 "J." :op2 "Florens")) :ARG1 (p3 / publication :name n :op1 "Computer" :op2 "Graphics") :polarity (a / amr-unknown) :time (b / before :op1 (n2 / now) :duration (t / temporal-quantity :quant 5 :unit (y / year))))
Our procedure to map a surface form to the underlying knowledge graph includes a similarity search in case an exact match is not successful. All identified named entities will be tried to be mapped in the KG. For Computer Graphics we would receive publications with the same name, as e.g. https://dblp.org/rec/books/daglib/0067138 or https://dblp.org/rec/journals/cg/HillM92. Therefore, we added a check if the surface form is included in the labels of publishers. If so, the node is excluded from the mapping process 2.
As described in [
LC-QuAD 2.0)
For DBLP-QuAD, we took part at the Scholarly QALD challenge 20233 and evaluated our approach on the test datasets of the development and the final phase. The settings as stated above for QALD-9 and LC-QuAD 2.0 did not achieve satisfying results on the DBLP-QuAD test datasets. With these settings, we achieved a maximum recall of 0.74 and maximum precision of 0.61. Therefore, we evaluated diferent weighting settings and achieved the best results on using only the context score – ignoring the popularity measure and the label score. In addition, we changed the context information from only the parent node to all name nodes in the AMR graph of the question and could increase recall and precision with this adjustment. Finally, we 2Publications stemming from the placeholder dictionary as described in Section ?? are flagged to be included in the mapping even though they share their name with a publisher. 3https://kgqa.github.io/scholarly-QALD-challenge/2023/ expanded the context information used for calculating the context score by using all words from the question filtered by stop words. Thereby, we achieved the best result overall with a recall of 0.926 and precision of 0.76 (1= 0.8353) on the test dataset of the final phase.
The following example depicts a typical disambiguation case. The question Did Xiao L. publish in INTERSPEECH in the last 2 years? from the DBLP-QuAD train dataset contains the highly ambiguous reference to an author Xiao L.. There are 93 authors with the abbreviated name Xiao L. listed in DBLP RDF. The author with most publications (as in our defined popularity measure) is a person with the name Xiao Liu4 with 152 publications5. But the author required for the query as in the train dataset is a person with the name Xiao Li6 with only 36 publications. In this case the context INTERSPEECH tips the scales and leads to the correct author. But, the dataset also contains many questions where authors are mentioned without any context. For all-purpose questions, such as Did Heinrich Heine die in Paris?7, the most popular named entity with the name Paris would come into mind and is referenced as correct named entity in the respective QA dataset. But obviously, this rule does not apply for named entities in the DBLP-QuAD dataset and maybe also in general in datasets of very specific domains 8.
Therefore, the lessons learned from the entity linking challenge on DBLP-QuAD questions are the following: • an extensive list of alternative labels for authors is essential • the descriptive information of entities in the dictionary has to be collected thoroughly • the context from the question is the most important information for the disambiguation Hence, in this case for the DBLP-QuAD dataset, context matters even more than for datasets of a general domain.
We presented examination and adapted approach on entity linking for the scholarly domain using AMR graphs and an extensive entity dictionary. We showed that major adaptations are required for the scholarly domain and especially DBLP-QuAD compared to the original approach based on all-purpose knowledge graphs, such as Wikidata and DBpedia.
With this approach, we achieved a recall of over 92% and precision of over 76% and thereby winning the Scholarly QALD challenge 2023 on entity linking.
Our examination shows the importance of context for the disambiguation in a specific domain and especially when very ambiguous surface forms have to be resolved. We assume that this might due to the specific domain of the KG and the QA dataset. But, we need to further examine this issue on other domain-related KGs and datasets. 4https://dblp.org/pid/82/1364-4 5at least for our status of DBLP RDF 6https://dblp.org/pid/66/2069-6 7Question with uid 4303 in LC-QuAD 2.0 train dataset 8which needs to be proven on more datasets