The domain of Entity Linking (EL) has been researched thoroughly, resulting in numerous approaches and a level of maturity in the field. In this dissertation, we aim at (1) creating an orchestrated, continuously-improving system through addition of approaches both existing and ones that are yet to come. On the same note, our framework shall (2) allow for Knowledge Graph (KG)- and potentially languageagnostic EL processes through cross-KG techniques, enabling further exploration of KG-dependence in the domain of EL. Finally, we will boost the ease of (3) reproducibility and (4) comparison between EL systems, their underlying techniques, as well as orchestration frameworks as a whole.
In the world of EL, there exist a plethora of approaches attempting to solve the issue of detecting and correctly disambiguating mentions from plain text documents. Many of these approaches are - at least primarily - bound to specific KGs, making system comparison problematic and portability of results to other knowledge bases questionable. Additionally, - with a few exceptions - utilizing only specific parts of a given system’s pipeline to further be processed by other systems is relatively hard, consequently increasing a researcher’s load even when intending to only research specific steps. Thus, proper system comparability and step1-related result explanation become potentially tedious tasks, for instance when attempting to solely improve Entity Disambiguation (ED), whilst intending to keep existing Mention Detection (MD) and Candidate Generation (CG) techniques constant.
Inter-System vs. Intra-System We broadly di↵erentiate between two types of
systems utilised for our orchestration: inter- and intra-system. The former refers
to orchestration across multiple full-fledged EL systems, applying End-to-End
EL systems’ results and combining them in specific ways. Among these are
approaches, such as Babelfy [
As such, we define the following problems our dissertation intends to alleviate and potentially resolve: P1 Multitude of ’Closed’ Heterogeneous Systems.
P2 Lack of Cross-KG and Robust KG-Agnostic EL Orchestration. P3 EL System Evaluation (Comparability, Result Reproducibility and Explainability). 1.1
While there exist a multitude of systems to choose from - each with their own
set of peculiarities -, to the best of our knowledge, a centralised way of utilising
existing approaches’ underlying steps without considerable development e↵ort
has yet to be released. In the case of open source projects, development and
adjustments may yield wanted results, but the accompanying e↵ort and required
underlying data structures render the task tedious, if not infeasible. Further,
taking into account licensing as well as privacy concerns - prohibiting
distribution - regarding potentially-utilised knowledge sources, makes it apparent where
limits of system, as well as result reproducibility may be reached. Therefore, our
goal is to create an orchestration framework, allowing for ease of research and
development in the domain of EL without having to worry about every step.
Alike GERBIL [
Lack of Cross-KG and Robust KG-Agnostic EL Orchestration.
Although there exist KG-Agnostic EL frameworks, to the best of our knowledge
existing orchestration approaches do not approach the domain of working with
multiple KGs simultaneously (Cross-KG) nor work for potentially ”any” wanted
knowledge base (KG-Agnostic). The lack of system portability between KGs can
be problematic for evaluation, maintenance as also ensuring the use of a system
in the long run - either due to lack of community interest for a given KG or
changes in regards to licensing. In the past, EL approaches have strongly relied on
Wikipedia/DBpedia [
GERBIL [
RQ1 How can we leverage multiple KG systems’ advantages for various settings? RQ2 When does which system(s) yield optimal results? RQ3 To what extent do domains impact EL results? RQ4 How to utilise KG-bound EL systems for agnostic tasks?
RQ5 How can we achieve and evaluate the level of KG-Agnosticity of various systems? Consequent to our research questions, we make the following hypotheses: H1 Di↵erent linking systems work to di↵erent extents based on target and underlying KGs.
H2 Domains of plain texts a↵ect the quality of results strongly (e.g. political vs.
geographical, see Obama2 and Obama3).
H3 Constraining a given system to a specific KG potentially lowers quality of results.
H4 EL research is dampened by a development overhead, rendering system comparability and reproducibility dicult, especially when varying between underlying knowledge bases. 3
As can be seen in GERBIL[
In order to achieve a sensible and highly extensible orchestration system, we have started by implementing our own KG-Agnostic embeddings-based EL approach, codenamed Agnos. Through its use of multiple KGs and various mechanisms for MD, CG and ED, we identified the degree of generalization, data structures and processes must provide in order to allow for maximal inter-compatibility with existing systems, whilst simultaneously minimizing the loss of linker-specific information. Both inter-system and intra-system compatibility is warranted and an initial REST API is provided. The code base for our EL approach is accessible at https://git.scc.kit.edu/wf7467/agnos mini, additionally to our project including initial orchestration capabilities at https://github.com/kmdn/Agnos mini. Each step from mention detection to pruning is handled by individual interfaces, allowing for definition of REST API-based connectivity for modularity of each desired step.
Once we have a relatively mature system set up, we will initially apply supervised learning techniques for our cross-KG approach. These will be supported by owl:sameAs predicates, allowing for a transparent translation layer of interconnections between KGs. Further down the line, we will additionally shift to exploiting KG alignment techniques to further facilitate the use of a plethora of KGs - both commonly available, as well as custom and potentially private ones. Due to our learning approach, we will further subdivide our experiments into various domains in order to properly assess their relevance and influence on result quality. 5
We shall assess our work from an EL point of view by using GERBIL along with
its performance metrics for evaluation - if data sets fulfill required parameters,
such as that of size and domain relevance for our experiments. Unfortunately,
GERBIL requires a reference knowledge base for data sets which may increase
diculty for a KG-Agnostic evaluation. Therefore, we will either develop other
data sets or use existing ones (e.g. [
In terms of further research to come, we will develop our framework for purposes of maximising intercompatibility of EL systems through usage of the NLP Interchange Format (NIF4). Additionally to applying supervised classification techniques, we will attempt to additionally remodel the problem in order to tackle it from di↵erent angles, perchance enabling the use of large corpora without the necessity of tedious creation of silver and gold standards. We will further refine our framework, publishing the finalised interfaces and code templates, once we consider them mature enough. Alongside framework developments, we will set up an openly-available REST API, along with a variety of configurations to choose from for execution of specific EL systems, as well as orchestration baselines.
For evaluation purposes, we would like to increase the range of data sets applied to our task. We will be targetting various gold standards of sucient size (e.g. CoNLL2011), but would like to contribute to the community by creating our own, as well as separating these into various domains to enable the impact of domain to the field of research.
On a similar note, we will be training our system - among others - on specific domains and evaluating them on the source domain, as well as various target domains in an attempt to evaluate domain dependency.
Finally, as our project progresses, we will switch from relatively simple owl:sameAs link usage to additionally making use of alignment techniques, increasing system agnosticity and likely yielding advantages from a multitude of approaches, aggregated into a single framework.