The majority of transmitted information consists of written text, either printed or electronically. Extraction of this information from digital resources requires the identification of important entities. While Named Entity Recognition (NER) is an important task for the extraction of factual information and the construction of knowledge graphs, other information such as terminological concepts and relations between entities are of similar importance in the context of knowledge engineering, knowledge base enhancement and semantic search. While the majority of approaches focusses on NER recognition in the context of the World-Wide-Web and thus needs to cover the broad range of common knowledge, we focus in the present work on the recognition of entities in highly specialized domains and describe our approach to ontology-based entity recognition and annotation (OER). Our approach, implemented as a first prototype, outperforms existing approaches in precision of extracted entities, especially in the recognition of compound terms such as German Federal Ministry of Education and Research and inflected terms.
Two realms define the range in which entity recognition has to take place. One realm needs to cover a large and broad range of common entities, related to common knowledge and contained in the broad range of web resources and documents, largely consisting of factual information about named entities. The other one covers highly specialized information in monothematic application domains and has a strong focus on the terminology used in the domain. Although it often covers also a large, but still limited set of entities, these entities are usually identified by complex names, such as *
This work has been partially supported by the "Wachstumskern Qurator – Corporate Smart Insights" project (03WKDA1F) funded by the German Federal Ministry of Education and Research (BMBF). chemical compounds Acetylsalicylic Acid, job titles Servicetechniker für Windkraftanlagen, etc.
Recognition of entities in the first realm is known under the term named entity recognition (NER). Since these approaches are often based on large common corpora, they are usually generic and domain-independent, but applicable to a broad range of application areas. Although they can cope with text in arbitrary domains, these approaches have problems recognizing all important entities in an application domain. Because of this incompleteness, they can achieve only limited recall. Further, their precision is limited by missing information.
We summarize approaches of the second realm, which are not limited to named entities, under the more general term entity recognition (ER). These approaches rely on given background knowledge about the entities in a particular domain. Thus, they are domain-dependent and applicable to a smaller range of domains, but configurable by the background knowledge. Their goal is to detect as much relevant domain entities as possible, enabling thus higher recall and precision. If such an approach uses knowledge formalized as ontology1, we term it ontology-based entity recognition (OER).
An example of NER in the first realm is – besides others – DBpedia Spotlight, an
open source annotation tool for recognizing named entities in text and linking them to
DBpedia resources [
Approaches of the first realm often require disambiguation mechanisms to decide
which interpretation of a named entity is in a certain context intended. DBpedia
Spotlight either tries to identify the right interpretation automatically or offers the user to
select one of the interpretations [
Entity recognition in the second realm is based on the availability of given controlled vocabularies, which may originate from lists of terms, taxonomies, thesauri up to ontologies. Although several papers describe and evaluate their systems, they are usually not publically available. One exception here is the BioPortal Annotator2, which performs ER and annotation of documents based on a larger number of biomedical ontologies. But the BioPortal Annotator has problems too: if more than one ontology is used for annotation its results are highly redundant. Its ability to recognize compound terms is limited and, even if just one ontology is used for ER, it is rather slow.
Disambiguation plays a minor role in this second realm, since the number of
polysemic terms in controlled vocabularies is usually rather small. Although a term, like
1 We use the term ontology in the sense of [
For complementing DBpedia Spotlight, we decided to follow a similar approach as
[
The architecture of OER consists of two parts working in two subsequent phases: during the first compilation phase, a language-dependent lemmaCache is initialized and the terminology of a knowledge model is precompiled into a lookupDictionary.
The second annotation phase uses the precompiled lemmaCache and the lookupDictionary for the annotation of texts as shown in Fig. 1. For the initialization of the lemmaCache different sources are consulted depending on the language. For German the lemmaCache is initialized on the base of a dump of Morphys morphology dictionary3, which allows lemmatizing more than 400.000 German word forms directly from the start. This cache is augmented during runtime via the API of Wortschatz Leipzig with additional lemmas derived from 1.000.000 sentences from Wikipedia or a news corpus. For English the lemmas are derived currently from Wordnet only. 3 http://www.danielnaber.de/morphologie/ (last access Dec., 13th 2019)
A knowledge model of a domain is used as source for the derivation of a controlled vocabulary consisting of (term,URI)-tuples. The terms, which may consist of single tokens or token sequences, form the controlled vocabulary for the entity recognition. The URIs build the values used for annotating the recognized entities.
By default the terms are derived for knowledge models in OWL, RDFS and SKOS from rdfs:label, skos:predLabel, skos:altLabel or skos:hiddenLabel. However, since a knowledge model may consist of more complex structures built from concepts, preferred terms and their preferred labels, we also allow users of OER to define their own derivation pattern for (term,URI)-tuples via a user-defined SPARQL call. For simpler knowledge models, we also allow the specification of (term,URI)-tuple via CSV.
Especially, for German it is important that the entity recognition can recognize different spelling variants of the same entity. German is famous for its compound nouns, creating new nouns by connecting adverbs (Soforthilfe), adjectives (Dreirad), verbs (Fahrlehrer) and nouns (Mädchenhandelsschule). However, under certain circumstances parts of a compound can be separated by a hyphen in order to improve legibility and to avoid ambiguity (Mädchen-Handelsschule). As experience has shown during the analysis of search queries, authors and users often even separate these parts incorrectly by blanks (Robert Koch Institut)4. These deficiencies may also occur combined in different variations, e.g. (Johannes Gutenberg-Universität Mainz). In order to recognize all these variations easily as referring to the same entity or concept, we compute them in advance and store them in a two-layered prefix tree structure. 2.3
The first layer is based on a radix tree that builds a termStore. Each lemmatized term contained in a term sequence is used as key of the termStore to store and access a unique id for each lemma. The list of unique ids of each term sequence are used subsequently as key in a prefix tree called sequenceStore forming the second layer. These lists of unique ids are used to store and access URIs of entities and concepts corresponding to term sequences.
Thus, a term sequence like gewählter Abgeordneter des deutschen Bundestags, will be lemmatized and normalized as gewählt abgeordnete des deutsch bundestag which in turn is translated into a numerical list, e.g. [2643, 92, 83634, 12344. This encoding of the term sequence is used as list-based key to access the URI of the corresponding knowledge model concept in the sequenceStore. This translation safes space through the numerical encoding of strings and allows mapping different flexions of labels, such as gewählten Abgeordneten des deutschen Bundestag, to the same URI.
This two-layered data structure is initially set up during the compilation phase and used during run-time to scan a given text in order to recognize and annotate compound terms. Suppose that the knowledge model contains two additional concepts with the labels gewählte Abgeordnete and deutscher Bundestag. Assume further that we like to annotate the following text: Als gewählte Abgeordnete des deutschen Fischzüchterverbandes reisen sie nach Berlin und treffen die gewählten Abgeordnete des deutschen Bundestags.
The recognition and annotation process simply scans the tokenized text from the beginning until a term contained in the termStore is reached (see Fig. 2). In the example, this is gewählte. Starting from this term a lookahead is performed searching for the longest sequence of terms, which are contained in the termStore and which form a term sequence contained in the sequenceStore. As soon as a subsequent term is not included in the termStore, the lookahead process terminates and delivers the longest termSequence still contained in the termStore together with its length (see Fig. 3). annotate (text, lc, ld): /* lc (lemmaCache), ld (lookupDictionary) */ tt := tokenize(text); s := p := 0; at := ’’; for token in tt: p := p + 1 if s > 1: s := s – 1 continue elseif not lc.lemmatize(token) in ld.termStore: at := at + ’ ’ + token else: (phrase,l) := lookahead(tt[p+1:],[term],1) if not phrase == []:
URI = ld.get(phrase) if not URI == ’’: a := a.append((p,phrase,URI)) at := at + ’ ’ + wrapHTML(phrase) continue at := at + ’ ’ + token return (a,at) a := []
The lookahead identifies gewählte Abgeordnete des deutschen as a sequence of terms each included in the termStore. As soon as it determines that Fischzüchterverbandes is not included in the termStore, it will resort to the longest-term sequence found so far: gewählte Abgeordnete, since longer term sequences are not contained in the sequence store.
The longest-term sequences found are used to derive from the sequenceStore the corresponding annotation values, the length of the identified term sequences are used to skip the next n tokens in the scan pipeline.
Eventually, this process recognizes for the example text the annotations of the term sequences gewählte Abgeordnete and gewählte Abgeordnete des deutschen Bundestags. lookahead(tt, fp ,n): if tt == []:
return (n,fp) termfound := lc.lemmatize(tt[0]) in ld.termStore phraseFound := fp in ld.phraseStore if termFound or phraseFound: (ph,l) := lookahead(tt[1:],fp.append(tt[0]),n+1) if ph in ld.phraseStore:
return (ph,l) elseif phraseFound:
return (fp,n) return ([],n) In a first evaluation during the development, we compared this solution with DBpedia Spotlight on recruitment related German texts and with the BioPortal Annotator on medical texts in English using the MeSH ontology. In both cases, our system is able to identify compound terms in German as well as in English. For a first evaluation of OER’s annotations against DBpedia Spotlight, we used the Recruitment Thesaurus of Ontonym5 currently consisting of more than 16.000 concepts and more than 20.000 labels – partially multilingual. As illustration, the following text excerpt from Wikipedia leads to the annotations shown in Fig. 4: „Medizinisch-technischer Assistent (MTA) ist die Sammelbezeichnung für die vier Berufsbilder der technischen Assistenten in der Medizin und Tiermedizin im deutschen Gesundheitswesen. Sie umfasst im Einzelnen die Ausbildungsberufe:
Medizinisch-technischer Assistent – Funktionsdiagnostik (MTAF) Medizinisch-technischer Laboratoriumsassistent (MTLA oder MTA-L) Medizinisch-technischer Radiologieassistent (MTRA, MTA-R oder RTA) 5 A former spin-off (2008 - 2015) from the Freie Universität Berlin and the first author.
Veterinärmedizinisch-technischer Assistent (VMTA) Der Namensbestandteil „-assistent“ kann zur Verwechslung mit dem Beruf des medizinischen Fachangestellten (Arzthelfer) führen, der sich in Ausbildung und Tätigkeit aber deutlich unterscheidet.“
In a second evaluation, we compared OER with the BioPortal Annotator on medical
texts annotated with MeSH6 [
the BioPortal Annotator, as identified in
[
Currently OER is still in a prototype phase and has some limitations. One of these limitations is the lemmatization of German compound terms. Since such compounds usually do not appear in morphologic dictionaries, we intend to augment the lemmaCache by a simple approach for splitting compounds, lemmatizing their head term and joining the lemmatized fragments together. Another limitation is the treatment of the different notations of gender-neutral terms, which can be solved rather easily. Because of the nature of the texts and domains we like to process with the system, we deliberately ignored the question of disambiguation for the initial development.
Of course, one limitation slips in by the used knowledge models: only the terms
contained in the knowledge model, their lemma and word forms related to these lemmas
can be recognized by this approach. Therefore, the annotations will only be as good as
the knowledge models themselves. However, we do not regard this as a limitation;
instead, we consider it a feature, since it allows focusing on entities contained in the
knowledge model of a target domain [
Besides the lemmatization of compounds and the treatment of gender-neutral terms, an interesting, more experimental augmentation of the system would be the recognition of the semantic equivalence of certain noun phrases and compound nouns. Additionally the annotation process could be extended by annotating terms with categorical information and limiting the number of annotated terms based on a numerical measure of their specificity. Of course, further code optimizations and investigations of the quality of the annotations as well as of the speed of the annotation process need still to follow.
ont:Medizinisch-TechnischerAssistent ont:Medizinisch-TechnischerAssistent ont:Beruf ont:technischer_Assistent ont:Medizin ont:Tiermedizin ont:Deutsch ont:Gesundheitswesen ont:Ausbildungsberuf ont:Medizinisch-TechnischerAssistent ont:Funktionsdiagnostik ont:Medizinisch-TechnischerAssistent_fuer_Funktionsdiagonstik ont:Medizinisch-Technischer_Laboratoriumsassistent ont:Medizinisch-Technischer_Laboratoriumsassistent ont:Medizinisch-Technischer_Laboratoriumsassistent MTRA MTA-R Veterinärmedizinischtechnischer Assistent VMTA technischen Assistenten Medizin Tiermedizin dbp:Technischer_Assistent dbp:Medizin dbp:Veterinärmedizin Gesundheitswesen Medizinisch-technischer Assistent Funktionsdiagnostik dbp:Medizinisch-technischer_Assistent dbp:Medizinische_Untersuchung Medizinisch-technischer Assistent Radiologieassistent MTRA RTA dbp:Medizinisch-technischer_Assistent dbp:Radio_Television_Afghanistan 9 Name spaces of URIs are abbreviated. Terms and annotations found by either OER or DBpedia Spotlight alone are marked in green. Red marks wrong annotations and orange marks annotations, which are correct but not precise.