The goal of the research is to make first steps for automated anaphora and co-reference resolution in Lithuanian language with respect to limited preprocessing tools and resources, by combining concepts and algorithms from different text analysis phases for this purpose. Existing resolution methods are created for major languages, e.g., English, and usually are language-specific. On the base of analysis of existing methods, a taxonomy of anaphoric objects is created and initial algorithms are proposed for solving anaphoras and coreferences in Lithuanian language.
While amounts of available information are rapidly increasing, research in Natural Language Processing (NLP) field is becoming more and more important. Unfortunately, the most of the NLP work has focused on English and other major languages leaving this field underdeveloped for smaller languages. Due to this situation, the Lithuanian language lacks mature NLP tools and resources while some parts of NLP process have not been researched at all. Anaphora and co-reference resolution is one of such cases for Lithuanian language.
In NLP, the anaphora is an expression interpretation of which depends on another
expression in context [
The relationship between “He” and “Tom” is called an anaphora. In this case, “He” is an anaphoric object that refers to its antecedent “Tom”. Without being able to solve anaphoric expressions, we would not know why Tom skipped the school nor who was Copyright © 2015 by the authors. Copying permitted for private and academic purposes. This volume is published and copyrighted by its editors. sick. This information is very important when we try to extract semantic information from various texts.
Terms “anaphora” and “co-reference” are often used together or mistaken for each
other. Sometimes, anaphoric and co-referential relations can coincide, but it is not
always true. The co-reference is the equivalence relation between two text items
(words or phrases, having the same meaning) [
The wider problem with anaphora and co-reference resolution is in the fact that even for major languages this process remains semi-automated what is entirely unacceptable to desirable analysis of the existing textual information. This is caused by imperfection of pre-processing methods and tools, needed for preparing texts for anaphora and co-reference resolution, and the lack of reliable resources, e.g., annotated corpora for resolution algorithms, based on machine learning, etc.
Research questions. This research is devoted for making the first steps in filling the gap in anaphora and co-reference resolution in Lithuanian language. It raises the following research questions: 1. Can existing anaphora and co-reference resolution methods, designed for other languages, be adapted to Lithuanian language? Can quality assessments of these algorithms be comparable with assessments of those created for major languages? 2. What automated methods and algorithms can be developed with current availability of pre-processing tools and resources in Lithuanian language? 3. How semantic information can be increased with additional (not limited to anaphora) co-reference resolution?
The research methodology is based on the Design Science Research and
Information System Research Framework defined by Hevner et al. (2004) [
The rest of the paper is structured as follows. Section 2 overviews the related works. Section 3 presents the main idea of this research and provides some initial results that have been achieved. Section 4 draws conclusions and presents future works. 2
This section provides analysis of various anaphora resolution methods that were analysed in this dissertation.
Syntax based approaches. One of the earliest anaphora resolution methods was
proposed by Hobbs in 1977 [
Centring theory (CT). Centres link one utterance with other utterances in
discourse. Each utterance has one backward-looking centre and a number of possible
forward-looking centres that a particular utterance has evoked. Forward-looking
centres are ranked by discourse salience and grammatical rules; the highest rated centre is
called the preferred centre [
Salience factors. While salience plays a role in most of the approaches, usually it
is not considered as the main criteria for anaphora resolution. Notable exception is
RAP (Resolution of Anaphora Procedure) algorithm introduced by Lappin and Leass
in 1994 [
resolution strategies based on UNL were proposed for Tamil language [
approach is targeted at NLP purposes in general but can also be used for rather
simplistic anaphora resolution [
Statistical methods. One of the earliest statistical approaches was proposed by Ge
et al. in 1998 [
Machine learning. First learning system to achieve comparable results with other
approaches was presented by Soon et al. [
Comparison of approaches. The comparison of analysed resolution methods is presented in Table 1. The precision that was reported in the original research is only given. Recall is not given since some of the methods did not provide its evaluation. The evaluations were not done against the same corpora; therefore, their results are meant to give a general idea of the state of anaphora resolution.
The goal of the research is to make first steps for automated anaphora and coreference resolution in Lithuanian language with respect to limited pre-processing tools and resources, by combining concepts and algorithms from different text analysis phases for this purpose. In order to reach the goal, the following tasks were stated: 1. Analyse current methods and resources used for anaphora resolution in English and other major languages; 2. Develop rules and algorithms for anaphora and co-reference resolution in Lithuanian language; 3. Implement rules and algorithms for anaphora and co-reference resolution suitable to improve semantic analysis and search in Lithuanian text corpora; 4. Conduct experiment for evaluating suitability of created rules and algorithms; 5. Evaluate developed method with recall and precision measures as main criteria.
The principal schema of the approach is presented in Fig. 1. Currently, anaphora resolution algorithms can be based on morphological annotations and entities, recognized by Named Entity Recognition (NER) algorithms, whereas existing syntactic annotation tools for Lithuanian language have not reached the sufficient quality yet.
Co-reference resolution algorithms can be applied after semantic annotation. There are more possibilities for discovering co-references, but they also are based on existence of pre-processing methods, such as, e.g., Semantic Role Labeling, so currently they are beyond the scope of this research. 3.2
This research combines multiple approaches to anaphora taxonomy by extending the main morphology-based taxonomy with additional generalization sets for providing the better coverage on the anaphora phenomenon [14]. The distinction between categories of lexical semantics and domain semantics allows identifying anaphoric expressions from multiple viewpoints.
The created taxonomy reflects the actual situation that the same anaphoric object may be classified as a pronoun (morphological type), agent (lexical semantics type) and person (domain semantics type). Some part of anaphoric relations may be detected using morphological annotations; additional relations can be found from results of lexical semantic analysis, and yet another part can be discovered from the domain semantics represented in ontology. The generic domain semantics categories, characteristic for various domains, are extended with state, domain role and abstract object, which are important for anaphora resolution. The “abstract object” represents such words or phrases as “person”, “enterprise”, “young man”, etc., that can have anaphoric references. Similarly, domain roles as “president”, “teacher”, “politician”, etc., can be used for discovering anaphoric relations. Morphological classification is language specific, but lexical and domain semantic classification can be used for other languages too. 3.3
This section presents the proposed anaphora resolution algorithm (Fig. 2), which was created for Semantic Search Framework for Lithuanian Language. The algorithm was investigated on a corpus that collects articles from various Lithuanian Internet news sites focusing on political and economic matters.
Proposed resolution method focuses on the cases where anaphoric objects are personal pronouns (subtypes of main pronouns who in turn are subtypes of pronouns in morphological categorization) and used to express persons (subtypes of domain agents in domain semantics categorization). The algorithm searches for the valid pronoun for which anaphora resolution was not performed yet, and checks it against the pre-set list of invalid pronouns that usually are either pleonastic or tend not to refer to persons. If the pronoun is valid, we go backwards until we find a noun that is recognized as a person by NER. If a suitable noun is not found, we move backwards to the next sentence and perform the same search until we either find a suitable noun, or until we pass X sentences backwards from the pronoun; then we move forward Y sentences from the pronoun searching for a suitable noun.
If we find a suitable noun then we determine if it agrees in number and gender with the pronoun. If noun and pronoun agree in number and gender then their pair is added to anaphora annotations and we return to the first step.
The algorithm can be considered naive since it takes the first suitable noun that agrees in a number and gender as an antecedent (or postcedent), and the alternatives are not considered. The evaluation of the algorithm was done against corpora of 500 Internet news portal articles focusing on politics and economics. Algorithm managed to achieve 61% recall and 74% precision. 3.4
Co-reference relation means relation between equivalent objects. In the proposed approach, equivalent objects are identified after semantic annotation (Fig. 2), during which named entities, having the same meaning but, possibly, the different representation form, are marked as different individuals.
Currently developed intertextual co-reference resolution algorithm merges such individuals into a single entity. Here, “Valid types of nouns” is a list of nouns that algorithm can resolve, e.g., persons, locations, organizations; “Resolution rules” is a list of rules that are valid for specific type (or several types) of nouns. The algorithm was tested for entities, having various modifications of their names, e.g., John Smith, J. Smith, J. S., Mr. Smith, John Smith’as, John Smithas, etc., co-refering to the same entity. The experiment was conducted with 277784 articles having 3058015 individuals. After merging, the number of individuals has decreased till 77532 (i.e., about 39 times). Unfortunately, due to the early stage of development we currently cannot provide the evaluation of precision and recall of the proposed algorithm.
The paper presents the ideas and initial results after 2 years of research. The contribution of this research is the created taxonomy of anaphoric objects and algorithms for automated anaphora and co-reference resolution in Lithuanian language. Its uniqueness is in the fact that anaphoric relations and co-references are identified from multiple viewpoints via analysing categories of both lexical semantics and domain semantics. Anaphora and co-reference resolution algorithms are combined from different stages of the text pre-processing process. The research is done in the very early stage of coping with anaphora and co-reference resolution problem in Lithuanian language, with respect to imperfect pre-processing algorithms and limited resources. Therefore, the analyzed methods for other languages could not be adapted. However, the assessments of our algorithms are comparable with assessments of those created for major languages.
The future work is directed towards creating more sophisticated anaphora and coreference resolution algorithms using emerging tools and resources for Lithuanian language that are being developed simultaneously. In Proceedings of the 40th Annual Meeting of the Association for Computational Linguistics (ACL), pp. 104–111, Philadelphia, USA (July 2002) 14. Zitkus, V., Nemuraite, L.: Taxonomy of anaphoric expressions as a starting point for anaphora resolution in Lithuanian corpus. Informacinės technologijos (IVUS 2014), Kaunas, Technologija. pp. 177–182 (2014).