Anaphora resolution for social media texts is essential yet dificult task for text understanding. An important characteristic of anaphora is that it creates a connection between the antecedent and the anaphor buried in the anaphoric sentence. This paper outlines the methods used to locate anaphora and their antecedents in a particular text. The text is a social media tweet for the SocAnaRes-IL 2022 challenge that was part of FIRE 2022. The proposed model uses a Neural Co-reference Network for the anaphora resolution.
developed model is able to provide the correct resolution for 73 anaphors by attempting on 285 on anaphors.
The tasks of co-reference resolution [
The pairwise model has been used in the majority of earlier work on bridge anaphora
resolution, which make the assumption that the gold mention information is available [
The training dataset provided by FIRE 2022 [
This tab-delimited value file format was unsuitable for NLP tasks like entity recognition and vectorization. Since it was generated from tweet data it also contained unusable information like tweet id and in some cases had not been cleaned. There were rows in the documents that contained entire sentences with words separated by ’\n’ tokens. The data had to be pre-processed ifrst into a more usable format and the structure of the table had to be disrupted to accommodate the denoising and new tokenization. We generated a tokenized list of each tweet with new indices due to the removal of unnecessary tokens and addition of word tokens in the samples where tokenization was incorrectly done. This made the data more suited for our proposed model.
We utilised a statistical model pretrained on the English language as part of the NeuralCoref
network by huggingface. The first step of the process is to extract words or phrases that refer
to real entities. These include proper nouns like names and objects or possessives like ’My
brother’. The model then trains a set of features for each of these entities or ’mentions’. This is
done by taking an initial set of word embeddings and training them on the OntoNotes Corpus, a
large manually-annotated corpus used for coreference resolution tasks. This method of learning
features helps with coreference resolution tasks by segregating word vectors along attributes
like gender, which is helpful to the model when identifying antecedents for people’s pronouns.
However, it does not do a great job with coreference involving things and pronouns such as ’it’
or demonstrative pronouns like ’this’ and ’that’. To address these issues, the feature vector will
have to take into account contextual information surrounding the entity word or phrase. This
contextual information is added by averaging the feature vectors of words present around the
entity word and adding some other features based on phrase length, word location, speaker
information, etc. Once these vectors are ready the model train two separate neural networks to
identify antecedents. The first is a classification network that scores each ’mention’ to classify
whether it is the first instance of an entity or whether it has an antecedent. The second neural
networks generates a score for a ’mention’ and every possible antecedent in the text to identify
the most likely pair. The scores generated by both networks are non-probabilistic max-margin
objective scores. Once both these models are trained, the representations for each mention are
plugged into them and based on their scores the most probably antecedent is determined. Figure
1 shows the mechanisum for scoring the Co-references using neural network architecture [
The Result from the Table 1 can be improved by applying several pre-processing procedures, such as removing unnecessary number strings and precise tokenization for ’\n’ recognition in tweets.
In our three-step model, the first step of entity detection has minimal scope for improvement. However, the feature generation step could be further built upon. Neuralcoref uses contextual information surrounding the entity word to improve its representation vector. This includes averaging the vectors for the surrounding words as well as taking integer representations of factors like speaker and location. This can be further improved upon by using diferent methods to add context to the feature representation. With a larger dataset, the third step of classification neural networks could also be further improved by adding layers of complexity. With the current dataset size, using large and complex language models doesn’t yield results but that is liable to change as the number of datapoints increase.
The current model is operating on a small dataset with requirement of several pre-processing steps like removal of unnecessary numerical strings and accurate tokenization due for detection of ’\n’ in the tweets. A thorough pre-processing of the dataset will automatically lead to a jump in accuracy of the model.