This paper describes the application of named entity recognition and coreference resolution algorithms in the oil industry. Oil industry researchers and businesses generate large amounts of content every day. Managing them correctly is very important to get the most use of each article and document. Named entity recognition algorithms can automatically scan entire articles and reveal the most significant people, organizations, and places discussed in them, while coreference resolution combines each entity mention into clusters of mentions. Each cluster represents one entity across one document. These methods allow to simplify the analysis of large numbers of documents and articles for researchers, managers, engineers, etc.
Named entity recognition is a process where an algorithm takes a string of text (sentence or paragraph) as input and identifies relevant nouns (people, places, and organizations) that are mentioned in that string.
In practice, texts often have the same entities mentioned in various ways (anaphora, cataphora, split antecedents, coreferring noun phrases). Coreference resolution algorithms are used to address this problem and to combine all mentions of the same entity into one cluster.
These algorithms may greatly simplify the analysis of documents and articles. For example: They can be used to create efficient search engines. If for every search query the algorithm ends up searching all the words in millions of articles, the process will take a lot of time. Instead, if named entity recognition can be run once on all the articles and the relevant entities (tags) associated with each of those articles are stored separately, this could speed up the search process considerably. With this approach, a search term will be matched with only a small list of entities discussed in each article, leading to faster search execution. They can be used to improve content recommendation systems. This can be done by extracting entities from a document and recommending other documents that have the most similar entities mentioned in them. An online journal or publication site can hold millions of research papers and scholarly articles.
There can be hundreds of papers on a single topic with slight modifications. Information search can become complicated. Segregating articles by tags extracted using named entity recognition and coreference resolution can help find the desired article or document. They can be used to create ontology objects and object properties. They can be used to classify content for news providers: such algorithm can scan entire articles and reveal the most significant people, organizations, and locations discussed in them. There are several ways to make the process of customer feedback handling smooth by means of solving named entity recognition tasks. They can be used for automatic summarization systems: named entities are the important information of the text and increase the performance of identification of text segments that are further included in summarized data. This is especially important for the oil industry for two reasons: new technologies, cited in scientific papers, can save millions of dollars daily after implementation; thousands of documents are generated in every oil company every day, these documents often require meticulous analysis.
There are two main approaches to address the named entity recognition (NER) problem [
In this paper, we use a hybrid approach to this task in the Russian language: • An algorithm based on context-free grammar is used to extract some of the document’s entities, keywords, and attributes.
• Another algorithm based on conditional random fields and word vectorization using a pre-trained skip-gram word2vec model for the Russian language and POS tags.
The concept of conditional random fields (CRFs) [
We used a hybrid approach to extract entities from texts: extracted entities were merged together, removing the duplicating ones. Entities of the following types were extracted: • person; • organization; • location; • product; • event; • money.
Named entity extraction algorithms used morphological and part-of-speech tags to correctly label entities.
To train and validate models, we used the Dialogue-2016 dataset and additionally labeled documents (newspapers, fiction books, technical documents).
The coreference resolution algorithm is based on neural network, which is mostly derived
from previous work [
• Pre-trained Russian word2vec skip-gram vectors, morphology, and POS tags were used as features.
• An extracted named entity tag was added as a feature to help the network find coreferences between the entities extracted by the NER algorithm.
Named entity recognition (Table 1) and coreference resolution (Table 2) modules were tested on a holdout subsample of the dataset (randomly selected 10% of data). Metrics for entities were calculated for every word separately. Classes of entities were unbalanced: most of the words in the texts were not parts of entities
B-PER I-PER B-ORG I-ORG B-LOC I-LOC
71.7
Entity recognition and coreference resolution models were combined in a single pipeline, which also included document OCR, text preprocessing, and tokenizing. A web service was created, which included a search system based on the Elasticsearch framework. The extracted entities were used in ranging the search output. The system was loaded with oil industry-related documents: scientific articles and business documents. The documents in the search could be viewed with highlighted entities and coreferences. The agglomerative clustering method (using Doc2Vec model for feature extraction) and a simple named entity linking algorithm based on regular expressions were used to recommend similar documents to help the user quickly find relevant documents that are like the current document.
Examples of our web service GUI and processed text can be seen in the Figures 1 and 2 below:
Altogether, this system considerably improves information search efficiency and document analysis.
This article describes application of machine learning algorithms for natural language processing tasks. Named Entity Recognition and Coreference Resolution allows to improve search engines and helps to analyze documents faster. Future work includes optimization of algorithms and addition of summary extraction, Named Entity Linking, recommendation engine based on documents features. All of these features are aimed to optimize the process of text documents analysis.