This paper describes a named entity discovery and linking system, which compete the CCKS2017 question named entity discovery and linking task. We are facing challenges including short-text, small training samples and open domain, making the existing solutions unfeasible. In this paper, we propose a CRF + rules method to recognize the corresponding named entity, which employs several features, such as bag-of-word features, POS features, parsing features etc. As for entity linking, context information, popularity, word embeddings, and online public corpus are used. The experiment results show that, the F1 score of named entity discovery is 0.815, while the accuracy of the entity linking is 0.736. The overall F1 score is 0.600, which proves the effectiveness of our system.
With the development of Internet techniques, unstructured text has become one of the
most popular information carriers. As the basis of text analysis, Named Entity
Discovery (NED) and Entity Linking (EL) are widely studied. However, the
traditional NED techniques can only be applied to the situation with very few entity
types (e.g. persons, locations, organizations etc.), and the existing EL methods need
rich context information. In this task, we mainly encounter three difficulties. Firstly,
the boundary of Named Entity (NE) definition is fuzzy. For example, “苹果手机”
(Apple iPhone) is not a named entity, but “苹果” (Apple) is a named entity as a
company name; The second challenge is that the questions in training set are
short-texts with lots of noises; The last challenge is that the corpus is from open
domain and entity types are relatively much more.
Our work can be divided into two parts: NED and EL. The existing approaches of NED
are mainly based on statistical models, e.g. HMM (Hidden Markov Model), CRF
(Conditional Random Field) and DNN (Deep Neural Network) etc[
However, the statistical models of NED principally focus on very few entity types, it is unsuitable in open domain. As for EL, supervised approach is difficult to apply, with short-text and limited information about each sense of ambiguous named entity.
Our strategy is CRF + rules in NED. Firstly, bag-of-word features and POS features
from the training corpus are extracted with the pyltp tool[
The filter rules are divided into the following categories (more than 180 cases are included, while the size of training set is 1395): a) Version number filtering: the product number should be included as part of the named entities, while the version number should not. b) Suffix filtering: some suffixes affecting NED must be filtered. For instance, for the phrase "戴尔笔记本" (Dell Laptop), the real NE is "戴尔" (Dell), while "笔 记本" ( Laptop) is a generic entity cannot be included. c) Verb-prefix filtering: in the prediction results, there are very few named entities which has structure of "verb + noun", in which case the verb prefix needs to be filtered out.
Among the predicted named entities, if conjunctions exist, such as "and" etc., the results need to be split into more parts.
The CRF model-based result can be revised by re-identify rules. In the beginning, a named entity dictionary in the training set is constructed. By using the dictionary, we re-identify a new result based on all-matching rule. If the re-identified result does not overlap with the model-based result, it can be added to the final result set. In the case of overlap, when the model-based result is contained in re-identified result, the re-identified result should be added to the final result set. In other cases, the result is based merely on the model-based result. Just as one example, assuming the model-based result is “百度知道” (Baidu Zhidao), while the re-identified result is “百 度知道企业平台” (Baidu Zhidao Enterprise Platform), the final result can be revised as “百度知道企业平台” (Baidu Zhidao Enterprise Platform).
Several useful features can be used for EL, such as entity context information in original sentence, popularity of each entity’s sense, word co-occurrence in Baidu Zhidao1 for each entity’s sense etc.
In this task, information extension is necessary for the reason that each ambiguous named entity has several senses with less information. We collect each sense’s relative questions as candidate question sets by searching Baidu Zhidao and picking out the questions in top 5 pages. For example, a sense from knowledgeworks2 calling “小米（小米公司）” (Xiaomi Inc.) is extended to “小米公司的经营理念是什么?” (What is the business philosophy of Xiaomi Inc.) as one candidate question by searching Badidu Zhidao.
Firstly, all questions are segmented. Then stop words are removed. At last, the similarity of original question with each candidate question is calculated by using Jaccard Similarity, and those candidate questions with low similarity are removed.
The original and candidate questions need to be represented as vectors. In a question,
each word’s vector representation is gained by pre-trained word2vec[
v = N1 ∙∑1N wi (1)
The context similarity score of an entity’s sense can be calculated by the average cosine similarity of the original question with each one in the candidate set of this sense. Let o be the original question’s sentence vector, m be one of entity’s sense, V = {vi} be the sentence vectors in candidate set of the sense whose size is K. The context similarity score of o and mis shown as formula (2).
scorecontext(o, m) = 1 ∙ ∑lK=1 similaritcyosine(vo, vl)
K (2)
We collect every sense’s visits of a named entity from Baidu Baike3 as one index of popularity. But, some of entity’s senses in knowledgeworks are different from those in Baidu Baike, in which case edit distance is needed. For those entity’s senses never show up in Baidu Baike, we use the minimum score from other entity’s senses to ensure score’s smoothness. Let o be the original question, M be the entity’s senses 1 https://zhidao.baidu.com/ 2 http://knowledgeworks.cn:30001/ 3 https://baike.baidu.com/ in knowledgeworks, m ∈ M be one of entity’s sense, k ∈ M, k ≠ m be another entity’s sense different from m, S be the size of senses in Baidu Baike, “editDist” be the edit distance of m and the processing sense in Baidu Baike. The score for an entity’s senseis shown as formula (3). scorevisit(o, m) = max((1 ∙ ∑iS=1 visitsi ∙ editDist) , (
S
min scorevisit(o, k))) (3) k∈M,k≠m
For each entity, knowledgeworks provides a frequently-used sense calling primary sense. Let t be 1 or 0 presenting whether m is primary, g be the weight when m is not primary for score’s smoothness. The primary score is shown as formula (4).
scoreprimary(o, m) = g1−t
After processing irrelevant information filtration and Chinese segmentation for extended questions, word co-occurrence frequencies are calculated by counting each word shown up together in the two extension sets of original questions and candidate ones in the entity’s senses, and then normalized. Let o be the original question, m be one of entity’s sense whose size is N, counti, i ∈ N be the word co-occurrence of arbitrarily one of the entity’s senses. The score is shown as formula (5). (4) (5) (6) scoreco−occ(o, m) = ∑c1Nocuonutnit , i ∈ N
According to the four scores described above, the weighting method is shown as formula (6).
scorefinal(o, m) = α ∙ scorecontext + β ∙ scorevisits + γ ∙ scoreprimary + μ ∙ scoreco−occ
s. t. α + β + γ + μ = 1
The following Table 1 lists EL parameters for formulas described above. All parameters in Table 1 are adjusted by repeated testing. The editDist means the edit distance of senses. the g presents the weight of non-primary parameter. As for the combination parameters, the sentence context similarity α contributes most to EL; the visit β and primary γ presenting popularity are assigned the same value; the word co-occurrence μ also act as an important role for EL.
Method CRF CRF+Rule1 CRF+Rule1+Rule2 CRF+Rule1+Rule2+Rule3
The Table 3 shows EL results.
In this experiment, we use the dataset provided by CCKS2017 Task 1. Besides we
crawl tens of millions of questions from Baidu Zhidao and train a word2vec[
The following Table 2 lists NED results. According to the result, the best score of NED in training set is based on CRF with rules, which F1 is 0.815. The top precision of EL comes from the combination of four features, which is 0.736. So as to the overall F1 is 0.600.
In this paper, we propose some strategies for NED and EL to deal with open domain
short-text issues. Experiments show that our method has effective performance. In the
future, we are trying to apply NED by using Heuristic and DNN[
This work was supported in the Research on People's Heterogeneous Information Aggregation Technology, and Research and Development of Intelligent Question Answering System Based on Text Automatic Abstract Technology, and the Research on Key Techniques of Knowledge Map in Intelligent Home.