TECNE: Knowledge Based Text Classi cation Using Network Embeddings

Abstract

Text classi cation is an important and challenging task due to its application in various domains such as document organization and news ltering. Several supervised learning approaches have been proposed for text classi cation. However, most of them require a signi cant amount of training data. Manually labeling such data can be very time-consuming and costly. To overcome the problem of labeled data, we demonstrate TECNE, a knowledge-based text classi cation method using network embeddings. The proposed system does not require any labeled training data to classify an arbitrary text. Instead, it relies on the semantic similarity between entities appearing in a given text and a set of prede ned categories to determine a category which the given document belongs to.

Introduction

Text classi cation is gaining more and more attention due to the availability of a huge number of text data, which includes search snippets, news data as well as text data generated in social networks. Recently, several supervised approaches have been proposed for text classi cation [ 6,1 ]. However, they all require a signi cant amount of labeled training data. Manual labeling of such data can be a very time-consuming and costly task. Especially, if the text to be labeled is of a speci c scienti c or technical domain, crowd-sourcing based labeling approaches do not work successfully and only expensive domain experts are able to ful ll the manual labeling task. Alternatively, semi-supervised text classi cation approaches [ 5 ] have been proposed to reduce the labeling e ort. Yet, due to the diversity of the documents in many applications, generating small training set for the semi-supervised approaches still remains an expensive process [ 2 ]. Moreover, to cope with the problem of labeled data several dataless text classi cation methods have been proposed. Similar to our proposed approach, the methods do not require any labeled data, rather they rely on the semantic similarity between documents and the prede ned categories. However, the most prominent and successful dataless classi cation approaches cannot utilize the rich entity and category information in large-scale knowledge bases.

In this paper we demonstrate TECNE, an approach which classi es an arbitrary input text, according to a prede ne set of categories, without requiring any training data. The approach is able to capture the semantic relation between the entities represented in a text and the prede ned categories by embedding them into a common vector space using state-of-the-art network embedding techniques. Finally, the category of the given text can be derived based on the semantic similarity between entities (present in the given text) and a set of prede ned categories. The similarity is computed based on the vector representation of the entities and the categories. 2

Description of TECNE

Given a Knowledge Base KB, containing a set of entities E = fe1; e2; ::; eng and a set of hierarchically related categories C = fc1; c2; ::; cmg, where each entity ei 2 E is associated with a set of categories C0 C via a relation cat E C, such that cat(ei) = C0. The input of the system is an arbitrary text t, which contains a set of mentions Mt = fm1; : : : ; mkg that uniquely refer to a set of entities as well as a set of prede ned categories C0 C (from the underlying knowledge base KB). The output of TECNE is a score value for each category ci 2 C0 based on the semantic similarity between the given text t and the prede ned categories C0.

TECNE Overview The general work ow of TECNE presented in Figure 1 is similar to our previous study [ 4 ].

The input is a text t of an arbitrary length. Then, the classi cation task start with the detection of each entity mention present in t based on a prefabricated \Anchor-Text Dictionary" from Wikipedia. The Anchor-Text Dictionary contains all mentions and their corresponding Wikipedia entities. In our example the detected mentions are \IBM", \midrange computer" and \eServer".

As a next step, for each detected entity mention in t, the candidate entities are generated with the help of the Anchor-Text Dictionary. In our example TECNE: Knowledge Based Text Classi cation Using Network Embeddings these are \IBM", \Midrange computer" and \IBM eServer". Also, the prede ned categories (Sports, Technology, Culture, World) are mapped to Wikipedia categories. Finally, based on the entity and category embeddings [ 3 ] that have been precomputed from Wikipedia, the output of TECNE is a score for each prede ned category. Ideally, the most semantically related category to the entities present in the input text should have the highest score. Thereby, in the given example the category Technology has the highest score. More technical details about the approach and the evaluation of the system can be found in [ 4 ]. 3

Demonstration

A recorded video of our demonstration can be found here: https://goo.gl/pSxkcy TECNE is implemented in Java using a client-server architecture with communication over HTTP. The server is a RESTful web service, which is implemented using Spark1. Moreover, the client user interface is achieved by using Vaadin Framework2 as a Web Application. The system supports both service-oriented and user-oriented interfaces for classifying short/long text documents. The system accepts any arbitrary text that needs to be classi ed as an input. In the interest of convince, the system utilizes three di erent APIs that a user can use to provide an online text as an input to the system. The rst API3 is used to fetch an abstract of Wikipedia articles. Simply, a user can enter the name of the Wikipedia article and the abstract of the certain article would be fetched automatically. Figure 2 presents the screen shot of this service, where the input is the abstract of the Albert Einstein's Wikipedia page. The second API4 and the third API5 are used to retrieve long and short random news respectively from di erent web pages. Besides that, a user can select a prede ned sample sentence as an input or also manually enter any text without using the already provided data sources.

For the sake of simplicity, the system covers 4 di erent categories, i.e. the system can classify a text based on 4 di erent categories, Sports, Business, World, and Science-Technology. However, it can be easily extended to support higher number of categories for the classi cation purpose. For classifying a text, TECNE proceeds in 3 main steps as following: 1 http://sparkjava.com 2 https://vaadin.com/ 3 https://en.wikipedia.org/w/api.php 4 https://webhose.io/ 5 https://newsapi.org

Conclusion and Future Work

In this paper, we demonstrate TECNE, a system for knowledge based text classi cation using network embeddings. Future works also include the extension of TECNE towards enabling user to de ne a category list where the input text will be categorized accordingly.

References

1. Biswas , R. , Turker, R., Moghaddam , F.B. , Koutraki , M. , Sack , H.: Wikipedia infobox type prediction using embeddings . In: DL4KGS@ESWC ( 2018 )

2. Li , C. , Xing , J. , Sun , A. , Ma , Z. : E ective document labeling with very few seed words: A topic model approach . In: CIKM. pp. 85 { 94 . ACM ( 2016 )

3. Tang , J. , Qu , M. , Wang , M. , Zhang , M. , Yan , J. , Mei , Q. : Line: Large-scale information network embedding . CoRR ( 2015 )

4. Turker, R., Zhang, L. , Koutraki , M. , Sack , H.: \the less is more" for text classi cation . SEMANTICS ( 2018 )

5. Xuan , J. , Jiang , H. , Ren , Z. , Yan , J. , Luo , Z. : Automatic bug triage using semisupervised text classi cation . CoRR ( 2017 )

6. Zhang , X. , Zhao , J.J. , LeCun, Y.: Character-level convolutional networks for text classi cation . In: NIPS ( 2015 )

6 https://en.wikipedia.org/wiki/Albert Einstein