<!DOCTYPE article PUBLIC "-//NLM//DTD JATS (Z39.96) Journal Archiving and Interchange DTD v1.0 20120330//EN" "JATS-archivearticle1.dtd">
<article xmlns:xlink="http://www.w3.org/1999/xlink">
  <front>
    <journal-meta />
    <article-meta>
      <title-group>
        <article-title>Predicting Wikipedia Infobox Type Information using Word Embeddings on Categories</article-title>
      </title-group>
      <contrib-group>
        <contrib contrib-type="author">
          <string-name>Russa Biswas</string-name>
          <xref ref-type="aff" rid="aff0">0</xref>
          <xref ref-type="aff" rid="aff1">1</xref>
        </contrib>
        <contrib contrib-type="author">
          <string-name>Maria Koutraki</string-name>
          <xref ref-type="aff" rid="aff0">0</xref>
          <xref ref-type="aff" rid="aff1">1</xref>
        </contrib>
        <contrib contrib-type="author">
          <string-name>Harald Sack</string-name>
          <xref ref-type="aff" rid="aff0">0</xref>
          <xref ref-type="aff" rid="aff1">1</xref>
        </contrib>
        <aff id="aff0">
          <label>0</label>
          <institution>FIZ Karlsruhe</institution>
        </aff>
        <aff id="aff1">
          <label>1</label>
          <institution>Karlsruhe Institute of Technology, Institute AIFB</institution>
          ,
          <country country="DE">Germany</country>
        </aff>
        <aff id="aff2">
          <label>2</label>
          <institution>Leibniz Institute for Information Infrastructure</institution>
          ,
          <country country="DE">Germany</country>
        </aff>
      </contrib-group>
      <abstract>
        <p>Wikipedia has emerged as the largest multilingual, web based general reference work on the Internet. A huge amount of human resources have been invested in the creation and update of Wikipedia articles which are ideally complemented by so-called infobox templates de ning the type of the underlying article. It has been observed that the Wikipedia infobox type information is often incomplete and inconsistent due to various reasons. However, the Wikipedia infobox type information plays a fundamental role for the RDF type information of Wikipedia based Knowledge Graphs such as DBpedia. This stimulates the need of always having the correct and complete infobox type information. In this work, we propose an approach to predict Wikipedia infobox types by using word embeddings on categories of Wikipedia articles, and analyze the impact of using minimal information from the Wikipedia articles in the prediction process.</p>
      </abstract>
      <kwd-group>
        <kwd>Wikipedia</kwd>
        <kwd>Infobox tion</kwd>
        <kwd>Word Embeddings</kwd>
        <kwd>Text Classi ca-</kwd>
      </kwd-group>
    </article-meta>
  </front>
  <body>
    <sec id="sec-1">
      <title>-</title>
      <p>Wikipedia has become the most widely used and largest multilingual open
encyclopedia. Huge amount of human skills, expertise and e orts goes in for the
creation of Wikipedia articles. It comprises of both structured and unstructured
or free text. Structured data in Wikipedia is represented in the form of an infobox
containing property value pairs summarizing the information content of the
article. An infobox is a xed-format table usually added to consistently present a
summary of some unifying aspects that the articles share and sometimes to
improve navigation to other interrelated articles. Furthermore, infobox information
is widely used in di erent Knowledge Graphs (KGs) such as DBpedia.</p>
      <p>
        Wikipedia infobox templates are created and assigned based on the
categorical type of the article, i.e. articles belonging to a speci c genre or type should be
assigned the same template. The assignment of the infobox type to a Wikipedia
article is executed based on the discussions between the contributors and the
editors of the content of the Wikipedia article. However, no integrity tests are
conducted to determine the correctness of the infobox assignment. This leads to
the assignment of incomplete and or incorrect infobox type information which
eventually leads to erroneous RDF type information in the KGs [
        <xref ref-type="bibr" rid="ref2">2</xref>
        ].
      </p>
      <p>
        The prediction of Wikipedia infobox type can be viewed upon as a text
classi cation problem [
        <xref ref-type="bibr" rid="ref7">7</xref>
        ] in which the infobox types are nothing but the classes.
In this paper, it is intended to classify the Wikipedia articles by exploiting word
embeddings on the Wikipedia categories the articles belong to. Moreover, the
impact of using minimal information such as only the rst line of the abstract
against the whole abstract in the classi cation process has also been studied.
This work is inspired by the work done on Wikipedia infobox type prediction in
[
        <xref ref-type="bibr" rid="ref3">3</xref>
        ] and extends the approach by making use of Wikipedia categories.
2
      </p>
    </sec>
    <sec id="sec-2">
      <title>Related Work</title>
      <p>
        Wu et al.[
        <xref ref-type="bibr" rid="ref8">8</xref>
        ] proposed KYLIN, a method of automatically creating new infoboxes
and updating the existing incomplete ones, by learning a CRF extractor over
common attributes. An automated Wikipedia infobox type prediction has been
proposed by Sultana et al.[
        <xref ref-type="bibr" rid="ref6">6</xref>
        ] by training a SVM classi er over the rst
ksentences of the articles as well as categories and named entities. Bhuniyan et
al.[
        <xref ref-type="bibr" rid="ref1">1</xref>
        ] focuses on an automated NLP based infobox type prediction system.
      </p>
      <p>
        The work presented in this paper is inspired by another work on Wikipedia
infobox type prediction presented by Biswas et al.[
        <xref ref-type="bibr" rid="ref3">3</xref>
        ] in which the prediction
problem is converted to a classi cation problem, where word and graph
embeddings have been applied to generate the feature set for various classi ers.
However, in this work, our aim is to predict the Wikipedia infobox types
leveraging word embeddings on Wikipedia categories followed by a neural network
based classi cation process. The proposed method does not focus on the creation
of new infoboxes rather it helps to predict correct infobox types.
3
      </p>
    </sec>
    <sec id="sec-3">
      <title>Infobox Type Prediction</title>
      <p>In this work, the infobox type prediction problem is considered as a text
classication problem, in which infobox types are regarded as classes.</p>
      <p>Wikipedia Infobox Type Prediction
Features. Three features from the Wikipedia articles are being used for the
classi cation process.</p>
      <p>{ Table of Contents (TOC) is the collection of section headers and sub
headers of the Wikipedia articles.
{ Abstract (A) of the Wikipedia articles i.e. summary of the entire content.
{ Categories (C) is a list of Wikipedia's main categorization system,
intended to group together articles on similar subjects.</p>
      <p>
        Feature Vector. Word2Vec [
        <xref ref-type="bibr" rid="ref5">5</xref>
        ] word embeddings are applied to generate the
feature vectors. Word2Vec aims to learn the distributed representation for words
reducing high dimensional word representations while keeping linguistic contexts
of words.
      </p>
      <p>In this paper, the Google pre-trained word vectors 3 of length 300 are used to
generate word vectors for each word present in the TOC, abstract as well as for
the categories. The Google pre-trained word2vec model includes word vectors for
a vocabulary of three million words and phrases trained on roughly 100 billion
words from a Google News dataset.</p>
      <p>For each Wikipedia articlean abstract vector, a TOC vector and a category
vector are generated by performing vector addition on all the word vectors of
the abstract and normalized by the total number of words present in each of
these features. Finally, a document vector is generated by concatenating these
three vectors as shown in Figure 1.</p>
      <p>
        Classi cation. Two classi ers have been trained to predict the Wikipedia
infobox types. The aforementioned document vector is used as the feature vector
in the classi cation method using a Random Forest(RF) Classi er. For a
multilabel convolutional neural network(CNN), categories and TOC are considered
as free text and sentence classi cation [
        <xref ref-type="bibr" rid="ref4">4</xref>
        ] method has been used where each
Wikipedia article is considered as a sentence.
4
      </p>
    </sec>
    <sec id="sec-4">
      <title>Results</title>
      <p>The classi ers have been trained on the most popular 30 infobox types with 5000
articles for each type from the Wikipedia 2016 version. Features generated using
TF-IDF have been used as a baseline.</p>
      <p>The experiments established the fact that categories of Wikipedia articles
play a vital role to determine the infobox type. With CNN, categories can
predict the infobox types with a micro F1-score of 96.8% which is 0.7% better
than our previous results obtained when the prediction was based on the
entire abstract and TOC combined as shown in Table 1. Furthermore, the word
embeddings approach performs much better than the TF-IDF baseline, since
word embeddings are able to capture semantic similarities. For instance, the
Wikipedia article of the album The Wall by Pink Floyd is assigned to most of
the categories containing the word album. Furthermore, using categories only
results in higher scores than using the entire abstract. Moreover, the prediction
3 https://code.google.com/archive/p/word2vec/</p>
      <p>Feature Set</p>
      <p>With Embedding</p>
      <p>TF-IDF
RF(CV) RF(Split) CNN</p>
      <p>RF(CV) RF(Split)
results are slightly better if categories are considered together with the rst
sentence of the abstract instead of considering the whole abstract, which means
that less information is su cient to infer Wikipedia infoboxes. Furthermore, in
all the experiments word embeddings perform better than TF-IDF. However,
rather similar results are obtained when TOC and categories are combined with
the entire abstract as well as only with the rst sentence of the abstract.
5</p>
    </sec>
    <sec id="sec-5">
      <title>Conclusion</title>
      <p>In this paper, the achieved results strengthen the fact that Wikipedia categories
as well as minimal text plays a vital role in the prediction of infobox types.
In future we intend to design a semi-supervised approach to correct existing
Wikipedia infobox types and to predict infobox types for newly created articles.</p>
    </sec>
  </body>
  <back>
    <ref-list>
      <ref id="ref1">
        <mixed-citation>
          1.
          <string-name>
            <surname>Bhuiyan</surname>
            ,
            <given-names>H.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Oh</surname>
            ,
            <given-names>K.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Hong</surname>
            ,
            <given-names>M.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Jo</surname>
            ,
            <given-names>G.</given-names>
          </string-name>
          :
          <article-title>An Unsupervised Approach for Identifying the Infobox Template of Wikipedia Article</article-title>
          . In: CSE (
          <year>2015</year>
          )
        </mixed-citation>
      </ref>
      <ref id="ref2">
        <mixed-citation>
          2.
          <string-name>
            <surname>Biswas</surname>
            ,
            <given-names>R.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Koutraki</surname>
            ,
            <given-names>M.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Sack</surname>
          </string-name>
          , H.:
          <article-title>Exploiting Equivalence to Infer Type Subsumption in Linked Graphs</article-title>
          . In: European Semantic Web Conference (
          <year>2018</year>
          )
        </mixed-citation>
      </ref>
      <ref id="ref3">
        <mixed-citation>
          3.
          <string-name>
            <surname>Biswas</surname>
            ,
            <given-names>R.</given-names>
          </string-name>
          , Turker, R.,
          <string-name>
            <surname>Moghaddam</surname>
            ,
            <given-names>F.B.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Koutraki</surname>
            ,
            <given-names>M.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Sack</surname>
          </string-name>
          , H.:
          <article-title>Wikipedia infobox type prediction using embeddings</article-title>
          .
          <source>In: DL4KGS@ ESWC</source>
          (
          <year>2018</year>
          )
        </mixed-citation>
      </ref>
      <ref id="ref4">
        <mixed-citation>
          4.
          <string-name>
            <surname>Kim</surname>
            ,
            <given-names>Y.</given-names>
          </string-name>
          :
          <article-title>Convolutional Neural Networks for Sentence Classi cation</article-title>
          .
          <source>In: EMNLP</source>
          (
          <year>2014</year>
          )
        </mixed-citation>
      </ref>
      <ref id="ref5">
        <mixed-citation>
          5.
          <string-name>
            <surname>Mikolov</surname>
            ,
            <given-names>T.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Chen</surname>
            ,
            <given-names>K.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Corrado</surname>
            ,
            <given-names>G.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Dean</surname>
            ,
            <given-names>J.:</given-names>
          </string-name>
          <article-title>E cient Estimation of Word Representations in Vector Space</article-title>
          .
          <source>CoRR</source>
          (
          <year>2013</year>
          )
        </mixed-citation>
      </ref>
      <ref id="ref6">
        <mixed-citation>
          6.
          <string-name>
            <surname>Sultana</surname>
            ,
            <given-names>A.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Hasan</surname>
            ,
            <given-names>Q.M.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Biswas</surname>
            ,
            <given-names>A.K.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Das</surname>
            ,
            <given-names>S.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Rahman</surname>
            ,
            <given-names>H.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Ding</surname>
            ,
            <given-names>C.H.Q.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Li</surname>
            ,
            <given-names>C.</given-names>
          </string-name>
          :
          <article-title>Infobox Suggestion for Wikipedia Entities</article-title>
          . In: CIKM (
          <year>2012</year>
          )
        </mixed-citation>
      </ref>
      <ref id="ref7">
        <mixed-citation>
          7. Turker, R., Zhang,
          <string-name>
            <given-names>L.</given-names>
            ,
            <surname>Koutraki</surname>
          </string-name>
          ,
          <string-name>
            <given-names>M.</given-names>
            ,
            <surname>Sack</surname>
          </string-name>
          , H.:
          <article-title>"the less is more" for text classi cation,</article-title>
          .
          <source>In: SEMANTiCS</source>
          <year>2018</year>
          (
          <year>2018</year>
          )
        </mixed-citation>
      </ref>
      <ref id="ref8">
        <mixed-citation>
          8.
          <string-name>
            <surname>Wu</surname>
            ,
            <given-names>F.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Weld</surname>
            ,
            <given-names>D.S.</given-names>
          </string-name>
          :
          <article-title>Autonomously Semantifying Wikipedia</article-title>
          . In: CIKM (
          <year>2007</year>
          )
        </mixed-citation>
      </ref>
    </ref-list>
  </back>
</article>