=Paper=
{{Paper
|id=Vol-2319/ecom18DC_paper_3
|storemode=property
|title=Encoder-Decoder Neural Networks for Taxonomy Classification
|pdfUrl=https://ceur-ws.org/Vol-2319/ecom18DC_paper_3.pdf
|volume=Vol-2319
|authors=Makoto Hiramatsu,Kei Wakabayashi
|dblpUrl=https://dblp.org/rec/conf/sigir/HiramatsuW18
}}
==Encoder-Decoder Neural Networks for Taxonomy Classification==
https://ceur-ws.org/Vol-2319/ecom18DC_paper_3.pdf
Encoder-Decoder neural networks for taxonomy classification
Makoto Hiramatsu Kei Wakabayashi
Graduate School of Library, Information and Media Faculty of Library, Information and Media Science,
Studies, University of Tsukuba University of Tsukuba
Tsukuba, Ibaraki Tsukuba, Ibaraki
himkt@klis.tsukuba.ac.jp kwakaba@slis.tsukuba.ac.jp
ABSTRACT as mentioned earlier, the number of category paths is 3,695, which
This paper describes our taxonomy classifier for SIGIR eCom Rakuten is fairly large to be considered as a set of classes for ordinal machine
Data Challenge. We propose a taxonomy classifier based on sequence- learning classifier. Moreover, this approach independently treats
to-sequence neural networks, which are widely used in machine these category paths although a category path shares a part of an-
translation and automatic document summarization, by treating other category path of a similar product. It is expected that this fact
taxonomy classification as the translation problem from a descrip- causes more data sparseness issue and degrades the performance
tion of a product to a category path. Experiments show that our because the classifier has no way to find common patterns that are
method can predict category paths more accurately than baseline shared in two different category paths.
classifier. In this paper, we propose a taxonomy classifier based on Encoder-
Decoder neural networks. The key idea is to regard the category
CCS CONCEPTS path as a series of category names in each hierarchical level. From
this perspective, the taxonomy classification task can be converted
• Computing methodologies → Information extraction;
into a sequence-to-sequence problem, which has a text (i.e., a se-
KEYWORDS quence of words) of the product name as the input and a sequence
of category names as the output. In recent years, remarkable perfor-
Encoder-Decoder Neural Networks, Recurrent Neural Networks, mance has been demonstrated in the field of machine translation
Taxonomy classification and automatic summarization by using the model called neural
ACM Reference Format: network Encoder-Decoder architecture. We apply the Encoder-
Makoto Hiramatsu and Kei Wakabayashi. 2018. Encoder-Decoder neural Decoder model to the taxonomy classification task and evaluate
networks for taxonomy classification. In Proceedings of ACM SIGIR Workshop the performance. Experiments show that our approach can suc-
on eCommerce (SIGIR 2018 eCom Data Challenge). ACM, New York, NY, USA, cessfully predict category paths more precisely than the baseline
Article 4, 4 pages. https://doi.org/10.1145/nnnnnnn.nnnnnnn
approach that treats the task as a multi-class classification problem
and applies Random Forest.
1 INTRODUCTION
Taxonomy is the major classification schemes in organizing con- 2 DATASET
cepts. With the rapid growth of the e-commerce market accompa-
nying the development on the Internet, the number of products on Table 1: Histogram of the depth of category paths
e-commerce becomes enormous. In this situation, it is required to
develop methods that predict taxonomic categories automatically
Category depth Frequency of item
because it is costly to classify all the products manually.
Rakuten Data Challenge, which is a competition we participated, 1 8,172
provides a task to predict correct categories for each given product. 2 2,792
As a feature of this task, categories have a hierarchical structure. 3 228,888
This hierarchical structure corresponds to a taxonomy, which in- 4 344,472
dicates that items in a category are further classified into a sub- 5 166,165
category that contains further lower detail information. Each prod- 6 45,253
uct has a path in the taxonomy like “Clothing, Shoes & Accessories → 7 4,197
Shoes → Men → Boots”. 8 61
As an approach to solving this task, the most straightforward
approach is to train a multi-class classifier (e.g., Random Forest)
We have 800,000 records for training data and 200,000 records for
that predicts a category path as a class of a given product. However,
test data. Each record has a description of a product and a category
Permission to make digital or hard copies of part or all of this work for personal or
Copyright © 2018 by the paper’s authors. Copying permitted for private and academic purposes.
path. The number of labels in the training data is 3,695, and each
classroom
In: use is G.
J. Degenhardt, granted withoutS.fee
Di Fabbrizio, providedM.that
Kallumadi, copies
Kumar, areLin,
Y.-C. notA.made or distributed
Trotman, H. Zhao label is assigned to 868 items on average. The category (id=4015) is
(eds.): Proceedings
for profit of the SIGIR
or commercial 2018 eCom
advantage andworkshop, 12 July,
that copies bear2018, Ann Arbor,
this notice andMichigan, USA,
the full citation
published at http://ceur-ws.org
on the first page. Copyrights for third-party components of this work must be honored. most frequently assigned to products, which is assigned to 268,295
For all other uses, contact the owner/author(s). items. Figure 1 shows the histogram of the number of words in each
SIGIR 2018 eCom Data Challenge, July 2018, Ann Arbor, Michigan, USA description in the training data. The average number of words was
© 2018 Copyright held by the owner/author(s).
ACM ISBN 978-x-xxxx-xxxx-x/YY/MM. 10.92, and the standard deviation was 5.21. The maximum number
https://doi.org/10.1145/nnnnnnn.nnnnnnn of words was 58, and the minimum value was 1.
SIGIR 2018 eCom Data Challenge, July 2018, Ann Arbor, Michigan, USA Makoto Hiramatsu and Kei Wakabayashi
70000
60000
50000
frequency
40000
30000
20000
10000
0
0 10 20 30 40 50 60
number of words in product descriptions in the dataset
Figure 1: Histogram of the number of words in product descriptions in the dataset
Table 1 shows the histogram of the depth of category path in 3.2 Encoder-Decoder neural networks for
the training set. The depth of category path in training set is 4.01 taxonomy classifier
on average. In other words, each product has four categories on
Encoder-Decoder Neural Network is a type of neural network that
average. The maximum depth of the depth of category path was 8,
is actively studied in recent years [1, 3, 7], which shows very good
and the minimum depth was 1.
performance in various tasks such as machine translation and auto-
matic summarization. We will describe the Encoder-Decoder Neural
Network used in this research.
3 PROPOSED METHOD Figure 2 shows our Encoder-Decoder neural network with at-
3.1 Preprocessing tention mechanism [1]. Our model has two main functions called
encoder and decoder. An encoder function fenc takes an input
We used 20 % of the training dataset as the validation set to eval-
sequence of words x = (x 1 , x 2 , . . . , x n ) and a decoder function
uate models. As preprocessing, we lowercase a product name in
fdec predicts the probability of a category path sequence y =
training/validation/test sets with SpaCy 1 . We use both the original
(y1 , y2 , . . . , ym ). fenc outputs a sequnce of hidden states h = (h 1 , h 2 , . . . , hn ).
corpus and the lowercase corpus and compare classifier perfor-
To predict yt , fdec uses information from h and c t . A context vector
mances.
c t captures input sequence information to help predict an each label
For the weights of dense word representation layer, we use
yt . A context vector c t is defined as following:
GloVe [6] pre-trained embeddings trained on Gigaword and Wikipedia. Õ
GloVe contains the lowercase words in its vocabulary. The prepro- ct = at i hi , (2)
cessing of lowercase makes the vocabulary matchinд rate improve. i
We show the matchinд rate of two corpora in Table 2 where source and attention is defined as following:
means descriptions of products, which are inputs. matchinд rate is
ât i
defined by at i = Í , (3)
j â t j
ât i = att(hi , h̄t ), (4)
|VDat aset ∩ VGloV e | where att(ht , h̄i ) is an attention function. The attention function
matchinд rate = , (1) of our works is based on Luong et al. [4] defined as following:
|VDat aset |
att(hi , h̄t ) = hi T Wa h̄t , (5)
where h is the encoder state, h̄ is the decoder state and Wa is the
where VDat aset is the vocabulary of the dataset and VGloV e is the weight matrix that controls the contribution of each hi and h̄t .
vocabulary in the GloVe embeddings. 1 https://spacy.io
Encoder-Decoder neural networks for taxonomy classification SIGIR 2018 eCom Data Challenge, July 2018, Ann Arbor, Michigan, USA
Table 2: Vocabulary matching rate
Preprocessing The size of source vocabulary Matching rate
None 670,092 10.69%
lowercase 626,567 57.82%
Predict sequence (category path)
1608 1206 1632 4680
Attention
+
Reebok Zigpulse Track Shoes 1608 1206 1632 4680
Source sequence (product title) Target sequence (category path)
Figure 2: Encoder-Decoder Neural Network
Equation 6 is the log-probability for predicted sequence. We use the TF-IDF vectors for features of the product description
n
Õ representations. To implement the baseline, we use scikit-learn [5].
log p(y | x) = log p(yt | y