The product matching task aims to identify that if a pair of product deriving from di erent websites refer to the same product or not. While the accumulated semantic annotations of products make it possible to study deep neural network-based matching methods, product matching is still a challenging task due to su ering from the class imbalance and heterogeneity of textual descriptions. In this paper, we directly regard product matching as a semantic text matching problem and propose a pre-trained matching model based on both self- and inter-ensemble. BERT is the main module in our approach for binary classi cation of product pairs. We perform two types of ensemble methods: self-ensemble using stochastic weight averaging (SWA) for the same model, and inter-ensemble combing the prediction of di erent models. Additionally, the focal loss is adopted to alleviate the imbalance problem of positive and negative samples. Experimental results show that our model outperforms existing deep learning matching approaches. The proposed model achieves an F1-score of 85.94% on the test data which ranks second in the SWC2020 on Mining the Web of HTML-embedded Product Data Task One. Our implementation has been released 3.
In recent years, online shops (e-shops) are increasingly adopting semantic markup languages to describe their products to improve their visibility. Those semantic annotations are conducive to the further analysis of product o ers. However, di erent annotation systems used by e-shops often lead to data inconsistencies or con icts. Product matching is the task of identifying the similarity of product o er pairs, which is the fundamental technology to construct a uni ed system such as product knowledge graphs. Moreover, product matching can improve the e ciency and experience of online purchasing. As for customers, matching the Copyright c 2020 for this paper by its authors. Use permitted under Creative Commons License Attribution 4.0 International (CC BY 4.0). 3 https://github.com/englishbook/product-matching same product on di erent websites is convenient for them to compare and nd the best choice quickly. As to the e-commerce platform, the matching results can be used for product recommendation. Therefore, product matching is a crucial problem in the e-commerce domain.
Product o ers are described by the textual information (e.g. title, description,
and brand). We can think of product matching as a semantic text matching
problem. Although many previous works have been done on text matching and
have shown great success with deep neural networks [
To address those limitations, we propose a pre-trained matching model based
on both self- and inter-ensemble for product matching in this paper. 1) For
semantic complexity, we apply pre-trained BERT to model text pairs. BERT [
With the development of deep learning, large amounts of matching models based
on deep neural networks have emerged and shown their e ectiveness. Previous
works mainly focus on the siamese architecture [
Recently, pre-trained language models are proposed and have achieved
signi cant improvement in various NLP tasks, with state-of-the-art models such as
BERT [
The WDC product data corpus, the largest publicly available product data corpus, is released by the Web Data Commons project in 2018. The corpus consists of 26 million products originating from 79 thousand websites. Products in the corpus are described by these properties: id, cluster id, category, title, description, brand, price, and speci cation table. Products with the same cluster id indicate the same product. In the SWC2020 challenge product matching task 4, organizers provide a dataset containing matching and non-matching pairs of products only from Computers & Accessories category. It is sampled from the product data corpus according to the clusters (68K product pairs for training, 1.1K for validation, and 1500 for testing). We also utilize an extended training dataset 5 with all the four product categories derived by the same sample strategy (214K for training, and 4.4K for validation). Matching models can learn more information from extra categories and thus make a more accurate prediction on samples from Computers. The statistics of these training datasets are listed in Table 1.
Besides, some data preprocessing operations are performed before inputting product information into models. We remove stopwords (using NLTK) and lowercase all textual descriptions. 4 https://ir-ischool-uos.github.io/mwpd/index.html 5 http://webdatacommons.org/largescaleproductcorpus/v2/index.html
Matching Score
… T( E( … …
T) E)
T[&*+] BERT
E[&*+]
T(, E(, … …
T-, E-,
T[&*+] E[&*+] Input Layer [CLS] Tok 1 … Tok M [SEP] Tok 1’ … Tok N’ [SEP]
Product A Information
Product B
Information
Although products are described by many attributes, most of the elds contain NULL values. The title attribute of all products is lled, and the lling rate of the description attribute is relatively high. Therefore, we mainly focus on these two attributes. We concatenate the textual information of product pair by [SEP] token at rst, and then add [CLS] and [SEP] tokens at the beginning and end respectively as the input of BERT, x = [[CLS] tA dA [SEP ] tB dB [SEP ]]. BERT can model the input tokens through the multi-layer bidirectional Transformer encoder and generate high-level representations. The output state of BERT that corresponds to [CLS] token is used as the pair representation. We feed the representation into a fully connected layer with the sigmoid activation function and obtain the nal matching score p between two product o ers. where pt = p when the ground truth is 1, otherwise pt = 1 p. The weight factor 2 [0; 1] is set according to class frequency to balance the importance of positive and negative samples. For convenience, we de ne t similar to pt. Moreover, the focusing parameter 2 [0; 5] is introduced to di erentiate easy and hard examples. In that way, the samples that have been accurately classi ed contribute less to the loss so that model can focus training on few hard cases.
Final Result
Inter-ensemble Model 1
Model 2 …
Model m Self-ensemble
BERT
BERT …
BERT Epoch Epoch + 1
Epoch +
Self-ensemble: for the same model at di erent training. As illustrated in
Figure 2, we use the stochastic weight averaging strategy [
Inter-ensemble: for di erent models. Di erent models have their advantages. For example, models trained by the cross-entropy loss should predict more accurately on substantial non-matched samples, while models with the focal loss should perform better on few hard examples. In this paper, multiple models are obtained by training on di erent datasets, inputs, and loss functions. As illustrated in Figure 2, we average the prediction probability of these models as the nal results to combine the strengths of di erent models. 3.5
Many attributes are not fed into the model for training, but they are undoubtedly useful for product matching. In the SWC2020 challenge, we attempt to take full advantage of them to correct the prediction results. The values of category attributes are assigned to four uni ed categories. Two products belonging to di erent categories must be non-matched. For test pairs with prediction results of 1 but di erent categories, we correct their results to 0. The following experiments demonstrate the e ectiveness of the post-processing operation. Similarly, the brand and price can also be used for correction in the future.
4.1
Product textual information is padded or truncated to a xed length. The max
length is set to 64 and 200 for the input only with product title and the
concatenation of title and description, respectively. For pre-trained BERT, we initialize
the model by the weights of BERTbase. For focal loss, is set to 0.75, and
is set to 2 to focus on hard positive samples. The optimizer we adopt is Adam
[
The F1 score on the positive class is used as the evaluation metric. We have trained multiple models with di erent architectures and training strategies. Models trained on the All dataset are also evaluated on the validation set with four categories. Table 2 shows the experimental results of these models on the validation set in detail. We can nd that the performance of pre-trained BERT for the product matching task is signi cantly better than other classic matching models (e.g. CNN, ESIM). Incorporating focal loss and SWA strategy further improves the BERT models. Moreover, post-processing can indeed correct some prediction errors e ectively. After obtaining various models, we select several models that perform better on the validation set and try to integrate them by inter-ensemble strategy. Averaging the matching score predicted by multiple models can combine their strengths. The results of our ensemble models on the validation set in the Computers dataset are presented in Table 3. Ensemble models are generally better than single models. In the nal evaluation of the test data, we submitted the prediction result of our best ensemble model. As shown in Table 4, we achieve an F1-score of 85.94% on the test set which ranks second. The experimental results demonstrate the e ectiveness and generalization ability of our proposed model. In this paper, we propose a pre-trained matching model based on both selfand inter-ensemble for product matching. Pre-trained BERT is adopted as the base matching model. We incorporate the SWA strategy in the training phase to improve the generalization ability of models and combine the output of di erent models to make full use of their advantages. Experimental results show that our model achieves great improvement compared with existing state-of-the-art matching models.
An interesting direction of future work is to pre-train BERT on product data corpus so that it can learn more product description ways. Also, post-processing operations are worthy of further study, especially in industry practice.
Acknowledgments This work was supported by the National Key Research and Development Program of China (Grant No. 2019YFB1704402) and 2019 Tencent Marketing Solution Rhino-Bird Focused Research Program.