With the constant growth of the number of products on e-marketplaces, buyers feel hard to find and choose items that would satisfy all their needs and expectations. Search and filtering algorithms of recommender systems, although are striving to help users, still fail quite often due to incomplete and inaccurate description of items. The given work suggests to combine analysis of both item description and item image in order to construct groups of similar items. Since a person can define whether two items are similar or not looking at two images and a brief description, it is suggested to form a set of similar items based on users' judgments and then to extract the core of keywords for the specific type of products. Further, it is proposed to use the given core to evaluate the similarity of any new item added to the definite group. The case study deals with the building of the core of keywords for sneakers. The developed key attributes allow matching the items with a high precision, thus, proving the effectiveness of the method of the core construction.
E-commerce marketplaces are the inexhaustible source of goods that may satisfy the needs of any exacting client. But often, the task to find the necessary product becomes a real challenge for a customer. While the sellers are allowed to create multiple item records for the same product, the customers see the endless lists of recommended items that actually might be a single product. Further, looking deeper into that list, a person can realize that it is not an easy task to estimate if the products are the same since their descriptions, images, titles, prices may be too different.
Buyers need to get an excellent user experience while searching for products and
making purchases on the e-marketplaces. That is why to stimulate purchases,
e-commerce platforms are constantly improving the efficiency of collaborating filtering
algorithms that form the list of recommended items. One of the steps in this process is items
matching. Many e-commerce platforms such as Walmart check if the item already
exists in the catalogue before setting up a new one [
Items matching is usually based on a comparison of textual data provided in item descriptions. Firstly, this data is often incomplete or missed. Secondly, it is represented in a way that puts a strain on matching algorithms. Item’s attributes may be called differently and may contain un-normalized values that make the analysis more difficult. And sometimes the description of an item does not contribute much to the buyer’s perception. For example, such products as mobile phones, TV-sets, air humidifiers can be precisely described and identified via their technical parameters. On the other hand, clothes, shoes, bags are the products that do not have many attributes for description and even if they do, the show and layout of a thing play a bigger role for item’s acquisition. In this situation, items’ images should play a major role in matching.
The given paper represents the idea of images matching combined with matching textual descriptions of items. The developed pipeline for data processing allows to collect similar items based on expert judgments, extract their tags and build the core of key attributes that may be used for further matching of new items.
The rest of the paper is organized in the following way. Section 2 considers existing works in the area of images matching in general and in the e-commerce domain in particular. Section 3 describes the scheme of items processing and the tools used for collection of the initial set of items. The experiments with two datasets are given in Section 4. The analysis of results and conclusions are given in Sections 5 and 6. 2
Images matching is a complex problem discussed by many researchers in the context
of computer vision and augmented reality applications, 3D modeling, visual search, etc.
[
In addition to feature-based methods of image matching that are implemented
usually as neural networks, there is also a group of geometric point matching methods.
Researchers in [
Treating an image not only as a set of pixels but as an object that has its internal
sense has given a start to the direction of images semantic analysis. For example,
authors of [
The field of e-commerce has brought new statements of images matching problem.
Images matching is a non-trivial problem and although everyone realizes that it may
improve the efficiency of recommender systems, still researchers find weak points in
similarity measures of images and keep on working on new methods. The main goal of
images matching in e-marketplace applications is de-duplication of item records. Many
studies are aimed at finding identical images that may indicate identical products [
Generally speaking, images matching is a part of the bigger e-commerce items
matching problem. Many researchers suggest different ways to find similar items in the
e-stores [
Our previous works [
The main goal of our research is to find ways to improve the quality of the search for product offers on electronic trading platforms. We have suggested that the solution to this problem should be based on the structuring of item offers. From our point of view, the most natural way is to group similar items. The analysis shows that existing approaches, such as the use of filters or recommendations, are not always effective. The use of filters may limit the choice of products. For example, the product description does not contain information about all color options, the price includes shipping costs and does not match the selected range, an error may be made in the model name, the brand is not specified, etc. The results of the recommendation systems vary on different trading platforms and often use mixed algorithms that take into account not only the characteristics of goods but also the behavior of users. As a result, some products may not fall into the search selection or similarity group. On the other hand, we noticed that a person surfing via an e-marketplace always identifies similar products. In this case, the person relies on the image and a brief description of the item. The main idea of this study is to create a core of keywords that determine the description of a similarity group.
Analyzing a lot of up-to-date evaluation approaches, we found out the pairwise
comparison is the best way for humans. Following the Theory of Intelligence [
According to [
Therefore, just monitoring how people implement items matching we can obtain data about items similarity. We suggest applying pairwise comparison of item images to obtain similarity estimation. Based on the processing of the pairwise comparison matrix we can determine the groups of similar items. We make a hypothesis that the group of similar items has the common core description, that is why we suggest building the core of the keywords for each group. So, we can extend the group of similar items based on comparing the new item with the core of keywords. The general scheme of processing of items is presented in fig. 1.
Collecting images and descriptions is performed with the help of grabbing and parsing software. The raw data sets are inspected manually. We have developed our own mobile application for pairwise comparison of items. We suggest using a crowdsourcing approach in order to collect estimations of item matches. Due to this reason, we have asked volunteers to upload our mobile application and do comparisons. The developed software component is an Android-mobile client. The user can compare images in pairs and indicate the similarity or difference of items offered for comparison. The data provided by the user is collected in cloud storage and further can be used for the next steps of processing. The result of the comparison is sent directly to Firebase Cloud node via HTTP-protocol where it is validated and stored. Each user session has a unique ID to make data analysis more efficient. If the user wants to finish the comparing, the “Finish” button should be pressed and the application would be closed. The image comparison screen is given in fig. 2. Therefore, we collect images and descriptions of items, then estimate the similarity based on image matching, and finally process item description to build the core of tags for each group of similar items. The given study is focused on developing the key attributes of similar items based on the item’s image tag comparison. 4 4.1
In this paper, we have checked the idea that if we know item groups with similar objects, then in the future we can easily find the correct group for new items. Thus, the main idea to make an experiment is checking the assumptions: 1) that similar items have the same core of tags; 2) that group of a new object can be easily found if the core for a group of objects is known. In the first step, we create the dataset. Each item in our dataset has an image and a text description. The example of such an item is presented in Table 1. A brand-new, Unused, Unworn, Sneakers, Breathable, Lightweight, Comfort, G27706, Does not apply, Full Year Article, Leather, adidas, Fabric, Does not apply, 2010-2019, Low Top, Leisure, Rubberband, Unisex, Lacing, Casual, adidas Continental, adidas Originals, adidas Continental 80 , Standard, Casual Shoes,
White / Scarlet / Collegiate Navy, Fitness Studio & Training
For our experiments, we have transformed the item description into a set of tags. We prepared two datasets according to our data processing pipeline. The first dataset is the set of similar items gathered from the eBay trading platform (https://www.ebay.com/). We chose the data about white sneakers. The second dataset contains different items, for example, different types of shoes, bags, etc. from the same source. Table 2 shows information about datasets. After the analysis of our datasets, we decided to clean the data, because tags for all items vary a lot. For example, we have such tags as "M" or "9.5" as the size of shoes or "2010-2019", "B42000", "light-weight", "Running & Jogging". A lot of tags contain different punctuation such as "&", "-", ",", ")", "(", etc. All stages of the cleaning process are presented in Fig. 3.
Item description
Set of tags
Delete punctuations
Remove tags with length less that 3 symbols and stopwords Formation of new tags from long descriptions
Calculate frequency
of tags Detele repeating
tags Remove digits
Final core of tags In addition to removing punctuation and other characters, we needed to convert long tags (such as "Running & Jogging") to a set of short tags ("Running" and "Jogging"). For example, "White / Scarlet / Collegiate Navy" was transformed into "White", "Scarlet", "Collegiate" and "Navy". We decided to remove stopwords and tags with length less than 3 symbols, these tags are not informative for the dataset. In order to get the tag core, we also need to calculate the tag frequency for the collection and remove duplicate tags. The number of tags after each stage is shown in Fig. 4. 4.2
Having the cleaned tag list sorted by descending tag frequency, we created the tag core
for the Dataset_1. The algorithm of tag core creation was described in detail in work
[
We evaluate the results of the experiment in two stages. The results presented above show that the constructed core describes the dataset well. Inside the dataset items are similar, but not always the same products. The analysis showed that the obtained estimates for identical pictures are equal. In these cases, the sets of tags also match. This confirms the assumption that the grouping can be carried out on the basis of assessing the similarity of the product description and core of tags for the group. It should be noted that for identical items with different images, we find differences in similarity values and in the sets of tags respectively. This may be because different sellers provide such descriptions.
The main idea of building a core of tags is to be able to determine whether a new product will fall into a group of similar goods. In order to estimate the proposed way to matching items, we use the Dataset_2. It contains 162 items of sneakers, shoes, and some other goods. According to the data processing pipeline discussed above, the sets of tags for each item description are created. We evaluate manually all the items whether they match items from the Dataset_1. Then we compare manual estimates with the similarity score of each item. We use the 0.75 similarity score as an indicator to join the item group. As a result, we obtain Total Items = 162, True Positive Items = 101, True Negative Items = 19, False Positive Items = 14, False Negative Items = 28. Therefore, Accuracy = 0.74, Precision = 0.88, Recall = 0.89. Manually classified items are all sneakers with a similar shape or similar models without taking into account their color. Thus, we can conclude that the proposed approach is validated, but it should be improved by studying the similarity score more deeply. 6
The given work represents the experimental pipeline of the core of key attributes construction for e-commerce items. It is suggested to build such a core from the tags of the items that have been acclaimed as similar by the experts based on images comparison. The method proposed was proved to be suitable for constructing the core for sneakers. The two open questions in method application are: 1) the definition of the threshold for attributes inclusion to the core since different values lead to a different number of keywords included; 2) the adaptation of the threshold for the analysis of new items that have to be compared with the core.
The future directions of this research include testing the method on other types of products, not clothes (shoes) but household appliances, for example. Also, the application of the method for bigger initial samples (including millions of items) obviously will require its modification and additional testing. And finally, the evaluation of search and filtering algorithms performance with and without application of the keywords core is planned for future work.