Product search engines are a key factor for online business. Retrieving relevant products in an E-Commerce (EC) website is of the utmost importance, as a single EC website can have millions of products with very similar features. One aspect that is not widely studied in this scenario is the efect that the image of the product (notably aesthetic properties) shown to the customer has on the customer's interest. Previous studies have been able to link certain characteristics of images to our innate interest. For instance, it is known that bright images with several colors are more likely to attract one's attention than dark ones. However, these issues have been understudied in the EC context. In this context, we conduct experiments on real-world EC to analyze the efects that the product's image aesthetic has on the user interest (expressed in clicks) in the product. Experimental results show that this relationship exists and that it is more visible in some categories of products.
E-Commerce (EC) platforms have become a popular means to bring
greater shopping convenience to costumers around the globe. These
platforms bring a series of social and technical challenges, which
have not been extensively studied. Examples of these challenges
include problems related to the trust and familiarity that these
websites convey to the customers [
After stumbling upon a product in an EC website, as a result of
an explicit search or associated with an ad, the user’s decision to
click on such product (which, in turn, may lead to a purchase) may
be influenced by the image of the product shown to the user. Thus,
even though this is commonly overlooked or taken for granted,
the product’s image, alongside with its title, description and tags,
may be deciding factors between buying the product or not. The
quality of the product must be clearly conveyed through these
features in order to convince the customer to buy the product
[
The challenge of evaluating the aesthetics, beauty, or quality of
an image, music or any artistic work has been studied for quite
some time [
Even though this is a complex and noisy problem to model, there
are few general characteristics of artistic works that determine their
visual quality. For instance, in photography, exposure, rule of thirds,
contrast, and other characteristics are often carefully planned and
chosen by the photographer. A few authors, such as [
The present study tackles the aforementioned hypothesis by analyzing the relationship between the quality of a product’s image and the probability of it being clicked when presented as a result for a search query in a large EC website focused on crafts and personalized products. Our goal is to verify whether the image features, including features related to aesthetics, can explain, at least partially, those clicks. Our experimental results show that this relationship does exist, even though it seems to be noisy. The applied methodology could be used to improve the search results of EC, presenting more attractive products to the customers or to guide sellers towards improving the attractiveness of their products. 2
There are several works focused on the definition and evaluation
of the aesthetics of an image. In one of the earliest studies [
A more recent work [
The use of Convolutional Neural Networks (CNN) is explored in
[
There are very few works that try to correlate the quality of the
image of a product with the product popularity (e.g., click rate).
In [
We use use three sets of of automatically extracted features to capture the aesthetic of the product images. The first one corresponds to state-of-the-art aesthetic base features that capture the most fundamental aspects of the image (e.g. exposure and saturation). The second one is the GIST descriptor that captures scene categorization and image layout. The third and last set is the BOVW, a generic content-based set of features which describe the distribution of local patches within the image. 3.1
The base features consists of 66 attributes extracted from the
product’s image. These attributes are directly related to the main aspects
1https://research.google.com/ava/
of the image, including features that capture the exposure, contrast,
saturation, ratio, depth, sharpness and composition [
The GIST descriptor is a low-dimensional scene descriptor that
captures a set of characteristics, such as naturalness, roughness,
expansion and ruggedness. These characteristics are estimated using
spectral information and coarse localization. The image is
partitioned into a 4 × 4 regular grid and a histogram of gradients (with
20 bins) is computed for each of the 16 regions and 3 color channels.
Finally, all histograms are concatenated to form a 960D vector [
BOVW represents an image by a histogram of local features [
Our evaluation exploits real data from Elo7, the largest Brazilian emarketplace focused on creative and personalized products2. In this EC platform, sellers register their own products, uploading pictures and providing descriptive text data (e.g. title, description, price, tags). Product searches return a grid of products where the photos are prominent and textual data include only the product’s title and price. Since the sellers are responsible for this information, there is a lot of heterogeneity in terms of quality of the products’ text features and, specially, their pictures, making it an ideal scenario for this study. The data contains over four million products in 42 Image Aesthetics and its Efects on Product Clicks in E-Commerce Search diferent categories and a hundred thousand unique queries across two months (October and September of 2018).
In this work, departing from our main hypothesis, we associate the interest of users in a product with the quality of its image. More formally, given a search query q and a product p, we have the number of times in which p was retrieved after a customer searched for q (impressions) and how many times it was clicked. The interest score of a product with relation to a query is then calculated as the ratio between the number of clicks and impressions times the log clicks of the number of clicks: clickscore = impressions × log2 clicks.
The addition of the logarithmic term is to give an additional priority for products that have a high click count and are more popular. We only considered products that have more than a thousand impressions to try to eliminate eventual noise coming from products that appeared very few times to the customers. For example, a high click rate on a product that appeared very few times over many queries may suggest a customer looking for a very specific product that does not otherwise show up in other searches and therefore is not a clear indication that the product has a good image as perceived by larger set of users.
Based on these scores, we then label images that are above the 80th percentile as “highly clicked” images. Images that are bellow this percentile were considered “poorly clicked”. The idea is to check if image features, including features related to aesthetics, influence the amount of product clicks in e-commerce search.
Figure 1 shows two examples of product’s images from the website. Figure 2a and 2b are from products that have a high and low score, respectively. The first image has more colors, better lighting and is overall more attractive than the first one. 5
Two experiments are conducted. First, we verify whether there is a significant diference in the values of image features between highly relevant (clicked) products and the less clicked products. Second, we test the capacity of a machine learning model to predict whether or not a product will be highly clicked based on its image features. (a) f 5 – Exposure (b) f 8 – Average Intensity 5.1
To check whether there is a significant diference between products that are more frequently clicked against their counterparts, we selected two thousand queries that had products with over a thousand impressions. For each query, we compare image features of the highest and lowest ranked products (most and least “relevant” products, respectively). For each feature, we performed the Kolmogorov-Smirnov test to check whether the two distributions (associated with the lowest and highest ranked products according to their click scores) are significantly diferent. 42 out of the 66 features had p-values under the 0.05 threshold, meaning that they are statistically diferent. Figure 2 shows the density plot of two features with a p-value under 0.05.
By looking at the density plot (Figure 2) of the features under the two categories (most and least relevant) we can see that the distributions are slightly diferent, corroborating the idea that image features do influence users in the click decision. For instance, in Figure 3a we can see that images that are more clicked have a higher exposure, which makes sense, since brighter images tend to attract more attention. 5.2
We performed a second experiment to check how accurately we can
predict the quality of the product’s image based on its click data.
Given that the products have a category, and since the images vary
vastly from category to category, we built diferent models for each
category. Thus, we divided the prediction into 10 sets of products
corresponding to the ten most common categories. A thousand
images labeled as “highly clicked” and a thousand image labeled as
“poorly clicked” were put into each one of these ten groups. Finally,
we use three sets of features (Base, Base + GIST and Base + BOVW)
to train an SVM [
Table 2 shows the metrics F1, Precision and Recall for the predictions (in the test sets of the cross-validation procedure) of the ten categories and the three sets of features. Firstly, we can see that, although predictions are hard, as the categories have a mean F1 of 56.7% for the Base features (the worst being 53% and the best 61%), for most categories they are still statistically better than random (F1=0.5). Thus, there is indeed some predictive power in the set of Base features. Secondly, we can see that the GIST features only helped to improve the F1 score in four out of the ten categories, making only the recall higher whilst reducing the precision. The BOVW features improved the results in six of the ten categories, increasing both, recall and precision. This means that local features have an interesting potential in this application.
Finally, we note that clearly some categories are easier to predict than others. The categories “Candies” and “Children’s” had the highest F1 while categories such as “Invitations” and “Baby” had the lowest. This could be explained by the types of images that are presented in these categories. In the “Candies” category, we usually have products that have bright colors, with diferent shapes, sizes and textures that can attract attention and clicks. In this category, images that do not have many colors or that are darker may have a lower click rate. However, in the “Invitations” category, where we have very similar images to one another, mostly white paper invitations with similar texture, the image may not be a deciding factor for a product’s click rate.
Although there are some categories that are easier to predict than others, the precision is still low. This indicates that the image alone is not completely responsible for the user’s decision to click on a product. Other factors such as the price range and the title of the product may influence customers actions. But, as we can see from the results, a product’s image quality does have some influence and could possibly be used to improve search results for an EC platform. The question of how much the quality of a product’s image influences its click rate as compared to its other properties (such as title and price) is left as future work. 6
In this work we investigated how product images influence product clicks in e-commerce platforms. The detection of aesthetically “bad” or “good” images can be used to improve e-commerce search engines, and, consequently, customers’ satisfaction and revenue for e-commerce companies. They may also provide feedback to help sellers to make their product more attractive to customers. Our experiments show that image attributes such as brightness, colorfulness and contrast can influence product clicks. First, we analyzed how some of these features vary when comparing more frequently clicked products with less clicked ones. We then used machine learning to try to predict product clicks based on image features. The performed experiments show that there is potential in this technique, specially for some specific categories of products.
As future work, we intend to test other machine learning methods and to add other features, such as Fisher Vectors. We can also propose specialized models for diferent categories, since the influence of the images in product clicks seem to vary according to the product category. We plan to study in more detail why and how product categories difer in order to produce more accurate models to better predict product clicks from image quality. Finally, we intend to run additional comparative experiments (e.g., comparing images with product’s titles and prices) to deepen our understanding of a customer’s motivation to click on a product.
This work was partially supported by Elo7, the FAPEMIG-PRONEXMASWeb project – Models, Algorithms and Systems for the Web, process APQ-01400-14, as well as by the National Institute of Science and Technology for the Web (INWEB), CNPq and FAPEMIG.