In recent years, item-item collaborative filtering algorithms have been studied thoroughly in recommender systems. When applied in the context of fashion e-commerce these algorithms can be used to generate similar recommendations, personalize search results and to build a framework for creating clusters of similar products. The efficacy of these algorithms when applied in fashion domain, rely on accurately inferring a user's fashion taste and matching them to a product. Our work hinges around discovery of similar products using two different item-item collaborative filtering algorithms. We identify and address some unique challenges while applying these algorithms in the dynamic fashion e-commerce environment. We study and evaluate their performances through precision/recall measures, live A/B tests and baseline them against a content based approach. We also discuss effects of the transient nature of the industry on a user's fashion taste.
Fashion shopping is a complex intersection of product styles with users’ taste where products are usually described in terms of several attributes such as silhouette, line, hem length, color, fabric, waist length and so forth. These attributes are dynamic in nature and vary as trends in fashion change. However, these attributes are not typical representatives of a user’s fashion taste. Users rely more on look, feel, popularity and other intrinsic factors to identify their taste.
The advent and growth of fashion e-commerce, pose even more challenges towards providing users with relevant products. As the industry grows, there are thousands of products available in a catalogue with similar set of attributes. Also users’ needs in fashion are not specific as compared to hard goods like electronics, therefore they tend to browse significantly more products on a fashion portal before clicking on a particular product. In general, at Myntra we observe the average click depth i.e. number of products viewed before the first click, to be around 90. Correctly inferring a user’s intent in a session becomes critical for providing a better shopping experience. A user’s click on a prod-uct is a proxy for his/her interest in that product. We can utilize this information to narrow down his/her intent. Thereafter, serving a user with similar products to the clicked one, helps them navigate through the vast catalogue and find relevant products. We also use similar products to personalize search results as majority of queries on our platform are broad in nature like ’t-shirts’ and ranking results based on user’s interest improves the overall search relevance. Therefore, discovering similar products in fashion e-commerce becomes an interesting data mining and business problem. In this paper, we present our work on prescribing similar products using item-item collaborative filtering algorithms in a fashion context.
Similarity between items is usually computed in terms of content
based similarity, item-item based collaborative filtering [
Unlike content based similarity measures, item-item
collaborative filtering algorithms use user signals instead of a taxonomy to
compute item-item similarity. In the domain of e-commerce users’
feedback is not explicit [
The challenges with using these implicit feedback signals are following: • Quantifying the chosen signals by assigning relative weights to each. • Normalising efect of popular products i.e. products which have signals from a large number of users.
In this work, we look at two representations to model item-item collaborative filtering. We compare these two approaches with a content based approach, where we use the annotated taxonomy of a product to get a feature representation for it.
In the following sections of this work, we describe these three approaches in detail and compare their performance via A/B tests and precision-recall test. Finally, we try to conclude by inferring whether the transient nature of the industry afects user taste or not. 2
For the remaining sections of this paper, we will use the following terminology. Let P be the set of all products and S be the set of all user sessions. A session contains all activity by a user within a 30 minute window from the time he/she logs into the portal annotated by time-stamp. We record user signals like product list views, product clicks, addition to carts, orders placed and so on.
For all the three approaches mentioned below, we split our data at the article type (e.g Men-Tshirts, Men-Shirts, Women-Dresses) level, since products similar to a product pi should be from the same article type. Splitting the data at the article type level has the added advantage that it reduces the set of products from which we ifnd similar products for pi . 2.1
Each product in our system is annotated with attributes generated
from a manually curated taxonomy. These attributes are broadly
classified into two types, general attributes like brand, color, price
bands etc which are applicable for all article-types and article-type
specific attributes (ATSA) like collar type, sleeve length, neck type
for T-shirts. Each attribute is represented as a key-value pair e.g.
collar type of a t-shirt is the key and diferent collar types like
roundneck, v-neck, polo-neck are values. Each product is represented
as a vector in the real space, where each dimension represents an
attribute. We use binary values to populate a particular dimension
i.e. if the attribute represented by the dimension is a part of the
set of attributes which were annotated to the product, then we
populate the dimension as 1, else we populate it as 0. Along with
product attributes, we extract relevant bi-grams using log likelihood
ratio scores[
We analyse this approach here to benchmark the performance of item-item collaborative filtering algorithms. The major challenge with this approach is that the set of attributes representing a product is not comprehensive and could results in products not being annotated by some key information. 2.2
In this approach, we use an undirected weighted graph
representation to model product relationships in the system. In this
representation nodes represent products, edges represent associativity
between products and edge weights represent degree of
associativity between products. This approach was first introduced by
YouTube [
(0.081) (0.078) (0.071) (0.059) (0.056)
In order to to generate this graph, we use a well-known technique
known as association rule mining[
wi j = log−p1(i,j) ∗ log pp(i()ip,j()j) where p(i,j) is the probability of occurrence of the pair (pi ,pj ) and p(i) and p(j) is the probability of occurrence of pi and pj respectively. We can use the following formulations for the values of p(i,j), p(i), p(j): p(i,j) = ci j p(i) = ci
C C where, ci j = count of occurrences of pair (i,j), p(j) = ci =
X cik k ⊂P and
C = X i, j ⊂P
Then, to find similar products for a product pi , we locate pi in the graph, we consider it’s adjacent nodes and among them pick the top N neighbors, sorted in descending order by edge weight. Here N is a hyper-parameter which denotes the number of similar products we want to find for pi .
The advantage of using this formulation to generate edge weights, is that it normalises the efect of popular products. If we consider a pair of products (pi ,pj ) where pj is browsed more across all sessions as compared to pi , then the value of cj will be high, resulting in a high value of p(j). The high value of p(j) results in the PMI of (pi ,pj ) turning out to be low, even though the co-occurrence counts of (pi ,pj ) might have been higher as compared to other pairs containing pi .
One challenge with using this approach is the noise prevalent in the input data. Some pairs might have a low co-browsing count or low PMI score to form a meaningful edge. To tackle this challenge, we put a threshold on both the co-browsing counts and the edge weights generated using PMI scores. After experimenting, we found 5 to be a suitable threshold for co-browsing count and 0.15 to be a suitable threshold for PMI score. These thresholds change depending upon the number of products and user sessions. cj C ci j
Another source of noise is the session in consideration itself. This entire approach hinges around the assumption that products browsed in a session are similar i.e. there is some context to products browsed by a user in a session. If the number of products browsed in a session are low, say 2 or 3, or if diferent article types were browsed randomly in a session, without any intent, then these sessions will add noise to our system. To tackle this problem, we use the concept of coherent sessions. We define a session as coherent, if in that session, the user has browsed at least three products of the same article type, and not more than two article types.
As we have considered sessions to construct the graph, it results in edges being formed only between products which were present in the system at the same time. Therefore, this graph captures the ephemeral nature of fashion trends, because products present across two diferent points in time would never form an edge. So, while ifnding similar products from this graph for pi , the output products represent a trend from the time when pi was present. 2.3
In this approach, we represent each product as a vector of user signals in the real space, with each dimension representing a user. We consider three user signals i.e. product clicks, addition of products to cart and checkout for populating values in each dimension. We start by aggregating these user signals across S, for a product-user combination (pi ,ui ), and generate a chronologically sorted list of all the identiefid signals that user ui has generated for product pi , where pi ∈ P . If we denote a product click by C, addition to cart as T and checkout as O, one example of such representation could be (pi , ui ) = {C, C, C, T , O }
After generating this list, we quantify our user signals, based on past data. We estimate the relationship between number of product clicks, add to carts and checkout using a linear classification model with each session as a data point. In the linear classification model, we use product clicks and add to cart signals as input and the checkout event as the target value.
The weight for checkout event is considered as 1, since it is the target value in the classification model. So, for the products which a user has bought, we populate a value of 1 in that user’s dimension for those products. For products, which the user hasn’t bought but has clicked or added to cart, we populate it’s vector with weights that we obtained from the model. For users, who have not generated any of the above signals for a product, we don’t populate any value in their dimensions. This results in the vector for each product being sparse, since the subset of users who have generated signals for a product is very small as compared to all users.
After computing the vector for a product pi , we take it’s L2 norm and find it’s N similar products by finding it’s N nearest neighbors using the following formulation of cosine similarity as the distance metric.
cos (i, j ) =
p⃗i · p⃗j |pi | ∗ |pj | where p⃗i and |pi | represent the vector of pi and it’s magnitude, "·" represents the dot product between the two vectors.
In order to get accurate cosine similarities, we utilise a
compressed sparse row (CSR) [
We observe that products which are globally popular are dense as compared to products which are less popular. This results in popular products being close neighbors to many products, if their vectors are not normalised. Taking a L2 norm reduces the magnitude along each dimension of popular product and normalizes for global popularity.
In this approach, since we have aggregated user signals across S, it results in the system being agnostic of fashion trends. Unlike the item graph approach, we find multiple instances of linkage of products across time. Hence, while finding similar products for a product pi using this approach, we find that the output products do not necessarily reflect trends from the same time-frame as when pi was present in the system. 3 3.1
We split our sessions data into two non overlapping sets, a training set, which is used to generate the similar products, and a test set, which is used to evaluate the approaches. On an average, each session contains 5 products. The training set consists of ∼200M sessions with ∼12M unique users. Using this training set, we find similar products for ∼1.3M products. The test set is generated from ∼10M sessions with ∼500K unique users. To generate the test set of products for pi , we take each session in the test data, where
(0.712) (0.702) (0.588) (0.473) (0.405) pi was clicked, and we consider the product which was clicked immediately after pi in this session. Aggregating across all the sessions in the test data, we create a set of products which were clicked immediately after pi . Using the session test data, we were able to generate test sets for ∼330K products, with the average size of the set being 164.
We use a series of MapReduce computations [
3.2.1 Browsing behaviour by Gender. To analyse browsing behaviour by gender, we look at density statistics from both the item-item graph and product-user vectors.
For item-item graph, we define, the density of a graph as e d =
n ∗ (n − 1)/2 where e is the number of edges and n is the number of nodes. Density here represents the ratio of number of edges formed in a graph to all possible edges in the graph. Below we tabulate density statistics of graphs for some prominent article types
Men-Tshirts Women-Jeans Men-Jeans Women-Casual Shoes Men-Casual Shoes Women-Sports Shoes Men-Sports Shoes
As can be observed from the graph, for two comparable article types like Men-Jeans and Women-Jeans, the women article types have higher density as compared to men article types, despite women article types having lower number of nodes. It can be inferred from the higher density of women article types that women browse more products than men before making a selection.
For product user vectors, density of a sparse vector is defined as the ratio of number of non-zero values in a vector to it’s length. In the table below, we tabulate density statistics for vectors of diferent article types.
Men-Tshirts Women-Jeans Men-Jeans Women-Casual Shoes Men-Casual Shoes Women-Sports Shoes Men-Sports Shoes
Similar to item-item graph, we notice that women article types are more dense as compared to men products. The higher density of women product vectors corroborates our earlier inference that women products are browsed more as compared to men products.
3.2.2 Diference in Style Catalogued Date . Each product in our system is tagged with a style catalogued date (SCD), which represents the date when the product was introduced to the system. We calculate the average diference between SCD of a product and it’s similar products and further average it over an article type.
Fig 8 represents average diference in SCD, between input products and their similar products averaged over article types for the two approaches. It can be observed from the plot that average difference in SCD for product-user vectors is 2-3 times more than that of item-item graph. Hence, it can be inferred from the plot that similar products found using item-item graph are from the same time-frame, while similar products found using product user vectors link across time. This validates our claim that similar products found using item-item graph are cohesive of fashion trends, while those found using product user vectors are agnostic of them. 3.3
3.3.1 Precision Scores. Denoting the test set of pi by Ti and the set of similar products for it as Ri , we use the following formulation to calculate precision: precision = |Ri ∩ Ti | |Ri |
The size of Ri is a hyper-parameter, since we can set the number of similar products that we want to find for pi , and we denote it as N.
(0.151) (0.112) (0.108) (0.107) (0.100) Fig 4: Precision versus N for the three approaches Fig 5: Recall versus N for the three approaches Fig 6: Precision for diferent article types Fig 7: Recall for diferent article types
Fig 4 shows the average precision of the three approaches for diferent values of N. Fig 6 shows precision by article type for the three approaches calculated with a value of N as 25.
3.3.2 Recall Scores. Going with the same notations as above, we use the following formulation to calculate recall: recall = |Ri ∩ Ti |
|Ti |
Fig 5 shows the average recall of the three approaches for different values of N. Fig 7 shows recall by article type for the three approaches, with the same value of N as for Fig 6 i.e. 25.
It is evident from Fig 4-7 that the two collaborative filtering approaches significantly out perform the content based approach. Among the two item-item collaborative filtering approaches we calculate from the numbers of Fig 4 & 5 that product user vectors show an average improvement of 7% and 5.1% over item-item graph for precision and recall, respectively. Also we observe that product user vectors have better precision recall numbers across majority of article types as compared to item-item graph.
3.3.3 A/B Tests. For the A/B test, we render the similar
products on the product details page of the input product. We test the
above three approaches by assigning randomly selected 10% of
trafifc to each treatment ( 100k users for each treatment). We recorded
the number of product views and the number of clicks for the three
approaches over a period of three weeks. Figure 9 shows the CTR
(click through rate defined as ratio of clicks to views) recorded
during the test period for the three approaches averaged over each day
of the week. From the test, we observe that product user vectors
show an average CTR improvement of 5% over item-item graph
with a p-value[
In this paper, we have identified and addressed some of the challenges faced by item-item collaborative filtering approaches which are unique to fashion e-commerce domain. We formulate a new method to combine and quantify various user signals, which can be used as input to these approaches, in e-commerce domain.
We propose a new method to evaluate similar products in an ofline setting. We compare and evaluate three approaches to discovering similar products in both ofline and online tests. We observe a significant improvement in the performance of item-item collaborative filtering approaches as compared to the content based approach. Furthermore, among the two item-item collaborative ifltering approaches product user vectors perform noticeably better than item-item graph. We also compare the efects of fashion trends with respect to the two approaches and based on results of the tests, infer that a user’s preference to products is independent of rapidly changing fashion trends. 5
We thank Deepak Warrier, Ankul Batra, Sagar Arora, Ullas Nambiar, Ashay Tamhane and Kunal Sachdeva for their contributions Fig 8: Average diference in SCD by article type Fig 9: Average CTR over a period of 3 weeks in reviewing this work and their inputs to algorithm design and evaluation.