Products in an ecommerce catalog contain information-rich fields like description and bullets that can be useful to extract entities (attributes) using NER based systems. However, these fields are often verbose and contain lot of information that is not relevant from a search perspective. Treating each sentence within these ifelds equally can lead to poor full text match and introduce problems in extracting attributes to develop ontologies, semantic search etc. To address this issue, we describe two methods based on extractive summarization with reinforcement learning by leveraging information in product titles and search click through logs to rank sentences from bullets, description, etc. Finally, we compare the precision of these two models.
Many search engine frameworks like Solr [
A naive solution is to ignore these fields completely for search.
While this may improve precision, it would be at the cost of recall,
as relevant information might be lost. Such relevancy problems
are mitigated by having semantic search using methods like query
understanding. However, they require SKUs to have relevant
attributes (atomic phrases that provide more information about an
item [
Our contribution in this paper is, we demonstrate how Extractive Summarization can be used to rank sentences present in product description and bullets using product title and user queries obtained from click through log. One of the benefits of this method is, cost of obtaining training data is cheap and the model can be run on items that have little or no click data associated with it. We also provide comparison of the two models by measuring precision@k of relevant sentences in the summary. 2
Summarization is the process of shortening a text document in order
to create a summary while retaining major points of the original
document. There are two kinds of summarization techniques:
Abstractive and Extractive summarization. Abstractive summarization
involves using internal semantic representation and natural language
generation techniques to create the summary [
Recently, a lot of work has been done on Abstractive Summarization
using attentional encoder-decoder model that was proposed by
Sutskever et. al in [
Extractive Summarization was traditionally done using hand
engineered features, such as sentence position, length [
In this paper, we try to rank sentences using summarization
techniques for the purpose of improving search relevancy. There hasn’t
been lot of work done in this area. One of the work that is aligned
with our objective is from Ryen et. al [
Our objective is to rank sentences in product description and bullets
from a search perspective. Search perspective means that when we
extract attributes from sentences, they are relevant to the item and
are likely to be used in a search query for that item. Methods like
query understanding can benefit from ranked sentences as they use
attributes in SKU to match with the user’s intent. Higher ranked
sentences are more likely to contain relevant attribute than a lower
ranked sentences. Having a set of top ranked sentences would also
help in full text match by avoiding queries to match with irrelevant
sentences. We use Extractive Summarization to achieve this. Our
work is based on [
We use title and queries obtained from click through log as part of the target summary. Title is one of the key fields in ecommerce catalog provided by the merchant, it captures essential information about the item and queries can be thought of as keywords users think are relevant attributes for the product. The intuition is, having them in the target summary would allow the model to capture important sentences present in the description and bullets. We create two models, one that uses just the title as target summary and the second model that uses top five queries that led to clicks on the item, along with the title as target summary.
Finally, we choose top K sentences as determined by the model as our final summary. Since, ecommerce product description tend to be short and less repetitive, the issue of repetition and diversity in not a concern in our summarization task. 3.1
Figure 3 depicts network architecture of the extractive summarizer.
It aims to extract sentences {s1..sm } from a document D composed
of sentences {s1..sn } where n > m and labels them 1 or 0 based
on if they should be included in the summary or not. It learns
to assign a score p(yi |si , D, θ ) to each sentence which is directly
proportional to its relevance within the summary. Here, θ denotes
the model parameter, si denotes the ith sentence and D represents
the document. Summary is chosen by selecting the sentences with
top p(yi |si , D, θ ) score. Our network and the objective function is
based on the paper [
These components are described in detail below:
Sentence encoder is composed of convolutional encoder which
encodes a sentence into a continuous representation and is shown
to capture salient features [
Finally, Sentence extracter sequentially labels each sentence as 1 or 0 depending upon if the sentence is relevant or not. It is implemented using RNN with LSTM cells and a softmax layer. At time ti , it makes a binary prediction conditioned on the document representation and previously labelled sentences. This lets it identify locally and globally important sentences. Sentences are then ranked by the score p(yi = 1|si , D, θ ). Here si is ith sentence, D is the document, θ is the model parameter and p(yi = 1|si , D, θ ) is the probability of sentence si being included in the summary. We learn to rank by training the network in a reinforcement learning framework optimizing ROUGE.
We use a combination of maximum likelihood cross entropy loss and rewards from policy gradient reinforcement learning as objective function to globally optimize ROUGE. This lets the model optimize the evaluation metric directly and makes it better at discrimating sentences i.e it ranks the sentence higher if it appears often in the summary. 3.2
Reinforcement Learning is an area of machine learning where a
software agent learns to take actions in an environment to
maximize cumulative reward. It difers from supervised learning in the
way that labelled input/output pairs need not be provided nor are
sub-optimal actions need to be explicity corrected. Rather, the focus
is on the balance between exploration and exploitation. Exploitation
is the act of preferring an action that it has tried in the past and was
found to be efective, whereas exploration is the act of discovering
such actions, i.e. trying out actions that it has not selected before.
We conceptualize the summarization model in a reinforcement
learning paradigm. The model can be thought of as an agent
interacting with the environment, which consists of documents. The
agent reads the document D and assigns a score to each sentence
si ∈ D using the policy p(yi |si , D, θ ). We then rank and get the
sampled sentences as the summary. The agent is then given a
reward based on how close the generated summary is with the gold
standard summary. We use F1 score of ROUGE-1, ROUGE-2, and
ROUGE-L as the reward r . In our case, gold standard summary is
the title and user queries. Agent is then updated based on the
reward using the REINFORCE algorithm [
L(θ ) = − Eyˆ∼pθ [r (yˆ)] Here, pθ stands for p(y |D, θ ), where θ is the model parameter, D is the document and r is the reward.
REINFORCE algorithm is based on the fact that the expected reward function of a non diferentiable function can be computed as: ▽L(θ ) = − Eyˆ∼pθ [r (yˆ) ▽ log p(yˆ|D, θ )] Calculating expected gradient in the above expression can be expensive as each document can have very large number of candidate summaries. It can be approximated by taking single sample yˆ from pθ for each training example in a batch, following which the above expression gets simplified to: ▽L(θ ) ≈ ≈ −r (yˆ) ▽ log p(yˆ|D, θ )
Ín −r (yˆ) i=1 ▽ log p(yˆi |si , D, θ ) Since the REINFORCE algorithm starts with a random policy, and because our task can involve large number of candidate summaries for the document, training the model can be time consuming. So, we limit the search space yˆ with smaller number of high probability samples Yˆ consisting of top k extracts. The way we choose these top k extracts is, we select p sentences which have highest ROUGE scores on its own and then generate all possible set of combination using these p sentences with the constraint that maximum length of the extract can be m. We rank these against the gold summary using F1 score by taking mean of ROUGE-1, ROUGE-2 and ROUGE-L. We choose top k of these ranked summaries as Yˆ. During training, we sample yˆ from Yˆ instead of p(yˆ|θ , D). 3.3
We create two summarization models, one with title as its target summary (Model 1) and the other with title plus top five queries for which the product was clicked as the target summary (Model 2). Title and each query are treated as independent sentences when generating the reference summary. For input, we use product descriptions and bullets for both the models.
We preprocess the title, decription and queries before passing them to the model. Preprocessing step consists of sentence segmentation, tokenization, conversion of tokens into vocabulary id, truncation and padding to a fixed length. We use SKUs from grocery category of our catalog to evaluate the models. For Model 1 we used all the SKUs from the grocery category and for Model 2 we used a subset of SKUs from the category which had engagement above a certain threshold. Though Model 2 had fewer training data, it was richer since it had queries (top 5) associated with each SKU as part of the summary. One advantage of both methods is, it requires almost no manual efort to get the training data, thus is very cheap. Figure 4 describes how the two models are set up for training. Since our objective is to have better full text match or attributes from the ranked set of sentences, each sentence can be independent of each other. This insight is well aligned with the framework of reinforcement learning based extractive summarization that optimizes ROUGE. 4
Tfidf is one of the commonly used frequency driven approches for
weighting terms to measure importance of a sentence for extractive
summarization [
idf(t , D) = tfidf (t , d, D) = tf(t , d) · idf(t , D)
N |{d ∈ D : t ∈ d }| Here, tf(t , d) is the count of the term t in the document d. idf(t , D) is the inverse document frequency. N is the total number of documents in the corpus. |{d ∈ D : t ∈ d }| is the number of documents where the term t appears. If the term is not present in the corpus, it will lead to division by zero. To avoid this, it is a common practice to adjust the denominator to 1 + |{d ∈ D : t ∈ d }|. For baseline, we use tfidf based model. Our baseline consists of three aproaches that utilizes tfidf to score the sentences to select top K. For the first approach, we sum up (unweighted) tfidf score of the words to measure importance of a sentence and then select top K as the summary. Here, tf is computed at the sentence level and idf is across all the SKUs (documents).
For the second approach (weighted), we weigh the tfidf score of tokens in the description that also appear in the title by multiplying it with a factor of wi . The optimal wieght wi was found by using grid search method. In our case, it was found to be 2.
For the third approach (filtered), we sum up the tfidf score of only those tokens in description that appear in the title.
Figure 5 shows precision@k for the three models. As we can see from the graph, the weighted approach has highest precision@k, this shows that the words present in title does indicate which sentences are of relatively higher importance. However, it is also not a right strategy to exclude all the other words, as demonstrated by the higher precision@k of unweighted model over filtered model. Thus, in summary, boosting words present in title while also retaining other words for the computation of tfidf score of a sentence seems to yield best result among all the baseline approaches. 5
Our purpose of ranking is to find sentences that are relevant to the product and contain attributes of the product that customers might use in their search queries. This will improve results of full text match as well as query understanding, since it depends on matching user’s intent with attributes extracted from the SKU. To analyze this, we reviewed 100 SKUs randomly sampled from the grocery category and manually labeled the sentences based on whether they were relevant or not. We evaluated the model using precision@k, with k as 1,2 and 3.
Based on the evaluation of the three tfidf based models as described in the section 4, we chose weighted Model (the second approach)
This demonstrates that words present in title capture key information of the product being sold. Title is provided by the merchant, so it provides merchant’s point of view regarding what aspect of the product is important. Whereas, words present in user queries indicate the attributes of product that the user cares about. So combining these two sources of information is a good way to infer relevant sentences of description from a search perspective. Also, since not all SKUs (documents) have user clicks or may have comparatively less engagement data associated with it, creating a model leveraging title and click through log to find relevant sentences provides a way to generalize it to SKUs (documents) that have little or no engagement data.
We provide one instance from our evaluation set as an example. Figure 7 shows a sample product description that is fed to the model. Figures 8 and 9 show output of Model 1 and Model 2 respectively. Sentences that have keyword stufing tend to be grammatically incorrect, structurally dissimilar to the title and generally longer. Thus, the intuition is that summarization models described above 6
We implemented a framework to rank sentences from product description & bullets based on Extractive Summarization that uses reinforcement learning to optimize ROUGE and maximum likelihood cross entropy, thus enabling the model to learn rank the sentences. We compare two models, one that uses just the title and the other that uses queries from click through log along with the title. We show that these two models have higher precision in ifnding relevant sentences than the baseline which is a tf-idf based method to select top sentences. Typically, in search engines, such ifelds (product descriptions, bullets etc.) are either ignored or given a very low weight compared to fields like product title. Using this framework that ranks the sentences, we can assign a higher weight to ranked set of sentences. In addition, top N sentences from ranked set can also be used to extract attributes and help build the ontology. Our future plan involves, 1) measuring the precision with two separate models, one for description and one for bullets, as they tend to have diferent grammatical structure 2) investigate the efect of query length on the ranking of sentences 3) have an algorithmic method to decide on the cut of (Top N) for selecting top sentences from each SKU. This is because, as length of the content in each SKU varies, number of relevant sentences could be diferent.