This paper addresses the challenge of extreme multi-label query classification (XMQC) in e-commerce, where short, ambiguous queries must be categorized into a vast label space to improve search relevance. We propose a supervised attention-based neural network framework that leverages clickstream data for automatic multi-label query annotation. Our approach employs a DistilBERT language model fine-tuned with a sparsemax loss function to efectively model the sparse category distribution for each query. Experiments on a real-world e-commerce dataset demonstrate that our model outperforms baseline approaches, achieving 78.96 precision@1, 73.75 recall@1, and 78.21 nDCG@1 overall. The sparsemax loss enables the model to handle label sparsity and ambiguity, with strong performance on head, torso, and tail queries. Qualitative analysis shows the model's robustness to challenges like misspellings and the ability to identify relevant categories for ambiguous queries.
E-commerce platforms ofer customers access to catalogs with millions of products categorized
into taxonomies and thousands of hierarchically organized categories. Products are continuously
updated through inventory feeds from partners and suppliers, including images, attributes, and
other metadata utilized to populate the product detail pages. In such a dynamic environment,
product search engines must continuously reindex products and optimize relevance through
users’ behavioral signals when available [
Both in conventional and semantic product search [
This presents several challenges. Firstly, due to the large, unbalanced label set and ambiguity, a QC model must be framed as an extreme multi-label, multi-class classification problem in which a query can simultaneously belong to more than one non-mutually exclusive class. The model loss function must consider the multi-labels’ sparseness and allow the model to return a sparse posterior probability where some classes can have zero probability while others can represent a probability distribution among the sparse labels that sum to one.
Secondly, as we frame the task as a supervised learning problem, a substantial amount of high-quality annotated data is required. Since manual data annotation is expensive, time-consuming, and non-scalable for e-commerce trafic volumes, click-stream data that captures the user’s behavioral signals is the only viable option for annotating data for a supervised model. However, labels need to be carefully denoised to represent query categories accurately.
Finally, data partitioning must respect the natural distribution across classes and labels. Conventional stratified sampling used in single-label classification tasks must be revised to consider stratified sampling in a multi-label scenario, where the sparse label distribution is preserved across training, validation, and test data.
To overcome these challenges, we propose a supervised attention-based neural network
framework that leverages clickstream data to automatically annotate query search data using the
user’s behavioral signals in a multi-label fashion. To preserve the correct label distribution, we
adopted a multi-label stratified sampling technique [
We demonstrate that the above process is significantly better than multiclass models, reporting robust generalization capabilities for torso and tail data. The resulting sparse category distribution can also capture ambiguous queries that apply to a broader range of categories and generic queries that may be better represented by intermediate nodes in the taxonomy tree.
The rest of the paper is organized as follows. Section 2 provides an overview of query classification in e-commerce. Section 3 formulates the extreme multi-label query classification (XMQC) problem. Section 4 discusses multi-label sampling and probability estimation techniques. Section 5 details the experimental setup, including dataset preprocessing, evaluation metrics, and model configuration. Section 6 presents the main results, analysis, and discussion. Section 7 reviews related work, and Section 8 concludes the paper and suggests future research directions.
When searching for a particular item on an online retail platform, a user might input a search query briefly describing the product. Depending on how closely the search results align with the user’s initial intent, the user may click on a relevant product, modify their query to refine the results, or leave the site if the displayed products are not accurately related to what was expected.
Manually classifying user queries into product categories is challenging and time-consuming.
This dificulty arises from the complex interpretation of user intentions based on brief query
texts and the large number of categories found in an e-commerce catalog, which can easily reach
several thousand classes [
Using behavioral signals like clicks, adding items to the cart, and completing purchases ofers a practical method for automatically creating category labels. Annotating query classification datasets with behavioral signals suggests that a given query might interact with multiple taxonomy labels (e.g., product categories) to some extent. This interaction with various product categories could be viewed as a distribution of probabilities over the labels corresponding to a specific query. Approaching the issue as a standard multi-class problem allows each label to be treated independently from others, where they cannot coexist and have probabilities of zero or one in the training dataset. However, this approach fails to accurately represent real-world data by overlooking crucial insights into the ambiguous nature of search queries that can simultaneously pertain to multiple taxonomy category labels.
Figure 1 shows three product search query examples for home improvement and grocery domains from two large e-commerce organizations. For instance, in the home improvement domain, the query “number stencils for painting” would have relevant products in the category Stencils at the third level of the taxonomy tree, under Signs, Letters & Numbers and Hardware. A more realistic view of the problem should also consider the interaction with other labels in the taxonomy tree. Figure 1 also shows the categories that are selected less frequently but are still a legitimate category since number stencils can also be categorized as Craft Supplies under the broader Paint category.
Similarly, in the grocery domain, the query “chicken broth organic gluten free” may refer to products that are sharing the top two categories Canned Goods & Soup Stocks / Broth & Bullions but with relevant products in both leaf nodes Broth & Stocks and Bouillon & Base.
Yet, simply considering the presence of multiple labels is not suficient to correctly represent a query classification prediction model. A given query that has an interaction of 1% with label and 99% with label would be considered in the same way a query ′ that has 99% interaction with label and 1% interaction with label , producing a skewed prediction where the minority label could take precedence on the more popular usage of the query. This is particularly important when the predicted query labels are used as input features to optimize (or re-rank) a search result returning matching products from a catalog. Considering the first example in Figure 1, a search engine could return a majority of products from the Craft Supplies minority class rather than boosting results from the Stencils category, compromising the actual result relevance and potentially missing product conversion opportunities.
The image in Figure 2 illustrates the potential benefits of integrating query understanding capabilities, such as query classification, into an e-commerce search engine. By accurately classifying the intent behind the query “organic bananas”, the search engine can significantly improve the relevance of the results.
Without query understanding (left side), the search engine relies primarily on lexical matching, focusing on the individual terms “organic” and “bananas” independently. This leads to the retrieval of less relevant items such as organic baby food, tea, and onions, which happen to contain the word "organic" but are not related to the core intent of the query. The lack of understanding of the query’s true meaning results in a suboptimal user experience.
In contrast, when query understanding is integrated (right side), the search engine can classify the query “organic bananas” as belonging to the category of fruits. This deeper understanding allows the engine to prioritize and boost categories and products directly related to organic bananas, such as fresh bananas, plantains, mangoes, and other fruits. By leveraging the query classification, the search engine can assign higher relevance scores to these categories and products, pushing them to the top of the search results.
Besides query classification in the e-commerce domain, other domains have similar challenges.
For example, movies can have more than one genre label, and each label can also contribute
a diferent weight to the overall movie genre. Negative online behaviors classification, which
has been recently getting attention to improve social media and online content quality [
Extreme multi-label query classification (XMQC) tries to find the most relevant subset of class labels associated with a short query text from an extremely large number of categories.
In the XMQC problem for e-commerce, we are given a set of queries = {1,2,...,},
where each query is represented by a -dimensional feature vector x ∈ R. The feature
vector x can encode various attributes of the query, such as the textual content, user context, etc.
Additionally, we have an extremely large set of product categories = {1,2,...,}, where
typically ≫ and can be in the order of hundreds of thousands or millions in e-commerce
applications [
=1 ≥ 1.
The objective is to learn a classifier : R → {0,1} that accurately maps each query feature vector x to its corresponding label vector y^, i.e., (x) = y^ ≈ y.
However, instead of predicting the binary label vector directly, we aim to predict a sparse label distribution p ∈ Δ− 1, where Δ− 1 := {p ∈ R|1 p = 1,p ≥ 0} is the − 1 dimensional simplex. Each entry represents the probability of query belonging to category .
The − 1 dimensional simplex is a geometric object that represents the set of all probability distributions over discrete categories. Using the simplex is necessary for representing sparse label distributions, capturing label uncertainty, enabling the use of appropriate loss functions, and facilitating the application of label embedding techniques to handle the extremely large number of categories in e-commerce applications.
The objective is to learn a classifier : R → Δ− 1 that maps each query feature vector x
to its corresponding sparse label distribution p^, i.e., (x) = p^ ≈ p. This can be achieved by
minimizing a loss function that measures the discrepancy between the predicted label distribution
p^ and the ground truth distribution p, e.g., using KL divergence or sparsemax loss [
The learned classifier will predict the relevant categories for a given query , taking into account the label distribution’s sparsity. The label embedding approach helps reduce the computational complexity and mitigate the data sparsity issue associated with the extremely large label space.
To address the challenges of extreme multi-label query classification, we propose a
distillationbased approach using a pre-trained DistilBERT model [
DistilBERT is a distilled version of BERT [
In our approach, we first fine-tune the pre-trained DistilBERT model on the query-category pairs using the sparsemax loss function. The sparsemax loss is a sparse alternative to the softmax loss that encourages the model to predict sparse probability distributions. The sparsemax loss is defined as: sparsemax(;) = − + 12 ∑︁ (2 − 2())+ 21
∈() where is the input vector, is the target label, () is the support set of sparsemax(z), and () is the threshold function given by: () = ∑︀≤ ()() − 1
() with (1) ≥ (2) ≥ ... ≥ () being the sorted coordinates of , and () = max{ ∈ [] | 1+() > ∑︀≤ ()}.
The sparsemax loss has several desirable properties, including convexity, diferentiability
everywhere, and a connection to the Huber classification loss in the binary case [
During fine-tuning, we minimize the sparsemax loss between the predicted label distribution and the ground truth sparse label distribution. This allows the model to learn to predict sparse label distributions that align with the true label sparsity in the XMQC task.
During inference, we apply the sparsemax transformation to the output logits of the fine-tuned DistilBERT model to obtain a sparse label distribution over the categories. We then select the categories with non-zero probabilities as the predicted labels for the query.
By leveraging the pre-trained DistilBERT model and fine-tuning it with the sparsemax loss, our approach can efectively handle the extreme multi-label classification problem while being computationally eficient. The use of a pre-trained model allows us to benefit from the knowledge already distilled into the model, while the sparsemax loss encourages the prediction of sparse label distributions, which is crucial for XMQC.
Sechidis et al. [
Iterative stratification aims to ensure that the ratio of positive to negative examples for each label is approximately maintained in each subset of the data. It does this by greedily assigning examples to subsets based on the desired number of positive examples for each label in each subset.
More formally, let be the full dataset, = {1,...,} the set of labels, 1,..., the desired subsets, and 1,..., the desired proportion of examples in each subset. The algorithm calculates the desired number of examples in each subset as = || , and the desired number of positive examples for each label in subset as = || , where is the subset of containing positive examples of label .
The algorithm then iteratively assigns examples to subsets. In each iteration, it considers the label with the fewest remaining positive examples, and for each example of this label, assigns it to the subset that maximizes the current desired number of positive examples for this label .
The Labels Distribution (LD) measure is used to evaluate how well the ratio of positive to negative examples for each label is maintained in each subset compared to the full data. It is defined as: = 1 ∑︁⎛⎝ 1 ∑=︁1⃒⃒⃒⃒⃒ |||−||| − |||−||| ⃒⃒⃒⃒⃒ ⎞⎠
=1 where is the subset of containing positive examples of label .
The paper empirically shows that iterative stratification achieves lower LD (i.e., better label distribution) compared to stratification based on distinct label sets, especially when the ratio of distinct label sets to examples is large. This makes it particularly suitable for XMQC problems.
In the XMQC problem, each query is associated with a sparse label distribution pi ∈ Δ− 1, where represents the probability of query belonging to category . To estimate these probabilities, we leverage the click data associated with each query-category pair.
Let be the number of clicks received by category starting from query , and let = ∑︀=1 be the total number of clicks starting from query across all categories. We . estimate the probability as: =
For example, consider the query = “wood for crafts” in Table 1. The total number of clicks for this query is = 197. The category Outdoors/Outdoor Games & Toys/Kids Tools & Building Kits/Toy Miniatures received = 99 clicks, resulting in an estimated probability of = 19997 = 0.502538. Similarly, the category Paint/Craft Paint & Supplies/Craft Supplies received = 49 clicks, leading to an estimated probability of = 14997 = 0.248731.
This approach allows us to obtain a distribution over labels for each query, rather than relying on one-hot label vectors. The estimated probabilities capture the strength of association between query and category based on user click behavior.
To reduce label noise and ensure consistency, we apply a heuristic-based filtering approach. We remove labels with < 0.1, as the clicks for these labels are too sparse and may not reliably represent the query’s intent. Similarly, we remove queries with max < 0.4, as these queries lack a consistent category association and may introduce noise into the training process.
By estimating the label probabilities based on click data and applying noise reduction techniques, we obtain a sparse label distribution p for each query . This distribution provides a more informative representation of the query’s category associations compared to binary label vectors, capturing the uncertainty and multiplicity inherent in the XMQC task.
The sparse label distributions p serve as the target probabilities for training the DistilBERT model using the sparsemax loss, as described in the previous section. By learning to predict these sparse distributions, the model can efectively handle the extreme multi-label nature of the problem and provide more accurate category recommendations for queries.
Category
Flooring/Tile & Tile Accessories/Tile Home Decor/Furniture/Furniture Parts/Table Tops Home Decor/Wall Art & Decor/Wall Art Storage & Organization/Shelves & Shelving/Wall Mounted Shelving Building Supplies/Lumber & Composites/Plywood & Sheathing/Plywood Moulding & Millwork/Moulding/Crown Moulding
Moulding & Millwork/Moulding/Window & Door Trim/Window & Door Moulding
The data used for the experiments was collected from the clickstream of an online store of a large home improvement retail corporation. The dataset consists of query-category pairs, where the labels are inferred based on the frequency of user clicks on products belonging to specific taxonomy categories. In total, there are 4,462 categories in the taxonomy tree with diferent levels of nesting.
Each data instance consists of three elements: 1) the search query typed by the user , 2) the label referring to the product category related to the query , and 3) the number of clicks associated with the query-category pair in a specific time interval. Since a query can lead to clicks on products from diferent categories, queries are duplicated per label, and the total number of clicks for a query = ∑︀
=1 is also computed, where is the number of categories for the query (see also example in Table 1).
Following the same approach as described in [
Table 2 provides a description of the dataset in terms of query and label counts, as well as their distribution among head, torso, and tail queries. When considering the entire dataset, the majority of unique queries (86.4%) are associated with a single label. However, the distribution of single-label and multi-label queries varies significantly when examining head, torso, and tail queries separately.
In e-commerce data, tail queries often form a long tail distribution. By definition, tail queries have a click count of 1 and are, therefore, all single-label. For torso queries, the percentage of single-label queries drops to 66.2%, and for head queries, it further decreases to 8.5%.
Head queries represent 69% of total trafic, with multi-label head queries accounting for 67.9%. Combined, multi-label head and torso queries comprise 81.1% of total clicks (112,742,307), indicating their high relevance for conversion rates. Tail queries, on the other hand, are known to be
Total Head Torso Tail 138,965,331 96,233,946 33,705,963 9,025,422
100.0% 69.3% 24.3% 6.5% 112,742,307 94,425,519 18,316,788 0
81.1% 67.9% 13.2% 0.0% 14,841,471 102,619 5,713,405 9,025,422 18,234,809 647,811 8,559,713 9,025,422 12,818,844 8,748 3,784,674 9,025,422 86.4% 8.5% 66.2% 100% 3,121 3,102 3,115 3,109 2,992 2,057 2,865 2,869 challenging to classify due to their low frequency. Text preprocessing techniques are often applied to tail queries to reduce their variability and improve classification performance (see Section 5.2.1).
The query classification model is intended for the classification of unseen data. From Table 2, we observe that only 3,121 out of the total 4,462 categories are represented in the dataset. The number of well-represented categories, arbitrarily defined as those with more than 50 unique queries, is even lower. To increase the number of well-represented categories and mitigate data sparsity, we apply label pruning and aggregate queries on parent categories, as described in the following sections.
5.2.1. Preprocessing Product search queries often contain various extraneous characters and information that may not be directly relevant to the classification task. To reduce data noise and space dimensionality, it is beneficial to apply several preprocessing and normalization steps to the query data. These steps typically include measurements normalization, punctuation normalization and removal, non-ASCII characters removal, mixed alphanumeric tokens replacement, numeric tokens replacement, and lowercasing. The specific preprocessing steps and their order have been calibrated for the home improvement domain, where measurements, quantities, and specific SKU numbers are frequently used in search queries. The preprocessing pipeline has been designed and fine-tuned based on empirical analysis and domain knowledge to ensure optimal results for this particular domain. After preprocessing, the cleaned and normalized query data is ready to be used as input to the DistilBERT model for training and inference. 5.2.2. Noise Removal Search engine queries are not always directly related to product searches, and consequently, they may not correspond to any specific product category. To remove frequent non-product queries and improve the quality of the training data, we apply two threshold-based filtering techniques: label removal threshold and query removal threshold.
For multi-label queries, such as the example shown in Table 1, we remove all category labels with an estimated probability below the label removal threshold (). This step helps to Bathtub & Shower Caddies Hanging Shower Caddies Bathroom Bathroom 0 Accessories & Hardware Toilet Lid 31 Decals Pool Alarms
38 Pool Filtration0 & Skimmer
Systems Pool Skimme3r8
Systems
Outdoors Pools
Pool 68 Maintenance
Pool 0 Maintenance Equipment
Pool 16 Sanitation Systems
14 16 Pool Chemical14 Monitoring Systems eliminate labels that have a weak association with the query based on user click behavior.
If none of the remaining category labels for a given query has an estimated probability higher than or equal to the query removal threshold (), the entire query is removed from the dataset. This step helps to filter out queries that lack a strong association with any product category, even after removing weakly associated labels. The values of the label removal threshold and query removal threshold were determined through experimental tuning, with set to 0.1 and set to 0.4. These thresholds were found to efectively remove non-product queries while retaining relevant query-category pairs.
Referring back to the example in Table 1, the noise removal procedure would remove the last two rows, corresponding to the categories Moulding & Millwork/Moulding/Crown Moulding and Moulding & Millwork/Moulding/Window & Door Trim/Window & Door Moulding, as their estimated probabilities are below the label removal threshold of 0.1. However, the query “wood for crafts’‘ would be retained in the dataset because it has at least one category label (Outdoors/Outdoor Games & Toys/Kids Tools & Building Kits/Toy Miniatures) with an estimated probability above the query removal threshold of 0.4.
The noise removal procedure efectively filters out non-product queries, such as URLs,
frequently asked questions, commands, and chitchat (e.g., "we are," "sign out," "change store’‘ ).
5.2.3. Labels pruning
To reduce data sparsity for the category labels with less frequent clicks, a large catalog taxonomy
tree can be pruned to increase the density of less frequent queries. Labels with less than K-tagged
examples (e.g., K=50) can be merged with the upper taxonomy node and their labels are replaced
with the upper-level taxonomy label (see Figure 3). For each label in the taxonomy tree, the
number of examples per node is tracked to capture the real queries distribution. After applying
the pruning procedure, every leaf in the taxonomy tree will include at least K samples [
In the example illustrated in Figure 3, the taxonomy tree is pruned based on a threshold of K=50 samples per node. The Pool Chemical Monitoring Systems category, which has only 14 samples, is merged with its parent node Pool Maintenance. The Pool Skimmer Systems category, having 38 samples, is first merged with its sibling node Pool Filtration & Skimmer Systems, which is initially empty because it is not a leaf of the tree. The combined category now has 38 samples and is further merged
Total Head Torso Tail 129,130,839 87,715,635 32,395,413 9,019,791
100.0% 67.9% 25.1% 7.0% 38,198,328 26,511,025 11,687,303 0
29.6% 20.5% 9.1% 0.0%
14,719,934 95,505 5,604,638 9,019,791
16,699,427 133,475 7,546,161 9,019,791
13,030,572 64,234 3,946,547 9,019,791
88.5% 67.3% 70.4% 100%
2,964 2,774 2,964 2,964
2,964 656 2,795 2,869
with the parent node Pool Maintenance. The Pool Sanitation Systems category, which has only 16
samples, is also merged with Pool Maintenance. After these merging steps, the Pool Maintenance
category accumulates a total of 68 samples, surpassing the required threshold of 50 samples per node.
On the other hand, the Toilet Lid Decals category, which has only 31 samples, is merged with its
parent node Bathroom Accessories & Hardware. However, even after merging, the combined category
still does not reach the threshold of 50 samples. As a result, the Toilet Lid Decals category is
eliminated from the pruned taxonomy tree. After applying the pruning procedure, every leaf in the
taxonomy tree will include at least K samples, ensuring a minimum density of examples per category
label. Categories that fail to meet the threshold after merging with their parent nodes are removed
from the tree. This process helps to mitigate the issue of data sparsity for less frequent queries and
enables more efective training and prediction in the extreme multi-label classification setting.
5.2.4. Data split procedure
After preprocessing and pruning, the data is split into training, development, and test folds
using a K-fold stratified partitioning procedure for multi-label data as described in [
The adapted procedure allows weights to be used instead of raw query counts to compute the fold label requirements. Since a query can have multiple labels, each of these is multiplied by the query weight and added to the total label count. During the data splitting, the query weights are deducted from the fold label requirement values. The procedure ensures that the queries with higher weights are distributed first, thus maintaining the distribution of head/torso/tail queries across the folds. As a result, the data is split such that it keeps the folds disjoint in terms of samples while maintaining the same label distribution. This approach helps to mitigate the issue of missing labels or underrepresented classes that can occur when using the more traditional random sampling process to split data folds by taking into account the importance of each query-category pair based on user click behavior.
The data description after the application of the preparation steps is presented in Table 3. Notable changes include the overall decrease in the number of head queries and a significant increase in single-label head queries (from 8.6% to 67.3%). There is also a reduction in the number of multi-label queries and a decrease in the number of categories with 50+ unique queries among head queries, indicating shifts in category popularity. For the experimentation, the data was split into training (95%), development (2.5%), and testing (2.5%) folds.
We employ several evaluation metrics to assess the performance of our proposed XMQC approach, considering various aspects such as precision, recall, and ranking quality1. Let y ∈ {0,1} be the ground-truth label vector and p^ ∈ R be the predicted score vector for query .
Precision at K (P@K) measures the proportion of relevant categories among the top K predicted categories, while Recall at K (R@K) measures the proportion of relevant categories found in the top K predictions. They are calculated as follows: 1 ∑︁ 1 ∑︁
∈topK(p^) || ∈top-K(p^) where top-K(p^) represents the set of top K predicted categories, is the ground truth label, and is the set of true categories for query .
Normalized Discounted Cumulative Gain (nDCG@K) is a ranking-based metric that considers the position of relevant categories in the top K recommendations. It is calculated as: nDCG@K = DCG@K DCG@K = ∑︁ 2 − 1
IDCG@K =1 log2(+1) where DCG@K is the Discounted Cumulative Gain at K, and IDCG@K is the Ideal Discounted Cumulative Gain at K, representing the maximum possible DCG@K score. These metrics provide a comprehensive evaluation of the proposed XMQC approach, assessing its ability to recommend relevant categories, considering precision, recall, and ranking quality.
We trained our proposed model using the DistilBERT pre-trained language model [
The model was trained for 18 epochs using an Adam optimizer with a learning rate of 1e-5 and an
epsilon value of 1e-8. We employed the sparsemax loss function, which is a variation of the softmax
function that encourages sparse output distributions [
We also trained two baseline models for comparison: a DistilBERT model with the
crossentropy loss function and a fastText model. FastText is a lightweight text classification model
that represents documents as an average of their word embeddings and uses a linear classifier
[
To further analyze our model’s performance, we evaluated it on diferent parts of the query distribution: head, torso, and tail. Table 5 shows the comparison of metrics for these query subsets. The model achieves higher precision, recall, and nDCG scores for head queries compared to torso and tail queries. This is expected, as head queries have better query-category relevance.
However, our model still performs well on torso and tail queries, despite their lower query-category relevance due to less customer behavior information for identifying relevant categories. This can be attributed to the sparsemax loss function, which helps the model to focus on the most relevant categories for each query, even when the number of relevant categories is small. By encouraging sparsity in the output distribution, the sparsemax loss enables the model to handle the ambiguity and uncertainty present in torso and tail queries more efectively than the cross-entropy loss.
These results support our initial hypothesis that a sparse label model better addresses the query classification ambiguity in the head and torso distribution. By inducing sparsity in the label space, the sparsemax loss allows the model to make more confident predictions for queries with multiple relevant categories. Furthermore, the model’s ability to focus on the most relevant category for tail queries, where there is typically only one relevant category per query, highlights the benefits of the sparse labeling approach in handling the long-tail distribution of queries in e-commerce search.
The multi-label model could also help to identify specific use cases where the predicted posterior sparse probabilities provide further insights about the user’s intent. For instance, table 6 shows the query “leona silver”, which is a misspelling of the brand “Leonia Silver”. Despite the Query distribution
Query leona silver brrom 616295 lighting for ceiling spelling error, the model correctly identifies the most relevant categories, such as Flooring/Tile & Tile Accessories/Tile and Flooring/Tile & Tile Accessories/Tile Samples, with high probabilities. Similarly, for the misspelled query “brrom”. This demonstrates the robustness of the sparse label model in handling misspellings and variations in brand names.
For the user query specified as product SKU number, “616295”, the model assigns relatively uniform probabilities to multiple categories across the taxonomy, indicating that the SKU number does not provide enough semantic information to confidently predict a specific category.
For the query “lighting for ceiling”, the model assigns the highest probabilities to categories related to ceiling lighting, such asLighting & Ceiling Fans/Ceiling Lights/Flush Mount Lighting and Lighting & Ceiling Fans/Ceiling Lights/Pendant Lighting.
These examples showcase the strengths of our proposed approach in handling diverse query types, including misspellings, SKU numbers, and queries that map to generic taxonomy nodes. However, there are still opportunities for improvement. For queries lacking clear semantic information, such as SKU numbers, incorporating additional context from user sessions or product metadata could potentially enhance the classification performance. Additionally, the model’s ability to handle extremely rare or unseen queries can be further investigated and improved through techniques like few-shot learning or data augmentation.
Recent work on query classification in e-commerce has focused on leveraging hierarchical
category structures to improve representation learning and address challenges such as class imbalance
and query ambiguity. HCL4QC [
Ahmadvand et al. [
In contrast, our work emphasizes the multi-label aspect of query classification, recognizing that a product may belong to multiple categories simultaneously. We employ a sparse label approach using the sparsemax loss to efectively handle query ambiguity and data sparsity in XMQC.
Although large language models (LLMs) are seen as the future of e-commerce search [
In this paper, we presented a novel approach for extreme multi-label query classification in ecommerce using an attention-based neural network optimized with a sparsemax loss function. Our experiments on a large-scale e-commerce dataset validated the efectiveness of modeling queries as a sparse distribution over product categories. The proposed DistilBERT model with sparsemax loss outperformed baseline classifiers, demonstrating significant improvements in precision, recall, and nDCG across diferent query types. The sparsemax loss proved crucial for handling the sparsity and ambiguity inherent in XMQC, enabling the model to focus on the most relevant categories for each query. Notably, our approach showed strong performance on both frequent head queries and less common torso/tail queries, underlining its ability to capture meaningful category associations even with limited training data. Qualitative analysis highlighted the model’s robustness to real-world challenges such as misspellings and its capability to identify pertinent categories for ambiguous or broad queries. These findings underscore the practical value of our XMQC framework in enhancing query understanding and search relevance in e-commerce. Future research could explore incorporating additional context, addressing data sparsity through techniques like few-shot learning, and optimizing inference eficiency for real-time query processing.
We want to thank Amirhossein Tebbifakhr for the extensive data analysis and contribution to modeling ideas, Danilo Giulianelli real-time optimizations in the production system, and Federico Marinelli for an early proof of concept.