In this preliminary work, we present an approach for the augmentation of clustering with natural language explanations. In clustering there are 2 main challenges: a) choice of a proper, reasonable number of clusters, and b) cluster analysis and profiling. There is a plethora of technics for a) but not so much for b), which is in general a laborious task of explaining obtained clusters. We propose a method that aids experts in cluster analysis by providing an iterative, human-in-the-loop methodology of generating cluster explanations. In an illustrative example, we show how the process of clustering on a set of objective variables could be facilitated with textual metadata. In our case, images of products from online fashion store are used for clustering. Then, product descriptions are used for profiling clusters.
or entities difer from each other and that they
could be divided into distinct classes. Clarification
Data analysts and data scientists are often faced of the diferences between groups gets better and
with the task of describing phenomena in a way better along with the knowledge gained about the
that is understandable to the audience. On the one instances that form diferent groups. Finally, one
hand, they have some data that they can describe is giving names to those categories of entities. In
statistically, categorize, or predict events based on essence, clustering in machine learning is no diferent
it, etc. On the other hand, they want to deliver process.
their observations in some kind of narrative that Clustering is an intrinsically subjective task and
they can "sell" to decision-makers. There is even a requires human assessment [
Assigning labels to groups of similar objects is From our expertise in e-commerce and Industry
one of the ways how humans describe the world [
The more objective the data is, the more it is and tries to incorporate the knowledge in the next suited for modeling the phenomena, be it physi- iteration. cal, business, sociological or psychological in nature. Explainable AI approaches have become particuMetadata is more suitable for explaining the model larly important, and although most work is generally to the user, convincing her or him, and prompt- focused on supervised learning, some works have ing to make decisions and actions based on this been done to explain clusters. One of the most comknowledge. It is more prone to error because of its mon methods for understanding clustering methconventionality and subjectivity, but they speak to ods is visualization. By using low-dimensionality humans. embedding and displaying them in two- or three
In this work we propose a method that allows for dimensional dimensions, one can get an overview of
clustering dataset with objective data, and explain the clusters and their data. However, these
visualdiferences between clusters with metadata. We use izations are not always understandable and
explainXAI methods to explain diferences between clusters able.
using metadata which is perfectly understandable The decision tree is one of the inherently
interby humans, but may not be of suficient quality pretable algorithms. So one common way to explain
to perform valid clustering. The selection of the models is to use decision trees. Nevertheless, the
most interpretable metadata is iterative and human- critical point for explaining the decision tree is its
guided. In our example, we show how image-based depth because decision trees with high depth no
clustering can be enhanced with textual description longer are interpreted, so we must pay attention
of clusters. We argue that such an approach can to the depth of the tree produced. Using a small
lead to better utilisation of metadata for cluster decision tree to divide a dataset into k clusters
proanalysis purposes, which results in better under- vides explainable clusters, but this approach has a
standing of clusters which is the final goal of every trade-of between being explainable and accuracy.
clustering task. Furthermore, it allows for check- IMM algorithm [
The rest of the paper is organized as follows. In Besides, visualization or providing some
condiSection 2 we present current research in the area tions on features, using text data is reasonable to
of interactive clustering and human-guided cluster generate explanations to users. In [
Similarly, in many other methods that aim at
explaining diferences between discovered clusters,
the clustering task is transformed to classification
In the survey on interactive clustering [
Another approach is given in [
In all the cases the process is not iterative, nor human-guided. Finally, to the best of authors’ 3.1. Data preparation knowledge, neither of the approaches known in the literature divides data into objective part with a good quality for cluster algorithms, but poor explanation capabilities and metadata with possibly worse potential as clustering features, but better explanation capabilities and possible inconsistencies with objective data that should be fixed. Addressing these issues was the primary motivation of our work that will be described in more detail in the following sections.
The method requires 2 types of data: objective and metadata as defined in the previous section. In "step 0" method provides users with helper functions to prepare both types of data. For objective data there is a function that performs a reduction of dimensionality via SVD followed by normalization. It works on any numerical data, which could be as well as one-hot variables and continuous real values (floats).
User sets the percentage of explained variance left after SVD reduction. The optimal count of new dimensions could be determined automatically by 3. Cluster analysis with metadata our algorithm. This is done by probing diferent dimension counts with scipy.optimize package, so In this section, we will show how our method could the user does not need to do this manually. Regardbe applied to real-case scenarios. We choose an ing metadata which is textual, there are wrappers example from the e-commerce field because the built on top of the SpaCy3 and NLTK4 libraries. authors have experience working in this industry. Users can contact text columns, lemmatize, remove Specifically, we work with online stores to provide stopwords and perform TF-IDF vectorization. For them, among others, with recommendations of prod- numerical metadata, we found a way to incorporate ucts to their end-users (clients). them into textual explanations. For instance, the
In real-life scenarios, data about products is year could be re-coded as the label "year2022", which stored in product catalogs in shop databases, and will be easily interpreted along the pipeline. Other most often exchanged with so-called product feeds numerical variables could be re-coded to low/medi(XML documents). We used a public dataset from um/high bins, based on quartiles. Finally, the user Kaggle1. This dataset in terms of content resem- constructs the "Pipeline" object and initializes it bles a product feed for an online store of a medium with 2 datasets: objective and metadata. size product catalog. It consists of 44000 products with category labels, titles, and images. For the code accompanying this example see the GitHub 3.2. Assistance in clustering repository2. The first step corresponds to running the unsuper
As has been said before, we treat images as ob- vised clustering algorithm. Typically, the person
jective data. We used embeddings of images ob- performing the analysis starts with the dilemma of
tained via MobileNetV2 [
Labels could represent the most important pieces of metadata, such as the label, the observation id, Figure 4: Example images of products that were assigned and summary of description. The next clue is de- to the same cluster 13 based on the objective data. rived from the silhouette score on a plot in Figure 2.
The range of the number of clusters to be tested is provided in accordance with the previous clue. To speed-up computations, this plot could be obtained
metadata, here pruned to level 4.
the result. We would like to give users the freedom decides on its convergence.
to refine explanations. Thus, we provide her or
him with the possibility to influence explanations
by extending stopwords with his own terms. On 4. Summary
the other hand, we initialize the whitelist with
keywords like "year2022", defined in "step 0". Then In this work, we presented the method that allows
we use the TF-IDF vectorizer, taking into account for explaining clusters with concepts that could be
the aforementioned lists. Vectors are used to train more human-readable than the data which was used
decision tree classifiers. The size of the list of ad- as an input to clustering algorithm. We based our
ditional terms is under the control of a user. She method on the observation that diferent types of
or he can change it and interactively observe the data are suitable in diferent degrees to clustering
result in a Figure 6, which gives an insight into what and explaining tasks. We demonstrated the
feasiterms were relevant for classifier. Furthermore, for bility of our approach on the e-commerce example,
every cluster, example observations, word clouds where images were treated as input for clustering
and LIME explanation for description of random and textual descriptions of images as basis for
clusobservation are presented. For instance for the clus- ter descriptions.
ter 13 in Figure 4 one can see example products, In its current version, adaptability to the needs
word clouds that describe clusters in Figure 5 and of a human expert is provided by the possibility of
LIME [
The last stage is a plot of the word cloud of each nation or can directly influence the list of tokens cluster, using the same TF-IDF vectorizer. Plots by adding words to the whitelist or blacklist. After are accompanied by examples of observations. In each such a change, the expert can see how it afects addition, a user is presented an explanation gener- the classifier used as an explainer, both globally and ated with LIME for a random instance from a given at the level of random observations for a cluster. class. The whole process is iterative, and the expert One can modify the scope of the metadata and the way individual observations are presented. If the results, despite the changes, are not satisfactory, it may imply the need to go back to choosing the number of clusters.
We treat metadata as something fixed and given.
From our experience in e-commerce, we know that product descriptions are the result of the work of many e-store employees, but we do not want to interfere with them. For example, a product recommendation system in which the objective data would be product images and the online shopping behavior would be swithced on for a particular ecommerce after the explanation is accepted by a decision maker in the e-store. The explanations could be refined by technical support of the platform before being shown to the e-store employee.
The tool is intended for use by a sole data scientist. However, in other situations, collaborative knowledge engineering approaches are also possible.
Metadata could be created and enhanced in systems
such as LOKI5 [
In future work we would like to improve our method with several extensions. We will focus on automatically proposing number of clusters based on both embedding features with methods similar to T-SNE and metrics like silhouette score. We want to test clustering techniques other than k-means.
For instance, hierarchical clustering could be more suitable in e-commerce, where taxonomies of products are multilayer. Word clouds could be replaced with topic analysis with Latent Dirichlet Allocation or techniques derived from Natural Language Generation. Another interesting direction is to construct explanations with other modalities, like visual, by something more sophisticated than presenting example images. It could be done for instance with image captioning.
Predictive Maintenance) project funded by the National Science Center, Poland under CHIST-ERA programme Grant Agreement No. 857925 (NCN UMO-2020/02/Y/ST6/00070).
The work of Szymon Bobek has been additionally supported by a HuLCKA grant from the Priority Research Area (Digiworld) under the Strategic Programme Excellence Initiative at the Jagiellonian University (U1U/P06/NO/02.16).
The work of Samaneh Jamshidi was supported by CHIST-ERA grant CHIST-ERA-19-XAI-012 funded by Swedish Research Council.
tionally supported by Edrone Sp. z o.o.6, which provided computer resources for machine learning.