In this work we argue towards a new self-learning engine able to suggest to the analyst good transformation methods and weighting schemas for a given data collection. This new generation of systems, named SELF-DATA (SELF-learning DAta TrAnsformation) relies on an engine capable of exploring different data weighting schemas (e.g., normalized term frequencies, logarithmic entropy) and data transformation methods (e.g., PCA, LSI) before applying a given data mining algorithm (e.g., cluster analysis), evaluating and comparing solutions through different quality indices (e.g., weighted Silhouette), and presenting the 3-top solutions to the analyst. SELF-DATA will also include a knowledge database storing results of experiments on previously processed datasets, and a classification algorithm trained on the knowledge base content to forecast the best methods for future analyses. SELF-DATA's current implementation runs on Apache Spark, a stateof-the-art distributed computing framework. The preliminary validation performed on 4 collections of documents highlights that the TF-IDF and logarithmic entropy weighting methods are effective to measure item relevance with sparse datasets, and the LSI method outperforms PCA in the presence of a larger feature domain.
Data-driven analysis is a multi-step process, in which data scientists tackle the
complex task of configuring the analytics system to transform data into
actionable knowledge. Since collections of textual data are usually characterized
by a high variability, the knowledge extraction process is challenging and
requires a lot of expertise. The text mining is focused on studying algorithms
to find implicit, previously unknown, and potentially high-quality information
from a large collection of documents. Text mining activities include: (i) grouping
data/documents with similar properties or similar content [
Effectively performing the text mining process on textual data requires a
multi-step process involving different algorithms and for each one different
specific parameters should be manually set by the end-user. Furthermore, different
methods exist and the selection of the optimal ones is guided by the expertise
of the analyst. Innovative and scalable solutions need to be devised to suggest
to the analyst how configure the mining process to relieve the end-user of the
burden of selecting proper methods for the overall cluster analysis process [
This paper proposes SELF-DATA (SELF-learning DAta TrAnsformation),
a distributed engine to suggest to the analyst a good configuration of the
mining process to cluster a collection of documents into cohesive and well-separated
groups. SELF-DATA relies on different building components that characterize
the data distribution of collections of documents and learn how configure the
analytics process through self-tuning and self-learning strategies. Based on
previous processed datasets, SELF-DATA will also be able, in the future, to suggest
to the analyst the complete configuration of the mining process after the
characterization of the data distribution of a new unexplored dataset. SELF-DATA’s
current implementation runs on the Apache Spark [
This paper is organized as follows. Section 2 presents the SELF-DATA envisioned architecture and a brief description of its main building components, while Section 3 discusses the preliminary performed experiments. Finally, Section 4 draws conclusions and presents future developments of this work. 2
The SELF-DATA engine SELF-DATA (SELF-learning DAta TrAnsformation) is a distributed engine whose aim is to suggest to the analyst a good configuration of the whole mining process to cluster a textual data collection into correlated groups of documents with a similar topic. The components of the SELF-DATA envisioned architecture, as well as the interactions between such components, are shown in Figure 1. It relies on 5 main components exploited in 2 phases, i.e., learning and prediction phases.
Learning phase. During the learning phase, SELF-DATA learns good configurations of the whole mining process for specifics data distributions of textual data collections from previous processed datasets. To this aim SELFDATA relies on 4 main components able to (i) automatically configure the complete clustering activity (e.g., selecting the good weighting schema and transformation method, automatically setting the specific-algorithm input parameters) by exploiting self-tuning strategies; (ii) characterize different data distribution through the computation of various indices; (iii) store results of steps (i) and (ii) into a knowledge base; and (iv) exploit the knowledge base to learn which subsets of data distribution indices with the corresponding values are mainly able to forecast the best configuration of the cluster analysis process of textual data collections. Specifically, the following components characterize the SELF-DATA learning phase.
– PASTA (PArameter-free Solutions for Textual Analysis) [
PASTA explores different suitable data weighting strategies, based on local (Term-Frequency (TF) and Logarithmic term frequency (Log)) and global (Inverse Document Frequency (IDF)) weights and data transformation methods (e.g., PCA, LSI) before applying a clustering algorithm to gain better insights from the analysis. To streamline the analytics process PASTA includes two procedures to relieve the end-user of the burden of selecting proper values for algorithm-specific parameters. These procedures evaluate and compare solutions through different quality indices (e.g., Silhouettebased indices, rand index, f-measure) to identify a few good configurations (3-top) of the mining process that yield a good partition of the original dataset. PASTA’s current implementation runs on the Apache Spark framework, supporting parallel and scalable processing for analytics activities. – The Feature computation component is an engine capable of characterizing the data distribution of a collection of documents (corpus) through various indices including • # categories: the original number of topics in the dataset under analysis (if known); • Avg frequency terms: the average frequency of a term’s occurrence in the corpus; • Max frequency: the maximum frequency of a term’s occurrence in the corpus; • Min frequency: the minimum frequency of a term’s occurrence in the corpus; • # documents: the number of documents in the corpus; • # terms: number of terms in the corpus, with repetitions; • Dictionary: the number of different terms in the corpus (without repetition); • TTR: the relation between the dictionary variety and the total number of terms in the corpus. It is calculated as a ratio between Dictionary and # terms ; • Hapax %: the ratio between number of Hapax (number of terms with one occurrence) and the cardinality of the dictionary; • Guiraud Index: the ratio between the cardinality of the dictionary and the square root of # terms. It represents the lexical richness. – K-DB is a knowledge base storing results of 3-top solutions identified through PASTA on previously processed datasets, including data distribution characterization. Specifically, for each processed dataset K-DB stores one record for each solution selected by PASTA. Each record includes the values of the various indices characterizing the data distribution, the selected weighting schema and transformation method, with the suggested input parameter value for either the LSI or PCA transformation method, and the suggested value for the input parameter of the clustering algorithm (i.e., the desired number of cluster). – Building a prediction model addresses a classification problem trained on the knowledge base content to generate a prediction model to suggest to the analyst a good configuration/setting for the cluster analysis process on a given unexplored dataset. Many different approaches (e.g., data mining approaches, logic programming, statistics models) can be exploited for this complex and challenging task.
Prediction phase. Given a new document collection, SELF-DATA first applies the document processing step and then computes the set of features to characterize data distribution. Given the values of the data distribution indices, the prediction model is exploited to forecast a good configuration of the cluster analysis process. This step suggests to the analyst a good weighting schema (both local and global) and a transformation method, and good values for specificalgorithm input parameters to effectively cluster the data collection into a few well-separated and cohesive groups of documents addressing a similar topic. 3
Here we presented a preliminary implementation of the SELF-DATA system with its preliminary results.
Preliminary development. A preliminary implementation of some components of SELF-DATA has been developed.
The current implementation includes: (1) the feature computation building
block able to characterize the data distribution through some basic indices (see
Section 2 for further detail) and (2) an extended version of the PASTA
component presented in [
PASTA exploits various quality metrics to evaluate different configurations of the mining process.
Silhouette-based indices: purified and weighted Silhouette. The Silhouette
index [
Classification technique to assess the robustness of a given document
partition. PASTA builds a classifier using the same input features of the
clustering algorithm, and the class label assigned by the clustering algorithm as
target. PASTA integrates the random forest method [
Characterization of the data distribution. Table 1 shows the proposed data distribution indices to characterize data distribution of the considered datasets. The Guirand index is a good parameter to distinguish datasets with a very sparse data distribution (D1 and D2 in Table 1) from less sparse (D3 and D4) datasets. Datasets with high values for lexical richness are usually characterized by a high variability in the data dictionary, thus more sparse data distribution. The TTR index also allows to better differentiate sparse data distribution from more dense data distributions. In fact, the TTR value for D3 is one order of magnitude higher than the TTR values of the others.
ID # categories Avg frequency terms
Max frequency Min frequency # documents # terms Dictionary
TTR
Hapax % Guiraud Index
Selecting good configurations for the whole process of cluster analysis. For each dataset PASTA has been run eight times, once for each combination of a weighting function and a transformation method. Table 2 shows the 3-top solutions identified by PASTA for dataset D1 and by considering the LSI method and all weighting functions. The optimal solution is reported in bold.
For D1 PASTA identifies for K (i.e., number of relevant dimensions for LSI) 34, 37 and 112 dimensions to be considered for the cluster analysis. Given these numbers of dimensions, PASTA selects kcls = 10 (input parameter for the Kmeans algorithm) as the optimal partition. We observed that PASTA usually 1 It is the weighted harmonic mean of precision and recall selects as optimal partition the experiment exploiting a low-medium number of dimensions (terms). The higher the K, the more variable the data distribution is and the more complex the cluster activity will be. Thus, the Silhouette-based indices (purified P S and weighted W S) tend to slightly decrease when a large number of terms featuring in each document is analyzed. Furthermore, the results reported in Table 2 highlight that the TF-IDF, Log-IDF and TF-Entropy weighting functions are able to better differentiate the weighting terms, thus identifying a larger number of clusters (associated with different topics in the same category) than the one discovered by Log-Entropy.
We also compared the optimal configurations selected by PASTA when it explores LSI with respect to the ones obtained with PCA. Table 3 shows the best configuration selected by PASTA for both LSI and PCA and for all weighting functions. Results are obtained on D3 and D4 datasets. For datasets D1 and D2 the LSI method outperforms PCA (detailed results are omitted to lack of space), whereas for dataset D3 and D4 PCA yield a better final partition than LSI. Thus, we can conclude that the LSI method outperforms PCA in the presence of a larger data domain, and usually a more sparse data distribution. 4
This paper presents a challenging vision for a self-learning engine capable of predicting how to configure the cluster analysis process of a collection of textual data. A preliminary implementation of some components has been presented and evaluated on four real collections of documents. We plan to develop the complete SELF-DATA system in a future work. D3 LSI D3 PCA D4 LSI D4 PCA