Clustering algorithms generally suffer from some well-known problems for which the Self Organizing Maps (SOM) algorithms are adept at handling. While there are many variants of the SOM algorithm, software programmes that implement the SOM algorithms have tended to give varying results even when tested on the same data sets. This can have serious implications when the goal of the clustering is novelty detection. In this study a comparison of the performance of some SOM clustering software was carried out and results presented.
In the clustering process data is grouped in such a way that the
intra-cluster similarity is maximized while the inter-cluster
similarity is minimized. Data can be described by either
categorical or numeric features. Due to the differences in the
characteristics of these two kinds of data, attempts to develop
criteria functions for mixed data have not been very successful
[
In a SOM, the neurons (clusters) are organized into a grid
which is usually two-dimensional, but sometimes
onedimensional (or (rarely) three or more-dimensions. The reason
for using one- and two dimensional grids is that space
structures of higher dimensionality cause problems with data
display and cannot be displayed on the monitor. The SOM
working algorithm is a variant of multidimensional vectors
clustering of which the Kmeans clustering algorithm is an
example of this type of algorithm [
The SOM neural network uses a competitive learning algorithm
and is a method for unsupervised learning, based on a grid of
artificial neurons whose weights are adapted to match input
vectors in a training set. The SOM algorithm is fed with
feature vectors, which can be of any dimension. The algorithm
for the training of the SOM [
Each node has a set of neighbors. When a node wins a competition, the neighbor‟s weights are also changed but not as much as that of the winning node. The further the neighbor is from the winner, the smaller its weight change. The SOM update rule for the weight vector of the unit i is given mathematically as: ………………2 where t represents the sample index for each presentation of a sample „x‟ hc(x),i represents the neighborhood function around the winner unit „c‟, with neighborhood radius r(t).
The neighborhood function is like a smoothing kernel that is
time-variable. It is a decreasing function of the distance
between the the ith and cth reference vectors on the map grid.
The neighborhood function is usually expressed as the
Gaussian function which can be expressed mathematically as:
…………………3
where
ά(t) represents the learning rate factor and takes values 0< ά(t)<1
σ(t) represents the width of the neighborhood function which
decreases monotomically with the regression steps.
A simpler definition of the neighbourhood function given by
Kohonen [
There are some visual displays that are used to "determine" where
the natural cluster boundaries are in the SOM. Some of the visual
tools that can be used are Histograms [
A rich variety of versions of the basic SOM algorithm have
been proposed. Some of the variants aim at improving the
preservation of topology by using more flexible map structures
instead of the fixed grid. Some of these methods however
cannot be used for visualization as easily as the regular grid.
Some variants aim at reducing the computational complexity
of the SOM [
Being able to set the neighborhood kernel function and to set the start value for the neighborhood function (learning radius): The neighborhood function determines how strongly the processing elements are connected to each other. Neighborhoods of different sizes in different neuron configurations (e.g. rectangular and hexagonal lattices) can be used. The simplest neighborhood function is the bubble (winnertakes-all): it is constant (or 1) over the whole neighborhood of the winner unit and zero elsewhere. Usually the neighbourhood function is expressed as a Gaussian function and as expected using the winnertakes-all function retrieves less clusters than the Gaussian function.
Being able to set the activation function and weight initialization methods: Before the training, initial values are given to the prototype vectors of the SOM. The SOM is very robust with respect to the initialization process, however, when properly accomplished it allows the algorithm to converge faster to a good
solution. Initialization procedures that have been used are: Random initialization, where the weight vectors are initialized with small random values; Sample initialization, where the weight vectors are initialized with random samples drawn from the input data set; Linear initialization, where the weight vectors are initialized in an orderly fashion along the linear subspace spanned by the two principal eigenvectors of the input data set.
Being able to set the choice of cooling strategy during training: for example linear or exponential.
Being able to set the distance measure to be used, for example, Euclidean, Manhattan and Maximum value: It is noted that the distance measure between data points is an important component of a clustering algorithm. If the components of the data instance vectors are all in the same physical units then it is possible to use the simple Euclidean distance metric to successfully group similar data elements. The Euclidean distance in a two or three-dimensional space measures is the actual geometric distance between objects in the space. However, it has been observed that even the Euclidean distance can sometimes be misleading, because of the way the mathematical formula used to combine the distances between the single components of the data feature vectors into a unique distance measure that can be used for clustering purposes is computed. Different formulas lead to different clustering‟s. Therefore, domain knowledge must be used to guide the formulation of a suitable distance measure for each particular application.
Being able to set the scaling technique to be used: for
example z-transform, (0,1) transform, (1,-1) transform
or none, depending on the clustering goal and data set.
Being able to set the starting and stopping learning rate:
The learning rate is a decreasing function of time
between [
Being able to set the training algorithm to be used: for
example batch, on-line, hybrid etc. The batch algorithm
has been shown to be faster [
Good data visualization options: for example histograms, hinton charts, weight charts (maps), UMatrix, P-Matrix etc. Good result analysis and presentation functions: computation of vital statistics for evaluating the quality of the clustering for example, mean, standard deviation (or variance), correlation coefficient, t-test etc.
This work presents a comparative study of the performance some SOM clustering software when tested on the same data set. Results were presented and reasons for the observed variations presented. The study also presents the desirable features that standard SOM software should have.
Agro metrological data for FRIN headquarters, Ibadan, Nigeria was used. The data set had 254 records and the attributes in the data set were: Year (numeric), Month (text), Total Rainfall in millimeters (numeric), Minimum Temperature in Celsius (numeric), Maximum Temperature in Celsius (numeric), Relative Humidity and Fire Danger Index (numeric). The SOM software used were: NNClust, Pittnet Neural Network Educational Software and RapidMiner Studio. The NNClust software was programmed to use only the Gaussian neighbourhood function and the Euclidean distance measure. The user can input the learning rate and starting neighbourhood size. The software automatically normalizes the input data between -1 and 1 and has features for generating data/result statistics and data visualization such as weight maps and radar charts. The Pittnet software also uses the Gaussian neighbourhood function and Euclidean distance metrics. The user also defines the starting learning rate and it also automatically normalizes the data between 0 and 1. It is a DOS based program that saves its result in a text file and has no data analysis or data visualization ability. RapidMiner studio (Community Edition) has facilities for selecting parameters for defining the learning rate, neighbourhood radius and can choose either to normalize the data or not. It also has an array of tools for statistical data analysis and data visualization.
Using the three software‟s clusters was generated. The arithmetic mean of each cluster group was also computed. The arithmetic mean is a measure of central tendency which describes the central location of data and is usually used with other statistical measures such as the standard deviation because it can be affected by extreme values in the data set and therefore be biased. The standard deviation describes the spread of the data and is a popular measure of dispersion. It measures the average distance between a single observation and its mean.
The meteorological data was clustered using NNClust SOM clustering software with a starting learning rate of 0.9 and was trained over 100 epochs. The software accepts only numeric values. Non numeric values are treated as missing values which are replaced by the column mean. The software was set to identify a maximum of ten clusters, however only eight clusters were generated. The software uses the number of clusters specified to create the SOM grid. The mean and standard deviation of the eight clusters were computed. Increasing the training cycle did not improve the results. Table 1 presents the summary of the eight clusters, while figure 2 presents the chart of the cluster means. The meteorological data was trained using the Pittnet software with a starting learning rate of 0.9 and was set to train for 100 epochs, although the software stops training as soon as the maximum number of clusters have been generated. The software requires the user to specify the number of clusters expected apriori. This number is used in conjunction with the number of input signals (attributes) to determine the SOM grid size. Expected number of clusters was set to ten. The software identified only four clusters. The mean and standard deviation of the clusters were computed. Table 2 presents the summary of the clusters, while figure 3 presents the chart of the cluster means.
TheRapidMiner Studio software was used to cluster the meteorological data set using a starting learning rate of 0.9 and was trained over a 100 epochs. The expected number of clusters was set at ten and the software generated ten clusters. Table 3 presents the summary of the cluster means with their standard deviations while figure 4 presents a chart of their cluster means. 3.1
The quality of the clusters identified in the data by the three software‟s can be inferred from a comparison of the mean and standard deviation of the clusters. If the value of the standard deviation is low, then the clustered records are within the same range. However if the value is high this suggests the presence of outliers in the clustered data records. For example table 4 presents the clustered records for cluster 2 (table1) for the NNClust software which is representative of the trend observed in the clusters identified by the software. Interpreting the cluster is indecisive when the values in the Total Rainfall fields are considered. The field has a mean of 142.05 and a standard deviation of 136.011711.
Similarly considering the clusters identified by the Pittnet software in table 2 the same trend is observed. Table 5 presents the records for cluster 4 (table 2) for the Pittnet software cluster results. It can be observed that the cluster is consists of data records which have the same value for the FireDangerIndex attribute. However, considering the Total Rainfall field which has a mean value of 39.74444 and a standard deviation of 43.34732. The high standard deviation value implies that there are outlier data values in the clustered records.
The clusters identified by the RapidMiner software presented
in table 3 were easier to interpret. They followed the expected
rainfall pattern which is known for the region where the data
was collected [
Some of the problems found in the literature about clustering algorithms are: Most clustering techniques are based on distance calculations which are very sensitive to ranges of variables, therefore the values have to be normalized. Normalization however is a subjective function, and these transformations cannot be carried out without creating biases; The presence of outliers in data sets create problems in data clustering based on distance calculations when they have not been identified and removed from the data set; Handling categorical variables (non-numeric data, non-numeric variables, categorical data, nominal data, or nominal variables) are a problem for most clustering algorithms, and even when data encoding methods are used they can introduce extra biases due to the number of values which the encoding introduces in the categorical variables; The selection of variables also has a large influence on clustering results, while the assigning of different weights for variables and categorical values can be used, when many variables and categorical values are involved, it can affect the clustering quality; Capturing patterns (or behaviors) hidden inside time-varying variables and modeling them is another problem and most clustering techniques do not possess this predictive modeling capability; Most clustering techniques were developed for laboratory generated simple data sets consisting of a few to several numerical variables; hence they can‟t be used for large data analyses that consist of many categorical complex data. Most common implementation of data clustering algorithms suffer from these problems, however, SOM‟s are very robust and are adept at handling these problems but this depends also on the goal of the algorithm‟s implementation (programming). Applications programmed for demonstration purposes cannot be used for large scale projects and some implementations are not flexible and do not give users much options. However if the various implementations of the conventional SOM algorithm (which are usually focused on the goals of the programmer) provides enough options to the user, it is still a very robust algorithm that can be used for both numerical, categorical and mixed data sets. Further work in this study is focused on the development of an open flexible SOM clustering tool with adequate features that can be used for research purposes. Portuguese conference on progress in Artificial Intelligence , pp 304 - 313, (Sringer-Verlag Berlin, Heidelberg ©2005) Cluster 1