<!DOCTYPE article PUBLIC "-//NLM//DTD JATS (Z39.96) Journal Archiving and Interchange DTD v1.0 20120330//EN" "JATS-archivearticle1.dtd">
<article xmlns:xlink="http://www.w3.org/1999/xlink">
  <front>
    <journal-meta />
    <article-meta>
      <title-group>
        <article-title>Using Self Organizing Maps to Visualize Age Related Changes in Lumbar Vertebrae and Intervertebral Discs</article-title>
      </title-group>
      <contrib-group>
        <contrib contrib-type="author">
          <string-name>Atif Ali Khan</string-name>
          <xref ref-type="aff" rid="aff1">1</xref>
        </contrib>
        <contrib contrib-type="author">
          <string-name>Daciana Iliescu</string-name>
          <xref ref-type="aff" rid="aff1">1</xref>
        </contrib>
        <contrib contrib-type="author">
          <string-name>Evor Hines</string-name>
          <xref ref-type="aff" rid="aff1">1</xref>
        </contrib>
        <contrib contrib-type="author">
          <string-name>Charles Hutchinson</string-name>
          <xref ref-type="aff" rid="aff3">3</xref>
        </contrib>
        <contrib contrib-type="author">
          <string-name>Robert Sneath</string-name>
          <email>robert.sneath@uhcw.nhs.uk</email>
          <xref ref-type="aff" rid="aff2">2</xref>
        </contrib>
        <aff id="aff0">
          <label>0</label>
          <institution>Atif.Khan; E.L.Hines;</institution>
          <addr-line>D.D.Iliescu</addr-line>
        </aff>
        <aff id="aff1">
          <label>1</label>
          <institution>School of Engineering, University of Warwick</institution>
        </aff>
        <aff id="aff2">
          <label>2</label>
          <institution>University Hospital Coventry and Warwickshire, NHS Trust</institution>
          ,
          <addr-line>Coventry</addr-line>
          ,
          <country country="UK">United Kingdom</country>
        </aff>
        <aff id="aff3">
          <label>3</label>
          <institution>Warwick Medical School, University of Warwick</institution>
        </aff>
      </contrib-group>
      <abstract>
        <p>A human spine is a complicated structure of bones, joints, ligaments and muscles which all undergo a process of change with age. This paper describes the use of artificial intelligence in visualization and better understanding of the progressive and degenerative changes in human lumbar spine. Visualizing this pattern of change will be helpful in finding the correlations among the spinal features and understanding of how a change in one feature affects others. The self-organizing map (SOM) is an efficient tool for visualization of multidimensional numerical data. It is capable of projecting high-dimensional data onto a regular, usually 2-dimensional grid of neurons. In this paper, SOM is used to visualize the pattern of change in lumbar spine features with the varying age. The paper gives an idea of how the information can be acquired from SOM representations and how the SOM can be best utilized in exploratory data visualization. Data from the lumbar spine MRIs of 61 patients (both male and female) were used in this study. The age of patients ranged from 2 to 93 years. Information for vertebral height, disc height and disc signal intensities were recorded from the MRI scans. SOM then transformed the larger feature space to a smaller one for getting a more meaningful relation between the spinal features. Complexity is reduced and the data set is represented in the form of 2D map which is easier to understand and provides visual description.</p>
      </abstract>
    </article-meta>
  </front>
  <body>
    <sec id="sec-1">
      <title>-</title>
      <p>
        A human spine is a complicated and key component of
human being. During the normal ageing process, spine
undergoes progressive and regressive changes which
presumably follow certain pattern. This research focuses
explicitly on the study of progressive and degenerative
changes occurring in the human lumbar spine with the
normal ageing process. This research work concentrates on
the identification and classification of age-related
variations in "human spine" with the help of Magnetic
Resonance Image (MRI). These scans of the lumbar spine
area belong to patients of different age groups. Back pain
is usually associated with the spine disorder. It is the
second most common reason for visits to the doctor’s
clinic, outnumbered only by the upper-respiratory
infections [
        <xref ref-type="bibr" rid="ref1 ref2 ref3">1, 2, 3</xref>
        ]. Back pain is one of the most common
reasons for missed work too. One-half of all working
Americans admit to having back pain symptoms each year
[
        <xref ref-type="bibr" rid="ref1 ref4">1, 4</xref>
        ]. Before finding the specific cause of back pain, it is
important to study the variation of spinal features with age
first and their correlation with one another [
        <xref ref-type="bibr" rid="ref5">5</xref>
        ]. These
variations and correlation among the features were studies
using the data collected from University Hospital Coventry
and Warwickshire, United Kingdom in the form of
magnetic resonance images (MRI) of the lumbar spine.
Scoring of features (feature selection and extraction) was
done under the expertise of an orthopedic surgeon and
radiologist. The model was designed and built using
selforganizing maps.
      </p>
      <p>
        A self-organizing map (SOM) is a special type of artificial
neural network which is trained using unsupervised
learning to produce a low-dimensional (typically
twodimensional), discretized representation of the input space
of the training samples, called a map. Unlike to other
artificial neural networks, self-organizing map uses a
neighborhood function to preserve the topological
properties of the input space [
        <xref ref-type="bibr" rid="ref6">6</xref>
        ]. One of the most
interesting aspects of SOMs is that they learn to classify
data without supervision. With this approach an input
vector is presented to the network (typically a multilayer
feedforward network) and the output is compared with the
target vector. If they differ, the weights of the network are
altered slightly to reduce the error in the output. This is
repeated many times and with many sets of vector pairs
until the network gives the desired output. Training a SOM
requires no target vector and learns to classify the training
data without any external supervision whatsoever [
        <xref ref-type="bibr" rid="ref7">7</xref>
        ]. To
study the variations and correlation of the spinal features, a
model was built which assigns patient a certain cluster to
which he/she resembles the most on the basis of his/her
spinal scores. This will help spine specialists to rank and
categorize patients on the basis of their spinal scores. This
research work will provide a better overview to the spine
specialists and the patients about the abnormal behavior if
any shown by their spine.
      </p>
    </sec>
    <sec id="sec-2">
      <title>II. Lumbar Spine</title>
      <p>
        A human spine consists of bones, joints, ligaments and
muscles. There are a total of 33 vertebrae in the human
spine: 7 in the neck (cervical region), 12 in the middle
back (thoracic region), 5 in the lower back (lumbar
region), 5 that are fused to form the sacrum and the 4
coccygeal bones that form the tailbone [
        <xref ref-type="bibr" rid="ref8">8</xref>
        ]. The anatomy of
human spine is shown in the figure 1 below. The focus of
this research work is to look at the age related changes in
the lumbar spine area. This lumbar spine area consists of
vertebrae L1, L2, L3, L4, L5 and intervertebral discs
between these vertebrae.
      </p>
    </sec>
    <sec id="sec-3">
      <title>III. Data Set</title>
      <p>The data set used in this research was taken from the
University Hospital Coventry and Warwickshire (UHCW),
United Kingdom. The raw data is in the form of Magnetic
Resonance Images (MRI) specifically of the lumbar spine
area. The format of data is Digital Imaging and
Communications in Medicine (DICOM). These magnetic
resonance images are the actual scans of the patients.
Figure 2, shows the lumbar spine MRI.</p>
      <p>Fig 2: Sagittal and an axial view of the lumbar spine MRI
Information associated with each MRI scan is the age and
gender of the patient which is used for SOM modeling.
3
2
4
3
3
4
6
4
4
2
35
2
MRI scans of 61 patients were selected to develop an initial
model. Age and gender distribution of patients are shown
in the Table 1 below. Ten groups were formed on the basis
of age decades as: G0: up to 10 years, G1: 11-20 years, G2:
21-30, years, G3: 31-40 years, G4: 41-50 years, G5: 51-60
years, G6: 61-70 years, G7: 71-80 years, G8: 81-90 years
and G9: 91 and above years of age).
There are lots of notable features which can be studied
from a lumbar spine MRI scan. The scoring criteria were
set to look initially on the vertebral height (L1, L2, L3, L4
and L5), disc height (T12-L1, L1-L2, L2-L3, L3-L4, L4-L5
and L5-S1) and disc signal (T12-L1, L1-L2, L2-L3, L3-L4,
L4-L5 and L5-S1). These 17 spinal features were used as
an input to build and test the initial model. These features
were measured and recorded from the lumbar spine MRIs
of 61 patients.</p>
    </sec>
    <sec id="sec-4">
      <title>IV. Methodology</title>
      <p>Self-organizing maps (SOMs) are a data visualization
technique invented by Teuvo Kohonen which reduces the
dimensions of data through the use of self-organizing
neural networks. As the humans simply cannot visualize
high dimensional data so this technique was created to help
us understand high dimensional data. The way SOMs go
about reducing dimensions is by producing a map of
usually 1 or 2 dimensions which plot the similarities of the
data by grouping similar data items together. So SOMs
accomplish two things, they reduce dimensions and
display similarities. The proposed model has a set of 17
input vectors arranged as columns in a matrix. SOM
groups or ranks each sample (patient) on the basis of
similarities in their 17 features and assigns certain location
to each sample in the map. Figure 3 below; shows the step
by step demonstration of the methodology used.</p>
    </sec>
    <sec id="sec-5">
      <title>V. Self-Organizing Maps</title>
      <p>
        Self-Organizing Map (SOM) is a data visualization
technique which helps to understand high dimensional data
by reducing data dimensions and displaying similarities
among data. According to Teuvo Kohonen; the
selforganizing map (SOM) is a new, effective software tool
for the visualization of high dimensional data. It converts
complex, nonlinear statistical relationships between
highdimensional data items into simple geometric relationships
on a low-dimensional display. As it thereby compresses
information while preserving the most important
topological and metric relationships of the primary data
items on the display, it may also be thought to produce
some kind of abstractions [
        <xref ref-type="bibr" rid="ref9">9</xref>
        ].
      </p>
      <p>
        SOM contains two processes: training and mapping. In
training process, it constructs the map using input samples.
After the training, it automatically classifiers a new input
sample in the mapping process. The map consists of
several neurons which associated with a weight vector that
has the same dimension as the input sample and a position
in the map. The neurons are arranged originally in physical
positions according to a topology function, such as a grid,
hexagonal, or random topology. The purpose of SOM is to
detect regularities and correlations in their input, and also
to recognize groups of similar input vectors [
        <xref ref-type="bibr" rid="ref10 ref11">10, 11</xref>
        ]. It can
adapt their future responses to that input accordingly in
such a way that neurons of competitive networks physically
near each other in the neuron layer respond to similar input
vectors.
With SOM, clustering is performed by having several units
compete for the current object. Once the data have been
entered into the system, the network of artificial neurons is
trained by providing information about inputs. The weight
vector of the unit is closest to the current object becomes
the winning or active unit. During the training stage, the
values for the input variables are gradually adjusted in an
attempt to preserve neighborhood relationships that exist
within the input data set. As it gets closer to the input
object, the weights of the winning unit are adjusted as well
as its neighbors [
        <xref ref-type="bibr" rid="ref12 ref13">12, 13</xref>
        ]. SOM training is shown below:
Initialization: All the connection weights are initialized
with small random values.
      </p>
      <p>Competition: For each input pattern, the neurons compute
their respective values of a discriminant function which
provides the basis for competition. The particular neuron
with the smallest value of the discriminant function is
declared the winner.</p>
      <p>Cooperation: The winning neuron determines the spatial
location of a topological neighbourhood of excited neurons,
thereby providing the basis for cooperation among
neighbouring neurons.</p>
      <p>Adaptation: The excited neurons decrease their individual
values of the discriminant function in relation to the input
pattern through suitable adjustment of the associated
connection weights, such that the response of the winning
neuron to the subsequent application of a similar input
pattern is enhanced.
Unlike other learning technique in neural networks,
training a SOM requires no target vector. A SOM learns to
classify the training data without any external supervision.
Each node's weights are initialized. If the input space is D
dimensional (i.e. there are D input units) we can write the
input patterns as:
And the connection weights between the input units i and
the neurons j in the computation layer can be written as:
wj = {wji : j = 1, …, N; i = 1, …, D}
“N” is the total number of neurons. To determine the best
matching unit, one method is to iterate through all the
nodes and calculate the Euclidean distance between each
node's weight vector and the current input vector. The
node with a weight vector closest to the input vector is
tagged as the BMU. The Euclidean distance is given as:
Where x is the current input vector and w is the node's
weight vector.</p>
      <p>Network Architecture
In SOM, the network is created from a 2D lattice of
'nodes', each of which is fully connected to the input layer.
Figure 6 shows a very small Kohonen network of 3 x 3
nodes connected to the input layer shown in dark blue.
Each node has a specific topological position (an x, y
coordinate in the lattice) and contains a vector of weights
of the same dimension as the input vectors. That is to say,
if the training data consists of vectors, X, of n dimensions:
(x1, x2, x3...xn). Then each node will contain a</p>
    </sec>
    <sec id="sec-6">
      <title>VI. Experimentation</title>
      <p>
        The measurements taken from the lumbar MRI of 61
patients were used to model the SOM. Each patient has 17
features which were used as input to the model. These 17
input variables are vertebral heights (5 variables), disc
height (6 variables) and disc signal (6 variables). So the
variables 1-5 are the vertebral height (L1-L5), variables
611 are the disc heights from T12/L1--L5/S1 and variables
12-17 are the disc signals from T12/L1—L5/S1
respectively. The inputs vertebral heights, disc heights and
disc signals have difference ranges. Initial model was built
without normalization of the inputs. Figure 7, below shows
the SOM model built on the basis of 17 input variables
without normalization. In this mode, final quantization
error was: 47.292 and final topographic error was: 0.00.
Figure 8, SOM and U-matrix without normalization of inputs
There are two separate parts of the SOM display. These
include the unified matrix or U-matrix, and the component
planes that are provided for individual variables [
        <xref ref-type="bibr" rid="ref14 ref15">14, 15</xref>
        ].
The U-matrix allows examination of the overall cluster
patterns in the input data set after the model has been
trained. [
        <xref ref-type="bibr" rid="ref16 ref17 ref18">16, 17, 18</xref>
        ] Each hexagonal cell represents
individual neurons, which are the mathematical linkages
between the input and output layers.
      </p>
      <p>
        The neurons are drawn into distinct clusters during model
training. Relative distances between neuron clusters are
displayed by the intensity of the colors, with dark color
representing greater distance [
        <xref ref-type="bibr" rid="ref19 ref20">19, 20</xref>
        ]. In the U-matrix
generated here, a strong cluster is apparent, occurring in
the top half (dark blue) and another one in the middle and SOM 23-Feb-2013
lower middle half (light blue). This indicates that most of
the input variables are covarying in one direction in
ndimensional space (where n is the number of input
variables). A different trend is seen when SOM is modeled
with normalized data. When the input variables are
normalized, following trend was seen as shown in figure 9
below.
In the component planes for individual variables, the color
coding corresponds to actual numerical values for the input
variables that are referenced in the scale bars adjacent to
each plot. Blue colors show low values and red corresponds
to high values. The relationships between each of the
variables are visualized by comparing the color patterns for
individual maps. In this manner, the relationships between
all of the variables entered into the model can be examined
simultaneously or in pair-wise combinations.
SOM model without input normalization showed final
quantitation error of 47.292. However, by the
normalization the inputs this quantization error is reduced
to 1.989. The final quantization error was: 1.989 and the
final topographic error was: 0.033. This shows that SOM
analysis with normalized input variables provides far
accurate and reliable results as compared to the results
without normalization. The first map in the figure 10
below is the unified distance matrix or U-Matrix which
represents overall behavior of the model. Variables 1 to 5
are the vertebral heights. Variable 6 to 11 are the disc
heights and variable 12 to 17 are the disc signal intensities
of all 61 patients. The color of the units (neurons) in the
map shows the behavior of the specific neuron. Similar
color shows that the neurons are located close to one
another or similarity among the samples.
Here in figure 11 above, matching the color code of each
variable with U-matrix it can be seen that vertebral heights
L1, L2, L3, L4, L5 (corresponding to variables 1, 2, 3, 4, 5
respectively) do not correlate with the age (dissimilarity
with U-matrix). Disc heights T12-L1, L1-L2, L2-L3,
L3L4, L4-L5 and L5-S1 (corresponding to variables 6, 7, 8, 9,
10, 11 respectively) show somewhat correlation with the
age. However, disc signal T12-L1, L1-L2, L2-L3, L3-L4,
L4-L5 and L5-S1 (corresponding to variables 12, 13, 14,
15, 16, 17 respectively) shows strong correlation with age.
      </p>
      <p>U-matrix
Variable5
Variable10
Variable15
d
d</p>
    </sec>
    <sec id="sec-7">
      <title>Conclusion</title>
      <p>The objective of the SOM analysis was to observe
interrelationships that exist between 17 variables that were
tested and thereby provides a basis for more advance
analysis. The SOM does not replace existing statistical
tools, but complements our ability to examine relationships
between disparate types of variables in a visual
presentation of the data. By visualizing the SOM results
obtained by normalized dataset, it was concluded that
lumbar spine vertebral height does not correlate with the
age whereas disc height shows somewhat correlation with
age. Disc signal intensities of lumbar spine show a strong
correlation with the age. In future, other spinal features
will be incorporated to study the spinal aging process in
more depth.</p>
    </sec>
    <sec id="sec-8">
      <title>Acknowledgments</title>
      <p>This project was partially supported by Warwick Impact
Fund, University of Warwick, United Kingdom. Authors
would like to thank the University Hospital Coventry and
Warwickshire (UHCW) NHS Trust, Coventry, United
Kingdom for providing valuable data in the form of
magnetic resonance images of the lumbar spine.</p>
    </sec>
  </body>
  <back>
    <ref-list>
      <ref id="ref1">
        <mixed-citation>
          [1]
          <string-name>
            <surname>Vallfors</surname>
            <given-names>B.</given-names>
          </string-name>
          <string-name>
            <surname>Acute</surname>
          </string-name>
          ,
          <article-title>Subacute and Chronic Low Back Pain: Clinical Symptoms, Absenteeism</article-title>
          and
          <string-name>
            <given-names>Working</given-names>
            <surname>Environment</surname>
          </string-name>
          .
          <source>Scan J Rehab Med Suppl</source>
          <year>1985</year>
          ;
          <volume>11</volume>
          :
          <fpage>1</fpage>
          -
          <lpage>98</lpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref2">
        <mixed-citation>
          [2]
          <string-name>
            <surname>Back</surname>
            <given-names>Health at Work. HSE</given-names>
          </string-name>
          <year>2005</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref3">
        <mixed-citation>
          [3]
          <string-name>
            <given-names>Back</given-names>
            <surname>Pain Patient Outcomes Assessment</surname>
          </string-name>
          <article-title>Team (BOAT)</article-title>
          .
          <source>In MEDTEP Update</source>
          , Vol.
          <volume>1</volume>
          <issue>Issue 1</issue>
          ,
          <source>Agency for Health Care Policy and Research</source>
          , Rockville.
        </mixed-citation>
      </ref>
      <ref id="ref4">
        <mixed-citation>
          [4] Department of Health Statistics Division.
          <article-title>The prevalence of back pain in Great Britain in 1998</article-title>
          . London: Government Statistical Service,
          <year>1999</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref5">
        <mixed-citation>
          [5]
          <string-name>
            <given-names>Atif</given-names>
            <surname>Khan</surname>
          </string-name>
          , Daciana Iliescu, Evor Hines, Charles Hutchinson, and Robert Sneath, “
          <article-title>Neural Network Based Spinal Age Estimation Using Lumbar Spine Magnetic Resonance Images (MRI)”</article-title>
          ,
          <source>proceedings of 4th International Conference on Intelligent Systems, Modelling and Simulation (ISMS2013)</source>
          ,
          <fpage>29</fpage>
          -
          <lpage>31</lpage>
          January, Bangkok Thailand, pp.
          <fpage>88</fpage>
          -
          <lpage>93</lpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref6">
        <mixed-citation>
          [6]
          <string-name>
            <surname>Kohonen</surname>
          </string-name>
          ,
          <string-name>
            <surname>Teuvo</surname>
          </string-name>
          (
          <year>1982</year>
          ).
          <article-title>"Self-Organized Formation of Topologically Correct Feature Maps"</article-title>
          .
          <source>Biological Cybernetics</source>
          <volume>43</volume>
          (
          <issue>1</issue>
          ):
          <fpage>59</fpage>
          -
          <lpage>69</lpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref7">
        <mixed-citation>
          [7]
          <string-name>
            <surname>Kohonen</surname>
          </string-name>
          , Teuvo, “
          <article-title>Self-organizing maps (SOMs)”</article-title>
          , Vol.
          <volume>30</volume>
          . Springer Verlag,
          <year>2001</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref8">
        <mixed-citation>
          [8]
          <string-name>
            <surname>McKenzie</surname>
            ,
            <given-names>Robin A.</given-names>
          </string-name>
          , and S. May. “The lumbar spine”,
          <source>Spinal</source>
          ,
          <year>1981</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref9">
        <mixed-citation>
          [9]
          <string-name>
            <surname>Kohonen</surname>
            ,
            <given-names>Teuvo. "</given-names>
          </string-name>
          <article-title>The self-organizing map"</article-title>
          <source>Neurocomputing 21.1</source>
          (
          <year>1998</year>
          ):
          <fpage>1</fpage>
          -
          <lpage>6</lpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref10">
        <mixed-citation>
          [10]
          <string-name>
            <surname>Barlow</surname>
            ,
            <given-names>Horace B.</given-names>
          </string-name>
          "
          <source>Unsupervised learning." Neural Computation</source>
          <volume>1</volume>
          , no.
          <issue>3</issue>
          (
          <year>1989</year>
          ):
          <fpage>295</fpage>
          -
          <lpage>311</lpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref11">
        <mixed-citation>
          [11]
          <string-name>
            <surname>Carpenter</surname>
            ,
            <given-names>Gail A.</given-names>
          </string-name>
          , and Stephen Grossberg, eds. “
          <article-title>Pattern recognition by self-organizing neural networks”</article-title>
          . MIT Press,
          <year>1991</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref12">
        <mixed-citation>
          [12]
          <string-name>
            <surname>Kohonen</surname>
            ,
            <given-names>Teuvo. "</given-names>
          </string-name>
          <article-title>The self-organizing map</article-title>
          .
          <source>" Proceedings of the IEEE 78, no. 9</source>
          (
          <year>1990</year>
          ):
          <fpage>1464</fpage>
          -
          <lpage>1480</lpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref13">
        <mixed-citation>
          [13]
          <string-name>
            <surname>Vesanto</surname>
            , Juha, and
            <given-names>Esa</given-names>
          </string-name>
          <string-name>
            <surname>Alhoniemi</surname>
          </string-name>
          .
          <article-title>"Clustering of the self-organizing map</article-title>
          .
          <source>" IEEE Transactions on Neural Networks</source>
          ,
          <volume>11</volume>
          , no.
          <issue>3</issue>
          (
          <year>2000</year>
          ):
          <fpage>586</fpage>
          -
          <lpage>600</lpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref14">
        <mixed-citation>
          [14]
          <string-name>
            <surname>Vesanto</surname>
            ,
            <given-names>Juha.</given-names>
          </string-name>
          <article-title>"SOM-based data visualization methods." Intelligent data analysis 3</article-title>
          , no.
          <issue>2</issue>
          (
          <year>1999</year>
          ):
          <fpage>111</fpage>
          -
          <lpage>126</lpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref15">
        <mixed-citation>
          [15]
          <string-name>
            <surname>Vesanto</surname>
            ,
            <given-names>Juha</given-names>
            , Johan Himberg, Esa Alhoniemi, and Juha
          </string-name>
          <string-name>
            <surname>Parhankangas</surname>
          </string-name>
          .
          <article-title>"Self-organizing map in Matlab: the SOM toolbox."</article-title>
          <source>In Proceedings of the Matlab DSP Conference</source>
          , vol.
          <volume>99</volume>
          , pp.
          <fpage>16</fpage>
          -
          <lpage>17</lpage>
          .
          <year>1999</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref16">
        <mixed-citation>
          [16]
          <string-name>
            <surname>Ultsch</surname>
            , Alfred; Siemon,
            <given-names>H.</given-names>
          </string-name>
          <string-name>
            <surname>Peter</surname>
          </string-name>
          (
          <year>1990</year>
          ).
          <article-title>"Kohonen's Self Organizing Feature Maps for Exploratory Data Analysis"</article-title>
          .
          <source>Proceedings of the International Neural Network Conference (INNC-90)</source>
          , Paris, France, July 9-
          <issue>13</issue>
          ,
          <year>1990</year>
          . 1. Dordrecht, Netherlands: Kluwer. pp.
          <fpage>305</fpage>
          -
          <lpage>308</lpage>
          . ISBN 978-0-
          <fpage>7923</fpage>
          -0831-
          <issue>7</issue>
          (
          <issue>0</issue>
          -7923-0831-X).
        </mixed-citation>
      </ref>
      <ref id="ref17">
        <mixed-citation>
          [17]
          <string-name>
            <surname>Ultsch</surname>
          </string-name>
          ,
          <string-name>
            <surname>Alfred</surname>
          </string-name>
          (
          <year>2003</year>
          ); U*
          <article-title>-Matrix: A tool to visualize clusters in high dimensional data</article-title>
          , Department of Computer Science, University of Marburg,
          <source>Technical Report Nr</source>
          .
          <volume>36</volume>
          :
          <fpage>1</fpage>
          -
          <lpage>12</lpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref18">
        <mixed-citation>
          [18]
          <string-name>
            <surname>Ultsch</surname>
            ,
            <given-names>Alfred.</given-names>
          </string-name>
          <article-title>"Maps for the visualization of highdimensional data spaces."</article-title>
          <source>In Proc. Workshop on Self organizing Maps</source>
          , pp.
          <fpage>225</fpage>
          -
          <lpage>230</lpage>
          .
          <year>2003</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref19">
        <mixed-citation>
          [19]
          <string-name>
            <surname>Vesanto</surname>
            ,
            <given-names>Juha</given-names>
            , Johan Himberg, Esa Alhoniemi, and Juha
          </string-name>
          <string-name>
            <surname>Parhankangas</surname>
          </string-name>
          .
          <article-title>SOM toolbox for Matlab 5</article-title>
          . Helsinki, Finland: Helsinki University of Technology,
          <year>2000</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref20">
        <mixed-citation>
          [20]
          <string-name>
            <surname>Ong</surname>
            , Jason, and
            <given-names>S. S.</given-names>
          </string-name>
          <string-name>
            <surname>Raza</surname>
          </string-name>
          .
          <article-title>"Data mining using selforganizing kohonen maps: A technique for effective data clustering &amp; visualization."</article-title>
          <source>In International Conference on Artificial Intelligence (IC-AI'99)</source>
          .
          <year>1999</year>
          .
        </mixed-citation>
      </ref>
    </ref-list>
  </back>
</article>