=Paper= {{Paper |id=None |storemode=property |title=Applying Constrained Clustering for Active Exploration of Music Collections |pdfUrl=https://ceur-ws.org/Vol-633/wom2010_paper8.pdf |volume=Vol-633 }} ==Applying Constrained Clustering for Active Exploration of Music Collections== https://ceur-ws.org/Vol-633/wom2010_paper8.pdf
                         Applying Constrained Clustering for
                        Active Exploration of Music Collections

                             Pedro Mercado                                              Hanna Lukashevich
         Fraunhofer Institute for Digital Media Technology                 Fraunhofer Institute for Digital Media Technology
                         Ehrenbergstr. 31                                                  Ehrenbergstr. 31
                        Ilmenau, Germany                                                  Ilmenau, Germany
                   mercpo@idmt.fraunhofer.de                                          lkh@idmt.fraunhofer.de

             Instituto Tecnológico Autónomo de México
                  Río Hondo No. 1, Tizapán, 64230
                             DF, México
               pmercadol@comunidad.itam.mx

ABSTRACT                                                                    1. INTRODUCTION
In this paper we investigate the capabilities of constrained                   During recent years the scientific and commercial inter-
clustering in application to active exploration of music col-               est in Music Information Retrieval (MIR) has significantly
lections. Constrained clustering has been developed to im-                  increased. Stimulated by the ever-growing availability and
prove clustering methods through pairwise constraints. Al-                  the size of digital music collections, automatic music index-
though these constraints are received as queries from a noise-              ing and retrieval systems has been identified as an increas-
less oracle, most of the methods involve a random procedure                 ingly important means to aid convenient exploration of large
stage to decide which elements are presented to the oracle.                 music catalogs. In order to supply the users with more ac-
In this work we apply spectral clustering with constraints                  curate and robust music exploration systems, automatically
to a music dataset, where the queries for constraints are                   extracted metadata like “music genre”, “style” or “mood” can
selected in a deterministic way through outlier identifica-                 be added to the conventional metadata e.g. artist name, al-
tion perspective. We simulate the constraints through the                   bum name and track title. Commonly this automatically
ground-truth music genre labels. The results show that con-                 extracted metadata is derived by means of collaborative fil-
strained clustering with deterministic outlier identification               tering or is generated by statistical classifiers that are pre-
method achieves reasonable and stable results through the                   trained on the restricted amount of labeled ground-truth
increment of the number of constraint queries.                              data. The exploration intentions of the end-user might not
                                                                            be expressed by the available training data. Hence the de-
                                                                            sirable exploration facets might stay unreachable.
Categories and Subject Descriptors                                             An alternative way of music exploration is to visualize the
H.3 [Information Systems]: Information Storage and Re-                      music collection or a part of it by placing similar songs close
trieval; H.5.5 [Information Systems]: Sound and Music                       to each other and non-similar songs far away from each other
Computing—methodologies and techniques                                      in some low-dimensional space projection. A comprehensive
                                                                            overview of the existing up to date systems and methods
                                                                            can be found in [18]. Similar goals can also be reached
General Terms                                                               with clustering algorithms that cluster (group) songs in a
                                                                            way that similar songs are joined in clusters and non-similar
Theory, Experimentation, Algorithms                                         songs appear in different clusters. Obviously, music has too
                                                                            many facets (aspects) for one “static” clustering that allows
                                                                            to use only one definition of similarity. In this paper we con-
Keywords                                                                    sider clustering with constraints as a complimentary fashion
Constrained clustering, outlier identification, spectral clus-              to music collection exploration. Here the user can express
tering, active semi-supervised learning, music information                  a particular point to clusterability of his/her music collec-
retrieval                                                                   tion by providing some feedback information in the form of
                                                                            constraints. Clustering with constraints has been already
                                                                            applied to a music collection by Peng et al. [15]. They simu-
                                                                            lated the generation of constraints by choosing random con-
                                                                            straint pairs from the classes in artist similarity graph. We
                                                                            propose to avoid using random constraints. In contrast, we
WOMRAD 2010 Workshop on Music Recommendation and Discovery,                 determine the optimal songs to be constrained via outlier
colocated with ACM RecSys 2010 (Barcelona, SPAIN)
Copyright c . This is an open-access article distributed under the terms    identification methods.
of the Creative Commons Attribution License 3.0 Unported, which permits
unrestricted use, distribution, and reproduction in any medium, provided
the original author and source are credited.
   The reminder of the paper is organized as follows. Sec. 2       most relevant ones. This function assigns high values to
provides some theoretic background on applied clustering           features which give better separability for a given number
and outlier identification methods. The conception of the          of clusters in the graph G. Therefore, features should be
conducted experiments is presented in Sec. 3. In Sec. 4 we         ranked in descending ordered through the given feature score.
bring some details on audio features, utilized dataset, eval-
uation scenarios and evaluation measures. The results are          2.3 Clustering Methods
presented and discussed in Sec. 5 and Sec. 6 concludes the            In this part we present two fundamental approaches con-
paper and brings some insights to the future research direc-       sidered in this paper: constrained clustering and spectral
tions.                                                             clustering.

2.     MATHEMATICAL BACKGROUND                                     2.3.1 Constrained Clustering
   In this section we present the fundamental concepts and            It is not always possible to get true labels, even for just a
methods used in this paper. We will always consider a data         portion of a dataset. In some circumstances it may be pos-
set X of n elements such that X = {x1 , x2 , . . . , xn }, where   sible to get information between pairs of elements. Wagstaff
xi ∈ Rm . See Sec. 4 for details on the used dataset.              et al. [21] proposed the addition of information through pair-
                                                                   wise constraints. They introduced two types of pairwise con-
2.1 Graph Laplacian                                                straints: namely Must Links (ML) if two elements should
   The fundamental tool related to spectral methods is the         be in the same cluster, and Cannot Links (CL) if two ele-
graph Laplacian. We present it briefly.                            ments should be in different clusters. This fundamental idea
   Let S ∈ Rn be a similarity matrix related to dataset X,         has been already applied for center initialization through
G = (E, V ) a similarity graph where E and V are the sets          weighted farthest traversal heuristic by Basu et al. [4] and
of edges and vertexes, respectively and W its corresponding        even generalized to kernel and graph methods by Kulis et
weighted adjacency
                P matrix. Let D be the degree matrix,              al. [11]. In particular, they exposed the manner in which
which has dii = n j=1 wij and zero elsewhere. Then, the un-        the information from given constraints can be added to this
normalized (L), Symmetric (Lsym ) and Random Walk (Lrw )           clustering methods. Given an affinity matrix W and sets of
Laplacians are:                                                    ML and CL, we define T as the constraint matrix, where for
                                                                   each pair of points (xi , xj )
                       L     =D−W
                    Lsym     = D−1/2 LD−1/2                 (1)                                   
                                                                                                   mij , for a ML,
                     Lrw     = D−1 L .                                         T = {tij } : tij =   −mij , for a CL,            (4)
                                                                                                   0,      otherwise,
     Some of the properties that the Laplacians hold are:

     1. They are symmetric positive semi-definite.                 where each mij is an arbitrary scalar. Then, the matrix
                                                                   which summarizes the side information is
     2. They have n real non-negative eigenvalues
                                                                                            W′ = W + T                           (5)
     3. The multiplicity of the smallest eigenvalue, which is
        always zero, is equal to the number of connected com-      and can be used for both kernel and graph clustering meth-
        ponents of G                                               ods.
2.2 Spectral Feature Selection
                                                                   2.3.2 Spectral clustering
   The properties of Laplacian operators have been already
extended to feature selection methods. In particular Zhao             Spectral clustering has received a considerable amount of
et al. [22] have developed a filter method based on properties     attention, due to its surprising results and easy implementa-
of Laplacians. We present it briefly.                              tion. We present the general framework related to Random
   Given a graph G, its corresponding weighted adjacency           Walk and Symmetric Laplacians. For more details, we refer
matrix W and degree matrix D, let λj and ξj be the eigenval-       to Luxburg [14].
ues and eigenvectors of the corresponding symmetric Lapla-            Let G and W be respectively the similarity graph and its
cian Lsym with 0 ≤ λ1 ≤ · · · ≤ λn . Then the score of the         weighted adjacency matrix obtained from a given similarity
feature Fi can be measured through the following function:         matrix. Depending on the type of Laplacian, the Matrix U
                                                                   is obtained as following:
                            k
                            X                                         • for Random Walk Laplacian we get the first k general-
                 ϕ(Fi ) =         (γ(2) − γ(λj ))α2j ,      (2)
                                                                        ized eigenvectors u1 , . . . , uk from the generalized eigen-
                            j=2
                                                                        value problem Lu = λDu and store them column-wise
where γ is a rational function, k is a number of clusters and           in a matrix U ∈ Rn×k .
αj is a cosine of the angle between the eigenvector ξj and
the weighted feature fbi which is defined as                          • for Symmetric Laplacian we get the first k eigenvectors
                                      1/2
                                    D fi                                u1 , . . . , uk of Lsym , store them column-wise in a matrix
                      fbi =                    ,            (3)         U ∈ Rn×k and normalize each row of U .
                                  k D1/2 f i k
where f i is the feature vector corresponding to Fi .              Afterwards, k-means algorithm is applied to cluster the rows
  Score function in eq. (2) considers the same criteria as         of matrix U , where each row is the embedding of the ele-
spectral clustering, where the first k eigenvectors are the        ments of the given dataset.
2.4 Outlier Identification Methods
  Application of the outlier identification is motivated by
the intrinsic nature of music, that is in some sense full of
outliers. Clustering constrained on extremes rather than
“randoms”, covers more of the problematic pieces. In this
study we apply the following outlier identification methods:

LOF Local Outlier Factor (LOF) was proposed by Breunig
   et al. [6]. It can be interpreted as an outlierness degree
   and gives the possibility to rank the items through it.
   As the name suggests, the outlierness of each element
   is restricted to local neighborhoods.

RRS Ramaswamy et al. [16] considered that the distance
   of each point to its kth nearest neighbor determines if
   it is an outlier or not. Hence, the larger the distance,
   the more chances for the item to be an outlier. Further
   we address this outlier detection method as RRS.

Both methods provide the possibility to rank the items
through their outlier degree. This allows to choose the order
in which the elements will be exposed to be constrained.


3.    CONCEPTION OF EXPERIMENTS
   In this section we explain the integration of the exposed
concepts and the setup of the experiments. The process
steps described in this section are summarized in Figure 1.
   Given a data set X and a set of pairwise constraints in
all experiments we aim to get the cluster assignments. Each
item in the dataset is represented with a feature vector xi ,
i = 1, . . . , n, where n is a number of elements in the dataset.
Not all dimensions in xi are equally profitable for the sim-
ilarity relations between the items in the dataset. In order
to select the most appropriate feature dimensions, we ap-                 Figure 1: Flow chart diagram of experiments (see
ply a spectral feature selection method as stated in Sec. 2.2.            Sec. 3 for details)
In our experiments the rational function in eq. (2) is set to
γ(x) = x4 . Given a data set related to the selected features,
the similarity relations between the items are captured via               Must-Link sets. The corresponding weighted adjacency ma-
the correlation coefficient kernel K, where K(xi , xj ) is equal          trix is defined as
to the Pearson correlation coefficient between vectors xi and
                                                                                              W = SKN N + T ,                          (7)
xj as
                          P                                               where T is the corresponding constraint matrix pointed in
                                (xi,k − x̄i )(xj,k − x̄j )                eq. (4). Here the elements tij of the constraint matrix T
     K(xi , xj ) = pP k                     pP                    , (6)
                          k (x i,k − x̄i )2     k (xj,k − x̄j )
                                                                2
                                                                          are set to the maximal (out of main diagonal) value of ad-
                                                                          jacency matrix W for ML, and to tij = −wij for CL. Next,
where x̄i and x̄j are the empirical means of vectors xi and xj            we use either the Symmetric or Random Walk Laplacian
respectively. The matrix related to the kernel is symmetric               (see eq. (1)) and apply spectral clustering (see Sec. 2.3.2),
positive semi-definite.                                                   receiving cluster assignments as outputs.
   The correlation coefficient kernel K is utilized to deter-                For our work we consider the following six experiments
mine the K Nearest Neighbors matrix (KNN), where in-                      where outlier identification methods and particular Lapla-
deed the neighborhood of each song is conformed by the                    cians are combined as presented in Table 1.
K most correlated songs. Here the parameter K was chosen
as K = log2 (n), where n is the number of elements (songs)
in the dataset. In addition to the KNN matrix we calculate                       Table 1: Configuration of experiments
the Symmetric K Nearest Neighbors matrix (SKNN), where
the KNN matrix is symmetrized through the insertion of                       Short name     Laplacian         Outlier Identification
missing non-mutual neighbor connections. The KNN ma-                         Sym RAW        Symmetric         Random
trix is utilized by the outlier detection methods introduced                 Sym LOF        Symmetric         LOF
in Sec. 2.4. At this stage we also consider the possibility of               Sym RSS        Symmetric         RRS
getting outliers random-wise just for the sake of comparison                 RW RAW         Random Walk       Random
of traditional presented scores in the literature.                           RW LOF         Random Walk       LOF
   For a set of identified outliers we get constraints from                  RW RSS         Random Walk       RRS
a noiseless oracle and through the corresponding extended
4.   EVALUATION SETUP
                                                                       Table 2: ISMIR 2004 benchmark dataset
   In this section we provide some details on the evaluation
setup. First of all, we briefly introduce audio features used                Genre                Number of songs
for compact and informative representation of audio tracks.                  Classical            320
Afterwards, we describe musical dataset involved in the ex-                  Electronic           115
periments. Finally, we bring some insights to the evaluation                 Jazz and Blues       26
scenarios and the evaluation measures used to estimate the                   Metal and Punk       45
effectiveness of proposed clustering algorithms.                             Rock and Pop         101
                                                                             World music          122
4.1 Audio Features
   We utilize a broad palette of low-level acoustic features
and several mid-level representations [5]. These mid-level
                                                                 with a single Gaussian. In addition, each feature dimension
features are computed on 5.12 seconds excerpts and observe
                                                                 is normalized by mean and standard deviation. All in all,
the evolution of the low-level features. With the help of mid-
                                                                 each music track is represented with a feature vector having
level representations, timbre texture [19] can be captured by
                                                                 2342 feature dimensions.
descriptive statistics as well as by including additional mu-
sical knowledge. To facilitate an overview the audio feature     4.2 Dataset Description
are subdivided in three categories by covering the timbral,
                                                                    In our experiments we use the “Training” part of the
rhythmic and tonal aspects of sound.
                                                                 ISMIR2004 Audio Description Contest Dataset1 . This data-
   Although the concept of timbre is still not clearly defined
                                                                 set includes 729 music tracks that are manually subdivided
with respect to music signals, it proved to be very useful for
                                                                 into 6 genre categories as presented in Table 2.
automatic music signal classification. To capture timbral
                                                                    In the context of this work genre labels are not directly
information, we use Mel-Frequency Cepstral Coefficients,
                                                                 employed in the traditional classification scenario. Instead
Spectral Crest Factor, Audio Spectrum Centroid, Spectral
                                                                 of that we use the genre labels to generate constraints for
Flatness Measurement, and Zero-Crossing Rate. In addi-
                                                                 the clustering algorithm. As such, two songs belonging to
tion, modulation spectral features [1] are extracted from the
                                                                 the same genre are considered to be connected with a must-
aforementioned features to capture their short term dynam-
                                                                 link constraint. Likewise two songs that belong to different
ics. We applied a cepstral low-pass filtering to the modula-
                                                                 genres are connected with a cannot-link constraint. The
tion coefficients to reduce their dimensionality and to decor-
                                                                 details on the choice of the constrained songs are provided
relate them as described in [7].
                                                                 in Sec. 3.
   All rhythmic features used in the current setup are derived
from the energy slope in excerpts of the different frequency-    4.3 Evaluation Scenarios
bands of the Audio Spectrum Envelope feature. These com-
                                                                    Traditionally constrained clustering is evaluated on the
prise the Percussiveness [20] and the Envelope Cross-
                                                                 entire dataset – both on constrained and on non-constrained
Correlation (ECC). Further mid-level features [7] are derived
                                                                 part – and the improvement of performance is shown over
from the Auto-Correlation Function (ACF). In the ACF,
                                                                 the number of pairwise constraints (see e.g. Basu et al. [3]).
rhythmic periodicities are emphasized and phase differences
                                                                 This approach is not optimal for the estimation of general-
annulled. Thus, we compute also the ACF Cross-Correlation
                                                                 ization capabilities of the clustering algorithm. Seeing the
(ACFCC). The difference to ECC again captures useful in-
                                                                 evaluation scores for the entire dataset, it is rather hard
formation about the phase differences between the different
                                                                 to estimate if the improvement is coming through the ris-
rhythmic pulses. In addition, the log-lag ACF and its de-
                                                                 ing amount of constrained songs or through the general im-
scriptive statistics are extracted according to [10].
                                                                 provement of clustering quality. In addition to the common
   Tonality descriptors are computed from a Chromagram
                                                                 scores for the entire dataset (further denoted as All dataset)
based on Enhanced Pitch Class Profiles (EPCP) [12], [17].
                                                                 we perform the evaluation on the part of the dataset that
The EPCP undergoes a statistical tuning estimation and
                                                                 is not involved in any constraints (further denoted as Test
correction to account for tunings deviating from the equal
                                                                 dataset).
tempered scale. Pitch-space representations as described in
                                                                    Interpretation of the number of pairwise constraints is also
[8] are derived from the Chromagram as mid-level features.
                                                                 not trivial. For instance, ten pairwise constraints can involve
Their usefulness for audio description has been shown in [9].
                                                                 just five songs if the constraints are provided in a manner of a
   Clustering music tracks that are described with a set of
                                                                 complete graph. On the other hand, ten pairwise constraints
audio features having different time resolution still remains
                                                                 can also concern twenty songs if each constraint connects a
a challenging task. The feature matrices of different songs
                                                                 distinct pair of songs. Instead of the number of the pairwise
can be hardly involved in clustering algorithm directly. To
                                                                 constraints we account for the percentage of the dataset in-
tackle this issue, we model each feature dimension of one
                                                                 volved in constraints.
song following a so called “bag-of-features” approach [2].
Here feature values for each dimension are modeled by a          4.4 Evaluation Measures
single Gaussian, so that each feature dimension within a
song is represented by the sample mean and standard devi-          We have applied several metrics for cluster evaluation.
ation of the feature values. In addition, for each dimension     One of the most traditional evaluation measures for clus-
of low-level and mid-level features we calculate the differ-     tering [3] is normalized mutual information (NMI). NMI is
ences between the neighbor frames. This forms so called          an information-theoretic measure which shows the amount
delta features that have already proved their efficiency for     1
                                                                   http://ismir2004.ismir.net/genre_contest/index.
MFCCs. We likewise model each dimension of delta features        htm
of information shared by ground-truth cluster assignments         significant when we only have a small number of features.
(represented with a random variable Y ) and estimated clus-       For a given number of features, the percentage of songs in-
ter assignments (represented with a random variable Z):           volved in constraints is augmented by five percent in each
                                                                  step, starting with 0 and stopping at 75.
                              2 · I(Y ; Z)                           Its worth to note, that all experiments with random se-
                  NMI =                    ,               (8)    lection of songs to be constrained, have been run 10 times,
                             H(Y ) + H(Z)
                                                                  and that all clustering evaluation measures are the means of
where I(Y ; Z) = H(Y ) − H(Y |Z) is the mutual information        these runs. In addition, a random base line clustering was
between Y and Z, H(Y ) is a marginal entropy of Y , and           used as a reference, where the items of each ground-truth
H(Y |Z) is the conditional entropy of Y given Z.                  class were randomly uniformly distributed over k estimated
  As additional information-theoretic evaluation measures         clusters. Resulting values of normalized mutual informa-
we use the normalized conditional entropies by Lukashe-           tion are presented in Fig. 3. It is possible to note that the
vich [13] developed for evaluating song segmentation. These       RAW scores tend to be more smooth over the incrementing
scores – in the context of this paper named over-clustering       size of the constrained set than scores for the outlier iden-
(So ) and under-clustering (Su ) – give some insights to the      tification methods. Clustering results of LOF and RRS for
origin of the clustering errors. The errors caused by the frag-   small number of constraints – constrained data set smaller
mentation of true clusters are captured by over-clustering So     than 30% – from both Symmetric and Random Walk Lapla-
defined as                                                        cians are comparable to the clustering results with random
                                 H(Z|Y )                          constraints. On the other hand, for the high number of
                      So = 1 −           ,                  (9)   constraints and the high amount of features, the cluster-
                                 log2 NZ
                                                                  ing results of both LOF and RRS and for both Laplacians
and erroneous connection of elements of different clusters        are significantly better than clustering results with random
into one cluster is reflected by under-clustering Su              constraints, bringing an improvement of up to 0.22 points of
                                H(Y |Z)                           NMI.
                     Su = 1 −           ,                 (10)       In fact, with more than 32 features the results are con-
                                log2 NY
                                                                  siderably better for almost all sets of constraints. Cluster-
where NY and NZ is a number of clusters in ground-truth           ing with RRS seems to suffer from some instability, yet the
and estimated cluster assignments respectively.                   differences between RRS an RAW with the Random Walk
  Pairwise F-measure is defined as the harmonic mean of           Laplacian are considerable while taking into account more
pairwise precision and pairwise recall. Let MY be a set           than 32 features.
of song pairs that are in the same cluster in the ground-            We present the scores of clustering evaluation measures for
truth clustering, ex. pairs of songs having the same genre        all experiments in Fig. 2. As a representative example we
label. Likewise let MZ be a set of identically labeled song       look at the clustering results with 512 feature dimensions.
pairs that are in the same cluster according to the estimated     Here the scores for Symmetric Laplacian with RAW and
cluster assignments. Then pairwise precision (Pp ), pairwise      with LOF are considerably lower. On the other hand, the
recall (Rp ), and pairwise F-measure (Fp ) are defined as         best results are obtained from RRS with both Random Walk
                             |MZ ∩ MY |                           and Symmetric Laplacians.
                     Pp =               ,
                               |MZ |
                                                                  6. CONCLUSIONS
                          |MZ ∩ MY |                                 In this paper we presented a system for the active explo-
                     Rp =            ,
                            |MY |                                 ration of music collections via spectral clustering with con-
                                                                  straints. For the experiments we simulated the constraints
                             2 · Pp · Rp                          through the ground-truth class labels of the audio genre
                      Fp =               ,                (11)
                              Pp + Rp                             dataset. Alongside with determining the constraint candi-
                                                                  dates in a random manner, we investigated two different
where | · | denotes the number of the corresponding pairs.
                                                                  outlier identification methods. Additionally we looked into
Note that Basu et al. [3] used a slightly modified definition
                                                                  a spectral feature selection method and proved the perfor-
of pairwise F-measure, where they considered only the pairs
                                                                  mance of clustering for two versions of Laplacian for spectral
of points that do not have explicit constraints between them.
                                                                  clustering.
In our case we do not embed this information explicitly into
pairwise F-measure. In contrast, we make difference between
two evaluation scenarios – entire dataset and test part not       7. ACKNOWLEDGMENT
involved in constraints – as described in Sec. 4.3.                  This work has been partly supported by the German re-
   To simplify the comparison with the work of Peng et al. [15]   search project GlobalMusic2One 2 funded by the Federal Min-
we additionally take into consideration accuracy and purity       istry of Education and Research (BMBF-FKZ: 01/S08039B).
performance measures.                                             Additionally, the Thuringian Ministry of Economy, Employ-
                                                                  ment and Technology supported this research by granting
5.   RESULTS                                                      funds of the European Fund for Regional Development to the
                                                                  project Songs2See 3 , enabling transnational cooperation be-
  In this section we present the results for the experiments
                                                                  tween Thuringian companies and their partners from other
stated in Table 1 of Sec. 3. Each of the experiments is run
                                                                  European regions.
over the following quantities of features: 16, 32, 64, 128,
                                                                  2
256 and 512 determined through the powers of two. This                see http://www.globalmusic2one.net
                                                                  3
log-line scale is used considering that improvement is more           see http://www.songs2see.net
                                        Symmetric Laplacian                                                      Random Walk Laplacian
                             All                                   Test                               All                                     Test
               1                                  1                                    1                                     1
                                                                                                                                                               16
              0.8                                0.8                                 0.8                                    0.8                                32
                                                                                                                                                               64
RAW           0.6                                0.6                                 0.6                                    0.6


        NMI
                                                                                                                                                               128
              0.4                                0.4                                 0.4                                    0.4                                256
              0.2                                0.2                                 0.2                                    0.2                                512
                                                                                                                                                               RB
               0                                  0                                    0                                     0
                    0   25         50      75          0      25          50   75          0     25         50        75          0      25          50   75


               1                                  1                                    1                                     1

              0.8                                0.8                                 0.8                                    0.8

 LOF          0.6                                0.6                                 0.6                                    0.6
        NMI




              0.4                                0.4                                 0.4                                    0.4

              0.2                                0.2                                 0.2                                    0.2

               0                                  0                                    0                                     0
                    0   25         50      75          0      25          50   75          0     25         50        75          0      25          50   75


               1                                  1                                    1                                     1

              0.8                                0.8                                 0.8                                    0.8

 RRS          0.6                                0.6                                 0.6                                    0.6
        NMI




              0.4                                0.4                                 0.4                                    0.4

              0.2                                0.2                                 0.2                                    0.2

               0                                  0                                    0                                     0
                    0   25         50      75          0      25          50   75          0     25         50        75          0      25          50   75

                                                               Percentage of dataset involved in constraints



Figure 3: Normalized Mutual Information (Y axis) versus the percentage of the dataset that is involved in at least one constraint (X axis),
different number of selected features and different evaluated subsets (All data set and Test set). Each row is related to a particular outlier
identification method. The two first columns are related to the Symmetric Laplacian and the following two columns to the Random Walk
Laplacian. Results of columns 1 and 3 are related to the All data set, while columns 2 and 4 are related only to the Test data set.
                            Sym Raw        Sym LOF                           Sym RRS      RW Raw           RW LOF                   RW RRS        RB


                       1                                                1                                                      1

                      0.8                                              0.8                                                    0.8




                                                     Accuracy
                      0.6                                              0.6                                                    0.6




                                                                                                           Purity
 NMI




                      0.4                                              0.4                                                    0.4

                      0.2                                              0.2                                                    0.2

                       0                                                0                                                      0
                        0   25        50      75                         0         25      50         75                        0   25       50        75
                       1                                                1                                                      1
 Pairwise F−measure




                      0.8                                              0.8                                                    0.8




                                                                                                           Under−clustering
                                                     Over−clustering




                      0.6                                              0.6                                                    0.6

                      0.4                                              0.4                                                    0.4

                      0.2                                              0.2                                                    0.2

                       0                                                0                                                      0
                        0   25        50      75                         0         25      50         75                        0   25       50        75
                                                   Percentage of dataset involved in constraints



Figure 2: Values of several evaluation measures versus the percentage of the dataset that is involved in at
least one constraint (X axis). Evaluation measures, starting from upper left plot and going to lower right
plot: Normalized Mutual Information, Accuracy, Purity, Pairwise F-measure, Over-Clustering and Under-
Clustering. In each of these plots experiments presented in Table 1 are evaluated. The number of selected
features is fixed to 512. Curves plotted with ‘crosses’ state for random baseline clustering.


8.                    REFERENCES                                                             Retrieval (ISMIR), 2005.
 [1] L. Atlas and S. S. Shamma. Joint acoustic and                                       [6] M. M. Breunig, H.-P. Kriegel, R. T. Ng, and J. Sander.
     modulation frequency. EURASIP Journal on Applied                                        LOF: Identifying density-based local outliers. ACM
     Signal Processing, 2003:668–675, 2003.                                                  SIGMOD Record, 29(2):93–104, June 2000.
 [2] J.-J. Aucouturier, B. Defreville, and F. Pachet. The                                [7] C. Dittmar, C. Bastuck, and M. Gruhne. Novel
     bag-of-frames approach to audio pattern recognition:                                    mid-level audio features for music similarity. In Proc.
     A sufficient model for urban soundscapes but not for                                    of the Int. Conf. on Music Communication Science
     polyphonic music. J. Acoust. Soc. Am.,                                                  (ICOMCS), Sydney, Australia, 2007.
     122(2):881–891, 2007.                                                               [8] G. Gatzsche, M. Mehnert, D. Gatzsche, and
 [3] S. Basu, A. Banjeree, E. Mooney, A. Banerjee, and                                       K. Brandenburg. A symmetry based approach for
     R. J. Mooney. Active semi-supervision for pairwise                                      musical tonality analysis. In Proc. of the 8th Int.
     constrained clustering. In Proceedings of the 2004                                      Conf. on Music Information Retrieval (ISMIR),
     SIAM International Conference on Data Mining                                            Vienna, Austria, 2007.
     (SDM-04), pages 333–344, 2004.                                                      [9] M. Gruhne and C. Dittmar. Comparison of harmonic
 [4] S. Basu, M. Bilenko, and R. J. Mooney.                                                  mid-level representations for genre recognition. In
     A probabilistic framework for semi-supervised                                           Proc. of the 3rd Workshop on Learning the Semantics
     clustering. In Proc. of the 10th ACM SIGKDD Int.                                        of Audio Signals (LSAS), Graz, Austria, 2009.
     Conf. on Knowledge Discovery and Data Mining                                       [10] M. Gruhne, C. Dittmar, and D. Gaertner. Improving
     (KDD), pages 59–68, 2004.                                                               rhythmic similarity computation by beat histogram
 [5] J. P. Bello and J. Pickens. A robust mid-level                                          transformations. In Proc. of the 10th Int. Society for
     representation for harmonic content in music signals.                                   Music Information Retrieval Conf. (ISMIR), Kobe,
     In Proc. of the 6th Int. Conf. on Music Information
     Japan, 2009.
[11] B. Kulis, S. Basu, I. Dhillon, and R. Mooney.
     Semi-supervised graph clustering: a kernel approach.
     In Proc. of the 22nd Int. Conf. on Machine Learning
     (ICML), 2005.
[12] K. Lee. Automatic chord recognition from audio using
     enhanced pitch class profile. In Proc. of the Int.
     Computer Music Conf. (ICMC), New Orleans, USA,
     2006.
[13] H. Lukashevich. Towards quantitative measures of
     evaluating song segmentation. In Proc. of the 9th Int.
     Conf. on Music Information Retrieval (ISMIR), pages
     375–380, Philadelphia, USA, 2008.
[14] U. Luxburg. A tutorial on spectral clustering.
     Statistics and Computing, 17(4):395–416, December
     2007.
[15] W. Peng, T. Li, and M. Ogihara. Music clustering
     with constraints. In Proc. of the 8th Int. Conf. on
     Music Information Retrieval (ISMIR), Vienna,
     Austria, 2007.
[16] S. Ramaswamy, R. Rastogi, and K. Shim. Efficient
     algorithms for mining outliers from large data sets. In
     Proceedings of the 2000 ACM SIGMOD international
     conference on Management of data, pages 427–438,
     2000.
[17] M. Stein, B. M. Schubert, M. Gruhne, G. Gatzsche,
     and M. Mehnert. Evaluation and comparison of audio
     chroma feature extraction methods. In Proc. of the
     126th AES Convention, Munich, Germany, 2009.
[18] S. Stober and A. Nürnberger. A multi-focus zoomable
     interface for multi-facet exploration of music
     collections. In Proc. of the 7th Int. Symposium on
     Computer Music Modeling and Retrieval (CMMR),
     Málaga, Spain, 2010.
[19] G. Tzanetakis and P. Cook. Musical genre
     classification of audio signals. IEEE Transactions on
     Speech and Audio Processing, 10(5):293–302, 2002.
[20] C. Uhle, C. Dittmar, and T. Sporer. Extraction of
     drum tracks from polyphonic music using independent
     subspace analysis. In Proc. of the 4th Int. Symposium
     on Independent Component Analysis (ICA), Nara,
     Japan, 2003.
[21] K. Wagstaff and C. Cardie. Clustering with
     instance-level constraints. In Proc. of the 17th Int.
     Conf. on Machine Learning (ICML), pages 1103–1110,
     2000.
[22] Z. Zhao and H. Liu. Spectral feature selection for
     supervised and unsupervised learning. In Proc. of the
     24th Int. Conf. on Machine Learning (ICML), 2007.