The PAN 2017 Author Clustering task examines the two application scenarios complete author clustering and authorship-link ranking. In the first scenario, one must identify the number (k) of different authors within a document collection and assign each document to exactly one of the k clusters, where each cluster corresponds to a different author. In the second scenario, one must establish authorship links between documents in a cluster and provide a list of document pairs, ranked according to a confidence score. We present a simple scheme to handle both scenarios. In order to group the documents by their authors, we use k-Medoids, where the optimal k is determined through the computation of silhouettes. To determine links between the documents in each cluster, we apply a predefined compressor as well as a dissimilarity measure. The resulting compression-based dissimilarity scores are then used to rank all document pairs. The proposed scheme does not require (text-)preprocessing, feature engineering or hyperparameter optimization, which are often necessary in author clustering and/or other related fields. However, the achieved results indicate that there is room for improvement.
Author clustering (AC) is a relatively new sub-discipline in the field of authorship
analysis and is offered again by PAN [
The PAN 2017 Author Clustering task examines two application scenarios: complete author clustering and authorship-link ranking. In the first scenario, one must identify k ? Corresponding author. 1 A shared task is an event, where researchers and practitioners aim to solve or at least make progress on open academic problems. (the number of different authors within D) while assigning each D 2 D exactly to one cluster C 2 C = fC1; C2; : : : Ckg. In the second scenario, one must establish authorship links between the documents fDa; Db; Dc; : : :g in each cluster C and provide a list of document pairs (Da; Db), ranked according to a confidence score 2 [0; 1], where indicates how likely Da and Db are to be written by the same author.
We present a simple AC approach based on the k-Medoids algorithm and the
computation of so-called silhouettes to determine the optimal k. Instead of using distances
computed through well-known metrics such as Manhattan or Euclid, we decided to
experiment with compression-based dissimilarity scores. To compute these scores we
apply a compression-based model consisting of a predefined compressor and a
dissimilarity measure designed for compressed text files. Compression-based models have
been applied widely across different authorship analysis tasks including authorship
attribution [
Our approach has a number of benefits. First, it does not require the explicit definition, selection and/or extraction of features as these are implicitly handled by the compression model. Second, our approach does not rely on a threshold which is often mandatory to judge whether two documents are written by the same author. Third, our approach does not involve machine learning methods and, thus, also not requires hyperparameter optimization (which is typically needed for classification/recognition). Fourth, the approach does not even need a specific preprocessing regarding the documents, which further reduces its complexity. 2
This section describes our approach, which is broken down into both scenarios complete author clustering and authorship-link ranking. 2.1
Task 1: Complete author clustering
Compressing distances: As mentioned in Section 1 we waive the usage of a traditional
distance function and instead use a compression-based dissimilarity measure. Given this
measure, we can determine the "nearness" between two documents. However, before we
can use this measure we require a compressor to obtain the compressed representation
of the documents. Here, we decided to use one of the most powerful available
compressor PPM2 (Prediction by Partial Matching), which has been used excessively in various
fields and domains and led to promising results. Once the documents are compressed
via PPM, we apply a dissimilarity function to measure how (dis-)similar two documents
are to each other. As a dissimilarity function we chose the CBC (Compression-based
Cosine) measure, proposed by Sculley and Brodley [
Clustering via k-Medoids: In the PAN 2017 Author Clustering task the simplification
is taken that all documents are single-authored. In practice this is not very realistic as
it can often occur that documents (or text fragments such as paragraphs, sentences or
phrases) are authored by different authors. However, we take advantage of the fact that
all documents within the PAN corpora are single-authored and chose a simple
partitional clustering algorithm that generates disjoint clusters. As a clustering algorithm
we decided to use k-Medoids (proposed by Kaufman and Rousseeuw [
The most common realization of the k-Medoids clustering method is the PAM
(Partitioning Around Medoids, [
Measure the quality of the clustering. Since for each problem the number of authors k
is not known beforehand, a strategy is needed to measure the clustering quality, in order
to determine the "optimal" k. Our strategy is based on the computation of silhouettes
(proposed by Rousseeuw [
SC = 1 nC
X s(D) D2C Here, The calculation of a silhouette value s(D) is calculated as follows: 2 In fact we use the PPMd variant, implemented in the C# library SharpCompress, offered by Adam Hathcock available under https://github.com/adamhathcock/sharpcompress. As a concrete implementation we used Michael Bone’s port of Dmitry Shkarin’s PPMd Variant I Revision 1. 3 Note that we skip the case n = 1, as we assume that for each problem there are two or more corresponding authors.
Algorithm 1: k-Medoids, adapted to compression-based dissimilarity scores. return C = fC1; C2; : : : ; Ckg; 1. Let s(D) 2 [ 1; 1] denote a silhouette value for a document D 2 D, which was assigned to a cluster Ca. We first compute a(D) = the average dissimilarity of D to all other documents in the same cluster Ca. 2. For every other cluster C 6= Ca, we calculate the average dissimilarity b(d) = dist(D; C) between D and each document in C. The cluster with the smallest average dissimilarity to D is denoted by Cb. 3. Finally, we compute s(D) as follows: For the case that the initial cluster comprises only one document (jCaj = 1) or that a = b holds, we set s(D) = 0. For the case that a(D) < b(D) we calculate s(D) = 1 ba((DD)) and otherwise s(D) = ba((DD)) 1. 2.2
Task 2: Authorship-link ranking In order to establish authorship-links within each cluster, we first modified the CBC measure in order to calculate similarity (instead of dissimilarity) scores as follows: CBCsim(x; y) = Given CBCsim(x; y), we applied it on each document pair within a cluster and sorted the resulting list in a descending order. Note that the authorship-link ranking step could be also performed through an arbitrary authorship verification method. However, we tried to keep the approach as compact as possible. Therefore, we only made use of PPM to compress the documents and calculate their similarity to each other by using CBCsim( ). 3
Since our approach does not require any type of training, there was no need to split the given training corpus into two sub-sets in order to apply hyperparameter learning on one set and the evaluation on the second set. Besides the PAN 2017 AC training corpus we also used the training corpus from PAN 20164. The results regarding both corpora are listed in Tables 1-6. 3.1
PPM: Optional parametrization As stated in this papers, our scheme does not require any type of training. However, this is only true, because we used a predefined (hard coded) parametrization regarding the PPM compressor within the involved C# library. In fact, there are two tweakable parameters (AllocatorSize and ModelOrder) that aim to improve the compression results. For AllocatorSize, we could not observe any influence regarding the author clustering results, irrespective of which values were used. Therefore, we waived 4 Note that at the time this paper was written, the test corpus was not publicly released. to train an "optimal" value for this parameter and, instead, used the default setting of 224 = 16; 777; 216.
In contrast, we observed for ModelOrder slight variations regarding the author clustering results, during initial experiments. Hence, we applied our scheme on both training datasets (PAN 2016 and PAN 2017), in order to consider, if it make sense to discard training and, instead, to use the default parameter setting of 6 (in total there are 15 possible values, ranging from 2 to 16). As can be inferred from the results (given in Figure 1), the default parameter setting is very close to the average across all possible parameter settings. As a consequence, we decided to discard the training for this parameter and to use the default (hard-coded) setting. 3.2
Other experiments Besides k-Medoids we also experimented with the density-based clustering method DBSCAN (Density-Based Spatial Clustering of Applications with Noise.), where we also used compression-based dissimilarity scores rather than distances. Our intention was to eliminate the determination of k, not only to reduce the approach’s complexity, but also to save runtime as only one scan through the documents is needed. However, instead of the expected reduction it added more complexity as both density parameters " (maximum radius of the neighborhood) and minPts (minimum number of points required to form a dense region) require training. In addition, it turned out that after training DBSCAN still performed worse than k-Medoids on both training corpora PAN-2016 and PAN-2017. On average, DBSCAN achieved only 80% of k-Medoids’ F-Bcubed scores. Therefore, we discarded this approach. 4
We proposed an experimental approach to cluster texts by their authors by using kMedoids with compression-based dissimilarity scores. On the plus side, our approach is quite simple and entirely independent from feature engineering, threshold determination (regarding the authorship-link ranking sub-task), (text-) preprocessing as well as hyperparameter optimization. On the negative side, the proposed approach does not perform very well, which might have a number of reasons. We noticed for example (after the submission deadline of the software) that the compression-based dissimilarity measure does not fulfill even one of the required properties of a real distance-based metric, which are identity5, symmetry6 and triangle inequality. Especially the symmetry 5 For example, when we compress a document x and apply CBC(x; x) we obtain as a dissimilarity measure the score : 0.117647. This value is somehow confusing as we might expect 0 when we are used to work with real distance metrics. 6 For example, consider we have two different documents x and y. Computing CBC(x; y) returns 0.6459, while CBC(y; x) returns 0.6852. 16 15 14 13 12 r 11 e rd 10 O l 9 e do 8 M d 7 M P P 6 5 4 3 2 property leads to an unexpected behavior, due to the implication that the order of the compressed documents matters when applying the compression-based dissimilarity on them. As future work we therefore need to examine for which cases compression-based models are applicable. Currently, we believe that they are well suited for establishing authorship-link rankings, but for clustering alternative strategies might be more promising (and reliable).
Acknowledgments This work was supported by the German Federal Ministry of Education and Research (BMBF) in the funded project EWV (award number: 13N13500).