In this paper, we describe the algorithm that has been used to carry out our plagiarism detection within the context of PAN10 competition. Our system is based on the LempelZiv distance, which is applied to extract structural information from texts. Then the algorithm tries to find outliers in the vector of distances between each fragment of the text and the whole document itself.
Next figure shows the global architecture for our intrinsic plagiarism algorithm.
This module fragments the original text in blocks. Our software offers two different possibilities: 1) fragmentation by sentences, and 2) fragmentation by paragraphs. The minimum size allowed for the fragments or text blocks is a configurable parameter in our system. It is necessary, since over a small fragment is not valid to detect the presence of plagiarism.
The current version of our algorithms includes, among others, the implementation
of the next definitions for distances:
Basile distance: proposed by Basile and others, that define a distance between two
texts x and y from its n-grams ([
RHonore distance: as described in [
Our algorithms can use one or a subset of the available distances by means of a
configurable parameter. In our detection of intrinsic plagiarism for PAN10 we have
only taken into account the LempelZiv distance, since it has been shown that
measures based on Kolmogorov complexity (using a lossless compression algorithm)
are a good way to extract structural information from texts for the intrinsic plagiarism
detection [
Next step consists of detecting which distance can be considered as an outlier in
the vector of distances between each fragment of the text and the whole document
itself. Our software implements three classical ways of detecting an outlier in a list of
data [
Interval aggregation is an optional module that can be used in the output of our system. It aggregates a group of separated detected plagiarism intervals into one interval when interval separation is smaller than a configurable threshold. It permits detecting as a unique plagiarized block some close blocks that were separated by the fragmenter. For PAN10 we did not use this interval aggregation module.
With respect to PAN10 competition, as stated above, we have only participated in the intrinsic plagiarism detection task, because of (software or hardware) bad performance of our system for external plagiarism. In this case, the configurable parameters of our plagiarism detector are: fragmentation level (sentence, paragraph), minimum length of interval (minimum length for being considered a valid sentence or paragraph), use of interval aggregation (true, false), aggregation interval (minimum distance between intervals for aggregation), minimum fragment length (minimum fragment length for plagiarism detection), active comparison distances (Basile, LempelZiv, RHonore), outlier detection method (standard deviation, percentiles, MAD), α, β and γ weights for outlier detection. Our settings, after from different tests on the training corpus PAN-PC-09, were: fragmentation level = paragraph, minimum length of interval = 200, use of interval aggregation = false, aggregation interval = 50, minimum fragment length = 200, active comparison distances = only LempelZiv, outlier detection method = standard deviation, weights for outlier detection γ = 3.0.
The detection performance that our system achieves on the training corpus PAN-PC-09, using the PAN evaluation measures, was: recall=0.185225576213, precision=0.075230788299, overall=0.0743645119788, granularity=1.71111111111.
As we noted earlier, we have only participated in the intrinsic plagiarism detection task. Since the results of the competition cover the detection of both intrinsic and external plagiarism globally, and not separately, the overall results had to be necessarily worse. In that sense, we are sure that we can greatly improve our current system with our future work. In any case, the results have not been too good at the moment. Our future work will include, in fact, the following tasks: 1) Improve intrinsic and external plagiarism performance; 2) Combine intrinsic and external plagiarism; 3) Develop the Internet module; 4) Implement new detection distances; 5) Implement new outlier detection methods; 6) Implement 'obfuscation' detection algorithms; 7) Implement a report generator module.
This work has been partially supported by the Spanish Center for Industry Technological Development (CDTI, Ministry of Industry, Tourism and Trade), through the CONTENIDOS A LA CARTA project, INGENIO 2010 Programme, AVANZA I+D 2008.