In this paper, we describe our approach at the PAN@CLEF2013 plagiarism detection competition. In sub-task of Source Retrieval, a method combined TF-IDF, PatTree and Weighted TF-IDF to extract the keywords of suspicious documents as queries to retrieve the plagiarism source document is proposed. In sub-task of Text Alignment, a method based on sentence similarity is presented. Our text alignment algorism and similar sentences merging algorism, called Bilateral Alternating Merging Algorithm, are described in detail.
The great development of Internet makes it easier for people to search, copy, save,
and reuse online sources. Copying another author’s text and claiming its authorship is
called plagiarism[
The task of source retrieval is to retrieve all plagiarized sources while minimizing
retrieval costs[
TF–IDF[
Pat tree is an efficient data structure successfully used in the area of keywords
extraction in the field of information retrieval. It was developed by Gonnet[
We believe that the words exist in the paragraph title and chapter headings contain more information. We give each term a weight according to its position. The final weighted TF-IDF value of each term is calculated by its TF-IDF value timing its weight. The weight is set to 3.6 if the term appears in the title of the article, while the weight is 2.5 when the term locates the first sentence or the last sentence of each paragraph.
All queries from a given suspicious document are extracted on the basis of the above methods and they are executed sequentially according to their priority.
Firstly, we extracte the top 20 words in the suspicious document according to their TF-IDF weights. The top 5 ranked keywords combine into the first query, and the next top 5 ranked keywords form the second query, and so on. In this manner, we get four groups of queries, and each query contains 5 terms.
Secondly, we get many groups of keywords based on PatTree which consist of different numbers of terms. We choose the keywords which composed by 2, 3, 4 and 5 terms and rank them by their sum of TF-IDF value and the frequency counting in the suspicious documents. We call them 2-gram, 3-gram, 4-gram and 5-gram PatTree keywords. One 2-gram PatTree keyword, one 3-gram PatTree keyword and two 4-gram PatTree keywords are regards as following four queries. Especially, 5gram PatTree keywords are chosen to all the other queries. We discard the keywords whose the number of nouns and verbs under threshold 3. The queries which have no any term existing in the list of top 10 weighted TF-IDF also are filtered out by us.
Table 1 shows the combination of queries.
Rank 1 to 2 3 4 5 to 6 7 to 8 other
Top 10 TF-IDF 2-Gram PatTree Keyword 3-Gram PatTree Keyword 4-Gram PatTree Keyword
Top 10 to 20 TF-IDF 5-Gram PatTree Keyword
Table 1. Combination of queries.
After committing all queries we extract from a suspicious document, we decide whether to actually download the web pages or not according to the cosine similarity between the snippet and the suspicious document. We extract about 40 queries for each suspicious document.
According to the five performance measures that PAN@CLEF2013 employs, the more web pages are download, the better recall could be got. For example, if the downloading number of web page for each document returned is set to 1000, the recall can reach 0.5719. By setting this value 600 we could lower the total number of web pages downloaded by 400 with only 2.05% of recall lost. To ensure the evaluation measures of Number of queries submitted, Number of queries until the first actual source is found and Number of downloads until the first actual source is downloaded better in this evaluation, we try to improve the measure of precision and recall of web pages downloaded regarding actual sources of a suspicious document, and reduce the measure of Number of web pages downloaded.
In the experiment, we have found that the queries constructed by top 10 TF-IDF words got a better result. The experiment result without using snippet filter on the training data of Source Retrieval is shown in Table 2.
Given a pair of documents, the task of text alignment is to identify all contiguous
maximal-length passages of reused text between them[
Algorithm 1 describes the Text Alignment algorithm of plagiarism detection. Algorithm 1. Text Alignment algorithm of the plagiarism detection Input: Suspicious document dplg，the plagiarism alternative documentation set Dsrc Output: The collection of the pairs(splg , ssrc) from the plagiarism fragment of the suspicious document splg and the plagiarism source document fragment ssrc Pre-Processing dplg and Dsrc Feature represent dplg and Dsrc for each dsrc ∈ Dsrc {
Measure sentences similarity of dplg and dsrc } Merge similar sentences to similar passages (splg , ssrc)← Post-Processing the pairs of similar passages return the set of passage pairs (splg , ssrc)
The result of seeding is that we get a number of scattered one-to-many sentence pairs. We designed an algorithm called Bilateral Alternating Merging Algorithm to combine these fragments into a larger fragment as plagiarism fragment. The description of the algorithm is shown as follows.
Input: Similar sentence list casesList Output: The collection of the pairs(splg , ssrc) from the plagiarism fragment of the suspicious document splg and the plagiarism source document fragment ssrc 1 public void merger(List<String> casesList, int sign) { 2 if (sign = -1) { 3 leftSort(casesList); 4 if (end >= leftEndge && end <= rightEndge) { 5 resultList.add(dto); 6 } else { 7 for (i = 1..casesList.size()) { 8 if (|now – last| > dist) { 9 checkAdjacent(); 10 merger(); 11 } 12 } 13 } else { 14 rightSort(casesList); 15 if (endgeFirstInfs=endgeEndInfs && end >= leftEndge&& end <= 16 rightEndge) { 17 resultList.add(dto); 18 } else { 19 for (i = 1..casesList.size()) { 20 if (|now – last| > dist) { 21 checkAdjacent(); 22 merger(); 23 }
In paper [
The experiments are carried out in the PAN@CLEF2012 training data set. Table 4 shows the results of our Text Alignment sub-task which applied different parameters.
02_no_obfuscation 03_artificial_low 04_artificial_high 06_simulated_paraphrase Overall
In the sub-task of Source Retrieval, we apply a method based on TF-IDF, Weighted TF-IDF and PatTree to extract the keywords to compose the queries of suspicious documents. The number of queries submitted is reduced by two filters. The performance measures of Number of queries until the first actual source is found and Number of downloads until the first actual source is downloaded get the promotion by combining and ranking the queries. To achieve better performance measure recall and precision, we try to keep a balance between Number of web pages downloaded and Precision and recall of web pages downloaded regarding actual sources of a suspicious document.
In the sub-task of Text Alignment, our method is more adaptable to obfuscation plagiarism. We achieve the better performance in sub-corpus 03-random-obfuscation, 04-translation-obfuscation and 05-summary-obfuscation. But the performance for sub-corpus 02-no-obfuscation still remains to rise.
There are many high-performance methods in dealing with no-obfuscation plagiarism, therefore the current results show that it is possible to build up a classifier based on different plagiarism types. Furthermore, we can use different methods to deal with different plagiarism problems to obtain a better performance. Moreover, more efforts are required to figure out more plagiarism features to be used in the design of the plagiarism type classifier to improve the performance.
Acknowledgements This work is supported by NSF of China(61272384). Remark This work was done in Heilongjiang Institute of Technology.