In this paper we report on our plagiarism detection system which is used to process the PAN plagiarism corpus for the tasks of Candidate Document Retrieval and Detailed Comparison. To retrieve the plagiarism candidate document by using ChatNoir API, a method based on tf*idf to extract the keywords of suspicious documents as queries is proposed. An Lucene ranking method is used for plagiarism candidate document reduction. And a detailed comparison algorithm to get the web pages that are actually sources for plagiarized passages is applied. To extract all plagiarism passages from the suspicious document and their corresponding source passages from the source document, a plagiarism detection method combined with semantic similarity and structure similarity is proposed. Semantic similarity is calculated by Vector Space Model while structure similarity is calculated by our own method. We use information retrieval to get candidate pairs of sentences from suspicious document and potential source document. A method which is called Bilateral Alternating Sorting is applied to merge pairs of sentences. Those plagiarism candidate result pairs are screened in post-processing.
The rapid development of network technology, including large numbers of search engines, document
repositories, translation software systems, not only provides people with the various knowledge
acquisition channel, but also opens the door for text plagiarism. Plagiarism generally refers to the
illegitimate use of someone else’s information, text, ideas, etc. without proper reference to the original
source of these borrowings [
In recent years, many well-known organizations carried out evaluation, international competitions
and conferences focused on the plagiarism detection. And PAN[
In this paper, we introduce a method for Candidate Document Retrieval and Detailed Comparison sub-tasks. In the Candidate Document Retrieval sub-task, a method based on tf*idf to extract the keywords of suspicious documents as queries was proposed. A scoring method was used to plagiarism candidate document ranking. In the Detailed Comparison sub-task, a plagiarism detection method based on Vector Space Model(VSM) and Overlapping Measure Model at the sentence level was presented. Bilateral Alternating Sorting was designed to merge the pairs of plagiarism sentences, and those plagiarism candidate result pairs were screened in post-processing.
The rest of the paper is organized as follows. Section 2 describes the overview of related work. Then, Section 3 and Section 4 describe the method for Candidate Document Retrieval sub-task and Detailed Comparison sub-task while Section 5 includes the evaluation results. Finally, Section 6 discusses the main points of this study and proposes future work directions.
Most often, copy is an open copying, and plagiarism mainly refers to plagiarize other's language, charts,
formulas or research ideas and then edited, pieced together, modified and added to their own papers,
writings, project applications, data files, computer code material, and so on, and set him/herself up as
the author. The research of formal language text plagiarism starts earlier for its strict formal syntax,
clear semantics of expression, easy analysis and processing. Since Ottenstein [
The core problem of text plagiarism detection is to determine whether the plagiarism exists and
how to measure the similarity degree. For text similarity problem, many researchers have put forth
some effective detection methods. It mainly includes (1) Similarity calculation method based on
statistics, such as [
Finger Printing and Word Frequency are the mainly method to recognize the plagiarism. Finger Printing is fast, simple and effective, suitable for large-scale computing. Word Frequency method first statistics the number of each word in the document to constitute the feature vector of the document, then use vector dot product, cosine law, the correlative frequency model, etc. to measure the similarity of two documents. Word Frequency statistical method has a high precision but its speed in not fast than Finger Printing technology.
Although a lot of plagiarism detection systems are better to complete the simple text copy detection, detection for English has also made some achievements, there are also some questions and works which have not received much attention yet.
First, plagiarism detection filed has not effective techniques to filter the plagiarism source in massive data corpus. The corpus of reference documentation is limited to the data of several G in the existing plagiarism detection system. However, over time, reference documentation set of documents to be detected grows increasingly large and source document set is not limited to be a few G magnitude data. Massive data processing and the growing amount of data increase the difficulty of plagiarism detection. Existing methods are powerless in the face of dealing with in the data of several T. Timely and effective in a limited time to find suspicious from the source document to be detected is the key to the establishment of effective plagiarism detection system.
Second, the performance of existing systems which are based on matching and statistics techniques is not satisfactory for the practical application. They still have the wrong check, leakage check, especially non-straightforward copy which has a low distinction degree, and has difficult to achieve accurate identification of plagiarism. And they can not deal with the complex text plagiarism, especially for two articles which have the same meaning and converting the writing method.
Given a suspicious document and a web search engine, the task is to retrieve a set of candidate source
documents that may have served as an original to plagiarize from[
First, each suspicious document s is preprocessed, including stemming, removing stop words and replacing figures. The queries for every s are coming from the top queryGroup*queryLength terms which are the top n sorted by tf*idf values from high to low in each paragraph of s, where queryGroup is the group number of queries and queryLength is the term number of each query group. In testing phase, the queries we used is 2*5.
ChatNoir API is applied to retrieve the plagiarism candidate document for each query group. Then, put retrieved top n results in the result set of plagiarism candidate document and use ChatNoir API to download them, where n=10.
The plagiarism candidate document is preprocessed into non-overlapping plaintext passages. We index them and use query of 2*5 for each passage of each suspicious document to retrieve by using an Lucene scoring method for filtering the plagiarism candidate in [11].
And we take top n of retrieving results of each query as results. Last, we use an algorithm to get the web pages that are actually sources for plagiarized passages which will be described in following PAN Detailed Comparison Task.
Given a pair of suspicious document and potential source document, the task is to extract all
plagiarized passages from the suspicious document and their corresponding source passages from the
source document [
In the pre-processing part, the suspicious documents and plagiarism candidate source documents will be processed in some ways, including removal of special characters and whitespace, case transformation, removal of stopwords and stemming.
Since the passage is the smallest unit that an author expresses an independent and complete view and the sentence is the basic structure of one passage, we choose sentences as the chunks. The following is the steps of processing the candidate plagiarism passages detection.
Step1: Suspicious documents and source documents are divided according to the sentence. After that we index all the sentences in the source documents. Each sentence in the suspicious documents will be retrieved in the index as a query. This kind of process is called sentence similarity retrieval. We regard suspicious passage S and reference passage R in source document as pairs of plagiarism candidate sentence which their cosine distance is greater than t1 to get semantic similarity, as shown in formula 1: n k1 wSk wRk Sim(S,R) cos n t1 ( wSk2 )( kn1 wRk2 ) k1 (1) where Sim(S,R) is the similarity degree of S and R, Θ is document vector angel, WSk and WRk are the weight of S and R respectively,t1 is threshold. We used t1=0.42.
Step2: We screen plagiarism candidate sentence to get structure similarity by using formula 2. 2 Min(NIS (t),NIR (t))
tIS IR T
t2 IS IR (2) where NIs(t) and NIR(t) are the number of the terms which are overlapping in the suspicious sentence and reference sentence, Min(NIs(t),NIR(t)) is the smallest one of NIs(t) and NIR(t) , t2 is the threshold. We used t2=0.32.
Those sentence pairs which are not only in line with formula 1 but also formula 2 will be regarded as the plagiarism candidate sentence pairs.
Step 3: Merge the scattered plagiarism candidate sentence pairs which are got by above process method. This process is the recovery of a complete plagiarism case. We design a Bilateral Alternating Sorting algorithm to merge the suspicious sentence and reference sentence which guarantee the suspicious sentence and reference sentence are adjacent. Because of patent pending, this method is inconvenient stated here. The passage pairs after merging are called the candidate result pairs.
We use formula 2 to screen candidate result pairs which perhaps the non suspicious passage pairs in post-processing phase .We used t3=0.30 this time.
The results of Candidate Document Retrieval sub-task and the Detailed Comparison sub-task are summarized in Table 1 and Table 2.
Our method is evaluated by PAN2012@CLEF and compared with the other plagiarism detection systems. The evaluation results of our method in the competition were excellent. With the PAN-09, PAN-10, PAN-11 and PAN-12 corpora, our method showed a great advantage and produced a high performance. Results show that our system’s overall performance, especially the recall is higher than most of the other methods for most kinds of plagiarism cases. The plagiarism detection method we proposed is flexible and scalable, the time limit is reasonable. In our case we only needed one mainstream server to run the complex plagiarism detection system. Furthermore, we will aim to determine the threshold boundary more reasonably. The synonym replacement and the translation would also be introduced into the plagiarism system. We will work on a better Candidate Document Retrieval method.
Acknowledgements. This work is supported by NSF of China(60970057,60903083). Remark: This work was done in Heilongjiang Institute of Technology.