In this paper, we report our approach in detecting external plagiarism. For the pre-processing stage, we identify non-English documents and translate them into English using an online translator tool. Then we index and retrieve the top documents that are similar to the suspicious documents. We divide the retrieved documents into passages where each passage contains twenty sentences. The plagiarism is detected by identifying the number of overlapped words between suspicious and source passages.
Nowadays, plagiarism happen easily and more difficult to detect. With the advances of technology, especially the Internet, plagiarism can happen across languages and has different level of obfuscation. People can easily copy and paste, paraphrase, or translate websites, papers, or other sources from the Internet without mentioning its source and acknowledge it as their own work. This situation motivates in constructing an accurate automatic plagiarism detector. A plagiarism detector is a tool to detect if a suspicious document contains plagiarized work.
In recent years, some research in the text plagiarism detection have been published
and developed. Mozgoyov et.al.
According to Potthas et al.
Grozea et.al.
In this paper we report our approach in detecting plagiarism (external plagiarism). The remaining of this paper is organized as follows: section 2 discusses our methods in plagiarism detection, section 3 describes the evaluation and section 4 is the conclusion.
In this section, we describe the method that we use in our plagiarism detection. There are four main steps in our detection method such as preprocessing stage, finding candidate documents, extract similar passages, and post-processing stage. 2.1
The pre-processing phase is mainly analyzing the corpus. The PAN ’10 corpus1 consists of 11.148 source documents and 15.925 suspicious documents. The corpus not only contains English documents but also several other languages. The external plagiarism cases also include the cross-lingual plagiarism cases. So, at the beginning we identify the language used in the documents using an automatic language identifier. The result shows that the non-English documents only occur in the source document set. The language identifier recognizes 10.480 English documents, 474 German documents, and 194 Spanish documents. Then we translate all non-English documents into English using an online language translator. We substitute the non-English documents in the corpus with their translated documents. 1 http://pan.webis.de/ 2.2
The procedure in finding candidate documents is the same as document retrieval using suspicious document as queries. In this phase, we index the overall source documents and use suspicious documents as queries. We use Lucene2 to index and retrieve the corpus. Lucene is an open source information retrieval system based on combination of Boolean Model and Vector Space Model. During the indexing process, we remove the stopwords, however we do not apply any stemming algorithm. In this work, for each suspicious document (as query), we retrieve the 10 most similar source documents. 2.3
We divide the top 10 source documents and suspicious documents into small passages. Each passage contains 20 sentences. Then we index and retrieve passages that are similar to the sections found in the source documents. We only use the top-5 similar source passages for each suspicious passage.
In the post-processing phase, we analyze both of the pair passages. We filter the
top-5 most similar source passages by removing pair passages that have low
similarity score. After that, we compute the overlapping n-grams
We don’t have time to try our method using the training corpus, so the evaluation
is only done using the testing corpus. Based on the evaluation measure given by
the organizer
Our result show that our method performs quite good precision score (we were 4th for this parameter), but it has very low recall score. In other words, for the precision score, 91.14% of our detections are correct while 8.86% are incorrect. On the other hand, the recall means that our detector can only detect 26.2% of the overall plagiarism cases.
Based on our result, we need to explore further in terms of plagiarism with different level of obfuscation. The translation process at early stage is quite effective to overcome cross-language plagiarism, but in the detailed step, passage retrieval and n-grams overlapping technique just can handle exact match plagiarism. Plagiarism using word modification such as the use of synonym, word reordering, and paraphrasing still can’t be identified using our method.
We report our participation in identifying external plagiarism in CLEF 2010. We apply N-grams overlapping words to measure the plagiarism between pair passages found in the documents. Our result achieves high precision (0.9114), but still low in terms of recall (0.2620). This method can identify the cross-language plagiarism, however it fails to detect plagiarism with various word modifications. In the future we will include words variations and develop method to detect plagiarism with different level of obfuscation.