Plagiarism is well known problem of the day. Easy access to print and electronic media and ready to use material made it easy to reuse the existing text in new document. The severity of the problem is much reduced in monolingual context by the automated and tailored effort made by the research community but the issue is yet not properly addressed in cross language (CL) text reuse. Any story or article written in any source language like Urdu is simply translated in target language like English and translator claims it as his own. Availability of standard and simulated resource address the issue and act as test bed for analyzing and implementing available plagiarism detection approaches. The research work is aimed at enriching the available cross- language corpus and on the other hand providing a benchmark corpus to Cross Language Plagiarism (CLP) domain.
Text reuse is the process of developing a new document using the data of existing documents. Plagiarism is a most familiar type of text reuse. In general, plagiarism is considered as reuse of thoughts, procedures, outcomes, or words without clearly showing the original source. The size of text that is reused varies from case to case. In some conditions authors use only phrases, sentences or passages to create new document while in some conditions, word by word document is reused to create a new document. To create a new document data can be collected from different source documents. In some conditions entire document of original text is reused to create new document. Possible ways to detect plagiarism are (1) Intrinsic Plagiarism Detection- indicating whether all passages written by single author and (2) Extrinsic Plagiarism Detection- pointing all sources from where passages are used to create the suspicious document [18].
Plagiarism has crossed the language boundaries now like Urdu to English or any.
Translational technologies are giving new ways of plagiarism, known as cross language
plagiarism (CLP). In cross language plagiarism, source material is translated from one
language to another and then translated data is reused to develop a new document
without giving references of the original source. Generally such unattributed text reuse is
also labeled as plagiarism [
Availability of ready to use data in different formats and in multiple languages on internet is also boosting the case of CLP. Student assignments, and newspaper stories and articles are hot domains for CLP as education and information has no barriers and boundaries. CLP needs to develop a benchmark corpus having source and target language document pairs to detect any level of plagiarism.
Urdu is a language with more than 100 million native speakers1. Few corpuses
are developed for cross-language information retrieval (CLIR) [
This research is aimed at generating a standard corpus in Urdu-English language pairs. The corpus will serve as base for CLP detection and analyzing multiple evaluation techniques in context of performance. Three levels of plagiarism (Near Copy, Light Revision, and Heavy Revision) enabled it to detect plagiarism at different levels. Automated and manual effort to generate suspicious document made the corpus more realistic and precise.
The rest of the paper is organized as follows. Section 2 summarizes the related work. Section 3 describes corpus generation process in detail. Analysis about corpus is presented in section 4. Finally, section 5 concludes the paper. 2
Generation of corpus using simulated and artificial approach as recommended by
Potthast et al. is in practice now [
In spite of the fact that research community is addressing the plagiarism issue potentially, it is majorly yet limited to monolingual aspect. The minor effort made in cross lingual aspect of the problem is also limited to few European languages like Spanish and German as source and English as suspicious in source-suspicious language pairs.
Different cross lingual corpora like English-Spanish [19] and English-German
corpus [
Applying CLP detection approaches on multiple corpora have also been done by
Ceska et al. [
For the PAN 2015 Text Alignment task, we submitted a cross-language corpus (UrduEnglish language pair) for evaluating the performance of CLP detection system. The CLUE corpus contains simulated cases of plagiarism (source fragments are in English and suspicious ones in English). 3.1
Generation of Source-Suspicious Fragment Pairs To generate source-suspicious fragment pairs, we collected source texts from two domains: (1) computer science and (2) general essay topics. All the source fragments were collected from Wikipedia (http://ur.wikipedia.org/wiki/urdu in footnote). It is likely that the amount of text reused for plagiarize may vary from a phrase, sentence, paragraph to entire document. Therefore, the source fragments were divided into three categories: (1) small (less than 50 words), (2) medium (50-100 words) and (3) large (100-200) words. Table 1 shows the distribution of source-suspicious fragment Paris.
To generate simulated cases of plagiarism participants (volunteers), who were university students (undergraduate and postgraduate) were asked to rewrite the source fragment (in Urdu) to generate the plagiarized fragment (in English) using one of the three methods.
i. Near Copy: Participants were told to automatically translate the source fragment to generate the plagiarized fragment. ii. Light Revision: Using this approach, the plagiarized fragment was created in two steps. In the first step source fragment (in Urdu) is automatically translated into English. In the second step, the translated fragment is passed through an automatic text rewriting tool called Article Rewriter1 to generate the plagiarized fragment (i.e. light revision of the source fragment). iii. Heavy Revision: Participants were instructed not to use the automatic machine translation tools for generating heavy revision of the source text. Instead, they were asked to manually translate the original source text in such a way that it looks like a paraphrased copy of the source text. The proposed corpus contains total 1000 documents (500 source documents (in Urdu) and 500 suspicious documents (in English)). All the documents in the corpus are collected from freely available online resources. A document in the corpus belongs to the domain of computer science or general topics. Computer science topics (Total 50) mainly includes: Free software, Open Source, Binary Numbers, Database Normalization, Artificial intelligence, Robotics, Mobile Apps, Yahoo, MSN, Google, Whatsapp, Android, twitter, Facebook, RUBY language, Gmail, Skype, Daily motion, HTML and few others. General domain topics were also same in count and mainly include: Global warming, Muhammad Iqbal, Capitalism, Bookselling, Mosque, Pakistan Air Force, Two-Nation theory, Cricket, Fashion, Capitalism, Lahore Forte, Badshahi Masjid, Globalization and few others. Out of 500 suspicious documents, 270 are plagiarized and remaining 230 are non-plagiarized. Only one source-plagiarized fragment pair was inserted into one source-suspicious document pair. Computer science source-plagiaries fragment pairs were inserted into computer science source-suspicious documents and similarly source-plagiarized fragment pairs on general topics were inserted into sourcesuspicious document pairs which belonged to the domain of general topics.
Out of 270 source-plagiarized fragment pairs, 180 are from Computer Science domain and 90 from General topics domain.
All the source-plagiarized fragment pairs were randomly inserted into source- suspicious document pairs. 4
The developed corpus is divided into source and suspicious documents. Although the manual revision is done on each source fragment to generate its NC, LR and HR version but the order of sentences was kept same. Manual revision was done to overcome issues generated by automatic translation tools outcome. Providing Source (Urdu) version to participant for generating its Heavy Revision (HR) made the plagiarized text more realistic. 5
The paper describes the corpus creation process for detection of plagiarism in cross language domain of Urdu-English pairs. The corpus can be used as benchmark or test bed for upcoming tasks of performance evaluation among different plagiarism detection techniques. In future we intend to increase the size of corpus. 6
Following data sets were observed and most of the xml features including length and offset of the fragment inserted in source and suspicious documents were found correct in all data sets. A mismatch was also found in few cases due to newline and some special characters. Dataset wise other findings are described as: – cheema15-training-dataset-english
Different folders are used to consider cases of plagiarism at undergrad, Master and Ph. D levels. Fragments are inserted at character level at random places. Source to suspicious ratio is on to one as single source fragment is used to make a document suspicious. Obfuscation strategy is almost paraphrasing with good quality Pair Entry / Example suspicious-document0099source-document0391.xml suspicious-document0259source-document0189.xml suspicious-document0309source-document0321.xml suspicious-document0386source-document0186.xml suspicious-document0485source-document0447.xml
Type /Artificial / Simulated Simulated
Quality of Plagiarism
Well paraphrased Simulated Simulated Simulated Simulated
Good Well paraphrased Well paraphrased – palkovskii15-training-dataset-english
Multilingual features although described but obfuscation is limited to English only. Fragments are inserted at word level at random places in suspicious document.
Source and suspicious documents are of large size and from general domain. Pair Entry / Example suspicious-document00021source-document02467.xml suspicious-document00067source-document02563.xml suspicious-document00081source-document03075.xml suspicious-document00380source-document00270.xml suspicious-document00407source-document02140.xml
Type /Artificial / Simulated Artificial
Quality of Plagiarism
NEAR COPY Artificial Artificial translation-chain translation-chain – mohtaj15-training-dataset-english
Multiple fragments are inserted in single document at random places. In most of the cases 3 fragments are inserted at character level. Placement of fragments is at random places in source and suspicious documents. Although in few cases fragments in source and suspicious documents were found irrelevant but dataset is well composed overall. Large sized documents from general domain are used.
Type /Artificial / Simulated Artificial
Quality of Plagiarism NEAR COPY Pair Entry / Example suspicious-document110926source-document307308.xml suspicious-document179883source-document517709.xml suspicious-document235057source-document534046.xml suspicious-document102450source-document106487.xml suspicious-document405184source-document26685.xml suspicious-document105415source-document149775.xml suspicious-document157936source-document198805.xml
Artificial Artificial Artificial Simulated Artificial
Simulated – kong15-training-dataset-chinese
Same text is used to suspect many documents. Small sized dataset with only 4 suspicious and 78 source documents. Suspicious text is inserted at consecutive locations probably at character level. Both source and suspicious documents are in Chinese but documents also have large English text in few cases. Quality of plagiarism cannot be judged. Poor Good Good Good – khoshnava15-training-dataset-persian
A data set with 720 suspicious and 802 source documents. Almost one-to-one source to suspicious ratio is there. Artificial type of plagiarism cases with no obfuscation strategy mostly. Both source and suspicious documents are in Persian therefore quality of plagiarism cannot be judged. – Asghari15-training-dataset-english-persian
Large data set with 15959 source and 5470 suspicious documents. Most of the Plagiarism cases are artificially generated. Due to English to Persian nature quality of plagiarism cannot be judged properly. Formation of dataset is fine. – alvi15-training-dataset-english
A data set with 70 source and 90 suspicious documents. Three types of obfuscation strategies are used: character substitution, synonym replacement and human retelling. One source fragment is used in different obfuscation strategies to suspect the suspicious document. Insertion is at sentence level and almost near copy or exact copy of source fragment is used in suspicious documents. There is some difference in the source length, source offset, suspicious length and suspicious offset because of new line character.
Pair Entry / Example Type /Artificial / Simulated suspicious-document00003.txt Retelling source-document00002.txt suspicious-document00043- Retelling source-document00018.xml suspicious-document00102- Retelling source-document00040.xml suspicious-document00128- Automatic source-document00078.xml suspicious-document00039- character-substitution source-document00010.xml suspicious-document00078- character-substitution source-document00020.xml suspicious-document00099- character-substitution source-document00025.xml Quality of Plagiarism Good Good Good Good Well paraphrased Well paraphrased Well paraphrased
We are thankful to all volunteers for their valuable contribution in construction of this corpus. 18. Stein, B., zu Eissen, S.M., Potthast, M.: Strategies for retrieving plagiarized documents. In: Proceedings of the 30th annual international ACM SIGIR conference on Research and development in information retrieval. pp. 825–826. ACM (2007) 19. Stein, B., Rosso, P., Stamatatos, E., Koppel, M., Agirre, E.: 3rd pan workshop on uncovering plagiarism, authorship and social software misuse. In: 25th Annual Conference of the Spanish Society for Natural Language Processing (SEPLN). pp. 1–77 (2009)