AraPlagDet is the first shared task that addresses the evaluation of plagiarism detection methods for Arabic texts. It has two subtasks, namely external plagiarism detection and intrinsic plagiarism detection. A total of 8 runs have been submitted and tested on the standardized corpora developed for the track. This overview paper describes these evaluation corpora, discusses the participants' methods, and highlights their building blocks that could be language dependent.
Despite the lack of large-scale studies on the prevalence of
plagiarism in the Arab world, the large number of news on this
phenomenon in media1 attests its pervasiveness. There are also
some studies that show the lack of awareness on the definition and
seriousness of plagiarism among Arab students [
AraPlagDet shared task involves two sub-tasks, namely: External plagiarism detection and Intrinsic plagiarism detection. Each participant was allowed to submit up to three runs in one or both sub-tasks. From 2009 to 2011, PAN2 plagiarism detection competitions have been organized with these two sub-tasks3.The evaluation corpora in these competitions were mostly English. Thus, AraPlagDet is the first plagiarism detection competition on Arabic documents.
External and intrinsic plagiarism detection tasks are significantly different approaches for plagiarism detection. In the external plagiarism detection sub-task, participants were provided with two collections of documents, namely suspicious and source, and the task is to identify the overlaps (exact or not) between them. In the intrinsic plagiarism detection sub-task, participants were provided with suspicious documents and the task is to identify in each document the inconsistencies with respect writing style. This approach is useful when the potential sources of plagiarism are unknown, and this is still a less explored area in comparison with the external approach.
A total of 18 teams and individuals from different countries (six of them are not Arab) registered in the shared task, which shows the 2 http://pan.webis.de 3 Since 2012 PAN plagiarism detection competition focuses on the external approach. interest of practitioners and researchers in this topic. However, only three participants submitted their runs.
Given a document d and a potential source of plagiarism D’,
detecting plagiarism by the external approach consists in
identifying pairs (s , s’) from d and d’ (d’ ∊ D’) respectively, such
that s and s’ are highly similar. This similarity could has many
levels: s is an exact copy of s’, s was obtained by obfuscating s’
(e.g. paraphrasing, summarizing, restructuring ...etc) or s is
semantically similar to s’ but uses different words or even
different language. This problem has been tackled by many
researchers in the last decade using a plethora of techniques
related to information retrieval and near-duplicate detection.
Techniques are used on the one hand, to retrieve the source d’
from D’, and on the other hand, to make an extensive comparison
between d and d’. Examples of techniques used to compare
passages include character n-grams and kernels [16] and
skip-ngrams and exact matching [30]. The last trend is to adapt methods
to detect a kind of plagiarism obfuscation. For instance,
SanchezPerez et al.’s method [39] is oriented to detect plagiarism cases
that summarize the source passage. See Section 4 for more details
on the building blocks of external plagiarism detection methods.
Given a document d, detecting plagiarism by the intrinsic
approach consists in identifying in d the set of passages S, such
that each s ∈ S is different from the rest of the document with
respect to writing style. Then, techniques used in this approach
consist in finding the best textual features that are able to
distinguish the writing style of different authors in one document.
It is obvious that intrinsic plagiarism detection is strongly related
to authorship attribution [42], paragraph authorship clustering [12]
and detection of inconsistencies in multi-author documents [
All the aforementioned methods were tested on English corpora,
namely PAN plagiarism detection corpora. Methods developed
and tested on Arabic documents are very few [
We describe in this section the evaluation corpus and the submitted methods in the external plagiarism detection sub-task.
The collection of a large number of documents incorporating real plagiarism may be difficult and hence not very practical. Therefore, plagiarism detection corpora are usually built automatically or semi-automatically by creating artificial plagiarism cases and inserting them in host documents4. To this end, it is essential to compile two sets of documents: i) the source documents, from which passages of text are extracted; and ii) the suspicious documents, in which the aforementioned passages are inserted after undergoing (optionally) obfuscation processing.
To build our corpus for external plagiarism detection sub-task
(ExAra-2015 corpus), we used documents from the Corpus of
Contemporary Arabic (CCA)5 [
Such a variety is desirable, because it makes the plagiarism detection corpus more realistic. ─ Each document is tagged with its topic, which is a favorable feature in the process of creating artificial suspicious documents. In this process, which attempts to imitate real plagiarism, the inserted plagiarism cases should topically match the topic of the suspicious (host) document. ─ The corpus is freely available and their developers were keen to have copyright permissions from the owners of the collected texts to use them for research purposes7.
Besides CCA, we included in our corpus –specifically in the source documents set– hundreds of documents from Arabic Wikipedia. We collected them manually by selecting documents that match the topics of the suspicious documents. These documents have been incorporated in the corpus to baffle the detection, and only few cases have been created from them. Surprisingly, we realized8 that many of the collected Wikipedia articles (notably biographies) contain exact or near exact copies of large passages from the CCA documents. This fact resulted in plagiarism cases that are not annotated in the corpus. To address this issue, we applied a simple 5-grams method to identify these cases of ‘real’ plagiarism between the suspicious documents and the collected Wikipedia documents, and we discarded from the corpus the Wikipedia documents involving the detected passages9.
We created two kinds of plagiarism cases: artificial (created automatically) and simulated (created manually). For the automatically created cases, we used the strategy of phrase shuffling and word shuffling. To avoid producing cases that have the same pattern of shuffling, we applied to the cases of the test 4 There is also the manual approach, which consists in asking a group of people to write essays and plagiarize. This method produces realistic plagiarism, however it is costly in terms of material and human resources and time [36]. 5 http://www.comp.leeds.ac.uk/eric/latifa/research.htm 6 http://ar.wikipedia.org 7 We contacted Eric Atwell (the co-developer of CCA) who gives us the permission to use CCA documents in our corpus. 8 We started to be aware of this issue thanks to AraPlagDet participants who pointed out the existence of some plagiarism cases that have not been annotated in ExAra sample corpus which has been released before the official training corpus. 9 Annotating the plagiarism in these documents would be a better solution but we chose to discard them because of time limitation. corpus a different algorithm than the one used for the training corpus.
Regarding manually created plagiarism, we employed two obfuscation strategies: synonym substitution and paraphrasing. Both of them are described below.
To create plagiarism cases with this obfuscation, we did the following: ─ Manually replaced some words with their synonyms. We used as source of synonyms Almaany dictionary10, the Microsoft Word synonym checker, Arabic WordNet Browser11, and the synonyms provided by Google translate12. It should be noted that an Arabic singular noun may have multiple plural forms that are synonymous. For example, the word 'ةريزج' (jazira– island) has the plurals 'رئازج' (jazair) and 'رزج' (juzur). 10 http://www.almaany.com/ 11 http://globalwordnet.org/arabic-wordnet/awn-browser/ 12 https://translate.google.com ─ Added diacritics (short vowels) to some words, where diacritics in Arabic are optional and their inclusion or exclusion are orthographically acceptable. Consequently, we can have for a word w whose length is n letters, at least 2n different representations. For example, the different representations of the word 'قح ' (haq– truth) with and without diacritics are depicted in Fig.1. wٌitقhoحutletters’ diacrٌitics.
Fig. 1. Different representations of the same word with and
We decided to substitute words with their synonyms manually (no
matter it is time-consuming) after many attempts to perform this
task automatically. Despite our efforts to obtain exact synonyms
by using part of speech tagging and lemmatization, our attempts
produced either passages with totally different meanings from the
original ones (poor precision) or very few passages with
substituted words (poor recall). These unsuccessful attempts could
be respectively attributed to:
(i) The high ambiguity of Arabic language: researchers estimated
the average number of ambiguities for a token in Arabic
language is 8 times higher than in most other languages [15].
Therefore, it is not surprising to find it difficult to select
automatically the appropriate synonym in a given context.
(ii) The limited coverage of lexical resources: in our experiments
we used Arabic WordNet as a source of synonyms.
Unfortunately, this resource, which is one of the most
important and freely available linguistic resources for Arabic,
contains only 9.7 % of the estimated Arabic vocabulary [
Cases produced with this obfuscation strategy are the most realistic ones in our corpus. This is because the passages to be obfuscated have been selected manually from the source and then paraphrased manually. The results are plagiarism cases that are very close in terms of topic to the suspicious documents that host them. In this type of obfuscation, all kinds of modifications were applied (restructuring, synonym substitution, removing repetitions …etc.), provided that the meaning of the original passage is maintained.
Due to the dullness and slowness of the manual process13, we produced 338 cases with synonym substitution obfuscation and only 44 cases with paraphrasing obfuscation. See Table 1 for more detailed statistics.
Three participants submitted their runs. Since multiple submissions were allowed, two participants submitted three runs. Therefore, we collected a total of seven runs. Two participants 13 We are aware about the possibility to use the crowdsourcing to allow the creation of a large number of plagiarism cases manually [37]. However, apart from some few volunteers we crafted the cases (of the synonym substitution and paraphrasing) ourselves because of the lack of financial resources. among the three submitted working notes describing their methods. Following, we summarize the work of these two participants.
External plagiarism detection methods involve mainly two phases: the source retrieval and the text alignment [35]. For a given suspicious document d, the source retrieval phase consists of selecting from the available set of source documents D, a subset D' of documents that are the most likely source of plagiarism. Text alignment is the process of extensively comparing d with each document in D' in order to determine the similar passages. Fig.2 depicts the building blocks of these two phases. PAN competition series on plagiarism detection has contributed significantly to defining these phases and setting their terminology14. Therefore, the detailed explanation of these phases with their building blocks could be found in PAN overview papers [31–35, 38]. In this paper, we are just adopting this terminology to describe the methods of participants.
Keyphrase extraction: extracting keyphrease form each chunk.
Queries formulation: combining keyphrases and creating one (or more) query for each chunk.
Search Control : scheduling and submitting the queries to a search engin that indexes the source documents.
Candidate Filtering: selecting from the search results the (source) documents that are worthy of the text alignment phase.
Seeding: extracting units (relativelly short) of text from the suspcious and the source documents and detecting matchs between them.
Passage Filtering: discarding passages judged irrelivant.
We describe in this subsection the methods of Magooda et al.[23]
and Alzahrani[
First, it should be noted that our evaluation corpus could be considered medium-sized especially in comparison with the PAN competition corpora [31–35, 38]. Furthermore, we did not determine in the competition the retrieval techniques to use. Nonetheless, to avoid being merely a lab method, it is important for any plagiarism detection approach to deal with large sets of documents by using appropriate retrieval techniques. Magooda et al. in their three methods used the Lucene search engine and two indexing approaches as shown in Table 2. Therefore, their methods could be used with a large collection of source documents, and could be adapted to be deployed online with a commercial search engine, which is an obvious solution to adopt if the source of plagiarism is the web as pointed out by Potthast et al.[33].
As for Alzahrani’s method, it is clear that it is not ready to be employed if the web is the source of plagiarism for two reasons: i) its retrieval model is not structured to be used with search engines. (for example, there is no query formulation, see Table 2); and ii) it is based on fingerprinting all the source documents, and entails an exhaustive comparison between the n-grams of the suspicious document and each source document, which is not workable if the source of plagiarism is extremely large, as the web. Nonetheless, her method could be feasible when the source of plagiarism is local and not too large, as in the case of detecting plagiarism between students’ assignments. Still, even with the intension to be used offline, this method could possibly use retrieval techniques based, for example, on inverted indexes instead of fingerprints similarity to allow for the processing of a large number of documents in reasonable time. Malcolm and Lane [25] discuss the importance of scalability even for offline plagiarism detectors.
Regarding this aspect, Magooda et al. reported the use of twolanguage dependent processing in the source retrieval phase: stemming queries before submitting them to the search engine and extracting named entities. In the text alignment phase, words are stemmed in the skip-gram approach. Moreover, their methods preprocess the text by removing diacritics and normalizing letters15. Alzahrani method is nearly language independent. The only reported language-specific process was stop words removal. It was applied as a pre-processing step on suspicious and source documents.
ٌةايحلاٌدنهللٌءارزوٌةسيئرٌلوأ ٌ"يدناغٌاريدنإ "ٌتشاع اهتابلقتٌلكبٌةيسايسلا ٌهايحلاٌدنٌهللٌءا ٌرَزٌوٌة ٌسَيئٌرٌلو اٌ"ىدناغٌاريدن ا"ٌت ٌشَا ٌعَ اٌَهتاَبٌلقتٌلٌ كبٌٌةيٌسايٌسلا Fig. 3. Two passages with the same words but the 2nd passage contains some letters with diacritics (highlighted in green) and a substitution of some interchangeable letters (highlighted in yellow). A simple plagiarism detector may fail to match them. Since the external plagiarism detection is a retrieval task, we think that challenges of Arabic IR hold for Arabic plagiarism detection. Arabic IR is challenging because the high inflection of Arabic and the complexity of its morphology. Arabic stems are derived from a set of a few thousand roots by fitting the roots into stem templates. Stems can accept attached prefixes and suffixes that include prepositions, determiners, and pronouns. Those are sometimes obstacles to match similar texts [22]. Moreover, unlike many other languages, Arabic writing includes diacritics that are pronounced, but often not written. As opposed to the Latin languages, the use of diacritics in Arabic is not restricted to some letters, they could be rather placed on every letter. Indeed, in Arabic IR, diacritics are typically removed [13, 17]. Another issue that affects Arabic IR and consequently Arabic plagiarism detection is the fact that Arabic has some letters that are frequently used interchangeably such as: (ي , ى), (ا,ٌأ , إٌ,ٌآ) and (ه,ٌةٌ ) hence the need of a letter normalization pre-processing. If the orthographic normalization (diacritics removal and letter normalization) is not employed, a plagiarism detection system may fail to match similar passages even if they have exactly the same words. See Fig. 3 for an illustration. 4.3
We employed a simple baseline, which entails detecting common chunks of word 5-grams between the suspicious documents and the source documents and then merging the adjacent detected chunks if the distance between them is smaller than 800 characters. Short passages (< 100 characters) are then filtered out. Since it is primarily based on matching n-grams, it should detect mainly plagiarism cases that are not obfuscated.
The methods were evaluated using the character-based macro precision and recall in addition to the granularity, and ranked using the plagdet that combines these measures in one measure. All these measures are computed using the set of the plagiarism cases annotated in the corpus (the actual cases) and the set of the plagiarism cases detected by the method (the detected cases). The precision and recall count the proportion of the true positive part in each detected and actual case respectively. An average of these proportions is then computed. Their formulas are presented in the equations 1 and 2 where S is the set of the actual plagiarism cases and R is the set of the detected plagiarism cases. A plagiarism detection method may generate overlapping or multiple detections for a single plagiarism case. Thus, granularity is used to average the number of the detected cases for each actual case as depicted in the formula 3. is the set of the actual cases that have been detected, and are the detected cases that intersect with a given actual case s. It is clear that the optimal value of the granularity is 1, and it means that for each actual case, at most only one case has been detected (i.e. not many overlapping or adjacent cases).
To rank methods a combination of the three measures is applied in the plagdet as expressed in the formula 4 where F1 is the harmonic mean of precision and recall. – – – –
Keep the pair where passages are equivalent, else discard it if: - passages length < threshold - the number of the words matches < threshold
∈ ∈ ∈ ∈ ∈ See [37] for more information on plagiarism detection evaluation measures. Table 4 provides the performance results of the participants’ methods as well as the baseline on the test corpus.
As shown in Table 4, four methods outperform the baseline in terms of the plagdet. In terms of precision, of the majority of methods are good, but none of them performed better than the baseline. Regarding the recall, the best three methods have acceptable scores, but the rest of methods’ scores are more or less close to the baseline. All the methods have a granularity of more than 1.05, which is not a very good score in comparison with what has been achieved by the state-of-the-art methods (see for example PAN2014 competition results [34]).
The goal of this section is to provide an in-depth look at the behavior of methods. Table 5 presents the performance of participants’ methods on the test corpus according to some parameters namely cases length, type of plagiarism and obfuscation. (1) (2) (3) (4)
Interestingly, Table 5 reveals that the three methods of Magooda et al. are the only ones that detect cases with word shuffling obfuscation. This explains the low overall recall of Palkovskii [29] and Alzahrani methods. It seems that the algorithm employed to shuffle words generates cases that are difficult to detect by the fingerprinting approach used in Alzahrani source retrieval phase. Magooda_1 and Magooda_2 methods perform better than Magooda_3 with respect to word shuffling cases. This is thanks to the common words approach which is able to match similar passages no matter the order of words. Regarding the impact of the case length, all the methods perform better with medium cases.
All the methods achieved a very high recall in detecting cases without obfuscation. Whereas the manual paraphrasing cases are the most challenging to detect after the word shuffling cases.
Typically, it is easy to obtain a reasonable precision. This could be observed in the majority of the results in Table 4. This behavior was observed also in PAN shared task on plagiarism detection [34]. Since Palkovskii_2 method is the least precise among all the submitted methods, we have been keen to understand the underlying reason behind its poor precision score. An examination of its outputs revealed that around 60% of the utterly false positive cases (cases whose precision is 0) stem from documents with religious content. We went one step further and looked into the text of these cases. It turned out that the phrase "ملسوٌهيلعٌالهٌىلص " was the underlying seed of many false positive cases. This phrase, which translates as "may Allah honor him and grant him peace", is a commonly used expression in Arabic (written and even spoken) after each mention of the prophet Muhammad. Another kind of false positive cases that stem from religion-related texts, are quotations from Quran and Hadith (sayings of the prophet Muhammad). Some false positive cases in the Palkovskii_2 run and even in the other methods’ runs belong to that kind. For instance, Quranic verses represent 6% of the utterly false positive cases in Magooda_2 run.
The detected plagiarism case in the suspicious document ٌةيفوصلاٌ تاعامجلاٌ كلذكوٌ ،ةنسلاٌ راصنأو ٌ ملسوٌ هيلعٌ ٌاله ىلص ٌ دمحم ٌريثأتبٌةينيدٌةعزنٌهيدلٌتناكٌيذلاٌظفاحٌنكل ٌ...ةددعتملاٌاهفئاوطوٌاهقرطب ٌيهو ٌ ملسوٌ هيلعٌٌالهىلص ٌدمحمٌانديسٌ بابشٌ ةعامجٌيفٌهتيغبٌدجوٌهتأشن ٌ.8391ٌ ماعٌ ةاتفلاٌ رصموٌ نيملسملاٌ ناوخلإاٌ نعٌ نوقشنمٌ اهسسأٌ ةعامج ٌظفاحٌ مضناٌ ناوخلإاٌ ةلظمٌ تحتٌ اوناكٌ كاذنآٌ ن ينيدتملاٌ ةيبلاغٌ نأٌ مغرو ٌنورهجيٌ مهنأٌ اهئانبأٌ يفٌ ىريٌ ناكٌ هنلأٌ ؛ 8391ٌ ماعٌ دمحمٌ بابشٌ ةعامجل ٌ،ركنملاٌنعٌنوهنيوٌفورعملابٌنورمأيٌمئلاٌةمولٌٌالهيفٌنوشخيٌلاوٌقحلاب
ٌملسو هيلعٌٌالهىلص ٌي بنلاٌل وقبًٌلا مع ٌريمأوٌك لمٌن يبٌمهدنع ٌق رفٌلا The detected plagiarism case in the source document ٌنوكيٌنأٌلمتحي ٌ"ةعامجللٌقرافملاٌنيدلاٌنمٌقراملا "ٌ:ملسوٌهيلعٌٌالهىلص ٌنأٌ ىلإٌ اذهٌ هيأرٌ يفٌ ةيميتٌ نباٌ دنتسيو ٌ .دترملاٌ لا ٌ ق يرطلاٌ عطاقٌ ب راحملا ٌي ضرٌةشئاع ٌن ع ٌوحنلاٌاذهٌىلع ٌةرسَّفمٌت ءاجٌدقٌ،روكذملاٌث يدحللٌةياور ٌنأ ٌ-اهنعٌٌالهيضر -ٌةشئاعٌنعٌهدنسبٌدوادٌوبأٌهاورٌامٌوهٌكلذوٌ،اهنعٌاله ملسوٌهيلعٌٌالهىلص ٌٌالهلوس رFig. 4. A detected plagiarism case by Palkovskii_2 method.
It is obvious that this case has been detected because the common phase "ملسو هيلع اله ىلص " ("may Allah honor him and grant him peace") has been used as a seed. The extension step produces a pair of passages that are not similar.
It is an important feature for any plagiarism detection system to not consider common phrases and quotations as plagiarism cases unless they appear as a part of a larger plagiarism case. In Arabic texts and notably in texts about religious topics, quotations from Quran and Hadith are very common. Moreover, there are some religious phrases that could be repeated many times in documents. The expression "ملسوٌ هيلعٌ الهٌ ىلص " ("may Allah honor him and grant him peace") is an example of such common phrases. In the ExAra test corpus, it appears 185 times in the suspicious documents and 171 times in the source documents. This increases the risk of obtaining many short false positive cases. Still, this issue could be addressed simply by filtering out the very short detected cases. In the baseline method for example, we apply such a filter and we obtain very high precision. The problem is that the common religious phrase may appear many times even in the same document. For example the expression "ملسوٌ هيلعٌ الهٌ ىلص " ("may Allah honor him and grant him peace") occurs 29 times in the ‘suspicious-document0014’ and 52 times in ‘sourcedocument00223’. This increases not only the risk of obtaining short false positive cases (of some few words) but also longer cases when the adjacent seeds are merged in the extension step (see Section 4.2.1). We observed many cases of this kind in Palkivskii_2 method output. See Fig. 4 for an illustration. Citing religious texts is common in Arabic writing. Moreover, many of the Arab countries are incorporating religion in their public schools curricula [14]. Therefore, we believe in the need to have plagiarism detectors that are able to cope with the characteristics of this kind of Arabic texts.
Only one participant submitted a run to this sub-task. Following, we describe the corpus, the method and its evaluation.
Sources of plagiarism are omitted in the intrinsic plagiarism detection evaluation corpus. Thus, a plagiarism case in this corpus is defined by its position and its length in the suspicious document only. For AraPlagDet intrinsic plagiarism detection sub-task, we used the InAra corpus [8] for the training phase. For the test phase, we built another corpus which had similar characteristics to the training one. Table 6 provides statistics on both training and test corpora. As shown in this table, all the cases are without obfuscation. This is because the goal is to evaluate the ability of methods to detect the style shift, and obfuscating the plagiarism cases may bear more difficulties to the task. Further information on the creation of these corpora could be found in [9] and [8].
Most of intrinsic plagiarism detection methods in the literature entail five main building blocks which are depicted in Fig. 5.
These are inspired from the authorship verification approach [44] and have not been changed in the past decade. Case length very short
short .753 .763 .693 .935 .978 .616 .493 .404 .747 .600 .679 .483 .431 .470 .548 .548 1.017 1.000 1.019 1.000 1.011 1.005 1.000 1.000 .741 .672 .677 .637 .594 .531 .519 .465 .862 .853 .807 .997 .998 .925 .647 .551 .850 .783 .818 .513 .505 .494 .554 .554 1.011 1.003 1.009 1.008 1.083 1.020 1.002 1.002 .850 .814 .807 .674 .634 .635 .596 .551 word shuffling .890 .871 .890 .000 .000 .000 .000 .000 .657 .492 .657 .000 .000 .000 .000 .000 1.081 1.044 1.081
document to uniform units such us paragraphs, sentences, or sliding window of N words or characters.
Style features extraction: representing each chunk (in some methods the whole document as well) as a vector of features.
Plagiarized fragments identification: using heuristics to decide whether the chunk is plagiarized or not based on its features.
Post-processing: merging ajacent chunks or/and filtering out some detected passages.
In this section, we describe the method of Mahgoub et al.[24], which is the only participant in the intrinsic plagiarism detection sub-task. Mahgoub et al. reported in their working notes that their method is similar to the one proposed by Zechner et al.[46]. It is based on computing the cosine distance between the Vector Space Model (VSM) of the suspicious document and the VSM of each chunk. Table 7 describes the method according to the generic framework depicted in Fig. 5.
It seems that features extraction is the most affected part by the language of the processed document. Three features extracted in Mahgoub et al. method are dependent to the language: it is obvious that any language has its own approaches for POS tagging and its own list of stop words. Moreover, Arabic, being a right-to-left language, has some punctuation marks adapted to that, such as the comma (،) and the question mark (؟).
We used a method based on character n-gram classes as features and naïve Bayes as a classification model. It is almost the same method described in [11] but with some modifications in the length of the sliding window in the segmentation strategy. This method is language-independent, and it allows for obtaining performance values comparable to the ones of the best intrinsic plagiarism detection methods namely Oberreuter and Velásquez [27] and Stamatatos [43] methods. The evaluation measures are the same used for the external plagiarism detection (see section 4.3.2)
As shown in Table 8, Mahgoub et al.’s method performance is lower than the baseline. This is in line with the performance of the original method [46] that obtained a plagdet score of 0.177 on the PAN09 corpus [38].
Unlike the external approach, we think that the performance of the intrinsic approach could be influenced by the document length and the percentage of plagiarism it incorporates. Table 9 presents the performance of Mahgoub et al. and the baseline methods on the test corpus according to the aforementioned parameters in addition to the case length. The segmentation strategy of the baseline does not produce short chunks, and that is why the precision is not computed in detected short cases. However, the actual short cases are detected with high recall. For both methods, the best performance is obtained in the medium cases, the short documents and the documents with much plagiarism. Nonetheless, since we have only two methods, we cannot generalize any observed pattern.
AraPlagDet is the first shared task on plagiarism detection on Arabic texts. Participants were allowed to submit up to three runs in both the external and intrinsic plagiarism detection sub-tasks and a total of eight systems were finally submitted. In the external plagiarism detection sub-task most of the submitted methods were able to detect cases without obfuscation with a high performance. The obfuscated cases were more or less challenging. This is consistent with methods tested on PAN corpora [7]. As for the intrinsic plagiarism detection, it is still a very challenging task. We hope that the evaluation corpora we developed will help to foster research on Arabic plagiarism detection from both perspectives. precision granularity
Many thanks to AraPlagDet participants for their devoted work on developing and testing their methods on AraPlagDet corpora. The work of the 3rd author was in the framework the DIANAAPPLICATIONS-Finding Hidden Knowledge in Texts: Applications (TIN2012-38603-C02-01) research project. The research of the 4th author was carried out in the framework of the grant provided by the Council for the Development of Social Science Research in Africa (CODESRIA) Ref. SGRT. 38/T13. recall b u o g h a M e n i l e s a
B 1.000 1.000 [29] [30] [31] [32] [33] [34] [35] [36] [37] [38] [39] [40]