Many texts with unknown sources may contain so-called anomalous parts, which may have been artificially inserted from another source or author, either by mistake or on purpose. The detection of these anomalies is a vital task for detecting plagiarism, verifying authenticity, as well as cleaning texts from parts that would cause their inconsistency. In this paper we propose a novel approach to anomaly detection in texts. The approach consists of two steps, the first step lies in detecting the potential sentences that are considered to be the borders between the anomalous and non-anomalous texts. In the second step, we pair up those sentences to find the anomalous parts. To obtain the semantic encoding of the sentences, we use a pre-built 2 model to individually encode the words that make up the sentence. We propose four methods for aggregating words in sentence embedding. One of them is based on multiplying the embedded elements with its Inverse Document Frequency (IDF) score, another uses a special TF-IDF metric calculated using a Term Frequency (TF) and . Once the sentence embeddings are obtained, a BIdirectional Long-Short Term Memory BI-LSTM network is used to detect sentence shifts that are on the boundaries of anomalous texts. We then use streaming semantic comparison to filter out false positives and false negatives from the boundary sentences and match them. Our approach was tested on a corpus originating from the competition External Plagiarism Detection in Arabic Text PAN 2015 (ExAraCorpusPAN2015), where we artificially replaced parts of the source text with semantically similar parts coming from another text. Our methods, based on embedding sentences obtained using Part-Of-Speech tag (POS-Tag) weights, achieved the highest F1-score value 0.95.
data, including: anomaly detection using deviations in high dimensional data [1], unsupervised clustering methAnomaly detection in texts is the task of identifying parts ods [2, 3], rule-based systems [4], and deep learning of texts which difer from other parts of the text from the [5, 6, 7]. Generative Adversarial Networks (GANs) and point of syntax or from the point of view of semantics. the adversarial training process have been recently emThe dificulty of the problem is in the fact that it is not ployed to solve this problem and they yield remarkable known in advance where to look for anomalies in the results [8, 9]. Although many anomaly detection methtext, how extensive the anomalous parts are and to what ods are applicable to text documents, special approaches extent diferent parts of the text should be considered can be applied to the information contained in the texts to anomalies. This problem has applications in several areas, provide better results. Our goal was to create a sentence for example in detecting plagiarism, in attempts to falsify embedding based on word embeddings and determine texts or in attempts to insert misleading information into the boundaries of anomalous parts of the text. Verify the texts. When solving this problem, it is necessary to follow implemented algorithm on Arabic texts. the formal (syntax) and content (semantics) essence of In the paper in Section 2, we present some of the meththe text. It is advisable to monitor both essences at the ods that influenced our approach to the solutions In Secsame time and not divide the problem into two tasks. tion 3, the proposal of the solution is elaborated, empha
Anomaly detection is also known as outlier detection sizing the new elements. Section 4 is devoted to simulated or novelty detection. Various approaches are used to datasets preparation, Section 5 contains the experimental solve the problem of finding anomalies in non-textual setup, results and their evaluation. In the final Section 6, an overall evaluation of the method and a proposal for possible improvements of the presented solution are given.
ITAT’23: Conference on Information technologies – Applications and Theory, September 22–26, 2023, Tatranské Matliare, Slovakia * Corresponding author. † These authors contributed equally. a$bdzuollwtaanh.sezdoaplmlaakr@imsti@udbewntu.u.epdjus..slyk ((AZ.. SAzlompalárikm);i); 2. Anomaly Detection in Texts asmaa.salem@bwu.edu.ly (A. Salem); gabriela.andrejkova@upjs.sk (G. Andrejková) A new GAN-based text anomaly detection method is pre0000-0002-2956-0496 (G. Andrejková) sented in [10], In the method, an adversarial regularized © 2023 Copyright for this paper by its authors. Use permitted under Creative Commons License Attribution 4.0 autoencoder (ARAE) is used to reconstruct normal senInternational (CC BY 4.0).
CEUR Workshop Proceedings (CEUR-WS.org) tences and detects anomalies via a combined anomaly multitask classification in [ 21] but it doesn’t solve anomascore based on the building blocks of ARAE. An overview lies. Our attention is focused on solving the problem of of methods for Group Anomaly Detection can be found in ifnding anomalies in Arabic texts using semantics and [11], but to contextual anomalies are given only general some stylistic features of the text. attention here.
Identifying the style of the author of the given document using stylometric functions can contribute to the 3. Proposed Solution detection of anomalies in the texts. The described method in [12] contributes to the detection of plagiarism in texts In order to detect semantic anomalies from text we need based on the formal aspect of the text. Plagiarism in Ara- (1) to find a way to encode of semantic contents of senbic texts was solved in [13, 14]. In [15], it is presented tences, (2) to determine the position of the anomalous an anomaly detection method – Context Vector Data De- part of the text and (3) to evaluate the quality of the given scription (CVDD), which builds upon word embedding solution. models to learn multiple sentence representations that capture multiple semantic contexts via the self-attention 3.1. Semantic content encoding mechanism. Since natural language text is dificult to interpret by
The introduction of a new language representation machines, our first step is to create vector embeddings model called BERT, which stands for Bidirectional En- of the words and sentences that make up a given text. coder Representations from Transformers [16] was a sig- To obtain the semantic content of words that make up nificant step in language representations. BERT inspired a sentence, we can simply make use of Aravec Word2Vec us to look for improved solutions, however, in the article [22]. Aravec is only capable of encoding singular words, we present a solution that provides quite good results. In therefore we need a method to create an embedding [17], there were used multiple machine learning methods that describes the semantic content of an entire sentence. such as sentence transformers, auto encoders, logistic There are multiple ways to obtain sentence embeddings. regression and distance calculation methods to predict Let = 1, ..., be a sentence consisting of anomalies in English texts and his system gave F1-score words and = 1, ..., be their embeddings, where value 0.86 for used Cross-lingual Natural Language Infer- represents the − ℎ word and its corresponding ence dataset plus injected Stanford Sentiment Treebank. embedding. Suggested methods for sentence embedding
The paper [18], 2008, introduces the main challenges in are: (1) averaging of words embeddings, (2) inverse docArabic text processing and describes the proposed unsu- ument frequency embeddings, (3) threshold of similarity pervised learning model for detecting anomalous Arabic embeddings, and (4) extraction of top keyword. textual information, but the problem is on classification if text contains some anomaly. Authors used various Arabic Web sites for text collections. Three main types 3.1.1. Averaging of words embeddings of text were used to represent anomalous segments (200 The simplest way to create a sentence embedding is to - 500 words), religious description text, social text and average all of its word embeddings. novels. All 3 evaluation criteria (cosine similarity, city block distance and Chebychev distance measure) have () = ∑︀=1 (1) achieved a significant increase in the detection rates (> 75 %) in terms of the detected anomalous segments in the Top 10 (indicates how many times each anomalous While such a method may be efective, not every word segment appeared in the top 10 of the ranked list). contributes to the overall meaning of the sentence to the
In the paper [19], we solved problem: "To cover stylis- same degree. tic dissimilarities among text segments of the same long English or Arabic text." Using clustering and Convolu- 3.1.2. Inverse document frequency embeddings tional Neural Networks (CNN) we evaluated 40 English Another method lies in calculating the Inverse Document and 40 Arabic text (using texts from pan-plagiarism- Frequency (IDF) value of each word from the corpus and corpus-2011.part1.rar, http://ksucorpus.ksu.edu.sa) and using it to weight each word of the embedding. IDF was we have got accuracy values in interval ⟨60; 74⟩% for proposed by Jones [23] in 1972, and has since been exEnglish texts and ⟨53; 96⟩% for Arabic text. tremely widely used. The IDF value penalizes words that
Our first approach to finding positions of anomalies are common in the entire corpus by calculating in how was using HTM networks for English and Arabic texts many documents the word is present in. It is calculated [20]. We achieved better results for English texts, so we as now turned our attention to Arabic texts. Contextual se- (, ) = (2) mantic embeddings were analyzed from point of Arabic | ∈ : ∈ | where is the collection of all documents from the cor- word its predicted part-of-speech tag (POS-Tag). Some pus, || = , is document, and | ∈ : ∈ | parts of speech contribute more to the meanings of a is the number of documents that appears in. In our sentence than others. Nouns, adjectives and verbs concase we consider every sentence a diferent document tribute more to the overall meaning than pronouns and and calculate the IDF values across all sentences of all prepositions. documents from our text corpus. With our IDF values, we We have calculated the relative weight of POS-Tags by can calculate IDF weighting based sentence embeddings first calculating the cosine distance between the embedding of the word with a given POS-Tag and the overall () = ∑︀=1 (, ) * (3) sentence embedding gained by calculated by a simple pooling. We have averaged and normalized these values.
We can then calculate sentence embeddings by multiply3.1.3. Semantic similarity TF-IDF embeddings ing the word embedding with its POS-Tag weight, as We can also use a modified TF-IDF metrics (the TF-IDF follows two statistics, term frequency ( ) and inverse document frequency. TF-IDF is frequently used in a language processing [24]. The is relative frequency of word within document () = ∑︀ =1 POS-weight(POS-Tag()) *
(7) 3.2. Positions of anomalous parts , After calculating the sentence embeddings, we will use (, ) = ∑︀ (4) some methods to detect anomalous positions. Since ′∈ ′, anomalies are generated from the same semantic cluster, where , is the number of the word in the document simply using one-class clustering methods is inadvisable. . During the search for anomalous sentences, it is possi
TF-IDF multiplies the values with the term fre- ble to use only relative relationships between sentences, quency metric, TF-metric or their embeddings. Let’s assume that the neural network trained on the initial sentences will predict the TF-IDF(, ) = (, ) * (, ) (5) following sentences and gradually the given text will be learned. If the predicted sentence is very diferent from -metrics calculates the number of times a given the original sentence (we will use some metrics for comterm appears in the given text (not necessary full docu- parison), then the original sentence does not fit into the ment). The -metric will be sampled over a window text and can be considered anomalous. The problem here of sentences, however it won´t be calculated over the is in the setting of the similarity/dissimilarity threshold, exact matches, rather, it will be calculated over words which we found experimentally. Sentences that are not that belong to the same context via embedding similarity. similar are marked as anomalous. In further processing,
To find the threshold of embedding similarity we can they are used to determine anomalous sections. Howuse a dictionary of synonyms as well as an thesaurus. We ever, it is possible that the anomalous section will consist calculate the average cosine distance between the embed- of only one sentence. After that, it is still necessary to dings of every word and their corresponding synonyms create sections using the marked anomalous sentences. from the dictionary of synonyms. Then, we calculate Our proposed methods: the average cosine distance between the embeddings of every word and their corresponding definitions from the 3.2.1. Anomaly thresholds in document thesaurus. Averaging these two values, we obtain the threshold embedding distance. Using this distance, we We can make use of a recurrent neural network, such as can calculate the values. This will ensure that the a Bidirectional − network [25]. The network TF-IDF will be the words that are most relevant to dis- uses a context window of previous sentences in order tinguishing it from its neighbors. The final embedding to predict the embedding of the next sentence. This premetric is calculated as diction can then be compared to the original embedding using the cosine distance metric between its embeddings, () = ∑︀=1 TF-IDF(, ) * (6) which will give us a measurement of how well the net work is able to predict the given sentence. Alternatively, where is word in sentence . the prediction success can be calculated as the prediction error of this sentence compared to the mean prediction error of the previous sentences. Such a metric takes the 3.1.4. POS-Tag relevance embeddings predictive strength of the network into account too, as when the data is noisy a low prediction score might be We have also developed key phrase extraction method that lie in part-of-speech tagging, which assigns to each found even in a non-anomalous part of the text, however certain sentence embeddings are. The idea behind this the prediction score is not that out of the ordinary. method is the following: the non-anomalous parts that
How well the network is able to predict a given sen- make up the majority of the text come from the same tence is a good indicator if an anomaly threshold is found. source and have the same semantic makeup. Therefore An anomaly threshold can either be a non-anomalous an autoencoder trained on such data will be able to reconsentence followed by an anomalous sentence, or in a struct them with less error than the anomalous parts that reverse situation an anomalous sentence followed by a come from diferent sources and possess a diferent senon-anomalous sentence. The outputs of the BI-LSTM mantic makeup. By calculating the cosine distance of the network will serve as our potential anomaly thresholds. reconstruction, we then calculate the average distance between two potential anomalous sections. 3.2.2. Anomaly sections After we set the threshold for anomaly detection, we can label some sentences as anomalous and others as Now that we have potential anomaly threshold sentences, non-anomalous. If we find a non-anomalous sentence bewe still need to identify which sections lying between two tween two potential anomalous sections, we can use the points are anomalous and which are not. If we merely created embeddings to compare the two anomalous sentagged them in an alternating fashion, it would lead to a tence embeddings and if the match passes a given threshlarge number of errors, as a single faulty division point old, flag the given non-anomalous sentence as anomalous. could mean that we misclassify our entire dataset. We We also look at the borders of anomalous sections. need some way of determining the anomalous nature of Let = 1, ..., be the labelling for the source text individual sections. = 1, ...., where has two possible values, – origi
We can assume that most anomalous sections are rel- nal and – anomalous. After the labelling has been creatively short compared to non-anomalous sections. We ated, we localize the starting point of each anomalous seccan pair up indices of division points that are located tion (− 1, ), and we compare the embedding similarity close to one another, specifically 20 sentences from each (via cosine distance) of the pair − 2, − 1 and the pair other. While we can create possible pairings of anoma- − 1, . If the similarity between − 1, was greater, lous parts that are located close to one another, there are we change our labelling of − 1 from − 1 to − 1, since individual division points that cannot be paired. They otherwise the labelling remains unchanged. We also might be a false positive or they correspond to beginning examine the relative similarities of pairs − 1, and or ending that has not been found yet. For each isolated , +1 and if we find that the similarity was greater in index, we construct multiple artificial sections that are case of − 1, , we change our labelling for from − 1 created varying distances before or after it. to − 1. We perform this operation until we are certain
Autoencoders, described in detail in [26] are unsuper- of the true anomaly border. We perform an analogous vised neural networks that aim to create a representation steps at the end of each anomalous section (− 1, ). of data that selects only the most relevant parameters, which can be used to reconstruct the original data. Autoencoders consist of two main parts: the encoder, which 3.3. Evaluation criteria converts the input into an encoding (usually of lesser Given that we know the positions of the anomalous parts dimension than the input), and a decoder that tries to in the texts of a prepared dataset, it is possible to use reconstruct the input from the encoding. Using simple the classical evaluation of the achieved results, namely feed-forward neural network, the encoding ℎ be calcu- accuracy, precision, recall and F1-Score. lated as: ℎ = ( + )
(8) where is the input, is an activation function, is a weight matrix and is the bias. This encoding can then be used to obtain ′, the reconstruction of the input. The reconstruction is calculated as:
on dataset created from a 1000 documents from various sources written in the Arabic language. The documents range from file sizes of 5 kB containing 20 sentences to ′ = ′( ′ℎ + ′) (9) those being as long as 12728 sentences and having a file where ′, ′, ′ might be diferent from , and . size of 3736 kB. The average sentence number of used
We use the sentence embedding methods described texts is 62 sentences with an average file size of 96.3 kB. in section 3.1 to encode every sentence of the text. We Dataset 1: The insertion of anomalies will be an artifitrained an deep autoencoder model on the sentences of cial process, where we will substitute small parts of texts the text. Autoencoders generalize the data they encode, (several sentences) from a diferent text of the dataset in and as such they can be used to detect how common order to create artificial anomalies. For each text, we first
decide whether the given text should be anomalous and if multiplies the word embeddings with their POS-Tag relit is we generate a random number of anomalies, ranging evance score, before averaging them. This method is from 1 to 10. Therefore, the number of anomalies in each described in subsection 3.1.4. and will henceforth be retext would range from 0 (non-anomalous, some texts are ferred to as POS-Tag pooling. The third method we have not modified) to 10 (most anomalies). implemented multiplies the word embeddings with the
When generating an anomaly in a text, we create a sen- IDF score of the given word before averaging them. This tence ofset, which will mark where the anomaly should method is described in subsection 3.1.2. and will hencestart. We then randomly choose another text from the forth be referred to as IDF pooling. The final method we dataset, that is diferent from the original one, from where have implemented multiplies the word embeddings with we will substitute sentences. We also decide on a sen- a specialized TF-IDF score where the term frequency is tence ofset in the other text, starting from which, we will calculated based on semantic similarity before averaging substitute a randomly generated number of sentences, them. This method is described in subsection 3.1.3. and ranging from 2 to 20 into the original text. This substitu- will henceforth be referred to as TF-IDF pooling. tion is only one sided, so that the sentences in the original We have used these sentence embeddings in conjuncfrom the given ofset and a given length will be substi- tion with a recurrent neural network with the task of tuted by the sentences from the other text, but the other predicting the embedding of the next sentence. text will remain unchanged. We repeat this process for The networks consists of two BI-LSTM layers with every anomaly in a given text, meaning that a given text a ℎ activation function connected to a dense feedcan have substituted parts from multiple sources. After forward layer with a activation function. The we have performed all substitutions, we save our newly input of the network consists of 5 previous sentence anomalous text, as well as create another document for embeddings, = 5, and the task of the network is to each text in which we will save the number of anomalies, predict the embedding of the next sentence. We chose the and for each anomaly their starting ofset and the length given value of based on our previous experience with of that anomaly calculated in number of sentences. Arabic texts (the sentences in the texts are quite long), but
Dataset 2: We have also created a modified dataset we will analyze this context length in future experiments. that replaces parts of texts from other semantically re- By comparing the prediction with the ground truth we lated texts. First, we use TF-IDF to extract the top 10 key- obtained anomaly scores used to mark where a given words of each texts and then use the K-means algorithm anomaly section potentially ends or begins. to cluster texts into semantic clusters. We use an identi- The software was created in Python using available cal process to create an anomaly text with one diference. libraries for working with texts and neural networks, When we create a new anomalous text, we’re swapping such as gensim, NLTK, keras and tensorflow . another text from the same cluster. This ensures that two texts and by extension the swapped sentences are 5.2. Results semantically similar, making our anomaly detection task more dificult but more relevant for real world usage.
tomatic semantic and syntactic analysis, therefore we evaluated several methods, four of which we describe in 5. Experiments the article and compare their application on two simulated datasets. The simulation of the datasets consisted 5.1. Experimental Setup in inserting part of the texts from other texts, while the We have created 500 anomalous texts in the Arabic lan- insertion positions are random but known for evaluaguage to use in our prediction algorithm using methods tion purposes. A non-trivial step is also merging anomadescribed in the Dataset preparation section. We have lous sentences into continuous parts of anomalous texts implemented 4 methods of semantic sentence embed- (anomalous sections). dings using Aravec Skip-Gram embeddings trained on The achieved results of these four methods on two data from Wikipedia, with a vector size of 300. datasets can be found in Tables 1 and 2. The obtained
To find the given threshold of embedding similarity results show that the most suitable method is the TF-IDF we have used an Arabic dictionary of synonyms as well pooling for both datasets. The TF-IDF evaluates two statisas an Arabic thesaurus [27]. tics, term frequency and inverse document frequency.
The most simple method involved simply average pool- This means that it records more information about the ing all the non-stop word embeddings of a sentence to cre- sentences in the text. ate our sentence embeddings. This method is described in The worst results were obtained for Average pooling, subsection 3.1.1. and will henceforth be referred to as Av- which uses the simplest approach, averaging word emerage pooling. The second method we have implemented beddings in a sentence. However, it is surprising that even the more complicated Dataset 2 reaches a value of
by the Average pooling and POS-Tag pooling methods, as their eficiency seems to decrease with a larger context window as they do not capture the relative uniqueness of the sentences due to lacking and IDF metric.
Jafari (2022) [17] showed that the outlier detector which uses a transformer based model adds more context values into anomaly detection models. The developed model was tested on Cross-lingual Natural Language Inference (XNLI) corpus which is the extension of the Multi-Genre NLI. The Stanford Sentiment Treebank (SST) is a corpus with fully labeled parse trees was used as auxiliary dataset which samples from it to be injected to XLNI dataset. A sample of 1000 English text taken from SST set injected to XNLI set to create a dataset in which XNLI sample text are normal and the SST samples are outliers. The results in the Table 4 show precision 92 %. 0.54 in the precision parameter. This means that the Dataset Valid Preciaveraging of word embeddings contributes significantly XNLI+SST Sample sion Recall F1 Score to important information in sentence embeddings too.
Another interesting thing of note is the trade-of re- Jafari [17] 3490 0.92 0.828 0.86 lationship between the precision and recall values. The Average pooling and POS-Tag pooling methods have a lot lower precision than recall. Meaning the algorithm 6. Conclusion tagged more sections than necessary. But the methods using IDF pooling and TF-IDF pooling suddenly start to In the article, we described four sentence embedding change the trade-of relationship, where the precision val- methods, which can be used to represent the semanues are approaching or even surpassing the recall values. tic content of sentences. These alongside a recurrent We believe the reason for it might be the inherent ability neural network were used to determine the ofsets of of the IDF metric that takes into account the semantic anomalous sentences. We then used the reconstruction content of other sentences. Therefore the semantic em- distances of sentence embeddings from an autoencoder beddings it produces are more unique. model trained on the entire text to determine the anomalous sections. These methods achieved good results on Table 3 simulated datasets. The presented methods can provide Sensitivity of k in regards to F1 score results on Dataset 2 warnings about positions of sections that could be anomalous in unknown texts.
Dataset 2 k=3 k=5 k=7 k=9 Our next goal is to use transformers in sentence embedPAOveSr-aTgaegppoooolliinngg 00..6691 00..6607 00..6547 00..5489 dings and to improve the algorithm for creating anomaIDF pooling 0.75 0.77 0.77 0.76 lous sections. Our further research will be oriented to TF-IDF pooling 0.86 0.87 0.86 0.84 the use of range-based performance metrics instead of point-based metrics in sentence embeddings, because a sentence is a sequence of words in a text and in pointbased metrics these words are taken as a set and could occur in diferent permutations. Word order is important in the sentence.
We have also opted to explore the sensitivity of 1score for the parameter (the number of previous sentences) for our various pooling algorithms on Dataset 2 using the 1 score metric and summarized the results in Table 3. We have determined = 5 as the optimal choice for our best performing pooling methods IDF pooling and Acknowledgments TF-IDF pooling. These two methods aren’t all that sensitive to change in the window size and we believe that a The research is supported by the Slovak Scientific Grant context window of 5 sentences is the optimal trade-of Agency VEGA, Grant No. 1/0177/21 “Descriptional between providing just enough past information while and Computational Complexity of Automata and Algoalso being capable of dealing with smaller anomalies. rithms”. [15] L. Ruf, Y. Zemlyanskiy, R. Vandermeulen,
T. Schnake, M. Kloft, Self-attentive, multi-context [1] A. Ghoting, S. Parthasarathy, M. Otey, Fast one-class classification for unsupervised anomaly mining of distance-based outliers in high- detection on text, Proc. of the 57th Annual Meeting dimensional datasets, Data Mining and of the Association for Computational Linguistics Knowledge Discovery 16 (2008) 349–364. (2019) 4061–4071.
doi:10.1007/s10618-008-0093-2. [16] J. Devlin, C. Ming-Wei, K. Lee, K. Toutanova, Bert: [2] A. Christy, G. Meeragandhi, S. Vaithyasubrama- Pre-training of deep bidirectional transformers for nian, Cluster based outlier detection algorithm for language understanding, Proc. of NAACL- HLT healthcare data, Procedia Computer Science 50 (2019) 4171–4186.
(2015). doi:10.1016/j.procs.2015.04.058. [17] A. Jafari, A deep learning anomaly detec[3] A. Jayasimhan, J. Gadge, Anomaly detection using tion method in textual data, https://arxiv.org a clustering technique, Inter. Journal of Applied abs/2211.13900 (2022) 8.
Information Systems 2 (2012) 5–9. doi:10.5120/ [18] N. Abouzakhar, B. Allison, L. Guthrie, Unsuperijais12-450391. vised learning-based anomalous arabic text detec[4] W.-K. Wong, A. W. Moore, G. F. Cooper, M. M. Wag- tion, in: N. Calzolari (Ed.), Proc. of the Sixth Interner, Rule-based anomaly pattern detection for de- national Conference on Language Resources and tecting disease outbreaks, American Association Evaluation, (ELRA), Marrakech, Morocco, 2008. for Artificial Intelligence (2002). [19] A. Salem, A. Almarimi, G. Andrejková, Text dis[5] J. Mu, X. Zhang, Y. Li, J. Guo, Deep neural network similarities predictions using convolutional neural for text anomaly detection in siot, Computer Com- networks and clustering*, in: J. Paralič (Ed.), Proc. of munications 178 (2021) 286–296. doi:https:// the IEEE World Symposium on Digital Intelligence doi.org/10.1016/j.comcom.2021.08.016. for Systems and Machines, IEEE, Košice, Slovakia, [6] G. Pang, C. Shen, L. Cao, A. V. D. Hen- 2018.
gel, Deep learning for anomaly detection: [20] Z. Szoplák, G. Andrejková, Anomaly detection in A review 1 (2020). doi:https://doi.org/10. text documents using htm networks., Proc. ITAT 48550/arXiv.2007.02500. 2021 (2021) 20–28. [7] O. Gorokhov, M. Petrovskiy, I. Mashechkin, Convo- [21] A. zahra El-Alami, S. O. E. Alaoui, N. E. Nahnahi, lutional neural networks for unsupervised anomaly Contextual semantic embeddings based on finedetection in text data LNCS 10585 (2017) 500–507. tuned arabert model for arabic text multi-class catdoi:10.1007/978-3-319-68935-7_54. egorization, J. of King Saud University – Computer [8] F. Mattia, P. Galeone, M. Simoni, E. Ghelfi, A survey and Information Sciences 34 (2021) 8432–8428.
on gans for anomaly detection (2019). [22] W. Antoun, F. Baly, H. Hajj, Arabert: Transformer[9] T. Y. Yap, Text anomaly detection with arae-anogan, based model for arabic language understanding,
Illinois Wesleyan University (2020). Proc. of the 4th Workshop on Open-Source Arabic [10] E. M. Billah Nagoudi, D. Schwab, Semantic Corpora and Processing Tools, with a Shared Task similarity of arabic sentences with word embed- on Ofensive Language Detection, Marseille, France, dings v1 (2017) 18 – 24. URL: https://hal.science/ European Language Resource Association (2020) 9– hal-01683485. 15. [11] A. Feroze, A. Daud, T. Amjad, M. K. Hayat, Group [23] K. S. Jones, A statistical interpretation of term specianomaly detection: Past notions, present insights, ifcity and its application in retrieval, J. of Documenand future prospects, SN COMPUT. SCI 2,119 tation 28 (1972) 11–21.
(2021). [24] D. Jurafsky, J. H. Martin, Speech and language [12] A. Saini, M. R. Sri, M. Thakur, Intrinsic plagia- processing (3rd ed., https://web.stanford.edu/ jurafrism detection system using stylometric features sky/slp3/14.pdf (2023) 1–29. and dbscan, 2021 Inter. Conference on Computing, [25] M. Schuster, K. K. Paliwal, Bidirectional recurrent Communication, and Intelligent Systems (2021) 13– neural networks, IEEE TSP 45(11) (1997) 2673–2681. 18. [26] D. Bank, N. Koenigstein, R. Giryes, Autoencoders, [13] I. Bensalem, P. Rosso, S. Chikhi, A new corpus for CoRR abs/2003.05991 (2020). URL: https://arxiv.org/ the evaluation of arabic intrinsic plagiarism detec- abs/2003.05991. arXiv:2003.05991. tion, CLEF 2013, LNCS 8138 (2013) 53–58. [27] S. Alali, A. A. Sheikh, S. Al-Ahmad, The net dic[14] I. Bensalem, Plagiarism detection: A focus on the tionary in the arabic language, https://www. alaraintrinsic approach and the evaluation in the arabic bimag.com/books/27725 (2019) 1–772. language (2020). doi:10.13140/RG.2.2.25727. 84641.