Stop words are very important for information retrieval and text analysis investigation tasks of natural language processing. Current work presents a method to evaluate the quality of a list of stop words aimed at automatically creating techniques. Although the method proposed in this paper was tested on an automatically-generated list of stop words for the Uzbek language, it can be, with some modifications, applied to similar languages either from the same family or the ones that have an agglutinative nature. Since the Uzbek language belongs to the family of agglutinative languages, it can be explained that the automatic detection of stop words in the language is a more complex process than in inflected languages. Moreover, we integrated our previous work on stop words detection in the example corpus” bynviestigating how to automatically analyse the detection of stop words in Uzbek texts. This work is devoted to answering whether there is a good way of evaluating available stop words for Uzbek texts, or whether it is possible to determine what part of the Uzbek sentence contains the majority of the stop words by studying the numerical characteristics of the probability of unique words. The results show acceptable accuracy of the stop words lists.
The application of Natural Language Processing (NLP) tasks in real-life scenarios are getting more frequent than ever before, and there is huge research getting involved with different approaches to enhance the quality of such tasks. An important aspect of many NLP tasks that make use of tasks, such as information retrieval, text summarization, context-embedding, etc., relies on a task of removing unimportant tokens and words from the context under focus. Such data are known as stop words. Therefore, it is desired that some automatic method should be developed to identify stop words that either make no change in the meaning of the context (or do very little) and remove them. from the context.
In this work, we are addressing the problem of automatic detection of stop words for the
lowresource agglutinative Uzbek language, and evaluate the proposed methods. The existing literature
that deal with stop words removal task for the Uzbek language [
The scientific term "stop words" is popular in the field of natural language processing, and its definition we focus in this work is as follows: If the removal of those words from the text not only does not change the context meaning but also leaves the minimum number of words possible that can still hold the meaning of the context, then such words can be called stop words for this work.
For instance, the following examples are shown to better explain what words would be considered in given sentences, and what the final context would become after removing those stop words: ● “Men bu maqolani qiynalib yozdim”. (I wrote this article with difficulty). After removing the stop words me(“n”, b“u”, q“iynalib”) the context becom“eMs: aqolani yozdim”.(I wrote the article.); ● “Har bir inson baxtli bo’lishga haqlidir” (Every person has the right to be happy). After removing the stop wordshar(“”, b“ir”), the context become“sI:nson baxtli bo’lishga haqlidir” (Person has right to be happy).
Such definition is an extension of the traditional definition of stop words by including more words than the actual expectations but still including the traditional stop words.
The Term Frequency - Inverse Document Frequency (TF-IDF) method [15] was used to detect stop words in Uzbek texts. TF-IDF is a numerical statistic that is intended to reflect how important a word is to a document in a corpus, the method acknowledges words with the lowest TF-IDF values as less important to the semantic meaning of the document and proposes these words as stop word candidates.
In our previous work[
The existing fundamental papers that deal with stop words in general, let alone for the Uzbek
language, barely address the quality of the automatically detected list of stop words. This statement
also applies to our previous work, where a preliminary manual expert observation of a part of the lists
(only unigrams) was done. To the authors‟ kno,wtlheedrgee was no in-depth observation of the
accuracy of the automatically constructed lists of stop words for agglutinative languages. For
instance, [
The words were sorted by the TF-IDF value in descending order and the lowest 5 percent of them
were tagged as stop words. We used this method to automatically detect stop words in the corpus [
The rest of the paper is structured as follows: We start by explaining the related works in the field of stop word removal, as well as the Uzbek language itself in Section 2, followed by the main methodology of the paper in Section 3, which includes the creation of probability distribution law of TF-IDF of unique words (Section 3.1), the numerical characteristics of the probability of unique words (Section 3.2), and the evaluation of the created method using a small selected chunk (Section3.3). The accuracy of the method for automatic detection of stop words in Uzbek texts, which is based on TF-IDF, is presented in Section 4. The last section of the paper presents conclusions and future work (Section 5).
Uzbek language belongs to the family of Turkic languages. There has been some research on the
Uzbek language mostly in the last few years. Most of the research done on Turkic languages can be
applied to the Uzbek language as well, using cross-lingual learning and mapping approaches,
alongside some language-specific additions. The paper [
The paper Ошибка! Источник ссылки не найден. provides the first annotated corpus for
polarity classification for the Uzbek language. Three lists of stop words for the Uzbek language are
presented in [
Matlatipov et. al [10] propose the first electronic dictionary of Uzbek words‟ endings invariants for morphological segmentation pre-processing useful for neural machine translation.
The article [11] presents the algorithm of cosine similarity of Uzbek texts, based on TF-IDF to determine similarity. Another work on similarity in Uzbek, but this time on semantic similarity of words, a decent amount of work went on the creation and evaluation of a semantic evaluation dataset that possesses both similarity and relatedness scores Ошибка! Источник ссылки не найден..
The scientific novelty of the methodology used in this work can be shown as follows: ● The creation of probability distributions law based on TF-IDF scores of unique words; ● Thorough investigation of numerical characteristics of the probability of unique words; ● Better evaluation of the stop words detection method‟s accuracy; Summarising the automatic detection of the position of stop words in given Uzbek texts.
In our previous work[
Madatov et. al [
In order to determine the position of the stop words throughout the school corpus, we investigate the probability distribution law of TF-IDF scores of stop words.
Word weight and its probability. Select a word from the set of unique
words extracted from a corpus. For future references these two assumptions are valid: a word
represents a unique word from a corpus and a corpus represents the “School corpus” presented in
previous work [
The probability of can be calculated using the following formula: ∑ . We match for each word. Now ∑ .
The probability density function. Suppose unique words are distributed independently in the total corpus. In that case, word can be applied multiple times. In order to escape repeating the word
We consider only the first appearance of this word. For each word observe i as a random variable. As the probability density function of the unique words, we get the following function: f(i) can be considered as the probability density function of word .
In the Cartesian coordinate plane, observe i on the OX axis and observe along the OY axis. Figure 1 presents the described observations extracted from the “School corpus”.t toWoebsenreveed thie position of stop words along with the corpus.
This section presents numerical characteristics of the probability of unique words. They are calculated by the following formulas: ∑ the mathematical expectation of the unique words ∑ √ ∑
– dispersion of the unique words – standard deviation of the unique words
of the unique words third central moment of the unique words
The asymmetry of the theoretical distribution
The variety of words increases gradually with grades in the school literature. It means that the probability density function of unique words is not symmetrical. One may predict it without a mathematical way. However, mathematically, the data in Table 1, especially, , confirms that the probability density function is asymmetric.
The stop words are distributed along the axis (not grouped at one part of the axis); represented by orange dots in Figure 2. 3.3.
This section presents the probability density function of unique words of selected work from the corpus. Each book from the corpus is devoted to one topic.
The prediction: Every book consists of the culmination part of the topic, the rest can be stop words. That is why we investigated just one book.
A random book was selected from the range of 25 books (in the corpus): 11th class literature. The book consists of 12837 unique words. The same process that was presented in Section 3.2 was applied to just the selected part of the corpus in order to create the probability density function of unique words. Figure 3 shows the probability density function of 11th class literature unique words.
Table 2: Distribution analysis of the selected single book
We obtain Figure 4 by the rule of stop words detection method, as mentioned in [
means that the probability density function is asymmetric.
The values were sorted in descending order and the lowest 5 percent of them are candidates for stop words. Figure 4 graphically represents the process, words with probability less than are candidates to be a stop word ( = 0,00001034371184).
The number of these candidates is 642. 85,8% of these words is located outside of the interval . On the left side of the interval there are 545 stop words and on the right side are 6 stop words. The same facts can be observed graphically on Figure 5 (Taking into the account the numerical characteristics of 5% words of selected work and comparing Figure 3 and figure 4 we detected their position along with the text).
The accuracy of the presented method if confirmed using the following reasoning: Let suppose hypothesis H0: Stop words of the selected document (11th class literature) are located outside of the interval (E-σ,E+σ); and alternative hypothesis H1: Stop words of the selected document (11th class literature) are located inside of the interval (Eσ,E+σ).
The critical value – Z (Z-score or Standard score) is obtained using this Equation:
.; where N=12837, =6419, =7076.62, √ In the presented task |Z|≈21,526. Z is located reject the null hypothesis.
This is the basis for rejecting the H1 hypothesis.
Throughout the work performed in this paper, we presented a natural extension of the already
presented previous research of the automatic detection of stop words in the Uzbek language [
From all theoretical investigations from previous sections, it can be concluded that, for a single genre, the majority of stopwords have the following nature: if , are located at the beginning parts of the text; if , are located at the ending of the text; if , are located at the beginning at the ending part of the text.
. on the le-σft, smideeanionfg Ethere is no reason to
In future works, we would like to use the results of this article as the basis for automatically extracting keywords and automatically extracting the abstract of a given text.
The authors gratefully acknowledge the European Commission for funding the InnoRenew CoE project (Grant Agreement $\#$739574) under the Horizon2020 Widespread-Teaming program and the Republic of Slovenia (Investment funding of the Republic of Slovenia and the European Union of the European Regional Development Fund).
The paper presents a natural extension of the already presented research of automatic detection of stop
words in Uzbek language[
a) The probability density is defined and later used to observe the accuracy of the automatic method for extraction of stop words of Uzbek language. b) The accuracy of the method that is presented in Section Ошибка! Источник ссылки не найден..
From this fact it can be concluded that, for a single genre, more of the stop words for texts: if , are located at the beginning parts of the text; if , are located at the ending of the text; if , are located at the beginning at the ending part of the text.
Further we use this result in the process of automatically extracting keywords from the given text and automatically extracting the annotation of the given text.
[10] S. Matlatipov, U. Tukeyev, M. Aripov. “Towards the Uzbek Language Endings as a Language Resource”, In: Advances in Computational Collective Intelligence. ICCCI 2020.
Communications in Computer and Information Science, vol 1287. Springer, Cham., (2020) [11] S. Matlatipov. "Cosine Similarity and its Implementation to Uzbek Language Data," Central
Asian Problems of Modern Science and Education: Vol. 2020 : Iss. 4 , Article 8, (2020). [12] I. Rabbimov, S. Kobilov, I. Mporas. Uzbek News Categorization using Word Embeddings and Convolutional Neural Networks. 2020 IEEE 14th International Conference on Application of Information and Communication Technologies (AICT). pp 1-5, (2020), doi:10.1109/AICT50176.2020.9368822 [13] I. Rabbimov, S. Kobilov. “Multi-Class Text Classification of Uzbek News Articles using Machine Learning”. Journal of Physics: Conference Series. (2020), doi: 10.1088/17426596/1546/1/012097 [14] R. M. Rakholia, J. R. Saini, "A Rule-Based Approach to Identify Stop Words for Gujarati Language," In Proceedings of the 5th International Conference on Frontiers in Intelligent Computing: Theory and Applications, pp. 797-806, (2017) [15] C. Sammut, G. Webb, eds. “Encyclopedia of machine learning”. Springer Science & Business
Media, (2011) [16] Salaev, Ulugbek, Elmurod, Kuriyozov, and Carlos, Gomez-Rodriguez. "SimRelUz: Similarity and Relatedness scores as a Semantic Evaluation dataset for Uzbek language". In Proceedings of the the 1st Annual Meeting of the ELRA/ISCA Special Interest Group on Under-Resourced Languages (pp. 199–206). European Language Resources Association, 2022. [17] Y. Wijeratne, N. de Silva, "Sinhala Language Corpora and Stopwords from a Decade of Sri Lankan Facebook," arXiv, 2020, doi: 10.2139/ssrn.3650976.