In today's world, numerous solutions using artificial intelligence are being actively developed to solve various tasks that people face every day. Accordingly, the results generated by artificial intelligence depend on the training datasets on which they were trained, in other words, the content of these datasets directly affects the final result. Lack of transparency regarding the sources and characteristics of the data used to train AI algorithms reduces confidence in the results obtained. In this case, users are often unable to appreciate the potential biases or discriminatory elements built into these algorithms. Insufficient awareness of the content of educational datasets increases the risk of spreading unfair or inaccurate decisions, which can have serious consequences for individuals and society as a whole [1].
Means for evaluating the representativeness of a textual data set in accordance with the principles of ethical non-discrimination are currently lacking. This is especially relevant for socially important and sensitive tasks according to SDG3 (good health and well-being), SDG4 (quality education) and SDG16 (peace, justice, and strong institutions), for example, detecting cyberbullying, determining the emotional state of people based on text messages, etc. The lack of attention to ethical components when creating and using datasets leads to bias in algorithms, which negatively affects the fairness and reliability of the decisions made [2].
Well-known datasets for training neural networks, for example [3,4], are actively used by researchers, because they have a large amount of data, but they were not validated by the authors regarding representativeness according to the principle of fairness, and therefore, the use of such datasets for training artificial intelligence algorithms may potentially violate ethical principles and, hence, have a low reliability of the decisions made.
The representativeness of the data in the datasets not only affects the accuracy of the results and models, but is also closely related to the principles of FATE (Fairness, Accountability, Transparency, Ethics) in the use of data and development of artificial intelligence technologies. If dataset does not include adequate representation of all social, demographic, or cultural groups, it can lead to discriminatory patterns that prioritize one group over another, so are not fair. The representativeness of datasets according to ethical principle of FATE is achieved by correct balancing according to various ethical aspects: gender, religious, age, etc. [5].
The main contribution of the paper is the development and validation of an approach to the analysis and formation of representative text samples of data according to the principle of fairness of FATE for subject areas.
Further, in chapter 2, a review of works related to the topic of the study, namely the formation of representative text samples and the issue of impartial representation of demographic groups according to the principle of justice, is carried out. Chapter 3 offers a description of the method of analysis and formation of representative samples of text data, the datasets used for further experimental studies of the effectiveness of the given method are given and described. Chapter 4 contains an experimental study. Section 5 presents the results and discussion. Chapter 6 concludes the work.
Many works have been devoted to the study of the representativeness of text samples and the fair and unbiased representation of demographic groups in them, since the concepts of representativeness, fairness and impartiality are important in the creation of ethical and fair machine learning models [6]. Natural language processing tools are widely used for this purpose [7]. Recently, authors have increasingly paid attention to the issue of representativeness of data in samples, but the current state suggests that data sets have gaps in the representation of gender and race, and the complex nature of demographic variables makes classification difficult and inconsistent. Thus, the question of representativeness of data in sets that include people with disabilities and the elderly is considered. The authors recommend increasing representativeness by adding samples for underrepresented groups, including by collecting additional data or using synthetic data methods to improve representation of minorities and people with disabilities.
In the article [8], the authors raise the important problem of sample representativeness in the context of machine learning and artificial intelligence, emphasizing the need for accurate representation of population data. The main strategy that the authors propose to achieve high quality models is the use of stratified samples, which allow to reduce the variability between subgroups and accurately reflect the proportions between different categories in the population.
The authors of the study [9] consider biases arising both from class imbalances in the data and from sensitive (protected) characteristics such as race or gender. The approach increases model accuracy by balancing classes and reduces dependence on sensitive features, which improves group fairness.
IBM researchers have developed an open-source AI Fairness 360 toolkit for evaluating and reducing discrimination in machine learning models [10]. The main purpose of the toolkit is to detect bias based on attributes such as race, gender or age, and to provide methods for representation of all given social groups at different stages of model development.
The article [11] highlights the problem of intersectional biases in natural language processing (NLP) models, namely the unrepresentative and biased representation of different groups of people in textual datasets. The results showed that although existing debiasing methods (for example, for BERT or RoBERTa) preserve the predictive accuracy of the models well, their ability to reduce intersectional biases is limited.
The authors of [12] propose a specialized model of machine learning to detect and minimize bias in textual data, in particular, in news articles. The authors claim that their approach is effective because of deep models and transformative architectures that are able to detect and correct biases at different stages of machine learning.
The article [13] presents the problem of gender bias in natural language processing models, solving it using two main approaches: statistical and causal fairness. Researchers use techniques such as counterfactual data augmentation for causal debiasing, as well as resampling and revaging methods for statistical debiasing. The results showed that the combination of these techniques allows for significant bias in the models by both statistical and causal metrics.
Article [14] is devoted to solving the problem of intersectional bias in the predictions of machine learning models, in particular deep neural networks. Researchers propose a new method based on the Apriori algorithm for automatically detecting biased subgroups in data. It allows efficient generation of frequent subgroups and calculation of fairness metrics for them.
In [15], the authors identify and classify bias in natural language processing using transformer models such as BERT. The authors explore different ways to identify bias, including identifying social characteristics such as gender, race, religion, and sexual orientation.
The study [16] examines the problem of cyberbullying, which is a threat to people based on different characteristics, such as religion, age, ethnicity, and gender. The data set used by the authors has been modified with ethical considerations in mind, which ensures responsible AI.
The cited works show that the formation of representative and unbiased samples is a relevant research area, however, most of the works are devoted either to the detection of unbiasedness or to the analysis of the representativeness or unbiasedness of data samples, however, data samples must be modified to achieve compliance with FATE principles.
So, summarizing, it is possible to highlight the features of the modern approach, which is applied to the development of AI models (Fig. 1). However, this approach does not take into account existing ethical principles and non-discriminatory, representative presentation of existing population subgroups, which should be applied to obtain AI models. The purpose of the work is to ensure compliance with the ethical aspects (gender, religious, age, etc.) of the FATE principle of justice [5] for educational datasets, which consists in creating a method of analysis and formation of representative (according to the specified aspects) text samples of data. To achieve the specified goal, it is necessary to propose a method that will implement the following research tasks:
β’ to develop an approach to the analysis and formation of relevant representative datasets according to the principle of fairness of FATE for subject areas. β’ to investigate the effectiveness of the proposed approach, by using it for the applied analysis of the text dataset and bringing it to a representative view according to the aspects of the FATE principle of justice: gender, age and religion.
In contrast to the existing approach to training AI models (see Fig. 1), the study proposes a new approach (Fig. 2), which will ensure the representativeness and ethical correctness of the datasets used for training AI models. In order to implement the proposed approach, we will present: the information model and presentation of the task of forming representative samples of text data as an optimization task; steps of the method of analysis and formation of representative samples; a way to obtain a typical ML model for the ethical aspect; description of the composition of the datasets for the study.
The problem of obtaining a representative, ethically unbiased text dataset can be presented in the framework of an information model of the following form:
where DS is the text dataset for analysis and correction, DSΚΉ is the text dataset after correction, C is the set of classes of the subject domain of the dataset, A is the set of ethical aspects, M is the set of trained machine learning models (separate for each ethical aspect), F is the objective function minimizing the deviation between current and desired ratios for all ethical aspects.
In (1), the initial dataset DS and the corrected dataset DSΚΉ can be represented as:
where D is the set of elements of the DS dataset, Metadata is the set of metadata of the DS dataset, DΚΉ is the set of elements of the DSΚΉ dataset, MetadataΚΉ is the set of metadata of the DSΚΉ dataset.
Each element of the set of elements of the dataset D in (2) and each element of the set of elements of the dataset DΚΉ in (3) is a tuple of the following form:
where the attribute text is the textual content of element d or dΚΉ; cx is the class of the subject area of the dataset to which the element belongs, cx β C; ACx is a set of classes of dataset element belonging to ethical aspects.
Thus, in (4) cx and ACx are the marking (marking) of the content of the text element. In (4), the set of classes of membership of the dataset element DS or DSΚΉ in (1) to the ethical aspects Ax is presented in the form of a tuple:
where ax -classes of element belonging to ethical aspects; k is the number of ethical aspects to be analyzed, k = |Ax|.
At the same time, in (5) according to (1) Ax β A, and classes of dataset elements belonging to ethical aspects are elements of the corresponding sets, unique for each of the ethical aspects:
The Metadata set of the DS dataset in (2) includes:
where nDS is the number of elements in D, nDS = |D|; ANDS is the set of quantities of dataset elements belonging to each class of each ethical aspect from Ax; ATDS is the set of available proportions of items for each class relative to the total for each ethical aspect from Ax, nΚΉDS is the target number of elements in DΚΉ; ANΚΉDS is the set of target quantities of dataset elements belonging to each class of each ethical aspect from Ax; ATΚΉDS is the set of target proportions of elements for each class relative to the total amount for each ethical aspect from Ax.
At the same time, in (8), each element anDS,i of the set ANDS corresponds to a separate i-th ethical aspect and is represented by a tuple of the following form:
where nDS,i,1 is the number of elements in the dataset of the 1st class of the i-th ethical aspect, nDS,i,2 is the number of elements in the dataset of the 2nd class of the i-th ethical aspect, nDS,i,j is the number of elements in the dataset of the j-th class of the i-th ethical aspect, k is the number of classes of the i-th ethical aspect.
Similarly to (9), in (8) the proportions of the elements atDS,i of the i-th ethical aspect are represented by a tuple of the following form:
where tDS,i,1 is the ratio of the number of elements in the dataset of the 1st class of the i-th ethical aspect to the total number of elements in the dataset, tDS,i,2 is the ratio of the number of elements in the dataset of the 2nd class of the i-th ethical aspect to the total number of elements in the dataset, tDS,i,j is the ratio of the number of elements in the dataset of the i-th class of the i-th ethical aspect to the total number of elements in the dataset.
At the same time, for the values ( 9) and (10) in accordance with (8) for each i-th ethical aspect, the equality holds:
In contrast to (8), the set of MetadataΚΉ of the DSΚΉ dataset in (3) includes:
where nΚΉΚΉDS is actually the number of elements in DΚΉ obtained as a result of adjustment, nΚΉΚΉDS = |D|; ANΚΉΚΉDS is the set actually obtained as a result of adjusting the quantities of dataset elements belonging to each class of each ethical aspect from Ax; ATΚΉΚΉDS is the set actually obtained as a result of adjusting the proportions of elements for each class relative to the total number for each ethical aspect from Ax.
Thus, in ( 8) and ( 13), ( 9) and ( 11) hold for ANΚΉDS and ANΚΉΚΉDS, and ( 10) and ( 12) hold for ATΚΉDS and ATΚΉΚΉDS .
Thus, according to (4), ( 6) and (7), the text dataset D has the number of elements n = nDS = |D| and can be presented in the form:
where C = {c1, c2, β¦, ck}, where k is the number of classes of dataset D, m is the number of ethical aspects.
According to ( 6) -( 10), the solution of the problem is aimed at obtaining the dataset DΚΉ, which contains the total number of elements nΚΉ = nΚΉDS = |DΚΉ|, quantitatively balanced according to the ethical aspects Πi from the set of ethical aspects Π:
where each aspect Ai contains classes Π‘i and target proportions of classes Tij or each element of class Π‘; Π‘ is the set of classes of the ethical aspect Ai , C = {c1, c2, β¦,cj}; j is the number of classes of the ethical aspect of Ai.
To balance the dataset for each ethical aspect, it is necessary to use trained or train an appropriate number of classifier models, which can be as deep learning models, for example, BERT, LSTM, GRU, as well as machine learning models of Logistic Regression, Naive Bayes, Support Vector Machines, k-Nearest Neighbors etc. [17], and according to (1), the set of trained models of classifiers M is presented in the form:
Thus, within the framework of the proposed information model, it is necessary to perform the transformation D β DΚΉ with the condition of maximal correspondence nΚΉΚΉDS β nΚΉDS, ANΚΉΚΉDS β ANΚΉDS ΡΠ° ATΚΉΚΉDS β ATΚΉDS.
The study proposes to reduce the task of building a representative, ethically unbiased dataset to the task of multi-criteria optimization. The optimization task consists in minimizing the deviation between the current and desired class ratios, taking into account the limitations on the number of samples in the classes and the possibilities of generating synthetic data.
Input data: textual dataset DS, set of ethical aspects A, requirements for representative distribution DSΚΉ.
The goal of the problem: to create a representative sample for all ethical aspects that achieves the target class proportions for each ethical aspect D β DΚΉ.
Variables: xij -number of samples of class Cj in aspect Ai after sequestration and augmentation.
The objective function F is the minimization of the deviation between the current and desired ratios for all ethical aspects simultaneously, taking into account constraints ( 18) -( 21):
Limitations of the task: 1) the sum of all class samples within one aspect is equal to the target number of samples for this aspect (4):
where ni is the number of classes in the aspectAi; 2) the number of samples for each class should correspond to the target proportion of classes:
3) the estimated number of samples cannot be negative:
4) the ability to add new samples should match the ability to generate new data for each class and aspect:
where Gij is the maximum possible number of samples of class Cj in aspect Ai, that can be added.
Based on the set optimization task of forming a representative dataset (17), we present the steps of the method of analysis and formation of representative samples of text data.
Method for analysis and formation of representative text datasets is presented in the form of three consecutive stages: preprocessing, analysis of representativeness according to ethical aspects and representative adjustment of dataset. Each stage consists of its own steps, which are shown in Figure 3.
The input data of the method is the dataset DS for analysis, which according to ( 2) and ( 8) contains the target number of nΚΉDS elements, the set of ethical aspects A with subsets of classes, the target proportions of ATDS classes and the number of ANΚΉDS elements in the classes of ethical aspects, respectively, the trained set of models Π for each ethical aspect from A, which uses balanced samples for each ethical aspect for training.
At stage 1, a sample of text data in D β DS is pre-processed, namely, the removal of noninformative text fragments such as punctuation marks, numbers and special characters [18]. Removal of emoticons is not performed, as in many cases including emoticons in the analysis improves the accuracy of machine learning models used to classify texts based on emotional or mood content [19]. Incorrect records (empty, uninformative, etc.) are also deleted.
At stage 2, an analysis of the representativeness of the sample of textual data is carried out, taking into account ethical aspects. First, it is necessary to vectorize and classify each element βd β D of the data sample using separate machine learning models m β M for each of the ethical aspects Ai β A. The existing proportions of ANDS and ATDS classes for each of the ethical aspects are determined. The amount of shortage or excess of elements of each class for each of the ethical aspects is calculated. After that, the sufficiency of the data in the sample for augmentation is analyzed (minimum availability of samples of the relevant classes, etc.).
The deletion operation is performed to remove redundant elements of each class for each of the ethical aspects with minimal damage to other distributions, for which the optimization problem of selecting redundant elements in the framework of (17), which should be removed to achieve the target proportions of classes, is solved.
The add operation is performed to create new items using one of the known methods, for example, the SMOTE method [20]. Requirements are created in the form of the necessary combination of classes of each of the ethical aspects for each new element, for which the optimization problem of forming requirements for the missing elements is solved within the framework of (17). The output data of the method is a text dataset DΚΉ β DSΚΉ, which has the required volume nΚΉDS and is balanced according to the required proportions ATΚΉDS according to the selected ethical aspects Ax β A.
The steps of the method of analyzing and generating representative samples of text data will allow you to generate text samples that are non-discriminatory and unbiased and reflect a proportional representation of the sample samples to the actual demographic subgroups of the population, which will affect the accuracy and transparency of training machine learning models for solving various problems.
To test the method of analysis and formation of representative samples of textual data, an input dataset was formed based on two datasets "Cyberbullying Classification" [3] and "Cyberbully Detection Dataset" [4]. The "Cyberbullying Classification" dataset contains 46,017 tweets, which are labeled by types of cyberbullying into 6 classes. The "Cyberbully Detection Dataset" contains 99,989 tweets, which is also labeled by type of cyberbullying. Both datasets are unlabeled for gender, age group, religion, and ethnicity of the message author.
To train machine learning models, which will be used to label the input dataset, datasets were used on the example of three ethical aspects of the principle of justice of gender, age and religion.
The English-language dataset "Tweet Files for Gender Guessing" [21], which contains 34,146 unique text entries, which are divided into two classes: female and male, with 17,073 entries in each class, was used to train ML based on the ethical aspect of the gender of the author of the message. On the basis of the English-language dataset "CyberBullying Detection Dataset" [22], which contains 20109 test samples, a sample was created for training the classifier and marking the input dataset according to the religious ethical aspect. The dataset in Italian "TAG-it Dataset Distribution" [23] was translated into English and used to bring to a representative view the working dataset by age and contains 21,948 text messages divided into age classes: 0-19, 20-29, 30-39, 40-49, 50-100 years old.
Since the classes in the given datasets are not balanced and have a different number of samples, which will negatively affect the quality of training of machine learning models, all classes in the datasets were balanced in terms of number. The final number of samples in each class of training samples for ML training according to ethical aspects is shown in Fig. 4.
To study the effectiveness of the method of analysis and formation of a representative sample of text data, a software implementation was created using the Python programming language. The tensorflow library (https://www.tensorflow.org/) was used to classify the input dataset on cyberbullying based on gender, age, and religion. In Fig. 5 shows an example of classification based on the religious basis of the FATE-principle of justice. To form a set of trained ethical machine learning models, which are separate for each ethical aspect, various classifier models were analyzed, and to select the best of them, their quality was evaluated by statistical indicators, such as Accuracy, Precision, Recall, and F1-score [24]. Both deep learning models, such as BERT, GPT, LSTM, GRU, etc., and classifiers such as Logistic Regression, Naive Bayes, Support Vector Machines k-Nearest Neighbors, etc., were studied [25]. After that, the classifier is trained on the selected ML model on the annotated dataset for the ethical aspect.
As a result, different architectures were chosen as classifiers: FastForest classifiers, SVM and LSTM, BERT deep learning models [26]. Thus, machine learning models such as FastForest, SVM, LSTM, and BERT are effective tools for solving text classification tasks, including determining a person's gender, religion, and age based on user text posts. Classical approaches such as FastForest and SVM have also demonstrated their effectiveness in text classification. FastForest works efficiently with large datasets and prevents overtraining. SVM, in turn, is known for its ability to work with high-dimensional data, which is especially useful for text classification, where each word or phrase can be represented as a separate feature [27]. Deep learning models, such as LSTM and BERT, are able to recognize complex patterns in text sequences, preserving the context at all stages of analysis [28]. A distinctive feature of LSTM is its ability to retain information about previous parts of the text, which makes this model effective for complex classification tasks where the overall context of the message is important. Studies have shown that such a model can achieve an accuracy of up to 92% in text classification tasks [29]. The BERT model, in turn, is characterized by the ability to analyze the text in two directions, that is, to take into account both the previous and subsequent context of words [30].
To analyze and form a representative sample of text data for the target proportions of classes, to form a representative sample of text data by age and gender, the population of Ukraine was taken. According to the M. V. Ptukh Institute of Demography and Social Research of the National Academy of Sciences of Ukraine (https://idss.org.ua/forecasts/nation_pop_proj), as of July 2023, the total population of Ukraine is estimated at 3,559,6216 people. The following number of people is represented in each age subgroup: age group 0-19 years 6,659,068 people, 20-29 years 3,623,143 people, 30-39 years -6,022,345 people, 40-49 years -5,431,140 people, 50 -100 years -13,860,520 people. Regarding the gender structure of the population of Ukraine in 2023: 1,695,1527 are women, and 1,864,689 are men (idss.org.ua). Note that within the scope of this work, the cisgender group is considered in the analysis of the gender ethical aspect.
To study the effectiveness of the method of analysis and formation of a representative selection of text data described in the work, several machine learning models were trained. The results of calculating static metrics such as Accuracy, Precision, Recall and F1-score [24] of machine learning models for the gender, age and religious ethical aspects are shown in Table 1. For different classes, different levels of linear resolution were obtained: according to religion using the BERT classifier, which showed the best result of the trained machine learning models for the task of classifying text samples according to the religious ethical aspect, the data turned out to be well separated, according to gender using the LSTM classifier, which showed the best performance compared to other models, the data turned out to be moderately separable, and according to age, using the SVM classifier, it was poorly separable.
In addition, it was found that the dataset is not representative, because the classes of various ethical aspects have a number of text samples that do not correspond to the proportions of the demographic subgroups of the population of Ukraine, thus they need balancing to acquire a representative appearance.
Therefore, according to the steps of the method of analysis and formation of a representative sample of text data, a sample of text data needs data augmentation to form a representative sample. For this, it is necessary to solve the optimization problem, for the correct removal of redundant elements of each class according to each of the ethical aspects, with further augmentation of the data sample to the target requirements (number of elements and proportions of classes).
Table 2 presents the percentages of samples by age in the sample of textual data and individuals of the population in age-demographic subgroups, and also calculates the new distribution of the sample classes if only one ethical aspect -age -was taken into account.
Percentage ratios of samples by age in the sample of text data and individuals of the population in age demographic subgroups, % Table 3 presents the percentages of samples by gender in the sample of textual data and individuals of the population in gender demographic subgroups, and also calculates the new distribution of sample classes if only one ethical aspect -gender -was taken into account.
The deviation of the sample distributions by classes of the age-ethical aspect of the dataset, transformed according to the created method, from the ideal representative distribution was obtained: minimum 0.01%, maximum 0.04%, average 0.02%, and for the gender ethical aspect: minimum 0.03%, maximum 0.03%, average 0.03 %. However, the optimization task of forming a representative sample of textual data is a multi-criteria one, in which the criteria are the formation of a sample based on age and gender ethical aspects, so the goal is to minimize the deviation between the current and desired class ratios, taking into account the limitations on the number of samples and the possibility of generating new data. As a result of solving the optimization problem for the formation of a representative sample by age and gender ethical aspects on the example of demographic subgroups of the population of Ukraine, a representative sample of text data was obtained by augmentation, the balance of classes of which is presented in Table 4, Fig. 6 and Fig. 7.
Distribution of samples in the formed representative sample after data augmentation as a result of solving a multi-criteria optimization problem
The deviation of the sample distributions by classes of age and gender ethical aspects of the dataset simultaneously, transformed according to the created method, from the ideal representative distribution was obtained: minimum 0.00%, maximum 0.04%, average 0.02%.
So, as a result of performing the steps of the analysis method and forming representative samples of text data, a text sample was formed, which is non-discriminatory and unbiased and reflects the representation of sample samples proportional to the real demographic subgroups of the population of Ukraine.
Thus, the goal of the study was achieved through the development of the method for analysis and formation of representative text datasets, designed for the analysis and formation of representative text samples of data according to the principle of fairness of FATE for subject areas.
To investigate the effectiveness of the analysis method and the formation of a representative presentation of the text dataset, software was created that uses machine learning models to classify texts according to various ethical aspects -age, gender, religion, ethnicity, etc. Thus, to classify the text samples in the sample according to the age-ethical aspect, SVM was used, LSTM was used for gender, and BERT was used for religious ones, which are the best indicators of statistical metrics.
As a result of the practical application of the developed method, it was established that the available dataset is not representative compared to the objective data of demographic statistics, so a multi-criteria optimization problem was solved and the dataset was transformed into a representative one in terms of age and gender ethical aspects. The obtained deviations of the sample distributions by classes of ethical aspects of the dataset transformed according to the created method from the ideal representative distribution were: minimum 0.00%, maximum 0.04%, average 0.02%, under the conditions of the initial volume of the dataset 47,692 elements, the minimum initial number of samples in the class 1007 elements, the maximum initial number of samples in the class is 28,112 elements. The studied efficiency proves that the developed method allows performing the analysis of the representativeness of text datasets and bringing them to a representative form according to various aspects of the FATE fairness principle.
The obtained results contribute to improvement of representativeness of text datasets and fair and unbiased representation of demographic groups in them, which increases trust in decisions made by artificial intelligence, and complies with goals SDG3 (good health and wellbeing), SDG4 (quality education) and SDG16 (peace, justice, and strong institutions).
Further plans for improving the method of analysis and formation of representative samples of text data are the formation of not only a non-discriminatory sample by the number of samples, but also the search and removal of samples of text samples that contain a biased attitude towards representatives of various demographic subgroups, according to the ethical aspects of the FATE-principle of justice.
Also, the prospects for further research are the use of the developed method for adjusting textual datasets of subject areas and their use for solving applied problems, such as detection and classification of cyberbullying, analysis of the emotional tonality of messages, detection of the physical and mental state of users based on their posts, etc. Detecting performance gains from using ethically balanced text datasets will provide feedback for improving the developed method.
During the preparation of this work, the authors used Grammarly in order to: grammar and spelling check; DeepL Translate in order to: some phrases translation into English. After using these tools/services, the authors reviewed and edited the content as needed and take full responsibility for the publication's content.