The article considers the problem of determining the construction of a linear order of alternatives based on multiple comparisons of alternatives given by experts. In the electoral systems of different countries of the world, this approach is called preferential voting. The problem has wide practical applications and is successfully used in various spheres of human life. It is proposed to use an algebraic approach to solve this problem. In this case, to formalize the problem, the metric of rank mismatch of alternatives, sometimes called the Cooke metric, is used. In determining the resulting ranking, it is proposed to use simplifying heuristics that reflect different approaches to understanding the optimality of the resulting ranking. The problem of determining the resulting ordering of alternatives based on incomplete rankings or multiple comparisons is solved. A short list of classical voting methods and a scheme for solving the problem is given. An algorithm for determining the leader in the team and ensuring conflict-free preliminary voting is proposed and substantiated.
Many practical situations can be formalized in the class of group ranking problems. The number of alternatives to be ranked can be in the tens or hundreds, and the number of people whose preference systems must be considered can reach hundreds of thousands. It is known that the problem of group ordering of alternatives is NP-hard, and it is impossible to solve it with exact algorithms at large dimensions. This fact should be considered when building a mathematical model of this problem and when choosing methods for determining the resulting ranking. At the same time, in practice, there are often situations when experts are unable to reliably rank all the alternatives of the initial set that characterizes the decision-making situation. In such a case, experts are offered to make multiple comparisons of alternatives or to build an incomplete ranking. The advantage of the ranking procedure is the simplicity of this method of structuring information. However, when the number of alternatives approaches 20, it becomes difficult for experts to organize them, because this procedure requires establishing relationships between the entire set of selected alternatives. The ability to operate with large amounts of information is limited by human psychological characteristics, so in such cases, experts may make significant mistakes.
In various subject areas, practical situations do not require a formal straightforward solution but can be successfully resolved by applying special procedures. Such procedures smooth out the diversity of expert judgments, generalize the original problem and pre-coordinate the views of various decision-makers. Participants in the decision-making process pre-evaluate possible solutions to the problem, and thus the entire team of experts prepares in advance for a compromise solution. Such procedures cannot be spontaneous or implemented spontaneously: they must be prepared, formalized, justified, and researched in advance.
Preferential voting (preferential choice of alternatives) is an election system in which a voter has the opportunity to rank candidates for a position or the implementation of certain projects according to his or her preferences [1]. This area of research is one of the main ones in proportional open-list PR and flexible-list PR systems [2].
The peculiarity of the preferential system of choosing alternatives is that the voter (expert, team member, decision maker, etc.) votes not for one but for several candidates, whom he/she wants to see elected in the first place, whom - in the second, etc. [3, 4] In the political life of different countries, the institution of preferential voting aims to allow voters not only to vote for the list of candidates of a particular party but also to express their preference for certain candidates within this list, to promote their election and to increase their rating in party lists by taking into account the people's will [5, 6].
Today, preferential voting is officially and successfully used in various countries around the world: Australia; Austria; Belgium; Denmark; Estonia; India; Ireland; Italy; Malta; The Netherlands; New Zealand; Norway; Poland; Slovakia; Czech Republic; Sweden; Sri Lanka; other countries.
It is generally recognized that preferential voting allows for a more accurate expression of the will of voters and contributes to the representation of a wider range of political opinions in parliaments or other authorities [7]. Therefore, the study of issues related to preferential voting is extremely relevant.
At the same time, it is necessary to study various components of this problem: • models of preferential choice of alternatives; • methods for determining compromise solutions; • the degree of satisfaction of experts with the results of solving the problem; • opportunity to manipulate the preferential choice; • other aspects of preferential selection.
The purpose of the article is to develop approaches to solving the group ranking problem based on incomplete rankings or multiple comparisons of alternatives given by experts. Based on the proposed approaches, it is necessary, in particular, to study the procedure of preliminary preferential voting for groups consisting of several dozen members.
To create a tool for comparing incomplete orders of alternatives, a mathematical model should be developed that will contain an approach that will allow determining the distances between incomplete rankings and multiple comparisons of alternatives.
The study proposes a flexible mathematical apparatus for modelling the problems of preferential voting and adequate application of the analysis of the judgments of a team of experts in practice. The purpose of the study requires the development of an algorithm for determining the priority of alternatives and ensuring the independence of determining and setting individual preferences of experts to calculate the compromise ranking of alternatives.
In general, to achieve the research objective in practical situations and to ensure the implementation of fair election procedures, the following tasks should be performed: • develop and implement a procedure for confidential voting; • to give all voters equal rights to express their will: to select, organize and justify a subset of alternatives; • ensure the storage of information provided by experts; • • • to implement and enforce the procedure for protocolizing the election procedure; to calculate the reasonable resulting rankings of alternatives; develop and implement procedures for interpreting the results of collective choice.
(
Let k experts carry out a preliminary examination of n alternatives or candidates for the position from set A . According to the terms of the examination, the members of the team (experts, voters, etc.) at the first stage must set multiple comparisons or incomplete rankings of alternatives [8-10], and n . An additional requirement may be a fixed value of for all experts. The upper limit of the value is due to the psycho-physiological capabilities of a person. It cannot be greater than 9, for example, given the Miller number 7 2 .
We will denote the subset of candidates selected by i team member, on which the team member establishes the order relation in the form of multiple comparisons, by Ai , i I 1,..., k, n . Note that at the initial stage, the number of team members and the number of alternatives coincide: k n , although in general, the values of k and n may differ significantly. Without loss of generality, we assume that
Ail aij ... aij , ,
1 for all team members: i I , l 1,..., n. .
In some practical situations, such a procedure can be considered as an intermediate stage in the transition from informal leadership, i.e. the process of influencing the team with the help of one's abilities and skills, to formal team management, i.e. the process of influencing team members based on the position held.
In other words, as a result of such elections, the team determines priorities among informal leaders without conflict and ensures a smooth transition to administrative leadership and the fulfilment of several tasks: • identification of a personality for whom team members recognize the right to make the most responsible decisions affecting their interests; • determination of the relative importance of the person officially responsible for managing the team and organizing its activities; • individuals from the set of informal leaders are recognized and respected in the team.
It is necessary to find the resulting ranking of alternatives consistent with the given multiple comparisons of experts, taking into account the metric of rank mismatch and various criteria. That is, it is necessary to find a reasonable order that is closest to the given expert multiple comparisons.
When formalizing decision support tasks using expert technologies, a mathematical apparatus is used that has proven itself in many practical problems [11]: • selection tasks; • binary relationships; • criterion-based selection.
In addition, heuristics are used in one way or another to solve problems arising from the application of expert technologies. In some cases, experts are asked to participate in the formation of an adequate model of the problem, in others - to choose a formula for aggregating information, in others - to determine the ratio of preference on certain elements of the mathematical model, in others to name the probability of certain events, in others - to choose metrics and criteria for determining the best solutions to the problem, etc. Moreover, in many practical situations, expert evaluation is the only possible way to solve real problems. The successful use of expert assessments in this process is largely due to the perfection of the mathematical apparatus for analyzing and processing expert information [12].
Such approaches can be applied in problematic teams [13, 14]. The purpose of the procedure for preliminary ranking of alternatives may be: • minimize the level of conflict that is brewing or has already entered an active stage; • a soft test of the level of support for potential or informal leaders in the team, as well as for candidates for the position of administrative head of the team; • assess existing groupings in the team, i.e. structure its polarity, identify individual leaders in subgroups of the team and thus obtain more information about the team, distribution of sympathies among team members, level of popularity and number of leaders in the team, etc; • determine the competence ratios of the members and/or the level of satisfaction with the final choice, with the experts with the highest relative competence obviously being the most realistic and able to be directly involved in conflict resolution.
It should be noted that in the interpretation of this particular problem, a team is a group of people united by common activities, common interests, a declared common goal, or the need to focus their efforts on a project. The team in which the preliminary survey is conducted is usually characterized by the following factors: • having a common goal; • regular joint activities aimed at achieving the goal; • the presence of formal and informal leadership; • established business and psychological relationships between team members.
In this regard, it is necessary to additionally note the peculiarities and main factors of the preliminary rating: • all members of the team know each other; • the assessment of applicants by team members is reliable; • each individual assessment by a team member does not reflect the personal qualities of the candidates for the position they are elected to, but only the attitude of team members towards leaders.
At the same time, the organizers of a preferential poll should not focus on the negative and should not organize an anti-leadership poll.
4.2.
Incomplete information is an attribute of decision-making situations. Incompleteness of expert information is a natural phenomenon that often arises in practice and is one of the types of uncertainty along with vagueness, inaccuracy, unreliability, uncertainty, incorrectness, inadequacy, insufficiency, etc. Incomplete data and the inability to supplement them naturally accompany experts and decisionmakers in their activities in various subject areas [15].
The ability to handle large amounts of information is limited by human psychological characteristics, so experts may make significant mistakes when they need to handle large amounts of information. Asking experts to perform an unstructured task means deliberately creating unreliable initial data in advance. Incomplete information is a natural manifestation of objective decision-making situations: not only the personal characteristics of individual experts but also of the teams that perform the expertise.
To successfully overcome this problem, a flexible approach to the ordering of alternatives can be applied, when each expert can propose a partial ordering of a subset of the entire set of alternatives. At the same time, researchers can take into account the situation when some experts may not be aware of the specifics of individual alternatives related to the overall problem and the specifics of the functioning of the entire system.
Definition 1. A multiple comparison or incomplete ranking R(н) of a set A is a ranking in which not all n alternatives of the set A have an order relation defined. In this case, the Ai A,i 1,..., k, inclusion relation is strict. That is, in the set A there is at least one alternative that is not in the subset Ai ,i 1,..., k.
Remark 1. Multiple comparisons differ from incomplete rankings in the number of alternatives presented to the experts for comparison or alternatives they proposed and ordered. For multiple comparisons, this value is 3-7 alternatives, and for incomplete rankings, it is more than 7 alternatives.
In this formulation of the problem, the problem is that in the tasks of collective ranking of alternatives, as a rule, situations are considered when experts set individual rankings in a fixed space with a given number of alternatives. In the classical algebraic approach, each expert has to set his or her preferences on the set of all alternatives without exception, and the resulting (compromise, collective, group, aggregated, integrated, integral, etc.) ranking is found on the set of all possible rankings of the given set of alternatives. The resulting ranking found in this way is considered to be the generalized opinion of a group of experts involved in solving the problem of linear (complete) ordering of a given set of alternatives. In this case, we speak of the application of the axiom of nondisplacement [16]. To develop a comprehensive system of interaction between system elements, we will use the formalism of binary relations, which is conveniently expressed as a directed graph or as a matrix of pairwise comparisons. In the presence of a significant number of comparison alternatives, the construction of such a matrix is a laborious process. Therefore, in practice, in addition to the procedure of pairwise comparisons of alternatives, multiple comparisons or incomplete rankings are often used to reduce the labor costs of experts. The results of multiple comparisons usually consist of complete relationships between 3-7 alternatives. However, achieving completeness, consistency, transitivity, coherence, etc. and meeting the requirements for these indicators in full requires a lot of effort and time from experts. To solve the problem of ranking based on individual multiple comparisons or incomplete rankings, the number of alternatives in which remains less than the entire set of alternatives to be ranked.
4.3.
The expert group members often do not agree on the subset of proposed alternatives, the preferences of experts on the selected subset, the competence of experts, the interests of the parties, the priorities chosen, the emphasis of evaluation, aspects of formalization, the idea of model idealization, etc. Therefore, at the first stage of solving the problem, an approach based on combining the subsets specified by the experts and introducing additional heuristics should be used.
At the first stage of solving the problem, the subsets of multiple comparisons of alternatives given by experts are combined into a single set A , which includes all candidates proposed by experts ai A, i 1,..., n,
k A Ai , A i1
A, where A is the initial set of all team members (experts), A is the union set of subsets of candidates selected by team members.
Different variants of the relationship between subsets are allowed: Ai1 Ai2 , Ai1 Ai2
, Ai1 Ai2 , i1,i2 J.
It is also necessary to introduce heuristics for aggregating (smoothing) individual multiple
comparisons. After applying the heuristics, the set (
Heuristic E1. There is a single team, i.e., a graph that corresponds to the preferences of the team
members, revealed on the basis of the previous vote, although this does not exclude the situation of
polarized leadership.
(
Proposition. For selection and ordering, only the kernel of the form (
Even after the domain of valid solutions has been narrowed to a subset of (
Heuristic E2. The least popular candidates can be removed from the final subset A A, without significantly affecting the final solution.
Combining all the alternatives given by experts into a single set ai A,i 1,..., n, to determine the resulting ranking R* is the initial stage of solving the problem described in this paper.
To narrow down the union set (
Heuristic E3. The candidate who participated in one multiple comparison and was ranked 3rd in this multiple comparison is excluded from the combined set A A .
If, after applying heuristic E3, the union set A A remains large, the following heuristics are applied sequentially until the union set A A is sufficient to directly search through all possible permutations of the alternatives included in this union set.
Heuristic E4. The candidate who participated in one multiple comparison and was ranked 2nd in this multiple comparison is excluded from the combined set A A .
Heuristic E5. A candidate who participated in one multiple comparison and was ranked 3nd in this multiple comparison is excluded from the combined set A A .
Heuristic E6. The candidate who participated in two multiple comparisons and was in third place in both of them is excluded from the combined set A A .
Heuristic E7. The candidate who participated in two multiple comparisons and was in second place in both of them is excluded from the combined set A A .
It should be noted that if the size of the combined set A A is large, additional heuristics can be added by analogy. The sequence of their application may vary, depending on the structure of preferences set by team members. In general, the problem of the influence of heuristics on the level of satisfaction of team members requires additional study and computational experiments.
The formalization of the problem of determining the resulting ranking for incomplete expert rankings and multiple comparisons of alternatives can take different directions. The most common method of finding the resulting ranking of alternatives is to calculate the median of the given rankings. This group of methods for generalizing expert information is the most reliable and mathematically sound. The solutions to the problem, which are determined by applying different metrics and different criteria, are the medians of the linear orders specified by the experts. In particular, this is due to the fact that the theory of measurement provides for the determination of average values in the form of medians for ordinal scales. Therefore, the methods of average ranks, etc. in such cases look at least approximate and questionable: they cannot provide confidence in the adequate choice of a leader.
If you use a scoring system to solve this problem, there will be a bias because all the team members are interconnected. After all, an organization is a living organism, and indirectly, the opinions of each member are not autonomous. Therefore, evaluating possible applicants in terms of points would mean not taking into account, destroying the interconnections between the team members.
In such cases, instead of scoring, the evaluation of relations in the form of following approaches can be used: • matrices of pairwise comparisons between alternatives; • ranking of alternatives; • multiple comparisons between alternatives; • incomplete rankings on a set of alternatives.
The most common method for finding the resulting ranking of alternatives is to calculate the median of the given rankings. This group of methods for generalizing expert information is the most reliable and mathematically sound. The solutions to the problem that are determined by applying different metrics and different criteria are the medians of the linear orders specified by the experts.
Methods of processing expert information are divided into three main groups: • statistical methods; • scaling methods; • algebraic methods.
The essence of algebraic methods is that a distance is set on the set of valid ratings and the resulting rating is defined as the one whose distance to the ratings of experts according to a certain selected criterion is minimal. Based on the algebraic approach, we will determine the coefficients of relative competence of experts.
An important way to present expert information is to rank alternatives [17, 18]. In many practical tasks, it is necessary to use multiple comparisons or incomplete rankings. At the same time, classical methods of voting theory have been applied to the problems of determining a generalized ranking of alternatives. However, in such cases, it is promising to use algebraic methods of calculating the median, which reflects the collective opinion of experts.
The expert reveals more about his preferences, structure, interrelationships between alternatives in his view, and prospects for leadership in the team.
The set of all alternatives given by experts of the form (
The distances between the rankings of alternatives are determined using metrics [19, 20]: • Cooke's metric of the mismatch of ranks (places, positions) of alternatives in individual rankings [19].
Where ril is the rank of the i-th alternative in the ranking of the l-th expert Rl ,l L, 1 ril n, •
Heming's metrics [20].
d R j , Rl ri j ril ,
iI d B j , Bl 0,5 bisj bils ,
iI sI where Bl bils , l L, i, s I , are the pairwise comparison matrices (PCMs) corresponding to the rankings Rl , l L; • Euclid's metrics; • a vector of preferences, the elements of which are the number of alternatives that precede each alternative in the ranking.
In this study, we will use the rank mismatch metric, most referred to as the Cooke metric, to
determine the resulting ranking of alternatives. The criteria most often used in such cases:
• additive;
• minimax.
(
The most common method for finding the resulting ranking of alternatives is to calculate the by B . The set of rankings given by experts will be denoted by R A , and the set of corresponding matrix of pairwise comparisons by R B .
For the case considered in this paper, the power of the sets and R A R B n , Rl RA, Bl RB , l L . It is clear that RA R , RB B . In general, the power R B is the same: R A of the set
B 2nn1/ 2 . But for the method described in this paper, we will assume that R B n!, since we are not interested in the non-transitive elements of the solution space B .
For the Cooke metric (a metric of mismatching the ranks of alternatives) of the form (
RCS CS Arg mRinR lL d R, Rl .
When using the egalitarian criterion, the HV median (compromise) is calculated [8]: RГВ ГВ Arg min max d R, Rl .
RR lL
For the Heming metric (
RВГ ВГ Arg min max d B, Bl .
BB lL In addition, such tasks may consider the coefficients of expert competence: 1,..., k .
The criterion functions used to determine the medians (
The tasks of preferential voting are being solved today: • using the classical methods of choice theory (Condorcet, Borda, Simpson, Copeland, Nanson, alternative votes, relative majority, etc.); • The single transferable vote (STV) system is an electoral system of quota-preferential voting, which aims to minimize the loss of votes and ensure proportional representation of all population groups.
However, the algebraic approach we used to solve this problem is sound, transparent, and has proven itself in practical problems. The algebraic approach has various variations, which we explore in this paper.
Cook's metric of the form (
To determine the consistent ranking of candidates, we will introduce an additional heuristic.
Heuristic E8 (maximum satisfaction of preferences). It means that an expert wants all the
alternatives in his multiple comparison to become winners in the sequence he sets, i.e., he wants to
(
Then it is easy to see that when the options from which a certain ranking is chosen depend on the sequence of indices of the alternatives of the same three, it is possible to determine the distances for choosing a compromise ranking.
For the sequences ai1 ... ai3 ... ai2 , and ai2 ... ai3 , the distances are 2. For the sequences ai2 distances are 4.
...
ai3 ...
ai1 , ai3 ...
ai 1 ...
ai ...
1 ai2 , and ai3 ...
ai2 ...
a , the i 1 d Ml 1 ril 2 ril 3 ril ,
1 2 3
Heuristic E4 (moderate reciprocity). In other words, each team member understands that everyone cannot be a winner in a fair vote, that the probability of satisfying all wishes is negligible. Moreover, no one can win, but it is necessary to at least maintain the individual order set by each member of the team: it is important for victory.
In this case, the distances from the given individual multiple comparisons to the resulting ranking R0 ai01 ai02 ... ai0 , are determined as follows: dVl ri10 ri1l ri20 ri2l ri30 ri3l . l I. .
Note that when v=4, we have 3 multiple comparisons that are at a distance of 2 from the possible candidate configurations in the resulting ranking, 7 multiple comparisons at a distance of 4, 9 multiple comparisons at a distance of 6, and 4 multiple comparisons at a distance of 8.
When v = 5, the distribution of distances has the following indicators. 4 multiple comparisons - 2, 12 multiple comparisons - 4, 24 multiple comparisons - 6, 35 multiple comparisons - 8, 24 multiple comparisons - 10, 20 multiple comparisons - 12.
The tasks of group (collective, collegial, democratic) ordering of alternatives allow modeling practical situations in various subject areas. The peculiarity of the generally accepted formulation of such tasks is the set of alternatives fixed for all members of the expert group, which in many cases significantly reduces the quality of the expertise. Currently, there are studies that allow for the possibility of setting incomplete rankings in collective ranking problems [8, 12], but algebraic methods for calculating the median have not yet been applied to such problems.
A flexible approach to the ordering of alternatives, when each expert can propose a partial ordering of a subset of the entire set of alternatives selected by him or her, and the search for the full resulting ranking of alternatives by algebraic methods have not been proposed by researchers. After all, this makes it possible to consider a situation when some experts may not be aware of the peculiarities of the functioning of individual elements from the set of alternatives.
• Transportation logistics: the sequence of visits to nodes along the route; • Reverse warehouse logistics: the sequence of unloading goods from a warehouse or transport; • Management of a trust company: priority of investing in shareholdings; • Purchase of vehicles or other equipment (thermal imagers, Kevlar helmets) for the Armed
Forces of Ukraine; • Preliminary election of the head of the department; • Backpack problem: the importance of having items; • Submitting for awards under the quota; • Formation of a track of Ukrainian songs; • Building a library with space constraints; • Nominating candidates for the student parliament;
Creating a restaurant and preliminary study of demand with the help of a focus group: planning a restaurant menu is an important element of the hotel and restaurant business; Forming a repertoire of films for collective viewing by a group of colleagues; Purchase of a collection of fashion brands for sale in a clothing store; Choosing a website layout program, where experts rank the alternatives in the order they think is best to worst, from best to worst website layout program; Purchase of cars for the company - for ease of maintenance, etc; Selecting a project for funding at a university research center; Organizing a corporate holiday and determining the menu of the festive table; Providing a "cushion" of goods in the warehouse; Purchasing laptops for the office: for support and maintenance, it is necessary that all devices are unified, so it is appropriate to choose one model among all of them; Preparation and approval of a set of information security measures for the organization [12, 21].
A computational experiment was conducted to verify the proposed approach and the developed method. The method of prioritizing candidates based on preferential voting was applied to the preliminary selection of the leaders of one of Taras Shevchenko National University of Kyiv. This voting was organized with the aim of further presenting the most authoritative candidate for his official appointment to an administrative position.
The employees of the department (members of the team) were asked to confidentially set their preferences through Google Sheets: to select three leaders in the department and set a priority - the most desirable candidate in the first place, the second - the candidate desirable after him, the third the desirable candidate in case the above leaders are not identified as winners in the compromise ranking of leaders. To ensure confidentiality, all possible leaders were numbered. A total of 16 members of the department took part in the survey. After the subsets of leaders were combined, they were renumbered. The results of the voting are shown in Table 1.
E1
E2
E3
E4
E5
E6
E7
E8
E9
E10 E11 E12 E13 E14 E15 E16
Since the combined set of leaders contained 7 candidates, there was no need to apply heuristics to narrow this set. To illustrate and further argue the results of the calculation, the number of places assigned by all members of the department to the candidates they proposed was calculated. These results are summarized in Table 2. 6 5 7 7 5 2
L3 3 4 2 9 2 5 4 2 3 1
After that, a direct search of all possible permutations of the 7 alternatives was carried out, i.e., 7!
possible rankings of team leaders were considered. To determine the distances in the ranking space,
we used the Cook's metric of the form (
After applying the minimum-maximum criterion of the form (
A confidential expert survey of all department employees after the compromise solution was determined confirmed that the two rankings of candidates for the position of head of the department generally satisfied all voting participants. This indicates the effectiveness of the developed method and the possibility of its application for preliminary identification of leaders in teams.
In many tasks, events that need to be ordered using incomplete expert rankings must be performed in parallel or even occur simultaneously. Therefore, it is logical to formalize the task in the class of collective quasi-order computation [22, 23].
It is also promising to develop parallel algorithms using artificial intelligence methods, the use of which in the described approaches can contribute to a synergistic effect in the process: • formalization and further optimization of business processes; • solving information recovery tasks in terms of expert preferences based on group ranking; • applying the formalisms of the collective ranking problem to a wide class of classical combinatorial problems in the descriptions of the corresponding formulations to adapt and interpret the formulation [24, 25].
Thus, a procedure for preliminary preferential voting for teams consisting of several dozen members is proposed and studied. This task is formalized, and additional heuristics are introduced to substantiate the approaches that will be used to determine the resulting ranking of candidates for the position.
The problem of ordering a subset of alternatives is considered. Based on this expertise, an algebraic approach to the analysis of incomplete rankings is proposed. The classical methods of group selection are compared with the algorithm for determining the median with incomplete initial information. The scheme of solving the problem in the above formulation is described.
Thus, this paper investigates approaches to determining the resulting ranking based on incomplete expert rankings and obtains the following main results: • the concept of incomplete ranking was introduced; • the problem of determining the group ranking of alternatives based on incomplete expert rankings or multiple comparisons is proposed; • approaches to aggregation of expert data are considered, taking into account the peculiarities of incomplete information received from experts; • algorithms for solving the problems of calculating collective ranking based on individual multiple comparisons were developed; • computational experiments were conducted to study the described method of preferential voting and the features of solving ranking problems. 10. References International Scientific and Practical Conference "Information Technologies and Security" (ITS 2021), Kyiv, Ukraine, December 9, 2021 / CEUR Workshop Proceedings, 2021, 3241, pp. 169– 180. [13] Cao, PP., Zheng, J., Wang, S. et al. Large group decision-making considering multiple classifications for participators: a method based on preference information on multiple elements of alternatives. Complex Intell. Syst. (2023). https://doi.org/10.1007/s40747-023-01209-x [14] Elkind, E. Distance rationalization of voting rules / E. Elkind, P. Faliszewski, A. Slinko. // Social
Choice and Welfare. – 2015. – Vol. 45, No. 2. – Pp. 345–377. [15] Hnatiienko H., Snytyuk V., Tmienova N., Zatonatskiy D. and Zhylinska O. Mathematical Support of the Task of Determining the Strategic Directions of Development and Priorities of the Organization / CEUR Workshop Proceedings, Volume 3347, Pages 169 – 184, 2022 // 9th International Scientific Conference "Information Technology and Implementation", IT and I 2022, Kyiv, November 30 - December 02, 2022. [16] Hnatiienko H., Snytyuk V. A posteriori determination of expert competence under uncertainty / Selected Papers of the XIX International Scientific and Practical Conference "Information Technologies and Security" (ITS 2019), pp. 82–99 (2019). [17] D’Ambrosio, A. Kemeny’s axiomatic approach to end consensus ranking in tourist satisfaction /
A. D’Ambrosio, V.A. Tutore. // Statistica Applicata. – 2008. – Vol. 20, No. 1. – Pp. 21–32. [18] Alnur, A. Experiments with Kemeny Ranking: What Works When? / A. Alnur, M. Meila. //
Mathematical Social Sciences. – 2012. – Vol. 64, No. 1. – Pp. 28-40. [19] Cook, W.D. Distance-based and adhoc consensus models in ordinal preference ranking. / W.D.
Cook. // European Journal of Operational Research. – 2006. – No. 172. – Pp. 369–385. [20] Amodio, S. Accurate algorithms for identifying the median ranking when dealing with weak and partial rankings under the Kemeny axiomatic approach // S. Amodio, A. D'Ambrosio, R.
Siciliano. // European Journal of Operational Research. – 2015. – No. 249. – Pp. 667–676.
[21] Babenko, T., Hnatiienko, H., Vialkova, V. Modeling of the integrated quality assessment system
of the information security management system / CEUR Workshop Proceedings, Volume 2845,
2021, Pages 75-84 // 7th International Conference ""Information Technology and Interactions"",
IT and I 2020; Kyiv; Ukraine; 2 December 2020 through 3 December 2020; Code 168286.
[22] Wang, J.; Zhou, X.; Li, S.; Hu, J. An Extended TODIM Method and Applications for
MultiAttribute Group Decision-Making Based on Bonferroni Mean Operators under Probabilistic
Linguistic Term Sets. Symmetry 2023, 15, 1807. https:// doi.org/10.3390/sym15101807.
[23] Ali, G., Afzal, A., Sheikh, U. et al. Multi-criteria group decision-making based on the
combination of dual hesitant fuzzy sets with soft expert sets for the prediction of a local election
scenario. Granul. Comput. (2023). https://doi.org/10.1007/s41066-023-00414-w.
[24] M.A. González, A. Oddi, R. Rasconi Multi-objective optimization in a job shop with energy costs
through hybrid evolutionary techniques. Proceedings of the 27th International Conference on
Automated Planning and Scheduling (ICAPS-2017) (2017), pp. 140-148.
[25] M.T. Emmerich, A.H. Deutz A tutorial on multiobjective optimization: fundamentals and
evolutionary methods. Nat Comput, 17 (