This paper examines the current state of expert decision-making technologies, proposing a formalization of these technologies while addressing key challenges in the field, including the potential for manipulation. It provides an overview of areas in human life where manipulation might occur and presents the stages of expert decision-making technologies, along with a framework outlining their sequence and interconnections. The paper investigates the challenges that may arise at each stage of applying expert technologies and examines their underlying causes in detail. Particular attention is given to the issue of choice manipulation, identifying specific domains where such manipulation might occur. A definition of manipulation within the context of expert evaluation is introduced, and heuristics are formulated to support the study of this issue in expert technology applications. Finally, the paper proposes the foundational principles of a methodology to counteract manipulation in expert evaluations.
Expert evaluation tasks are widely used across various fields of human activity, serving as a critical tool for formalizing and structuring situations in diverse subject areas. In many cases, expert technologies are the only viable means of arriving at an acceptable solution to a practical problem [1, 2]. However, since these solutions are inherently not optimal but represent compromises, they are particularly susceptible to manipulation. This study focuses on developing the foundational principles of a methodology to counter manipulation in the application of expert technologies [3].
Expert technologies are a powerful and often indispensable tool for addressing a wide range of practical problems [4, 5]. Today, tasks across various fields of human activity are effectively tackled through the application of expert knowledge and its judicious use [6, 7]. These technologies are particularly well-suited for solving poorly structured or entirely unstructured problems [8, 9]. However, it is crucial to carefully examine potential anomalies that may arise during the application of expert knowledge. Additionally, the possibilities for manipulation when using expert technologies must be thoroughly formalized and addressed [10].
2.1. Formal representation of expert decision-making technologies
⟨ , , , , , , , , , , ⟩, where − a set of attributes (objects, options, alternatives, parameter sets); −a set of constraints; −a set of criteria; −a set of measurement scales by criteria and alternatives; −mapping the set of valid alternatives to the set of criterion scores; −a set of generalizations of criteria; −a set of formal characteristics of experts; −a set of goals set by researchers; −a system of preferences set by a decision maker, a team of decision makers in collective decision-making, or a group of experts at various stages of preparation for decision-making; −problems that arise at different stages of the application of expert technologies; −manipulation (forgery) of choice at different stages of decision-making in case of problems with the use of expert technologies.
All the components of tuple (1) have a great impact on the quality and efficiency of expert technologies in decision-making situations. In this paper, special attention will be paid to the last two components of tuple (1) - the problems of using expert technologies at different stages of their application( ), and some aspects of manipulation in the use of expert technologies( ) . In order to reduce the risks of manipulation of expert technologies, the methodological foundations of countering manipulation will be considered.
Manipulation accompanies us in all spheres of life. Today, various aspects of manipulation are widely studied [11]. This phenomenon is the subject of research in various fields of science - sociology, psychology, political science, philosophy [12], ethics [13, 14], management [15], choice and decisionmaking theory [16, 17], mathematical statistics, expert evaluation [18], etc.
Manipulation is an action targeted at process participants with the aim of imperceptibly influencing and changing the results expected by the process participants in their favor.
Today, this phenomenon is being comprehensively studied in order to both improve the tools of influence and successfully counteract such influences. The successful result of such research is the prevention of manipulation and early detection of manipulation attempts.
● In the sociological sciences, the manipulation of mass consciousness is comprehensively studied; ● Political science is the study of various forms of political manipulation that influence the behavior of voters, groups of individuals, and society as a whole in order to gain power; ● psychologists study the phenomena of psychological manipulation and the possibility of targeted influence on people.
There are also various areas of research related to this topic: ● Information manipulation is an interdisciplinary field with a wide range of research topics; ● organizational manipulation, which explores additional opportunities for influencing members of the workforce in parallel with the development of organizational theory; ● managerial manipulation, which aims to use additional levers of control in companies to achieve management goals; ● domestic manipulation, which has become widespread and is studied by representatives of various fields of research; ● a wide range of other aspects of manipulation.
Formalization of the task of expert evaluation
10.1. Solving a specific problem using mathematical methods and computer technology
At each stage outlined in Table 1 and visualized in Figure 1, various problems naturally and inevitably arise. These challenges clearly increase the potential for and likelihood of manipulation. To explore possible manipulation strategies and implement measures to counteract them, it is essential to examine the essence and characteristics of each stage of expert technology application in greater depth. Let us now consider the attributes of each stage.
researcher
The elements of vector (3) have the following meaning:
−Some or most members of the expert panel do not meet the requirements for experts because they cannot perform such operations:
). −highlighting the best object; −selection of an unordered subset of the best objects; −selection of an ordered subset of the best objects; - ranking (ordering) the entire set of objects; - orderly breakdown of objects (stratification, group ordering); - disordered division of objects (classification); −The purpose of the expert evaluation is incorrectly chosen; −The researchers misunderstood the goals that were set for them; −Inadequately selected tool for expertise and decision support; −The problem of expert evaluation is incorrectly solved; −The results of the expert evaluation contradict the company's status quo; −Distrust of scientifically based methods; −Informal ties in the company are stronger than the recommendations of the expert group; −The management did not use the results of the expert evaluation task: there is no will to implement innovative solutions.
5.2. Diagnosis of the problem, formulation of the expert evaluation task, preliminary situational analysis, detection and identification of problems, (3) (4) (5) problem statement
The elements of vector (4) have the following meaning: −Experts often misidentify the emphasis and main tasks; = ( at a different level of competence; −A global problem is replaced by partial ones or a general problem is identified that is solved −The task is formulated inadequately and the group of experts performs unproductive work; −A superficial analysis is carried out without knowledge of the specifics of the situation, company traditions, analysis of the background, tracking the dynamics of activities and data trends; −Problems are identified based on information from interested managers and those who know how to present the issues in a favorable light for their unit;
−Deep and significant problems remain undiscovered and are further obscured behind the illusion of a scientific study; −An unacceptably generalized, rough statement of the problem is made; −An unreasonably detailed statement of the problem is made; −An inappropriate class of models is selected to model the situation. 5.3. Formalization of the expert evaluation task: compilation of a list of criteria, task constraints, construction of measurement scales, formation of a set of alternatives
The elements of vector (5) have the following meaning:
−Inadequate formalization of the problem is carried out - the use of formal methods in those sections and in ways that are inappropriate, unreasonable and ineffective in a particular situation; −Lack of formalized procedures for structuring the problem area in the consultants' arsenal; −A rigorous form of dialog between consultants and experts in the process of identifying their knowledge; unclear definition of roles in peer review procedures; −Insufficient responsibility of consultants in the process of peer review moderation and −Consultants are not sufficiently qualified to select and use formal expert evaluation models; −The list of criteria for the task is not carefully compiled; −The task criteria selected by experts and consultants are not characteristics of the degree of achievement of the sub-goals of the global goal; characteristics of a person and only according to arithmetic laws; −The
measurement scales are built without taking into account the psychological −The set of alternatives is formed by a directive method, without the use of creativity and expert technologies; alternatives: −The consultants did not explain, and the experts failed to generate a sufficient number of −Excessive, logically unjustified increase in the complexity of the task: an excessive number of criteria, failure to take into account the dependence of criteria, etc; the number of alternatives, parameters, etc.
−Increasing the dimensionality of the problem by having too many gradations on the scales, take into account the same aspect of the consequences; −The selected criteria are not general and cannot serve as a measure for all alternatives; −The criteria selected for modeling the problem at hand can be represented as constraints; −The set of criteria does not meet the principle of completeness: −using some additional criteria may lead to a change in the decision; −rejecting some selected criteria does not change the decision; −The criteria do not meet the principle of simplicity: −the requirement of non-redundancy is not met - different criteria from the set should not −the minimum requirement is violated - the set of some should contain as few criteria as possible; identification of possible challenges and prospects; specifics of the process being modeled and evaluated; −The task's limitations are set formally, without a deep analysis of the retrospective and −The measurement scales are selected from a traditional set, without taking into account the (6)
The elements of vector (6) have the following meaning: −Simulation of multicriteria in the selection of a mathematical model; −The choice of the class of models was limited to the choice of an additive model using artificially assigned values of the coefficients of expert competence or ignoring the level of their competence;
−A mathematical model was chosen that does not correspond to the peculiarities of the subject area and, at the initial stages of solving the expert evaluation problem, creates distrust of the expert group in the adequacy of the modeling; the solution offset the effort of building the model; results of modeling; −An adequately formulated mathematical model does not have software to implement it, so −The use of simplified methods to solve a complex mathematical problem, which negates the lead to significant deviations and loss of the right direction of the solution; organization methods; −Finding a solution is practically a simple matter of searching through the possible options; −The chosen method is not sufficiently formalized, and there are no clear rules of operation; −The absurdity of finding a solution in such circumstances is declared as a principle; −The criteria for assessing the level of ideas put forward have not been developed, which can −The results obtained are extremely dependent on the preparation and conduct of the group's −A significant part of the success of the examination depends on the team leader. 5.5. Formation of the expert group, formation of the rules of work of the expert group
The elements of vector (7) have the following meaning: −The problem of the size of the expert committee is not taken into account, since the number of experts significantly affects the accuracy of the group assessment;
−When choosing the number of experts, the need to achieve a compromise between accuracy and labor intensity of the examination was not taken into account;
−Reducing the number of experts leads to a decrease in accuracy, as the final results are significantly influenced by each expert: the point of forming expert opinions by a group of specialists is lost;
−An increase in the number of experts increases the accuracy of the assessment, but complicates the organization of the assessment, extends the timeframe for its conduct, complicates the processing of survey results, and offsets the influence of each expert on the group assessment, which is not compensated by an increase in the reliability of the assessments;
−Increasing the expert group to create the illusion of its significance at the expense of lowskilled specialists; problems in expert evaluation tasks;
−The problem of individual selection of experts arises, which remains one of the most difficult (collective, collegial) decision-making:
−The need for maximum attention of the decision maker and experts to this stage, although there are no methods of forming an expert committee that guarantee full objectivity of the −Insufficient attention to errors caused by poor structuring of the problem and insufficient information about the object, process or phenomenon being modeled;
−Application of procedures that ensure control over the occurrence of errors caused by the interest of experts in the results of the examination, which necessarily affects the reliability of the −Failure of the decision-maker and consultants to take into account the peculiarities of group subordinated to the company's goals; −Distributed responsibility; −High probability of making a risky decision; −The influence of personal preferences on expert assessments is great; −Low degree of influence of personal preferences of each expert on the resulting expert −The need to implement procedures that ensure that the goals of each expert are −Breadth of judgment, inconsistency, and complexity of assessment; −Long time spent on the evaluation procedure; −There is still a possibility of accepting an unsatisfactory, manipulated expert opinion; −There is little dependence on a single expert to produce a collective expert opinion; −Failure of the organizers of the expert evaluation to ensure that its participants are fully informed of the information directly related to the subject matter of the expert evaluation; −Failure of consultants to ensure maximum creative activity of experts; −Inability to ensure the independence of an expert within a hierarchical organization; −Lack of proper communication and interaction between experts; −Restriction of creative independence and activity of experts.
The elements of vector (8) have the following meaning: −The problem of quantitative-qualitative transformation: experts often express their opinions in qualitative rather than quantitative terms;
−The transformation of quantitative into qualitative (and vice versa) can be carried out using special methods of the theory of representative measurements, which is part of the statistics of nonnumerical objects; acceptable alternatives; identification of acceptable alternatives; −Failure of the consultants to explain to the experts the difference between possible and −Unscrupulous, overly pessimistic or exaggeratedly optimistic attitude of experts to the −A formal arithmetic approach to generating a set of valid alternatives, forecasts, etc.
The elements of vector (9) have the following meaning: = (8) (9) −The problem of choosing the number of rounds of examination when applying the Delphi method: objectivity increases, but the costs of the procedure (both time and material) increase; −Lack of anonymity when using the
method of commissions (meetings, conferences, seminars, roundtables): a group of experts participating in meetings and expressing their opinions is guided mostly by the logic of compromise; −Collective responsibility for the final expert assessment; −Reflection of the company's management hierarchy in the course of the examination; −Demotivation of experts due to the belief that the collective assessment does not depend on the expert - and this is often the case; −Insufficient preparation of the examination; −Imperfection of the expert technologies used; −Use of unreasonable methods of comparing contradictory judgments; −Imperfection of the methods used to process expert information; −Different understanding by experts of the purpose of the examination; −Opposition of experts' interests; −Expert opinions contain general phrases with minimal or no numerical data; −Insufficient use of scientifically based methods of expert evaluation; −Unreasonable comparison of different groups of objects under study; −Systemic errors in the formation of professional groups of experts; −Incompleteness of information, which arises from the fact that different experts will always have different knowledge of the event and its level of uncertainty; hundreds of times; examination; leads to an increase in the level of uncertainty rather than a decrease; the involved experts significantly distorts the information; −The ambiguity of some questions that may be misunderstood by the expert, which ultimately −The lack of competence and/or interest in some of the certain results of the expert review of −Errors in the mathematical model used lead to a misuse of the results obtained from experts; −Excessive enthusiasm for quantitative assessments leads to incorrect results - it is necessary to limit oneself to qualitative assessments of alternatives;
−Use ranking only in cases where it is impossible or inappropriate to use direct assessments: objects with neighboring ranks may have a difference in the intensity of the feature by tens or −When ranking alternatives, there may be too many, which affects the results of the −When ranking, some experts may consider certain objects to be incomparable and not include them in the overall set, allowing them to make incomplete rankings;
−Experts must clearly understand what they are evaluating and on what scale to avoid situations where experts evaluate the same indicator characterizing an object based on different premises: that is, the construction of scales must be psychologically sound;
−If the number of alternatives being evaluated is sufficiently large, the procedure of pairwise comparison of all possible pairs becomes laborious for the expert;
−Assuming the consistency of the expert's assessments, a single presentation of each alternative in conjunction with any other is almost sufficient: but this assumption is too strong and can lead to a significant loss of information.
−Expert procedures should help to ensure the objectivity of experts; −Ensuring that emotional and psychological factors are avoided;
The elements of vector (10) have the following meaning [19, 20]:
−When formulating rules for assessing the competence of experts, take into account the basic requirements for experts: −be able to generalize knowledge; −recognize and identify problems; −draw plausible conclusions from incomplete information; −explain their judgments; −Reconstruct and reorganize your information; −determine whether the problem is within his or her competence; −to be compared with pre-known true answers of an objective nature, although not all types of expert evaluation tasks can be selected for situations for which true answers are known in advance; −consistency, stability, repeatability in the expression of preferences, when experts are offered the same questions, the same objects, but separated in time and sequence of presentation; −the ability to build a complex solution rule taking into account all the criteria without simplifying the strategy during expert evaluation;
−transitivity, which is a very common condition, sometimes called the basic rule of inference; expert is rather indifferent;
−The results of experts' comparisons of different objects often do not correspond to the postulate of transitivity - non-transitivity is characteristic of judgments about objects to which the
−To determine the level of competence of experts using self-assessment, the expert determines the degree of his or her awareness of the issue under study; information is obtained about the level of self-confidence of the expert, not about his or her actual competence; −When determining the mutual assessment of experts' competence, confrontation between experts and their coalitions can be detected;
−Confrontation between experts necessarily distorts the actual competence of experts.
= (
The elements of vector (11) have the following meaning [21, 22]: information; problems are possible; personal relationships, etc. problem, weed out their erroneous judgments, etc. - a synergistic effect; −The risk of insufficient competence of the expert in the subject matter of the evaluation; −Antagonism between some experts; −Getting into the sphere of corporate, business or criminal interests; −Lack of specificity in the results of the examination; −Impossibility to track the consequences of the examination results; −The problem of organizing communication between experts within the sessions; −The ban on communication between experts ensures the independence of their opinions; −Sometimes, knowing the opinions of others, an expert can get deeper into the essence of the −Incomplete information during an examination in absentia prevents the collection of more −During an in-person examination in the form of a free discussion, social and psychological −Problems related to professional or official inequalities of commission members, their
The elements of vector (12) have the following meaning [23, 24]: = ( −The problem of dissidents' opinions: there is no need to unreasonably exclude from the expert commission or ignore the assessments of those whose opinions differ from the majority opinion - instead of an unqualified expert, you can exclude the specialist who has penetrated the essence of the problem the most;
−Part of the expert information is lost because the opinions of experts are not fully taken into account, as they may have several assessments that differ in the degree of confidence; −Shortcomings in the processing and analysis of data obtained during the expert survey, which do not take into account the level of significance of the characteristics of experts for each question in the questionnaire and do not cover the full range of expert opinions, which in aggregate reduces the effectiveness of the expert review; account; −The degree of confidence of the expert in each of the answer options is not taken into −Measurement methods and their different accuracy can have a significant impact on the result, sometimes leading the researcher to opposite conclusions.
The elements of vector (13) have the following meaning [25, 26]: procedures; committee can sometimes lead to biased expertise; −The problem of consistency of expert opinions is an important section in most expert −Attempts to ensure maximum consistency in the opinions of the members of the expert −In order to ensure consistency of opinions, the organizers of the examination deliberately select the "right" experts;
−The organizers of the examination should take into account the so-called group viewpoints, when experts are divided into groups that have different (but identical within the group) viewpoints; in the absence of a single point of view, which is also an important result; −There is no need to draw premature conclusions about the bias of the expertise: it may result −Take into account the mathematical and statistical features of methods for establishing the level of consistency of expert opinions; individual expert information; and interpreting them.
−Determination by consultants of the resulting expert opinion based on uncoordinated −The complexity of forming the final expert opinion: reconciling the data obtained, analyzing reconsider their point of view, even if it is wrong.
(14) (15)
The elements of vector (14) have the following meaning: conduct of the examination; −Experts are not sufficiently informed about the activities of units in related areas; −The company's corporate culture does not imply compromise in the interaction between −There are insufficient mathematical, psychological, and organizational tools to ensure the −The company's poor performance discipline does not allow for high-quality preparation and −An unbalanced employee incentive system significantly affects the quality of feedback; −Possible subjectivity of experts: experts may be captive to their perceptions and reluctant to
The elements of vector (15) have the following meaning: = ( −Low effectiveness of conclusions (results of examination); −Expert opinions are template-based: they do not take into account all aspects of the company's activities; the procedure in general and the generalized expert decision in particular; −Failure to fulfill one of the stages of expert evaluation leads to a decrease in confidence in −Experts are formal in their approach to explaining the motives for their participation in the review and arguing their position;
−The inability or unwillingness of consultants to provide illustrations and adequate interpretation of the results of the examination sometimes reduces the entire work done to nothing.
Stage 15 can be represented as a vector The elements of vector (16) have the following meaning: −Lack of forecasting based on the results of the analysis; −Lack of constructive recommendations for making the necessary decisions to improve performance; not binding and detached from the main business processes; evaluation emphasize its untimeliness and artificiality.
−Formal acceptance of the results of expert evaluation does not ensure its "legitimization"; −The perception of the final expert assessment by the majority of the company's divisions as −The defects and uncertainties that arise in the development of requirements for expert evaluation; executors;
The elements of vector (17) have the following meaning: −Insufficient will of the company's management to implement this solution; −There is insurmountable resistance and sabotage of the decision on the ground; −Insufficient training of local specialists to support the implementation of the solution; −There is no modern material base to support the decision - it is not timely; −Unrealistic estimate of the timeframe and budget based on the results of the expert −Unqualified preparation of a set of necessary measures to achieve the declared goal; −Unreasonable determination of the required amount of resources; −Artificial and unreasonable appointment of executors of relevant measures; −Illogical distribution of work, resources, and performers across objects, tasks, and deadlines; −Failure to conduct or unfair conduct of instructional and methodological activities with −Failure to provide timely and full assistance to contractors in case of difficulties. (16) (17) 5.16. Develop criteria for achieving the goal, which are indicators of the quality of decision implementation
The elements of vector (18) have the following meaning: −Formal attitude to the selection of criteria; −The criteria are not based on information available to the company; −Determining the values of the criteria requires a lot of labor; −The criteria are not indicators of the quality of the implementation of this particular expert assessment; −The developed criteria do not reflect the hierarchical structure of the company; −When developing and implementing the criteria, the areas of responsibility of the departments were not taken into account or they were violated intentionally or due to incompetence.
Stage 17 can be represented as a vector = ( , . . . , ) (18) (19) (20) The elements of vector (19) have the following meaning: −Incorrectly selected data that does not reflect the real state of solution implementation; −There are no technical and communication capabilities for monitoring; −Unscrupulousness of the personnel responsible for monitoring; −Lack of systems that support data aggregation; −Irregular monitoring and response to monitoring results; −Continuous changes in requirements, information links, unreasonable changes in the functions of departments, etc; evaluation results approved for implementation; implemented; −Lack of work on identifying the causes of deviations in the implementation of the expert −Lack of control over compliance with the main characteristics of the solution being −Insufficient or unfair control over compliance with the implementation deadlines; −Unprofessional and inconsistent monitoring of the status of the problem situation and the 5.18. Evaluation of the results of the implementation of a decision made on the process of implementing the results of the expert assessment.
basis of expert evaluation
The elements of vector (20) have the following meaning:
= ( , . . . , ) −Lack of evaluation of the results of the solution implementation; −Inadequate assessment of the effectiveness of the consequences of the decision; −Incorrect placement of emphasis on the results obtained and the responsibility of the participants in the implementation of the decision;
−Lack of motivation for the results of the implemented solution; costs of organizing and conducting the survey; expert evaluation; implement the expert evaluation; −The cost of an expert survey is unreasonably high, as it involves high expert salaries and the −Lack of periodic assessment of the actual effectiveness of decisions made on the basis of −The lack of a procedure for predicting the final effective period of the decisions taken to −Lack of regulations for determining the need to adjust actions or make a new decision; −Failure to anticipate the need to accumulate and systematize experience and develop algorithms for implementing standard decisions based on expert evaluation.
It is clear that each classification is schematic. However, modeling of processes, objects, and phenomena is a necessary element of research for the purpose of structuring and in-depth study.
It should be noted that potential problems, in particular, are or may become potential opportunities for manipulation. In particular, problems open the way to manipulating choices. Problems can arise at every stage of the application of expert technologies. And they, in turn, are a convenient environment for "formalized" manipulation. This is explained by the significant presence of subjective factors at all stages of expert evaluation. And also the illusion of the omnipotence of any mathematical methods.
There are many different definitions of the concept of manipulation that are used to study the problem of manipulation in various fields of science [27, 28]. problem among the compromise solutions of the form (1).
Choice manipulation is the facilitation of choosing an acceptable solution to the expert evaluation different ways. calculations.
In most cases, this is not deception, substitution or falsification of facts [29]. Manipulation consists in choosing such tools for formalizing the problem that will satisfy the wishes of some interested group of people in legitimately making the "right" decision [30, 31]. Moreover, in such situations, the use of mathematical methods, models, and human psychological characteristics to obtain a biased decision that is sufficiently justified to have signs of objectivity [32, 33].
We will introduce a number of heuristics that allow us to formalize the task of identifying manipulation opportunities, structuring them, and counteracting such attempts [34, 35].
Heuristics H1. Since problems arise at all stages of expert evaluation, they can be solved in Heuristics H2. Accordingly, the results of problem solving can be used in the interests of participants in the process, customers of the expertise or final beneficiaries.
Heuristics H3. All decisions can be justified and supported by appropriate mathematical Heuristics H4. It is possible to investigate the quality of the decision justification in the expert evaluation task and offer more reasonable solutions. That is, in this area, the secret can be made explicit 7. Methodological basis for countering manipulation in the use of expert technologies
To improve the quality of solving decision support tasks in complex poorly structured organizational systems, taking into account the subjective component, as well as to ensure the functioning of the system in conditions of constant change, we consider the methodological foundations of decision support. The methodology is applied at all stages of decision support in an organizational system and ensures obtaining reliable information from all participants in decisionmaking situations [30]. At the stage of obtaining expert information, decision makers have the opportunity to set their preferences in a convenient form in ordinal or cardinal scales in terms of the subject area [36].
The essence of the methodology developed by the author is to develop and implement approaches and ways to solve the problem of effective use of the subjective component of decision-making for the functioning of personnel management systems. We can offer a number of approaches that are the methodological basis for counteracting manipulation in the application of expert technologies. The methodology should be based on the following elements: ● Formalization of all stages of expert evaluation; ● Identification and recording of all heuristics used, taking into account the definition of heuristics and their manifestations: postulate, hypothesis, axiom, presumption, etc; ● Explicit presentation of heuristics that are used to solve a problem and, accordingly, significantly affect the solution; ● Application of soft computing, in particular, methods for constructing membership functions for a fuzzy set: layering; frequency of values; approximation by a triangular or trapezoidal membership function, etc; ● A group of methods for metricizing ordinal preferences on a set of alternatives has been developed that allow automatic transition to interval cardinal measurement scales; ● A group of adaptive methods for determining weighting coefficients has been developed; ● Unification of methods for determining the weighting of objects, parameters, criteria, expert competence, or the weight of information sources; ● The relationship between matrices of pairwise comparisons, rankings, and weighting coefficients in cardinal measurement scales is established; ● Study of the dynamics of decision-making by coalitions; ● Application of adaptive methods and rationale: why specific heuristics are used; ● Visualization, illustration, and interpretation of alternatives; ● Determining the distances from the ideal solution and alternative solutions from each other; ● Use of preferential ranking rather than simple selection; ● Using indirect methods instead of direct methods: taking into account the limited capabilities of people; ● Identification of intersections and limits of similarity between the criteria bundles that are used; ● Identify ideal rankings and calculate trade-offs between them.
By using subject matter models that are more aligned with expert preferences, the developed methodology will enable users to work in terms of subject matter and improve the credibility of the recommendations provided by the ODA.
This paper examines the stages involved in expert decision-making technologies, presenting a framework that outlines their sequence and interrelationships. It also investigates the challenges encountered during the application of these technologies and explores their underlying causes. Furthermore, the paper proposes a foundational methodology to counteract manipulation in expert evaluations. The proposed methodology aims to mitigate various forms of manipulation, both by identifying early indicators and analyzing the outcomes of decisions. It also offers recommendations for preventive measures to safeguard the decision-making process across all stages of preparation. Future research will refine and enhance this methodology, strengthening its effectiveness in addressing manipulation attempts.
References [1] Mosier, K. L. (1997). Myths of expert decision making and automated decision aids. In C. Zsambok & G. Klein (Eds.), Naturalistic decision making. NJ: Erlbaum. [21] Babenko, T., Hnatiienko, H., Vialkova, V. Modeling of the integrated quality assessment system of the information security management system / CEUR Workshop Proceedings, Volume 2845, 2021, Pages 75-84 // 7th International Conference ""Information Technology and Interactions"",
[22] Babenko, T., Hnatiienko, H., Vialkova, V. Modeling of information security system and automated assessment of the integrated quality of the impact of controls on the functional stability of the organizational system // Selected Papers of the XX International Scientific and Practical Conference "Information Technologies and Security" (ITS 2020), Kyiv, Ukraine, December 10, 2020 / CEUR Workshop Proceedings, 2021, 2859, pp. 188-198. [23] Szádoczki, Z., et.al. Incomplete pairwise comparison matrices based on graphs with average degree approximately 3. Annals of Operations Research, 2023, 326(2), pp. 783-807. DOI 10.1007/s10479-022-04819-9 [24] Mulesa, O., Kachmar, O., Myronyuk, I., Horvat, P., Radivilova, T., Kykyna, Y. (2022). Designing Semi-Automated Decision-Making Expert Systems for Healthcare Tasks. 2022 IEEE 17th International Conference on Computer Sciences and Information Technologies (CSIT), 235-238. doi: https://doi.org/10.1109/ csit56902.2022.10000448 [25] Tsyganok, V.V., Kadenko, S.V., Andriichuk, O.V. Simulation of expert judgments for testing the methods of information processing in decision-making support systems. Journal of Automation and Information Sciences, 2011, 43(12), pp. 21-32. DOI: 10.1615/JAutomatInfScien.v43.i12.30 [26] Tsyganok, V.V., Kadenko, S.V. On sufficiency of the consistency level of group ordinal estimates.
Journal of Automation and Information Sciences, 2010, 42(8), pp. 42-47. DOI: 10.1615/JAutomatInfScien.v42.i8.50 [27] Klenk, M., & Jongepier, F. (2022). Manipulation online: Charting the field. In M. Klenk & F.
Jongepier (Eds.), The philosophy of online manipulation (pp. 15-48). Routledge. [28] Heidler, D. S. & Heidler, J. T. (2018). The rise of Andrew Jackson: myth, manipulation, and the making of modern politics: Basic Books. [29] Hwang, T. (2019). Maneuver and manipulation: on the military strategy of online information warfare. Carlisle, PA: Strategic Studies Institute and U.S. Army War College Press. [30] Klenk, M. (2023). Algorithmic transparency and manipulation. Philosophy and Technology, 36, 1-20. https://doi.org/10.1007/ s13347-023-00678-9 [31] Maes, G., & Van Hootegem, G. (2019). A systems model of organizational change. Journal of
[32] Dodonov, A. et al.: Information Operations Recognition. From Nonlinear Analysis to Decision
[33] Palko, D. et al. Cyber Security Risk Modeling in Distributed Information Systems. Applied
Sciences (Switzerland) 2023, 13, 2393. https://doi.org/10.3390/app13042393 [34] Babenko, T., Hnatiienko, H., Bigdan, A. Model for determining the protection level of a complex system / CEUR Workshop Proceedings, Volume 3132, Pages 156 – 165, 2022 // 8th International Scientific Conference "Information Technology and Implementation", IT and I 2021, Kyiv, 1
[35] Voloshyn O.F., Mashchenko S.O. Decision-making models and methods: teaching. manual for students higher education closing / Voloshyn O.F., Mashcenko S.O. - 3rd ed. - K.: Lyudmila Publishing House, 2018. - 292 p. [36] Hrechko Viktoriia; Hrygorii Hnatienko;Tetiana Babenko. An intelligent model to assess information systems security level // 2021 Fifth World Conference on Smart Trends in Systems Security and Sustainability (WorldS4), London, United Kingdom, 29-30 July 2021/ Date Added to IEEE Xplore: 19 August 2021, Pp 128 – 133, DOI: 10.1109/WorldS451998.2021.9514019