Ways of Counteracting Possible Manipulations Within the AHP on The Base of Weighted Linear Equations

Abstract

The paper considers a problem of enhancing the quality of pairwise comparisons within the Analytic Hierarchy Process (AHP). A situation when an expert, who is accountable for providing judgments in the form of pairwise comparisons, is not a person of a good integrity. They want to boost up a certain alternative but don't want to claim its advantage explicitly. Then applying procedures for rectifying inconsistency definitely shall result in order violations, and the system of automated decision making should decide what to do with that. An approach based on weighted systems of linear algebraic equations is considered. Some ways of choosing appropriate weights for counteracting manipulations are suggested.

1. Introduction

Systems and tools for automated algorithmic-driven decision making are widely proliferated now.

A very significant place among the algorithms applied in these systems is taken by the Analytic

Hierarchy Process (AHP) suggested by T.Saaty [1-6 et al.]. AHP typically considers different connected levels of hierarchy, each of them involves estimations given by experts – usually in the form of pairwise comparisons between given alternatives. But there are many problems related to improving initial pairwise comparisons even if there is only one level of hierarchy.

Let there be n alternatives making a set

= { 1, … , }, and let M be a pairwise comparison matrix (PCM) provided by an expert. As a matter of fact, the matrix M represents some relation of preference.

= log , , = ̅1̅̅,̅̅ where is a chosen logarithm base. In this paper there is no reason for stipulating any specific values for

though this question might matter in some other contexts. What is really essential is that transforming the initial PCM to the logarithmic form allows to get additive pairwise comparisons [6].

2022 Copyright for this paper by its authors.

It is very helpful for analyzing consistency and for developing procedures for improving it, which is the main point of the paper.

The great problem is that initial PCMs directly provided by experts may be not of sufficient quality.

Classical concepts of cardinal and ordinal consistency are ubiquitous, a lot of different indicators of inconsistency as well as plenty of approaches for rectifying inconsistency have been suggested [1, 6, 13-24 et al.], many of them are frankly heuristic. Since Saaty had suggested his famous inconsistency threshold (numbering 0.1) that has been permanently debated and reviewed (one well-known recommendation is that this threshold should be reduced to 0.05). Anyway, such approaches may yield good results in normal situations but not be so good if things are not that normal. Many studies are focused on the problem of mere improving consistency which often is confused with the problem of quality, whereas those problems are not the same. First of all, for an arbitrary square positive matrix claimed to be a PCM (even if it is generated randomly and moreover even if it is not an inversesymmetric, in other terms reciprocal, matrix) we can easily construct the ideally consistent matrix with the same Perronian vector [e.g., 6]. If a procedure for rectifying inconsistency came to such a matrix with preserving the initial disorder, such a result obviously would not be good.

It appears very important to make plausible assumptions about possible sources of inconsistency. It

may result from various “benign” factors such as objective difficulty with estimating alternatives, lack of the experts’ awareness, errors caused by overlooks, inaccuracy, distraction etc. But there may be factors of another sort, which can be characterized as “malignant”. An expert, who is accountable for providing judgments in the form of pairwise comparisons, may be not a person of good integrity. Suppose they want to boost up a certain alternative which is doubted to be the best one, but don’t want to claim its advantage explicitly. Then, being aware of the algorithms for decision making implemented in the given system, they likely shall manipulate with pairwise comparisons to deceive the algorithms and force them to make an improper resolution. In addition to this, a board responsible for organizing the process of decision making may be not of sufficient integrity as well. They can, for example, involve dummy technical alternatives, impose irrelevant criteria etc. Such strategies are integrally related to situations of so-called order violations.

An order violation is a situation when ( ) < ( ) whereas ≻ . Order violations are

ultimately inevitable if the relation represented by a PCM M is non-transitive, but such situations may occur for transitive relations as well and do so even if the given PCM is comparatively consistent.

Many techniques for enhancing consistency, being applied to an initial PCM which is ordinally but

not of sufficient cardinal consistency and features an order violation, shall result in getting another PCM, which will be free of this flaw. A resulting PCM may be very consistent and sometimes be in a good accordance with the initial Perronian vector, this will be showcased below. Such a result may be good for “benign” situations (though may be not). But if there is a manipulation, it’s just a false improvement of the given PCM. It is exactly what the manipulator wanted, which is to let algorithms make a judgment desirable for the manipulator despite their tricky estimations seemingly contradicting to that. And then a blame of an improper decision may be put on designers of algorithms but not on the manipulator. Technically, in such a situation when an expert states in their PCM that A is better than B but wants the algorithms to decide that B is better than A, just an order violation, which is an intentional order violation, shall be an integral part and a main goal of the whole manipulation. If such a foul play really took place, and an order violation has been detected, the latter may be considered as a telltale sign of a manipulation – but that might be not that case, that could be merely an accidental mistake.

Basically, there may be an opportunity to consult other experts and to form an average PCM based on many estimations. But sometimes this may be impossible. In addition to this, experts’ opinions may be not independent, and/or they may be biased because of a common influence. So, we are still regarding the “pure” case when there is only one expert.

In this paper some ways to combining techniques for enhancing consistency of initial PCMs with counteracting possible manipulations resulting to order violations have been suggested and discussed.

2. Some examples of deliberate order violations

Firstly, we are going to use the standard Saaty scale. Let there actually be a competition between two alternatives: A1 and A2. 186

Example 1

The first example is a very basic and well-known one, but we will consider its parametrical form.

Suppose an expert wants to boost up the alternative A2, but they don’t want to state that explicitly. Then

they might try to manipulate by imposing a technical, obviously a worse alternative A3 and providing a pairwise comparison matrix (PCM) as follows: A1 gets a slight (quantified as 1) preference over A2 and A3, and A2 gets more significant preference 2 over A3. Let’s denote such a parametrized PCM as 0( 1, 2), then

The bad news for the manipulator is that within the standard Saaty scale A2 wins if only 1 = 2, 2 = 9. Then the PCM takes a view 0( 1, 2) = ( 0( 2, 9 )) = ( 0( 2, 9 )) ( 3 ) ≈ 0.2804 0.58

One can recognize a PCM having such a consistency ratio as an unacceptable one.

To conceal their manipulations, experts and organizers might try to impose some other base for decision making, for instance as in the following example.

Example 2

The number of technical alternatives might be increased. Let an overall number of alternatives be n,

among which only A1 and A2 be the real competitors, and the other n-2 alternatives be the technical ones. Like the previous example, A1 gets the slight preference 1 over A2 and the technical alternatives, A2 gets more significant preference 2 over the technical alternatives, and the other (technical) alternatives are on a par. Then the resulting parametrized PCM takes the view ′( , 1, 2) = (

The performed experiments show that such a PCM ensures a victory for A2 just already if = 6, 1 = 2, 2 = 3. In this case the Perronian vector approximately equals (0.2801, of scales stipulates that the following grade of preference is quantified times larger than the previous one, is a certain parameter). Then for defining preferences between i-th and j-th alternatives we can which is the distance between those alternatives in terms of grades. So, the Example 3 Let n=3, that is there are three alternatives, and the situation is like that in Example 1. Experiments carried out, for instance, in [25] show that the deliberate order violation is gained if 1 … ( , ) = ( − 12 12 1 … 23 … …) = ( −1 0 −1 1 0 −4 1 4) 0 or if 23 is larger than 4.

For = 1.2, the Perronian vector approximately equals (0.3682,

0.3912, 0.2406)

The index of consistency approximately equals 0.0296. Though applying such indices to transitive scales with different parameters is a special and not very clear issue, it should be the less the better anyway. The examples given above illustrate deliberate order violation means that what an expert wants to achieve is different from what they claim when constructing the PCM. Of course, it would be reasonable to ask the expert for explaining the situation. But the process of decision making can be very automated, and such a facility can be unavailable. Therefore, it appears important to develop algorithms and automated procedures aimed at detecting possible manipulations and counteracting them, or in other words, those robust to manipulations. To say it more accurately, detecting order violations is easy but counteracting is not.

3. A weighted system of linear equations

algebraic equations was considered in [25].

An approach to enhancing quality of pairwise comparisons based on solving systems of linear

Let M be a pairwise comparison matrix provided by an expert, and C is its logarithmic form

Then the regarded system can be written in the following form: where H and b are the matrix and the right side of the system of linear algebraic equations with

= respect to :

= , = ̅1̅̅,̅̅, = ̅̅+̅̅̅1̅̅, ̅̅ ∀ , > , > + − estimations or of trust to them.

are weighting coefficients which can be clearly interpreted as degrees of certainty about experts’ ( 1 ) ( 2 ) ( 3 )

Basically, this system follows the idea of applying the logarithmic least square method for getting more consistent PCMs [6, 26, 27]. It focuses on analysis of triads and on minimizing distances between what is needed for cardinal consistency and what is really provided.

The system (1-3) is over-determined, it contains ( − 1)/2 unknowns and

equations. Equations forming the group ( 2 ) were named expert equations because they are reflecting experts’ estimations. Equations of the group ( 3 ) were named consistency, or equidistant, equations, because they ensue from the requirements of cardinal consistency if those requirements are written in the logarithmic form. It’s easy to show that the system has a single (pseudo)solution which can be obtained with the help of the Moore-Penrose pseudo-inversion [28].

The obtained shall form the new PCM. Such a process can be performed iteratively. The issue of convergence within iterative improvements has been studied in [29, 30]. It was shown that under certain conditions the consequence of PCMs converges to the ideally consistent PCM with zero consistency index, and this can be confirmed experimentally. But, as it was mentioned before, it is necessary to control the process so that a risk of transmitting a disorder in the initial data to final ( −1) 2 + ( −1)( −2) 6 resolutions would be as low as possible.

As a tool for such a control we suggest using weighting coefficients for the equations in (1-3). The

idea of weighting sources of information has been discussed, for instance, in [31] but this idea can be implemented in different ways. And the main problem is how to pick out the coefficients .

Solving the unweighted system is not very helpful for counteracting intentional order violations. As

it was mentioned before, in this case that can in fact enhance consistency of the PCM but hardly its quality. Experiments carried out in [24] confirm that such enhancements, which can be done once or iteratively, eventually result in the consistent PCM with the changed directions of preferences. Whereas in the initial matrix we had 1 ≻ 2, in the resulting PCM we can get 2 ≻ 1, and this is exactly the arranged order violation.

Let’s illustrate this on the Example 3. Since a transitive scale is being used in that example, the

pairwise comparison matrix C is presented in the logarithmic form from the very beginning.

For = 1.2 its “classical” exponential form is

The unweighted system (2-3) for this PCM takes a view

which is the approximate limit of consequent PCMs.

Its consistency index practically equals 0. Its Perronian vector equals

So, according to ∗ 2 ≻ 1, and ( 2) > ( 1). There is no order violation now because it has been committed before, in the course of iterations. Yet, as we have mentioned before, this is not a satisfactory solution. So, such algorithms based on unweighted equations shall probably justify intentional order violations planned by experts who had manipulated. What algorithms of automated decision making should do is decide whether to accept an order violation resulting from their work or not. Let’s now look at the problem how to find proper values for the weighting coefficients . = ( 0.8333 0.8333

Its solution yields the following updated PCM (in the exponential form) After 10 iterations we come to the matrix

1 1 1 1 ∗ = ( 1.0627 0.6535 manipulations by changing weights of

Surely, algorithm-based counteracting manipulations can’t be determined and straightforward even

after an order violation has been detected. A system of decision making should have a set of rules aimed at considering a wide range of factors and principles, and this makes the problem rather complicated and intricated. We are going to discuss some heuristic rules of such a sort and look at how these rules may come in use for getting more or less appropriate coefficients in ( 1-3 ).

Hardly there is any reason for changing weights of consistency equations but changing those for

expert equations appears promising. In this paper we are regarding the following heuristic rules:  pairwise comparisons have a priority  considering measures of inconsistency  analyzing strongly connected components

Let’s illustrate these rules one by one.

Pairwise comparisons have a priority

Meaningfully this rule has a following interpretation: if an expert explicitly stated in PCM that for

the alternatives A1, A2 1 ≻ 2, and there is no additional opportunity to consult them, then all transformations must remain this advantage and the relation ( 1) > ( 2) should eventually hold.

Unfortunately, following this rule may be problematic if there are multiple order violations or the given

PCM is ordinally inconsistent at all. Technically, it surely can be applied for a single order violation.

But this rule is too strict and can potentially cause problems if the detected order violation was an accidental error but not a deliberately planned result. Certainly, more attention should be paid to order violations between alternatives competing for an overall victory.

We are going to illustrate this rule on the Example 3. Let’s entrench the preference 1 ≻ 2 by

taking weighting coefficients = (1.5, 1, 1, 1).

The iterative process described above leads to the consistent PCM

1 1.051 1.6171 ∗ = ( 0.9515 1 1.5487)

0.6184 0.6499 1 with the Perronian vector (0.3891,

= ( − (( )))2 are the largest.

Let’s build the matrix ( ) = ( , , = ̅1̅,̅̅̅̅) for the Example 3. It approximately equals with the Perronian vector and with eliminated order violation.

But the regarded case was a very simple one, and the source of the manipulation has been distinctly indicated by the criteria of maximum deviation. In more tricky cases the situation may be more complicated.

Example 4 corresponding PCM equals (here the standard Saaty scale is applied)

As a matter of fact, it is the Example 2 with the parameters = 6, 1 = 2, 2 = 3. The for the pair ( 1, 2 ), which reflects the preference of 1 over 2. In such a situation decreasing weighting coefficient for the corresponding equation (for instance, by putting it to 0.75 whereas the other coefficients remain to equal 1) seems not to be helpful. The resulting Perronian vector approximately equals (0.2531, 0.2983, 0.1122, 0.1122, 0.1122, which means that the advantage of the second alternative has been even increased.

In this case the manipulation was more concealed and less concentrated. Instead, increasing it

according to the first heuristic rule (pairwise comparisons have a priority) yields much better results. For example, putting the coefficient to 1.25 yields the Perronian vector (0.2859, 0.2648, 0.1123, 0.1123, 0.1123, 0.1123) and the first alternative wins.

Analyzing strongly connected components

Basically, picking out and analyzing strongly connected components (SCC) in the preference graph

related to the given PCM appears to be quite useful if the ordinary consistency doesn’t hold, that is the initial relation of preference is non-transitive. The idea is to build separate PCMs for each SCC and then to combine them with a specially constructed PCM connecting these SCCs.

Realizations of such an idea can be very different. For example, the following heuristic approach

might be applied: for separate PCM ( ) within the k-th SCC denoted by preferences could be got directly from the initial PCM M. But for making the procedure more flexible and adjustable we suggest that these coefficients should rather be calculated by the formula

( ) = 1, 1 ≤ 1 is a smoothing coefficient designed to reduce a scatter of values within one SCC. Preferences between SCCs are calculated by averaging with the additional treating. More technically, given the initial PCM M, the PCM ( ) for combining SCCs can be calculated by the following formula: ( ) =

2 ∑( , )∈ | | = {( , ): ∈ , ∈ } where and are the k-th and the l-th SCCs, 2 ≥ 1 is a sharpening coefficient designed to make difference between compared SCCs more distinct, and coefficients stand for relations between and .

Then the value of the -th alternative can be calculated as follows:

where ( )should be obtained from the comparison matrix within the SCC , and ( ) is a value ascribed to on the base of inter-CSS PCM ( ). Then the obtained vector of values should be normalized so that the sum of its components would equal 1.

In addition to this, we consider an extended preference graph which contains additional relations

resulting from order violations. Let’s try to apply this technique to the Example 4. The initial PCM is ordinally consistent. Formally, this means that each alternative constitutes a separate SCC with a single element. Since we have an order violation in the pair ( 1, 2 ), the arc ( 2,1 ) should be added to the extended graph. So, we have SCCs in the extended graph as follows:

1 = {1,2}, 2 = {3}, 3 = {4}, 4 = {5}, 5 = {6}

Comparisons within each SCC but 1 are trivial and yield

( 5 )

It can be shown that taking 1 = 2 = 1 leads to not very good results. Let’s take 1 = 0.5, 2 = 1.5. Then for 1 we get the PCM 1( ) = 1, = ̅2̅̅,̅5̅ with the Perronian vector

(0.5858, 0.4142) Then we obtain the inter-CSS pairwise comparison matrix with the Perronian vector

(0.4970, 0.1257, 0.1257, 0.1257, 0.1257)

Finally, calculating by (5) and normalizing the resulting vector yields the following distribution of values among the alternatives:

(0.2912, 0.2059, 0.1257,

The order violation has been eliminated.

5. Conclusions and discussion

In this paper the main attention is paid to possible manipulations with pairwise comparison matrices within the AHP-based decision making which can be deliberately committed by experts accountable for forming such judgments. The question is that a manipulator may want to boost up a certain alternative, but they don’t want to be accused of non-integrity and to declare an advantage of that alternative explicitly. Then the manipulator would like to make up tricky PCMs, and what they want to achieve by doing that is force algorithms of decision making to reverse some of their judgments. This is the pure order violation, and it’s the deliberate and planned one. Therefore, if a manipulation really takes place, it is typically accompanied with order violations. So, if an order violation is detected, it should be considered as a signal of possible manipulation though the violation might be caused by other, more benign reasons like overlooks, inaccuracy etc.

So, AHP-based algorithms for decision making should become more robust to possible

manipulations and to acquire some techniques of tackling order violations with the aim of counteracting such manipulations. The strategy of dealing with order violations can’t be easy and straightforward, it should be based on a set of parametrized rules and involve intelligent combining such rules.

An approach to enhancing consistency of pairwise comparisons on the base of iterative solving weighted systems of linear equations is being developed in the paper. An integral part of the suggested approach is to combine enhancing consistency itself with deciding what to do with detected order violations. Some rules aimed at picking out weights of these equations are regarded and illustrated in the paper. These rules are quite simple, other approaches certainly must exist. For instance, it seems promising to apply different techniques of reinforcement learning. Typically, the AHP-based decision making is more complicated than it was presented in the paper. It usually comprises some levels of hierarchy, at least one of them is related to criteria which possible decisions depend on. There is a vast room for foul plays with these criteria, and the issue how to make algorithms of automated decision making more robust deserves special research. It appears promising to combine the approach presented in this paper with the multi-level model “state-probability of choice” for multiagent decision making [32]. A game view on the problem is worth to be developed. This means considering a game in which a manipulator can try different strategies of achieving their goals, and a system of decision making should implement strategies of detecting manipulations and counteracting them.

Such considerations appear to be useful for many practical applications, such as financial privacy of the telecommunications space [33] or prioritizing cybersecurity measures using incomplete data [34].

6. References

References

[1] T.L. Saaty , The Analytic Hierarchy Process, McGraw-Hill , New York, 1980 .

[2] M. Brunelli , Introduction to the Analytic Hierarchy Process , Springer, Cham, 2015 .

[3] O.S. Vaidya , S. Kumar , Analytic hierarchy process: An overview of applications , European Journal of Operational Research 169-1 ( 2006 ) 1 - 29 .

[4] A. Ishizaka , A. Labib . Review of the main developments in the analytic hierarchy process . Expert Syst. Appl . 38 ( 2011 ) 14336 - 14345 .

[5] W. Ho , Integrated analytic hierarchy process and its applications. A literature review , European Journal of Operational Research 186-1 ( 2008 ) 211 - 228 .

[6] W. Koczkodaj , L. Mikhailov , G.Redlarski, M. Soltys , J. Szybowski , G. Tamazian , E.Wajch, Kevin Kam Fung Yuen, Important Facts and Observations about Pairwise Comparisons, Fundamenta Informaticae 144 ( 2016 ) 1 - 17 . doi: 10 .3233/FI-2016-1336

[7] E. Choo , W. Wedley , A Common Framework for Deriving Preference Values from Pairwise Comparison Matrices , Comput. Oper. Res. 31-6 ( 2004 ) 893 - 908 .

[8] E. Choo , W. Wedley, Estimating ratio scale values when units are unspecified , Computers and Industrial Engineering 59 ( 2010 ) 200 - 208 . doi: 10 .1016/j.cie. 2010 . 04 .001.

[9] V. Tsyganok , S. Kadenko , O. Andriichuk , Usage of Scales with Different Number of Grades for Pair Comparisons in Decision Support Systems , International Journal of the Analytic Hierarchy Process 8 -1 ( 2016 ) 112 - 130 . doi: 10 .13033/ijahp.v8i1. 259 .

[10] R. Janicki , Y. Zhai , On a pairwise comparison-based consistent non-numerical ranking , Logic Journal of IGPL 20-4 ( 2012 ) 667 - 676

[11] B. Cavallo , L. D'Apuzzo , A general unified framework for pairwise comparison matrices in multicriterial methods , Int. J. Intell. Syst . 24 ( 2009 ) 377 - 398 .

[12] R.E. Jensen , An alternative scaling method for priorities in hierarchical structures , Journal of Mathematical Psychology , 28 - 3 ( 1984 ) 317 - 332

[13] W. Koczkodaj , S. Szarek , On distance-based inconsistency reduction algorithms for pairwise comparisons , Logic Journal of IGPL 18-6 ( 2010 ) 859 - 869

[14] O. Yu , Assessing and Improving Consistency of a Pairwise Comparison Matrix , in: Analytic Hierarchy Process. Portland International Conference on Management of Engineering and Technology ( 2017 ), pp. 1 - 6 . doi: 10 .23919/PICMET. 2017 . 8125304 .

[15] M.M. Potomkin , M.V. Nikolaienko , D.I.Grazion , Improvement of Analytic Hierarchy Process based on the Refinement of the Procedures for the Formation of Pairwise Comparison Matrices , Cybern Syst Anal 56 ( 2020 ) 603 - 610 . doi: https://doi.org/10.1007/s10559-020-00277-y

[16] L. Mikhailov , S. Siraj , Improving the Ordinal Consistency of Pairwise Comparison Matrices , in: Proceedings of the XI International Symposium for the Analytic Hierarchy Process ISAHP-201 , 2011 . doi: 10 .13033/isahp.y2011. 128 .

[17] S. Siraj , L. Mikhailov , J. Keane , A heuristic method to rectify intransitive judgement in pairwise comparison matrices , European Journal of Operational Research 216 ( 2012 ) 420 - 428 .

[18] J. Barzilai , Consistency measures for pairwise comparison matrices , J. Multi-Criteria Decis. Anal . ( 1998 ) 123 - 132 .

[19] W. Koczkodaj , M. Orłowski , Computing a consistent approximation to a generalized pairwise comparisons matrix , Computers & Mathematics with Applications , 37 - 3 ( 1999 ) 79 - 85

[20] Y. Iida , Ordinality Consistency Test About Items and Notation of a Pairwise Comparison Matrix , in: AHP. Proceedings of the X International Symposium for the Analytic Hierarchy Process , 2009

[21] M. Brunelli , Recent Advances on Inconsistency Indices for Pairwise Comparisons, A Commentary, Fundam. Inform 144 ( 2016 ) 321 - 332

[22] W. Koczkodaj , R. Szwarc , On axiomatization of inconsistency indicators for pairwise comparisons , Fundam. Inform 132 ( 2014 ) 485 - 500

[23] A.Z. Grzybowski , New results on inconsistency indices and their relationship with the quality of priority vector estimation , Expert Syst. Appl . 43 ( 2016 ) 197 - 212

[24] R. Ramanathan , L.Ganesh, Group Preference Aggregation Methods Employed in AHP : An Evaluation and Intrinsic Process for Deriving Members' , Weightages European Journal of Operational Research 79 ( 1994 ) 249 - 265

[25] O. Oletsky , On Constructing Adjustable Procedures for Enhancing Consistency of Pairwise Comparisons on the Base of Linear Equations , in: CEUR Workshop Proceedings , 3106 ( 2021 ) pp. 177 - 185 .

[26] S. Bozóki , V. Tsyganok , The (logarithmic) least-squares optimality of the arithmetic (geometric) mean of weight vectors calculated from all spanning trees for incomplete additive (multiplicative) pairwise comparison matrices , International Journal of General Systems 48-4 ( 2019 ) 362 - 381 . doi: 10 .1080/03081079. 2019 .1585432

[27] L. Csato , A characterization of the Logarithmic Least Squares Method , European Journal of Operational Research , 276 -1 - ( 2019 ) 212 - 216 .

[28] G.H. Golub , C.F. Van Loan , Matrix computations (3rd ed.). Baltimore: Johns Hopkins , 1996

[29] W. Holsztynski , W.Koczkodaj, Convergence of Inconsistency Algorithms for the Pairwise Comparisons , IPL. 59-4 ( 1996 ) 197 - 202 .

[30] W. Koczkodaj , J. Szybowski, The limit of inconsistency reduction in pairwise comparisons , Int. J. Appl. Math. Comput. Sci . 26 ( 2016 ) 721 - 729

[31] S. Kadenko , Defining relative weights of data sources during aggregation of pair-wise comparisons , in: http://ceur-ws. org/ Vol-2067/paper7.pdf

[32] E. Ivokhin , O. Oletsky , Restructuring of the Model “State-Probability of Choice” Based on Products of Stochastic Rectangular Matrices, Cybern Syst Anal . Vol. 58 - 2 ( 2022 ) 242 - 250 . https://doi.org/10.1007/s10559-022-00456-z

[33] V. Kraevsky , O. Kostenko , O. Kalivoshko , N. Kiktev , I. Lyutyy , Financial Infrastructure of Telecommunication Space: Accounting Information Attributive of Syntalytical Submission , in: 2019 IEEE International Scientific-Practical Conference Problems of Infocommunications, Science and Technology (PIC S&T) , Kyiv, Ukraine, 2019 , pp. 873 - 876 , doi: 10.1109/PICST47496. 2019 .9061494

[34] H. Hnatiienko , N. Kiktev , T. Babenko , A. Desiatko , L. Myrutenko, Prioritizing Cybersecurity Measures with Decision Support Methods Using Incomplete Data, in: Selected Papers of the XXI International Scientific and Practical Conference "Information Technologies and Security" (ITS 2021 ), Kyiv, Ukraine, December 9, 2021 , pp. 169 - 180 . https://ceur-ws. org/ Vol- 3241 /paper16.pdf