On Constructing Adjustable Procedures for Enhancing Consistency of Pairwise Comparisons on the Base of Linear Equations

Abstract

The flexible and adjustable procedure aimed at improving consistency of pairwise comparison matrices by transforming initial matrices obtained from experts is suggested. Inconsistencies may arise because of unintentional mistakes and unawareness of experts, as well as because of deliberate manipulations and fraud. This procedure is based on solving a system of linear equations, one group of which corresponds to the experts' judgments, and the other group reflects the requirements of consistency. For constructing such equations, transitive scales of preferences between alternatives are used. For taking into account possible unreliability of the experts' judgments, different weights can be assigned to different equations of the system. It is shown in the paper that the suggested procedure can be either performed one-time or recurred iteratively. An effect of uncontrolled changes in preference directions of pairwise comparisons in the course of such iterations was detected. Some experiments are described. Much attention within these experiments is paid to counteracting inconsistencies caused by possible manipulations. fraud detection Analytic Hierarchy Process, pairwise comparisons, consistency, systems of linear equations,

1. Introduction. Methods and tools

It is commonly recognized that one of the main problems related to the Analytic Hierarchy Process (AHP) [1-5] is how to ensure satisfactory consistency of pairwise comparisons within it. Possible inconsistency may be caused both by inevitable difficulties experts encounter on stating their judgments about the alternatives and by deliberate manipulations which may be done by some experts. An example of such a manipulation will be shown below.

There are some known approaches related to possible ways of detecting inconsistencies in pairwise comparison matrices and of transforming these matrices so that they would become more consistent [6-11]. It’s worth mentioning that automated changing of pairwise comparison matrices may entail significant contradictions with the experts’ estimates and therefore reduce experts’ motivation and overall confidence in results of the whole process. On the other hand, inconsistencies may result from incompetency, manipulations and fraud.

Therefore, procedures for automated detecting inconsistencies in pairwise comparison matrices and especially for transforming these matrices should be combined with repeated queries to experts for changing their initial estimates. There is still a need of developing effective techniques aimed at mitigating inconsistencies in the initial comparison matrices provided by experts and maybe at detecting possible incompetence, fraud and manipulations.

2021 Copyright for this paper by its authors.

An approach which is being developed in this paper is based on forming an explicit system of linear algebraic equations. Some of these equations correspond to the experts’ opinions, and the others reflect desired features of consistency.

For specifying pairwise comparison matrices and then for getting such equations, so-called transitive scales of comparisons are applied.

Some weight coefficients reflecting reliability degrees of these equations will be introduced. This allows to take into account information about experts and to ensure combined adjustable reasoning based of their judgments.

Some numerical examples illustrating the approach are described.

2. Transitive scales

The main idea of applying transitive scales [11] to pairwise comparisons is as follows. If we introduce the parameter  1, which determines how many times the value of one grade of preference scale is larger than the other, then the minimal gradation of preference should be evaluated as  , the next gradation makes  2 etc.

Transitive scales are the important type of scales that might be applied for building pairwise comparison matrices within the AHP. Speaking more generally, there is a number of studies aimed at exploring different types of scales and unifying them [6, 11]. Though transitive scales are not very widely used in AHP, they have many positive features. We are going to use these scales in order to get a straightforward way to making initial pairwise comparison matrices more consistent and reliable by means of constructing a system of linear equations.

If transitive scales are actually applied for getting pairwise comparison matrices, experts have to estimate amounts of gradations distancing an importance of each alternative from that of all others. A real comparison matrix is unequivocally determined by the degrees of preference between specific alternatives and by the value of  . More technically, if the evaluated degree of preference of the i-th alternative over the j-th one equals C  cij , then the corresponding element of the pairwise comparison matrix shall be calculated as

And vice versa

aij  C cij  log aij

One of the most important reasons for using transitive scales for different practical applications is that a spread between the maximal and the minimal values of importance can be too large and unnatural, and then the proper choice of  allows to reduce it. The less is the parameter  , the less is the spread. An example of using such scales and a discussion about this matter can be found in [12].

Another reason for using transitive scales is that they facilitate getting explicit and simple linear equations for mitigating inconsistencies in pairwise comparisons. Now we are going to illustrate this.

3. A simple example of manipulations

Let’s consider the following situation. There are three alternatives 1, 2, 3. An expert, who is not of sufficient integrity, wants to promote the alternative 2 and to ensure its win, but they don’t want to be accused of dishonesty because of making the direct statement that 2 is better than 1. Then they may postulate levels of preferences across the alternatives as follows:

Taking into account ( 1 )-( 2 ) and the fact that a pairwise comparison matrix should be inversedsymmetric, we get the following matrix:    1 M ( 1 )   1   1    1 1 pairwise comparison matrix. For the given matrix M ( 1 ) its normalized main eigenvector equals

So even though the pairwise comparisons indicate an advantage of 1 over 2, the alternative 2 gets an overall win.

However, the actual value of  doesn’t matter significantly in this particular context, which was confirmed experimentally. For example, taking   2 gives the following distribution of importance: so the winner doesn’t change.

But what is a genuine nature of such a situation? Technically, the pairwise comparison matrix itself represents a transitive relation between alternatives but there is an inconsistency of the other sort in this example. Let ai be an i-th alternative, and v  (v1,..., vn ) be a normalized main eigenvector of the comparison matrix. As the vector v represents the distribution of importance values among the alternatives, and v(i) vi represents an importance value of the i-th alternative ai , it is considered to be desirable that the following consistency relation should be true: ai a j  v(i)  v( j)

But this relation doesn’t hold for the example under consideration.

4. Equations for enhancing consistency

The common requirements related to consistency have been formulated in [1, 11-18 etc.]. The main of them is the following one: i, j, k : aij  aik  akj i, j, k : cij  cik  ckj

By using ( 1 )-( 2 ) and by taking logarithm of ( 3 ) we can perform a linearization and get some linear relations, which shall describe requirements for consistency in a linear form. In addition to this those relations should explicitly involve levels of preferences: ( 4 )

For a consistent pairwise comparison matrix, relations like ( 4 ) should be satisfied for all of its elements. But on the other hand, experts are posing specific values of elements reflecting their opinions about preferences across the alternatives. Combining all these considerations together, we come to the following explicit system of linear equations with respect to new improved estimates xij , given the expert estimates cij : xij  cij i, j, k xik  xkj  xij  0 ( 3 ) ( 5 )

The first group of equations in ( 5 ) represents the experts’ judgments, and the other group reflects the requirements of consistency. Let’s look at the structure of ( 5 ) more carefully. It is as follows:     b   ...x   1  ,    ...

... where

 c12  b   ...  ,    cn1,n   1 0 0      ... ... ...

 0 0 1  so  is the unit matrix.

It means that the first group of equations itself forms a diagonal unit matrix with an evident solution, therefore adding any extra equation to it inevitably makes the entire system ( 5 ) redundant. Typically this system shall be inconsistent, i.e. it is probably going to have no solutions. But in this case, we can try to find a pseudo-solution by means of Moore-Penrose pseudo-inversion [19].

It is well-known that the techniques of the Moore-Penrose pseudo-inversion are closely related to the least square method. Use of the least square method and of its logarithmic variation for analyzing pairwise comparison matrices was discussed, for example, in [11, 20, 21].

After solving ( 5 ) we can build the improved pairwise comparison matrix on the base of the obtained solution of the system.

5. A simple numerical example

Let’s illustrate the techniques described in the previous section on a very simple numerical example. Let’s take preference levels as follows: c12  1, c13  1, c23  1 and let   1.2.

This produces the pairwise comparison matrix    1  M ( 2 )   1   1   1  1         1 

 x13  1 Its index of consistency approximately equals 0.0018.

By using ( 5 ) we can get the following system of linear equations x12  1 ( 6 )

This system is obviously inconsistent and has no solutions. By applying Moore-Penrose pseudoinversion we can obtain the pseudo-solution of ( 6 ): x12  x23  x13  0 Its index of consistency has been really reduced and now approximately equals 0.00012.

6. Adjusting the procedure

The described procedure of “improving” pairwise comparisons and of making them more consistent can be made more flexible. We can regard not an entire system ( 7 ) but some subsets of its equations instead. By doing so, we can adjust the procedure and change possible solutions in a desired direction.

Let’s come back to the previous example. If we want the resulting matrix to become consistent, we can reject some experts’ judgments. For example, in ( 6 ) we can leave out the equation x13  1 For such a reduced system we will get the solution x12  1., x13  2., x23  1.

which meets the requirement ( 7 ).

7. Weighting equations

In addition to this, we may not limit adjustment facilities to leaving out some equations of ( 5 ) only. We can consider equations as those of different importance, and according to this consideration different weight coefficients can be assigned to different equations. The idea of increasing importance of more reliable judgments by assigning larger weight coefficients to them is more or less known, it has been discussed in a number of papers, for instance in [22, 23]. But this idea can be implemented in different ways.

Generally speaking, the weighted least squares method can be used for solving tasks of such a sort. But first of all we are going to illustrate a more simple approach, which involves replacing two conflict equations with their convex combinations. This means that from two equations (w1, x)  b1 and (w2, x)  b2 we can move on to the single equation ( 7 ) Here (,) stands for a dot product, and  [0.1] .

Let’s illustrate this approach.

8. Counteracting manipulations

Let’s come back to the example considered in the Section 3. As it was mentioned before, an expert is trying to promote the alternative 2 by means of posing irrelevant pairwise comparisons. The preferences are and the system ( 5 ) in this case takes the form (q1) : x13  1 (q ) :x12  (1  )x23  4  3 (q4 ) : x12  x23  x13  0   2 3 0.4474

If a judgment seems to be not very reliable, its weight can be reduced. So, if q1 is considered to be more reliable than q3 , its weight should be bigger than that of q3 . Then we may put, for example, . In this case we shall get the distribution among alternatives as follows: (q1) : x12  1 (q2 ) : x13  1 (q3 ) : x23  4 (q4 ) : x12  x23  x13  0 x11  0., x13  2., x23  3. 0.3682 0.3912

0.2406 x13  (1 )x23   4(1 ), 0   1 or after performing some trivial transformations x13  (1  )x23  4  3 Replacing q2 and q3 with ( 9 ) leads to the following system: Its pseudo-solution, which can be obtained by Moore-Penrose pseudo-inversion, is as follows: If we get the corresponding pairwise comparison matrix for   1.2, its normalized main eigenvector shall equal

This means that the alternative 2 retains its advantage.

Let’s try introducing weight coefficients for different equations. We are going to use a strategy of replacing conflict pairs of equations with their convex combinations. In our case we have such a conflict pair formed by equations q2 and q3 . Meaningfully, these equations correspond to two contradictory judgments. First of them means that the alternative 1 has a slight advantage over the alternative 2, and the second one means that the alternative 2 has a significant advantage over the alternative 3. The convex combination of q2 and q3 in accordance with ( 8 ) shall be written as ( 8 ) ( 9 )

0.1539 given  

, the distribution of importance among alternatives is as follows: On the contrary, decreasing  leads to more distinct supremacy of the alternative 2. For example, 1

Obviously, the issue of specifying weight coefficients is a very important issue, but we don’t consider it in details in this paper. It is closely related to the issues of trust and reliability of judgments [24]. In simple cases like given above the contradictions can be detected by a straightforward preliminary analysis. Really, if an expert postulates that the alternative 1 is better than the alternative 2, he should not estimate the preference of 2 over 3 much more than of 1 over 3, and therefore the weight of such a judgment and consequently that of the corresponding equation q3 can be reduced. An external expertize might be helpful. But the whole issue needs a special study.

9. Iterative improvement of consistency

The process of enhancing consistency of pairwise comparisons can be made iterative. Really, after getting an improved matrix with a better index of consistency we can apply the same procedure to the next matrix and so on. For illustrating this process, we started with the matrix M ( 1 ) from the Section 1. After 6 iterations had been performed, we got a matrix  1  M *    0.33  2.33  with a practically zero index of consistency.

Given   1.2, the normalized main eigenvector of this matrix is invariable. But there is another interesting fact worth paying attention to. The resulting matrix M * is much more consistent than the initial one. But now the alternative 2 has an advantage even in pairwise comparisons.

More interesting situations may arise if the initial pairwise comparison matrix is initially nontransitive. For example, such a matrix might contain a cycle of preferences like the following: a1  a2  a3  a4  a1 or in a more complicated case a1  a2  a3  a4  a2

and a2  a6

The principal difference between these situations is that there is only a single strongly connected component in the first case, and there are three of them in the second case. In that case it appears reasonable to perform a decomposition, namely to build separate pairwise comparison matrices for each strongly connected component, then to solve corresponding equations aimed at improving their consistency if needed, and finally to combine the obtained results, maybe on the basis of a pairwise comparison matrix which represent preferences between separate components. But issues related to non-transitivity need to be specially explored. 10.Conclusions and discussion

A way for improving consistency of pairwise comparison matrices within the Analytic Hierarchy Process is suggested in the paper. It is suggested that raw matrices of pairwise comparisons should undergo some transformations in order to replace them with other matrices having better consistency than the original ones.

The main idea of the suggested approach is to construct a system of linear algebraic equations, and elements of the transformed matrix can be obtained from the solutions of this system. For building the system of equations, using transitive scales with a parameter, which determines how many times the value of one grade of preference scale is larger than the other, is suggested.

Typically a system of linear equations constructed within this approach shall be inconsistent, but we can obtain its pseudo-solution by means of Moore-Penrose pseudo-inversion.

As a summary, the suggested scheme of transformation aimed at enhancing consistency of pairwise comparisons can be represented as the sequence of the following steps:  getting the system of linear algebraic equations based on the initial comparison matrix;  solving the system and getting preferences across the alternatives; this step can be performed once or be recurred iteratively;  getting the final pairwise comparison matrix.

The procedure appears to be quite flexible and adjustable. Firstly, we can use not all possible equations, but a selected subset of them. Secondly, as long as different experts’ judgments and other requirements may have different degrees of importance and reliability, we may introduce weights for corresponding equations. It would be possible to use the weighted least squares method, but in the paper another approach, which is based on convex combinations of equations, is illustrated.

Numerical experiments described in the paper showcase different aspects of regarded issues.

The other aspect is the following. Inconsistencies in comparison matrices often may be caused by inevitable mistakes, but they may be also related to unawareness, low competence and even to nonintegrity of experts. In these cases, initial inconsistencies may not be mitigated by the described iterative process. Moreover, it can even aggravate these inconsistencies.

Some experiments delivered above illustrate how these factors can be mitigated by assigning different weights to different equations. Generally speaking, it is reasonable to diminish weights of judgments and of corresponding equations which are not reliable enough and to reduce influence of these judgments by doing so.

A wider context may be related to the problem of automated building rankings among experts and their judgments. This might be of great importance, e.g. for addressing the problem of incompetence or that of a fraud detecting. If such an intelligent automated system detects suspicious or improper judgments, it should at least diminish estimated degrees of their reliability and thereby the weights of corresponding equations, or exclude these judgments and corresponding equations at all. Especially this refers to situations when such irrelevant judgments affect rankings of alternatives likewise it happened with the matrix described in the sections 3 and 8, and this can be detected by the system.

Automated changing of pairwise comparison matrices may reduce experts’ motivation and overall confidence in results of the whole process. In order to avoid this, it may be useful to consult experts so they may be requested to provide necessary clarifications and explanations, or maybe they can change their judgments. Developing new techniques of implementing iterative procedures for refining pairwise comparisons combined with prompt experts’ and operators’ interventions appears to be helpful for many industrial applications (e.g. [25]), especially for those involving conveyor architectures and a control over emerging and renewable units. 11.Acknowledgements

We are grateful to the anonymous reviewers for their helpful suggestions. 12.References

References

[1] T.L. Saaty. The Analytic Hierarchy Process. McGraw-Hill , New York, 1980 .

[2] M. Brunelli . Introduction to the Analytic Hierarchy Process , Springer, Cham, 2015 .

[3] O.S. Vaidya , S. Kumar . Analytic hierarchy process: An overview of applications . European Journal of Operational Research 169 ( 1 ) ( 2006 ), pp. 1 - 29 .

[4] A. Ishizaka , A. Labib . Review of the main developments in the analytic hierarchy process . Expert Syst. Appl . 2011 , 38 , pp. 14336 - 14345 .

[5] W. Ho . Integrated analytic hierarchy process and its applications. A literature review . European Journal of Operational Research 186 ( 1 ) ( 2008 ), pp. 211 - 228 .

[6] V.V. Tsyganok , S.V. Kadenko , O.V. Andriichuk . Usage of Scales with Different Number of Grades for Pair Comparisons in Decision Support Systems . International Journal of the Analytic Hierarchy Process . Vol. 8 ( 1 ) ( 2016 ), pp. 112 - 130 . doi: 10 .13033/ijahp.v8i1. 259

[7] W. Koczkodaj , J. Szybowski. The limit of inconsistency reduction in pairwise comparisons . Int. J. Appl. Math. Comput. Sci . 2016 , 26 , pp. 721 - 729 .

[8] B. Cavallo , L. D'Apuzzo . A general unified framework for pairwise comparison matrices in multicriterial methods . Int. J. Intell. Syst . 2009 , 24 , pp. 377 - 398 .

[9] Iida , Y. Ordinality Consistency Test About Items and Notation of a Pairwise Comparison Matrix, in AHP . Proceedings of the X International Symposium for the Analytic Hierarchy Process , 2009 .

[10] L. Mikhailov , S. Siraj . Improving the Ordinal Consistency of Pairwise Comparison Matrices, in Proceedings of the XI International Symposium for the Analytic Hierarchy Process ISAHP- 2011 ( 2011 ). doi: 10 .13033/isahp.y2011. 128

[11] E. Choo , W. Wedley . A Common Framework for Deriving Preference Values from Pairwise Comparison Matrices . Comput. Oper. Res . Vol. 31 ( 6 ) ( 2004 ), pp. 893 - 908 .

[12] O.V. Oletsky , E.V. Ivohin . Formalizing the Procedure for the Formation of a Dynamic Equilibrium of Alternatives in a Multi-Agent Environment in Decision-Making by Majority of Votes . Cybern Syst Anal Vol. 57 ( 2021 ), pp. 47 - 56 . doi: https://doi.org/10.1007/s10559-021-00328-y.

[13] O. Yu . Assessing and Improving Consistency of a Pairwise Comparison Matrix in the Analytic Hierarchy Process . Portland International Conference on Management of Engineering and Technology ( 2017 ), pp. 1 - 6 , doi: 10.23919/PICMET. 2017 . 8125304 .

[14] M. Brunelli . Recent Advances on Inconsistency Indices for Pairwise Comparisons. A Commentary. Fundam. Inform . 2016 , 144 , pp. 321 - 332 .

[15] W. Koczkodaj , R. Szwarc . On axiomatization of inconsistency indicators for pairwise comparisons . Fundam. Inform . 2014 , 132 , pp. 485 - 500 .

[16] J. Barzilai . Consistency measures for pairwise comparison matrices . J. Multi-Criteria Decis. Anal . 1998 , 7 , pp. 123 - 132 .

[17] E. Choo , W. Wedley . Estimating ratio scale values when units are unspecified . Computers and Industrial Engineering . 59 ( 2010 ), pp. 200 - 208 . doi: 10 .1016/j.cie. 2010 . 04 .001

[18] A.Z. Grzybowski . New results on inconsistency indices and their relationship with the quality of priority vector estimation . Expert Syst. Appl . 2016 , 43 , 197 - 212

[19] G.H. Golub , C.F. Van Loan. Matrix computations (3rd ed.). Baltimore: Johns Hopkins . 1996 .

[20] S. Bozóki , V. Tsyganok. The (logarithmic) least-squares optimality of the arithmetic (geometric) mean of weight vectors calculated from all spanning trees for incomplete additive (multiplicative) pairwise comparison matrices . International Journal of General Systems . 2019 . Vol. 48 ( 4 ) ( 2019 ), pp. 362 - 381 . doi: 10 .1080/03081079. 2019 . 1585432 .

[21] L. Csato . A characterization of the Logarithmic Least Squares Method , European Journal of Operational Research , 276 ( 1 ) ( 2019 ), pp. 212 - 216

[22] R. Ramanathan , L. Ganesh . Group Preference Aggregation Methods Employed in AHP: An Evaluation and Intrinsic Process for Deriving Members ' Weightages European Journal of Operational Research 79 ( 1994 ), pp. 249 - 265 .

[23] S. Kadenko . Defining relative weights of data sources during aggregation of pair-wise comparisons , in http://ceur-ws. org/ Vol-2067/paper7.pdf.

[24] H. Hnatiienko , N. Tmienova , A. Kruglov A. ( 2021 ) Methods for Determining the Group Ranking of Alternatives for Incomplete Expert Rankings . In: Shkarlet S., Morozov A. , Palagin A . (eds) Mathematical Modeling and Simulation of Systems (MODS' 2020 ). MODS 2020. Advances in Intelligent Systems and Computing , vol 1265 . Springer, Cham ( 2020 ), pp. 217 - 226 . doi: https://doi.org/10.1007/978-3- 030 -58124-4_ 21 .

[25] N. Kiktev , H. Rozorinov , M. Masoud . Information model of traction ability analysis of underground conveyors drives . 13th International Conference Perspective Technologies and Methods in MEMS Design, MEMSTECH 2017 - Proceedings , 2017 , pр. 143 - 145 . DOI: 10 .1109/MEMSTECH. 2017 . 7937552 .