=Paper=
{{Paper
|id=Vol-3106/Short_1.pdf
|storemode=property
|title=On Constructing Adjustable Procedures for Enhancing Consistency of Pairwise Comparisons on the Base of Linear Equations
|pdfUrl=https://ceur-ws.org/Vol-3106/Short_1.pdf
|volume=Vol-3106
|authors=Oleksiy Oletsky
|dblpUrl=https://dblp.org/rec/conf/intsol/Oletsky21
}}
==On Constructing Adjustable Procedures for Enhancing Consistency of Pairwise Comparisons on the Base of Linear Equations==
https://ceur-ws.org/Vol-3106/Short_1.pdf
On Constructing Adjustable Procedures for Enhancing
Consistency of Pairwise Comparisons on the Base of Linear
Equations
Oleksiy Oletsky
National University of Kyiv-Mohyla Academy, Skovorody St.,2, Kyiv, 04070, Ukraine
Abstract
The flexible and adjustable procedure aimed at improving consistency of pairwise
comparison matrices by transforming initial matrices obtained from experts is suggested.
Inconsistencies may arise because of unintentional mistakes and unawareness of experts, as
well as because of deliberate manipulations and fraud. This procedure is based on solving a
system of linear equations, one group of which corresponds to the experts’ judgments, and
the other group reflects the requirements of consistency. For constructing such equations,
transitive scales of preferences between alternatives are used. For taking into account
possible unreliability of the experts’ judgments, different weights can be assigned to different
equations of the system.
It is shown in the paper that the suggested procedure can be either performed one-time or
recurred iteratively. An effect of uncontrolled changes in preference directions of pairwise
comparisons in the course of such iterations was detected.
Some experiments are described. Much attention within these experiments is paid to
counteracting inconsistencies caused by possible manipulations.
Keywords 1
Analytic Hierarchy Process, pairwise comparisons, consistency, systems of linear equations,
fraud detection
1. Introduction. Methods and tools
It is commonly recognized that one of the main problems related to the Analytic Hierarchy Process
(AHP) [1-5] is how to ensure satisfactory consistency of pairwise comparisons within it. Possible
inconsistency may be caused both by inevitable difficulties experts encounter on stating their
judgments about the alternatives and by deliberate manipulations which may be done by some
experts. An example of such a manipulation will be shown below.
There are some known approaches related to possible ways of detecting inconsistencies in pairwise
comparison matrices and of transforming these matrices so that they would become more consistent
[6-11]. It’s worth mentioning that automated changing of pairwise comparison matrices may entail
significant contradictions with the experts’ estimates and therefore reduce experts’ motivation and
overall confidence in results of the whole process. On the other hand, inconsistencies may result from
incompetency, manipulations and fraud. Therefore, procedures for automated detecting
inconsistencies in pairwise comparison matrices and especially for transforming these matrices should
be combined with repeated queries to experts for changing their initial estimates. There is still a need
of developing effective techniques aimed at mitigating inconsistencies in the initial comparison
matrices provided by experts and maybe at detecting possible incompetence, fraud and manipulations.
II International Scientific Symposium “Intelligent Solutions” (IntSol-2021), September 28–30, 2021, Kyiv-Uzhhorod, Ukraine
EMAIL: oletsky@ukr.net
ORCID: 0000-0002-0553-5915
©️ 2021 Copyright for this paper by its authors.
Use permitted under Creative Commons License Attribution 4.0 International (CC BY 4.0).
CEUR Workshop Proceedings (CEUR-WS.org)
177
An approach which is being developed in this paper is based on forming an explicit system of
linear algebraic equations. Some of these equations correspond to the experts’ opinions, and the others
reflect desired features of consistency.
For specifying pairwise comparison matrices and then for getting such equations, so-called
transitive scales of comparisons are applied.
Some weight coefficients reflecting reliability degrees of these equations will be introduced. This
allows to take into account information about experts and to ensure combined adjustable reasoning
based of their judgments.
Some numerical examples illustrating the approach are described.
2. Transitive scales
The main idea of applying transitive scales [11] to pairwise comparisons is as follows. If we
introduce the parameter 1 , which determines how many times the value of one grade of
preference scale is larger than the other, then the minimal gradation of preference should be evaluated
as , the next gradation makes etc.
2
Transitive scales are the important type of scales that might be applied for building pairwise
comparison matrices within the AHP. Speaking more generally, there is a number of studies aimed at
exploring different types of scales and unifying them [6, 11]. Though transitive scales are not very
widely used in AHP, they have many positive features. We are going to use these scales in order to
get a straightforward way to making initial pairwise comparison matrices more consistent and reliable
by means of constructing a system of linear equations.
If transitive scales are actually applied for getting pairwise comparison matrices, experts have to
estimate amounts of gradations distancing an importance of each alternative from that of all others. A
real comparison matrix is unequivocally determined by the degrees of preference between specific
alternatives and by the value of . More technically, if the evaluated degree of preference of the i-th
alternative over the j-th one equals C c ij , then the corresponding element of the pairwise
comparison matrix shall be calculated as
aij C
(1)
And vice versa
cij log aij
(2)
One of the most important reasons for using transitive scales for different practical applications is
that a spread between the maximal and the minimal values of importance can be too large and
unnatural, and then the proper choice of allows to reduce it. The less is the parameter , the less is
the spread. An example of using such scales and a discussion about this matter can be found in [12].
Another reason for using transitive scales is that they facilitate getting explicit and simple linear
equations for mitigating inconsistencies in pairwise comparisons. Now we are going to illustrate this.
3. A simple example of manipulations
Let’s consider the following situation. There are three alternatives 1, 2, 3. An expert, who is not of
sufficient integrity, wants to promote the alternative 2 and to ensure its win, but they don’t want to be
accused of dishonesty because of making the direct statement that 2 is better than 1. Then they may
postulate levels of preferences across the alternatives as follows:
c12 1, c13 1, c23 4
It means that the alternative 1 gets a slight preference over alternatives 2 and 3 but the preference
of the alternative 2 over 3 is significant. And that is a contradiction.
178
Taking into account (1)-(2) and the fact that a pairwise comparison matrix should be inversed-
symmetric, we get the following matrix:
1
1
M (1) 1 4
1 1
1
4
Taking 1.2 gives the following distribution of importance among the alternatives, which
typically can be obtained as the Perronian vector, that is the normalized main eigenvector of the
pairwise comparison matrix. For the given matrix M (1) its normalized main eigenvector equals
0.3682 0.3912 0.2406
So even though the pairwise comparisons indicate an advantage of 1 over 2, the alternative 2 gets
an overall win.
However, the actual value of doesn’t matter significantly in this particular context, which was
confirmed experimentally. For example, taking 2 gives the following distribution of importance:
0.4068 0.5125 0.0807
so the winner doesn’t change.
But what is a genuine nature of such a situation? Technically, the pairwise comparison matrix
itself represents a transitive relation between alternatives but there is an inconsistency of the other sort
in this example. Let ai be an i-th alternative, and v (v1 ,..., vn ) be a normalized main eigenvector of
the comparison matrix. As the vector v represents the distribution of importance values among the
alternatives, and v(i ) vi represents an importance value of the i-th alternative ai , it is considered to
be desirable that the following consistency relation should be true:
ai a j v(i ) v( j )
But this relation doesn’t hold for the example under consideration.
4. Equations for enhancing consistency
The common requirements related to consistency have been formulated in [1, 11-18 etc.]. The
main of them is the following one:
i, j , k : aij aik akj (3)
By using (1)-(2) and by taking logarithm of (3) we can perform a linearization and get some linear
relations, which shall describe requirements for consistency in a linear form. In addition to this those
relations should explicitly involve levels of preferences:
i, j , k : cij cik ckj
(4)
For a consistent pairwise comparison matrix, relations like (4) should be satisfied for all of its
elements. But on the other hand, experts are posing specific values of elements reflecting their
opinions about preferences across the alternatives. Combining all these considerations together, we
come to the following explicit system of linear equations with respect to new improved estimates xij ,
given the expert estimates cij :
xij cij
i, j, k xik xkj xij 0
(5)
179
The first group of equations in (5) represents the experts’ judgments, and the other group reflects
the requirements of consistency. Let’s look at the structure of (5) more carefully. It is as follows:
b
... x 1 ,
... ...
where
c12
b ... ,
c
n 1, n
1 0 0
... ... ...
0 0 1
so is the unit matrix.
It means that the first group of equations itself forms a diagonal unit matrix with an evident
solution, therefore adding any extra equation to it inevitably makes the entire system (5) redundant.
Typically this system shall be inconsistent, i.e. it is probably going to have no solutions. But in this
case, we can try to find a pseudo-solution by means of Moore-Penrose pseudo-inversion [19].
It is well-known that the techniques of the Moore-Penrose pseudo-inversion are closely related to
the least square method. Use of the least square method and of its logarithmic variation for analyzing
pairwise comparison matrices was discussed, for example, in [11, 20, 21].
After solving (5) we can build the improved pairwise comparison matrix on the base of the
obtained solution of the system.
5. A simple numerical example
Let’s illustrate the techniques described in the previous section on a very simple numerical
example. Let’s take preference levels as follows:
c12 1, c13 1, c23 1
and let 1.2 .
This produces the pairwise comparison matrix
1
1
M ( 2) 1
1 1
1
Its index of consistency approximately equals 0.0018.
By using (5) we can get the following system of linear equations
x12 1
x13 1
x23 1
x12 x23 x13 0 (6)
180
This system is obviously inconsistent and has no solutions. By applying Moore-Penrose pseudo-
inversion we can obtain the pseudo-solution of (6):
x12 0.75, x13 1.25, x23 0.75
Since (6) is initially inconsistent, the relation
x12 x23 x13 0 (7)
still doesn’t hold. But let’s look at the new pairwise comparison matrix. It equals
1 x12 x13
M (1) x12 1 x23
x13
x 23 1
1 0.75 1.25 1. 1.1465 1.2560
0.75
1 0.8722
0.75
1. 1.1465
1.25 0.75 1 0.7962 0.8722 1.
Its index of consistency has been really reduced and now approximately equals 0.00012.
6. Adjusting the procedure
The described procedure of “improving” pairwise comparisons and of making them more
consistent can be made more flexible. We can regard not an entire system (7) but some subsets of its
equations instead. By doing so, we can adjust the procedure and change possible solutions in a desired
direction.
Let’s come back to the previous example. If we want the resulting matrix to become consistent, we
can reject some experts’ judgments. For example, in (6) we can leave out the equation
x13 1
For such a reduced system we will get the solution
x12 1., x13 2., x23 1.
which meets the requirement (7).
7. Weighting equations
In addition to this, we may not limit adjustment facilities to leaving out some equations of (5) only.
We can consider equations as those of different importance, and according to this consideration
different weight coefficients can be assigned to different equations. The idea of increasing importance
of more reliable judgments by assigning larger weight coefficients to them is more or less known, it
has been discussed in a number of papers, for instance in [22, 23]. But this idea can be implemented
in different ways.
Generally speaking, the weighted least squares method can be used for solving tasks of such a sort.
But first of all we are going to illustrate a more simple approach, which involves replacing two
conflict equations with their convex combinations. This means that from two equations
(w1, x) b1 and (w2 , x) b2
we can move on to the single equation
181
((w1 (1 )w2 ), x) b1 (1 )b2 (8)
Here (,) stands for a dot product, and [0.1] .
Let’s illustrate this approach.
8. Counteracting manipulations
Let’s come back to the example considered in the Section 3. As it was mentioned before, an expert
is trying to promote the alternative 2 by means of posing irrelevant pairwise comparisons. The
preferences are
c12 1, c13 1, c23 4
and the system (5) in this case takes the form
(q1 ) : x12 1
(q2 ) : x13 1
(q3 ) : x23 4
(q4 ) : x12 x23 x13 0
Its pseudo-solution, which can be obtained by Moore-Penrose pseudo-inversion, is as follows:
x11 0., x13 2., x23 3.
If we get the corresponding pairwise comparison matrix for 1.2 , its normalized main
eigenvector shall equal
0.3682 0.3912 0.2406
This means that the alternative 2 retains its advantage.
Let’s try introducing weight coefficients for different equations. We are going to use a strategy of
replacing conflict pairs of equations with their convex combinations. In our case we have such a
conflict pair formed by equations q2 and q3 . Meaningfully, these equations correspond to two
contradictory judgments. First of them means that the alternative 1 has a slight advantage over the
alternative 2, and the second one means that the alternative 2 has a significant advantage over the
alternative 3. The convex combination of q2 and q3 in accordance with (8) shall be written as
x13 (1 ) x23 4(1 ), 0 1
or after performing some trivial transformations
x13 (1 ) x23 4 3 (9)
Replacing q2 and q3 with (9) leads to the following system:
(q1 ) : x13 1
(q ) : x12 (1 ) x23 4 3
(q4 ) : x12 x23 x13 0
If a judgment seems to be not very reliable, its weight can be reduced. So, if q1 is considered to be
more reliable than q3 , its weight should be bigger than that of q3 . Then we may put, for example,
2
. In this case we shall get the distribution among alternatives as follows:
3
0.4474 0.1798 0.3728
Therefore, the alternative 1 wins.
182
On the contrary, decreasing leads to more distinct supremacy of the alternative 2. For example,
1
given , the distribution of importance among alternatives is as follows:
3
0.1846 0.6615 0.1539
Obviously, the issue of specifying weight coefficients is a very important issue, but we don’t
consider it in details in this paper. It is closely related to the issues of trust and reliability of judgments
[24]. In simple cases like given above the contradictions can be detected by a straightforward
preliminary analysis. Really, if an expert postulates that the alternative 1 is better than the alternative
2, he should not estimate the preference of 2 over 3 much more than of 1 over 3, and therefore the
weight of such a judgment and consequently that of the corresponding equation q3 can be reduced. An
external expertize might be helpful. But the whole issue needs a special study.
9. Iterative improvement of consistency
The process of enhancing consistency of pairwise comparisons can be made iterative. Really, after
getting an improved matrix with a better index of consistency we can apply the same procedure to the
next matrix and so on. For illustrating this process, we started with the matrix M (1) from the Section
1. After 6 iterations had been performed, we got a matrix
1 0.33 2.33
M * 0.33 1 2.67
2.33 2.67 1
with a practically zero index of consistency.
Given 1.2 , the normalized main eigenvector of this matrix is
0.3682 0.3912 0.2406
which is the same as that of the initial matrix. So the Perronian vector of the matrix remained
invariable. But there is another interesting fact worth paying attention to. The resulting matrix M * is
much more consistent than the initial one. But now the alternative 2 has an advantage even in pairwise
comparisons.
More interesting situations may arise if the initial pairwise comparison matrix is initially non-
transitive. For example, such a matrix might contain a cycle of preferences like the following:
a1 a2 a3 a4 a1
or in a more complicated case
a1 a2 a3 a4 a2 and a2 a6
The principal difference between these situations is that there is only a single strongly connected
component in the first case, and there are three of them in the second case. In that case it appears
reasonable to perform a decomposition, namely to build separate pairwise comparison matrices for
each strongly connected component, then to solve corresponding equations aimed at improving their
consistency if needed, and finally to combine the obtained results, maybe on the basis of a pairwise
comparison matrix which represent preferences between separate components. But issues related to
non-transitivity need to be specially explored.
10.Conclusions and discussion
A way for improving consistency of pairwise comparison matrices within the Analytic Hierarchy
Process is suggested in the paper. It is suggested that raw matrices of pairwise comparisons should
183
undergo some transformations in order to replace them with other matrices having better consistency
than the original ones.
The main idea of the suggested approach is to construct a system of linear algebraic equations, and
elements of the transformed matrix can be obtained from the solutions of this system. For building the
system of equations, using transitive scales with a parameter, which determines how many times the
value of one grade of preference scale is larger than the other, is suggested.
Typically a system of linear equations constructed within this approach shall be inconsistent, but
we can obtain its pseudo-solution by means of Moore-Penrose pseudo-inversion.
As a summary, the suggested scheme of transformation aimed at enhancing consistency of
pairwise comparisons can be represented as the sequence of the following steps:
getting the system of linear algebraic equations based on the initial comparison matrix;
solving the system and getting preferences across the alternatives; this step can be performed
once or be recurred iteratively;
getting the final pairwise comparison matrix.
The procedure appears to be quite flexible and adjustable. Firstly, we can use not all possible
equations, but a selected subset of them. Secondly, as long as different experts’ judgments and other
requirements may have different degrees of importance and reliability, we may introduce weights for
corresponding equations. It would be possible to use the weighted least squares method, but in the
paper another approach, which is based on convex combinations of equations, is illustrated.
Numerical experiments described in the paper showcase different aspects of regarded issues.
The other aspect is the following. Inconsistencies in comparison matrices often may be caused by
inevitable mistakes, but they may be also related to unawareness, low competence and even to non-
integrity of experts. In these cases, initial inconsistencies may not be mitigated by the described
iterative process. Moreover, it can even aggravate these inconsistencies.
Some experiments delivered above illustrate how these factors can be mitigated by assigning
different weights to different equations. Generally speaking, it is reasonable to diminish weights of
judgments and of corresponding equations which are not reliable enough and to reduce influence of
these judgments by doing so.
A wider context may be related to the problem of automated building rankings among experts and
their judgments. This might be of great importance, e.g. for addressing the problem of incompetence
or that of a fraud detecting. If such an intelligent automated system detects suspicious or improper
judgments, it should at least diminish estimated degrees of their reliability and thereby the weights of
corresponding equations, or exclude these judgments and corresponding equations at all. Especially
this refers to situations when such irrelevant judgments affect rankings of alternatives likewise it
happened with the matrix described in the sections 3 and 8, and this can be detected by the system.
Automated changing of pairwise comparison matrices may reduce experts’ motivation and overall
confidence in results of the whole process. In order to avoid this, it may be useful to consult experts
so they may be requested to provide necessary clarifications and explanations, or maybe they can
change their judgments. Developing new techniques of implementing iterative procedures for refining
pairwise comparisons combined with prompt experts’ and operators’ interventions appears to be
helpful for many industrial applications (e.g. [25]), especially for those involving conveyor
architectures and a control over emerging and renewable units.
11.Acknowledgements
We are grateful to the anonymous reviewers for their helpful suggestions.
12.References
[1] T.L. Saaty. The Analytic Hierarchy Process. McGraw-Hill, New York, 1980.
[2] M. Brunelli. Introduction to the Analytic Hierarchy Process, Springer, Cham, 2015.
[3] O.S. Vaidya, S. Kumar. Analytic hierarchy process: An overview of applications. European
Journal of Operational Research 169(1) (2006), pp. 1-29.
184
[4] A. Ishizaka, A. Labib. Review of the main developments in the analytic hierarchy process.
Expert Syst. Appl. 2011, 38, pp. 14336–14345.
[5] W. Ho. Integrated analytic hierarchy process and its applications. A literature review. European
Journal of Operational Research 186(1) (2008), pp.211-228.
[6] V.V. Tsyganok, S.V. Kadenko, O.V. Andriichuk. Usage of Scales with Different Number of
Grades for Pair Comparisons in Decision Support Systems. International Journal of the Analytic
Hierarchy Process. Vol.8(1) (2016), pp. 112-130. doi: 10.13033/ijahp.v8i1.259
[7] W. Koczkodaj, J. Szybowski. The limit of inconsistency reduction in pairwise comparisons. Int.
J. Appl. Math. Comput. Sci. 2016, 26, pp.721–729.
[8] B. Cavallo, L. D’Apuzzo. A general unified framework for pairwise comparison matrices in
multicriterial methods. Int. J. Intell. Syst. 2009, 24, pp. 377–398.
[9] Iida, Y. Ordinality Consistency Test About Items and Notation of a Pairwise Comparison Matrix,
in AHP. Proceedings of the X International Symposium for the Analytic Hierarchy Process, 2009.
[10] L. Mikhailov, S. Siraj. Improving the Ordinal Consistency of Pairwise Comparison Matrices, in
Proceedings of the XI International Symposium for the Analytic Hierarchy Process ISAHP-2011
( 2011). doi: 10.13033/isahp.y2011.128
[11] E. Choo, W. Wedley. A Common Framework for Deriving Preference Values from Pairwise
Comparison Matrices. Comput. Oper. Res. Vol.31(6) (2004), pp.893-908.
[12] O.V. Oletsky, E.V. Ivohin. Formalizing the Procedure for the Formation of a Dynamic Equilibrium of
Alternatives in a Multi-Agent Environment in Decision-Making by Majority of Votes. Cybern Syst
Anal Vol.57 (2021), pp. 47-56. doi: https://doi.org/10.1007/s10559-021-00328-y.
[13] O. Yu. Assessing and Improving Consistency of a Pairwise Comparison Matrix in the Analytic
Hierarchy Process. Portland International Conference on Management of Engineering and
Technology (2017), pp. 1-6, doi: 10.23919/PICMET.2017.8125304.
[14] M. Brunelli. Recent Advances on Inconsistency Indices for Pairwise Comparisons. A
Commentary. Fundam. Inform. 2016, 144, pp. 321–332.
[15] W. Koczkodaj, R. Szwarc. On axiomatization of inconsistency indicators for pairwise
comparisons. Fundam. Inform. 2014, 132, pp. 485–500.
[16] J. Barzilai. Consistency measures for pairwise comparison matrices. J. Multi-Criteria Decis.
Anal. 1998, 7, pp. 123–132.
[17] E. Choo, W. Wedley. Estimating ratio scale values when units are unspecified. Computers and
Industrial Engineering. 59 (2010), pp. 200–208. doi: 10.1016/j.cie.2010.04.001
[18] A.Z. Grzybowski. New results on inconsistency indices and their relationship with the quality of
priority vector estimation. Expert Syst. Appl. 2016, 43, 197–212
[19] G.H. Golub, C.F. Van Loan. Matrix computations (3rd ed.). Baltimore: Johns Hopkins. 1996.
[20] S. Bozóki, V. Tsyganok. The (logarithmic) least-squares optimality of the arithmetic (geometric)
mean of weight vectors calculated from all spanning trees for incomplete additive
(multiplicative) pairwise comparison matrices. International Journal of General Systems. 2019.
Vol.48(4) (2019), pp. 362-381. doi: 10.1080/03081079.2019.1585432.
[21] L. Csato. A characterization of the Logarithmic Least Squares Method, European Journal of
Operational Research, 276(1) (2019), pp. 212-216
[22] R. Ramanathan, L.Ganesh. Group Preference Aggregation Methods Employed in AHP: An
Evaluation and Intrinsic Process for Deriving Members’ Weightages European Journal of
Operational Research 79 (1994), pp. 249-265.
[23] S. Kadenko. Defining relative weights of data sources during aggregation of pair-wise
comparisons, in http://ceur-ws.org/Vol-2067/paper7.pdf.
[24] H. Hnatiienko, N. Tmienova, A. Kruglov A. (2021) Methods for Determining the Group Ranking
of Alternatives for Incomplete Expert Rankings. In: Shkarlet S., Morozov A., Palagin A. (eds)
Mathematical Modeling and Simulation of Systems (MODS'2020). MODS 2020. Advances in
Intelligent Systems and Computing, vol 1265. Springer, Cham (2020), pp.217-226. doi:
https://doi.org/10.1007/978-3-030-58124-4_21.
[25] N. Kiktev, H. Rozorinov, M. Masoud. Information model of traction ability analysis of
underground conveyors drives. 13th International Conference Perspective Technologies and
Methods in MEMS Design, MEMSTECH 2017 – Proceedings, 2017, pр. 143–145. DOI:
10.1109/MEMSTECH.2017.7937552.
185