Ways of Enhancing Recommendations Aimed at Improving Positions of Alternatives on the Base of AHP

Abstract

Suggestions on enhancing the approach to elaborating recommendations about improving alternatives, which are to be chosen, on the base of the Analytic Hierarchy Process, and those on supplementing the recommender system based on this approach, are made in the paper. These suggestions are aimed both at implementing various options for recommendations about actual betterment of alternatives and at justifying possible satisfactory choices. Within the latter, possibilities related to fixing inconsistencies in the initial pairwise comparison matrices are discussed and illustrated. Analytic Hierarchy Process, pairwise comparisons, recommender system, position improving, fixing inconsistency, counteracting manipulations (D.Dosyn) ORCID: 0000-0002-0553-5915 (O.Oletsky); 0000-0002-6573-2814 (O.Tryhub); 0009-0000-3909-6722 (I.Franchuk); 0000-0003-4040-4467 Workshop Proceedings

1. Introduction

The common approach to estimating given alternatives involves computing values of those alternatives with a certain formula. Given n alternatives and q criteria affecting their values, one of the most frequently used formula for getting values for each j-th alternative is as follows: where

is the separate estimation obtained for the j-th alternative by the separate k-th criterion, coefficients are normalized, which means the following: and is the weighting coefficient reflecting importance of the k-th criterion. Typically, all those and

∑ =1 = 1,

∑ =1

= 1, = ∑ =1 ∑ = 1 =1 The values are typically referred to as the global priorities of alternatives.

A lot of decision support systems are based on the well-known and commonly used Analytic Hierarchy Process (AHP) [1-6 et al.]. It is an expert-based methodology, and the coefficients and are obtained from the pairwise comparison matrices (PCMs) among alternatives and criteria respectively. Typically, given PCM A, the values for each item can be calculated as the components of the Perronian (normalized main eigenvector) of A. As an approximation, geometric means of the A’s EMAIL: [e-mail] CEUR

ceur-ws.org rows are frequently regarded instead. If and are calculated, the two-level AHP for obtaining relative weights of alternatives on the base of given criteria mathematically comes down to the formula given above. Some AHP-based decision support systems are aimed not only at estimating alternatives but on elaborating recommendations for how to improve positions of certain alternatives as well. One of such systems was presented in [ 7 ], this paper considers how to refine the approach this system is based on and to make it more flexible.

2. The basic approach

The basic approach has been described in detail in [ 7 ]. But some ideas were not articulated enough precisely and concisely in that paper, and now we are going to fix this point.

The algorithm is aimed at forming recommendations for approving the position of a certain alternative among other alternatives from the given set A, say ∗, if the best alternative, say Abest, has been already pointed out by means of necessary calculations on the base of the experts’ estimations. These recommendations shall meet the following requirements:  ∗ shall not lose, i.e., its global priority shall not become less than that of the alternative Abest;  the number of needed position steps relating to the initial position of the chosen alternative ∗ should be minimal.

A position step is a change in the position of an alternative by one step within a certain grading scale. For example, the standard grading scale assuming values {1 , 1 , 1 , … , 1 , 1,2, … 9}, which was suggested 9 8 7 2 by T.Saaty [ 1 ], is widely used. Within this scale, one position step is a change from 1 to 1 , from 6 to 7, 4 3 or so. The system has to calculate the number of position steps for each criterion, and to sum these numbers up.

An utterly important, but arguable question is how to choose the alternative, in relation to which the number of position steps is to be calculated. At least, the following options might be considered:  for each criterion, to compare the alternative ∗ with the alternative , which has got the highest global priority.

 for each criterion, to compare the alternative ∗ with the alternative , which has got the highest priority namely with respect to this criterion.

 for each criterion, to compare the alternative ∗ with the alternative , which has got the middle-rank global priority.

 for each criterion, to compare the alternative ∗ with the alternative , which has got the middle-rank priority namely with respect to this criterion.

In the software system presented in [ 7 ], the latest approach is implemented. This means that the alternative ∗ is being compared by each criterion with a certain alternative, which turns out average namely by this criterion, even maybe ∗ with itself. Authors think that other approaches can lead to elaborating a lot of too excessive recommendations. For facilitating analysis of recommendations, the number of which may be rather large, the system provides their sorting with respect to various features.

3. The practical example

Let’s consider a sample task of purchasing vehicles for military purposes. The technical information given below appears to make no secret and can be found in open public sources. But, for evident reasons, the names of specific alternatives (i.e., the trademarks of considered units) are coded in the paper. Let there be 5 alternatives , = 1, … 5. Let the experts have picked out 7 criteria to compare these alternatives as follows:

 features of crew needed for operating the vehicle and controlling it (this criterion is called CREW in the provided example).

 how the vehicle is protected (called PROTECTION).  how the vehicle is armed (called ARMAMENT).  how mobile is the vehicle (called MOVABILITY).

 how the vehicle is adapted to overcome obstacles and other hard conditions of a road or of a surface (called PASSABILITY).

 how silent is the vehicle (called SILENCE).  facilities for maintaining and repairing the vehicle (called MAITAINABILITY).

The hierarchical representation of the described task, which is typical within the context of the AHP is as follows:

Experts are to provide pairwise comparisons across alternatives with respect to each criterion (which numbers 7), as well as those among criteria (one pairwise comparison matrix). After entering all this information into the system, a decision maker shall be ready to get initial rankings and further recommendations by pressing the button Calculate. Let the resulting screen be as shown in Fig. 2. One can see that the best choice is the alternative 5. But it is possible that this alternative cannot be chosen for some objective reasons (for instance, it is too costly, there is a lack of such vehicles, or so). However, there is another satisfactory alternative, say 1. Then the following task arises: how this alternative ought to be changed so that it will become the best, and the number of such improvements would be as small as possible. After making recommendations, the screen will look like this (Fig. 3).

Overall, there are 128 alternatives; they are sorted with respect to the number of position steps needed for 1 to become the best choice. One can see that for achieving this goal it’s enough to make two position steps: to move from 1 to 2 in comparison with the alternative 2 with respect to the PASSABILITY criteria, and to move from 1 to 1 in comparison with the alternative 3 with respect to 2

4. Some problems and possible improvements

Currently there are several restrictive problematic issues related to the described system. The approach is aimed at improving the position of a certain alternative, but this doesn’t affect comparisons between other alternatives. In practice, this may not be true. The implemented approach may be characterized as greedy in the sense that it doesn’t admit any possibilities which consider lowering the position of the regarded alternative with respect to any criterion. But sometimes the opposite may be helpful. For instance, it may be helpful to lower the position of an alternative with respect to some lowweighted criterion and to compensate this by raising its position by another, high-weighted criterion. The more serious issue is the following. Currently the algorithm is aimed at minimizing merely the number of steps needed to bring the chosen alternative to the desired position, and all possible steps are considered to be of equal value. But the fact is that different steps may be of very different difficulty, and this aspect definitely must be regarded. However, now the system doesn’t take this into account, and this matter is to be cared by those who make final decisions. So, introducing weights reflecting difficulties of possible position steps would make recommendations produced by the system much more sensible and reasonable Surely, this means the need for gathering more additional information from experts, whereas usually it is very difficult to exactly predict in advance which information of such a sort would be needed. So, the following optimization problem can be suggested: given the current winning alternative for achieving a situation when ( ∗) ≥ ( and the chosen alternative ∗, to find a sequence of position steps ( 1, 2, … )

), where (∙) is the estimation of the alternative, and ∑ ( ) →

, where ( ) is the difficulty of the position step . position steps are to be regarded. But there is a much more disputable point, namely, how to determine the alternative to be promoted. If this was easy, why we would not be able to easily determine the winner manually, without using the system at all? Some recommendations can be given, for instance, the candidate for promotion would be the alternative which has gained the second place, or so, but this matter needs to be rigorously explored. From this point of view, the system might consider different alternatives and ascribe to them different weights depending on their position, difficulty of improving them, or so. Anyway, in this case the system should take into account not only relative rankings of alternatives, but their quantitative estimations (∙) as well. This matter depends on the used grading scale in large measure, and sometimes this dependence becomes crucial.

Unlike many other systems aimed at elaborating recommendations on positioning certain alternatives, the approach described above doesn’t admit any gaming with pairwise comparisons, not saying with those affecting weighting coefficients of criteria. It actually presumes that the expert judgments are absolutely trustworthy and are not to be questioned and changed. This often may be good if the judgments are reliable indeed, and if it really goes about needed betterments of certain alternatives. But these assumptions seem to be too optimistic, and sometimes the situation may be very different.

Let’s consider the following situation. Based on the initial judgments, the winner is an alternative A, but it can’t be chosen because of some reasons. There is another passable alternative B, and the system provides recommendations how to improve it, as it has been showcased above. But the problem is that the recommended improvements may be too costly and take too long time, which may appear unacceptable. Anyway, in this case a person who makes a decision has nothing else to do but to choose B, even though that would contradict to the recommendations. What a recommender system really could do in such a situation is help the decision maker with justifying their choice, this means implementing more flexible and less strict facilities.

For instance, for the example given above it would be reasonable to introduce an additional criterion related to needed costs, and the system might suggest increasing the weight of this criterion in some reasonable way. Another possible direction is connected to fixing existing errors and problematic issues in the initial judgments, which may be not reliable enough. Firstly, for the judgments in the form of pairwise comparisons the problem of consistency exists. Even ordinal inconsistencies, not saying cardinal ones, are rather typical. Order violations, which means situations when ( ) < ( ) whereas ≻ , happen too frequently as well, and those errors may occur even for ordinally and more or less cardinally consistent PCMs. Possible approaches to improving consistency of PCMs have been considered in many papers, such as [6, 8-24 et al.]. Typically, such an improvement is about replacing the initial PCM with another, more consistent one. In [ 24 ] “benign” errors caused by lack of knowledge, overlooks, inaccuracy, distraction etc. and “malignant” ones caused by non-integrity of experts, are being distinguished, and these types of errors require different types of tackling.

Anyway, the described system might be supplemented with facilities aimed at reducing inconsistency in initial PCMs by replacing them with more consistent ones. Certainly, such a replacement may lead to changes in positions of alternatives, but those changes definitely shall be recognized as completely rightful, rule-based and justified, maybe except situations which are obviously or supposedly dubious. Let’s illustrate this on a numerical example.

5. A way to enhancing consistency

Let there be three alternatives: 1, 2 and 3, and 3 criteria: 1, 2, 3. Let pairwise comparison matrices among alternatives by separate criteria be as follows (here we are using the standard grading scale suggested by Saaty though many other alternative possibilities for building grading scales are known [6, 25-29 et.al.]): Let the PCM across criteria be as follows: 1 1 ( ) = ( 2) 1 1 1 2 1 1

Here the criterion 3 is estimated better than 1. Then the AHP yields values for alternatives as follows: 0.3728

The alternative 1 is pointed out to be the best. But the initial PCMs are not consistent enough. Let’s try to improve consistency of the PCM ( ) among criteria as it has been described in [ 23, 24 ]. According to that approach, for finding the updated PCM (in the logarithmic form) we might try to solve the over-defined system of linear equations

Here the first three equations correspond to estimations given by experts (in the logarithmic form), and the last one reflects the requirement of cardinal consistency. Unfortunately, this straightforward approach shall not be helpful. Indeed, we will get a more consistent PCM, but it will have the same Perronian vector, therefore nothing is going to change. What we can do is introduce weighting coefficients for equations reflecting how trustworthy are specific judgments as it has been suggested in [ 23, 24 ]. In our case, we may increase the role of the judgment 13 = −1, for instance by ascribing weighting coefficient 2 to it. Thus, the system of equations takes the view {

12 = 0 2 13 = −2

23 = 0 12 + 23 − 13 = 0 From that we are getting the updated PCM among criteria

1 0.8079 0.5274 ′( ) = ( 1.2377 1 0.8079)

1.8962 1.277 1 which yields the following distribution of values among alternatives: 0.3672

Now 2 wins. A question how to choose weights which should be increased and how to get weighting coefficients for judgments is a very special issue. These coefficients could be obtained by some preliminary analysis, or by getting additional consultations from experts, or in some other way. Some considerations about this issue have been discussed in [ 24 ], but the matter is really tricky. Now we are only illustrating the main idea and the possibility for its implementation.

6. Conclusions and discussion

In the paper we have discussed the approach, which was implemented in the software system combining decision making and recommender features on the base of the Analytic Hierarchy Process. This approach is aimed not only at choosing the best alternative on the base of the experts’ judgments in the form of pairwise comparisons, but also at elaborating recommendations on how to improve the position of a certain alternative if this is needed. An illustrative example has been provided. But there are possibilities to refine the approach itself and the recommender system, some of these ways have been suggested and discussed in the paper. We are distinctly differentiating two directions for possible supplementing of the described system as follows:  implementing different options aimed at elaborating recommendations for actual improvement of alternatives.

 elaborating recommendations aimed at reasonable justifying of choices made by a decision maker.

Possibilities connected with the first direction have been discussed in detail in the Section 4. What about the second direction, an approach aimed at fixing cardinal inconsistencies in the initial PCMs has been outlined, and the numerical example has been provided. This approach needs further development, and other possibilities definitely exist. Another sensible classification of possible enhancements considers how reliable are judgments provided by experts. Introducing weighting coefficients reflecting measures of trust to different judgments seems to be a good idea. It has been showcased in the Section 5 and discussed in more detail in [ 24 ]. But on the other hand, implementing such an option may make the system more vulnerable for many kinds of undesired manipulations. Basically, the more sophisticated and flexible the system becomes and the more degrees of freedom it ensures, the more possibilities for dubious gaming and manipulating with it are arising. There may be many “malignant” manipulations including but not limited to intentional inconsistencies, this matter was discussed in [ 24 ]. Some of these manipulations even imply that a manipulator does not want to express their preferences directly, but they are aware of the algorithms implemented in the recommender system, and on the base of this they want the system to change things in the direction desirable for the manipulator.

In our opinion, judgments connected with comparing criteria, which are being used for assessing alternatives, are especially hazardous or vulnerable from this point of view. The system definitely should differentiate such “malignant” manipulations from “benign” errors related to possible lack of experts’ knowledge, inaccuracy, overlooks, or so, and tackle these very different cases in very different ways. Some considerations about this issue including but not limited to possible ways of counteracting possible manipulations can be found in [ 24 ], but the question obviously needs further investigation. So, within prospects of our further research we are going to elaborate ways for combining both directions of enhancing the decision making system described above, namely suggesting recommendations for actual improvement of certain alternatives, on the one hand, and counteracting inconsistencies and manipulations, on the other. A game aspect is worth mentioning as well. We can imagine that there may be some influencers, each of which wants the system to promote a certain alternative. Then we can regard a possible competition among these influencers, and this competition may be modelled and explored by means of methods of the game theory. Regulations for setting reasonable rules of the game should be elaborated as well, some known approaches to this are developing now within the algorithmic game theory [ 30, 31 ]. These methods can also be used in information security systems [ 32, 33 ].

7. References

References

[1] T.L. Saaty , The Analytic Hierarchy Process, McGraw-Hill , New York, 1980 .

[2] M. Brunelli , Introduction to the Analytic Hierarchy Process , Springer, Cham, 2015 .

[3] O.S. Vaidya , S. Kumar , Analytic hierarchy process: An overview of applications , European Journal of Operational Research 169-1 ( 2006 ) 1 - 29 .

[4] A. Ishizaka , A. Labib . Review of the main developments in the analytic hierarchy process . Expert Syst. Appl . 38 ( 2011 ) 14336 - 14345 .

[5] W. Ho , Integrated analytic hierarchy process and its applications. A literature review , European Journal of Operational Research 186-1 ( 2008 ) 211 - 228 .

[6] W. Koczkodaj , L. Mikhailov , G.Redlarski, M. Soltys , J. Szybowski , G. Tamazian , E.Wajch, Kevin Kam Fung Yuen, Important Facts and Observations about Pairwise Comparisons, Fundamenta Informaticae 144 ( 2016 ) 1 - 17 . doi: 10 .3233/FI-2016-1336.

[7] R. Trygub , O. Trygub , V. Gorborukov , Researching Semistructured Problems of Multicriteria Optimization Using the Software System , Scientific Notes of National University of Kyiv-Mohyla Academy . Computer Science , Vol. 151 ( 2013 ) 79 - 88 .

[8] W. Koczkodaj , S. Szarek , On distance-based inconsistency reduction algorithms for pairwise comparisons , Logic Journal of IGPL 18-6 ( 2010 ) 859 - 869 .

[9] O. Yu , Assessing and Improving Consistency of a Pairwise Comparison Matrix , in: Analytic Hierarchy Process. Portland International Conference on Management of Engineering and Technology ( 2017 ), pp. 1 - 6 . doi: 10 .23919/PICMET. 2017 . 8125304 .

[10] M.M. Potomkin , M.V. Nikolaienko , D.I.Grazion , Improvement of Analytic Hierarchy Process based on the Refinement of the Procedures for the Formation of Pairwise Comparison Matrices , Cybern Syst Anal 56 ( 2020 ) 603 - 610 . doi: https://doi.org/10.1007/s10559-020-00277-y.

[11] L. Mikhailov , S. Siraj , Improving the Ordinal Consistency of Pairwise Comparison Matrices , in: Proceedings of the XI International Symposium for the Analytic Hierarchy Process ISAHP-201 , 2011 . doi: 10 .13033/isahp.y2011. 128 .

[12] S. Siraj , L. Mikhailov , J. Keane , A heuristic method to rectify intransitive judgement in pairwise comparison matrices , European Journal of Operational Research 216 ( 2012 ) 420 - 428 .

[13] J. Barzilai , Consistency measures for pairwise comparison matrices , J. Multi-Criteria Decis. Anal . ( 1998 ) 123 - 132 .

[14] W. Koczkodaj , M. Orłowski , Computing a consistent approximation to a generalized pairwise comparisons matrix , Computers & Mathematics with Applications , 37 - 3 ( 1999 ) 79 - 85 .

[15] Y. Iida , Ordinality Consistency Test About Items and Notation of a Pairwise Comparison Matrix , in: AHP. Proceedings of the X International Symposium for the Analytic Hierarchy Process , 2009 .

[16] M. Brunelli , Recent Advances on Inconsistency Indices for Pairwise Comparisons, A Commentary, Fundam. Inform 144 ( 2016 ) 321 - 332 .

[17] W. Koczkodaj , R. Szwarc , On axiomatization of inconsistency indicators for pairwise comparisons , Fundam. Inform 132 ( 2014 ) 485 - 500 .

[18] A.Z. Grzybowski , New results on inconsistency indices and their relationship with the quality of priority vector estimation , Expert Syst. Appl . 43 ( 2016 ) 197 - 212 .

[19] R. Ramanathan , L.Ganesh, Group Preference Aggregation Methods Employed in AHP : An Evaluation and Intrinsic Process for Deriving Members' , Weightages European Journal of Operational Research 79 ( 1994 ) 249 - 265 .

[20] S. Bozóki , V. Tsyganok , The (logarithmic) least-squares optimality of the arithmetic (geometric) mean of weight vectors calculated from all spanning trees for incomplete additive (multiplicative) pairwise comparison matrices , International Journal of General Systems 48-4 ( 2019 ) 362 - 381 . doi: 10 .1080/03081079. 2019 . 1585432 .

[21] W. Holsztynski , W.Koczkodaj, Convergence of Inconsistency Algorithms for the Pairwise Comparisons , IPL. 59-4 ( 1996 ) 197 - 202 .

[22] W. Koczkodaj , J. Szybowski, The limit of inconsistency reduction in pairwise comparisons , Int. J. Appl. Math. Comput. Sci . 26 ( 2016 ) 721 - 729 .

[23] O. Oletsky , On Constructing Adjustable Procedures for Enhancing Consistency of Pairwise Comparisons on the Base of Linear Equations , in: CEUR Workshop Proceedings , 3106 ( 2021 ) pp. 177 - 185 .

[24] O. Oletsky , D. Dosyn , Some Ways of Counteracting Possible Manipulations Within the AHP on The Base of Weighted Linear Equations , in: CEUR Workshop Proceedings , 3347 ( 2022 ) pp. 185 - 194 .

[25] E. Choo , W. Wedley , A Common Framework for Deriving Preference Values from Pairwise Comparison Matrices , Comput. Oper. Res. 31-6 ( 2004 ) 893 - 908 .

[26] E. Choo , W. Wedley, Estimating ratio scale values when units are unspecified , Computers and Industrial Engineering 59 ( 2010 ) 200 - 208 . doi: 10 .1016/j.cie. 2010 . 04 .001.

[27] V. Tsyganok , S. Kadenko , O. Andriichuk , Usage of Scales with Different Number of Grades for Pair Comparisons in Decision Support Systems , International Journal of the Analytic Hierarchy Process 8 -1 ( 2016 ) 112 - 130 . doi: 10 .13033/ijahp.v8i1. 259 .

[28] B. Cavallo , L. D'Apuzzo , A general unified framework for pairwise comparison matrices in multicriterial methods , Int. J. Intell. Syst . 24 ( 2009 ) 377 - 398 .

[29] R.E. Jensen , An alternative scaling method for priorities in hierarchical structures , Journal of Mathematical Psychology , 28 - 3 ( 1984 ) 317 - 332 .

[30] T. Borgers , Krahmer D. , Strausz R. An introduction to the theory of mechanism design . Oxford University Press, 2015 .

[31] T. Roughgarden . Twenty lectures on algorithmic game theory . Cambridge University Press, 2016 .

[32] Hnatiienko , H. , Kiktev , N. , Babenko , N. , Desiatko , A. , Myrutenko , L. Prioritizing Cybersecurity Measures with Decision Support Methods Using Incomplete Data // Selected Papers of the XXI International Scientific and Practical Conference "Information Technologies and Security" (ITS 2021 ), Kyiv, Ukraine, December 9, 2021 . CEUR Workshop Proceedings, 2021 , 3241 , pp. 169 - 180 .

[33] Palko , D. ; Babenko, T. ; Bigdan , A. ; Kiktev , N. ; Hutsol , T. ; Kuboń , M. ; Hnatiienko , H. ; Tabor, S. ; Gorbovy , O. ; Borusiewicz , A. Cyber Security Risk Modeling in Distributed Information Systems . Applied Sciences (Switzerland) 2023 , 13 , 2393. https://doi.org/10.3390/app13042393