Ranking‐dependent Decision Support Methods for Weakly‐structured Domains

Abstract

   The paper presents an overview of decision support methods based on pair-wise comparisons of alternatives, that take ordinal relations between them into consideration. The accent is made on the original method of expert pair-wise comparisons, based on a certain order of alternative estimation. This order is defined based on a preliminary rough ranking of compared alternatives. The described method allows us to improve the credibility of expert session results, obtain more consistent expert data, and, if necessary, reduce the labor intensity of expert estimation through reduction of required number of pair-wise comparisons.

Keywords

•  1  decision-making support
• expert estimation
• pair-wise comparison matrix
• alternative ranking
• weakly-structured subject domain

1. Introduction 

2. Overview  of  available  methods  that  take  the  information  on  alternative  ranking into account 

In order to be able to reduce the number of PC, the expert session organizer might need

some additional information on the alternatives and preference structures. For example, he might follow some certain preference structures, corresponding to minimum-diameter regular graphs, as described in [5,6]. Also, he might use the information on ordinal relation between compared alternatives.

In fact, a set of PC methods envisions usage of information on alternative ranking. Some

ordinal factorial analysis methods rely on alternative rankings only [7,8]. Many of rankingdependent approaches are based on experiments in cognitive psychology, such as the ones described in [9]. These experiments indicated that people estimate objects more precisely if these objects are presented to them in the order of decrease of the estimation criterion (i.e., from largest to smallest, from best to worst etc).

Historically, the first method using this approach was, probably, TOPSIS [10]. In this

method, “ideal best” and “ideal worst” alternatives are generated based on multi-criteria expert estimates of alternatives. After that, alternatives are rated according to their distance from these ideal objects. We should also mention approaches used by Harker [11] and Wedley [12], which rely on preliminary rough ranking of alternatives. During the last decade several mono-criterial PC methods emerged, which relied on information on the ordinal relation between the alternatives. For instance, best\worst method [13,14] envisions comparing all alternatives with the best and the worst one in the set (instead of complete enumeration of pairs). TOP2 (best\second best) method [15] envisions comparing all alternatives from the set only with the first two alternatives in the ranking. Best\worst and TOP2 methods are illustrated by Fig. 1. b  Fig. 1 Best\worst (a) and TOP2 (b) methods for the case of  7  alternatives.  3. Comparisons of the most distant alternatives: outline of the method 

Another method, using the information on preliminary alternative ranking, has been

suggested by the authors. It has been outlined in several conference papers, for instance, in [16]. Recently we have obtained some more data based on experimental results. In the following sections we will share the most recent findings related to the method.

The approach envisions comparing the alternatives in the order of decreasing distances between them in the ranking. So, PC are performed in queues.

Let the alternatives be numbered according to their preliminary ranking: ⋯ , where is an alternative with number and rank , 1, , while is the general number of alternatives. Then, the sequence of expert PC (which ensures the greatest adequacy of expert session results to the expert’s inner judgments) is as follows: (ranks differ by 1

; (ranks differ by 2; or ,

(ranks differ by 3; , or … or , (ranks differ by 1 .

Once PC input is finished, alternative weights can be calculated using eigenvector or any queue 1: queue 2: queue 3: … queue 1 , , ,

: other chosen method.

or or , , , or                     4. Outcomes of experimental research of the method 

Experimental research of the method, outlined in [16], indicated that the relative weights

of alternatives, calculated using eigenvector based on PCM, corresponding to the suggested PC sequence, were more adequate to experts’ judgments about subject domains they considered themselves competent in (in comparison to weights, obtained based on other PC sequences).

Subject domains and problems chosen by respondents for decomposition were rather

diverse. Some of the common subject domain examples, chosen by experts, are as follows:

Development of a data analysis system development for a smart home

Smartphone selection (based on design, functionality, aesthetical features)

Hotel selection

Analysis of competitors

Developing a communication platform for a university department

Starting a smart surveillance camera development project

Choosing a bank for getting a deposit

Car selection

Choosing a cinema to watch a premiering movie

Choosing a city to live in (based on location, living standards etc)

Choosing a platform for app development

Tour selection

Choosing a foreign university for a master program

POS system development

Selecting a laptop for personal use

Starting up a new band

Development of a role-play (action, shooter) game

Choosing a Smart TV

Promoting an IT startup

Choosing a music streaming service 34

 Choosing a mobile app for photo editing

So, as we can see, in comparison to previous experiments, which focused on one and the

same subject domain or criterion (such as [12]), our experiment has been a more universal one.

Our recent research shows that even the criteria PCM, constructed according to the suggested PC sequence, are, generally, more consistent than PCM, constructed according to other sequences.

Several hundreds of experiment instances were conducted. Out of these, 83 meaningful instances, devoid of rank reversals, have been selected.

Table 1 shows the filtered (screened) results of the experiment. The 1st column indicates the number of experiment instance. The 9th column indicates how the expert him(her)self has ranked the PC sequences (or orders) A (most distant alternatives compared first), B (random order), and C (closest alternatives compared first). The 10th (last) column indicates the ranking of sequences A, B, and C, according to the value of consistency ratio CR of PCM, obtained based on respective sequences.

Table 1. Aggregated experiment results  Continue with the Table 1.  #  results was not an initial priority and consistency studies have been conducted only recently, however, their results still speak in favor of the suggested approach. 5. Comparing the method with other ranking‐based PC methods 

One of obvious directions of future research would be comparison of the suggested PC

method (“most distant objects first”) with other similar ranking-based methods (such as TOP2 and best\worst). We have a simulation-type experiment in mind. Its phases are as follows:

1) generate a random weight vector;

2) reconstruct an ideally consistent PCM (ICPCM) based on the generated vector;

3) fluctuate the ICPCM with some noise matrix;

4) calculate the weight vector from the fluctuated PCM, using different methods (PC of the

most distant objects first, TOP2, best\worst);

5) calculate and compare weight calculation errors across the three methods.

At the same time, such an experiment is problematic to conduct before we clarify

some issues.

First, both TOP2 and best\worst are incomplete PC methods. They do not require experts 3 comparisons (see Fig. 1) and fill just 2 3 cells of the respective PCM: to fill all PCM cell. Each of these two methods requires the respondent to perform just 2 1 2), or, if necessary, to reduce the number of comparisons to minimum ( 1 Fig. 3 Spanning tree, obtained for 7 alternatives based on their ranking and suggested PC order 

So, the question “how many of the above-mentioned queues and PC from these queues

will we need to perform within the experiment?” remains open.

Second, we need to determine, how to complete an incomplete PCM (as in order to apply the eigenvector method for weight calculation, we will need a complete matrix) based on available comparisons.

Third, it is unclear, which methods of weight calculation, besides eigenvector method, should we should apply (LLSM, combinatorial method, GM, other [1-4]).

6. Application of the method to enumeration of spanning trees 

Much of recent research (including [3,4]) has been dedicated to reduction of computational

complexity of the combinatorial method of spanning tree enumeration. In [4] it has been shown that the trees of smaller diameter accumulate smaller expert errors, than trees with larger diameter. So, it makes sense to consider these trees in the first place during aggregation.

As alternative ranking provides additional information on relations between alternatives, this information can be used for modification of the combinatorial method. Each spanning tree is a graph, consisting of 1

edges. So, in order to sort the spanning trees from more informative to less informative ones, we can assign a rating (or utility) to each edge, based on the number of PC queue this edge belongs to: where: ∑ ∑

, 1, 0,

∈ , ∉ ;

So, spanning trees can be ranked and sorted according to their ratings (or utilities).

The suggested ratings provide an alternative order of spanning tree enumeration. Spanning

trees, containing more information, should be enumerated in the first place during aggregation. While the approach outlined in [4] is based solely on the diameter of spanning trees and does not depend on specific object numbers, the approach suggested here relies on alternative ranks and, thus, is not invariant in terms of alternative numbers. At the same time, it is interesting to note that the trees containing just comparisons of neighboring alternatives are the “least informative” ones in terms of both graph diameter and comparison sequence.

To illustrate the sorting of spanning trees according to their ratings, let us consider the examples of 4 and 5 alternatives. Ranking of the spanning trees for 4 alternatives is shown (1) (2) in Table 3. Sorting of these trees is shown on Fig. 4. Ranking of the spanning trees for 5 alternatives is shown in Table 4. Sorting of these trees is shown on Fig. 5. Fig. 5. Sorting of spanning trees by rating (utility) for 5 alternatives 

7. Conclusions 

In the paper we have presented an outline of expert estimation methods, based on preliminary ranking of alternatives. It has been shown that consideration of ordinal relation between alternatives allows us to reduce the number of PC without significant losses and distortions of expert information. We have suggested an original PC method, based on comparisons of the most distant objects in the ranking. Experimental results show that the method allows us to achieve better credibility and consistency of expert data. Moreover, if necessary, it allows us to reduce the number of required PC to the minimum. Additionally, it allows us to improve the computational complexity of combinatorial method of weight calculation. Further research will be focused on comparison of the method with other rankingdependent PC methods, such as best\worst and TOP2. 8. References  [1] Saaty, T. (1980). The analytic hierarchy process. New York: McGraw-Hill. [2] Bozoki, S., Tsyganok, V. (2019). The (logarithmic) least squares optimality of the arithmetic (geometric) mean of weight vectors calculated from all spanning trees for incomplete additive (multiplicative) pairwise comparison matrices. International Journal of General Systems 48(4), 362-381. [3] Lundy, M., Siraj, S., Greco, S. (2017). The Mathematical Equivalence of the “Spanning Tree” and Row Geometric Mean Preference Vectors and its Implications for Preference Analysis. European Journal of Operational Research 257(1), 197-208. [4] S. Kadenko, V. Tsyganok, Z. Szádoczki, & S. Bozóki (2021). An update on combinatorial method for aggregation of expert judgments in AHP. Production, 31: 1-17. doi: 10.1590/01036513.20210045. [5] S. Kadenko, V. Tsyganok, Z. Szádoczki, S. Bozóki, P. Juhász & Oleh Andriichuk (2021).

Improvement of Pair-wise Comparison Methods Based on Graph Theory Concepts. CEUR Workshop Proceedings, Vol. 3241, 46-55. [6] Z. Szádoczki, S. Bozóki, H. Tekile (2022). Filling in pattern designs for incomplete pairwise comparison matrices: (quasi-)regular graphs with minimal diameter. Omega, 107, doi: 10.1016/j.omega.2021.102557. [7] S. Kadenko (2013). Defining the relative weights of alternative estimation criteria based on clear and fuzzy rankings. Journal of Automation and Information Sciences, 45(2), 41-49. [8] S. Kadenko (2008). Determination of parameters of criteria of “tree” type hierarchy on the basis of ordinal estimates. Journal of Automation and Information Sciences, 40(8), 7-15. [9] Stevens, S. S. & Galanter, E. (1957) Ratio Scales and Category Scales for a Dozen Perceptual

Continua, Journal of Experimental Psychology, Vol. 54, No 6: 377-411. [10] Hwang, C.L. & Yoon, K. (1981) Multiple attribute decision making: methods and applications: a state-of-the-art survey. Berlin, New York: Springer-Verlag. [11] Harker, P. T. (1987). Shortening the comparison process in the AHP. Mathematical Modelling, 8, 139- 141 [12] W.C. Wedley (2009). Fewer Comparisons – Efficiency via Sufficient Redundancy, in: Proceedings of the 10th International Symposium on the Analytic Hierarchy/Network Process, Multi-criteria Decision Making, Pitsburg, Pensilvania, USA, July 30 – August 2, 2009. URL: https://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.553.2890&rep=rep1&type=pdf [13] Rezaei, J. (2015) Best-worst multi-criteria decision-making method. Omega, 53, 49-57, https://doi.org/10.1016/j.omega.2014.11.009. [14] Rezaei, Jafar (2016). Best-worst multi-criteria decision-making method: Some properties and a linear model. Omega. 64: 126–130. doi:10.1016/j.omega.2015.12.001. [15] Szádoczki, Z., Bozóki, S., Juhász, P. et al. (2023) Incomplete pairwise comparison matrices based on graphs with average degree approximately 3. Ann Oper Res 326, 783–807. https://doi.org/10.1007/s10479-022-04819-9 [16] Andriichuk, O., Tsyganok, V., Kadenko, S., & Porplenko, Y. (2020) Experimental Research of Impact of Order of Pairwise Alternative Comparisons upon Credibility of Expert Session Results, in: Proceedings of the 2020 IEEE 2nd International Conference on System Analysis & Intelligent Computing (SAIC), pp. 1-5, http://dx.doi.org/10.1109/SAIC51296.2020.9239126

References