Visualizing methods of multi-criteria alternatives for pairwise comparison procedure A.A. Zakharova1,2, D.A. Korostelyov1,2 zaa@tu-bryansk.ru | nigm85@mail.ru 1 Keldysh Institute of Applied Mathematics Russian Academy of Sciences, Moscow, Russia 2 Bryansk State Technical University, Bryansk, Russia The article deals with the problem of choosing a preferred alternative in a pairwise comparison procedure. The difficulties of applying this procedure in a case of using alternatives with a large number of criteria are noted. It is proposed to supplement the procedure of expert pairwise comparison with visualization tools of multi-criteria alternatives. The paper considers several visualization methods for multi-criteria alternatives for pairwise comparison procedures: histograms, two-dimensional graphs, three-dimensional surfaces, probability distribution diagrams, visualization based on modifications of radar and radial diagrams, as well as combined methods. It described an experimental study of the application of the considered method for the task of determining the preferred alternative by the example of choosing one of two OpenFoam solvers (rhoCentralFoam and pisoCentralFoam), with the help of which estimates of the accuracy of calculating the inviscid flow around a cone were obtained. Π•ach solver is characterized by 288 criteria. It is shown that the use of some of the methods considered does not make it possible for the expert to make a choice. In this case, a good result was obtained using methods for constructing three-dimensional surfaces, probability distribution diagrams, as well as using the combined method based on modified radar diagrams. It is concluded that the rhoCentralFoam solver is more preferable if there are no additional criteria for ranking the criteria. The possibility of using the combined method in combination with the ranking procedure of criteria (or their groups) during decision-making is also noted. Keywords: multi-criteria alternatives, visual pair comparison, alternatives visualization, decision support. when comparing the site design at the stage of its design 1. Introduction [6]. In decision theory, one of the basic tasks is ranking Thus, we have the task of visualizing data sets alternatives [1, 2]. This allows setting their priority in characterizing alternatives. Consider and analyze several relation to the task at hand. There are various methods approaches and methods that can be applied to visualize that allow such ranking, some of which are expert. multi-criteria alternatives. Among the expert ranking methods, the method of 2. Visualization methods pairwise comparisons [3] has proven itself well, the essence of which is to provide the expert alternately with Data preparation pairs of alternatives for comparison, during which he prefers one of them. In particular, this procedure used in The visualization procedure begins with a step one of the classical decision-making methods β€” the requiring initial data preparation. hierarchy analysis method developed by T. Saati [4], in 1. All input values should be given to a numeric format which it is necessary to construct matrices of pairwise - relevant methods of decision theory may be used comparisons for all levels of the hierarchy. for these purposes [7]. The use of pairwise comparisons is usually effective 2. Normalization of data per segment [0; 1] taking into in cases where each alternative is well reflected in the account the direction of the criterions optimization expert’s perception or is characterized by a small number (maximization or minimization). For these purposes, of criteria (usually no more than 10) [5]. In the case when the formulas can be used: β€² 𝑣𝑣𝑖𝑖,𝑗𝑗 βˆ’π‘£π‘£π‘šπ‘šπ‘šπ‘šπ‘šπ‘š,𝑗𝑗 an expert needs to compare new for him alternatives with β€’ 𝑣𝑣𝑖𝑖,𝑗𝑗 = – in case of maximization; π‘£π‘£π‘šπ‘šπ‘šπ‘šπ‘šπ‘š,π‘—π‘—βˆ’π‘£π‘£π‘šπ‘šπ‘šπ‘šπ‘šπ‘š,𝑗𝑗 a large number of criteria, this can cause difficulties for β€² π‘£π‘£π‘šπ‘šπ‘šπ‘šπ‘šπ‘š,π‘—π‘—βˆ’π‘£π‘£π‘–π‘–,𝑗𝑗 him. Therefore, in such situations, it is necessary to use β€’ 𝑣𝑣𝑖𝑖,𝑗𝑗 = – in case of minimization, π‘£π‘£π‘šπ‘šπ‘šπ‘šπ‘šπ‘š,π‘—π‘—βˆ’π‘£π‘£π‘šπ‘šπ‘šπ‘šπ‘šπ‘š,𝑗𝑗 additional tools, for example, reducing the dimension, or where i – alternative number (1 ≀ 𝑖𝑖 ≀ 𝑁𝑁), j – criteria statistical processing of criteria values. However, even number (1 ≀ 𝑗𝑗 ≀ 𝐾𝐾), N – count of alternatives, K – count applying these approaches, there is still a chance of not of criteria, vmax,j, vmin,j – maximum and minimum possible getting the desired result. For example, in the case of value of j-th criteria. Values vmax,j, vmin,j usually calculating statistical characteristics, we can get determined on the basis of their physical meaning. conflicting data in a situation where the mathematical However, if there are problems with their definition, then expectation for an alternative is better, but the variance is they can be calculated by the formulas: worse. Therefore, additional tools are needed that could π‘£π‘£π‘šπ‘šπ‘šπ‘šπ‘šπ‘š,𝑗𝑗 = min�𝑣𝑣𝑖𝑖,𝑗𝑗 οΏ½, help the expert decide. One such methods may be visual analytics - when for π‘£π‘£π‘šπ‘šπ‘šπ‘šπ‘šπ‘š,𝑗𝑗 = max�𝑣𝑣𝑖𝑖,𝑗𝑗 οΏ½. each alternative a corresponding visual image is constructed that characterizes the set of values of its After the data is prepared, you can proceed to criteria. Visualization is able to present the alternative as visualize them. For these purposes, several different a holistic image, and it will be easier for an expert to approaches and methods can be applied, in each of which make his choice with its help. It should be noted that the we will consider the visualization of two alternatives and visual comparison of alternatives is currently already the features of their visual pairwise comparison. being applied and shows a good result, for example, Copyright Β© 2020 for this paper by its authors. Use permitted under Creative Commons License Attribution 4.0 International (CC BY 4.0) Histograms in the criteria that the expert can undoubtedly notice with a pairwise comparison. One of the most accessible and simple methods of In addition, when comparing two histograms, the visualization of multidimensional data is diagrams, and expert’s perception can be significantly affected by the among them, the most accessible in the procedure of order of columns (criteria), therefore, when using this pairwise comparison can be called histograms. In this approach, it is appropriate to think over the ordering or case, two main approaches to their application can be grouping of criteria based on their features, for example, distinguished. by a degree of importance. 1. Both alternatives are shown on a common In addition to using this visualization method in the histogram. (Fig. 1). In addition, you can use the pairing comparison procedure, it can also be useful in option of overlapping columns to focus on the ranking the criteria, as well as grouping them. This deviation of values by criteria. makes it possible to use this method as a preparatory stage for the construction and comparison of visual images of alternatives. Two-dimensional graphics and three- dimensional surfaces Other common data visualization methods are two- dimensional graphs and three-dimensional surfaces. Their application can be effective when there are areas with a noticeable difference in the values of the criteria, and due to interpolation, these areas are more pronounced. In the case of two-dimensional graphs, the X axis can be interpreted as the serial number of the comparison Fig. 1. Visual comparison of two alternatives in a common criterion j (1 ≀ 𝑗𝑗 ≀ 𝐾𝐾) and at Y axis normalized criterion β€² histogram value – 𝑣𝑣𝑖𝑖,𝑗𝑗 (1 ≀ 𝑖𝑖 ≀ 2) is plotted. If each criterion corresponds to a numerical value (for example, time), 2. Each alternative is visualized on separate then it can also be used to determine the X coordinate histograms. (Fig. 2). (only provided that all these values are pairwise different) (Fig. 3). Fig. 2. Visual comparison of two alternatives on separate histograms Note that in the 2nd approach, it is important to use identical parameters of the diagrams (color, size, etc.) in order to reduce the subjectivity of the perception of visual images, describing the data sets of the Fig. 3. Visual comparison of two alternatives on a two- corresponding alternatives, and to enable the expert to dimensional graph focus on a holistic perception of visual images. However, for the application of visualization based When determining preferences among alternatives on surfaces in three-dimensional space, a prerequisite is based on visual images, an expert can be guided by his the presence or ability to display a set of criteria on the perception of the degree of filling of the columns of axis X and Y: 𝑓𝑓𝑓𝑓(𝑗𝑗) = π‘₯π‘₯, 𝑓𝑓𝑓𝑓(𝑗𝑗) = 𝑦𝑦, 1 ≀ 𝑗𝑗 ≀ 𝐾𝐾. This diagrams, including and the total area of all columns (the mapping can be set based on the physical meaning of the second approach is more suitable for these purposes) or criteria, or by using the grouping of criteria (for example, the subjectively averaged deviation of the criteria using clustering methods) (Fig. 4). columns of the two alternatives (the first approach is more suitable for these purposes). The use of histograms is justified for those situations when it is necessary to see the whole alternative as a whole, and the number of criteria is not too large (about 5-20). Moreover, the effectiveness of this visualization method is additionally achieved if ranking approaches or non-linear scales were used in the normalization process since this allows you to get quite noticeable differences Probability distribution diagram Probability distribution diagrams can also be used as another approach to visualizing data sets describing alternatives. For these purposes, the interval [0; 1], to β€² which all normalized values of the criteria (𝑣𝑣𝑖𝑖,𝑗𝑗 ) belong, on M equal intervals in length (for example, by 5 or 10) depending on the number of criteria. These intervals are located on the abscissa axis. After that, the number Π‘i,m Fig. 4. Visual comparison of two alternatives using a surface (1 ≀ π‘šπ‘š ≀ 𝑀𝑀) of hits of the normalized criteria values for in three-dimensional space each of the intervals is determined, which then allows you to determine the corresponding probabilities: 𝑝𝑝𝑖𝑖,π‘šπ‘š = 𝐢𝐢𝑖𝑖.π‘šπ‘š Similar to histograms, visual images for the pairwise , used as values on the ordinate axis when plotting 𝐾𝐾 comparison procedure in the case of two-dimensional (Fig. 5). graphs and three-dimensional surfaces can also be As in previous approaches for this visualization constructed both on one diagram, and on two method, it is also possible to build diagrams on a single neighboring ones with the same parameters. diagram, or separately. The choice of preference in the The expert can make his choice based on the area (for comparison of the probability distribution the expert can surfaces) or the intervals (for curves) of the zones where give an alternative, which is characterized by the one alternative prevails over another. Moreover, in the displacement of the probability distribution to the right case of surfaces built in three-dimensional space, a (towards the interval with the highest criteria of values). prerequisite is the availability of tools that allow you to This type of chart is conveniently displayed on a single rotate and zoom in on the surface so that the expert can diagram while applying transparency of the columns so choose the most suitable angles for comparison. Thus, it that the differences are accented (Fig. 5). is important to provide an interactivity property. This The effectiveness of this approach becomes method allows you to compare visual alternatives significantly higher when the number of criteria K is entirely based on the subjective perception of the sufficiently large (for example, hundreds and thousands). coverage area of one surface of another, and also with its With its help, not only visual images can be obtained, but help you can determine the various relationships of the also quantitative probabilistic characteristics of the groups of criteria that are used as the coordinates of the dominance of one alternative over another. Also, this measurements along the abscissa and ordinates. method allows us to group criteria, but it does not allow them to be ranked. Fig. 5. Visual comparison of two alternatives using a probability distribution diagram Λ— sectors with radii proportional to the criteria of Visual images based on radar and radial alternative (Fig. 6); diagrams Λ— sectors with radii proportional to the roots of the As noted in [8] visual images of alternatives can also criteria of alternative (Fig. 7); be built on the basis of methods based on the radar and Λ— radar diagram with a permutation of criteria by radial diagrams. The following visualization options are grouping large values side by side (Fig. 8); proposed: Λ— radar diagram with a permutation of criteria, taking advantages and disadvantages, and often some of them into account their alternation (alternately clockwise may not be useful in the visual comparison itself, but in are the criteria with larger and smaller values) (Fig. the preparatory stage, the purpose of which is to 9). determine the order or grouping of criteria (as histograms When using this method of visualization, the main and 3D surface), as well as the integral quantitative emphasis is on the fact that the best alternative occupies characteristics of visual images - brightness, area of a larger area, and also that the visual image is brighter prevalence, statistical characteristics (probability due to the use of gradient fills (in the center, the color is distribution diagram), etc. And already these more neutral – green, and on the periphery – more characteristics allow, for example, to set a specific order contrast – red). of permutation and grouping of criteria during visualization (radial and radar diagrams). Thus, it is advisable to move from the task of comparing a single visual object to the task of comparing a group of visual objects that characterize an alternative or its components. For this purpose, an integrated Fig. 6. Visual comparison of two alternatives using a radial approach is proposed, consisting in the sequential diagram with sector radii proportional to the values of the presentation of a series of visual images obtained using criteria different methods, until the expert makes a choice. For this expert to select the first represented whole visual images. If with their help he cannot determine the preferred alternative, then by means of visual analysis he tries to identify groups of general criteria, and also, if possible, to rank and filter them. Further, the selected groups can be visualized separately and placed in a table Fig. 7. Visual comparison of two alternatives using a radial grid - on the right are the images for the components of diagram with sector radii proportional roots of values of the one alternative, and on the left for the other (Fig. 10). criteria Fig. 8. Visual comparison of two alternatives using a radar diagram with a permutation of criteria by grouping large values side by side Fig. 9. Visual comparison of two alternatives using a radar diagram with a permutation of criteria, taking into account their alternation If one of these visualization methods is used, the expert, when paired, selects the alternative that seems to him subjectively brighter and larger in area. The methods described in [8] represent a more universal visualization mechanism, because they allow one to take into account the order and grouping of criteria, and also for them integral quantitative Fig. 10. Visual comparison of two alternatives using the integrated method characteristics (brightness, area) can be determined. Complex method In such a set of visual images, there is a high probability that in a number of rows it will be possible to When comparing alternatives by only one visual choose a preference. If for one alternative there are more image, it is not always possible to choose the preferred such preferences than for another, then you can make a one from them. This is because the comparison is usually choice in favor of this alternative. based on the color, shape, area or volume of the visual objects defining the respective alternatives. At the same time, different visualization methods have different 3. Experiments well as defined for two norms (L1, L2) four parameters (Ux, Uy, p, ρ). Analyzing this visual image (Fig. 12), one Let us analyze the application of the considered can notice that the blue color (rCF solver) prevails on the methods on the example of the alternative (solvers) surface over red (pCF solver), so the expert can choose described in the works [9, 10]. As noted in [8], out of five this alternative (rCF slover). solvers, two give the best results – rhoCentralFoam and pisoCentralFoam (rCF ΠΈ pCF). Given the fact that the number of comparison criteria for these two alternatives is quite large, we will use visual images built on different diagrams. In Fig. 11 is a visual comparison using histograms. Fig. 12. Visual comparison of two solvers using surfaces in three-dimensional space Fig. 11. Visual comparison of two solvers using histograms Fig. 13 shows the results of the visualization of alternatives using a probability distribution diagram. To For most experts, this comparison will not be build it, we used a partition of the values of the criteria unambiguous, because the images are very similar, and at the same time on both diagrams, there are both areas into 10 intervals. In this diagram (Fig. 13), it can be noted with the best values and the worst. that the blue color largely prevails in the columns [0.6; We will get an approximately similar result when 0.7), [0.8; 0.9) and [0.9; 1.0], which correspond to the using a two-dimensional graph, however, constructing a probability of falling into intervals with a higher value (rank). This means that for the normalized values of the surface in three-dimensional space can give a more rCF solver, the probability of obtaining a better solution interesting result. This method visualization is possible because criteria can be grouped due to the fact that they is higher. Therefore, the choice of an expert, in this case, were obtained during computational experiments by will most likely also be made in favor of this alternative varying two parameters – angle Ξ² (in range 10-35Β° with (rCF solver). step 5Β°) and Mach numbers (in range 2-7 with step 1), as Fig. 13. Visual comparison of two solvers using a probability distribution diagram Given the possibility of decomposing criteria into solver) is significantly preferable (due to a more uniform subsets, we consider the use of complex visual images shape and a subjectively larger area of images). If we based on petal families of diagrams. The decomposition assume that these criteria are peer-to-peer, then it will be will be carried out based on various parameters (Ux, Uy, difficult for an expert to determine preference. p, ρ) and norms (L1, L2), i.e. for each alternative, we However, if these criteria can be ranked (for example, construct eight diagrams (Fig. 14). the criteria of the L1 block are preferable to the criteria of Analyzing these visual images, it can be noted that the L2 block, or vice versa), then it will be easier for an for the first four rows (L1 norm) due to a more uniform expert to make a choice because for this, it will suffice to shape and subjectively somewhat larger area of images compare either only the upper images or only the lower the second alternative (pCF solver) looks preferable, ones. however, for the L2 norm (5-8 rows) first alternative (rCF 4. Conclusion References The analysis of visualization methods of alternatives [1] CastaΓ±Γ³n-Puga M., Sanchez M., Aguilar L., for the pairwise comparison procedure showed that, RodrΓ­guez-DΓ­az A. Applied Decision-Making. depending on the properties of the source data and their Applications in Computer Sciences and criteria, various approaches can be both effective and not. Engineering. – 2019. DOI: 10.1007/978-3-030- Therefore, it is appropriate to attempt to use several 17985-4. different visualization methods and their combination in [2] Alinezhad A., Khalili J. New Methods and conjunction with a decomposition of the source data. In Applications in Multiple Attribute Decision Making this case, it is possible on some methods to see that one (MADM). – 2019. DOI: 10.1007/978-3-030-15009- alternative is better than another due to the subjective 9. perception of the area of predominance, brightness, [3] David H.A. The Method of Paired Comparisons. smoothness of forms, etc. (2nd edition, revised). – Oxford University Press, Berlin. – 1988. [4] Saaty, T.L., 1980. Analytic Hierarchy Process: Planning, Priority Setting, and Resource Allocation. – New York: McGraw-Hill. – 1980. [5] Averchenkov V.I., Miroshnikov V.V., Podvesovskiy A.G., Korostelyov D.A. (2014) Fuzzy and Hierarchical Models for Decision Support in Software Systems Implementations. Proceedings of the 11th Joint Conference, JCKBSE 2014 (eds. A.G. Kravets et. al.), Communications in Computer and Information Science, 466, Springer International Publishing, pp. 410-421. DOI: 10.1007/978-3-319- 11854-3_35 [6] Practical application of the paired comparison method. – URL: https://usabilitylab.ru/blog/boevoe- primenenie-metoda-parnyix-sravnenij/. [7] Figuera J., Greco S. and Ehrgott M. (Eds). Multiple Criteria Decision Analysis: State of the Art Surveys. – New York: Springer, 2005. – DOI: 10.1007/b100605. [8] Zakharova A.A., Korostelyov D.A., Fedonin O.N. Visualization Algorithms for Multi-criteria Alternatives Filtering (2019). Scientific Visualization 11.4: 66 - 80, DOI: 10.26583/sv.11.4.06 [9] Bondarev A.E., Kuvshinnikov A.E. Analysis of the Accuracy of OpenFOAM Solvers for the Problem of Supersonic Flow Around a Cone // ICCS 2018, Lecture Notes in Computer Science (LNCS) 10862. – P. 221–230, 2018. – DOI:10.1007/978-3-319- 93713-7_18. [10] Bondarev A., Kuvshinnikov A. Comparative Fig. 14. Visual comparison of two solvers using a series of radar diagrams Estimation of QGDFoam Solver Accuracy for Inviscid Flow Around a Cone // IEEE The The greatest effect in the pairwise comparison Proceedings of the 2018 Ivannikov ISPRAS Open procedure can be achieved by visualization of groups of Conference (ISPRAS-2018). – P. 82-87, 2018. – initial criteria combining with a ranking of DOI: 10.1109/ISPRAS.2018.00019. decomposition parameters. Research in this direction can About the authors be quite promising. Those, we can thereby reduce the dimension of the initial data set, which will also allow us Zakharova Alena A., Doctor of Engineering Science, to apply traditional decision-making methods, and the Keldysh Institute of Applied Mathematics, Moscow and comparison of criteria, in this case, can be based on the professor of the department β€œComputer Science and Software” considered visualization methods or supplemented by of Bryansk State Technical University, Bryansk, Russia. E- mail: zaa@tu-bryansk.ru. them. Korostelyov Dmitriy A., Candidate of Engineering Science, Keldysh Institute of Applied Mathematics, Moscow Acknowledgments and associate professor of the department β€œComputer Science The work was supported by Russian Science and Software” of Bryansk State Technical University, Bryansk, Foundation grant β„– 18-11-00215. Russia. E-mail: nigm85@mail.ru.