=Paper=
{{Paper
|id=Vol-2763/CPT2020_paper_s4-2
|storemode=property
|title=Visualizing Methods of Multi-criteria Alternatives for Pairwise Comparison Procedure
|pdfUrl=https://ceur-ws.org/Vol-2763/CPT2020_paper_s4-2.pdf
|volume=Vol-2763
|authors=Alena Zakharova,Dmitriy Korostelyov
}}
==Visualizing Methods of Multi-criteria Alternatives for Pairwise Comparison Procedure==
https://ceur-ws.org/Vol-2763/CPT2020_paper_s4-2.pdf
Visualizing methods of multi-criteria alternatives for pairwise
comparison procedure
A.A. Zakharova1,2, D.A. Korostelyov1,2
zaa@tu-bryansk.ru | nigm85@mail.ru
1
Keldysh Institute of Applied Mathematics Russian Academy of Sciences, Moscow, Russia
2
Bryansk State Technical University, Bryansk, Russia
The article deals with the problem of choosing a preferred alternative in a pairwise comparison procedure. The difficulties of
applying this procedure in a case of using alternatives with a large number of criteria are noted. It is proposed to supplement the
procedure of expert pairwise comparison with visualization tools of multi-criteria alternatives. The paper considers several visualization
methods for multi-criteria alternatives for pairwise comparison procedures: histograms, two-dimensional graphs, three-dimensional
surfaces, probability distribution diagrams, visualization based on modifications of radar and radial diagrams, as well as combined
methods. It described an experimental study of the application of the considered method for the task of determining the preferred
alternative by the example of choosing one of two OpenFoam solvers (rhoCentralFoam and pisoCentralFoam), with the help of which
estimates of the accuracy of calculating the inviscid flow around a cone were obtained. Πach solver is characterized by 288 criteria. It
is shown that the use of some of the methods considered does not make it possible for the expert to make a choice. In this case, a good
result was obtained using methods for constructing three-dimensional surfaces, probability distribution diagrams, as well as using the
combined method based on modified radar diagrams. It is concluded that the rhoCentralFoam solver is more preferable if there are no
additional criteria for ranking the criteria. The possibility of using the combined method in combination with the ranking procedure of
criteria (or their groups) during decision-making is also noted.
Keywords: multi-criteria alternatives, visual pair comparison, alternatives visualization, decision support.
when comparing the site design at the stage of its design
1. Introduction [6].
In decision theory, one of the basic tasks is ranking Thus, we have the task of visualizing data sets
alternatives [1, 2]. This allows setting their priority in characterizing alternatives. Consider and analyze several
relation to the task at hand. There are various methods approaches and methods that can be applied to visualize
that allow such ranking, some of which are expert. multi-criteria alternatives.
Among the expert ranking methods, the method of
2. Visualization methods
pairwise comparisons [3] has proven itself well, the
essence of which is to provide the expert alternately with Data preparation
pairs of alternatives for comparison, during which he
prefers one of them. In particular, this procedure used in The visualization procedure begins with a step
one of the classical decision-making methods β the requiring initial data preparation.
hierarchy analysis method developed by T. Saati [4], in 1. All input values should be given to a numeric format
which it is necessary to construct matrices of pairwise - relevant methods of decision theory may be used
comparisons for all levels of the hierarchy. for these purposes [7].
The use of pairwise comparisons is usually effective 2. Normalization of data per segment [0; 1] taking into
in cases where each alternative is well reflected in the account the direction of the criterions optimization
expertβs perception or is characterized by a small number (maximization or minimization). For these purposes,
of criteria (usually no more than 10) [5]. In the case when the formulas can be used:
β² π£π£ππ,ππ βπ£π£ππππππ,ππ
an expert needs to compare new for him alternatives with β’ π£π£ππ,ππ = β in case of maximization;
π£π£ππππππ,ππβπ£π£ππππππ,ππ
a large number of criteria, this can cause difficulties for β² π£π£ππππππ,ππβπ£π£ππ,ππ
him. Therefore, in such situations, it is necessary to use β’ π£π£ππ,ππ = β in case of minimization,
π£π£ππππππ,ππβπ£π£ππππππ,ππ
additional tools, for example, reducing the dimension, or where i β alternative number (1 β€ ππ β€ ππ), j β criteria
statistical processing of criteria values. However, even number (1 β€ ππ β€ πΎπΎ), N β count of alternatives, K β count
applying these approaches, there is still a chance of not of criteria, vmax,j, vmin,j β maximum and minimum possible
getting the desired result. For example, in the case of value of j-th criteria. Values vmax,j, vmin,j usually
calculating statistical characteristics, we can get determined on the basis of their physical meaning.
conflicting data in a situation where the mathematical However, if there are problems with their definition, then
expectation for an alternative is better, but the variance is they can be calculated by the formulas:
worse. Therefore, additional tools are needed that could π£π£ππππππ,ππ = minοΏ½π£π£ππ,ππ οΏ½,
help the expert decide.
One such methods may be visual analytics - when for π£π£ππππππ,ππ = maxοΏ½π£π£ππ,ππ οΏ½.
each alternative a corresponding visual image is
constructed that characterizes the set of values of its After the data is prepared, you can proceed to
criteria. Visualization is able to present the alternative as visualize them. For these purposes, several different
a holistic image, and it will be easier for an expert to approaches and methods can be applied, in each of which
make his choice with its help. It should be noted that the we will consider the visualization of two alternatives and
visual comparison of alternatives is currently already the features of their visual pairwise comparison.
being applied and shows a good result, for example,
Copyright Β© 2020 for this paper by its authors. Use permitted under Creative Commons License Attribution 4.0 International (CC BY
4.0)
Histograms in the criteria that the expert can undoubtedly notice with
a pairwise comparison.
One of the most accessible and simple methods of
In addition, when comparing two histograms, the
visualization of multidimensional data is diagrams, and
expertβs perception can be significantly affected by the
among them, the most accessible in the procedure of
order of columns (criteria), therefore, when using this
pairwise comparison can be called histograms. In this
approach, it is appropriate to think over the ordering or
case, two main approaches to their application can be
grouping of criteria based on their features, for example,
distinguished.
by a degree of importance.
1. Both alternatives are shown on a common
In addition to using this visualization method in the
histogram. (Fig. 1). In addition, you can use the
pairing comparison procedure, it can also be useful in
option of overlapping columns to focus on the
ranking the criteria, as well as grouping them. This
deviation of values by criteria.
makes it possible to use this method as a preparatory
stage for the construction and comparison of visual
images of alternatives.
Two-dimensional graphics and three-
dimensional surfaces
Other common data visualization methods are two-
dimensional graphs and three-dimensional surfaces.
Their application can be effective when there are areas
with a noticeable difference in the values of the criteria,
and due to interpolation, these areas are more
pronounced.
In the case of two-dimensional graphs, the X axis can
be interpreted as the serial number of the comparison
Fig. 1. Visual comparison of two alternatives in a common
criterion j (1 β€ ππ β€ πΎπΎ) and at Y axis normalized criterion
β²
histogram value β π£π£ππ,ππ (1 β€ ππ β€ 2) is plotted. If each criterion
corresponds to a numerical value (for example, time),
2. Each alternative is visualized on separate then it can also be used to determine the X coordinate
histograms. (Fig. 2). (only provided that all these values are pairwise
different) (Fig. 3).
Fig. 2. Visual comparison of two alternatives on separate
histograms
Note that in the 2nd approach, it is important to use
identical parameters of the diagrams (color, size, etc.) in
order to reduce the subjectivity of the perception of
visual images, describing the data sets of the Fig. 3. Visual comparison of two alternatives on a two-
corresponding alternatives, and to enable the expert to dimensional graph
focus on a holistic perception of visual images.
However, for the application of visualization based
When determining preferences among alternatives
on surfaces in three-dimensional space, a prerequisite is
based on visual images, an expert can be guided by his
the presence or ability to display a set of criteria on the
perception of the degree of filling of the columns of
axis X and Y: ππππ(ππ) = π₯π₯, ππππ(ππ) = π¦π¦, 1 β€ ππ β€ πΎπΎ. This
diagrams, including and the total area of all columns (the
mapping can be set based on the physical meaning of the
second approach is more suitable for these purposes) or
criteria, or by using the grouping of criteria (for example,
the subjectively averaged deviation of the criteria
using clustering methods) (Fig. 4).
columns of the two alternatives (the first approach is
more suitable for these purposes).
The use of histograms is justified for those situations
when it is necessary to see the whole alternative as a
whole, and the number of criteria is not too large (about
5-20). Moreover, the effectiveness of this visualization
method is additionally achieved if ranking approaches or
non-linear scales were used in the normalization process
since this allows you to get quite noticeable differences
Probability distribution diagram
Probability distribution diagrams can also be used as
another approach to visualizing data sets describing
alternatives. For these purposes, the interval [0; 1], to
β²
which all normalized values of the criteria (π£π£ππ,ππ ) belong,
on M equal intervals in length (for example, by 5 or 10)
depending on the number of criteria. These intervals are
located on the abscissa axis. After that, the number Π‘i,m
Fig. 4. Visual comparison of two alternatives using a surface (1 β€ ππ β€ ππ) of hits of the normalized criteria values for
in three-dimensional space each of the intervals is determined, which then allows
you to determine the corresponding probabilities: ππππ,ππ =
πΆπΆππ.ππ
Similar to histograms, visual images for the pairwise , used as values on the ordinate axis when plotting
πΎπΎ
comparison procedure in the case of two-dimensional (Fig. 5).
graphs and three-dimensional surfaces can also be As in previous approaches for this visualization
constructed both on one diagram, and on two method, it is also possible to build diagrams on a single
neighboring ones with the same parameters. diagram, or separately. The choice of preference in the
The expert can make his choice based on the area (for comparison of the probability distribution the expert can
surfaces) or the intervals (for curves) of the zones where give an alternative, which is characterized by the
one alternative prevails over another. Moreover, in the displacement of the probability distribution to the right
case of surfaces built in three-dimensional space, a (towards the interval with the highest criteria of values).
prerequisite is the availability of tools that allow you to This type of chart is conveniently displayed on a single
rotate and zoom in on the surface so that the expert can diagram while applying transparency of the columns so
choose the most suitable angles for comparison. Thus, it that the differences are accented (Fig. 5).
is important to provide an interactivity property. This The effectiveness of this approach becomes
method allows you to compare visual alternatives significantly higher when the number of criteria K is
entirely based on the subjective perception of the sufficiently large (for example, hundreds and thousands).
coverage area of one surface of another, and also with its With its help, not only visual images can be obtained, but
help you can determine the various relationships of the also quantitative probabilistic characteristics of the
groups of criteria that are used as the coordinates of the dominance of one alternative over another. Also, this
measurements along the abscissa and ordinates. method allows us to group criteria, but it does not allow
them to be ranked.
Fig. 5. Visual comparison of two alternatives using a probability distribution diagram
Λ sectors with radii proportional to the criteria of
Visual images based on radar and radial alternative (Fig. 6);
diagrams Λ sectors with radii proportional to the roots of the
As noted in [8] visual images of alternatives can also criteria of alternative (Fig. 7);
be built on the basis of methods based on the radar and Λ radar diagram with a permutation of criteria by
radial diagrams. The following visualization options are grouping large values side by side (Fig. 8);
proposed:
Λ radar diagram with a permutation of criteria, taking advantages and disadvantages, and often some of them
into account their alternation (alternately clockwise may not be useful in the visual comparison itself, but in
are the criteria with larger and smaller values) (Fig. the preparatory stage, the purpose of which is to
9). determine the order or grouping of criteria (as histograms
When using this method of visualization, the main and 3D surface), as well as the integral quantitative
emphasis is on the fact that the best alternative occupies characteristics of visual images - brightness, area of
a larger area, and also that the visual image is brighter prevalence, statistical characteristics (probability
due to the use of gradient fills (in the center, the color is distribution diagram), etc. And already these
more neutral β green, and on the periphery β more characteristics allow, for example, to set a specific order
contrast β red). of permutation and grouping of criteria during
visualization (radial and radar diagrams).
Thus, it is advisable to move from the task of
comparing a single visual object to the task of comparing
a group of visual objects that characterize an alternative
or its components. For this purpose, an integrated
Fig. 6. Visual comparison of two alternatives using a radial approach is proposed, consisting in the sequential
diagram with sector radii proportional to the values of the presentation of a series of visual images obtained using
criteria different methods, until the expert makes a choice.
For this expert to select the first represented whole
visual images. If with their help he cannot determine the
preferred alternative, then by means of visual analysis he
tries to identify groups of general criteria, and also, if
possible, to rank and filter them. Further, the selected
groups can be visualized separately and placed in a table
Fig. 7. Visual comparison of two alternatives using a radial grid - on the right are the images for the components of
diagram with sector radii proportional roots of values of the one alternative, and on the left for the other (Fig. 10).
criteria
Fig. 8. Visual comparison of two alternatives using a radar
diagram with a permutation of criteria by grouping large
values side by side
Fig. 9. Visual comparison of two alternatives using a radar
diagram with a permutation of criteria, taking into account
their alternation
If one of these visualization methods is used, the
expert, when paired, selects the alternative that seems to
him subjectively brighter and larger in area.
The methods described in [8] represent a more
universal visualization mechanism, because they allow
one to take into account the order and grouping of
criteria, and also for them integral quantitative Fig. 10. Visual comparison of two alternatives using the
integrated method
characteristics (brightness, area) can be determined.
Complex method In such a set of visual images, there is a high
probability that in a number of rows it will be possible to
When comparing alternatives by only one visual choose a preference. If for one alternative there are more
image, it is not always possible to choose the preferred such preferences than for another, then you can make a
one from them. This is because the comparison is usually choice in favor of this alternative.
based on the color, shape, area or volume of the visual
objects defining the respective alternatives. At the same
time, different visualization methods have different
3. Experiments well as defined for two norms (L1, L2) four parameters
(Ux, Uy, p, Ο). Analyzing this visual image (Fig. 12), one
Let us analyze the application of the considered
can notice that the blue color (rCF solver) prevails on the
methods on the example of the alternative (solvers)
surface over red (pCF solver), so the expert can choose
described in the works [9, 10]. As noted in [8], out of five
this alternative (rCF slover).
solvers, two give the best results β rhoCentralFoam and
pisoCentralFoam (rCF ΠΈ pCF). Given the fact that the
number of comparison criteria for these two alternatives
is quite large, we will use visual images built on different
diagrams. In Fig. 11 is a visual comparison using
histograms.
Fig. 12. Visual comparison of two solvers using surfaces
in three-dimensional space
Fig. 11. Visual comparison of two solvers using histograms
Fig. 13 shows the results of the visualization of
alternatives using a probability distribution diagram. To
For most experts, this comparison will not be
build it, we used a partition of the values of the criteria
unambiguous, because the images are very similar, and
at the same time on both diagrams, there are both areas into 10 intervals. In this diagram (Fig. 13), it can be noted
with the best values and the worst. that the blue color largely prevails in the columns [0.6;
We will get an approximately similar result when 0.7), [0.8; 0.9) and [0.9; 1.0], which correspond to the
using a two-dimensional graph, however, constructing a probability of falling into intervals with a higher value
(rank). This means that for the normalized values of the
surface in three-dimensional space can give a more
rCF solver, the probability of obtaining a better solution
interesting result. This method visualization is possible
because criteria can be grouped due to the fact that they is higher. Therefore, the choice of an expert, in this case,
were obtained during computational experiments by will most likely also be made in favor of this alternative
varying two parameters β angle Ξ² (in range 10-35Β° with (rCF solver).
step 5Β°) and Mach numbers (in range 2-7 with step 1), as
Fig. 13. Visual comparison of two solvers using a probability distribution diagram
Given the possibility of decomposing criteria into solver) is significantly preferable (due to a more uniform
subsets, we consider the use of complex visual images shape and a subjectively larger area of images). If we
based on petal families of diagrams. The decomposition assume that these criteria are peer-to-peer, then it will be
will be carried out based on various parameters (Ux, Uy, difficult for an expert to determine preference.
p, Ο) and norms (L1, L2), i.e. for each alternative, we However, if these criteria can be ranked (for example,
construct eight diagrams (Fig. 14). the criteria of the L1 block are preferable to the criteria of
Analyzing these visual images, it can be noted that the L2 block, or vice versa), then it will be easier for an
for the first four rows (L1 norm) due to a more uniform expert to make a choice because for this, it will suffice to
shape and subjectively somewhat larger area of images compare either only the upper images or only the lower
the second alternative (pCF solver) looks preferable, ones.
however, for the L2 norm (5-8 rows) first alternative (rCF
4. Conclusion References
The analysis of visualization methods of alternatives [1] CastaΓ±Γ³n-Puga M., Sanchez M., Aguilar L.,
for the pairwise comparison procedure showed that, RodrΓguez-DΓaz A. Applied Decision-Making.
depending on the properties of the source data and their Applications in Computer Sciences and
criteria, various approaches can be both effective and not. Engineering. β 2019. DOI: 10.1007/978-3-030-
Therefore, it is appropriate to attempt to use several 17985-4.
different visualization methods and their combination in [2] Alinezhad A., Khalili J. New Methods and
conjunction with a decomposition of the source data. In Applications in Multiple Attribute Decision Making
this case, it is possible on some methods to see that one (MADM). β 2019. DOI: 10.1007/978-3-030-15009-
alternative is better than another due to the subjective 9.
perception of the area of predominance, brightness, [3] David H.A. The Method of Paired Comparisons.
smoothness of forms, etc. (2nd edition, revised). β Oxford University Press,
Berlin. β 1988.
[4] Saaty, T.L., 1980. Analytic Hierarchy Process:
Planning, Priority Setting, and Resource Allocation.
β New York: McGraw-Hill. β 1980.
[5] Averchenkov V.I., Miroshnikov V.V.,
Podvesovskiy A.G., Korostelyov D.A. (2014) Fuzzy
and Hierarchical Models for Decision Support in
Software Systems Implementations. Proceedings of
the 11th Joint Conference, JCKBSE 2014 (eds. A.G.
Kravets et. al.), Communications in Computer and
Information Science, 466, Springer International
Publishing, pp. 410-421. DOI: 10.1007/978-3-319-
11854-3_35
[6] Practical application of the paired comparison
method. β URL: https://usabilitylab.ru/blog/boevoe-
primenenie-metoda-parnyix-sravnenij/.
[7] Figuera J., Greco S. and Ehrgott M. (Eds). Multiple
Criteria Decision Analysis: State of the Art Surveys.
β New York: Springer, 2005. β DOI:
10.1007/b100605.
[8] Zakharova A.A., Korostelyov D.A., Fedonin O.N.
Visualization Algorithms for Multi-criteria
Alternatives Filtering (2019). Scientific
Visualization 11.4: 66 - 80, DOI:
10.26583/sv.11.4.06
[9] Bondarev A.E., Kuvshinnikov A.E. Analysis of the
Accuracy of OpenFOAM Solvers for the Problem of
Supersonic Flow Around a Cone // ICCS 2018,
Lecture Notes in Computer Science (LNCS) 10862.
β P. 221β230, 2018. β DOI:10.1007/978-3-319-
93713-7_18.
[10] Bondarev A., Kuvshinnikov A. Comparative
Fig. 14. Visual comparison of two solvers using a series of
radar diagrams Estimation of QGDFoam Solver Accuracy for
Inviscid Flow Around a Cone // IEEE The
The greatest effect in the pairwise comparison Proceedings of the 2018 Ivannikov ISPRAS Open
procedure can be achieved by visualization of groups of Conference (ISPRAS-2018). β P. 82-87, 2018. β
initial criteria combining with a ranking of DOI: 10.1109/ISPRAS.2018.00019.
decomposition parameters. Research in this direction can
About the authors
be quite promising. Those, we can thereby reduce the
dimension of the initial data set, which will also allow us Zakharova Alena A., Doctor of Engineering Science,
to apply traditional decision-making methods, and the Keldysh Institute of Applied Mathematics, Moscow and
comparison of criteria, in this case, can be based on the professor of the department βComputer Science and Softwareβ
considered visualization methods or supplemented by of Bryansk State Technical University, Bryansk, Russia. E-
mail: zaa@tu-bryansk.ru.
them.
Korostelyov Dmitriy A., Candidate of Engineering
Science, Keldysh Institute of Applied Mathematics, Moscow
Acknowledgments
and associate professor of the department βComputer Science
The work was supported by Russian Science and Softwareβ of Bryansk State Technical University, Bryansk,
Foundation grant β 18-11-00215. Russia. E-mail: nigm85@mail.ru.