=Paper=
{{Paper
|id=Vol-2845/Paper_15.pdf
|storemode=property
|title=A Technique for Structuring of Group Expert Judgments Formed Under Complex Forms of Ignorance
|pdfUrl=https://ceur-ws.org/Vol-2845/Paper_15.pdf
|volume=Vol-2845
|authors=Igor Kovalenko,Yevhen Davydenko,Alyona Shved
|dblpUrl=https://dblp.org/rec/conf/iti2/KovalenkoDS20
}}
==A Technique for Structuring of Group Expert Judgments Formed Under Complex Forms of Ignorance==
https://ceur-ws.org/Vol-2845/Paper_15.pdf
A Technique for Structuring of Group Expert Judgments Formed
Under Complex Forms of Ignorance
Igor Kovalenko, Yevhen Davydenko and Alyona Shved
a
Petro Mohyla Black Sea National University, 68 Desantnykiv street, 10, Mykolaiv, 54003, Ukraine
Abstract
The main provisions of the technique of structuring the aggregated expert assessments in
order to synthesize group opinion, which are formed under the influence of two or more types
of ignorance, generated by uncertainty, inaccuracy, inconsistency, contradiction, has been
proposed. To synthesize mathematical models for their implementation, the mathematical
notation of the theory of evidence (DST) and the theory of plausible and paradoxical
reasoning (DSmT) is used. Different quantified measure of uncertainty level in DST has been
considered. A procedure for selecting the technique (rule) for aggregation expert judgments
formed in the frame of Dempster-Shafer’s model, based on quantitative measures of
uncertainty, has been proposed. The proposed procedure allows to obtain the aggregated
probability masses that provides the lowest achievable uncertainty level.
Keywords 1
Information technology, ignorance, decision-making, expert evidence, combination rule
1. Introduction
The theory of choice and decision-making investigates mathematical techniques and models for
organize the processes of synthesis of optimal strategic and managerial decisions in social, economic,
technical, organizational and other systems. In practice, making effective decisions is impossible
without the experience and knowledge of specialists (experts).
Current trends in the development of information technologies, striving to obtain the results of
expert surveys, processing and analysis of expert assessments with a higher quality, while reducing
the time allotted for making a decision, contribute to the complexity of the examination tasks. The
choice and making of optimal and effective decisions for solving complex problems becomes much
more complicated in situation of multi-criteria, multi-alternativeness, especially when solving semi-
structured (mixed) and unstructured tasks, i.e. such tasks in which qualitative (not or partially
formalized), little-known, uncertain factors prevail, especially if there is a tendency to increase their
number. The situation is aggravated by the presence of ignorance of various nature, which has a
negative impact on the processes associated with the acquisition and analysis of initial data
(statistical, analytical, expert information). In such situation, a person cannot, at the heuristic level,
guarantee an effective decision-making taking into account all the conflicting factors that affect the
achievement of the goal of the problem under consideration.
This, in turn, creates the preconditions for the synthesis of a complex of formalized mathematical
models focused on the intellectual support of the decision-making processes under complex forms of
ignorance, multi-criteria and multi-alternativeness.
The purpose of the research is to present and formalize the main ideas of the technology for
structuring expert knowledge under complex (combined) types of ignorance caused by uncertainty,
inaccuracy, inconsistency or / and contradiction of expert knowledge. The basis of which is the
procedure for identifying different forms of ignorance, their possible combinations, as the basis for
the selection and application of methods for the analysis of expert assessments, which make it
IT&I-2020 Information Technology and Interactions, December 02–03, 2020, KNU Taras Shevchenko, Kyiv, Ukraine
EMAIL: ihor.kovalenko@chmnu.edu.ua (I. Kovalenko); davydenko@chmnu.edu.ua (Y. Davydenko); avshved@chmnu.edu.ua (A. Shved)
ORCID: 0000-0003-2655-6667 (I. Kovalenko); 0000-0002-0547-3689 (Y. Davydenko); 0000-0003-4372-7472 (A. Shved)
© 2020 Copyright for this paper by its authors.
Use permitted under Creative Commons License Attribution 4.0 International (CC BY 4.0).
CEUR Workshop Proceedings (CEUR-WS.org)
149
possible to correctly operate with the analyzed set of initial data in the conditions of the revealed type
of ignorance.
2. Basics of Dempster-Shafer theory and Dezert-Smarandache theory
To solve the problem of group expert assessments structuring under complex (combined) types of
ignorance such as uncertainty, inaccuracy, inconsistency and conflict, effective results can be
obtained by using the DST [1, 2, 6, 22] and DSmT [26, 27] models and techniques.
The DST techniques allows to handle correctly with expert preferences formed under conditions of
uncertainty, inconsistency (medium level) and allows to correctly operate with possible ways of
intersection and union of expert evidence which can be formed in the processes of identifying and
analyzing of expert date and knowledge. The DSmT is an extension of the notation of the DSmT, and
allows to operate with deeper forms of ignorance, in particular with combinations of uncertainty and
inaccuracy, uncertainty and non-specificity, etc.
Let a group of experts {E j | j 1, t} , evaluating some initial set of objects {Aj | j 1, m}
frame of discernment), have formed profiles of expert preferences {B j | j 1, t} , each
B j {X l | l 1, s} , X l , reflects the preferences (choice) of the expert Ej and satisfies one of the
following systems of rules.
1. The initial data set A is a set of the mutual exclusion and exhaustive elements (DS model).
In this case, Bj is a 2А-dimensional vector, each element of which is obeyed the next
system of rules [1, 2, 6, 22]:
1. X l {} ;
2. X l { Ai } ;
(1)
3. X l {Ai | i 1, k} , k < n;
4. X l { Ai | i 1, n} .
2. The initial data set A is a set of the mutual exhaustive elements (DSm model). In this case,
Bj is a DА-dimensional vector, each element of which is obeyed the next system of rules
[26, 27]:
1. Conditions, that match (1).
(2)
2. If X l , X k D , than X l X k D and X l X k D .
For each Bj, j 1, t , a vector m j {ml | l 1, s} will be obtained whose elements (basic
probability assignment, bpa) satisfy the conditions:
0 m( X l ) 1, ( X l ), m() 0, m( X ) 1 ,
X l
j (3)
where Λ corresponds to 2А, or DА respectively.
3. Measures of uncertainty in DST
There are two main types of uncertainty: non-specificity or imprecision, and conflict in the theory
of evidence.
The first of them (non-specificity) allows to determine how the bpa’s of corresponding focal
elements are imprecise, and is directly related to the cardinality of the formed focal elements. The
non-specificity manifests itself in a situation when several elements of the frame of discernment are
not defined (not specified).
Hartley's weighted entropy allows to quantify the degree of non-specificity [7]:
X j , X j
N (m) m( X j )log 2 X j , 0 N (m) log 2 . (4)
F. Smarandache, A. Martin and C. Osswald [28] are introduced the concept of the "degree of
specificity" S (m) [0,1] :
150
S (m) 1 d (m, mS ) , m d (m, m X ) d (m, mY ), m( X ) m(Y ) , (5)
where as a metric d (m, mS ) can be used any measure that characterizes the distance between the
selected groups of evidence; the values of ms satisfy the conditions:
m( X )
mS ( X max ) 1 , X max arg max , X 2 , X
A
(6)
X
The second type of uncertainty (conflict) allows to identify and quantify the discrepancy
(contradiction) both within the group of evidence and between several groups of evidence.
There are two types of conflict in DST: internal or auto-conflict and general or global conflict.
Auto-conflict is a type of conflict that occurs only within a group of evidence [8, 20]. Global conflict
indicates inconsistency among selected groups of evidence (represented by m1 and m2), and includes
both inconsistency within individual evidence (related to individual m-functions) and inconsistency
between selected groups of evidence (between m1 and m2) [9].
Conflict is characterized by differences in the selection and evaluation of the elements of the frame
of discernment, and may be the result of the confusion, dissonance, discord and strife [10, 14, 15, 31]:
Conf (m) m( X j )log 2 f ( X j ) , (7)
X j 2
where the function f(Xj) can take the values of the functions Bel (C ) m( X j ) , Pl (C ) m( X j ) ,
X j C , X j 2 X j C , X j 2
X j Xi
betP(C ) CX j
X j m( X j ) , or Con( X j ) m( X i )
Xj
.
X j 2 , X j X i 2
Global uncertainty by G. Klir and B. Parviz [14] is the sum of its components: conflict and non-
specificity:
T (m) Conf (m) N (m) , (8)
where N(m) is Hartley's weighted entropy (4); Conf(m) is a conflict measure (7).
The contradiction of the group of evidence can be defined as a weighted contradiction of all focal
elements of the group of evidence [28], Contrm [0,1] :
Contrm m( X j )d (m, mX j ) ,
(9)
X j 2
1, i j, i 1, k ;
where X i , X j : mX j ( X i )
0, i j.
4. The technique of structuring the aggregated expert assessments formed
under complex forms of ignorance
The proposed technology is designed to solve the problem of analysis (ranking, clustering, ranking
clusters) of group expert assessments under multi-criteria, multi-alternativeness and complex forms of
ignorance (uncertainty, inaccuracy, inconsistency, conflict) in order to synthesis a final (aggregated)
expert assessment.
The generalized scheme of the proposed information technology of structuring the expert date and
knowledge in the context of complex forms of ignorance and synthesis of group decisions is shown in
Fig. 1.
Let's consider the main ideas of the proposed technology. Let { Ai | i 1, n} be a set of
alternatives, on which certain limitations can be imposed: mutually exclusive and / or mutually
exhaustive elements, which determines the type of model in frame of which expert evidence will be
formed.
Let a group of experts {E j | j 1, t} have formed profiles of expert preferences
{B j | j 1, t} on the set { Ai | i 1, n} . The profile Bj formed by the expert Ej reflects his
151
preferences regarding all analyzed elements of the set A, and corresponds to one of the systems of
rules (1) or (2), respectively (depending on the chosen analysis model).
Figure 1: A structure of information technology for the analysis of expert assessments formed under
complex forms of ignorance
152
For each expert the same instruction has been presented, which prescribes what they should do
with the set A. The instruction contains information about a scale measurement type, within which
experts express their preferences, which in turn affects the information received from experts (words,
conditional gradations, numbers, rankings, binary relations or other objects of non-numerical nature).
The profile Bj formed by the expert Ej reflects his preferences, expressed within a given scale, with
respect to the elements of the set A. The expert himself decides which elements (or selected groups of
elements) of the set A will be evaluated. Thus, the profile of preferences Bj formed by Ej may contain:
estimates expressed with respect to all elements of the set A; the assessments expressed regarding the
preferred elements of the set A; the estimates expressed regarding the selected groups of preferred
elements of the set A.
Next, the set of expert assessments {Bi | i 1, n} is fed to the input of the block of ignorance
nfi identification, nf i NF , in this case we are talking about such types of ignorance nfi as
uncertainty, inaccuracy, inconsistency, conflict or their combinations that can be simultaneously
present in the knowledge system. In the ignorance identification block, a system of identification
criteria Ci {c(ji ) | j 1, z} , i 1, p , of the analyzed forms of ignorance NF {nfi | i 1, p} is formed.
On the basis of formed Ci {c(ji ) | j 1, z} a system of decision rules SRi {Rl(i ) | l 1, h} for nfi
identification is developed. For nfi identification it can be used one or combination of features, which
allows to unambiguously establish the presence of nfi in the initial data (knowledge) set. The absence
of nfi is recognized if for all set of proposed ignorance identification criteria Ci confirmed absence of
nfi ( j : c (ji ) absence of nf i ; ); the presence of nfi is recognized if there is at least 1 criterion c (ji )
(from a given set Ci) signals the presence of nfi ( j : c (ji ) nf i ).
First, it is necessary to form a set of criteria Ci {c(ji ) | j 1, z} , which, in turn, are considered as
indicators of the presence of ignorance in the analyzed data (knowledge) set.
For identification the above forms of ignorance (and their combinations), it is proposed to use the
following features:
1. The structure of expert evidence.
2. The level of conflict.
3. Indicators of the quality of the received evidence: level of auto-conflict; the degree of
specificity of the generated evidence, etc.
4. The degree of inconsistency of the formed expert evidence.
5. Limitations which are imposed on the frame of discernment A.
The next step is to form a system of decision rules SRi {Rl(i ) | l 1, h} for analyzed forms of
ignorance identification.
Based on the formed decision rules Rl(i ) , a rule for choosing a method for modeling the above
forms of ignorance (and their combinations) can be obtained:
P , if l : Rl(i ) absence nf i ;
Bj 1
P2 , if l : Rl nf i ;
(i )
where P1 indicates that expert evidence are no contradict, have a high (acceptable) quality, and
consistent; P2 indicates that expert evidence have a high (not acceptable) level of conflict.
If B j P1 , then it is assumed that the expert evidence are consistent (they are characterized by
close evidence, the presence of a low / insignificant level of conflict), and may indicate a high
(acceptable) quality of expert evidence.
In this case, combination techniques (rules) can be used to find the aggregated expert judgments
formed in frame of DST or DSmT data-model [1, 13, 19, 21, 24-27, 29, 32, 33]. The algorithm for the
complex use of the combination techniques (rules) for finding group solution by combining expert
assessments formed in frame of DST model has been proposed. A schematic generalized algorithm
for choosing a combination rule is shown in Fig. 2.
153
Figure 2: A general algorithm of combination techniques (rules) choice
154
Let a set of {Pi | i 1, k} potential combination rules be given. It is proposed to choose a rule
P , mcombP mi P m j , that minimizes the value of the total uncertainty of the combined bpa’s
min T (mcombP ) . Formally, the procedure for choosing a combination rule can be represented as next
successive stages. At the first stage, from the set of available combination rules {Pi | i 1, k} , a
subset is selected that satisfies a set of specified criteria С {сi | i 1, q} .
The data model (DST or DSmT model), expert’s competence coefficients, conflict level
information, ignorance level, degree of interaction and the structure of expert evidence can be
considered as criteria for the choice of combination techniques. Recommendations for the choice of
combination rules based on the analysis of a number of criteria are given in [1, 21, 24, 25]. As a
result, the initial set {Pi | i 1, k} will be narrowed down to subset {Pi | i 1, z} , z k , that
obtained by excluding from the set P, rules that do not satisfy the formed set of criteria
The choosing the combination rule based on the analysis of quantitative characteristics of
uncertainty has done on the second stage.
At first, the combination rule Pl is selected that maximizes the value of measure (5) that
reflects the degree of specificity of the combination result max( S (mi Pl m j ) ),. S (mi Pl m j ) 1 .
Next, the combination rule Pr is selected that minimizes the value of the measure (9) that
reflects the degree of contradiction of the combination result
min( Contr(mi Pr m j ) ), Contr(mi Pr m j ) 0 .
If Pl Pr , then a combination rule is selected that satisfies the following condition:
Pl , T m i Pl m j T m i Pr m j ;
P
Pr , T m i Pl m j T m i Pr m j .
(10)
where T is a measure of global uncertainty calculated by eq. (8).
If B j P2 , then it was revealed that there is inconsistency (conflict) of expert evidence, which
indicates the presence of several subgroups of experts with similar assessments, or the presence of so-
called dissident experts (one or more experts with estimates significantly different from those of the
main group).
As a result, three tasks arise:
1. Identification and exclusion of conflicting (contradictory) evidence (experts).
2. Partitioning (clustering) of the original set of expert evidence into homogeneous (with an
acceptable conflict level) subgroups.
3. Aggregation of conflicting (contradictory) evidence in order to find a group assessment.
The solution to the first task lies in the use of different approaches (techniques, measures) allowing
to quantify similarities and differences of expert opinions. These techniques use different distance
metrics [3, 4, 11, 12, 30] and allow to calculate the level of conflict between the focal elements of
several groups of evidence [8-10, 14, 15, 20, 31], for example, the degree of conflict between an
expert and the rest of t-1 experts [20]. In this case, both the nature of the selected subsets of frame of
discernment (including singletons) and the values of the obtained bpa’s are taken into account.
For example, let {a, b, c, d} be a frame of discernment, thus
Case 1: the evidence of experts
E1 : m{a} 0.1; m{b} 0.9 ;
E2 : m{a} 0.9 ; m{b} 0.1 ;
are contradictory (the same elements of the frame of discernment are evaluated, but they are assigned
conflicting bpa’s);
155
Case 2: the evidence of experts
E1 : m{a} 0.4 ; m{b} 0.6 ;
E2 : m{c} 0.6 ; m{d} 0.4 ;
are also contradictory (there are no common assessed elements of frame of discernment; when the
evidence is combined, their intersections will give ).
To solve the second task in [16, 17, 23] has been proposed a procedure for structuring group
experts evidence formed under uncertainty and inconsistency, which allows to split the original set of
experts evidence into subgroups E {G1}, {G2} ,…, {Gq}, …, {Gp} (Gq E, {Gq} = {E1,…,Er},
t ≥ r ≥ 1, t ≥ p ≥ 1), which are characterized by agreed expert evidence. And determine such Ej that do
not belong to any group (Ej Gq, provided that |Gq| = 1). The judgments of such experts Ej are
significantly different from those of the main group. Further, within each of the formed subgroups,
aggregate group estimates can be obtained. Provided that p = 1 (and, as a consequence, t = r), the
opinions of the entire group E are considered consistent. If a tendency pt and r1 arises, further
analysis is inappropriate.
To solve the third task, it is suggested to use one of the proportional conflict redistribution rules.
Each of which (depending on the rule) is based on different approaches aimed at redistributing the
local or general mass of bpa’s on the subsets involved in the conflict. Thus, providing mechanisms for
combining evidence with a very high level of conflict. In [24-27] notes that PCR5 is the only rule
whereby the shares of each local conflict bpa’s are redistributed to the subsets involved in the conflict
in proportion to the relevant bpa’s for these subsets. This technique makes it possible to achieve the
most correct redistribution of the local conflict bpa’s, but it entails certain computational difficulties.
The next step is the choice of the mathematical formalism for structuring expert data and
knowledge formed under identified type of ignorance. If the absence of ignorance nfi is confirmed,
then the task of expert judgments structuring is reduced to solving the problem of finding the
aggregated (generalized) expert judgments. If the analysis reveals the presence of nfi (in this case the
initial data set is characterized by inconsistency), then the task of expert judgments structuring is
reduced to solving the problem of partition the initial data set into several clusters of experts with
close (agreed, consistent, homogenous) evidence, for their subsequent analysis and search for an
aggregated estimate within each of the selected subgroups.
In the case when the selection of subgroups of experts and the search for aggregated assessments
within the selected subgroups is not acceptable, it is advisable to determine the reason for the spread
of expert assessments, identify experts whose assessments violate the consistency of the general set of
evidence, and conduct a second survey (possibly with adjustments to the composition of the expert
group, changing the expert survey procedure, etc.) in order to obtain high quality expert evidence.
The result of the processes occurring in this block is the information prepared (structured) for
decision making, which meets the set goals of the analysis.
The final stage is the interpretation of the results of structuring and the development of a group
solution.
5. Conclusion
The main ideas of the technology of structuring the aggregated expert assessments formed under
multi-criteria, multi-alternativeness and specific types of ignorance (caused by uncertainty,
inaccuracy, inconsistency, conflict / contradiction) has been proposed.
The proposed technology is a set of systematized techniques and methods of intellectual support of
decision making processes in various spheres of human activity.
This creates the basis for the development of integrated information technologies for intellectual
support of the decision-making process, which allow solving typical decision-making problems in
various subject areas, taking into account the type of expert measurements scale, the method of
156
modeling the revealed type of ignorance, the form of presentation of expert judgments (crisp, interval,
fuzzy) and the complex of modeled types of ignorance.
A procedure for evidence combination rule selection has been proposed. For each pair of
combined expert judgments a rule is selected that minimizes the value of the degree of inconsistency
and maximizes the value of the degree of specificity of the result of combination.
6. Acknowledgements
The proposed mathematical models and approaches were obtained as a result of research supported
by the state research project: “Development of modern information and communication technologies
for the management of intellectual resources to decision-making support of operational management”
(research project no. 0121U107831, financed by the Government of Ukraine).
7. References
[1] K. K. Annamdas, Evidence based uncertainty models and particles swarm optimization for
multiobjective optimization of engineering systems, Ph.D. thesis, University of Miami, Coral
Gables, Florida, 2009.
[2] M. J. Beynon, B. Curry, P. Morgan, The Dempster-Shafer theory of evidence: an alternative
approach to milticriteria decision modeling. Omega 28. 1 (2000), pp. 37–50.
DOI:10.1016/S0305-0483(99)00033-X.
[3] A. Bhattacharyya, On a measure of divergence between two statistical populations defined by
their probability distribution. Bulletin of the Calcutta Mathematical Society 35 (1943), pp. 99–
110.
[4] F. Cuzzolin, A geometric approach to the theory of evidence. Transactions on Systems, Man, and
Cybernetics. Part C: Applications and Reviews 38. 4 (2007), pp. 522–534.
DOI:10.1109/TSMCC.2008.919174.
[5] Y. Davydenko, I. Kovalenko, A. Shved, The basic concepts of the normative theory of the
synthesis of information technologies for decision support, in: Proceedings of the 15th
International conference "Computer sciences and Information technologies", CSIT 2020,
Zbarazh, Ukraine, 2020.
[6] A. P. Dempster, Upper and lower probabilities induced by a multivalued mapping. Annals of
Mathematical Statistics 38. 2 (1967), pp. 325–339. DOI:10.1214/aoms/1177698950.
[7] D. Dubois, H. Prade, A note on measures of specificity for fuzzy sets. International Journal of
General Systems 10. 4 (1985), pp. 279–283. DOI:10.1080/03081078508934893.
[8] T. George, N. R. Pal, Quantification of conflict in Dempster-Shafer framework: a new approach.
International Journal Of General System 24. 4 (1996), pp. 407–423.
DOI:10.1080/03081079608945130.
[9] Y. He, Uncertainty quantification and data fusion based on Dempster-Shafer theory, Ph.D. thesis,
Florida State University, Tallahassee, Florida, 2013.
[10] U. Hohle, Entropy with respect to plausibility measures, in: Proceedings of 12th IEEE
International Symposium on Multiple-Valued Logic, MVL 82, Paris, France, 1982, pp. 167–169.
[11] A. L. Jousselme, D. Grenier, E. Boss´e, A new distance between two bodies of evidence.
Information Fusion 2. 2 (2001), pp. 91–101. DOI:10.1016/S1566-2535(01)00026-4.
[12] A. L. Jousselme, D. Grenier, E. Boss´e, Analyzing approximation algorithms in the theory of
evidence. Sensor Fusion: Architecture, Algorithms and Applications VI. 4731 (2002): 65–74.
[13] T. Inagaki, Interdepence between safety-control policy and multiple-sensor schemes via
Dempster-Shafer theory. Transaction on Reliability 40. 2 (1991), pp. 182–188.
[14] G. J. Klir, B. Parviz, A note on measure of discord, in: Proceedings of the 8th Conference on
Uncertainty in Artificial Intelligence, UAI’92, Stanford, CA, USA, 1992, pp. 138–141.
DOI:10.1016/B978-1-4832-8287-9.50023-2.
157
[15] G. J. Klir, A. Ramer, Uncertainty in the Dempster-Shafer theory: a critical re-examination.
International Journal of General System 18. 2 (1990), pp. 155–166. DOI:
10.1080/03081079008935135.
[16] I. I. Kovalenko, A. V. Shved, Clustering of group expert estimates based on measures in the
theory of evidence. Naukovyi Visnyk Natsionalnoho Hirnychoho Universytetu 4. 154 (2016), pp.
71–78.
[17] I. Kovalenko, Y. Davydenko, A. Shved, Formation of consistent groups of expert evidences
based on dissimilarity measures in evidence theory, in: Proceedings of the 14th International
conference "Computer sciences and Information technologies", CSIT 2019, Lviv, Ukraine, 2019,
pp. 113–116. DOI: 10.1109/STC-CSIT.2019.8929858
[18] I. Kovalenko, Y. Davydenko, A. Shved, K. Antipova, Methodology for the synthesis of
information technologies for ignorance modeling: the key concepts, in: Proceedings of the 1st
International Workshop "Information-Communication Technologies & Embedded Systems",
ICT&ES 2019, Mykolaiv, Ukraine, 2019, pp. 233–240.
[19] I. Kovalenko, Y. Davydenko, A. Shved, Searching for Pareto-optimal solutions, in:
N. Shakhovska, M. Medykovskyy (Eds.), Advances in Intelligent Systems and Computing IV,
CCSIT 2019, Advances in Intelligent Systems and Computing 1080, Springer, Cham, 2020,
pp. 121–138. DOI:10.1007/978-3-030-33695-0_10.
[20] A. Martin, A. L. Jousselme, C. Osswald, Conflict measure for the discounting operation on belief
functions, in: Proceedings of the 11th International Conference on Information Fusion, FUSION
2008, Cologne, Germany, 2008, pp. 1–8.
[21] K. Sentz, S. Ferson, Combination of evidence in Dempster-Shafer theory, Technical Report
SAND 2002-0835, Sandia National Laboratories, Albuquerque, NM, 2002.
[22] G. Shafer, A mathematical theory of evidence, Princeton University Press, Princeton, 1976.
[23] A. Shved, I. Kovalenko, Y. Davydenko, Method of detection the consistent subgroups of expert
assessments in a group based on measures of dissimilarity in evidence theory, in: N. Shakhovska,
M. Medykovskyy (Eds.), Advances in Intelligent Systems and Computing IV, CCSIT 2019,
Advances in Intelligent Systems and Computing 1080, Springer, Cham, 2020, pp. 36–53.
DOI:10.1007/978-3-030-33695-0_4.
[24] F. Smarandache, An in-depth look at information fusion rules and the unification of fusion
theories. Computing Research Repository (CoRR) cs.OH/0410033 (2004).
[25] F. Smarandache, Unification of Fusion Theories (UFT). International Journal of Applied
Mathematics and Statistics 2 (2004): 1–14.
[26] F. Smarandache, J. Dezert, Representation of DSmT, in: Advances and Applications of DSmT
for Information Fusion, volume 1, American Research Press, Rehoboth, 2004, pp. 3–35.
[27] F. Smarandache, J. Dezert, Advances and applications of DSmT for information fusion.
Collected works, volume 2, American Research Press, Rehoboth, 2006.
[28] F. Smarandache, A. Martin, C. Osswald, Contradiction measures and specificity degrees of basic
belief assignments, in: Proceedings of the 14th International Conference on Information Fusion,
FUSION 2011, Chicago, USA, 2011, pp. 475– 482.
[29] Ph. Smets, The combination of evidence in the transferable belief model. Pattern analysis and
Machine Intelligence 12. 5 (1990), pp. 447–458. DOI:10.1109/34.55104.
[30] B. Tessem, Approximations for efficient computation in the theory of evidence. Artificial
Intelligence 61 (1993), pp. 315–329. DOI:10.1016/0004-3702(93)90072-J.
[31] R. R. Yager, Entropy and specificity in a mathematical theory of evidence. International Journal
of General Systems 9. 4 (1983), pp. 249–260. DOI:10.1080/03081078308960825.
[32] R. R. Yager, On the Dempster-Shafer framework and new combination rules. Information
Science 41. 2 (1987), pp. 93–137. DOI:10.1016/0020-0255(87)90007-7.
[33] L. Zhang, Representation, independence and combination of evidence in the Dempster-Shafer
Theory of Evidence, in: M. Fedrizzi (Ed.), Advances in the Dempster-Shafer Theory of
Evidence, John Wiley & Sons, New York, 1994, pp. 51–69.
158