Multi criteria decision making involves evaluation of various alternative solutions upon a set of criteria. The result of multi criteria decision making is the best alternative which secures the highest score with the predefined criteria. Usually, these criteria are weighted in an order to represent their stake in the final selection. In multi criteria decision making this step is very critical for the selection of the right product. Often these criteria are traced backed to a set of multi-disciplinary stakeholders participating in the evaluation process. Often these stakeholders differ upon their weighting of a particular criteria with other stakeholders. In this paper we provide a holistic criteria weighting method which allows to assimilate the different criteria weights from different stakeholders to provide a single set of criteria weights, uniformly acceptable by all stakeholders. This paper shows how the criteria weights can be simulated and used priorly to find the design solutions. The simulations results can be used to accept or refute the particular design alternative.
Decision making is pervasive phenomenon in systems engineering (SE) projects. Some decisions involve single decision maker (DM). The responsibility of such a decision depends upon the sole single DM. Often, SE decisions involve more than one DM, thus adding fabric of complexity to the problem. Decision making becomes more complex, when multiple criteria get involved in the problem. Such problems, where multiple criteria and multiple decision makers are involved in a problem are called multi criteria decision making or analysis (MCDM/MCDA) problems.
A systems engineering project involves multiple stakeholders and multiple criteria decision analysis [1]. Such problems need attention and a formal methodology, to provide pertinent solutions. Variety of methodologies exist in the literature which provide more or less acceptable solutions, such as the famous AHP technique [2], Multiattribute utility theory [3], ELECTRE Methods [4–6], PROMETHEE [7], TOPSIS[8], etc. In this paper, we are concerned with only the first step of the decision making problem, i.e., criteria weighting. In this paper we present a methodology for criteria weighting in the context of a systems engineering projects. Our criteria weighting approach can provide weights for a variety of decision making techniques. Our approach is based on the classical preference modelling technique given by Fishburn et al. [9].
A systems engineering project typically involves a crowd of multi-disciplinary stakeholders. Success of a project depends upon the decisions and choices made during the analysis of architectures & alternatives, and during the selection of components during the detail design phase [10]. Systems engineering is about making the right decisions to achieve the development of a product which is exactly demanded by their clients and stakeholders. These right decisions do not come automatically from a thin air, rather precise metrics are needed to evaluate the appropriate alternatives and right design components. As these decisions are based on these multiple criteria, they need to be weighted to measure their impact on the final decision choice. This task of providing weights to the criteria may seem trivial for a single decision maker, but when multiple stakeholders are involved this task becomes fairly difficult. As the different stakeholders differ upon the weights for various criteria, while each stakeholder being correct in his own view.
It is interesting to see that industries seldom use techniques which demand high cognitive loads on DMs. Even if a techniques is more correct technically but leads to high cognitive load, it will hardly find usage in industry. Industries prefer simple techniques to pacify majority of its non-technical stakeholders. Our technique claims to pose less cognitive load to the stakeholders as compared to the other technique.
In this paper, we provide a technique which allows to assimilate the various criteria weights from the different stakeholders to provide a single array of criteria weights which can be uniformly accepted by all the different decision makers or stakeholders.
The major contributions of this paper are as follows: — It provide a holistic way to integrate the different criteria weights of different stakeholders to provide a single weights using the classical preference modelling. — It show that how all stakeholders are uniformly satisfied with the proposed technique.
This paper is organized as follows: Section 2 presents the criteria weighting problem. Section 3 presents the state of art of criteria weighting problems. Section 4 presents our novel approach. Section 5 presents an example of our approach. Section 6 discusses about the advantages and disadvantages of our approach, and the industrial readiness assessment. Section 7 concludes and presents future perspectives. 2
In a systems engineering project, it is of great importance that most of stakeholders are satisfied with the various decisions taken during the product development and with the final resulting end product. A higher satisfaction among the stakeholders can be guaranteed if the various stakeholders criteria weights are taken into account in a transparent and holistic manner.
The criteria weighting is critical part of the MCDM problem during analysis of alternatives in a system engineering project. Different stakeholders own different views towards the different criteria, their perception of weights differ from each other which makes it very tedious and difficult to come up with an agreement on a particular set of criteria weights. This often causes the conflicts among the stakeholders, which may halt the progress of the project, if it remains unresolved. The criteria weighting problem refers to the problem of generating a single array of criteria weights from multiple arrays of criteria weights emerging from different stakeholders. Often, such problems start with demanding the DMs’ weights for all the involved criteria and then provide a mechanism to find the mean weight. This approach assumes that the DMs’ know their particular weights, which is often not supported by any evidence. The criteria weighting problem faces four critical challenges: — Evidence of validity for criteria weights. — Transparency of DMs’ participation in criteria weighting. — Scalability of criteria weighting. — Acceptable Cognitive load on DMs.
Evidence of validity for criteria weights refers to the means which can prove that the assumed criteria weight is correct for a DM or to be able to reason why it is correct for a particular DM. Transparency of DM’ participation in criteria weighting is necessary to atleast make sure that the criteria weights are not illicitly decided in a manner to favour a particular DM’ preferences, or in other words to make sure that every participant DM is satisfied with his bit of contribution in the process. The scalability of the techniques for a sufficiently large number of criteria which may arise in a systems engineering project. In a systems engineering project the number of broad criteria hardly rise more than ten. These broad criteria later can be divided in to multiple criteria in next level. The criteria weighting technique should be able to address this hierarchy of criteria. The Fourth most important challenge is about the amount of cognitive load that a technique poses on the DM. If majority of DMs find it difficult to use the technique for weighting the criteria, then the technique has less chances to be used in the process. Whereas, if a technique which poses less cognitive load on the DMs, can easily win over others and find acceptability in the approach, even if it provides less accurate results.
Stm
c1 c2 ··· cn St1 w11 w12 · · · w1n St2 w21 w22 · · · w2n ... ... ... . . . ... wm1 wm2 · · · wmn
w
=⇒ Gr w1 w2 · · · wn
(
Eq (
In the literature of multi-criteria decision making, often the problem of criteria weighting comes along the multi-criteria decision analysis. The vast literature on criteria weighting techniques found mentions of large number of different techniques, but as in this paper we try to solve criteria weighting technique for systems engineering, we’ll stay with the techniques which have earlier found usage in the systems engineering project. Criteria weighting methods can be divided into two types internal and external. Popularly known internal methods are Entropy method, Regression method, Variance method, and LINMAP method. The external methods can further be divided into two types with DMs matrix such as: Swing method, Utility method, and without DMs matrix such as: SMART Method [11], SMARTER method, Eigenvector, Minimum weighted squares method, weighted sum method (WSM), weighted product method (WPM) [12]. Reference point method for vector optimization, AHP method, PROMETHEE [7] method are equally popular to solve this problems. There are many other techniques based on the visual aids for selecting the right stakeholder weight.
The previously mentioned techniques depend completely on human judgement about the preference weighting to get the weight, but seldom human can provide reasoning about the preference weight. To best of our knowledge, in the literature of decision making, there is no comprehensive way to reason the weight of the decision maker, to allocate systematically the weights at the various levels of the preferences. In this respect our work is very different from the previous work. We provide a mechanism to weight the human perception and link it with the mathematical formulations to derive the criteria weight. To have more robust criteria weighting, we have used multiple algorithms to find the weight.
Another aspect of the criteria weighting problem is about the uniform satisfaction among the stakeholders, pointed out by the well known Arrow’s impossibility theorem [13], which states that DM can find no procedure that can combine individual’s rankings of alternatives to obtain single unified rankings. Recent research works have tried to address criteria weighting problem using many other different techniques such as card playing [14], using hybrid approaches of many different techniques, but in almost all of them they started with assumption about the particular criteria weight. 4
The proposed technique uses the classical preference modelling [9, 15, 16] for representing stakeholder preferences.The proposed approach is four step process. We assume that all the necessary DMs (stakeholders) and criteria are already identified for the system under study. 4.1
Prerequisite to technique – ‘D’ is the set of decision makers (DMs) involved in the concerned conflict, where 2 ≤ |D| < ∞. – ‘C’ is the set of distinguishable criteria, satisfying 2 ≤ |C| < ∞. For each stakeholder set ‘C’ consists of three distinct subsets H, M, L, such that: {H} ∪ {M} ∪ {L} = {C}, and {H} ∩ {M} = {M} ∩ {L} = {H} ∩ {L} = {φ }. Where cardinality of each set can be different. The criteria subsets are such that the perceived utility is in the order p(H) > p(M) > p(L). – For each i ∈ D, there is set of preference relationships { i, >i, ∼i} defined over ‘C’. – For each i ∈ D, a complete binary relation i on C, specifies DM i’s strong preference over C. If s, t ∈ C, then s i t means the DM i strongly prefers s to t. – For each i ∈ D, a complete binary relation >i on C, specifies DM i’s weak preference over C. If s, t ∈ C, then s >i t means the DM i weakly prefers s to t. – For each i ∈ D, a complete binary relation ∼i on C, specifies DM i’s indifference over C. If s, t ∈ C, then s ∼i t means that s to t are indifferent to DM i. – The relation and > are asymmetric, ∼ is symmetric and reflexive and the triple { i, >i, ∼i} is complete.
– The preference relationships can be transitive for a particular stakeholder. 4.2
Technique The proposed approach can be divided into four steps as follows: Criteria Categorization All the decision makers involved in the decision process should categorize the agreed set of criteria in to three sub sets high preference (H), medium preference (M), low preference (L), according to their perception of utility of the criteria. Hence, the sorting of criteria is such that the perceived utility is in the order p(H) > p(M) > p(L).
Preference modelling over criteria Each DM i ∈ D, creates the preference matrix Pi,
over the criteria j ∈ C, given by Eq.(
c1 c2 ··· c j c1 0 p12 · · · p1 j c2 p21 0 · · · p2 j Pi = ... ... ... . . . ...
c j p j1 p j2 · · · 0
where the value pab of a DM i ∈ D is given by Eq(
For each categories of solution, system experts could add approximative criteria values. The chassis could be in carbon, magnesium, aluminum or in polyethylene. For each technology, subject matter experts are able to approximatively weight each element of this structure.
For each technology, the expert is able to make a “categorization” for alternative
solutions based on the proposed criteria set, completely without the specifications, it
is not accurate sizing, but he knows a ranking of basic technologies. This simulation
does not attempt to validate a solution but tries to establish a more precise relationship
between a criterion and a family of solution. Consider the choice of the two following
stakeholders. The sales engineer wants a conveyance which is economical, reliable and
robust. It will therefore fill the his preference matrix given by Eq.(
The ecological criterion may not be a priority for him, but the solution proposed would not be very attractive commercially. It would be difficult to place the product in the market demand. Similarly, considering the mayor of the city, responsible for public transport system. He wants an ecological and innovative solution, that represents modernity and dynamism of the city. He does not realize that his preference matrix may indicate towards the solution of the latest technologies that are likely to be less reliable and expensive to maintain for a product designed for intensive use. In these two examples, we see that the perception of a stakeholder to the criteria is necessarily biased. The head of the city was not opposed to having a robust and reliable, but he can interpret these criteria with his knowledge, which are a priori limited on technologies. His perception represents the criteria “robust” and “reliable” with respect to its repository of life for the project in question: a modern car, a scooter, ... To summarize, this simulation is a means proposed for each stakeholder should help when choosing a value for the weight of importance of the design criteria. Simulation offers the ability to view a draft solution based on criteria elements informed by expert systems.
Generating scores The scores are generated using a simple process consisting of a maximum of j number of moves, where j = |C|, in which every DM starts marking the criteria set C. The scores can be generated using the utility functions shown in Figure 2, depending upon the risk averseness, proneness or neutrality of the decision maker. Below, we define some risk neutral utility functions that we employ in our example in Section 5. 1. Every DM starting with the most preferred configuration to the least preferred state. 2. Individual score of every configuration is calculated and the one with the highest value.
The scores of every configuration can be calculated using the equation (
score(ci) = (k − j.i) k Ui(Xi)
More Risk Averse
Less Risk Averse Less Risk Prone
Risk Neutral More risk prone
Score 14 12 10 8 6 4 2
RA RP RRNN12
RN3 Xi 0 1 2 3 4 5 6 Rank where k, j are index terms used to generate different types of utility functions.
Normalised score(ci) =
score(ci)
∑i ci
(
Once the scores are calculated they are normalized to obtain the score. 5
To show the ease of usage of our technique, we present here an example of Criteria
weighting problem. We take an example of a hybrid car. The design team in a
automotive industry wants to design a hybrid car and they need to weight design criteria
to proceed with the selection of alternatives. Let the set of DMs be D = {a, b, c, d, e},
the criteria set be C = {c1, c2, c3, c4, c5, c6}. The interpretation of various criteria is
explained in Table 1.
Step 1 The DMs categorize criteria set according to their perception of criteria as shown
in Table 2.
Step 2 The DMs create their preference matrices according to their categorization from
step 1 as shown in Table 2. The preference matrix of stakeholder a, b, c and d are
shown in Eq.(
Criteria with higher preference are marked early and one with lower are marked
later in the order of preference.
(
After the marking is done, the DMs agree on a set of score generating algorithm. By
using the score generating functions as mentioned in Eq.(
Once the criteria weight matrix is available, the final criteria can be obtained by
either by the mean of weights obtained for each criteria or by accepting any particular
criteria array. In this example we took the mean of the four array of criteria weights and
the resultant criteria weight array is represented by Eq.(
The addition to ease of application, our technique provides other multiple benefits. It helps a DM to understand his perception of criteria better and of the other DMs. It also helps to avoid conflicts among the DMs, which often arrive during the decision making process. The transparency of the approach allows easy negotiation of the criteria weights and hence maximum number of DMs are satisfied with their contribution. The low cognitive load that our technique demands can be important factor for acceptability of the technique. Our technique can be coupled with a variety of decision-making approaches, such as PROMETHEE, TOPSIS, WSM, WPM.
The decision makers are free to propose their own utility functions or scoring algorithms based on the ordinal ranking achieved. This allows a more conducive environment for criteria weight negotiation, as the whole process is transparent, with no black-box process involved.
In our approach, we considered that all the DMs have equal weight, but this may not be the case in a real systems engineering project. It is possible that the different stakeholders in the projects have different importance. But this is only upto Project manager to allocate the weights to various DMs. Our approach offers possibility to weight the different DMs involved in a project. In this case only one preference matrix would be required, i.e., only of Project manager. While, weighting DMs it would be advised to use a set of risk averse and risk neutral utility functions to weight them, in order to avoid penalizing the low ranked DMs.
There can be arguments that the utility score generation algorithms cannot be accepted as weighting mechanism. But the literature shows that, every individual DM has a utility function, which he uses consciously or unconsciously while providing scores directly as it usually happens. Here we attempted to provide a formalism to use this conscious/unconscious utility function, in order to help other DMs to understand the perceptions of each other. The benefit that our approaches provides over other is that, it involves the DMs to methodologically provide the ordinal ranking by first demanding them to attempt to categorize them in three categories. This categorization provides input to the next step, which can again be validated by corresponding DM; depending on the preference of the various DM’s, a range of scoring algorithm can be applied and weights can be obtained.
In the beginning our approach demands slightly more participation from the DMs, as compared to the other approaches, but once the weights are methodologically obtained they are certainly more reliable then the other contemporary approaches with least amount of conflict. Better decisions allow to design the right systems, with more stakeholders getting more confident about their product. 7
In this current work, we have provided a systems theory of how the criteria weights can be obtained using the classical theory of preference modeling. We call our technique Utility Rank Order Weighting (UROW) technique. This approach provides multiple benefits with compared to other existing approaches. Usually in systems engineering project, the engineers rely upon their intuition to provide weights, and later use other technique to combine the different DMs’ preferences. Our approach provides a formalism to this systems engineer intuition and hence provides the reasoning for the various weights achieved. Our approach is very easy to understand and use, and demands very low cognitive load from the engineers and stakeholders. It allows to formally provide the scores using the DM’ drawn utility functions: risk prone, risk averse, or risk neutral; it provides a mechanism to combine them together to come up with a uniformly acceptable solution. In future, we look forward to link the simulation of the DMs’ preferences with the design library, in order to shorten the decision time. Our approach can easily be applied to the class of methods which require information on the attributes to carry out a decision analysis.