Decision-making depends on the availability of information and the ability of decision-makers to manipulate this information. This paper proposes an approach that integrates the decision-relevant information, which is subject to uncertainty, to multi-criteria decision-making. An approach must enable decision-makers to explore the uncertainty and risk involved in their decisions. It arose from the theory of risk-based decision-making and the generalization of particular risk-based solutions in different domains. Multi-Attribute Utility Theory (MAUT) became the ground for the proposed approach. Utility functions appropriately account for the associated uncertainty and risk attitude of decision-makers. Three types of weighing policies reflected risk importance from decision-makers' point of view were introduced. MAUT seeks to trade-off among criteria and assigns a ranking to the alternatives. We examined the property of the proposed approach through a quantitative study on the alternatives ranking for decision-making in the transfer from face-to-face to online learning. The results showed that the risk-oriented approach reflected the risk-averse property and consequently provided the rankings similar to ones realized in the natural conditions.
Traditionally decision-making problems are related to constructing the preference
order to rank the alternatives to select the best one. For decision-making problems
where alternatives are compared with one criterion, this may be easily accomplished.
However, the most realistic cases should evaluate multiple conflicting attributes.
Conflicting attributes usually arise in a case when decision-makers should take into
attention the interest of different stakeholders. Multi-criteria decision-making (MCDM)
suggests many analytical frameworks that facilitate decision-makers to perform
tradeoffs between conflicting attributes [
In theory, multiple objectives defined by performance goals, that are often competing and conflicting with each other, cause the complexity of MCDM. However, uncertainty as well is one of the causes of decision-making complexity. If the decisionmaker knew precisely the outcome of each alternative, she would define precisely her preferences.
Ignoring uncertainty and its associated risk may simplify the decision-making process. However, as well it degrades the quality of decisions. Sometimes decisionmaker uses risk as one of the criteria to consider uncertainty. Sometimes risk analysis and decision-making are realized in parallel. Both approaches cause information loss. The first one does not reflect the risk from the points of view of stakeholders; the second one does not give the proper weight to dependencies between alternatives and risk sources.
In this paper, we present the approach to risk-oriented attributes definition. The involvement of risk-oriented attributes simplifies to make informed decisions under uncertainty and risk.
The remainder of the paper is structured as follows. Section 2 discusses some related work in the area of risk-based decision-making and MCDM. Section 3 describes the main components of the proposed approach. Section 4 demonstrates the practical use of the proposed approach through two application examples of educational decision-making problems. Section 5 concludes the paper and outlines some perspectives for future work. 2
Decision-making is the process of making choices by identifying objectives, gathering
information, and assessing or ordering alternatives. Risk-based decision-making
(RBDM) is “a process that organizes information about the possibility for one or more
unwanted outcomes into a broad, orderly structure that helps decision-makers make
more informed management choices” [
RBDM is an iterative, never-ending process. For example, in [
The paper [
Risk issues obstruct the selection of the cloud provider because of the reliability
and security of the cloud as a public platform. The automated categorization and
selection of the cloud provider based on risk metrics is quite a challenging task. The
authors of [
One additional example is risk-based testing described in [
As an MCDA method, the most promising one is representative of unique criterion
methods. Between them, the most popular are methods AHP [
In this paper, we encapsulate the risk-oriented assessment of alternatives into MAUT and examine how this approach works for some decision-making problems. 3
Let A = {a1, a2, ..., aN} is a finite nonempty set of alternatives among which the decision-maker must choose. As well, there is a collection of criteria C = {c1, ..., cM} to be satisfied by the different alternatives. Each criterion should be risk-oriented, i.e., formulated in terms of risk consequences. For example, in the case of “hackers’ attack” risk, the criteria could be “the volume of damaged data” or “the time for service recovery.” It is quite close to the concept of “impact” in traditional risk evaluation. However, usually impact is evaluated as “cost” of risk consequences, which is “derivate” of starting measure. Correspondingly, each criterion, i, is connected with a measurement scale Si∈R. So that, we can define the partial score si(aj) of alternative aj in accordance with criterion i.
We use MAUT as MCDM means so that we have to convert the partial scores into
utility value on a standardized scale [
Ui ( a j ) = 1 − si ( a j ) − smini , where smini and smaxi are the minimum and the maximum values of partial scores under criterion i, respectively.
Since different criteria might have different degrees of importance, the decisionmaker should define a weight, wi, associated with each criterion i, which represents the hazard of risk of this criterion. We can apply different weighing policies.
The first policy is uniform, where every criterion is given the same weight as the others: wi = 1 M ∀i = 1,..., M . (1) (2) (3) The numerical score obtained for each alternative determines the final ranking, where the highest value indicates the most preferred alternative.
The scheme of the whole process is shown in Fig. 3. This weighting policy can be applied to any tasks when decision-makers are not able to define risks order more precisely.
The second policy is customization by the decision-maker, where every criterion is given the weight reflecting its importance. The values wi should be normalized so that ∑ M
i=1 wi = 1 .
The third policy is customization by risk profiles, where every criterion is given the weight, which is assigned based on risk profiles. The difference between this policy and the previous one lies in the source of weighting: is it a decision-maker or third party information.
Finally, we aggregate all the utilities following an additive aggregation function as follows:
U ( a j )
M =∑ wi ×Ui ( a j ) .
i=1 The first step in this process is the identification of a set of criteria and suitable measurement scales. At this step, the decision-maker should involve other stakeholders to understand the criteria better. Utility assessment is a step of evaluating the partial scores of alternatives and the transition to utility values. Risk weighing is a step of definition and realization weighing policy. Finally, the aggregation utility function provides the possibility for ranking the alternative and choosing the best one. During all stages, a decision-maker is the driver of the process. 4
The examination of the proposed risk-oriented approach was established while faceto-face learning had been transformed for online learning due to quarantine limitations. We applied the proposed approach for two issues – choosing the delivery channel for courses and modifying the teaching materials. 4.1
The type of learning environment can lead to problems related to the scope and
instructional characteristics [
Lockdown caused the transfer to online learning but was unexpected by teachers. Therefore, there were a lot of potential problems. The primary of them composed the set of risk-oriented criteria: ─ r1: the frequency of breakdowns of the learning environment will be unacceptable; ─ r2: the time for adoption the course to learning environments will be inappropriate; ─ r3: the means provided by learning environments will not be enough for learning activities in a course; ─ r4: the teachers’ efforts to become proficient in learning environments will be enormous.
We got the teaching staff working in the Computer Systems Institute to take part in assessing. We used an ordinal scale with marks in the interval from 1 to 10. The medians of the partial scores given by a group of teachers are shown in Table 1.
In Table 1, the values of utility functions Ui(aj) were calculated according to (1). While calculating the aggregation U(aj), we used the first weighing policy (2).
As we see, the preference order was a1>a2>a4>a3, which was far from an expected order. In particular, group chats in Telegram with inadequate support of learning activities turned out better than powerful Google Classroom. Alternatives Risk-oriented criteria Utility function U r1 r2 r3 r4 U1 U2 U3 U4 a1 8 3 3 3 0,00 1,00 0,86 0,83 0,67 a2 8 3 2 5 0,00 1,00 1,00 0,50 0,63 a3 2 9 6 8 1,00 0,00 0,43 0,00 0,36 a4 2 7 9 2 1,00 0,33 0,00 1,00 0,58 In the mid of the semester, the dean office gathered data about learning environments involved in online teaching. The source of data was students who could have met different technologies in different courses. The results are shown in Fig. 4. The ranking resulting from the data agreed with the preference order above. The “other” technology in Fig. 4 represents different rare solutions such as communication through e-mail only, Microsoft Teams, face-to-face communications.
In this case, using risk-oriented criteria supplied the explanation for teachers’ choice. If we used traditional domain-oriented criteria, e.g., supported activities, provided means, types of test questions, we would get a different order and not understand the real choice. 4.2
After the first week of the lockdown, the teachers ascertained that teaching materials for face-to-face learning were not enough for online learning. Time pressure was too hard for transforming the course for e-learning. Therefore, the decision lay in enriching existing course materials by linking with third-party materials. There were considered four alternatives: ─ a1: the whole particular courses provided by the “Coursera for Campus” program; ─ a2: the predefined videos and/or assignments from the courses provided by the “Coursera for Campus” program; ─ a3: the set of recommended reading materials; ─ a4: the sequence of guided assignments.
We defined two sets of criteria to provide a comparison of the proposed approach with the traditional one.
The set of domain-oriented criteria was defined based on [
The set of risk-oriented criteria was defined based on [
To begin, we converted the partial scores into utility value. For the assessment with risk-oriented criteria, we used (1). A linear utility function for domain-oriented criteria was calculated as follows:
Ui ( a j ) = si ( a j ) − smini . (4) In both cases, we used the first weighing policy (2). Table 3 provides the result of aggregation. Alternatives Udomain Urisk a1 0,50 0,44 a2 0,45 0,31 a3 0,17 0,72 a4 0,83 0,54 To define which order was more realistic, we asked the students to assess the additional materials after the courses had been completed. In the survey, we used an ordinal scale with marks in the interval from 1 to 5. Fig. 5 shows the distributions of marks. It turned out that the preferences of students were closer to order built along with riskoriented criteria. The teachers can forecast the students’ behavior quire well; the riskoriented criteria took into account this forecast. Instead, the domain-driven criteria took into account only the theoretical point of view.
Overall, the risk-oriented approach seems to be more practical. It is known that humans are not risk-averse. Therefore, a risk-oriented approach can be more reasonable for decision-makers. 5
We proposed an approach that allows decision-makers to make informed decisions and raising their awareness of uncertainty and risk involved in their decisions. The approach relies on a risk account while defining the criteria of decision-making. As well, decision-makers are able to input their risk preferences by choosing the weighting policy. MAUT became the ground for seeking to trade-off among criteria and assigning a ranking to the alternatives.
We executed two experiments with different alternatives and sets of criteria. All assessments had been realized but not used in the real-life process. It gave the possibility to compare calculated rankings with “natural” ones. In both experiments, using a risk-oriented set of criteria provides the ranking correlated with observation in reallife processes. However, while this shows that the approach is reasonable, in reality, it might not always be feasible due to specific criteria definition.
The approach suffers from some limitations. First, it lacks an empirical evaluation of its usefulness with domain experts. Second, it lacks research on the impact of a utility function definition. We used only a linear utility function because it is the simplest solution. Both issues will be addressed in future work.
More evaluation studies are also needed to provide more evidence of the usefulness of a risk-oriented approach to support informed decision-making under uncertainty and risk. These studies should be expanded beyond controlled decision-making scenarios in a friendly environment.