In designing most systems, requirements analysts face many competing requirements, such as performance, usability, costs, and so forth. Ideally, analysts would like to quantitatively measure consequences of solutions on requirements and risks, and extract stakeholders' preferences in terms of numerical weights. However, during the early stages of requirements and system design, it is hard to quantitatively measure all factors on a similar scale and quantify stakeholders' preferences. This contribution proposes a semi-automated decision aid tool which allows the use of available but potentially incomplete quantitative and qualitative requirements and risk measures. It removed the need to elicit importance weights of requirements. Instead, stakeholders are asked how much they would relax the demand on one objective to better achieve another. The proposed tool extends the Even Swap method with formally defined rules for suggesting the next swap to decision stakeholders.
Requirements analysts need to make key decisions early in the project, such
as which architectural or design solution to employ [
Related Work: Faced with the typical absence of reliable quantitative data,
some Requirements Engineering (RE) techniques, such as i* [
Some decision analysis methods evaluate consequences of alternative
solutions in terms of precise and meaningful quantitative measures [
Problems: The main problems when making trade-offs among requirements to decide over alternative design solutions are: 1. Manual Prioritization: extracting stakeholders’ preferences over multiple criteria in terms of numerical importance weights is error-prone and laborintensive. 2. Incomparable Scales: aggregating requirements measures in different scales is usually error-prone or not possible. 3. Extensive Data Collection: eliciting required information to make an objective decision usually involves an extensive data collection from stakeholders. 4. Lack of Quantitative Risk Factors: quantitatively measuring the probability and damage of all risks is challenging, if possible at all. 5. Scalability: the decision problem may become complicated and impossible to be analyzed manually due to several requirements and/or alternatives.
Contributions: This paper describes a decision aid tool that addresses above
problems. The tool adopts the Even Swaps multi-criteria decision analysis
approach [
Although the Even Swaps method solves the problem 1, 2, and 3 (mentioned
above), it can fail in practice due to scalability issues: when several software
requirements and alternative solutions need to be considered, decision stakeholders
may not be able to determine the best swap among numerous possibilities [
– The tool suggests which requirements to select for the next swap.
We illustrate the use of the tool by analyzing a prototypical Digital Rights
Managements (DRM) player [
The DRM player contains two hard-coded credentials: Valid activation code
and Player key. A software cracker can use the DRM player without buying
activation code, either through static code analysis to extract the valid code or
by tampering the binary code to bypass the license checks. The main protection
strategies against Tampering the binary code attack are [
Obfuscation adds an overhead to the code which causes performance drop downs. 2. Tamper Proofing: Tamper proofing algorithms detect that the program has been modified. Once tampering has been detected, a tamper response is executed which usually causes the program to fail. 3. Distribution with Physical Token: Physical tokens are hardware-based protections that try to provide a safe environment for data, code and execution. By employing a physical token, the user needs to show possession of a token to use the software.
These alternative security solutions have side-effects on other goals such as portability, delay, and cost. The corresponding consequence table in Figure 1 contains heterogeneous data, i.e., different goals are evaluated in different scales and by different techniques. Some of the criteria are measurable variables that need to be minimized or maximized. For example, stakeholders are able to estimate Delay (G3) in milliseconds based on the properties and specification of alternatives. However, enough information is not available to quantitatively measure the risk of tampering attack, so consequences of alternatives on the damage and probability of this risk is evaluated in the ordinal scale of Low, Medium Low, Medium, Medium High, and High. 3
In an even swap, the decision analyst, collaborating with the stakeholders,
hypothetically changes the consequence of an alternative on one requirement, and
compensates this change with a preferentially equal change in the satisfaction
level of another requirement. Swaps aim to either make criteria irrelevant, in the
sense that both alternatives have equal consequences on the criteria, or create a
dominant alternative. Alternative A dominates alternative B, if A is better than
(or equal to) B on every criteria [
Notation Remark. A swap between two goals gx and gy that changes the satisfaction value of gx from x to x0 and compensates this change by modifying the satisfaction level of gy from y to y0 is written as:
(gx : x → x0 ⇐⇒ gy : y → y0) Case Study: The DRM Player Decision Scenario. We illustrate the Even Swaps method by analyzing and comparing the first two alternatives security solutions for the DRM player ( Figure 1): 1. Compare No security solution (A1) and Code obfuscation (A2) 2. Ask stakeholders: If the RiskDamg could be reduced from High to Medium, how much Delay they would tolerate instead of 50 ms delay? (RiskDamg : High → Medium ⇐⇒ Delay : 50ms →?)
Stakeholders agree with increasing the Delay to 500 ms. 3. Consequences of A1 are revised based on the above swap. The revised alternative 0 is called A1, which is a virtual alternative and subsidence of A1:
Consequences of A01 = {Medium, Medium, , 500ms, $0}
Consequences of A2 = {Medium, Medium, , 200ms, $20K} 4. RiskP rob, RiskDamg, and Portability are irrelevant decision criteria and can be removed:
Consequences of A01 = {500ms, $0} Consequences of A2 = {200ms, $20K}
on {Delay, Cost} 5. Ask stakeholders: If the Delay could be reduced from 500 ms to 200 ms, what Cost they would pay? (Delay : 250ms → 200ms ⇐⇒ Cost : $0→?)
Stakeholders agree to pay $10K for A1. 6. Consequences of A01 are revised based on the above swap.
Consequences of A01 = {200ms, $10K} Consequences of A2 = {200ms, $20K}
on {Delay, Cost} 7. Delay is irrelevant and is removed. 8. With respect to price, A01 dominates A2. 9. A2 is removed from the problem and the process continues by applying the Swap
Method to A1 and A3. 4
This paper introduces a semi-automated Even Swaps tool, that given a consequence table, suggests a chain of swaps to determine the overall best alternative. The algorithm consists of several Even Swaps cycles. In the beginning of each cycle, the algorithm selects a pair of alternatives for the next Even Swaps process. The algorithm suggests a chain of swaps to stakeholders intending to find the preferred alternative in the pair. The dominant alternative is kept in the list of solutions, but the dominated alternative is removed. The algorithm then selects another pair of alternatives to compare and a new cycle starts. These cycles continue until one alternative remains, which is the best solution overall. 4.1
The decision aid algorithm has two main goals: 1) minimizing the number of swapping steps required to make one of the alternatives dominant, and 2) generating swaps that are easy to make for stakeholders. Toward these goals, we develop a set of rules in the following sections for suggesting the next swap to stakeholders.
Goal one: Minimizing the number of swapping steps: The tool minimizes the number of swapping steps in 2 ways: 1) suggests swaps that help make one of the alternatives dominant, and 2) reuses previously made swaps to avoid asking repetitive swap queries from stakeholders.
Rule 1: Make swaps that help toward creating a dominated alternative. Swaps make one of the decision criteria irrelevant, which helps remove one goal from the decision problem in each step. Removing criteria one by one is a timeconsuming approach to apply the Even Swaps process. The tool suggests swaps that help toward making one of the alternatives dominated. That means if an alternative such as A is dominant for n goals like g1, g2, ... gn, and B is a better solution only for one goal, like g, then we need to remove g by swapping it with one of those n goals. By removing g, solution A might still be dominant with respect to all goals (g1, g2, ... gn). However, swapping any two goals from g1, g2, ... gn will not change the situation between A and B and neither of them becomes dominated with respect to all relevant and remaining goals.
In the Even Swaps process, between the pair of A1 and A2 in the DRM player scenario, with respect to RiskDamg, A2 is a better solution, and with respect to Delay and Cost, A1 is a better solution. Note that RiskP rb and P ortability are irrelevant. To reduce the number of swaps needed in the next step, RiskDamg must be swapped with either Delay or Cost, aiming to remove RiskDamg, so A1 would dominate A2 with respect to all relevant goals.
Rule 2: Pick the most reusable swaps. When stakeholders make a swap, their input can be reused for another alternative, without further consultation with human stakeholders. This reduces the number of swap queries from stakeholders. A swap from previous steps can be reused in the current step if the goals and their values are identical with the previous swap. Assume stakeholders have made a swap as (gx : x → x0 ⇔ gy : y → y0) in a previous cycle. Now to decide between two alternatives such as A and B in a different cycle, this swap is reusable iff: - Consequences of A on gx and gy are equal to x and y - Consequences of B on gx is x0 Under these conditions, alternative A can be replaced with A0 where consequences of A0 on gx and gy are revised to be x0 and y0. Thus gx becomes an irrelevant goal and can be removed.
Goal Two: Suggesting Easy Swaps: In addition to considering swaps
reusability, the algorithm suggests swaps that decision stakeholders would be willing to
make. Hammond et al. [
In addition, if consequences of two alternatives on the first goal of the swap are close, with an insignificant change, such a goal can become irrelevant. In this way, the goals that do not differentiate alternatives are eliminated from the problem earlier. The tool swaps the first goal with another goal for which consequences of alternatives are highly differentiable, because it would be more probable that the revised virtual alternative after applying the swap is still better than the other alternative.
Rule 3, make the easiest swap: When comparing two alternatives A and B, two goals such as g1 and g2 are swapped where the satisfaction level of A on g1 is minimum compared to any other goal, and the satisfaction level of g2 is the highest level compared to other goals. We define a distance factor between alternatives on every goal, which aggregates these desired properties into a value. The distance factor on the goal gx is 4(A, B, gx) and calculated as: 4(A, B, gx) = |ax − bx| + ax
maxgx where ax and bx are the consequences of A and B on gx. maxgx is the maximum satisfaction level of gx in the consequence table, and by dividing |ax − bx| + ax to the maximum value, distance factors of different alternatives are normalized to values in the scale of 0 to 2.
For example, the max of G4 in Figure 1 is (fully satisfied). For the goals that need to minimized, the lower their value is, the higher the satisfaction level would be. Thus, if the consequence of A on gx is ax, then ax is replaced with maxgx − ax. The tool revises the satisfaction level of goals in the consequence table in Figure 1 are as:
RiskP rob A1 Medium Low A2 Medium Low A3 Medium Low
A4 Medium High maxg Medium High
RiskDamg
0 Medium Low Medium High Medium High Medium High
G4 These modifications to the consequence table are only used for calculating the distance factor. Figure 2 shows the distance factors of A1 and A2 on the goals. (For calculations, the interval scale of Low to High is mapped to the interval values of 1 to 5, and , , , and are mapped to 3, 2, ,1, 0.) By applying rule 1, we concluded that RiskDamg must be swapped with either Delay or Cost. Based on rule 3, the lowest distance factor (RiskDamg) must be swapped with the highest distance factor (Cost).
Rule 4, Swap goals with tangible scales: Tangible goals that are measured
in absolute values, such as costs or delays, are easier to trade [
The automated Even Swaps tool takes a set of goals and a consequence table,
and in each round of the Even Swaps method, identifies a list of potential goals
to be swapped next. The list is first generated by applying rule 1. Then the list
is trimmed by only keeping the most reusable swaps (rule 2). To find the best
swap, the tool applies rule 3, and if still there are more than one possible swap
for the next step, rule 4 is applied. A demo of the proposed tool in this paper is
available at [
Given m goals, in each round, at most m − 1 swaps are made, and for n alternatives, at most n − 1 rounds of Even Swaps are needed. Thus, in the worst case scenario, with m − 1 × n − 1 swaps the best solution is identified. By reusing swaps and applying rule 1, we aim to minimize this number.
In this work, we adopt and enhance the Even Swaps [
A threat to practicality of the tool is the diversity of evaluation scales in the consequence table. Stakeholders may not be able to swap a goal measured in absolute values with a goal that is evaluated by qualitative labels such as and .