On the meaning and use of contribution links

Abstract

Contribution links are at the core of goal modelling languages of the i* family. They allow representation of how satisfaction of one goal is a ected by satisfaction of others assisting thereby deep and detailed understanding of the impact of low-level design decisions to high-level stakeholder objectives in various decision support scenarios. Several approaches have been proposed in the literature for representing and performing inferences with the construct. While theoretical arguments are typically evoked to support each such method, their usability and intuitiveness by users is also important for deciding which method is suitable for what task. In this paper, we o er a short summary of some of those approaches for treating contribution links and review a group of initial experimental studies we have conducted to understand how untrained users perceive the meaning of contribution links via observing the inferences users spontaneously make with them.

Keywords

• Conceptual Modelling sion Experimental Study

-

One of the most important features of goal modelling languages within the i* family [ 14,3,4 ] are contribution links. Such links allow the expression of the supposition that satisfaction of one goal within the model a ects satisfaction of another goal in some way. The construct is particularly useful for representing and exploring how various low-level options encoded within goal models a ect higher level stakeholder objectives, assisting thereby decision making when there is a lack of precise quantitative decision models or hard evidence.

Nevertheless, due to the abstract nature of the construct, it seems to be di cult to pinpoint its precise meaning and to subsequently nd an obviously e ective way to represent such meaning. The variety of ways found in the literature to represent and understand the semantics of contributions appear to be evidence of this di culty. Thus, there are qualitative contribution links of various kinds in which symbols and words are used to convey the quality and magnitude of the contribution as well as quantitative contribution links in which numbers, also of various formats, are employed together with symbols such as signs and subscripts to represent similar information. Newcomers to i* may likely be perplexed with regards to which of the various proposals to adopt for their speci c needs.

We believe that the problem is too central to i*'s usefulness and adoption potential to be ignored. In this paper, we o er a brief review of some of the proposals o ered by the literature so far (Section 2), followed by a presentation of a experimental research program we have been engaging in for exploring the intuitiveness of various contribution representation approaches (Section 3). We close with an outline of our medium term research agenda (Section 4). 2

Understanding and Representing Contributions

A contribution link A !l B from goal A to goal B generally shows that the satisfaction of goal B is a ected by the satisfaction of another goal A according to label l. The quality (e.g., positive or negative) and strength of contribution that is e ected to B depends on our understanding of the state of satisfaction of A and label l.

The literature o ers several ways for representing l. In qualitative frameworks, the label l can be a symbol such as \+", \ " signifying partial and \++",\ " su cient contribution [ 14,5,2 ]. As of iStar 2.0, words are proposed instead of symbols (\help",\hurt",\make",\break"). Labels can also be quantitative, i.e. a number from some numeric interval and, if relevant, signed [ 5,2,10 ]. Labels may also contain subscripts as in \0:2+D" and \ S " when more than one variable are used to denote goal satisfaction status [ 5 ].

Contribution labels allow users of the diagram perform inferences about the satisfaction status of one goal given the corresponding status of other goals in the diagram. Typically some notion and representation of belief or evidence about partial goal satisfaction is introduced to allow such inferences. In qualitative frameworks partial goal satisfaction is represented through associating the goal with a variable that takes values from some ordered set characterising \levels" of satisfaction (evidence/ belief), such as the set fN, P, Fg denoting No, Partial, Full satisfaction of a goal, respectively. Visually, various icons are used in place of symbols fN, P, Fg as annotations next to the goal they refer to [ 5,2 ]. In quantitative frameworks a continuous domain is used for the variable, such as [0.0, 1.0] [ 10 ], [0,100] or [-100,100] [ 2 ], again commonly represented as annotations next to the goal in question. Giorigini et al. [ 5 ] de ne two variables for each goal, one to capture satisfaction and one to capture denial, expressing thereby inconsistencies to our beliefs about satisfaction of goals.

The way by which contribution links A !l B can be used to perform inferences about partial goal satisfaction is expressed via rules that show how a given partial satisfaction level of goal A, say sat(A), a ects the partial satisfaction level of goal B, sat(B), based on what label l is { noting also that denial values den(A) and den(B) can also be considered. Moreover, in the general case, B is targeted by more than one goals A1; A2; : : : using contributions labelled with di erent labels, l1; l2; : : :. Thus, to fully de ne satisfaction of B we need rules which dictate (a) how the satisfaction level of each Ai and li are combined Approach

Quantitative E ect Aggregation

Qualitative E ect Aggregation

URN ([ 2 ]) Multiplication Grand Sum AHP-inspired ([ 10,13 ]) Multiplication Clustered Sums

Multiplication Max Evidence-based ([ 5 ]) Serial-Parallel Max

Min Max

Min

Min (Custom)

Max into an e ect from Ai, (b) how the corresponding e ects from all Ai should be aggregated to calculate satisfaction of B. There is variability in the literature with regards to both how e ects should be calculated and to how they should be aggregated.

In qualitative frameworks [ 14,5,2 ] a set of rules in logical or tabular form is de ned for deciding both the above. Given their two-value system, Giorgini et al. [ 5 ] follow an evidence maximization principle for aggregating such e ects. Amyot et al. [ 2 ] use a single value system and as such use a more complex function that explicitly labels con ict. In both, the strength of the contribution e ect is the minimum between the strength of the label and the satisfaction of the origin, noting that negative labels invert satisfaction into denial and denial into satisfaction. Aggregation however is di erent in Amyot et al. where strong and weak e ects are counted and compared separately to then combine in a hybrid addititive/maximization fashion marking co-presence of strong positive and negative e ects with \con ict" labels (Table 3 of [ 2 ]). We note that, in the context of such con cts, Horko et al. [ 7 ] suitably proposes human intervention for their resolution, instead of relying on rules.

Quantitative frameworks use algebraic expressions instead of rules and exhaustive tables. Amyot et al. [ 2 ] multiply satisfaction values of goals Ai (a number in [-100,100]) with the label li (also a number in [-100,100]). The satisfaction of B is calculated by adding up the results { as in sat(B) = P sat(Ai) li. In the AHP-based proposals by Liaskos et al. [ 10 ] and Maiden et al. [ 13 ], the same aggregation approach is followed, with the important di erence, however, that each goal can receive multiple groups of incoming contribution links, each group independently concerned with a speci c local decision. Thus, the AHP-based approach is not concerned with calculating a global satisfaction value that results from a total evaluation of a goal model, but rather sets of satisfaction values corresponding to options in decision problems expressed as OR-decompositions in the model. Another important di erence of that approach is that it does not de ne denial of goal, which greatly simpli es the problem of devising e ect and aggregation rules.

In their quantitative framework, Giorgini et al. avoid committing to a speci c way by which e ect of a contribution is calculated: it can be the product (sat(Ai) li) or a serial/parallel resistance model ( ssaatt((AAii))+llii ), while a model more similar to the qualitative arrangement is that of minimization (min(sat(Ai); li)). In all cases, aggregation follows a maximization principle { as in sat(B) = max(sat(A1) l1; sat(A2) l2; : : :) where represents any of the e ect calculation methods above. A summary of e ect calculation and aggregation approaches can be seen in Table 1, stressing that it is not exhaustive. 3

Evaluating The Intuitiveness Aspect

The variety of methods to represent and reason with contribution links, brings up the question of which of the methods is appropriate and for what purpose. Theoretical approaches, such as ontological analysis (e.g. [ 6 ]) or demonstrative appeals to e.g. expressiveness, exibility or amenability to tractable automated reasoning, are normally followed to measure usefulness of each option. However, an additional criterion is how the representations work for users in practice, i.e. how they are helping them use goal models to their bene t. In this context, we have been speci cally exploring how contribution links are spontaneously understood by users who are not trained to the exact semantics of such constructs. Our goal is to see if any version of the operational semantics we reviewed above appears to be more intuitive for users, i.e., more readily understood.

In our rst study of the kind [ 1 ], we focussed on quantitative contribution links. We developed a number of goal models with quantitative labels and trained a number of users on the abstract meaning of contribution links but without exposing them to any of the precise inference rules of Table 1. We then presented them with small-to-medium size goal models and asked them to perform forward reasoning, i.e., infer the satisfaction level of a top-level goal given the corresponding level of the leaf-level goals. They were speci cally given four options for the satisfaction of the top-level goal, each corresponding to the result that is acquired by following the rules of each method of Table 1. An additional factor was added: for some goal models, the weights li of contributions targeting a goal always add up to 1:0.

In the results, we rstly saw that some semantic choices where more preferred than others, with the AHP-Inspired (multiplication/sum) and the min/max being more popular and serial-parallel/max being the least popular. In other words when following the serial-parallel/max propagation rules we arrive at satisfaction levels that are not expected by untrained users. Moreover, in goal models in which incoming contribution labels where restricted to 1.0, users tended to pick the choice corresponding to the multiplication/sum rules, apparently, as we hypothesize, after spontaneously inferring that the meaning of contributions is that of share of contribution of each origin goal to the satisfaction of the destination goal. The results also show some e ect of size, with the min/max interpretation increasing in popularity as size increases.

In a di erent study [ 11 ], we took up qualitative contribution labels and the Giorgini et al. semantics of label propagation. This time we focussed exclusively on e ect calculation, by only considering two goals, one contributing to the other. Like before, we o ered various examples of such pairs of goals with di erent labels and satisfaction levels of the origin goal, and asked participants what they thought the satisfaction of the destination goal was. We then compared what they responded with the normative semantics. The most important nding is the perception problems of negative contributions especially combined with goal denial. According to the formal semantics, goal denial of the origin goal translates to goal satisfaction of the destination, when the contribution label is negative (\ " or \ "). However, our participants (note: rst year university students) did not assume that the two negatives combined will result to positive satisfaction. An additional interesting nding is that even in cases where the origin had no satisfaction or denial assigned to it, the participants assumed the destination to still have positive or negative values, interpreting contributions as generators of satisfaction or denial rather than mere propagators of such.

In our latest e ort [ 12 ], a direct comparison between qualitative and quantitative contribution links is attempted. Participants are presented with single decisions (OR-decompositions of goals) that are connected with an hierarchy of soft-goals through contribution links. They are asked to identify the option that satis es { in their opinion based on what they see { the top level goal the best. Participant responses matched much more frequently the multiplication/sum semantics in the quantitative models than the min/max semantics of the qualitative ones, an e ect we attribute to the familiarity of participants with interpretation, aggregation and comparison of numbers.

In parallel, we have also experimented with the impact of the way contributions are visualized to intuitiveness and correct use [ 9 ]. Using optimal decision detection exercises similar to the ones described above [ 12 ] we considered three di erent representations of contribution link based decision problems: traditional graphs, tree-maps and a combination of bar- and pie-charts. We found that the latter allowed for more accurate identi cation of the optimal decision. Hence, attempting to replace symbolic representations with visual ones appears to improve the task of making inferences in goal models. 4

Future Work

The main motivation of the presented research program is to establish goal models as useful decision support tools, worthy of the e ort investment to construct and maintain them. Key to this is the development of a deep understanding of contribution relationships in a way that also satis es user expectations and the development of intuitive ways to represent and perform inferences therewith. Our plans for future empirical exploration follow a number of directions. Firstly, continuing the path of the works mentioned earlier, we plan to turn to more qualitative empirical methodologies { similar to those of Horko and Yu [ 7 ] { aiming at understanding what goes in users' minds when confronted with a contribution link network and asked to perform reasoning with it. Secondly, we plan to make the plethora of associated automated reasoning techniques ([ 8 ] for survey) part of our investigation. Thus, we wish to explore the extent to which the way reasoners aggregate local contribution structures into a nal evaluation of interest coincides with user's intuition and also understand what a ects users' trust in the reasoner. Finally, we intend to continue exploring visualizations alternative to the traditional box-and-line ones, focussing on ways to replace symbolic representations of contribution and satisfaction with visual ones.

References

1. Alothman , N. , Zhian , M. , Liaskos , S.: User Perception of Numeric Contribution Semantics for Goal Models: an Exploratory Experiment . In: Proceedings of the 36th International Conference on Conceptual Modeling (ER'17) . pp. 451 { 465 ( 2017 )

2. Amyot , D. , Ghanavati , S. , Horko , J. , Mussbacher , G. , Peyton , L. , Yu , E.S.K. : Evaluating goal models within the goal-oriented requirement language . International Journal of Intelligent Systems 25 ( 8 ), 841 { 877 ( 2010 )

3. Amyot , D. , Mussbacher , G. : User Requirements Notation: The First Ten Years, The Next Ten Years . Journal of Software (JSW) 6 ( 5 ), 747 { 768 ( 2011 )

4. Dalpiaz , F. , Franch , X. , Horko , J.: iStar 2 . 0 Language Guide . The Computing Research Repository (CoRR) abs/1605 .0 ( 2016 )

5. Giorgini , P. , Mylopoulos , J. , Nicchiarelli , E. , Sebastiani , R. : Reasoning with Goal Models . In: Proceedings of the 21st International Conference on Conceptual Modeling (ER'02) . pp. 167 { 181 . London, UK ( 2002 )

6. Guizzardi , R.S. , Franch , X. , Guizzardi , G. , Wieringa , R.: Ontological distinctions between means-end and contribution links in the i* framework . In: Proceedings of the 32nd International Conference on Conceptual Modeling (ER 2013 ). pp. 463 { 470 . Hong-Kong , China ( 2013 )

7. Horko , J. , Yu , E.S.K. : Interactive goal model analysis for early requirements engineering . Requirements Engineering 21 ( 1 ), 29 { 61 ( 2016 )

8. Horko , J. , Yu , E.S. : Comparison and evaluation of goal-oriented satisfaction analysis techniques . Requirements Engineering (REJ) 18(3) , 1 { 24 ( 2011 )

9. Liaskos , S. , Dundjerovic , T. , Gabriel, G.: Comparing Alternative Goal Model Visualizations for Decision Making: an Exploratory Experiment . In: Proceedings of the 33rd Annual ACM Symposium on Applied Computing (SAC'18) . pp. 1272 { 1281 . Pau , France ( 2018 )

10. Liaskos , S. , Jalman , R. , Aranda , J.: On Eliciting Preference and Contribution Measures in Goal Models . In: Proceedings of the 20th International Requirements Engineering Conference (RE'12) . pp. 221 { 230 . Chicago, IL ( 2012 )

11. Liaskos , S. , Ronse , A. , Zhian , M. : Assessing the Intuitiveness of Qualitative Contribution Relationships in Goal Models: an Exploratory Experiment . In: Proceedings of the 11th ACM/IEEE International Symposium on Empirical Software Engineering and Measurement (ESEM'17) . pp. 466 { 471 . Toronto, Canada ( 2017 )

12. Liaskos , S. , Tambosi , W. : Factors a ecting comprehension of contribution links in goal models: an experiment . In: Proceedings of the 32nd International Conference on Conceptual Modeling (ER 2019 ) (to appear) . Salvador, Brazil ( 2019 )

13. Maiden , N.A.M. , Pavan , P. , Gizikis , A. , Clause , O. , Kim , H. , Zhu , X. : Making Decisions with Requirements: Integrating i* Goal Modelling and the AHP . In: Proceedings of the 8th International Working Conference on Requirements Engineering: Foundation for Software Quality (REFSQ'02) . Essen, Germany ( 2002 )

14. Yu , E.S.K. : Towards Modelling and Reasoning Support for Early-Phase Requirements Engineering . In: Proceedings of the 3rd IEEE International Symposium on Requirements Engineering (RE'97) . pp. 226 { 235 . Annapolis , MD ( 1997 )