Goal models help stakeholders understand project scenarios and make decisions. In prior work, we used Tropos evaluation semantics to allow for automated analysis over time; however, formal evaluation labels (e.g., (F; ?)) are di cult for users to interpret across a large model. In this paper, we present our work towards understanding the extent to which using colors in goal modeling a ects users' ability to make decisions. Speci cally, we are interested in studying if coloring intentions with evaluation information allows for better comparisons of initial states and simulations of future paths. To address this question, we developed a color visualization extension to BloomingLeaf, a goal model analysis tool, where the color of each node is changed based on either the initial evaluations or the resulting analysis over time. This then allows us to explore if and how color visualization assists the user with decision making. We present our implementation and initial evaluation of this extension.
Goal modeling allows stakeholders to model and visualize their domain and
analyze trade-o s [
In our work, we aim to improve the interpretability of model evaluations
and analysis. We investigate the use of color to convey information about the
ful llment of evaluations, both statically and as they change over time. Previous
work in the iStar community looks at improving clarity of goal models through
visualizations and making them more user-friendly [
Running Example: Social Distancing Dinner. In the midst of the
COVID19 pandemic [
(P; P) (P; F) (P; ?) (?; P) (?; ?) (i) Running Example: Social Distancing Dinner (ii) A lattice of evidence pairs in
Tropos with colored labels. satis ed over time) and another where minimize spread of COVID-19 is prioritized. For each scenario, BloomingLeaf returns evaluation labels for all timepoints in a week, and Emma is unsure how to proceed. How can she easily know whether cook at home or order takeout is satis ed more often? What about the proportion of time that practice social distancing is partially denied or fully denied? While traversing the simulation path, Emma is unsure how to interpret the data and unclear about the optimal course of action.
Contributions. In this paper, we explore our research question: To what extent does applying colors to individual intentions in models and simulation paths improve model understanding and allow for better decision making? We present Evaluation Visualization Overlay (EVO), an extension to BloomingLeaf which colors the initial satisfaction values (i.e., evidence pairs) of each intention in a goal model and, after running analysis, overlays the proportion of time points that each intention holds each evaluation. With EVO, users can better visualize future states of their model, make comparisons between intentions, and understand trends in ful llment over time.
The remainder of this paper is organized as follows. Sect. 2 introduces relevant background about evaluation labels and BloomingLeaf. Sect. 3 explores our visualization approach. Sect. 4 discusses our preliminary evaluation and compares our work with prior visualizations. We conclude in Sect. 5. 2
BloomingLeaf is an online tool used to construct and analyze goal models using
the Tropos language [
Interactions between intentions are expressed through links. Contribution
links (e.g., +, -, ++, {) indicate an element's in uence on another. For example,
in Fig. 1(i), in order for cook at home to be satis ed, fresh groceries must also be
satis ed, indicated by a ++, while cook at home helps practice social distancing,
indicated by a +. Decomposition links (and /or ) decompose an intention into child
goals. An intention with an and [resp. or ] decomposition requires all [resp. only
one] of its children to be satis ed. Functions indicate an intention's evolution.
For example, Fig. 1(i) assigns fresh groceries a Monotonic Negative function
(MN) because Emma's groceries will expire throughout the week. After creating
a model of intentions, links, initial values, and functions, users can analyze their
model over time by simulating single paths (not shown). These are sequences of
states consisting of evidence pairs for each element in the model over an ordered
set of time points [
The goal of Evaluation Visualization Overlay (EVO) is to visualize both the initial state and path analysis of a model over time. Each evidence pair is assigned a color, with bluer shades closer to satis ed and redder shades closer to denied, as illustrated in the background of Fig. 1(ii). Thus, (F; ?) and (P; ?) are shades of blue and (?; F) and (?; P) are shades of red. Con icting evaluations with evidence for both satisfaction and denial are shades of purple, with (F; P) [resp. (P; F)] assigned shades of purple closer to blue [resp. red]. These colors are used both in modeling and analysis mode when EVO is enabled.
Implementation. Working within the architecture of BloomingLeaf, we implemented EVO as an on-o toggle option in the top toolbar, with its mechanics implemented in the JavaScript front-end. For the visualization of a single path in analysis mode, BloomingLeaf's back-end provides the necessary data, which is then stored in an encapsulating object. This object contains an array that corresponds to the evidence pairs of each intention at each time point. EVO while Modeling. Activating EVO in the modeling mode changes the color of each intention to correspond to its user-set initial satisfaction value. This provides an overall visualization of the model's initial state, allowing users to understand the color scheme and initial evaluations.
In our running example, Emma rst prioritizes minimize spread of COVID-19, assigning it as (F; ?) with a Constant function. Knowing her groceries deplete over time, she sets the resource fresh groceries to (F; ?) with a Monotonic Negative function. Interested in the initial state of the model, she activates EVO. She sees the two intentions above are blue (not shown), while the rest remain their original color. Because the majority of her model doesn't change, Emma knows she cannot answer her dinner question with only the model's initial state. EVO while Analyzing the Resulting Model. After running a single-path simulation in analysis mode, activating EVO displays colored stripes on each element. The width of each colored stripe corresponds to the percentage of the path that the intention is assigned a given evidence pair, ordered left to right from most to least satis ed. Thus, the user can gauge the evaluations of intentions over time without needing to walk through the simulation state by state.
In our example, Emma, simulates a path in analysis mode and sees the evidence pairs in the corners of the intentions while stepping through time points. However, she is unable to remember data from each previous time point and so nds it impossible to make sense of the evidence pairs at once. She then turns on EVO to see how prioritizing the satisfaction of minimize spread of COVID-19 impacts her dinner decision (see Fig. 2). While there are variations in the evaluations, Emma sees that cook at home is more satis ed than order takeout, and that it is never denied. This indicates that to minimize the transmission of COVID-19 in her life she should choose to cook at home. Emma also observes that not get or transmit COVID-19 and practice social distancing are mostly satis ed, but notices that minimize economic impact has the majority of evidence towards denied.
In a second scenario, Emma wants to reduce job loss and is concerned about her local economy. She clears the model and sets minimize economic impact to fully satis ed with a Constant function. Starting with EVO o , she is able to discern that while order takeout is fully satis ed, minimize spread is denied for several time points. However, toggling EVO reveals the whole picture of her results (see Fig. 2). While order takeout and have dinner are overwhelmingly satis ed, the other intentions have mixed results, and many are denied. Given this information, Emma understands the consequences of interacting with others and decides to cook at home to keep her and others the most safe.
EVO helps Emma understand trends in the model and answer her trade-o question: Which dinner option is fully or partially satis ed for the greatest proportion of time points under her chosen constraints? She can also draw conclusions about the unpredictability of her scenario with EVO by seeing discrepancies among multiple single paths. To explore EVO's versatility, we took nine models ranging in size from 14 to 121 intentions, some more deterministic and others more stochastic, and analyzed them with EVO enabled. For each model, we considered what information was clear before activating EVO and what we could not understand without it. We consolidated our observations into bene ts of EVO and areas of improvement. Bene ts. The toggle operation makes the user interface easy to use. In the modeling mode, EVO shows initial evaluations to visualize the initial state of the model. The colors are intuitive, with bluer shades equating to more satis ed and redder shades to more denied. In analysis mode, EVO provides the user with the \big picture" of their model by visualizing the proportion of time points the intentions hold each evaluation. Since the order of colors is consistent, it's easy to compare evaluation proportions across di erent intentions and simulation paths. Improvements. The chosen color palette is not optimal for colorblind users. EVO can be improved by enabling a colorblind mode. An intention that starts (F; ?) and evolves to (?; F) has the same visualization as the reverse, because the colors are always in the same order. An additional view showing changes in order would di erentiate these two cases. EVO can also be improved to visualize any partial point of interest after simulating a single path. EVO is limited in the modeling view; it's only useful for select models with well-de ned initial states. Plan for Validation. We have con dence EVO bene ts users in drawing conclusions from goal models, given the distinct di erences observed between using it and not; however, we only provide an initial evaluation. To quantify and validate the functionality of EVO, we must test it among a broader range of users.
We plan to conduct a study comparing BloomingLeaf's usability in decision
making with and without EVO by providing di erent models to two randomized
groups of students trained in goal modeling. One group's tool will have EVO
enabled, while the other group will only use the simulated path. The students
will then answer questions about optimal decisions and evaluations over time.
Related Work. Aprajita proposed using heatmaps to depict overall satisfaction
(see Fig. 11 of the associated thesis [
EVO visualizes evaluation trends in single-path simulations as well as models'
initial states, which allows for easy comparison across intentions within models.
This work could have implications for decision making in other iStar tools [
Before drawing conclusions to our research question, a more thorough analysis across a broad range of potential users is required. Additionally, future work ensures our color palette is appropriately clear for all users. We are also developing a variation of EVO that visualizes the evaluations at every time point over a single path. This variation allows users to see changes of intentions over time instead of their unordered overall evaluation.
Acknowledgments. Varnum was supported by a Smith College STRIDE Scholarship.