Decision support systems in many cases are based on user interfaces used to collect preferences and requirements of users. For example, configurators in the automotive domain ask users to provide preference information regarding the car color and car engine. Stakeholders in release planning scenarios provide feedback on software requirements in terms of importance evaluations of different interest dimensions. In such scenarios, decision biases can trigger situations where users take suboptimal decisions. In this paper, we provide a short overview of example decision biases and report the results of an empirical study that show the existence of such biases in the context of release planning (configuration) decision making.
When interacting with decision support systems such as
recommender systems [
Recently, decision support for groups became increasingly
popular due to the fact that in many scenarios user groups are engaged
in decision processes [
The remainder of this paper is organized as follows. In Section 2, we exemplify decision biases on the basis of examples from the domain of software requirements engineering. In this context, we discuss the results of a related empirical study. In Section 3, we discuss issues for future work. We conclude this paper with Section 4.
The major goal of our study was to analyze biases in decision scenarios. The study focused on an analysis of the decision behavior of computer science students (N=222) working in groups of 6-8 persons in a software project. A structured questionnaire with A/B testing was used to analyze the decision behavior of students. The average time needed to complete the questionnaire was 4 minutes. In order to simulate decision scenarios, scenario descriptions where integrated in questions where needed. In the following, we provide an overview of the study results.
Framing. The way a decision alternative is presented can
influence a user’s decision behavior. One example of framing is that users
prefer meat that is characterized with ”80 percent lean” over meat
that is ”20 percent fat”. Another example is the framing of prices:
when comparing the offers of company x and y, the offer ”pellets
fore 24.50 per 100kg with a discount of 2.50 if the customer pays
with cash” from x appears to be the more attractive one compared
to the offer ”pellets for 22.00 per 100kg with a 2.50 surcharge if
the customer pays with credit card” from company y. The increased
attractiveness of x’s offer can be explained by prospect theory [
Framing: Study Results. In our study, we described a scenario where stakeholders had to estimate the acceptability of a given probability of successful project completion. In one setting, the probability was specified as ”probability of success”, in the other setting, the probability was expressed as ”failure probability”. In the first setting, study participants evaluated the acceptability on an average with 86 out of 100 points (1: not acceptable, 100: definitely acceptable). In the second setting, study participants evaluated the acceptability on an average with 77 points (out of 100).
Anchoring. It is known that preference visibility has various
negative impacts on the quality of a group decision. An example thereof
is anchoring where indicated reference values (e.g., item evaluations
of a group member) can have an influence on the evaluation behavior
of other group members. An example thereof is the visualization of
the average rating given to an item by a community: increasing the
shown average item rating results in a increased rating by individual
community members [
Anchoring: Study Results. In our study, the participants were asked
whether it is important for them to have as soon as possible
knowledge about the preferences (which requirements should be
implemented when?) of other stakeholders. Nearly 70 percent of the study
participants agreed on the fact that the mentioned preference
visibility is important. These persons are vulnerable to limited information
exchange which can result in suboptimal decisions [
Decoy Effects. Decisions are taken depending on the context in which the alternatives are presented. It can be the case that completely inferior decision alternatives added to a set of alternatives can change the selection behavior of users. Such alternatives are denoted as decoy items since they manage to draw the attention of users towards specific alternatives. An example of a decoy effect is asymmetric dominance which denotes a situation where a decoy alternative is dominated by an item T in all dimensions. Dominance is evaluated in terms of a pairwise comparison of attribute values characterizing the alternatives. An example of asymmetric dominance is shown in Table 1. Alternative c can be regarded as a decoy item since it is outperformed by alternative a in both dimensions (higher project returns and lower project efforts) and thus makes alternative a even more attractive compared to alternative b.
release a b c project returns 30.000 50.000 28.000 project efforts 15.000 35.000 16.000
Decoy Effects: Study Results. The study participants were asked to select one out of two alternative software release plans (characterized by the corresponding estimated returns and efforts). Release alternative c is completely dominated by release alternative a which has been selected in 86 percent of the cases (only 9 percent of the participants selected alternative b).
Table 2 includes a variant of the previous setting where alternative c is arranged near to alternative b. Compared to the setting shown in Table 1, the share of participants selected this alternative was only 77 percent wheres 22 percent of the participants selected alternative b. Consequently, the inclusion of inferior alternatives can trigger a shift in the selection behavior of stakeholders. One way to counteract such situations is to point out inferior alternatives or to simply delete these from the set of available options.
release a b c project returns 30.000 50.000 52.000 project efforts 15.000 35.000 40.000
Decision Strategies of Study Participants. In addition to the above
mentioned biases, the study participants were asked a couple of
questions regarding their practices in group decision making. First, early
knowledge about the preferences of other stakeholders was
considered as a positive element that helps to improve the quality of
requirements prioritization (84% of the study participants supported
this statement). However, as indicated in the literature, early
knowledge about the preferences of other stakeholders can have a negative
impact on decision quality since focusing on preferences triggers less
efforts related to the exchange of decision-relevant information [
Summarizing, biases in preference acquisition exist and can have
a negative impact on the outcome of the decision process. As a result
of our user study, it could be observed that study participants (in our
case Computer Science students) were often not aware of this and
thus vulnerable to such biases.
There are a couple of issues that are within the scope of our future
research. First, the majority of researchers still focuses on the
identification of new biases and the analysis of biases in specific decision
scenarios. A major goal of our ongoing and future work is to focus on
approaches to automatically identify potential sources of suboptimal
decisions and to adapt the underlying decision support. For example,
decoy effects can be predicted on the basis of a formal model [
In this paper, we discussed the results of a user study related to the existence of decision biases in preference acquisition. The results were discussed on the basis of an empirical study that was conducted with computer science students within the scope of a software engineering course. The outcomes of this study clearly indicate the existence of decision biases and suboptimal decision practices that can lead to suboptimal outcomes in group decisions. Our future work will include a.o. an analysis to which extent explanations can help to counteract decision biases. Furthermore, we will extend the scope of our user studies to industrial scenarios.
The work presented in this paper has been conducted within the scope of the Horizon 2020 project OpenReq (openreq.eu).