Group-based configuration systems support scenarios where a group of users configures a product/service. In those groupbased configuration scenarios where the knowledge of some group members regarding items is insufficient, an advice of experts is necessary in order to help members to evaluate products or services. This paper introduces a novel approach which takes advantage of the concept of liquid democracy that allows the delegation of group member votes to experts. Concerning the application of liquid democracy, we propose a new approach based on Multi-attribute Utility theory (MAUT)-based evaluation used to calculate the utility of configurable items. Compared to the traditional approach, the proposed MAUT-based evaluation focuses on the role of experts by assigning higher weights to them. Additionally, the respective expertise level of the experts is taken into account. Consequently, the main contribution of this paper consists in the improvement of group-based configuration by taking liquid democracy aspects into consideration.
Configuration [
In the context of group-based configuration, sometimes, some
group members may be unable to evaluate the dimensions of a given
set of items due to a knowledge gap. Hence, in order to precisely
evaluate items, group members have to invest much effort in order
to collect necessary information as well as to analyze items [
Recently, a variety of studies regarding liquid democracy have
been conducted for the purpose of making better use of the so-called
”wisdom of the crowds” [
The remainder of the paper is organized as follows. In Section 2, we describe a group-based configuration scenario in requirements engineering which is used as a working example throughout the paper. In Section 3, we discuss how liquid democracy can be applied to group-based configuration in order to transfer the rating power from group members to experts. Section 4 presents a new approach of MAUT-based evaluation to calculate the utility value of a requirement. Section 5 discusses how requirements can be assigned to releases based on their utility value, their effort estimation, existing dependencies between requirements, and the capacity of releases. Finally, Section 6 draws a brief conclusion and provides some ideas for future work. 2
For demonstration purposes, we introduce a group configuration scenario occurring in a small requirements engineering example project where we configure a release plan. In this context, we define a set of requirements (R1; R2; R3, and R4) for developing a sport watch. These requirements are defined by a group of engineers with longstanding experience and practical knowledge in requirements engineering. Each requirement is described by an id, a title, and a textual description (see Table 1).
Id R1 R2 R3 R4
Title Evaluation Software DataStorage Function GPS Display lighting
Description To evaluate the collected training data, an evaluation software is required. The evaluation software requires the connection and the access to the clock’s internal memory. The evaluation should contain measured information regarding the distance, the height, the average heart rate, and the calorie consumption.
To evaluate the measured data, a storage service is required. The internal memory is used for saving the measured information, such as the distance, the height, the average heart rate, and the calorie consumption. The stored data will be used by the evaluation software.
To identify the position, a GPS sensor is used. Based on the measured position and time information, the speed and the distance can be measured.
The sport watch needs a display lighting to be operated at dusk.
In this example, we assume a situation where a group of five stakeholders (i.e., users) reads requirements, evaluates them regarding different dimensions of requirements, and assigns them to different releases (i.e., release planning configuration). We defined two different releases which are shown in Table 2.
Given the sets of requirements and releases, we assume that each stakeholder evaluated requirements with regard to the following dimensions: risk, effort, and profit. The dimension of risk indicates ”the estimated risk for developing a requirement”, effort represents ”the estimated total work done for developing a requirement”, and profit corresponds to ”the estimated profit of a requirement”. These Releases Release 1 Release 2
Capacity (in hours) 260 260 dimensions are evaluated using ratings. Thereby, the ratings can lie in the range between 1 and 5, where an evaluation of 5 indicates a requirement with low risk, high profit, and low effort and an evaluation of 1 represents a requirement with high risk, low profit, and high effort.
The evaluation of stakeholders with regard to different dimensions of requirements is shown in Table 3. In this table, some group members did not sufficiently evaluate dimensions of a requirement (i.e., some dimensions were not evaluated by stakeholders). For instance, the first stakeholder (S1) did not evaluate the profit of the requirement R2. In addition, there also exist some stakeholders who did not evaluate any dimension of a requirement. For instance, the fourth stakeholder (S4) did not evaluate any dimension of the requirement R1. A possible reason for the existence of such missing evaluations can be lacking expertise or knowledge of some stakeholders regarding the meaning of some requirements. In this scenario, those stakeholders who do not know much about the content of some requirements have to invest a lot of effort in order to acquire enough necessary knowledge and to analyze the requirements. This triggers a high cost of the requirement evaluation process. Alternatively, the stakeholder could ask for the advice of some experts to provide more precise evaluations with regard to dimensions of requirements. In other words, the stakeholder directly passes his/her evaluation power to experts by using liquid democracy (see Section 3). In the context of requirements engineering, the expert can be a requirement engineer who has longstanding experiences and practical knowledge of requirements. The consultation of experts helps to precisely evaluate the dimensions of items.
Alternatively, in some cases empty evaluations could be trigged by the reason that the stakeholder does not want to evaluate some dimensions of a requirement. Moreover, he/she also does not want to delegate the rating power to anyone else. In this scenario, groupbased configuration systems will automatically check the quantity of complete evaluations of requirements and the configuration phase is only complete if this quantity is high enough. In this example, we assume that the quantity of complete evaluations should be equal to or greater than 80% of the total number of all evaluations. In other words, the configuration phase will not be finished until the quantity of available evaluations reaches 80%.
In addition to that, when evaluating a requirement, stakeholders can assign different weights to dimensions. Thereby, the weight is referred to as the importance of a dimension, i.e., the higher the importance of a dimension, the higher the weight. Different stakeholders could assign different weights to the same dimension. For instance, one stakeholder (e.g., developer) assumes that the effort of a requirement is the most important dimension, whereas another stakeholder (e.g., project manager) evaluates the profit to be the most important dimension of a requirement. In order to limit the scope of this paper, some simplifications have to be made. For the sake of simplicity, we assume that the importance (i.e., weight) of all dimensions for all stakeholders is equal and all dimensions have a weight of 1 from the stakeholders’ point of view (i.e., 8 s 2 stakeholders, weight(s, risk) = weight(s, profit) = weight(s, effort) = 1). 3
Liquid democracy is a hybrid voting model of participative
democracy which combines direct and representative democracy
approaches in order to empower electors [
In this paper, we use a liquid democracy approach in order to complete evaluations of dimensions which were not evaluated by stakeholders. As shown in Table 3, we can observe that stakeholders did not evaluate all dimensions of requirements. In this example, we assume that stakeholders S2, S4, and S5 need experts to complete their evaluations. Expert selection can be done by one of two approaches. The first approach is to select only one expert for the three aforementioned stakeholders. The second approach is to allow each group member to select his/her own expert. In our example, we choose the second approach where each stakeholder chooses different experts for different requirements. For instance, regarding the requirement which is related to the user interface, the stakeholder can choose an expert who has experiences in user interface design. For the data storage-related requirement, the stakeholder can choose an expert who has knowledge of data management. In our approach, the expert selection process is done automatically by the recommender system. That means, experts on a specific topic are automatically identified and recommended to the stakeholder. Alternatively, each stakeholder is allowed to select experts who are not included in the recommended list. In our approach, the recommender system suggests experts based on the expertise level. In the context of requirements engineering, the expertise level of an expert can be calculated based on the following criteria: working experience, skills, number of contributions in requirements engineering projects, and number of delegations received in the requirements engineering domain. The expertise level is in range of 1 to 5. The expertise level of 5 indicates excellent topicrelated knowledge, whereas the expertise level of 1 represents limited knowledge.
In this paper, we exemplify an expert recommendation process with 5 experts in the requirements engineering domain. Table 4 shows a recommended list of experts ranked in a descending order of the expertise level. In addition, stakeholders who want to delegate evaluations to other experts can select different experts for different requirements. As shown in Table 1, the development of the defined requirements requires deep knowledge with regard to different areas. Therefore, selecting an appropriate expert for each requirement helps to increase the overall quality of requirements engineering. The expert selection for stakeholders S2, S4, and S5 are depicted in Table 5. In this table, we can observe that stakeholders select different experts for different requirements. For instance, stakeholder S2 requires experts’ evaluations regarding the dimensions of requirements R1, R2, and R3. The stakeholder S2 chooses Expert3 for R1, Expert2 for R2, and Expert4 for R3. Furthermore, we can observe that the stakeholder S2 does not need any expert for R4 and this is represented by a dash symbol (”-”) in Table 5.
Experts Expert2 Expert5 Expert4 Expert1 Expert3
Expertise-Level (sport watch domain) 4.5 3.75 3.15 2.25 2.05 were not evaluated by the stakeholders S2, S4, and S5. The evaluations which were given by experts are shown (in bold) in Table 6. Next, the utility of each requirement has to be calculated and used as one of the important criteria to assign requirements to releases. The utility of each requirement is calculated based on Multi-attribute Utility Theory (MAUT) (see Section 4). 4
As already mentioned before, configurable items are usually
described by a set of dimensions. In this context, Multi-attribute
Utility Theory (MAUT) [
P s2stakeholders
d2dims w(s;d) w(s) jstakeholdersj (3)
An example of the utility calculation of a requirement is presented in Formula (4). In this example, for simplicity, we assume that all stakeholders assign the same weight (i.e., the weight of 1) for all dimensions of requirements (8s 2 stakeholders, 8d 2 dimensions, w(s; d)= 1). Additionally, we assume that each stakeholder has also the same importance (8s 2 stakeholders weight(s)= 1).
U tility(R2) = = + + =
P Pd2dPims eval(s;d) w(s;d) w(s) s2stakeholders d2dims w(s;d) w(s)
jstakeholdersj 1 2 1 + 4 1 2 1 + 3 (1 2 + 4:5) + 3 1
+ 5 1 + 1 1 + (1 2 + 4:5) + 1 5 1 + 4 1 4 1 + 4 (1 2 + 4:5) + 2 1
+ 1 + 1 1 + (1 2 + 4:5) + 1 3 (1 2 + 3:75) + 2 (1 2 + 3:75) + 4 1
(1 2 + 3:75) + (1 2 + 3:75) + 1 1 6 24:5 9 32 32:75
+ + + + = 3:354 5 2 8:5 2 8:5 12:5
Similarly, MAUT values of other requirements are also calculated by using Formulae 1 - 3. The MAUT values of requirements R1; R2; R3, and R4 are the following: M AU T (R1) = 3:266, M AU T (R2) = 3:354, M AU T (R3) = 3:380 and M AU T (R4) = 3:326. After the calculation of requirement utilities, requirements will be assigned to defined releases (see Section 5). 5
In Section 4, we showed how the utility value of a requirement can be calculated based on Multi-attribute Utility Theory (MAUT). The higher the MAUT value, the sooner the requirement will be implemented. In the context of requirements engineering, making a recommendation of requirements is referred to as release planning, i.e., to clarify which requirement should be implemented in which release. In release planning, stakeholders have to estimate the effort investing for each requirement. In our working example, the effort is referred to as the invested time (in hour) to implement a requirement. The higher the evaluation of effort, the lower the invested time. We assume that an evaluation of 5 corresponds to an effort of 50 hours and an evaluation of 1 corresponds to an effort of 250 hours. In order to calculate the effort of a requirement, we first calculate the average of evaluations with regard to the effort of the requirement given by all stakeholders. After that, the effort (in hour) is calculated using the Formula 5, where ef f ort(Ri; s) is the evaluation of the stakeholder s about the effort of the requirement Ri. (4) ef f ort(Ri) = 5
jstakeholdersj (5)
We exemplify the calculation of the effort of the requirement R1 as shown in Formula 6. The effort values of other requirements are calculated in a similar way and presented in Table 7. (6) requirements, dependencies between requirements, their utility values, and capacity of releases. The effort of each requirement in represented in the second column of the table.
In our example, release planning is done based on four criteria: the utility (MAUT) value, effort (measured in hours), the dependency between requirements, and the capacity of releases. Given the fact that requirement R3 achieves the highest utility (i.e., M AU T (R3) = 3:380) and its estimated time effort of 120 hours, R3 turns out to be the best candidate to be assigned to Release 1. Furthermore, it is reasonable that requirement R2 should follow R3 and hence also be assigned to Release 1, as R2 shows the second highest utility and there is still some remaining capacity left for Release 1 to cover R2 in this release (i.e., capacity(Release 1) = 260 hours). Next, requirement R4 has to be assigned to a release. The assignment of requirement R4 to the first release is not possible, because of the limited capacity ( remaining capacity (Release 1) = 10 hours and effort(R4) = 83.33 hours). Therefore, requirement R4 is assigned to the second release. Finally, requirement R1 is assigned to the second release. Based on the description of requirements shown in Table 1, we can observe that there is a dependency between R1 and R2 which is indicated as follows: ”The evaluation software requires the access to the clock’s internal memory”. In the context of requirements engineering, this means that the first requirement R1 (i.e, evaluation software) can not be implemented before R2 (i.e., data storage function) has been completed. However, in this scenario, the identified dependency will not trigger any changes in the release planning since the release plan in Table 7 shows that requirement R2 which is assigned to the first release (development period: from 2020-05-01 to 2020-07-01) will be implemented before all the requirements assigned to the second release (development period: from 2020-07-15 to 2020-09-15). With this final step, all requirements are assigned to releases and the requirements engineering process is complete. 6
In this paper, we introduced utility analysis concepts which focus on liquid democracy. These concepts allow the manual delegation of a stakeholder’s voting right to a domain expert. First, we described a scenario for the development of a sport watch which is used as a working example throughout this paper. Based on the working example, we applied liquid democracy in order to receive consultations from experts in situations where stakeholders do not have enough knowledge with regard to certain requirements. Afterwards, we proposed a novel approach of a MAUT-based evaluation which takes into account users’ and experts’ evaluations and assigns higher importance to expert consultations (i.e., evaluations). Finally, we proposed a group-based configuration for release planning where requirements were assigned to releases based on derived utility values, effort estimations, existing dependencies, and release capacities.
Within the scope of future work, we plan to integrate the proposed approach in a requirements engineering tool named INNOSENSR1. INNOSENSR is a modern innovative release planning tool which makes use of intelligent techniques in order to facilitate the complete requirements engineering process. In the current version of INNOSENSR, stakeholders have to evaluate requirements without getting any support from domain experts. In the future, we will integrate our approach into INNOSENSR in order to increase the requirements engineering quality.
The work presented in this paper has been conducted within the scope of the Horizon 2020 project OPENREQ (732463). 1 http://innosensr.com