=Paper=
{{Paper
|id=Vol-2019/commitmde_1
|storemode=property
|title=Supporting Consensus-based Sofware Development: a Vision Paper
|pdfUrl=https://ceur-ws.org/Vol-2019/commitmde_1.pdf
|volume=Vol-2019
|authors=Mathieu Lavallée,Guillaume Beaulieu,Michalis Famelis
|dblpUrl=https://dblp.org/rec/conf/models/LavalleeBF17
}}
==Supporting Consensus-based Sofware Development: a Vision Paper==
https://ceur-ws.org/Vol-2019/commitmde_1.pdf
Supporting Consensus-based Software Development:
a Vision Paper
Mathieu Lavallée Guillaume Beaulieu Michalis Famelis
Independent consultant Université du Québec à Montréal Université de Montréal
math.lavallee@gmail.com beaulieu.guillaume.2@courrier.uqam.ca famelis@iro.umontreal.ca
Abstract—Traditional, vertical organizational models of soft- More More
horizontal vertical
ware development have been challenged by more agile and col-
laborative structures. Recently, this has also been demonstrated
in the emergence of explicitly horizontalist organizational struc- Consensus-based Agile Traditional
tures, focused on consensus-based decision making. In this paper, communities (CBCs) "Flat" organizations management
Holarchies
we describe the principles and processes of these “Consensus-
Based Communities” (CBCs) and outline the main challenges Fig. 1: The continuum from vertical (i.e. hierarchical) to horizontal
they face as they try to implement “Consensus-Based Software (i.e. non-hierarchical) organizations.
Development” (CBSD). We express these as early, high level
requirements for a tool supported methodology. Based on these,
we present and analysis of existing tools that shows that no an “holarchy” [11], where decisions are delegated to self-
single tool provides complete support for consensus-based group managing teams. Often, organizations using self-managed
decision making. We thus outline directions for future research,
identifying opportunities for the development and deployement
teams still maintain a higher authority, simply reducing the
of model-based techniques in this emergent field. amount of middle management. In Figure 1 we position Agile
teams, that promote self-organized GDM, in the middle of the
I. I NTRODUCTION continuum, along with flat and holarchic organizations.
Software development is a team activity [1], [2], in which In this paper, we put the spotlight on the other end of the
internal team dynamics impact the quality of the produced continuum. This is populated by software development orga-
software. A classic formulation of this is “Conway’s Law”: nizations that intentionally adopt explicitly anti-hierarchical
Any organization that designs a system (defined and consensus-based modes of GDM and collaboration. For
broadly) will produce a design whose structure is a example, Koumbit, a web-hosting company based in Mon-
copy of the organization’s communication structure. treal, describes its internal processes as “non-hierarchical self-
[3] management” [12]. Loomio is a developer-owned cooperative
More recently, Nagappan et al. showed that organizational based in New Zealand that puts consent at the centre of its
structure was a better predictor of fault-proneness than tra- decision making process [13]. Other organizations like Zappos
ditional technical metrics [4]. A defining aspect of a team’s [14] and Medium.com [15] maintain horizontal principles.
organization and communication structure is Group Decision This is also often the case for open source communities,
Making (GDM), also known as Multi-Person Decision-Making such as Drupal, which favours a self-managed approach [16].
(MPDM), [5], [6], [7]. Traditionally, organization structures We call such organizations Consensus-Based Communities
are hierarchical and command based. However, the advent of (CBCs).
Agile software development has changed GDM processes, as CBCs face a set of unique challenges when developing
it tends to prefer horizontal structures [8]. Characteristically, software, mostly based on their philosophical approach to
the Agile Manifesto declares: GDM. The Agile Manifesto already pushed GDM away from
The best architectures, requirements, and designs the “Benevolent Dictator” management paradigm.
emerge from self-organizing teams. [9] Agile Software Development has completely dis-
In practice, the degree of self-organization varies widely pensed with the formal job title of the project man-
across different organizations. We illustrate this diversity in ager. Agile methodologies such as scrum seem to
Figure 1, as a continuum of organizational horizontality in distribute the erstwhile responsibilities of the project
software development teams. On the one end there is tra- manager into new roles. [8]
ditional “command-based” development, where upper levels CBCs affirm that team decisions must go a step further, and
of the organizational hierarchy have the final say on what ensure consensus building as much as possible, instead of
developers do. Moving from that extreme, some organizations majority decision-making. The rationale behind striving for
maintain the use of project managers who have the final say in consensus is based on a perceived flaw of democratic decision-
decisions while encouraging the building of a team consensus. making called “tyranny of the majority”:
Some organizations actively pursue a “flat” model [10] or The will of the majority may be seen as the will
of the whole group, with the minority expected to point out areas for improvement in Section VI. We discuss
accept and carry out the decision, even if it is against related work in Section VII and conclude in Section VIII.
their deeply held convictions and most basic needs.
It is possible for a voting group to look for solutions II. BACKGROUND
that would suit everybody, but it is more common for To better understand the unique challenges in CBSD, we
ideas with a majority backing to be pushed through. first outline the main characteristics of CBCs.
[17, p. 9] Consensus-based decision-making does not mean that ev-
This flaw can cause, for example a team with a majority of erybody must agree on everything. In practice, consensus
non-experts to overrule a minority of experts, leading to poor is typically “soft”, which means that certain forms of dis-
decisions. Instead, consensus-building requires team members agreement are acceptable. According to the Consensus Hand-
to convince all their colleagues, instead of unilaterally impos- book [17, p. 6]:
ing their propositions. This means that software developers in
Instead of simply voting for an item and having
CBCs cannot exploit the process in order to shut down their
the majority of the group get their way, a consensus
colleagues, but must provide facts and data able to persuade
group is committed to finding solutions that every-
them to change their mind.
one actively supports, or at least can live with.
CBCs such as the ones mentioned above also operate under
anti-hierarchical principles. Their perception is that software The aim is to avoid a situation where there are winners and
development has become so complex that not one person can losers. CBCs claim that this helps everybody feel involved in
see the whole picture. Each software developer is therefore the organization, and helps them learn new skills and be better,
a piece of the puzzle, each with their own expertise, and more complete software developers [12]. This further increases
decisions should then be made with everyone in mind [7]. the sense of agency and ownership towards decisions, since
Even though CBCs are currently not the norm in software each individual actively participates, ensuring that everyone’s
engineering, studying them can provide interesting insights on concerns are part of the process. A major benefit of consensus
team dynamics, and can help identify beneficial organizational decision-making is therefore the increased solidarity among
practices across the continuum of organizational horizontality. team members in cases where bad decisions are made. Faced
Broadly speaking, the recent move away from traditional with a negative result, since the whole team was engaged in
organizational practices can be understood as a concern about the decision together, no single individual or group can be
the degree to which they are appropriate for software de- singled out for blame [19].
velopment. Thus, interesting insights can be obtained from In the following, we describe a typical consensus-based
radically different organizational structures [10]. At the same GDM process, in terms of the activities performed and
time, since CBCs are software producing organizations in an the roles and responsibilities associated. Consensus-based
era of software ecosystems [18], understanding the impact of decision-making processes are adapted from the different kinds
their practices on software quality (cf. Conway’s Law) may of democratic practices surrounding them [20]. They are
have ramifications beyond their niche. adapted to the size and ability of the group.
In this paper, we focus on CBCs and propose a vision While the exact process varies from one organization to
for a comprehensive, model-based, tool supported method- another, we present here a basic model of typical activities in
ology for Consensus-Based Software Development (CBSD) consensus-based GDM [17, p. 16]. These are:
that addresses their unique set of challenges. We identify Introduction (A1): Introduction of the issue on which a
the specific needs of CBCs and study existing collaboration decision must be made;
tools to find gaps between the two and potential avenues of Discussion (A2): Discuss the issue and collect ideas;
research and development. Specifically, we make the following Concerns (A3): List concerns regarding the proposal;
contributions: Proposal (A4): Identify emerging proposals meeting con-
a) We highlight the existence of CBCs and describe their cerns;
unique processes and challenges; Refinement (A5): Discuss, clarify and amend the proposals;
b) We describe how consensus-based GDM can be sup- Consensus test (A6): Test for agreement, usually through a
ported for software development; straw poll. If not, return to the previous points;
c) We analyze the capabilities of existing tools, identify their Implementation (A7): Implement the decision.
limitations, and outline how model-based techniques can We show these activities diagrammatically in Figure 2.
be used to improve the state of CBSD practise. During a test for consensus (activity A6), a typical CBC
The rest of the paper is organized as follows: We present uses four types of votes for the straw poll [17, p. 16] with the
background information on CBCs in Section II, detailing following meaning:
their typical internal processes In Section III, we explain the Block (V1): The participant thinks the proposal is funda-
challenges raised by consensus-based GDM and in Section IV mentally wrong and should not be implemented by the
we outline the requirements for a tool framework for support- organization.
ing consensus-based software development. We analyze the Stand aside (V2): The participant cannot support the pro-
capabilities of existing GDM support tools in Section V and posal and is not willing to implement it. However they
R1. Facilitator includes the minutes of the meeting, along with the decisions
R4. Participant
and action items. It is recommended that action items be
A1. Introduction
tracked independently from the rest in order to avoid losing
R3. Hand takers
track of them and to prevent power concentration into the same
Issues F1.
hands [21].
to adress A2. Discussion Active listening
F2.
? Summarising
A3. Concerns III. C HALLENGES OF C ONSENSUS -BASED GDM
A4. Proposal
F3. Software development teams that adopt a consensus-based
Synthesis
V1. Block GDM process face a unique set of challenges. These stem from
A5. Refinement the need to work and reach consensus. Degenerations of the
process can introduce problems such as false consensus, where
V2. Stand aside Issues Consensus
raised A6. Consensus test an explicit agreement is reached, but where team members
V3. Reservation reached
? ? implicitly disagree. We group such challenges along seven
different dimensions.
Acceptable A7. Implementation V4. Agreement
One dimension (D1) is related to the heterogeneity of
issues communication [7]. For example, software engineering teams
? R2. Minute takers
include a variety of experts of different fields, depending on
Fig. 2: Consensus decision making model. the needs of the project. These different experts can be profi-
cient in different languages and techniques. For example, one
expert can be well-versed with the UML language while the
are willing to let the organization implement it without rest of the team is not. This creates a hurdle for communication
them. as the expert struggles to explain his point to non-experts.
Reservation (V3): The participant has some reservations, but Another dimension (D2) is related to the need to persuade
is willing to let the proposal be implemented and to work others in order to obtain consensus. Ideally, this persuasion is
for its implementation. based on real facts, brought by real experts, but in practice,
Agreement (V4): The participant supports the proposal and emotional factors can enter the equation. For example, “people
is willing to implement it. are easily persuaded by other people that they like” ([22],
The responsibilities associated with the good functioning quoted by [23]). Another example, “consistently slandering
of the team are traditionally assigned to a team manager. someone behind his back” [21, p. 33] is another tactic to
Instead, in self-managing teams, they must be delegated to persuade others using unsavory means.
team members. Consensus-based teams recommend to split This leads to the third dimension (D3) of how to ade-
responsibilities to different people to avoid a concentration of quatly generate alternatives for a complex issue. Ideally, GDM
powers which would re-create a hierarchical structure [21]. follows a rational decision-making approach. This approach
Specific roles include [17], [21]: is akin to a breadth-first approach, where all alternatives
Facilitators (R1): A neutral party whose responsibility is to are studied and compared. The opposite is the naturalistic
keep the meeting on topic and assist the decision-making decision-making approach. This approach is akin of a depth-
process. In some cases, no single person is assigned the first approach, where one easy to find solution is studied in-
role, instead relying on the participants, assuming that depth [24]. For example, a typical software development team
they have internalized it enough to self-discipline. mixes the two approaches, going in-depth with a small number
Minute Takers (R2): Responsible for keeping track of the of alternatives [25]. Generally this is because the complexity of
decisions and action items. software development context makes a breadth-first approach
Hand Takers (R3): Responsible of keeping track of whose impossible and requires limiting the breadth to what seems
turn it is to speak next and control the time allotted to possible at the time [24], [25].
each speaker. Generating and choosing the proper alternatives can be
Participant (R4): Each team member is responsible for self- difficult, especially with dysfunctional teams (D4). Issues of
discipline and for the good functioning of the process. “groupthink”, where there is a reluctance of criticizing ideas,
Facilitation is a complex role, which can be broken down are well documented [26]. Instances of polarized factions
into the following responsibilities [17, p. 31]: constantly blocking the propositions of the others are also
Active listening (F1): To assist speakers in expressing their a symptom of a dysfunctional team [26], [27]. Even before
ideas and its associated context. such problems arise, social stratification due to factors such
Summarising (F2): To reformulate the intervention of the as racism, sexism and class, generally makes marginalized
speaker in a clear and concise manner in order to confirm people take less share in the decision-making process. Worse,
its interpretation. that phenomenon is often justified by those who are silencing
Synthesis (F3): To merge the ideas in actual proposals po- themselves as they perceives themselves as less competent
tentially acceptable for all participants. [28]. Creating a context for genuine participation is an ongoing
The list of work products of a consensus-based process process that requires patience and dedication.
D2. D1.
Evaluate Manage the difference
and expertise
A1. and of languages
Introduce the issue UC7.
Identify expertise
A2. and UC6.
UC1. Support whiteboarding A7.
Discuss the issue helps Implement the
Present the issue helps and
and helps decision
A3. UC5.
List concerns and
helps
Supporting Monitor decisions
UC2. D5.
helps
A4. and
Manage the proposals
the CBSD and Self-monitor
Keep track of proposals
process decisions
and helps
A5.
Refine proposals helps
and UC4.
UC3. Detect consensus
Document the arguments
and
R2. and
Take the minutes and and A6.
Test for consensus
R3. F3. and and and and
Manage speakers Synthesize V1. V2. V3. V4.
interventions Record Record stand Record Record
block votes aside votes reservation votes agreement votes
Fig. 3: Use-cases to support for the CBSD process, using the i* modeling language.
The fifth dimension is related to the nature of anti- to the letter, but changes are actually frequent: Since there
hierarchical organizations (D5). Without a manager, the de- are no leader to impose a process, any meeting can result
velopers themselves must assume responsibility of the task in a process change. Hence, the bulk of the decision-making
typically assigned to managers. These include managing con- process is centred around educating participants on the current
flicting priorities between stakeholders, assuring that action organizational context. Documentation of such processes is
items are actually implemented, and giving more weight to contextual and reflects a community’s ongoing organizational
the intervention of experts [6], [10]. dynamics. Therefore, the given organizational context of the
Another challenge is related to documenting the rationale CBC (D7) will have an impact on its GDM, more so than
of the decisions made (D6). As [29] writes: in a traditional contexts using common GDM practices. The
rationale of the decision (D6) cannot be understood outside
In the field of software architecture, there has been a
this constantly evolving organizational context of the CBC
paradigm shift from describing the outcome of archi-
(D7).
tecting process mostly described by component and
connector (know-what) to documenting architectural IV. R EQUIREMENTS FOR CBSD SUPPORT TOOLS
design decisions and their rationale (know-how)
We identify the high-level requirements for CBSD tools,
which leads to the production of an architecture. [29]
based on the processes, activities, roles, and artifacts involved
A GDM support tool should therefore document decision in CBSD described in Section II, as well as the challenges
rationale to ensure that future developers can understand what outlined in Section III. We present the requirements in the
arguments supported the decision. Ideally, such a tool would be form of use-cases, listed below. We also give the requirements
able to see how decisions evolve [30]. For example, developers as goal models in Figures 3 and 4 using i*, a language for
should know that a decision can be changed if the underlaying modelling early requirements [31], [32].
arguments are no longer valid. UC1: During the introduction of an issue, present its context
But the technical context of the issue is not the only context in a concise yet understandable way (A1).
to manage. Since CBCs are not the norm, the GDM process UC2: Collect the ideas brought during the meeting (A2),
is generally a two-way educational process, first teaching obtain concerns from the participants and keep track of
newcomers about the GDM process, but also teaching ways them (A3). Turn ideas into actionable proposals that ev-
to address concerns in a constructive manner. With consensus- eryone can inspect and amend (A4). Record amendments
based GDM, communities transition from a discursive space to proposals (A5).
in which people can be either right or wrong to one where UC3: Document the arguments pertaining to each proposal
people are either more or less aware of the issues surrounding (A5) and the identity of the person(s) who brought the
its practices. To make things more complicated, as is the argument forward (R3). While it would be difficult to
case for any organization, the typical process is not followed support active listening (F1) and the reformulation of
V. W HAT IS C URRENTLY M ISSING IN CBSD S UPPORT
Supporting D7. T OOLS ?
CBSD Educate the
organizational
introspection context In this section we describe how existing tools and tech-
and nologies fit with the CBSD activities, roles and work products
helps helps described in previous sections. Ideally, CBSD support tools
UC8. helps UC10. should be compatible with the processes presented earlier, and
Evaluate alternatives Understand decisions address the challenges of consensus-based decision-making.
UC9. In practice, while some are covered by existing tools, other
and and
Detect dysfunctions functionalities are still missing. The list of tools analyzed
D6. is a preliminary, non-exhaustive list of tools, based on a
D3. and Document
Evaluate the breadth similar list published in the literature [33] as well as our own
D4. rationale
of the alternatives Identify groupthink and investigation.
studied other dysfunctional ArguNet [34] and Carneades [35] are tools for structuring
decision processes debates into arguments and logical propositions. They can
Fig. 4: Use-cases to support for CBSD introspection, using the i* record the ideas and arguments presented during a GDM
modeling language. meeting, using formal logic to encode their structure. They
treat arguments as logically true propositions which can either
support or oppose a proposal. They do not have voting
mechanisms or support for evaluating expertise. ArguNet uses
a directed node graph similar to concept models to link
participants’ intervention (F2), the tool should assist the arguments and logicial propositions. Carneades, on the other
Facilitator’s synthesis responsibility (F3). Each decision hand, uses a tree modeling langage similar to i* models in
should include the minutes of the relevant meetings in order to structure arguments.
order to understand better the context of the decision bCisive [36] is a tool for structuring debates. It provides
(R2). a richer modelling language (over fifty pictograms) than
UC4: Detect consensus, through mechanisms like straw polls ArguNet and Carneades. For example, it allows typing the
(A6). Straw polls should support the four types of votes sources of arguments, e.g., to differentiate between expert
(V1-V4). It should be possible to record the rationale of opinion and hearsay. Like Carneades, bCisive uses a tree
non-consenting votes (V1-V3). modeling langage similar to i* models.
UC5: Monitor the implementation of decisions after the meet- DebateGraph [37] is a mind-mapping tool offering the ca-
ing, to ensure it is carried out as decided (A7, D5). pability to structure data. It can link arguments and proposals,
UC6: Record relevant sketches (e.g., whiteboard photos). but its utility remains limited for GDM. Like ArguNet, it
Discussion can often lead to a refined model that can uses a directed node graph similar to concept models to link
be understood by all participants (D1). documents together.
UC7: Identify and evaluate the expertise relevant to the deci- Loomio [38] is a GDM tool useful for voting when there are
sion at hand (D2). already clearly-defined alternatives. It does not help in finding
and structuring the debate in order to find these alternatives. It
Introspection, reflection and self-improvement is a big part does not directly use a modeling language, but it uses simple
of the Agile philosophy [9], and the philosophy of CBCs. A graphs (pie charts, bar charts) to display the level of consensus.
CBSD support tool should also therefore support the following Planning Poker [39] is useful for choosing an appropriate
use cases: priority or estimate. Alternatives are clearly stated in the form
of numerical scales. The facilitator can choose among various
UC8: After implementing a decision, determine if the alter- scales (such as the Fibonacci sequence, powers of 2, Small-
natives studied were adequate or if the team should have Medium-Large, etc.) according to what is most appropriate for
spent more time finding alternatives (D3). a given context. Planning poker uses a pictogrammic modeling
UC9: Detect degenerated group decision making problems, langage based on common playing cards in order to facilitate
such as groupthink, unresolved conflict, etc. (D4). understanding.
UC10: Some decisions can have a long-term impact. Develop- Reddit [40] and Discourse [41] are discussion tools, where
ers years later might have to deal with the decisions made participants bring arguments based on an introduction state-
today and might want to understand why the specific ment. Reddit integrates a voting mechanism which ensures
decision was made (D6). Understanding the decision that entries supported by more participants are ranked first.
means understanding the organizational context wherein Discourse is closer to a mailing list and presents arguments in
the decision was made at the time. Participants must a temporal sequence. While Reddit does not use a modelling
therefore be educated on the organizational context in language, Discourse uses a simple pictogrammic model based
order to understand the related decisions (D7). on users’ avatars in order to show the state of a discussion. A
TABLE I: Results of the tool analysis based on the i* models in Figure 3 and 4. Tools are presented in alphabetical order.
Tool Argunet bCisive Carneades DebateGraph Discourse holaSpirit Loomio Planning Poker Reddit
UC1: Present issue A1 Yes Yes No No Yes No Yes Yes Yes
A2 Yes Yes Yes Yes Yes No No No Yes
A3 Partial Yes No Partial No No No No No
UC2: Manage proposals
A4 Yes Yes Yes No No No No No No
A5 Partial Partial No No Partial No No No Partial
A5 Yes Yes Yes Yes Yes No No No Yes
UC3: Document R2 No No No No Yes No No No Yes
arguments R3 No No No No Yes No No No Yes
F3 Yes Yes Yes Yes No Yes No Yes No
V1 No No No No No No Yes Yes Yes
V2 No No No No No No Yes No No
UC4: Detect consensus A6
V3 No No No No No No Yes No No
V4 No No No No No No Yes Yes Yes
A7 No No No No No No No Partial No
UC5: Monitor decisions D5 No No No No No No No Partial No
UC6: Whiteboarding D1 No Yes No Yes Yes No No No Partial
UC7: Identify expertise D2 No Yes No No No Yes No No No
UC8: Evaluate D3 No No No No No No No No No
alternatives
UC9: Detect
D4 No No No No No Partial No No No
dysfunctions
UC10: Understand D6 Yes Yes Yes No Yes No No No Yes
decisions D7 No No No No No No No No No
Discourse discussion is modeled with the avatar of the initiator, different paradigm used by the tools analyzed. About half
and avatars of the last users to reply to the discussion. the tools studied support a rational decision process (e.g.
holaSpirit [42] is an implementation of the “holacracy” [11] ArguNet, bCisive, Carneades), rooted in logical propositions
framework, which focuses on managing responsibilities and akin to Bayesian logic. These tools assume that arguments
meeting agendas. It includes a way to document tensions brought forward are equally true and build the GDM around
between participants, which can help diagnose some dys- argument structures. The other half support a naturalistic
functionalities. It can help identify expertise, based on the decision process (e.g. Loomio, Reddit, Planning Poker) and
assignment of responsibilities. The “holacracy” framework are rooted in fuzzy propositions. These fuzzy propositions are
relies on Venn diagrams to show how responsibilities can be not assumed to be true, and GDM is therefore built around
regrouped into roles. participants’ votes.
We summarize the results of our analysis in Table I. Note The importance of both rational and naturalistic approaches
that the activity “Refinement” (A5) appears in two columns to GDM has been detailed in published literature [24], [25]. On
and does not present the same results. The reason is that one the one hand, purely rational tools are limited in practice. On
analysis is taken from the point of view of management pro- the other hand, while some fuzzy argumentation models exist,
posals (UC2) while the other is based on argument documenta- they focus on artificial intelligence and are not supported by
tion (UC3). For example, a tool can permit edition of proposals any GDM tools [43]. This implies that a fuzzy argumentation
(A5 from UC2), but does not cover argument documentationm modelling support tool might be useful for practical GDM,
(A5 from UC3), therefore resulting in different results. and in extension, to CBSD.
Studying the results presented in Table I, it is possible to Results show only partial coverage of the “Refinement”
pinpoint incompatibilities between current tool functionalities activity (A5) from the point of view of proposal management
and the needs of CBSD. Some requirements are poorly covered (UC2). A few tools support the editing of proposals and
by current tools, while others are not covered at all. arguments, but none keep track of changes. This requirement
Results also show that many tools focus on one aspect of is important in practice because it is often necessary to come
GDM exclusively. This is coherent with the current tendency back to previous versions of proposals, e.g., when a change
away from a single tool able to do everything (e.g., Rational is blocked by participants. It is also important for postmortem
Rose) and toward multiple inter-communicating tools (e.g., evaluation of the GDM process, especially postmortem alter-
integrating Slack with JIRA and Bitbucket). It would therefore native evaluation (UC8) and team dysfunction identification
theoretically be possible to cover the needs of CBSD using (UC9).
multiple inter-communicating tools. Unfortunately, most of Among the uncovered requirements, none of the analyzed
these tools cannot do this right now, forcing a manual transfer tools support postmortem alternative evaluation (UC8). This is
of data from one tool to the next. For example, while none cause for concern since poorly framed problems are the source
of the tools analyzed provides direct support for decision of multiple software catastrophes [44]. It would be interesting
monitoring (UC5), it would be possible to support it through a to track changes made to the alternatives throughout the project
connection to a project management tool like Microsoft Project in order to detect when poor alternatives were chosen, or where
or Atlassian JIRA. radical changes to alternatives occurred during development.
However, one obstacle to this integration is the two widely No tools support the postmortem evaluation of team dys-
functions (UC9), except holaSpirit, which does it partially c) RQ3. Evolving decision models:: It is currently possi-
through the manual identification of tensions. With a tool able ble to make static argumentation models. It should be possible
to keep track of changes, it would be possible to see how the to take multiple snapshots of static models in order to reason
decision evolved throughout the project. It would therefore be on the evolution of the models over time. For reasoning on
possible to perform some reasoning not only on one static design decision rationale, the monitoring and management of
model, but on a serie of evolving models. It seems however compliance [51] and assurance [52] documentation over time
that at this time, no framework exists to study the evolution is relevant. Crucially, decision-making should be informed by
of argument models. the principled evaluation of alternatives. This can be accom-
Given the importance given to the management of expertise plished by integrating into CBSD techniques developed for
in the literature [7], [6], [44], it is surprising that more tools design space exploration [53]. Further, a support tool should
are not taking it into account. Only two tools support it, but integrate these in a metamodelling platform that would allow
they do not implement voting mechanisms. In other words, comprehensive management [54] and versioning [55].
expertise is not factored in any of the vote-based tools. This d) RQ4. Monitoring Team Dynamics:: Current tools pro-
limits the capability of stakeholders to assess the relative vide limited information on team characteristics, like expertise
merits of alternatives, as all proposals, arguments and votes (D2) or organizational context (D7). Since these characteristics
are considered equal, which might not be the case in practice. vary from time to time, and from decision to decision, it
would be important to keep track of them in order to support
VI. G OING F ORWARD better reasoning. For example, CBCs might accrue “consensus
Given the mismatch between the capabilities of existing debt” (akin to technical debt [56]) when provisional short term
tools and the need of CBCs, we propose a roadmap for decisions need to be made to address urgent contingencies.
research on model-based techniques for supporting CBSD. This debt should be recorded, monitored and resolved. Keep-
We focus on model-based techniques specifically because the ing track of team dynamics would enable better modelling of
Model Driven Engineering (MDE) approach allows describing the social network, which would connect to decision models
the various relevant artifacts at a high level of abstraction, with specific heuristics to detect degenerations. In this, it
which in turn helps integrate and analyze information from is crucial to integrate into CBSD insights from industrial
heterogeneous sources, such as discussions, decisions, design psychology [57], taking into account the specific philosoph-
artifacts, documentation, source code, etc. ical concerns of CBCs, especially centred around issues of
intersectionality [58].
A. Main research questions
Research on CBSD should at a minimum address the B. Epistemological approach
following:
CBCs have their roots in social movements (e.g., Loomio
a) RQ1. Inter-tool communication:: Developers should
originates from the Occupy Wall Street movement [38]), and
be able to seamlessly use chains of relevant tools. For example,
research in CBSD should take in account its origins and the
they could pick one tool for problem statement, a second
particular concerns that lie at its core philosophy. Specifically,
for proposals and a third for argument documentation (UC1,
the philosophy behind CBSD evolved from critiques of abuses
UC2, UC3), and be able to link the decision data to a project
of traditional organizational structures. From the viewpoint of
management tool (UC5) of their choice. It is a classic problem
social movements, inequalities extant in society are reproduced
in software engineering and model-driven development. We
as individual behavior inside organizations unless special care
propose to use metamodelling and language globalization [45]
is given to avoid them.
to create an infrastructure for interoperability between tools.
This includes, from the perspective of CBCs, academic
b) RQ2. Tool support for fuzzy argumentation:: Existing
researchers. This poses a practical and epistemological chal-
tools are primarily oriented toward rational decision-making.
lenge for research in CBSD, since traditional post-positivist
In practice, when facing complex decisions, developers favour
and constructivist epistemological stances might be met with
a more naturalistic approach. Tool support, as well as the
significant resistance and suspicion from CBCs. Instead, we
modelling languages and techniques used for representing
propose that critical theory is the appropriate epistemological
discussions, should reflect this practical need. The modelling
framework for research in this domain. Easterbrook et al.
of discussions is a difficult problem in itself in the field of
describe it as follows [59]:
computational linguistics where various approaches have been
developed to represent and analyze them (e.g., [46], [47], Critical Theory judges scientific knowledge by its
[48]). However, combining such approaches with MDE tech- ability to free people from restrictive systems of
niques, languages and tools is an open problem. One potential thought [60]. Critical theorists argue that research
avenue is to extend peliminary work on creating models of is a political act, because knowledge empowers dif-
software design discussions [49], [50] with techniques from ferent groups within society, or entrenches existing
design space modelling and exploration. We are currently power structures
working on a project to model online discussions and provide Within a critical-theoretic epistemological framework, re-
online feedback and analytics about them to the participants. searchers prioritize the use of participatory approaches, that
provide them with intimate insights and high levels of famil- non-technical stakeholders, etc. [63]. [7]’s model integrates
iarity with the teams being studied [61], as well as action a feedback mechanism which recognizes that consensus is a
research. Action research is a research methodology, where building, evolving process (RQ3). The objective of their model
researchers acknowledge that they are not objective observers is to support this evolving process by providing appropriate
and instead work together with a community to affect change, feedback to the participants, based on their expertise. As the
explicitly acknowledging their biases. The focus is on building authors write, engineering GDM “is a dynamic and iterative
relationships of mutual trust and accountability between re- process, composed of several rounds where the experts ex-
searchers and the communities they study, and on participatory press, discuss, and modify their preferences” [7].
theory building in order to address real problems withing the The importance of expertise must however be balanced
community, under the principle that research and action are with the concern of technocratic autocracy, “where expertise
indivisible [62]. is the basis of authority” [24]. The works of [24] outlines
In short, research on CBSD should be conducted in close the importance of managing concerns (RQ5) to avoid the
collaboration with CBCs themselves, ensuring that their real case where the decision-making process becomes the domain
needs are addressed in accordance to the specific concerns and of unchallenged experts. Their work also show the lack of
priorities in their social and political milieu. research into naturalistic decision-making in Agile software
development. They write that:
VII. R ELATED W ORK
Today’s software systems are becoming more com-
The research questions provide interesting new avenues for
plex because of the need to balance the often
the development of tool support for GDM within generic
disparate needs of diverse stakeholders, and the
CBCs. However, how does this can be mapped to CBSD?
growing complexity of the technology used. This
Will the theoretical tool presented herein provide any benefit
reality requires more unstructured decision-making.
for software engineering, at least within CBC contexts?
Second, the principles of agile software development
The work of [6] on Agile software development corrob-
align with the definition of naturalistic decision-
orates some of the CBC challenges. They are critical of
making. [24]
traditional decision theory being too rooted within the rational
decision-making paradigm (RQ2): There is therefore a need, at least within Agile software
development, for naturalistic GDM support tools (RQ2).
Normative decision theory views decision makers
In summary, the research questions identified are not generic
as idealized, rational, extremely intelligent beings
needs of CBCs, but can also be applied to software engineer-
who overcome their inner turmoil, shifting values,
ing. A number of these research questions are not only limited
anxieties, post-decision regrets, fear of ambiguity,
to CBSD, but could also provide benefits for Agile software
inability to perform intricate calculations and limited
development. For Agile contexts, we note especially the need
attention span to make rational, optimum choices.
for naturalistic GDM tool support (RQ2) and the need for
[6]
expertise management (RQ4).
They also see the self-managed nature of Agile software
development as challenging. They mention that traditional VIII. C ONCLUSION
authoritarian structures normally ensure that the voice of the We have outlined the group decision-making (GDM) pro-
experts carry more weight than the voice of others (RQ4). cess and its associated challenges in the context of Consensus-
They also mention that in Agile, decision are not always Based Software Development (CBSD), based on existing
implemented as discussed during meetings, if at all (UC5). practices within Consensus-Based Communities (CBCs). We
In practice however, Agile software development is not structured these in high-level requirements for supporting the
as self-managed as theoretically designed. [8], in a survey CBSD with appropriate tooling and analyzed a preliminary list
of Agile software developers, found that 67% of them still of CBSD-related tools to find gaps in the existing functional-
had a manager above them. These Agile teams come from ities. Based on this analysis, we posed outlined a vision for
traditional authoritarian organizations, or are within mixed research in CBSD, identigying four research directions and
organizations (Agile and non-Agile), and are still beholden outlining the epistemological and methodological challenges
to high management. for such research.
The works of [7], while not specific to software, does study
engineering teams. They note the heterogeneity of experts (D1) R EFERENCES
within modern engineering teams. For example, a software [1] T. Flanagan, C. Eckert, and P. J. Clarkson, “Externalizing tacit
development team might include a user interface expert, a overview knowledge: A model-based approach to supporting design
database expert, a security expert, etc. They therefore build teams,” Artificial Intelligence for Engineering Design, Analysis and
Manufacturing, vol. 21, no. 3, pp. 227–242, Jun. 2007. [Online].
a theoretical model to help engineering team manage this het- Available: http://dx.doi.org/10.1017/S089006040700025X
erogeneity of expertise, through a weighting approach (RQ4). [2] M. J. Turk, “Scaling a code in the human dimension,” in
This heterogeneity becomes especially difficult during soft- Proceedings of the Conference on Extreme Science and Engineering
Discovery Environment: Gateway to Discovery, ser. XSEDE ’13.
ware requirement negotiations, as it includes participants with New York, NY, USA: ACM, 2013, pp. 691–697. [Online]. Available:
varying backgrounds: domain experts, software developers, http://doi.acm.org/10.1145/2484762.2484782
[3] M. E. Conway, “How do committees invent?” Datamation, vol. 14, no. 4, Conference, ser. WSC ’15. Piscataway, NJ, USA: IEEE Press,
pp. 28–31, 1968. 2015, pp. 276–287. [Online]. Available: http://dl.acm.org/citation.cfm?
[4] N. Nagappan, B. Murphy, and V. Basili, “The influence of organizational id=2888619.2888648
structure on software quality: An empirical case study,” in Proceedings [28] P. Bourdieu, Distinction. Harvard University Press, 1984.
of the 30th International Conference on Software Engineering, ser. [29] M. Shahin, P. Liang, and M. R. Khayyambashi, “Architectural design
ICSE ’08. New York, NY, USA: ACM, 2008, pp. 521–530. [Online]. decision: Existing models and tools,” in 2009 Joint Working IEEE/IFIP
Available: http://doi.acm.org/10.1145/1368088.1368160 Conference on Software Architecture European Conference on Software
[5] A. Maturo and A. G. S. Ventre, “Models for consensus in multiperson Architecture, Sept 2009, pp. 293–296.
decision making,” in NAFIPS 2008 - 2008 Annual Meeting of the North [30] J. Tyree and A. Akerman, “Architecture decisions: demystifying archi-
American Fuzzy Information Processing Society, May 2008, pp. 1–4. tecture,” IEEE Software, vol. 22, no. 2, pp. 19–27, March 2005.
[6] M. Drury, K. Conboy, and K. Power, “Decision making in agile [31] E. S. K. Yu, “Towards modelling and reasoning support for early-
development: A focus group study of decisions and obstacles,” in 2011 phase requirements engineering,” in Requirements Engineering, 1997.,
Agile Conference, Aug 2011, pp. 39–47. Proceedings of the Third IEEE International Symposium on, Jan 1997,
[7] I. J. Pérez, F. J. Cabrerizo, S. Alonso, and E. Herrera-Viedma, “A pp. 226–235.
new consensus model for group decision making problems with non- [32] F. Dalpiaz, X. Franch, and J. Horkoff, “iStar 2.0 Language Guide,” Tech.
homogeneous experts,” IEEE Transactions on Systems, Man and Cyber- Rep., 2016.
netics: Systems, vol. 44, no. 4, pp. 494–498, 2014. [33] D. A. L. Garcı́a, T. d. J. Mateo Sanguino, E. C. Ancos, and I. F.
[8] Y. Shastri, R. Hoda, and R. Amor, “Does the ’project manager’ still de Viana González, “A debate and decision-making tool for enhanced
exist in agile software development projects?” in 2016 23rd Asia-Pacific learning,” IEEE Transactions on Learning Technologies, vol. 9, no. 3,
Software Engineering Conference (APSEC), Dec 2016, pp. 57–64. pp. 205–216, July 2016.
[9] K. Beck, M. Beedle, A. van Bennekum, A. Cockburn, W. Cunningham, [34] G. Betz, S. Cacean, and C. Voigt. (2016) Argunet - open-source
M. Fowler, J. Grenning, J. Highsmith, A. Hunt, R. Jeffries, J. Kern, argument mapping. [Online]. Available: http://www.argunet.org/editor
B. Marick, R. C. Martin, S. Mellor, K. Schwaber, J. Sutherland, and [35] T. F. Gordon. (2016) Carneades - tools for argument (re)construction,
D. Thomas. (2001) Principles behind the agile manifesto. [Online]. evaluation, mapping and interchange. [Online]. Available: http:
Available: http://agilemanifesto.org/principles.html //carneades.github.io/
[10] Rishipal, “Analytical comparison of flat and vertical organizational [36] ReasoningLab.com. (2017) bcisive - online decision mapping. [Online].
structure,” European Journal of Business and Management, vol. 6, Available: https://www.bcisiveonline.com/
no. 36, pp. 56–65, 2014. [37] P. Baldwin and D. Price. (2017) Debategraph. [Online]. Available:
[11] B. J. Robertson, “Organization at the leading edge: Introducing ho- http://debategraph.org/
lacracy,” Tech. Rep., 2007. [38] Loomio Cooperative Limited. (2016) Loomio - better decisions together.
[12] Koumbit. (2017) About us. [Online]. Available: https://www.koumbit. [Online]. Available: https://www.loomio.org/
org/en/about [39] Mountain Goat Software. (2017) Planningpoker.com - estimates
[13] Loomio Cooperative Limited. (2016) The loomio co-op handbook. made easy. sprints made simple. [Online]. Available: https://www.
[Online]. Available: https://www.loomio.coop/ planningpoker.com/
[14] Zappos Insights. (2017) Holacracy - flattening the organization [40] Reddit. (2017) Reddit. [Online]. Available: https://about.reddit.com/
structure and busting bureaucracy. [Online]. Available: https://www. [41] Civilized Discourse Construction Kit. (2017) What is discourse?
zapposinsights.com/about/holacracy - discourse - civilized discussion. [Online]. Available: http://www.
[15] A. Doyle. (2016) Management and organization discourse.org/about
at medium. [Online]. Available: https://blog.medium.com/ [42] holaSpirit SAS. (2017) holaspirit - application for holacracy and teal
management-and-organization-at-medium-2228cc9d93e9 organizations. [Online]. Available: https://www.holaspirit.com/
[16] Drupal. (2016) Drupal code of conduct. [Online]. Available: https: [43] J. Janssen, M. De Cock, and D. Vermeir, “Fuzzy argumentation net-
//www.drupal.org/dcoc works,” Vrije Universiteit Brussel, Tech. Rep., January 2008.
[17] Seeds for Change, A Consensus Handbook. Footprint Workers’ Co-op, [44] C. Edwards, “Agreeing to fail,” Engineering & Technology, vol. 4,
2013. no. 7, pp. 74–75, 2009. [Online]. Available: https://doi.org/10.1049/et.
[18] D. G. Messerschmitt, C. Szyperski et al., “Software ecosystem: under- 2009.0716
standing an indispensable technology and industry,” MIT Press Books, [45] B. Combemale, J. Deantoni, B. Baudry, R. B. France, J.-M. Jézéquel,
vol. 1, 2005. and J. Gray, “Globalizing modeling languages,” Computer, pp. 10–13,
[19] D. Zwieback, Beyond Blame, Learning from successes and failure. 2014.
O’Reilly Media, 2015. [46] D. P. Twitchell, M. Adkins, J. F. Nunamaker, and J. K. Burgoon, “Using
[20] A. Cornell, Oppose and Propose! AK Press, 2011. speech act theory to model conversations for automated classification
[21] D. Vannucci and R. Singer, Come Hell or High Water. AK Press, 2010. and retrieval,” in Proceedings of the International Working Conference
[22] R. Cialdini, “The science of persuasion,” Scientific American, vol. 284, Language Action Perspective Communication Modelling (LAP 2004),
pp. 76–81, 2001. 2004, pp. 121–130.
[23] F. J. Cabrerizo, F. Chiclana, M. R. Uren̋a, and E. Herrera-Viedma, [47] S. Joty and E. Hoque, “Speech act modeling of written asynchronous
“Challenges and open questions in soft consensus models,” in 2013 Joint conversations with task-specific embeddings and conditional structured
IFSA World Congress and NAFIPS Annual Meeting (IFSA/NAFIPS), models,” in Proceedings of the 54th Annual Meeting of the Association
June 2013, pp. 944–949. for Computational Linguistics, ACL, 2016, pp. 7–12.
[24] N. B. Moe, A. Aurum, and T. Dybå, “Challenges of shared decision- [48] R. P. Fawcett and B. L. Davies, Monologue as a turn in dialogue:
making: A multiple case study of agile software development,” Informa- Towards an integration of exchange structure and rhetorical structure
tion and Software Technology, vol. 54, no. 8, pp. 853–865, 2012, special theory, 1992, pp. 151–166.
Issue: Voice of the Editorial Board. [49] E. Black, P. McBurney, and S. Zschaler, Towards Agent Dialogue as a
[25] C. Zannier, M. Chiasson, and F. Maurer, “A model of design decision Tool for Capturing Software Design Discussions, 2014, pp. 95–110.
making based on empirical results of interviews with software design- [50] A. S. Mashiyat, M. Famelis, R. Salay, and M. Chechik, “Using devel-
ers,” Information and Software Technology, vol. 49, no. 6, pp. 637–653, oper conversations to resolve uncertainty in software development: A
2007. position paper,” in Proceedings of the 4th International Workshop on
[26] M. Van Mechelen, M. Gielen, V. vanden Abeele, A. Laenen, and Recommendation Systems for Software Engineering, 2014, pp. 1–5.
B. Zaman, “Exploring challenging group dynamics in participatory [51] S. Ghanavati, D. Amyot, and L. Peyton, “Towards a framework for
design with children,” in Proceedings of the 2014 Conference tracking legal compliance in healthcare,” in Advanced Information
on Interaction Design and Children, ser. IDC ’14. New York, Systems Engineering. Springer, 2007, pp. 218–232.
NY, USA: ACM, 2014, pp. 269–272. [Online]. Available: http: [52] S. Kokaly, R. Salay, V. Cassano, T. Maibaum, and M. Chechik, “A model
//doi.acm.org/10.1145/2593968.2610469 management approach for assurance case reuse due to system evolution,”
[27] A. Kalyanasundaram, W. Wei, K. M. Carley, and J. D. Herbsleb, in Proceedings of the ACM/IEEE 19th International Conference on
“An agent-based model of edit wars in wikipedia: How and when Model Driven Engineering Languages and Systems. ACM, 2016, pp.
is consensus reached,” in Proceedings of the 2015 Winter Simulation 196–206.
[53] S. A. Busari and E. Letier, “Radar: A lightweight tool for requirements [58] D. Atewologun, R. Sealy, and S. Vinnicombe, “Revealing intersectional
and architecture decision analysis,” in Proceedings of the 39th dynamics in organizations: Introducing ’intersectional identity work’,”
International Conference on Software Engineering, ser. ICSE ’17. Gender, Work & Organization, vol. 23, no. 3, pp. 223–247, 2016.
Piscataway, NJ, USA: IEEE Press, 2017, pp. 552–562. [Online]. [59] S. Easterbrook, J. Singer, M.-A. Storey, and D. Damian, “Selecting em-
Available: https://doi.org/10.1109/ICSE.2017.57 pirical methods for software engineering research,” Guide to advanced
[54] G. Brunet, M. Chechik, S. Easterbrook, S. Nejati, N. Niu, and M. Sa- empirical software engineering, pp. 285–311, 2008.
betzadeh, “A manifesto for model merging,” in Proceedings of the 2006 [60] C. Calhoun, Critical social theory: Culture, history, and the challenge
international workshop on Global integrated model management. ACM, of difference. Wiley-Blackwell, 1995.
2006, pp. 5–12. [61] T. C. Lethbridge, S. E. Sim, and J. Singer, “Studying software
[55] P. Brosch, G. Kappel, P. Langer, M. Seidl, K. Wieland, and M. Wimmer, engineers: Data collection techniques for software field studies,”
“An introduction to model versioning,” in Proceedings of the 12th Empirical Software Engineering, vol. 10, no. 3, pp. 311–341, 2005.
international conference on Formal Methods for the Design of Computer, [Online]. Available: http://dx.doi.org/10.1007/s10664-005-1290-x
Communication, and Software Systems: formal methods for model- [62] P. S. M. Dos Santos, G. H. Travassos, and M. V. Zelkowitz, “Action
driven engineering. Springer-Verlag, 2012, pp. 336–398. research can swing the balance in experimental software engineering,”
[56] P. Kruchten, R. L. Nord, and I. Ozkaya, “Technical debt: From metaphor Advances in computers, vol. 83, pp. 205–276, 2011.
to theory and practice,” Ieee software, vol. 29, no. 6, pp. 18–21, 2012. [63] H. In and S. Roy, “Issues of visualized conflict resolution,” in Proceed-
[57] K. W. Buffinton, K. W. Jablokow, and K. A. Martin, “Project team dy- ings Fifth IEEE International Symposium on Requirements Engineering,
namics and cognitive style,” Engineering Management Journal, vol. 14, 2001, pp. 314–315.
no. 3, pp. 25–33, 2002.